[clang] [compiler-rt] [llvm] [AArch64][compiler-rt] Add preserve-most outline atomics convention (PR #240709)
Roland McGrath via cfe-commits
cfe-commits at lists.llvm.org
Thu Nov 12 19:11:39 PST 2026
https://github.com/frobtech updated https://github.com/llvm/llvm-project/pull/240709
>From f9a60b39d06c4266f70f9872074c4da384874094 Mon Sep 17 00:00:00 2001
From: Roland McGrath <mcgrathr at google.com>
Date: Mon, 9 Nov 2026 20:08:27 -0800
Subject: [PATCH 1/5] [AArch64] Add preserve-most calling convention for
outline atomics
This adds the `+outline-atomics-preserve-most` subtarget feature, the
`--aarch64-outline-atomics-preserve-most` backend flag, and the Clang
driver flags `-moutline-atomics-preserve-most` and
`-mno-outline-atomics-preserve-most` for AArch64 targets.
Out-of-line atomic helper calls emitted by `-moutline-atomics`
(`__aarch64_{cas,swp,ldadd,ldclr,ldeor,ldset}*`) are currently lowered
as ordinary AAPCS64 calls, clobbering all caller-saved GPRs and
SIMD/SVE registers at every atomic operation. However,
`compiler-rt`'s `lse.S` implementation (as well as custom
runtime/kernel implementations using the same ABI) links statically
with hidden visibility and modifies only its argument and return
registers, `x15`-`x17`, and `nzcv`.
A single call-only calling convention,
`aarch64_outline_atomics_preservemost`, models that contract across
all 1-, 2-, 4-, 8-, and 16-byte helpers: the callee preserves all
registers except the argument/return registers (`x0`-`x4`), scratch
registers (`x15`-`x17`), and `nzcv`.
`LibcallLoweringInfo::setLibcallImplCallingConv` allows
`AArch64Subtarget::initLibcallLoweringInfo` to apply this
per-subtarget override without duplicating `RuntimeLibcallImpl` table
entries.
Because `libgcc`'s `__aarch64_cas16_*` implementation additionally
writes `x13` and `x14`, this convention remains opt-in rather than the
default for targets that may link against `libgcc`. The feature is
marked `InlineIgnore` so LTO can inline across translation units
compiled with different settings, and is suppressed on Arm64EC where
calls may traverse emulation thunks.
Assisted-by: Gemini
---
clang/docs/ReleaseNotes.md | 2 +
clang/include/clang/Options/Options.td | 12 ++
clang/lib/Driver/ToolChains/CommonArgs.cpp | 16 ++
.../AArch64/outline-atomics-preserve-most.c | 19 +++
clang/test/Driver/aarch64-features.c | 18 +++
.../test/Driver/unsupported-outline-atomics.c | 8 +
llvm/docs/ReleaseNotes.md | 2 +
.../llvm/Analysis/LibcallLoweringInfo.h | 18 ++-
llvm/include/llvm/AsmParser/LLToken.h | 1 +
llvm/include/llvm/IR/CallingConv.h | 4 +
llvm/lib/AsmParser/LLLexer.cpp | 1 +
llvm/lib/AsmParser/LLParser.cpp | 4 +
llvm/lib/IR/AsmWriter.cpp | 3 +
llvm/lib/IR/Function.cpp | 1 +
.../AArch64/AArch64CallingConvention.td | 9 ++
llvm/lib/Target/AArch64/AArch64Features.td | 7 +
.../Target/AArch64/AArch64ISelLowering.cpp | 1 +
llvm/lib/Target/AArch64/AArch64Options.td | 2 +
.../Target/AArch64/AArch64RegisterInfo.cpp | 10 ++
llvm/lib/Target/AArch64/AArch64Subtarget.cpp | 34 +++++
llvm/lib/Target/AArch64/AArch64Subtarget.h | 3 +
.../aarch64-outline-atomics-preserve-most.ll | 22 +++
...line-atomics-preserve-most-callee-error.ll | 7 +
.../outline-atomics-preserve-most-sve.ll | 28 ++++
.../AArch64/outline-atomics-preserve-most.ll | 142 ++++++++++++++++++
.../AArch64/outline-atomics-preserve-most.ll | 88 +++++++++++
26 files changed, 461 insertions(+), 1 deletion(-)
create mode 100644 clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c
create mode 100644 llvm/test/Bitcode/aarch64-outline-atomics-preserve-most.ll
create mode 100644 llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-callee-error.ll
create mode 100644 llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
create mode 100644 llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
create mode 100644 llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index 9c36e1e167945..fc88a9fee8037 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1347,6 +1347,8 @@ documentation](CIR/index.md) for more information regarding ClangIR's status and
`x86_64-unknown-uefi`, it follows the Microsoft ABI conventions (LLP64,
Microsoft C++ ABI, `char *` based `va_list`) and links with `lld-link`.
+- Added `-moutline-atomics-preserve-most` and `-mno-outline-atomics-preserve-most` for AArch64 targets to control whether out-of-line atomic helper calls preserve all registers except argument/return registers (`x0`-`x4`), scratch registers (`x15`-`x17`), and `NZCV`.
+
#### Android Support
- Enabled PAC and BTI by default for AArch64 Android targets.
diff --git a/clang/include/clang/Options/Options.td b/clang/include/clang/Options/Options.td
index d281f6552b4db..7df0f134e41b4 100644
--- a/clang/include/clang/Options/Options.td
+++ b/clang/include/clang/Options/Options.td
@@ -6388,6 +6388,18 @@ def moutline_atomics : Flag<["-"], "moutline-atomics">, Group<f_clang_Group>,
def mno_outline_atomics : Flag<["-"], "mno-outline-atomics">, Group<f_clang_Group>,
Visibility<[ClangOption, CC1Option, FlangOption]>,
HelpText<"Don't generate local calls to out-of-line atomic operations">;
+def moutline_atomics_preserve_most
+ : Flag<["-"], "moutline-atomics-preserve-most">,
+ Group<f_clang_Group>,
+ Visibility<[ClangOption, FlangOption]>,
+ HelpText<"Use a preserve-most calling convention for out-of-line atomic "
+ "helper calls on AArch64">;
+def mno_outline_atomics_preserve_most
+ : Flag<["-"], "mno-outline-atomics-preserve-most">,
+ Group<f_clang_Group>,
+ Visibility<[ClangOption, FlangOption]>,
+ HelpText<"Don't use a preserve-most calling convention for out-of-line "
+ "atomic helper calls on AArch64">;
def mno_implicit_float : Flag<["-"], "mno-implicit-float">, Group<m_Group>,
HelpText<"Don't generate implicit floating point or vector instructions">;
def mimplicit_float : Flag<["-"], "mimplicit-float">, Group<m_Group>,
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index db5250db329a9..6b48129c8d276 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -3067,6 +3067,22 @@ void tools::addOutlineAtomicsArgs(const Driver &D, const ToolChain &TC,
CmdArgs.push_back("-target-feature");
CmdArgs.push_back("+outline-atomics");
}
+
+ if (Arg *A =
+ Args.getLastArg(options::OPT_moutline_atomics_preserve_most,
+ options::OPT_mno_outline_atomics_preserve_most)) {
+ if (!Triple.isAArch64()) {
+ D.Diag(diag::warn_drv_moutline_atomics_unsupported_opt)
+ << Triple.getArchName() << A->getOption().getName();
+ } else if (A->getOption().matches(
+ options::OPT_moutline_atomics_preserve_most)) {
+ CmdArgs.push_back("-target-feature");
+ CmdArgs.push_back("+outline-atomics-preserve-most");
+ } else {
+ CmdArgs.push_back("-target-feature");
+ CmdArgs.push_back("-outline-atomics-preserve-most");
+ }
+ }
}
void tools::addOffloadCompressArgs(const llvm::opt::ArgList &TCArgs,
diff --git a/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c b/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c
new file mode 100644
index 0000000000000..37d110a48a2f9
--- /dev/null
+++ b/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c
@@ -0,0 +1,19 @@
+// RUN: %clang_cc1 -triple aarch64-unknown-linux-gnu -O2 -target-feature +outline-atomics -target-feature +outline-atomics-preserve-most -emit-llvm -o - %s | FileCheck %s --check-prefix=IR
+// RUN: %clang_cc1 -triple aarch64-unknown-linux-gnu -O2 -target-feature +outline-atomics -target-feature +outline-atomics-preserve-most -S -o - %s | FileCheck %s --check-prefix=ASM
+// RUN: %clang_cc1 -triple aarch64-unknown-linux-gnu -O2 -target-feature +outline-atomics -mllvm --aarch64-outline-atomics-preserve-most -S -o - %s | FileCheck %s --check-prefix=ASM
+// RUN: %clang_cc1 -triple aarch64-unknown-fuchsia -O2 -target-feature +outline-atomics -target-feature +outline-atomics-preserve-most -emit-llvm -o - %s | FileCheck %s --check-prefix=IR
+// RUN: %clang_cc1 -triple aarch64-unknown-fuchsia -O2 -target-feature +outline-atomics -target-feature +outline-atomics-preserve-most -S -o - %s | FileCheck %s --check-prefix=ASM
+// RUN: %clang_cc1 -triple aarch64-unknown-fuchsia -O2 -target-feature +outline-atomics -mllvm --aarch64-outline-atomics-preserve-most -S -o - %s | FileCheck %s --check-prefix=ASM
+
+int fetch_add_with_live_vals(_Atomic int *p, int v, int a, int b, int c, int d,
+ int e, int f) {
+ // ASM-LABEL: fetch_add_with_live_vals:
+ // ASM-NOT: stp x19
+ // ASM-NOT: stp x20
+ // ASM: bl __aarch64_ldadd4_acq_rel
+ // ASM: ret
+ __c11_atomic_fetch_add(p, v, __ATOMIC_ACQ_REL);
+ return a + b + c + d + e + f;
+}
+
+// IR: attributes #{{[0-9]+}} = { {{.*}}"target-features"="{{.*}}+outline-atomics,+outline-atomics-preserve-most{{.*}}"
diff --git a/clang/test/Driver/aarch64-features.c b/clang/test/Driver/aarch64-features.c
index d2dc5a20119ee..9a2aae82e0a07 100644
--- a/clang/test/Driver/aarch64-features.c
+++ b/clang/test/Driver/aarch64-features.c
@@ -87,6 +87,24 @@
// RUN: %clang --target=aarch64-unknown-managarm-mlibc -rtlib=libgcc \
// RUN: -### -c %s 2>&1 | FileCheck -check-prefix=CHECK-OUTLINE-ATOMICS-ON %s
+// RUN: %clang --target=aarch64-linux-gnu -rtlib=compiler-rt -moutline-atomics-preserve-most \
+// RUN: -### -c %s 2>&1 | FileCheck \
+// RUN: -check-prefixes=CHECK-OUTLINE-ATOMICS-ON,CHECK-OUTLINE-ATOMICS-PRESERVE-MOST %s
+
+// RUN: %clang --target=aarch64-linux-gnu -moutline-atomics -moutline-atomics-preserve-most \
+// RUN: -mno-outline-atomics-preserve-most -### -c %s 2>&1 | FileCheck \
+// RUN: -check-prefixes=CHECK-OUTLINE-ATOMICS-ON,CHECK-NO-OUTLINE-ATOMICS-PRESERVE-MOST %s
+
+// RUN: %clang --target=aarch64-unknown-fuchsia -moutline-atomics-preserve-most \
+// RUN: -### -c %s 2>&1 | FileCheck \
+// RUN: -check-prefixes=CHECK-OUTLINE-ATOMICS-ON,CHECK-OUTLINE-ATOMICS-PRESERVE-MOST %s
+
+// RUN: %clang --target=aarch64-unknown-fuchsia -moutline-atomics-preserve-most \
+// RUN: -mno-outline-atomics-preserve-most -### -c %s 2>&1 | FileCheck \
+// RUN: -check-prefixes=CHECK-OUTLINE-ATOMICS-ON,CHECK-NO-OUTLINE-ATOMICS-PRESERVE-MOST %s
+
// CHECK-OUTLINE-ATOMICS-ON: "-target-feature" "+outline-atomics"
// CHECK-OUTLINE-ATOMICS-OFF-NOT: "-target-feature" "+outline-atomics"
// CHECK-NO-OUTLINE-ATOMICS: "-target-feature" "-outline-atomics"
+// CHECK-OUTLINE-ATOMICS-PRESERVE-MOST: "-target-feature" "+outline-atomics-preserve-most"
+// CHECK-NO-OUTLINE-ATOMICS-PRESERVE-MOST: "-target-feature" "-outline-atomics-preserve-most"
diff --git a/clang/test/Driver/unsupported-outline-atomics.c b/clang/test/Driver/unsupported-outline-atomics.c
index 2e88528c2ec60..2cba7fb0d1dfb 100644
--- a/clang/test/Driver/unsupported-outline-atomics.c
+++ b/clang/test/Driver/unsupported-outline-atomics.c
@@ -13,3 +13,11 @@
// RUN: %clang -target riscv64 -mno-outline-atomics -S %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-NO-OUTLINE-ATOMICS-RISCV
// CHECK-NO-OUTLINE-ATOMICS-RISCV: warning: 'riscv64' does not support '-mno-outline-atomics'; flag ignored [-Woption-ignored]
// CHECK-NO-OUTLINE-ATOMICS-RISCV-NOT: "-target-feature" "-outline-atomics"
+
+// RUN: %clang -target x86_64 -moutline-atomics-preserve-most -S %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-PRESERVE-MOST-X86
+// CHECK-PRESERVE-MOST-X86: warning: 'x86_64' does not support '-moutline-atomics-preserve-most'; flag ignored [-Woption-ignored]
+// CHECK-PRESERVE-MOST-X86-NOT: "-target-feature" "+outline-atomics-preserve-most"
+
+// RUN: %clang -target x86_64 -mno-outline-atomics-preserve-most -S %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-NO-PRESERVE-MOST-X86
+// CHECK-NO-PRESERVE-MOST-X86: warning: 'x86_64' does not support '-mno-outline-atomics-preserve-most'; flag ignored [-Woption-ignored]
+// CHECK-NO-PRESERVE-MOST-X86-NOT: "-target-feature" "-outline-atomics-preserve-most"
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 687025c958221..0d64f1374edaf 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -303,6 +303,8 @@ Makes programs 10x faster by doing Special New Thing.
ARM64 ABI (calling convention, callee saved registers, stack probing and SEH
unwind information) and produces PE/COFF objects.
+* Added the `+outline-atomics-preserve-most` subtarget feature (and `--aarch64-outline-atomics-preserve-most` backend option) to lower AArch64 out-of-line atomic helper calls using `aarch64_outline_atomics_preservemost`, preserving all registers except `x0`-`x4`, `x15`-`x17`, and `NZCV`.
+
### Changes to the AMDGPU Backend
* Replaced `xnack` and `sramecc` target features with `amdgpu.xnack`
diff --git a/llvm/include/llvm/Analysis/LibcallLoweringInfo.h b/llvm/include/llvm/Analysis/LibcallLoweringInfo.h
index 2461bc7723ca1..22679bae055ba 100644
--- a/llvm/include/llvm/Analysis/LibcallLoweringInfo.h
+++ b/llvm/include/llvm/Analysis/LibcallLoweringInfo.h
@@ -20,6 +20,7 @@
#include "llvm/Analysis/RuntimeLibcallInfo.h"
#include "llvm/IR/RuntimeLibcalls.h"
#include "llvm/Pass.h"
+#include <cassert>
namespace llvm {
@@ -31,6 +32,8 @@ class LibcallLoweringInfo {
/// Stores the implementation choice for each libcall.
RTLIB::LibcallImpl LibcallImpls[RTLIB::UNKNOWN_LIBCALL + 1] = {
RTLIB::Unsupported};
+ /// Stores context-specific (e.g. subtarget) calling convention overrides.
+ DenseMap<unsigned, CallingConv::ID> CallingConvOverrides;
public:
/// Callback applying the caller's context-specific (e.g. subtarget) libcall
@@ -80,14 +83,27 @@ class LibcallLoweringInfo {
LibcallImpls[Call] = Impl;
}
+ /// Override the CallingConv that should be used for \p Call in this lowering
+ /// context.
+ void setLibcallImplCallingConv(RTLIB::LibcallImpl Call, CallingConv::ID CC) {
+ assert(Call != RTLIB::Unsupported &&
+ "Cannot override calling convention for unsupported libcall");
+ CallingConvOverrides[Call] = CC;
+ }
+
// FIXME: Remove this wrapper in favor of directly using
// getLibcallImplCallingConv
CallingConv::ID getLibcallCallingConv(RTLIB::Libcall Call) const {
- return RTLCI.LibcallImplCallingConvs[LibcallImpls[Call]];
+ return getLibcallImplCallingConv(LibcallImpls[Call]);
}
/// Get the CallingConv that should be used for the specified libcall.
CallingConv::ID getLibcallImplCallingConv(RTLIB::LibcallImpl Call) const {
+ if (!CallingConvOverrides.empty()) {
+ auto It = CallingConvOverrides.find(Call);
+ if (It != CallingConvOverrides.end())
+ return It->second;
+ }
return RTLCI.LibcallImplCallingConvs[Call];
}
diff --git a/llvm/include/llvm/AsmParser/LLToken.h b/llvm/include/llvm/AsmParser/LLToken.h
index 78e20282c8459..2d99e78ceac75 100644
--- a/llvm/include/llvm/AsmParser/LLToken.h
+++ b/llvm/include/llvm/AsmParser/LLToken.h
@@ -152,6 +152,7 @@ enum Kind {
kw_aarch64_sme_preservemost_from_x0,
kw_aarch64_sme_preservemost_from_x1,
kw_aarch64_sme_preservemost_from_x2,
+ kw_aarch64_outline_atomics_preservemost,
kw_msp430_intrcc,
kw_avr_intrcc,
kw_avr_signalcc,
diff --git a/llvm/include/llvm/IR/CallingConv.h b/llvm/include/llvm/IR/CallingConv.h
index 249c512dda532..413c5879d5204 100644
--- a/llvm/include/llvm/IR/CallingConv.h
+++ b/llvm/include/llvm/IR/CallingConv.h
@@ -297,6 +297,10 @@ namespace CallingConv {
/// stateless compartment.
CHERIoT_LibraryCall = 127,
+ /// Callee preserves all registers except X0-X4, X15-X17, and NZCV (used by
+ /// AArch64 out-of-line atomic helper calls).
+ AArch64_Outline_Atomics_PreserveMost = 128,
+
/// The highest possible ID. Must be some 2^k - 1.
MaxID = 1023
};
diff --git a/llvm/lib/AsmParser/LLLexer.cpp b/llvm/lib/AsmParser/LLLexer.cpp
index 36fcb4a7df7f2..417fed434228f 100644
--- a/llvm/lib/AsmParser/LLLexer.cpp
+++ b/llvm/lib/AsmParser/LLLexer.cpp
@@ -663,6 +663,7 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(aarch64_sme_preservemost_from_x0);
KEYWORD(aarch64_sme_preservemost_from_x1);
KEYWORD(aarch64_sme_preservemost_from_x2);
+ KEYWORD(aarch64_outline_atomics_preservemost);
KEYWORD(msp430_intrcc);
KEYWORD(avr_intrcc);
KEYWORD(avr_signalcc);
diff --git a/llvm/lib/AsmParser/LLParser.cpp b/llvm/lib/AsmParser/LLParser.cpp
index 6be0e010a235c..2ef79cb079168 100644
--- a/llvm/lib/AsmParser/LLParser.cpp
+++ b/llvm/lib/AsmParser/LLParser.cpp
@@ -2344,6 +2344,7 @@ void LLParser::parseOptionalDLLStorageClass(unsigned &Res) {
/// ::= 'aarch64_sme_preservemost_from_x0'
/// ::= 'aarch64_sme_preservemost_from_x1'
/// ::= 'aarch64_sme_preservemost_from_x2'
+/// ::= 'aarch64_outline_atomics_preservemost'
/// ::= 'msp430_intrcc'
/// ::= 'avr_intrcc'
/// ::= 'avr_signalcc'
@@ -2410,6 +2411,9 @@ bool LLParser::parseOptionalCallingConv(unsigned &CC) {
case lltok::kw_aarch64_sme_preservemost_from_x2:
CC = CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2;
break;
+ case lltok::kw_aarch64_outline_atomics_preservemost:
+ CC = CallingConv::AArch64_Outline_Atomics_PreserveMost;
+ break;
case lltok::kw_msp430_intrcc: CC = CallingConv::MSP430_INTR; break;
case lltok::kw_avr_intrcc: CC = CallingConv::AVR_INTR; break;
case lltok::kw_avr_signalcc: CC = CallingConv::AVR_SIGNAL; break;
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index b47458c28e499..9dc739a9e352c 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -361,6 +361,9 @@ static void printCallingConv(unsigned cc, raw_ostream &Out) {
case CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2:
Out << "aarch64_sme_preservemost_from_x2";
break;
+ case CallingConv::AArch64_Outline_Atomics_PreserveMost:
+ Out << "aarch64_outline_atomics_preservemost";
+ break;
case CallingConv::MSP430_INTR: Out << "msp430_intrcc"; break;
case CallingConv::AVR_INTR: Out << "avr_intrcc "; break;
case CallingConv::AVR_SIGNAL: Out << "avr_signalcc "; break;
diff --git a/llvm/lib/IR/Function.cpp b/llvm/lib/IR/Function.cpp
index c11e5f2ae1220..9fd405c6aed2a 100644
--- a/llvm/lib/IR/Function.cpp
+++ b/llvm/lib/IR/Function.cpp
@@ -1214,6 +1214,7 @@ bool llvm::CallingConv::supportsNonVoidReturnType(CallingConv::ID CC) {
case CallingConv::ARM64EC_Thunk_Native:
case CallingConv::RISCV_VectorCall:
case CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X1:
+ case CallingConv::AArch64_Outline_Atomics_PreserveMost:
case CallingConv::RISCV_VLSCall_32:
case CallingConv::RISCV_VLSCall_64:
case CallingConv::RISCV_VLSCall_128:
diff --git a/llvm/lib/Target/AArch64/AArch64CallingConvention.td b/llvm/lib/Target/AArch64/AArch64CallingConvention.td
index 5ad6045c09f50..8136d7af28778 100644
--- a/llvm/lib/Target/AArch64/AArch64CallingConvention.td
+++ b/llvm/lib/Target/AArch64/AArch64CallingConvention.td
@@ -702,6 +702,15 @@ def CSR_AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2
(sequence "X%u",19, 28),
LR, FP)>;
+// Out-of-line atomic helpers in compiler-rt (__aarch64_{cas,swp,ld*}*)
+// preserve all registers except argument/return registers (X0-X4),
+// scratch registers (X15-X17), and NZCV.
+def CSR_AArch64_Outline_Atomics_PreserveMost
+ : CalleeSavedRegs<(add (sequence "Z%u", 0, 31),
+ (sequence "P%u", 0, 15),
+ (sub (sequence "X%u", 5, 28), X15, X16, X17),
+ LR, FP)>;
+
// The SMSTART/SMSTOP instructions preserve only GPR registers.
def CSR_AArch64_SMStartStop : CalleeSavedRegs<(add (sequence "X%u", 0, 28),
LR, FP)>;
diff --git a/llvm/lib/Target/AArch64/AArch64Features.td b/llvm/lib/Target/AArch64/AArch64Features.td
index 07d01fae8856d..1897e3280ce97 100644
--- a/llvm/lib/Target/AArch64/AArch64Features.td
+++ b/llvm/lib/Target/AArch64/AArch64Features.td
@@ -665,6 +665,13 @@ def FeatureSMEF16F8F16 : ExtensionWithMArch<"sme-f16f8f16", "SME_F16F8F16", "FEA
def FeatureOutlineAtomics : SubtargetFeature<"outline-atomics", "OutlineAtomics", "true",
"Enable out of line atomics to support LSE instructions">;
+def FeatureOutlineAtomicsPreserveMost
+ : SubtargetFeature<"outline-atomics-preserve-most",
+ "OutlineAtomicsPreserveMost", "true",
+ "Use a preserve-most calling convention for out-of-line "
+ "atomic helper calls",
+ [], InlineIgnore>;
+
def FeatureFMV : SubtargetFeature<"fmv", "HasFMV", "true",
"Enable Function Multi Versioning support.">;
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index fbc7fdfc63c33..1c3b9d82ebf52 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -9285,6 +9285,7 @@ CCAssignFn *AArch64TargetLowering::CCAssignFnForCall(CallingConv::ID CC,
case CallingConv::SwiftTail:
case CallingConv::Tail:
case CallingConv::GRAAL:
+ case CallingConv::AArch64_Outline_Atomics_PreserveMost:
if (Subtarget->isTargetWindowsOrUEFI()) {
if (IsVarArg) {
if (Subtarget->isWindowsArm64EC())
diff --git a/llvm/lib/Target/AArch64/AArch64Options.td b/llvm/lib/Target/AArch64/AArch64Options.td
index 3cfea0ad209fc..b8d05bb416beb 100644
--- a/llvm/lib/Target/AArch64/AArch64Options.td
+++ b/llvm/lib/Target/AArch64/AArch64Options.td
@@ -232,6 +232,8 @@ defm : BoolField<"aarch64-use-tbi", "false",
"Assume that top byte of an address is ignored">;
defm : OptionalBoolField<"sve-use-scalar-inc-vl",
"Prefer add+cnt over addvl/inc/dec">;
+defm : OptionalBoolField<"aarch64-outline-atomics-preserve-most",
+ "Use a preserve-most calling convention for out-of-line atomic helper calls">;
defm : ValueField<"sve-vscale-for-tuning", "std::optional<unsigned>", "std::nullopt",
"Force a vscale for tuning factor for SVE">;
diff --git a/llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp b/llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
index d52841923cea8..303b02e72be9d 100644
--- a/llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
@@ -166,6 +166,11 @@ AArch64RegisterInfo::getCalleeSavedRegs(const MachineFunction *MF) const {
"only supported to improve calls to SME ACLE __arm_sme_state "
"and is not intended to be used beyond that scope.");
+ case CallingConv::AArch64_Outline_Atomics_PreserveMost:
+ report_fatal_error(
+ "Calling convention AArch64_Outline_Atomics_PreserveMost is only "
+ "supported for calls to out-of-line atomic helper routines.");
+
case CallingConv::Win64:
if (Darwin)
return CSR_Darwin_AArch64_AAPCS_Win64_SaveList;
@@ -250,6 +255,8 @@ AArch64RegisterInfo::getDarwinCallPreservedMask(const MachineFunction &MF,
return CSR_AArch64_SME_ABI_Support_Routines_PreserveMost_From_X1_RegMask;
if (CC == CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2)
return CSR_AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2_RegMask;
+ if (CC == CallingConv::AArch64_Outline_Atomics_PreserveMost)
+ return CSR_AArch64_Outline_Atomics_PreserveMost_RegMask;
if (CC == CallingConv::CFGuard_Check)
report_fatal_error(
"Calling convention CFGuard_Check is unsupported on Darwin.");
@@ -298,6 +305,8 @@ AArch64RegisterInfo::getCallPreservedMask(const MachineFunction &MF,
return CSR_AArch64_SME_ABI_Support_Routines_PreserveMost_From_X1_RegMask;
if (CC == CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2)
return CSR_AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2_RegMask;
+ if (CC == CallingConv::AArch64_Outline_Atomics_PreserveMost)
+ return CSR_AArch64_Outline_Atomics_PreserveMost_RegMask;
if (CC == CallingConv::CFGuard_Check)
return CSR_Win_AArch64_CFGuard_Check_RegMask;
if (MF.getSubtarget<AArch64Subtarget>().getTargetLowering()
@@ -738,6 +747,7 @@ bool AArch64RegisterInfo::isArgumentRegister(const MachineFunction &MF,
case CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X0:
case CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X1:
case CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2:
+ case CallingConv::AArch64_Outline_Atomics_PreserveMost:
if (STI.isTargetWindowsOrUEFI())
return HasReg(CC_AArch64_Win64PCS_ArgRegs, Reg);
return HasReg(CC_AArch64_AAPCS_ArgRegs, Reg);
diff --git a/llvm/lib/Target/AArch64/AArch64Subtarget.cpp b/llvm/lib/Target/AArch64/AArch64Subtarget.cpp
index 0adee4dc05611..3e31ef423599a 100644
--- a/llvm/lib/Target/AArch64/AArch64Subtarget.cpp
+++ b/llvm/lib/Target/AArch64/AArch64Subtarget.cpp
@@ -20,6 +20,7 @@
#include "GISel/AArch64LegalizerInfo.h"
#include "GISel/AArch64RegisterBankInfo.h"
#include "MCTargetDesc/AArch64AddressingModes.h"
+#include "llvm/Analysis/LibcallLoweringInfo.h"
#include "llvm/CodeGen/GlobalISel/InstructionSelect.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineScheduler.h"
@@ -545,6 +546,39 @@ bool AArch64Subtarget::useScalarIncVL() const {
return valueOr(CLOpts.sve_use_scalar_inc_vl, hasSVE2() || hasSME());
}
+bool AArch64Subtarget::useOutlineAtomicsPreserveMost() const {
+ if (isWindowsArm64EC())
+ return false;
+ return valueOr(CLOpts.outline_atomics_preserve_most,
+ OutlineAtomicsPreserveMost);
+}
+
+void AArch64Subtarget::initLibcallLoweringInfo(
+ LibcallLoweringInfo &Info) const {
+ if (!useOutlineAtomicsPreserveMost())
+ return;
+
+#define LCALLS(A, B) \
+ RTLIB::A##B##_RELAX, RTLIB::A##B##_ACQ, RTLIB::A##B##_REL, \
+ RTLIB::A##B##_ACQ_REL
+#define LCALL4(A) LCALLS(A, 1), LCALLS(A, 2), LCALLS(A, 4), LCALLS(A, 8)
+ static constexpr RTLIB::Libcall OutlineAtomicCalls[] = {
+ LCALL4(OUTLINE_ATOMIC_CAS), LCALLS(OUTLINE_ATOMIC_CAS, 16),
+ LCALL4(OUTLINE_ATOMIC_SWP), LCALL4(OUTLINE_ATOMIC_LDADD),
+ LCALL4(OUTLINE_ATOMIC_LDSET), LCALL4(OUTLINE_ATOMIC_LDCLR),
+ LCALL4(OUTLINE_ATOMIC_LDEOR),
+ };
+#undef LCALL4
+#undef LCALLS
+
+ for (RTLIB::Libcall LC : OutlineAtomicCalls) {
+ RTLIB::LibcallImpl Impl = Info.getLibcallImpl(LC);
+ if (Impl != RTLIB::Unsupported)
+ Info.setLibcallImplCallingConv(
+ Impl, CallingConv::AArch64_Outline_Atomics_PreserveMost);
+ }
+}
+
// If return address signing is enabled, tail calls are emitted as follows:
//
// ```
diff --git a/llvm/lib/Target/AArch64/AArch64Subtarget.h b/llvm/lib/Target/AArch64/AArch64Subtarget.h
index 17cf9f5d24765..70f5246270d51 100644
--- a/llvm/lib/Target/AArch64/AArch64Subtarget.h
+++ b/llvm/lib/Target/AArch64/AArch64Subtarget.h
@@ -178,6 +178,8 @@ class AArch64Subtarget final : public AArch64GenSubtargetInfo {
InstructionSelector *getInstructionSelector() const override;
const LegalizerInfo *getLegalizerInfo() const override;
const RegisterBankInfo *getRegBankInfo() const override;
+ void initLibcallLoweringInfo(LibcallLoweringInfo &Info) const override;
+ bool useOutlineAtomicsPreserveMost() const;
const Triple &getTargetTriple() const { return TargetTriple; }
bool enableMachineScheduler() const override { return true; }
bool enablePostRAScheduler() const override { return usePostRAScheduler(); }
@@ -435,6 +437,7 @@ class AArch64Subtarget final : public AArch64GenSubtargetInfo {
case CallingConv::Fast:
case CallingConv::Swift:
case CallingConv::SwiftTail:
+ case CallingConv::AArch64_Outline_Atomics_PreserveMost:
return isTargetWindowsOrUEFI();
case CallingConv::PreserveNone:
return IsVarArg && isTargetWindowsOrUEFI();
diff --git a/llvm/test/Bitcode/aarch64-outline-atomics-preserve-most.ll b/llvm/test/Bitcode/aarch64-outline-atomics-preserve-most.ll
new file mode 100644
index 0000000000000..3ca73f560e0ba
--- /dev/null
+++ b/llvm/test/Bitcode/aarch64-outline-atomics-preserve-most.ll
@@ -0,0 +1,22 @@
+; RUN: llvm-as %s -o - -f | llvm-dis | FileCheck %s
+; RUN: llvm-as %s -o - -f | verify-uselistorder
+
+declare aarch64_outline_atomics_preservemost i32 @helper_scalar(i32, ptr)
+; CHECK: declare aarch64_outline_atomics_preservemost i32 @helper_scalar(i32, ptr)
+
+declare aarch64_outline_atomics_preservemost i128 @helper_cas16(i128, i128, ptr)
+; CHECK: declare aarch64_outline_atomics_preservemost i128 @helper_cas16(i128, i128, ptr)
+
+define i32 @call_helper_scalar(i32 %v, ptr %p) {
+; CHECK-LABEL: define i32 @call_helper_scalar(
+; CHECK: %r = call aarch64_outline_atomics_preservemost i32 @helper_scalar(i32 %v, ptr %p)
+ %r = call aarch64_outline_atomics_preservemost i32 @helper_scalar(i32 %v, ptr %p)
+ ret i32 %r
+}
+
+define i128 @call_helper_cas16(i128 %e, i128 %d, ptr %p) {
+; CHECK-LABEL: define i128 @call_helper_cas16(
+; CHECK: %r = call aarch64_outline_atomics_preservemost i128 @helper_cas16(i128 %e, i128 %d, ptr %p)
+ %r = call aarch64_outline_atomics_preservemost i128 @helper_cas16(i128 %e, i128 %d, ptr %p)
+ ret i128 %r
+}
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-callee-error.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-callee-error.ll
new file mode 100644
index 0000000000000..bf1ad50a60df6
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-callee-error.ll
@@ -0,0 +1,7 @@
+; RUN: not --crash llc -mtriple=aarch64-unknown-linux-gnu -o - < %s 2>&1 | FileCheck %s
+; RUN: not --crash llc -mtriple=aarch64-unknown-fuchsia -o - < %s 2>&1 | FileCheck %s
+
+; CHECK: LLVM ERROR: Calling convention AArch64_Outline_Atomics_PreserveMost is only supported for calls to out-of-line atomic helper routines.
+define aarch64_outline_atomics_preservemost void @bad_callee() {
+ unreachable
+}
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
new file mode 100644
index 0000000000000..06fe8e84bf35a
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
@@ -0,0 +1,28 @@
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+sve,+outline-atomics -verify-machineinstrs < %s | FileCheck %s --check-prefix=DEFAULT
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+sve,+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+sve,+outline-atomics -verify-machineinstrs < %s | FileCheck %s --check-prefix=DEFAULT
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+sve,+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+
+; Verify that SVE scalable vector (z0-z31) and predicate (p0-p15) registers are
+; preserved across outline-atomic libcalls without scalable stack spills.
+define <vscale x 4 x i32> @ldset8_preserves_sve(ptr %p, i64 %v, <vscale x 4 x i32> %sv0, <vscale x 4 x i32> %sv1, <vscale x 4 x i1> %pg) nounwind {
+; DEFAULT-LABEL: ldset8_preserves_sve:
+; DEFAULT: addvl sp, sp,
+; DEFAULT: str z{{[0-9]+}},
+; DEFAULT: bl __aarch64_ldset8_acq_rel
+;
+; PRESERVE-LABEL: ldset8_preserves_sve:
+; PRESERVE-NOT: addvl
+; PRESERVE-NOT: str z
+; PRESERVE-NOT: str p
+; PRESERVE: bl __aarch64_ldset8_acq_rel
+; PRESERVE-NOT: ldr z
+; PRESERVE-NOT: ldr p
+; PRESERVE: add z0.s, p0/m, z0.s, z1.s
+; PRESERVE: ret
+ %old = atomicrmw or ptr %p, i64 %v acq_rel
+ %res = call <vscale x 4 x i32> @llvm.aarch64.sve.add.nxv4i32(<vscale x 4 x i1> %pg, <vscale x 4 x i32> %sv0, <vscale x 4 x i32> %sv1)
+ ret <vscale x 4 x i32> %res
+}
+
+declare <vscale x 4 x i32> @llvm.aarch64.sve.add.nxv4i32(<vscale x 4 x i1>, <vscale x 4 x i32>, <vscale x 4 x i32>)
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
new file mode 100644
index 0000000000000..053a7bd72eff1
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
@@ -0,0 +1,142 @@
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+outline-atomics -verify-machineinstrs < %s | FileCheck %s --check-prefix=DEFAULT
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+outline-atomics --aarch64-outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+outline-atomics,+outline-atomics-preserve-most -global-isel -global-isel-abort=1 -verify-machineinstrs < %s | FileCheck %s --check-prefix=GISEL
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -O0 -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=O0
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -O0 -global-isel=0 -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=O0
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=CSRMASK
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+outline-atomics -verify-machineinstrs < %s | FileCheck %s --check-prefix=DEFAULT
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+outline-atomics --aarch64-outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+outline-atomics,+outline-atomics-preserve-most -global-isel -global-isel-abort=1 -verify-machineinstrs < %s | FileCheck %s --check-prefix=GISEL
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -O0 -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=O0
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -O0 -global-isel=0 -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=O0
+; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=CSRMASK
+; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=CSRMASK
+; RUN: llc -mtriple=arm64ec-pc-windows-msvc -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=ARM64EC
+
+; Verify that live GPRs are held in preserved x5-x14 across 1/2/4/8-byte
+; outline-atomic libcalls without callee-saved stack spills.
+define i64 @ldadd4_preserves_gprs(ptr %p, i32 %v, i64 %a, i64 %b, i64 %c, i64 %d, i64 %e, i64 %f) nounwind {
+; DEFAULT-LABEL: ldadd4_preserves_gprs:
+; DEFAULT: stp x24, x23,
+; DEFAULT: stp x22, x21,
+; DEFAULT: stp x20, x19,
+; DEFAULT: bl __aarch64_ldadd4_acq_rel
+;
+; PRESERVE-LABEL: ldadd4_preserves_gprs:
+; PRESERVE-NOT: stp x
+; PRESERVE: bl __aarch64_ldadd4_acq_rel
+; PRESERVE-NOT: ldp x
+; PRESERVE: ret
+;
+; GISEL-LABEL: ldadd4_preserves_gprs:
+; GISEL-NOT: stp x
+; GISEL: bl __aarch64_ldadd4_acq_rel
+; GISEL: ret
+;
+; CSRMASK-LABEL: name: ldadd4_preserves_gprs
+; CSRMASK: BL &__aarch64_ldadd4_acq_rel, csr_aarch64_outline_atomics_preservemost
+; ARM64EC-LABEL: name: '#ldadd4_preserves_gprs'
+; ARM64EC: BL &"#__aarch64_ldadd4_acq_rel", csr_aarch64_aapcs
+ %old = atomicrmw add ptr %p, i32 %v acq_rel
+ %after = load volatile i32, ptr %p
+ %sum0 = add i32 %old, %after
+ %sum0z = zext i32 %sum0 to i64
+ %s1 = add i64 %a, %b
+ %s2 = add i64 %c, %d
+ %s3 = add i64 %e, %f
+ %s4 = add i64 %s1, %s2
+ %s5 = add i64 %s3, %sum0z
+ %res = add i64 %s4, %s5
+ ret i64 %res
+}
+
+; Verify that 16-byte CAS uses the same calling convention, keeping live values
+; in x5-x14 without callee-saved stack spills while treating x0-x4 as clobbered.
+define i128 @cas16_preserves_gprs(ptr %p, i128 %exp, i128 %des, i64 %extra) nounwind {
+; DEFAULT-LABEL: cas16_preserves_gprs:
+; DEFAULT: stp {{.*}}x19
+; DEFAULT: bl __aarch64_cas16_acq_rel
+;
+; PRESERVE-LABEL: cas16_preserves_gprs:
+; PRESERVE-NOT: x19
+; PRESERVE-NOT: x20
+; PRESERVE: bl __aarch64_cas16_acq_rel
+; PRESERVE: ret
+;
+; GISEL-LABEL: cas16_preserves_gprs:
+; GISEL-NOT: x19
+; GISEL-NOT: x20
+; GISEL: bl __aarch64_cas16_acq_rel
+; GISEL: ret
+;
+; CSRMASK-LABEL: name: cas16_preserves_gprs
+; CSRMASK: BL &__aarch64_cas16_acq_rel, csr_aarch64_outline_atomics_preservemost
+; ARM64EC-LABEL: name: '#cas16_preserves_gprs'
+; ARM64EC: BL &"#__aarch64_cas16_acq_rel", csr_aarch64_aapcs
+ %pair = cmpxchg ptr %p, i128 %exp, i128 %des acq_rel acquire
+ %old = extractvalue { i128, i1 } %pair, 0
+ %ext = zext i64 %extra to i128
+ %sum1 = add i128 %old, %des
+ %sum2 = add i128 %sum1, %ext
+ store volatile i128 %sum2, ptr %p
+ ret i128 %sum2
+}
+
+; Verify that NEON 128-bit vector registers (q0-q31) are preserved without spills.
+define <4 x i32> @swp8_preserves_simd(ptr %p, i64 %v, <4 x i32> %vec0, <4 x i32> %vec1) nounwind {
+; DEFAULT-LABEL: swp8_preserves_simd:
+; DEFAULT: stp q{{[0-9]+}}, q{{[0-9]+}}, [sp
+; DEFAULT: bl __aarch64_swp8_rel
+;
+; PRESERVE-LABEL: swp8_preserves_simd:
+; PRESERVE-NOT: stp q
+; PRESERVE-NOT: str q
+; PRESERVE: bl __aarch64_swp8_rel
+; PRESERVE-NOT: ldp q
+; PRESERVE-NOT: ldr q
+; PRESERVE: add v0.4s, v0.4s, v1.4s
+; PRESERVE: ret
+ %old = atomicrmw xchg ptr %p, i64 %v release
+ %res = add <4 x i32> %vec0, %vec1
+ ret <4 x i32> %res
+}
+
+; Verify the exact register boundaries:
+; - x4 (argument register) and x15-x17 (scratch) and NZCV are clobbered;
+; - x5 and x14 are preserved in place across the call.
+define i64 @exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary(ptr %p, i32 %v, i64 %x, i64 %y) nounwind {
+; PRESERVE-LABEL: exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary:
+; PRESERVE-NOT: mov {{x[0-9]+}}, x5
+; PRESERVE-NOT: mov {{x[0-9]+}}, x14
+; PRESERVE-DAG: mov [[SAVEX4:x[6-9]|x1[0-3]]], x4
+; PRESERVE-DAG: mov [[SAVEX15:x[6-9]|x1[0-3]]], x15
+; PRESERVE-DAG: mov [[SAVEX16:x[6-9]|x1[0-3]]], x16
+; PRESERVE-DAG: mov [[SAVEX17:x[6-9]|x1[0-3]]], x17
+; PRESERVE: bl __aarch64_ldclr4_relax
+; PRESERVE-DAG: mov x4, [[SAVEX4]]
+; PRESERVE-DAG: mov x15, [[SAVEX15]]
+; PRESERVE-DAG: mov x16, [[SAVEX16]]
+; PRESERVE-DAG: mov x17, [[SAVEX17]]
+; PRESERVE-DAG: cmp {{x[0-9]+}}, {{x[0-9]+}}
+; PRESERVE-DAG: csel x0,
+; PRESERVE-NOT: mov x5,
+; PRESERVE-NOT: mov x14,
+; PRESERVE: ret
+;
+; O0-LABEL: exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary:
+; O0: bl __aarch64_ldclr4_relax
+ %cmp = icmp eq i64 %x, %y
+ %r4 = call i64 asm sideeffect "", "={x4}"()
+ %r5 = call i64 asm sideeffect "", "={x5}"()
+ %r14 = call i64 asm sideeffect "", "={x14}"()
+ %r15 = call i64 asm sideeffect "", "={x15}"()
+ %r16 = call i64 asm sideeffect "", "={x16}"()
+ %r17 = call i64 asm sideeffect "", "={x17}"()
+ %old = atomicrmw and ptr %p, i32 %v monotonic
+ call void asm sideeffect "", "{x4},{x5},{x14},{x15},{x16},{x17}"(i64 %r4, i64 %r5, i64 %r14, i64 %r15, i64 %r16, i64 %r17)
+ %old64 = zext i32 %old to i64
+ %sel = select i1 %cmp, i64 %x, i64 %old64
+ ret i64 %sel
+}
diff --git a/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll b/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll
new file mode 100644
index 0000000000000..6c5f9b668d354
--- /dev/null
+++ b/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll
@@ -0,0 +1,88 @@
+;; Verify that +outline-atomics-preserve-most is InlineIgnore across ThinLTO
+;; module boundaries and lowers inlined atomicrmw operations using the
+;; preserve-most calling convention on both Linux and Fuchsia targets (via
+;; either function attributes or the backend flag).
+
+; RUN: split-file %s %t
+; RUN: sed -e "s,TRIPLE,aarch64-unknown-linux-gnu,g" %t/caller.ll | opt -module-summary -o %t/linux-caller.bc
+; RUN: sed -e "s,TRIPLE,aarch64-unknown-linux-gnu,g" %t/callee.ll | opt -module-summary -o %t/linux-callee.bc
+; RUN: llvm-lto2 run %t/linux-caller.bc %t/linux-callee.bc -o %t/linux-attr \
+; RUN: -r=%t/linux-caller.bc,caller,px \
+; RUN: -r=%t/linux-caller.bc,callee_atomic,l \
+; RUN: -r=%t/linux-callee.bc,callee_atomic,pl \
+; RUN: -r=%t/linux-callee.bc,callee_with_live_vals,
+; RUN: llvm-objdump -d -r %t/linux-attr.1 | FileCheck %s
+
+; RUN: llvm-lto2 run %t/linux-callee.bc -aarch64-outline-atomics-preserve-most -o %t/linux-flag \
+; RUN: -r=%t/linux-callee.bc,callee_with_live_vals,px \
+; RUN: -r=%t/linux-callee.bc,callee_atomic,pl
+; RUN: llvm-objdump -d -r %t/linux-flag.1 | FileCheck %s --check-prefix=FLAG
+
+; RUN: sed -e "s,TRIPLE,aarch64-unknown-fuchsia,g" %t/caller.ll | opt -module-summary -o %t/fuchsia-caller.bc
+; RUN: sed -e "s,TRIPLE,aarch64-unknown-fuchsia,g" %t/callee.ll | opt -module-summary -o %t/fuchsia-callee.bc
+; RUN: llvm-lto2 run %t/fuchsia-caller.bc %t/fuchsia-callee.bc -o %t/fuchsia-attr \
+; RUN: -r=%t/fuchsia-caller.bc,caller,px \
+; RUN: -r=%t/fuchsia-caller.bc,callee_atomic,l \
+; RUN: -r=%t/fuchsia-callee.bc,callee_atomic,pl \
+; RUN: -r=%t/fuchsia-callee.bc,callee_with_live_vals,
+; RUN: llvm-objdump -d -r %t/fuchsia-attr.1 | FileCheck %s
+
+; RUN: llvm-lto2 run %t/fuchsia-callee.bc -aarch64-outline-atomics-preserve-most -o %t/fuchsia-flag \
+; RUN: -r=%t/fuchsia-callee.bc,callee_with_live_vals,px \
+; RUN: -r=%t/fuchsia-callee.bc,callee_atomic,pl
+; RUN: llvm-objdump -d -r %t/fuchsia-flag.1 | FileCheck %s --check-prefix=FLAG
+
+; CHECK-LABEL: <caller>:
+; CHECK-NOT: stp x19
+; CHECK-NOT: stp x20
+; CHECK: bl {{.*}} <caller+{{.*}}>
+; CHECK-NEXT: R_AARCH64_CALL26 __aarch64_ldadd4_acq_rel
+; CHECK-NOT: ldp x19
+; CHECK: ret
+
+; FLAG-LABEL: <callee_with_live_vals>:
+; FLAG-NOT: stp x19
+; FLAG-NOT: stp x20
+; FLAG: bl {{.*}} <callee_with_live_vals+{{.*}}>
+; FLAG-NEXT: R_AARCH64_CALL26 __aarch64_ldadd4_acq_rel
+; FLAG-NOT: ldp x19
+; FLAG: ret
+
+;--- caller.ll
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128-Fn32"
+target triple = "TRIPLE"
+
+declare i32 @callee_atomic(ptr, i32)
+
+define i64 @caller(ptr %p, i32 %v, i64 %a, i64 %b, i64 %c, i64 %d) #0 {
+ %old = call i32 @callee_atomic(ptr %p, i32 %v)
+ %old64 = zext i32 %old to i64
+ %s1 = add i64 %a, %b
+ %s2 = add i64 %c, %d
+ %s3 = add i64 %s1, %s2
+ %res = add i64 %s3, %old64
+ ret i64 %res
+}
+
+attributes #0 = { nounwind "target-features"="+outline-atomics,+outline-atomics-preserve-most" }
+
+;--- callee.ll
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128-Fn32"
+target triple = "TRIPLE"
+
+define i32 @callee_atomic(ptr %p, i32 %v) #0 {
+ %old = atomicrmw add ptr %p, i32 %v acq_rel
+ ret i32 %old
+}
+
+define i64 @callee_with_live_vals(ptr %p, i32 %v, i64 %a, i64 %b, i64 %c, i64 %d) #0 {
+ %old = atomicrmw add ptr %p, i32 %v acq_rel
+ %old64 = zext i32 %old to i64
+ %s1 = add i64 %a, %b
+ %s2 = add i64 %c, %d
+ %s3 = add i64 %s1, %s2
+ %res = add i64 %s3, %old64
+ ret i64 %res
+}
+
+attributes #0 = { nounwind "target-features"="+outline-atomics" }
>From 80d95f68e510d4340405d80a1d3ffb406402f855 Mon Sep 17 00:00:00 2001
From: Roland McGrath <mcgrathr at google.com>
Date: Tue, 10 Nov 2026 12:28:27 -0800
Subject: [PATCH 2/5] [AArch64][compiler-rt] Use _preservemost symbols for
outline atomics
Use dedicated `__aarch64_*_preservemost` runtime helper symbol names
for `aarch64_outline_atomics_preservemost` instead of reusing the
ordinary `__aarch64_*` AAPCS64 symbols, and expose matching aliases in
`compiler-rt/lib/builtins/aarch64/lse.S`.
Using separate symbol names makes the calling convention ABI-safe when
linking against `libgcc`, prebuilt archives, or mixed-flag translation
units: callers with `-moutline-atomics-preserve-most` target
`__aarch64_*_preservemost`, while callers without it continue using
`__aarch64_*`. In `compiler-rt`'s `lse.S`, the existing routines
already modify only `x0`-`x4`, `x15`-`x17`, and `nzcv`, so aliasing
`__aarch64_*_preservemost` to `__aarch64_*` requires zero extra code.
Defining `AArch64OutlineAtomicPreserveMostImpls` in
`RuntimeLibcalls.td` with `AARCH64_OUTLINE_ATOMICS_PRESERVEMOST` also
registers these symbols in `LTO::getRuntimeLibcallSymbols` and removes
the per-subtarget calling convention override map from
`LibcallLoweringInfo`.
Assisted-by: Gemini
---
clang/docs/ReleaseNotes.md | 2 +-
.../AArch64/outline-atomics-preserve-most.c | 2 +-
compiler-rt/lib/builtins/aarch64/lse.S | 13 +
compiler-rt/lib/builtins/assembly.h | 11 +
llvm/docs/ReleaseNotes.md | 2 +-
.../llvm/Analysis/LibcallLoweringInfo.h | 18 +-
llvm/include/llvm/IR/RuntimeLibcalls.td | 30 +-
llvm/include/llvm/IR/RuntimeLibcallsImpl.td | 2 +
.../AArch64/AArch64CallingConvention.td | 2 +-
llvm/lib/Target/AArch64/AArch64Subtarget.cpp | 39 +-
.../outline-atomics-preserve-most-sve.ll | 2 +-
.../AArch64/outline-atomics-preserve-most.ll | 18 +-
.../AArch64/outline-atomics-preserve-most.ll | 4 +-
.../Util/DeclareRuntimeLibcalls/aarch64.ll | 500 ++++++++++++++++++
14 files changed, 591 insertions(+), 54 deletions(-)
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index fc88a9fee8037..21217fdd15907 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1347,7 +1347,7 @@ documentation](CIR/index.md) for more information regarding ClangIR's status and
`x86_64-unknown-uefi`, it follows the Microsoft ABI conventions (LLP64,
Microsoft C++ ABI, `char *` based `va_list`) and links with `lld-link`.
-- Added `-moutline-atomics-preserve-most` and `-mno-outline-atomics-preserve-most` for AArch64 targets to control whether out-of-line atomic helper calls preserve all registers except argument/return registers (`x0`-`x4`), scratch registers (`x15`-`x17`), and `NZCV`.
+- Added `-moutline-atomics-preserve-most` and `-mno-outline-atomics-preserve-most` for AArch64 targets to lower out-of-line atomic helper calls to `__aarch64_*_preservemost` entry points that preserve all registers except argument/return registers (`x0`-`x4`), scratch registers (`x15`-`x17`), and `NZCV`.
#### Android Support
diff --git a/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c b/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c
index 37d110a48a2f9..ab39c3e5093f0 100644
--- a/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c
+++ b/clang/test/CodeGen/AArch64/outline-atomics-preserve-most.c
@@ -10,7 +10,7 @@ int fetch_add_with_live_vals(_Atomic int *p, int v, int a, int b, int c, int d,
// ASM-LABEL: fetch_add_with_live_vals:
// ASM-NOT: stp x19
// ASM-NOT: stp x20
- // ASM: bl __aarch64_ldadd4_acq_rel
+ // ASM: bl __aarch64_ldadd4_acq_rel_preservemost
// ASM: ret
__c11_atomic_fetch_add(p, v, __ATOMIC_ACQ_REL);
return a + b + c + d + e + f;
diff --git a/compiler-rt/lib/builtins/aarch64/lse.S b/compiler-rt/lib/builtins/aarch64/lse.S
index a444d82892c36..b224fa8b480c3 100644
--- a/compiler-rt/lib/builtins/aarch64/lse.S
+++ b/compiler-rt/lib/builtins/aarch64/lse.S
@@ -116,6 +116,16 @@ HIDDEN(___aarch64_have_lse_atomics)
#endif
#define NAME(BASE) GLUE4(__aarch64_, BASE, SIZE, SUFF)
+#if !defined(__arm64ec__)
+// Note: lse.S routines only clobber x0-x4, x15-x17, and nzcv, satisfying both
+// the AAPCS64 and aarch64_outline_atomics_preservemost calling conventions.
+#define PRESERVEMOST_NAME(BASE) GLUE2(NAME(BASE), _preservemost)
+#define DEFINE_PRESERVEMOST_ALIAS(BASE) \
+ DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS_UNMANGLED(PRESERVEMOST_NAME(BASE), \
+ NAME(BASE))
+#else
+#define DEFINE_PRESERVEMOST_ALIAS(BASE)
+#endif
#if MODEL == 5
// Drop A for _sync functions.
#define LDXR GLUE3(ld, xr, S)
@@ -194,6 +204,7 @@ DEFINE_COMPILERRT_OUTLINE_FUNCTION_UNMANGLED(NAME(cas))
ret
#endif
END_COMPILERRT_OUTLINE_FUNCTION(NAME(cas))
+DEFINE_PRESERVEMOST_ALIAS(cas)
#endif // L_cas
#ifdef L_swp
@@ -215,6 +226,7 @@ DEFINE_COMPILERRT_OUTLINE_FUNCTION_UNMANGLED(NAME(swp))
BARRIER
ret
END_COMPILERRT_OUTLINE_FUNCTION(NAME(swp))
+DEFINE_PRESERVEMOST_ALIAS(swp)
#endif // L_swp
#if defined(L_ldadd) || defined(L_ldclr) || \
@@ -260,6 +272,7 @@ DEFINE_COMPILERRT_OUTLINE_FUNCTION_UNMANGLED(NAME(LDNM))
BARRIER
ret
END_COMPILERRT_OUTLINE_FUNCTION(NAME(LDNM))
+DEFINE_PRESERVEMOST_ALIAS(LDNM)
#endif // L_ldadd L_ldclr L_ldeor L_ldset
NO_EXEC_STACK_DIRECTIVE
diff --git a/compiler-rt/lib/builtins/assembly.h b/compiler-rt/lib/builtins/assembly.h
index 9195b3297b8c3..f72efbcfbd89d 100644
--- a/compiler-rt/lib/builtins/assembly.h
+++ b/compiler-rt/lib/builtins/assembly.h
@@ -355,10 +355,21 @@
#define END_COMPILERRT_OUTLINE_FUNCTION(name) \
CFI_END SEPARATOR \
.size SYMBOL_NAME(name), . - SYMBOL_NAME(name)
+#define DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS_UNMANGLED(name, target) \
+ .globl FUNC_SYMBOL(name) SEPARATOR \
+ SYMBOL_IS_FUNC(name) SEPARATOR \
+ DECLARE_SYMBOL_VISIBILITY_UNMANGLED(FUNC_SYMBOL(name)) SEPARATOR \
+ .set FUNC_SYMBOL(name), FUNC_SYMBOL(target) SEPARATOR \
+ .size SYMBOL_NAME(name), . - SYMBOL_NAME(target)
#else
#define END_COMPILERRT_FUNCTION(name)
#define END_COMPILERRT_OUTLINE_FUNCTION(name) \
CFI_END
+#define DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS_UNMANGLED(name, target) \
+ .globl FUNC_SYMBOL(name) SEPARATOR \
+ SYMBOL_IS_FUNC(name) SEPARATOR \
+ DECLARE_SYMBOL_VISIBILITY_UNMANGLED(FUNC_SYMBOL(name)) SEPARATOR \
+ .set FUNC_SYMBOL(name), FUNC_SYMBOL(target)
#endif
#ifdef __arm__
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 0d64f1374edaf..e4f940f8af9f3 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -303,7 +303,7 @@ Makes programs 10x faster by doing Special New Thing.
ARM64 ABI (calling convention, callee saved registers, stack probing and SEH
unwind information) and produces PE/COFF objects.
-* Added the `+outline-atomics-preserve-most` subtarget feature (and `--aarch64-outline-atomics-preserve-most` backend option) to lower AArch64 out-of-line atomic helper calls using `aarch64_outline_atomics_preservemost`, preserving all registers except `x0`-`x4`, `x15`-`x17`, and `NZCV`.
+* Added the `+outline-atomics-preserve-most` subtarget feature (and `--aarch64-outline-atomics-preserve-most` backend option) to lower AArch64 out-of-line atomic helper calls to `__aarch64_*_preservemost` entry points using `aarch64_outline_atomics_preservemost`, preserving all registers except `x0`-`x4`, `x15`-`x17`, and `NZCV`.
### Changes to the AMDGPU Backend
diff --git a/llvm/include/llvm/Analysis/LibcallLoweringInfo.h b/llvm/include/llvm/Analysis/LibcallLoweringInfo.h
index 22679bae055ba..2461bc7723ca1 100644
--- a/llvm/include/llvm/Analysis/LibcallLoweringInfo.h
+++ b/llvm/include/llvm/Analysis/LibcallLoweringInfo.h
@@ -20,7 +20,6 @@
#include "llvm/Analysis/RuntimeLibcallInfo.h"
#include "llvm/IR/RuntimeLibcalls.h"
#include "llvm/Pass.h"
-#include <cassert>
namespace llvm {
@@ -32,8 +31,6 @@ class LibcallLoweringInfo {
/// Stores the implementation choice for each libcall.
RTLIB::LibcallImpl LibcallImpls[RTLIB::UNKNOWN_LIBCALL + 1] = {
RTLIB::Unsupported};
- /// Stores context-specific (e.g. subtarget) calling convention overrides.
- DenseMap<unsigned, CallingConv::ID> CallingConvOverrides;
public:
/// Callback applying the caller's context-specific (e.g. subtarget) libcall
@@ -83,27 +80,14 @@ class LibcallLoweringInfo {
LibcallImpls[Call] = Impl;
}
- /// Override the CallingConv that should be used for \p Call in this lowering
- /// context.
- void setLibcallImplCallingConv(RTLIB::LibcallImpl Call, CallingConv::ID CC) {
- assert(Call != RTLIB::Unsupported &&
- "Cannot override calling convention for unsupported libcall");
- CallingConvOverrides[Call] = CC;
- }
-
// FIXME: Remove this wrapper in favor of directly using
// getLibcallImplCallingConv
CallingConv::ID getLibcallCallingConv(RTLIB::Libcall Call) const {
- return getLibcallImplCallingConv(LibcallImpls[Call]);
+ return RTLCI.LibcallImplCallingConvs[LibcallImpls[Call]];
}
/// Get the CallingConv that should be used for the specified libcall.
CallingConv::ID getLibcallImplCallingConv(RTLIB::LibcallImpl Call) const {
- if (!CallingConvOverrides.empty()) {
- auto It = CallingConvOverrides.find(Call);
- if (It != CallingConvOverrides.end())
- return It->second;
- }
return RTLCI.LibcallImplCallingConvs[Call];
}
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.td b/llvm/include/llvm/IR/RuntimeLibcalls.td
index c49e1c30784cb..6488b5ce5084d 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.td
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.td
@@ -15,6 +15,9 @@ include "llvm/IR/RuntimeLibcallsImpl.td"
class DuplicateLibcallImplWithPrefix<RuntimeLibcallImpl Impl, string prefix>
: RuntimeLibcallImpl<Impl.Provides, prefix#Impl.LibCallFuncName>;
+class DuplicateLibcallImplWithSuffix<RuntimeLibcallImpl Impl, string suffix>
+ : RuntimeLibcallImpl<Impl.Provides, Impl.LibCallFuncName#suffix>;
+
def IsOSDarwin : LibcallPredicate<[{TT.isOSDarwin()}]>;
def IsOSOpenBSD : LibcallPredicate<[{TT.isOSOpenBSD()}]>;
def IsOSWindows : LibcallPredicate<[{TT.isOSWindows()}]>;
@@ -2364,7 +2367,7 @@ def objc_sync_exit : RuntimeLibcallImpl<OBJC_SYNC_EXIT>;
// AArch64 Runtime Libcalls
//===----------------------------------------------------------------------===//
-defset list<RuntimeLibcallImpl> AArch64LibcallImpls = {
+defset list<RuntimeLibcallImpl> AArch64OutlineAtomicImpls = {
foreach MemSize = [1, 2, 4, 8, 16] in {
defm __aarch64_cas#MemSize
: AtomicOrderSizeLibcallImpl<"OUTLINE_ATOMIC_CAS"#MemSize>;
@@ -2382,12 +2385,23 @@ defset list<RuntimeLibcallImpl> AArch64LibcallImpls = {
defm __aarch64_ldeor#MemSize
: AtomicOrderSizeLibcallImpl<"OUTLINE_ATOMIC_LDEOR"#MemSize>;
}
+} // End AArch64OutlineAtomicImpls
+
+defset list<RuntimeLibcallImpl> AArch64OutlineAtomicPreserveMostImpls = {
+ foreach libcall = AArch64OutlineAtomicImpls in {
+ def libcall#_preservemost
+ : DuplicateLibcallImplWithSuffix<libcall, "_preservemost">;
+ }
+} // End AArch64OutlineAtomicPreserveMostImpls
+
+def __arm_sc_memcpy : RuntimeLibcallImpl<SC_MEMCPY>;
+def __arm_sc_memmove : RuntimeLibcallImpl<SC_MEMMOVE>;
+def __arm_sc_memset : RuntimeLibcallImpl<SC_MEMSET>;
+def __arm_sc_memchr : RuntimeLibcallImpl<SC_MEMCHR>;
- def __arm_sc_memcpy : RuntimeLibcallImpl<SC_MEMCPY>;
- def __arm_sc_memmove : RuntimeLibcallImpl<SC_MEMMOVE>;
- def __arm_sc_memset : RuntimeLibcallImpl<SC_MEMSET>;
- def __arm_sc_memchr : RuntimeLibcallImpl<SC_MEMCHR>;
-} // End AArch64LibcallImpls
+defvar AArch64LibcallImpls = !listconcat(AArch64OutlineAtomicImpls, [
+ __arm_sc_memcpy, __arm_sc_memmove, __arm_sc_memset, __arm_sc_memchr
+]);
def __arm_sme_state : RuntimeLibcallImpl<SMEABI_SME_STATE>;
def __arm_tpidr2_save : RuntimeLibcallImpl<SMEABI_TPIDR2_SAVE>;
@@ -2443,7 +2457,9 @@ def AArch64Libm : LibcallLibrary<"libm",
// AArch64 outline atomics and streaming-mode mem* helpers.
def AArch64OutlineAtomics : LibcallLibrary<"compiler-rt",
- (add AArch64LibcallImpls),
+ (add AArch64LibcallImpls,
+ LibcallsWithCC<(add AArch64OutlineAtomicPreserveMostImpls),
+ AARCH64_OUTLINE_ATOMICS_PRESERVEMOST>),
isAArch64_ExceptArm64EC>;
def AArch64SystemLibrary : SystemRuntimeLibrary<isAArch64_ExceptArm64EC,
diff --git a/llvm/include/llvm/IR/RuntimeLibcallsImpl.td b/llvm/include/llvm/IR/RuntimeLibcallsImpl.td
index 025d1d4e78095..8b0d025eed950 100644
--- a/llvm/include/llvm/IR/RuntimeLibcallsImpl.td
+++ b/llvm/include/llvm/IR/RuntimeLibcallsImpl.td
@@ -53,6 +53,8 @@ def SMEABI_PreserveMost_From_X0 : LibcallCallingConv<[{CallingConv::AArch64_SME_
def SMEABI_PreserveMost_From_X1 : LibcallCallingConv<[{CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X1}]>;
def SMEABI_PreserveMost_From_X2 : LibcallCallingConv<[{CallingConv::AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2}]>;
def AARCH64_VECTORCALL : LibcallCallingConv<[{CallingConv::AArch64_VectorCall}]>;
+def AARCH64_OUTLINE_ATOMICS_PRESERVEMOST
+ : LibcallCallingConv<[{CallingConv::AArch64_Outline_Atomics_PreserveMost}]>;
// Return type or argument type of library calls.
diff --git a/llvm/lib/Target/AArch64/AArch64CallingConvention.td b/llvm/lib/Target/AArch64/AArch64CallingConvention.td
index 8136d7af28778..e61c2bb4841e7 100644
--- a/llvm/lib/Target/AArch64/AArch64CallingConvention.td
+++ b/llvm/lib/Target/AArch64/AArch64CallingConvention.td
@@ -702,7 +702,7 @@ def CSR_AArch64_SME_ABI_Support_Routines_PreserveMost_From_X2
(sequence "X%u",19, 28),
LR, FP)>;
-// Out-of-line atomic helpers in compiler-rt (__aarch64_{cas,swp,ld*}*)
+// Out-of-line atomic preserve-most helpers (__aarch64_*_preservemost)
// preserve all registers except argument/return registers (X0-X4),
// scratch registers (X15-X17), and NZCV.
def CSR_AArch64_Outline_Atomics_PreserveMost
diff --git a/llvm/lib/Target/AArch64/AArch64Subtarget.cpp b/llvm/lib/Target/AArch64/AArch64Subtarget.cpp
index 3e31ef423599a..3c94c48492615 100644
--- a/llvm/lib/Target/AArch64/AArch64Subtarget.cpp
+++ b/llvm/lib/Target/AArch64/AArch64Subtarget.cpp
@@ -558,24 +558,35 @@ void AArch64Subtarget::initLibcallLoweringInfo(
if (!useOutlineAtomicsPreserveMost())
return;
-#define LCALLS(A, B) \
- RTLIB::A##B##_RELAX, RTLIB::A##B##_ACQ, RTLIB::A##B##_REL, \
- RTLIB::A##B##_ACQ_REL
-#define LCALL4(A) LCALLS(A, 1), LCALLS(A, 2), LCALLS(A, 4), LCALLS(A, 8)
- static constexpr RTLIB::Libcall OutlineAtomicCalls[] = {
- LCALL4(OUTLINE_ATOMIC_CAS), LCALLS(OUTLINE_ATOMIC_CAS, 16),
- LCALL4(OUTLINE_ATOMIC_SWP), LCALL4(OUTLINE_ATOMIC_LDADD),
- LCALL4(OUTLINE_ATOMIC_LDSET), LCALL4(OUTLINE_ATOMIC_LDCLR),
- LCALL4(OUTLINE_ATOMIC_LDEOR),
+ struct LibcallPair {
+ RTLIB::Libcall LC;
+ RTLIB::LibcallImpl Impl;
+ };
+#define LCALL(LC_BASE, IMPL_BASE, ORDER_UPPER, ORDER_LOWER) \
+ {RTLIB::OUTLINE_ATOMIC_##LC_BASE##_##ORDER_UPPER, \
+ RTLIB::impl___aarch64_##IMPL_BASE##_##ORDER_LOWER##_preservemost}
+#define LCALLS(LC_OP, IMPL_OP, SIZE) \
+ LCALL(LC_OP##SIZE, IMPL_OP##SIZE, RELAX, relax), \
+ LCALL(LC_OP##SIZE, IMPL_OP##SIZE, ACQ, acq), \
+ LCALL(LC_OP##SIZE, IMPL_OP##SIZE, REL, rel), \
+ LCALL(LC_OP##SIZE, IMPL_OP##SIZE, ACQ_REL, acq_rel)
+#define LCALL4(LC_OP, IMPL_OP) \
+ LCALLS(LC_OP, IMPL_OP, 1), LCALLS(LC_OP, IMPL_OP, 2), \
+ LCALLS(LC_OP, IMPL_OP, 4), LCALLS(LC_OP, IMPL_OP, 8)
+ static constexpr LibcallPair OutlineAtomicImpls[] = {
+ LCALL4(CAS, cas), LCALLS(CAS, cas, 16), LCALL4(SWP, swp),
+ LCALL4(LDADD, ldadd), LCALL4(LDSET, ldset), LCALL4(LDCLR, ldclr),
+ LCALL4(LDEOR, ldeor),
};
#undef LCALL4
#undef LCALLS
+#undef LCALL
- for (RTLIB::Libcall LC : OutlineAtomicCalls) {
- RTLIB::LibcallImpl Impl = Info.getLibcallImpl(LC);
- if (Impl != RTLIB::Unsupported)
- Info.setLibcallImplCallingConv(
- Impl, CallingConv::AArch64_Outline_Atomics_PreserveMost);
+ const RTLIB::RuntimeLibcallsInfo &RTLCI = Info.getRuntimeLibcallsInfo();
+ for (const auto &[LC, Impl] : OutlineAtomicImpls) {
+ if (Info.getLibcallImpl(LC) != RTLIB::Unsupported &&
+ RTLCI.isAvailable(Impl))
+ Info.setLibcallImpl(LC, Impl);
}
}
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
index 06fe8e84bf35a..3f65949c14cfa 100644
--- a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
@@ -15,7 +15,7 @@ define <vscale x 4 x i32> @ldset8_preserves_sve(ptr %p, i64 %v, <vscale x 4 x i3
; PRESERVE-NOT: addvl
; PRESERVE-NOT: str z
; PRESERVE-NOT: str p
-; PRESERVE: bl __aarch64_ldset8_acq_rel
+; PRESERVE: bl __aarch64_ldset8_acq_rel_preservemost
; PRESERVE-NOT: ldr z
; PRESERVE-NOT: ldr p
; PRESERVE: add z0.s, p0/m, z0.s, z1.s
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
index 053a7bd72eff1..0cf8d67d62bd2 100644
--- a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
@@ -26,17 +26,17 @@ define i64 @ldadd4_preserves_gprs(ptr %p, i32 %v, i64 %a, i64 %b, i64 %c, i64 %d
;
; PRESERVE-LABEL: ldadd4_preserves_gprs:
; PRESERVE-NOT: stp x
-; PRESERVE: bl __aarch64_ldadd4_acq_rel
+; PRESERVE: bl __aarch64_ldadd4_acq_rel_preservemost
; PRESERVE-NOT: ldp x
; PRESERVE: ret
;
; GISEL-LABEL: ldadd4_preserves_gprs:
; GISEL-NOT: stp x
-; GISEL: bl __aarch64_ldadd4_acq_rel
+; GISEL: bl __aarch64_ldadd4_acq_rel_preservemost
; GISEL: ret
;
; CSRMASK-LABEL: name: ldadd4_preserves_gprs
-; CSRMASK: BL &__aarch64_ldadd4_acq_rel, csr_aarch64_outline_atomics_preservemost
+; CSRMASK: BL &__aarch64_ldadd4_acq_rel_preservemost, csr_aarch64_outline_atomics_preservemost
; ARM64EC-LABEL: name: '#ldadd4_preserves_gprs'
; ARM64EC: BL &"#__aarch64_ldadd4_acq_rel", csr_aarch64_aapcs
%old = atomicrmw add ptr %p, i32 %v acq_rel
@@ -62,17 +62,17 @@ define i128 @cas16_preserves_gprs(ptr %p, i128 %exp, i128 %des, i64 %extra) noun
; PRESERVE-LABEL: cas16_preserves_gprs:
; PRESERVE-NOT: x19
; PRESERVE-NOT: x20
-; PRESERVE: bl __aarch64_cas16_acq_rel
+; PRESERVE: bl __aarch64_cas16_acq_rel_preservemost
; PRESERVE: ret
;
; GISEL-LABEL: cas16_preserves_gprs:
; GISEL-NOT: x19
; GISEL-NOT: x20
-; GISEL: bl __aarch64_cas16_acq_rel
+; GISEL: bl __aarch64_cas16_acq_rel_preservemost
; GISEL: ret
;
; CSRMASK-LABEL: name: cas16_preserves_gprs
-; CSRMASK: BL &__aarch64_cas16_acq_rel, csr_aarch64_outline_atomics_preservemost
+; CSRMASK: BL &__aarch64_cas16_acq_rel_preservemost, csr_aarch64_outline_atomics_preservemost
; ARM64EC-LABEL: name: '#cas16_preserves_gprs'
; ARM64EC: BL &"#__aarch64_cas16_acq_rel", csr_aarch64_aapcs
%pair = cmpxchg ptr %p, i128 %exp, i128 %des acq_rel acquire
@@ -93,7 +93,7 @@ define <4 x i32> @swp8_preserves_simd(ptr %p, i64 %v, <4 x i32> %vec0, <4 x i32>
; PRESERVE-LABEL: swp8_preserves_simd:
; PRESERVE-NOT: stp q
; PRESERVE-NOT: str q
-; PRESERVE: bl __aarch64_swp8_rel
+; PRESERVE: bl __aarch64_swp8_rel_preservemost
; PRESERVE-NOT: ldp q
; PRESERVE-NOT: ldr q
; PRESERVE: add v0.4s, v0.4s, v1.4s
@@ -114,7 +114,7 @@ define i64 @exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary(ptr %p, i32 %v, i64 %x
; PRESERVE-DAG: mov [[SAVEX15:x[6-9]|x1[0-3]]], x15
; PRESERVE-DAG: mov [[SAVEX16:x[6-9]|x1[0-3]]], x16
; PRESERVE-DAG: mov [[SAVEX17:x[6-9]|x1[0-3]]], x17
-; PRESERVE: bl __aarch64_ldclr4_relax
+; PRESERVE: bl __aarch64_ldclr4_relax_preservemost
; PRESERVE-DAG: mov x4, [[SAVEX4]]
; PRESERVE-DAG: mov x15, [[SAVEX15]]
; PRESERVE-DAG: mov x16, [[SAVEX16]]
@@ -126,7 +126,7 @@ define i64 @exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary(ptr %p, i32 %v, i64 %x
; PRESERVE: ret
;
; O0-LABEL: exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary:
-; O0: bl __aarch64_ldclr4_relax
+; O0: bl __aarch64_ldclr4_relax_preservemost
%cmp = icmp eq i64 %x, %y
%r4 = call i64 asm sideeffect "", "={x4}"()
%r5 = call i64 asm sideeffect "", "={x5}"()
diff --git a/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll b/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll
index 6c5f9b668d354..fa083c6ead7a8 100644
--- a/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll
+++ b/llvm/test/ThinLTO/AArch64/outline-atomics-preserve-most.ll
@@ -36,7 +36,7 @@
; CHECK-NOT: stp x19
; CHECK-NOT: stp x20
; CHECK: bl {{.*}} <caller+{{.*}}>
-; CHECK-NEXT: R_AARCH64_CALL26 __aarch64_ldadd4_acq_rel
+; CHECK-NEXT: R_AARCH64_CALL26 __aarch64_ldadd4_acq_rel_preservemost
; CHECK-NOT: ldp x19
; CHECK: ret
@@ -44,7 +44,7 @@
; FLAG-NOT: stp x19
; FLAG-NOT: stp x20
; FLAG: bl {{.*}} <callee_with_live_vals+{{.*}}>
-; FLAG-NEXT: R_AARCH64_CALL26 __aarch64_ldadd4_acq_rel
+; FLAG-NEXT: R_AARCH64_CALL26 __aarch64_ldadd4_acq_rel_preservemost
; FLAG-NOT: ldp x19
; FLAG: ret
diff --git a/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/aarch64.ll b/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/aarch64.ll
index 7b51109fb5b05..d1918f6dccaba 100644
--- a/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/aarch64.ll
+++ b/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/aarch64.ll
@@ -13,105 +13,205 @@
; LINUX-NEXT: declare i32 @_Unwind_RaiseException(ptr)
; LINUX-NEXT: declare void @_Unwind_Resume(ptr)
; LINUX-NEXT: declare i128 @__aarch64_cas16_acq(i128, i128, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_preservemost(i128, i128, ptr)
; LINUX-NEXT: declare i128 @__aarch64_cas16_acq_rel(i128, i128, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_rel_preservemost(i128, i128, ptr)
; LINUX-NEXT: declare i128 @__aarch64_cas16_rel(i128, i128, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_rel_preservemost(i128, i128, ptr)
; LINUX-NEXT: declare i128 @__aarch64_cas16_relax(i128, i128, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_relax_preservemost(i128, i128, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_cas1_acq(i8 zeroext, i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_preservemost(i8 zeroext, i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_cas1_acq_rel(i8 zeroext, i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_cas1_rel(i8 zeroext, i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_cas1_relax(i8 zeroext, i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_relax_preservemost(i8 zeroext, i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_cas2_acq(i16 zeroext, i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_preservemost(i16 zeroext, i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_cas2_acq_rel(i16 zeroext, i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_cas2_rel(i16 zeroext, i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_cas2_relax(i16 zeroext, i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_relax_preservemost(i16 zeroext, i16 zeroext, ptr)
; LINUX-NEXT: declare i32 @__aarch64_cas4_acq(i32, i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_preservemost(i32, i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_cas4_acq_rel(i32, i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_rel_preservemost(i32, i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_cas4_rel(i32, i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_rel_preservemost(i32, i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_cas4_relax(i32, i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_relax_preservemost(i32, i32, ptr)
; LINUX-NEXT: declare i64 @__aarch64_cas8_acq(i64, i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_preservemost(i64, i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_cas8_acq_rel(i64, i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_rel_preservemost(i64, i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_cas8_rel(i64, i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_rel_preservemost(i64, i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_cas8_relax(i64, i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_relax_preservemost(i64, i64, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldadd1_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldadd1_relax(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_relax_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldadd2_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldadd2_relax(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_relax_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldadd4_acq(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldadd4_acq_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldadd4_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldadd4_relax(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_relax_preservemost(i32, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldadd8_acq(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldadd8_acq_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldadd8_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldadd8_relax(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_relax_preservemost(i64, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldclr1_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldclr1_relax(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_relax_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldclr2_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldclr2_relax(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_relax_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldclr4_acq(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldclr4_acq_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldclr4_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldclr4_relax(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_relax_preservemost(i32, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldclr8_acq(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldclr8_acq_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldclr8_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldclr8_relax(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_relax_preservemost(i64, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldeor1_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldeor1_relax(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_relax_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldeor2_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldeor2_relax(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_relax_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldeor4_acq(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldeor4_acq_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldeor4_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldeor4_relax(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_relax_preservemost(i32, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldeor8_acq(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldeor8_acq_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldeor8_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldeor8_relax(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_relax_preservemost(i64, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldset1_acq(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldset1_acq_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldset1_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_ldset1_relax(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_relax_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldset2_acq(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldset2_acq_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldset2_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_ldset2_relax(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_relax_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldset4_acq(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldset4_acq_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldset4_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_ldset4_relax(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_relax_preservemost(i32, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldset8_acq(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldset8_acq_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldset8_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_ldset8_relax(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_relax_preservemost(i64, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_swp1_acq(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_swp1_acq_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_swp1_rel(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_rel_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i8 @__aarch64_swp1_relax(i8 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_relax_preservemost(i8 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_swp2_acq(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_swp2_acq_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_swp2_rel(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_rel_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare zeroext i16 @__aarch64_swp2_relax(i16 zeroext, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_relax_preservemost(i16 zeroext, ptr)
; LINUX-NEXT: declare i32 @__aarch64_swp4_acq(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_swp4_acq_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_swp4_rel(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_rel_preservemost(i32, ptr)
; LINUX-NEXT: declare i32 @__aarch64_swp4_relax(i32, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_relax_preservemost(i32, ptr)
; LINUX-NEXT: declare i64 @__aarch64_swp8_acq(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_swp8_acq_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_swp8_rel(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_rel_preservemost(i64, ptr)
; LINUX-NEXT: declare i64 @__aarch64_swp8_relax(i64, ptr)
+; LINUX-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_relax_preservemost(i64, ptr)
; LINUX-NEXT: declare double @__adddf3(double, double)
; LINUX-NEXT: declare float @__addsf3(float, float)
; LINUX-NEXT: declare fp128 @__addtf3(fp128, fp128)
@@ -605,105 +705,205 @@
; DARWIN-NEXT: declare i32 @_Unwind_RaiseException(ptr)
; DARWIN-NEXT: declare void @_Unwind_Resume(ptr)
; DARWIN-NEXT: declare i128 @__aarch64_cas16_acq(i128, i128, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_preservemost(i128, i128, ptr)
; DARWIN-NEXT: declare i128 @__aarch64_cas16_acq_rel(i128, i128, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_rel_preservemost(i128, i128, ptr)
; DARWIN-NEXT: declare i128 @__aarch64_cas16_rel(i128, i128, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_rel_preservemost(i128, i128, ptr)
; DARWIN-NEXT: declare i128 @__aarch64_cas16_relax(i128, i128, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_relax_preservemost(i128, i128, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_cas1_acq(i8 zeroext, i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_preservemost(i8 zeroext, i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_cas1_acq_rel(i8 zeroext, i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_cas1_rel(i8 zeroext, i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_cas1_relax(i8 zeroext, i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_relax_preservemost(i8 zeroext, i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_cas2_acq(i16 zeroext, i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_preservemost(i16 zeroext, i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_cas2_acq_rel(i16 zeroext, i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_cas2_rel(i16 zeroext, i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_cas2_relax(i16 zeroext, i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_relax_preservemost(i16 zeroext, i16 zeroext, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_cas4_acq(i32, i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_preservemost(i32, i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_cas4_acq_rel(i32, i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_rel_preservemost(i32, i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_cas4_rel(i32, i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_rel_preservemost(i32, i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_cas4_relax(i32, i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_relax_preservemost(i32, i32, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_cas8_acq(i64, i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_preservemost(i64, i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_cas8_acq_rel(i64, i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_rel_preservemost(i64, i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_cas8_rel(i64, i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_rel_preservemost(i64, i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_cas8_relax(i64, i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_relax_preservemost(i64, i64, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldadd1_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldadd1_relax(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_relax_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldadd2_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldadd2_relax(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_relax_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldadd4_acq(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldadd4_acq_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldadd4_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldadd4_relax(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_relax_preservemost(i32, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldadd8_acq(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldadd8_acq_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldadd8_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldadd8_relax(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_relax_preservemost(i64, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldclr1_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldclr1_relax(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_relax_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldclr2_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldclr2_relax(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_relax_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldclr4_acq(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldclr4_acq_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldclr4_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldclr4_relax(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_relax_preservemost(i32, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldclr8_acq(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldclr8_acq_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldclr8_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldclr8_relax(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_relax_preservemost(i64, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldeor1_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldeor1_relax(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_relax_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldeor2_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldeor2_relax(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_relax_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldeor4_acq(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldeor4_acq_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldeor4_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldeor4_relax(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_relax_preservemost(i32, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldeor8_acq(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldeor8_acq_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldeor8_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldeor8_relax(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_relax_preservemost(i64, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldset1_acq(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldset1_acq_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldset1_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_ldset1_relax(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_relax_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldset2_acq(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldset2_acq_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldset2_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_ldset2_relax(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_relax_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldset4_acq(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldset4_acq_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldset4_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_ldset4_relax(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_relax_preservemost(i32, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldset8_acq(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldset8_acq_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldset8_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_ldset8_relax(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_relax_preservemost(i64, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_swp1_acq(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_swp1_acq_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_swp1_rel(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_rel_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i8 @__aarch64_swp1_relax(i8 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_relax_preservemost(i8 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_swp2_acq(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_swp2_acq_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_swp2_rel(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_rel_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare zeroext i16 @__aarch64_swp2_relax(i16 zeroext, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_relax_preservemost(i16 zeroext, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_swp4_acq(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_swp4_acq_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_swp4_rel(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_rel_preservemost(i32, ptr)
; DARWIN-NEXT: declare i32 @__aarch64_swp4_relax(i32, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_relax_preservemost(i32, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_swp8_acq(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_swp8_acq_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_swp8_rel(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_rel_preservemost(i64, ptr)
; DARWIN-NEXT: declare i64 @__aarch64_swp8_relax(i64, ptr)
+; DARWIN-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_relax_preservemost(i64, ptr)
; DARWIN-NEXT: declare double @__adddf3(double, double)
; DARWIN-NEXT: declare float @__addsf3(float, float)
; DARWIN-NEXT: declare fp128 @__addtf3(fp128, fp128)
@@ -1140,105 +1340,205 @@
; WINMSVC-NEXT: declare i32 @_Unwind_RaiseException(ptr)
; WINMSVC-NEXT: declare void @_Unwind_Resume(ptr)
; WINMSVC-NEXT: declare i128 @__aarch64_cas16_acq(i128, i128, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_preservemost(i128, i128, ptr)
; WINMSVC-NEXT: declare i128 @__aarch64_cas16_acq_rel(i128, i128, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_rel_preservemost(i128, i128, ptr)
; WINMSVC-NEXT: declare i128 @__aarch64_cas16_rel(i128, i128, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_rel_preservemost(i128, i128, ptr)
; WINMSVC-NEXT: declare i128 @__aarch64_cas16_relax(i128, i128, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_relax_preservemost(i128, i128, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_cas1_acq(i8 zeroext, i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_preservemost(i8 zeroext, i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_cas1_acq_rel(i8 zeroext, i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_cas1_rel(i8 zeroext, i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_cas1_relax(i8 zeroext, i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_relax_preservemost(i8 zeroext, i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_cas2_acq(i16 zeroext, i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_preservemost(i16 zeroext, i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_cas2_acq_rel(i16 zeroext, i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_cas2_rel(i16 zeroext, i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_cas2_relax(i16 zeroext, i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_relax_preservemost(i16 zeroext, i16 zeroext, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_cas4_acq(i32, i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_preservemost(i32, i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_cas4_acq_rel(i32, i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_rel_preservemost(i32, i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_cas4_rel(i32, i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_rel_preservemost(i32, i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_cas4_relax(i32, i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_relax_preservemost(i32, i32, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_cas8_acq(i64, i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_preservemost(i64, i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_cas8_acq_rel(i64, i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_rel_preservemost(i64, i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_cas8_rel(i64, i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_rel_preservemost(i64, i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_cas8_relax(i64, i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_relax_preservemost(i64, i64, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldadd1_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldadd1_relax(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_relax_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldadd2_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldadd2_relax(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_relax_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldadd4_acq(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldadd4_acq_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldadd4_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldadd4_relax(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_relax_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldadd8_acq(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldadd8_acq_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldadd8_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldadd8_relax(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_relax_preservemost(i64, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldclr1_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldclr1_relax(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_relax_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldclr2_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldclr2_relax(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_relax_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldclr4_acq(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldclr4_acq_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldclr4_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldclr4_relax(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_relax_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldclr8_acq(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldclr8_acq_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldclr8_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldclr8_relax(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_relax_preservemost(i64, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldeor1_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldeor1_relax(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_relax_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldeor2_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldeor2_relax(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_relax_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldeor4_acq(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldeor4_acq_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldeor4_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldeor4_relax(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_relax_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldeor8_acq(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldeor8_acq_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldeor8_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldeor8_relax(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_relax_preservemost(i64, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldset1_acq(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldset1_acq_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldset1_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_ldset1_relax(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_relax_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldset2_acq(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldset2_acq_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldset2_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_ldset2_relax(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_relax_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldset4_acq(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldset4_acq_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldset4_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_ldset4_relax(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_relax_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldset8_acq(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldset8_acq_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldset8_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_ldset8_relax(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_relax_preservemost(i64, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_swp1_acq(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_swp1_acq_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_swp1_rel(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_rel_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i8 @__aarch64_swp1_relax(i8 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_relax_preservemost(i8 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_swp2_acq(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_swp2_acq_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_swp2_rel(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_rel_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare zeroext i16 @__aarch64_swp2_relax(i16 zeroext, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_relax_preservemost(i16 zeroext, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_swp4_acq(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_swp4_acq_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_swp4_rel(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_rel_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i32 @__aarch64_swp4_relax(i32, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_relax_preservemost(i32, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_swp8_acq(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_swp8_acq_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_swp8_rel(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_rel_preservemost(i64, ptr)
; WINMSVC-NEXT: declare i64 @__aarch64_swp8_relax(i64, ptr)
+; WINMSVC-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_relax_preservemost(i64, ptr)
; WINMSVC-NEXT: declare double @__adddf3(double, double)
; WINMSVC-NEXT: declare float @__addsf3(float, float)
; WINMSVC-NEXT: declare fp128 @__addtf3(fp128, fp128)
@@ -1669,105 +1969,205 @@
; MINGW-NEXT: declare i32 @_Unwind_RaiseException(ptr)
; MINGW-NEXT: declare void @_Unwind_Resume(ptr)
; MINGW-NEXT: declare i128 @__aarch64_cas16_acq(i128, i128, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_preservemost(i128, i128, ptr)
; MINGW-NEXT: declare i128 @__aarch64_cas16_acq_rel(i128, i128, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_rel_preservemost(i128, i128, ptr)
; MINGW-NEXT: declare i128 @__aarch64_cas16_rel(i128, i128, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_rel_preservemost(i128, i128, ptr)
; MINGW-NEXT: declare i128 @__aarch64_cas16_relax(i128, i128, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_relax_preservemost(i128, i128, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_cas1_acq(i8 zeroext, i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_preservemost(i8 zeroext, i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_cas1_acq_rel(i8 zeroext, i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_cas1_rel(i8 zeroext, i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_cas1_relax(i8 zeroext, i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_relax_preservemost(i8 zeroext, i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_cas2_acq(i16 zeroext, i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_preservemost(i16 zeroext, i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_cas2_acq_rel(i16 zeroext, i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_cas2_rel(i16 zeroext, i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_cas2_relax(i16 zeroext, i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_relax_preservemost(i16 zeroext, i16 zeroext, ptr)
; MINGW-NEXT: declare i32 @__aarch64_cas4_acq(i32, i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_preservemost(i32, i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_cas4_acq_rel(i32, i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_rel_preservemost(i32, i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_cas4_rel(i32, i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_rel_preservemost(i32, i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_cas4_relax(i32, i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_relax_preservemost(i32, i32, ptr)
; MINGW-NEXT: declare i64 @__aarch64_cas8_acq(i64, i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_preservemost(i64, i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_cas8_acq_rel(i64, i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_rel_preservemost(i64, i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_cas8_rel(i64, i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_rel_preservemost(i64, i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_cas8_relax(i64, i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_relax_preservemost(i64, i64, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldadd1_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldadd1_relax(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_relax_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldadd2_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldadd2_relax(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_relax_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldadd4_acq(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldadd4_acq_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldadd4_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldadd4_relax(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_relax_preservemost(i32, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldadd8_acq(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldadd8_acq_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldadd8_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldadd8_relax(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_relax_preservemost(i64, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldclr1_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldclr1_relax(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_relax_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldclr2_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldclr2_relax(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_relax_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldclr4_acq(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldclr4_acq_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldclr4_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldclr4_relax(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_relax_preservemost(i32, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldclr8_acq(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldclr8_acq_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldclr8_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldclr8_relax(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_relax_preservemost(i64, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldeor1_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldeor1_relax(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_relax_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldeor2_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldeor2_relax(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_relax_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldeor4_acq(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldeor4_acq_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldeor4_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldeor4_relax(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_relax_preservemost(i32, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldeor8_acq(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldeor8_acq_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldeor8_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldeor8_relax(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_relax_preservemost(i64, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldset1_acq(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldset1_acq_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldset1_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_ldset1_relax(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_relax_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldset2_acq(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldset2_acq_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldset2_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_ldset2_relax(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_relax_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldset4_acq(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldset4_acq_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldset4_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_ldset4_relax(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_relax_preservemost(i32, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldset8_acq(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldset8_acq_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldset8_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_ldset8_relax(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_relax_preservemost(i64, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_swp1_acq(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_swp1_acq_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_swp1_rel(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_rel_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i8 @__aarch64_swp1_relax(i8 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_relax_preservemost(i8 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_swp2_acq(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_swp2_acq_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_swp2_rel(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_rel_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare zeroext i16 @__aarch64_swp2_relax(i16 zeroext, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_relax_preservemost(i16 zeroext, ptr)
; MINGW-NEXT: declare i32 @__aarch64_swp4_acq(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_swp4_acq_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_swp4_rel(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_rel_preservemost(i32, ptr)
; MINGW-NEXT: declare i32 @__aarch64_swp4_relax(i32, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_relax_preservemost(i32, ptr)
; MINGW-NEXT: declare i64 @__aarch64_swp8_acq(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_swp8_acq_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_swp8_rel(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_rel_preservemost(i64, ptr)
; MINGW-NEXT: declare i64 @__aarch64_swp8_relax(i64, ptr)
+; MINGW-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_relax_preservemost(i64, ptr)
; MINGW-NEXT: declare double @__adddf3(double, double)
; MINGW-NEXT: declare float @__addsf3(float, float)
; MINGW-NEXT: declare fp128 @__addtf3(fp128, fp128)
@@ -2202,105 +2602,205 @@
; ANDROID-NEXT: declare i32 @_Unwind_RaiseException(ptr)
; ANDROID-NEXT: declare void @_Unwind_Resume(ptr)
; ANDROID-NEXT: declare i128 @__aarch64_cas16_acq(i128, i128, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_preservemost(i128, i128, ptr)
; ANDROID-NEXT: declare i128 @__aarch64_cas16_acq_rel(i128, i128, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_acq_rel_preservemost(i128, i128, ptr)
; ANDROID-NEXT: declare i128 @__aarch64_cas16_rel(i128, i128, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_rel_preservemost(i128, i128, ptr)
; ANDROID-NEXT: declare i128 @__aarch64_cas16_relax(i128, i128, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i128 @__aarch64_cas16_relax_preservemost(i128, i128, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_cas1_acq(i8 zeroext, i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_preservemost(i8 zeroext, i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_cas1_acq_rel(i8 zeroext, i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_acq_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_cas1_rel(i8 zeroext, i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_rel_preservemost(i8 zeroext, i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_cas1_relax(i8 zeroext, i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_cas1_relax_preservemost(i8 zeroext, i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_cas2_acq(i16 zeroext, i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_preservemost(i16 zeroext, i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_cas2_acq_rel(i16 zeroext, i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_acq_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_cas2_rel(i16 zeroext, i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_rel_preservemost(i16 zeroext, i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_cas2_relax(i16 zeroext, i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_cas2_relax_preservemost(i16 zeroext, i16 zeroext, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_cas4_acq(i32, i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_preservemost(i32, i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_cas4_acq_rel(i32, i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_acq_rel_preservemost(i32, i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_cas4_rel(i32, i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_rel_preservemost(i32, i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_cas4_relax(i32, i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_cas4_relax_preservemost(i32, i32, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_cas8_acq(i64, i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_preservemost(i64, i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_cas8_acq_rel(i64, i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_acq_rel_preservemost(i64, i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_cas8_rel(i64, i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_rel_preservemost(i64, i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_cas8_relax(i64, i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_cas8_relax_preservemost(i64, i64, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldadd1_acq_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_acq_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldadd1_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldadd1_relax(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldadd1_relax_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldadd2_acq_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_acq_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldadd2_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldadd2_relax(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldadd2_relax_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldadd4_acq(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldadd4_acq_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_acq_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldadd4_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldadd4_relax(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldadd4_relax_preservemost(i32, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldadd8_acq(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldadd8_acq_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_acq_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldadd8_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldadd8_relax(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldadd8_relax_preservemost(i64, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldclr1_acq_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_acq_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldclr1_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldclr1_relax(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldclr1_relax_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldclr2_acq_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_acq_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldclr2_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldclr2_relax(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldclr2_relax_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldclr4_acq(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldclr4_acq_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_acq_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldclr4_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldclr4_relax(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldclr4_relax_preservemost(i32, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldclr8_acq(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldclr8_acq_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_acq_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldclr8_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldclr8_relax(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldclr8_relax_preservemost(i64, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldeor1_acq_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_acq_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldeor1_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldeor1_relax(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldeor1_relax_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldeor2_acq_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_acq_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldeor2_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldeor2_relax(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldeor2_relax_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldeor4_acq(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldeor4_acq_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_acq_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldeor4_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldeor4_relax(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldeor4_relax_preservemost(i32, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldeor8_acq(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldeor8_acq_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_acq_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldeor8_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldeor8_relax(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldeor8_relax_preservemost(i64, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldset1_acq(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldset1_acq_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_acq_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldset1_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_ldset1_relax(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_ldset1_relax_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldset2_acq(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldset2_acq_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_acq_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldset2_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_ldset2_relax(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_ldset2_relax_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldset4_acq(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldset4_acq_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_acq_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldset4_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_ldset4_relax(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_ldset4_relax_preservemost(i32, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldset8_acq(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldset8_acq_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_acq_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldset8_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_ldset8_relax(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_ldset8_relax_preservemost(i64, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_swp1_acq(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_swp1_acq_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_acq_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_swp1_rel(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_rel_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i8 @__aarch64_swp1_relax(i8 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i8 @__aarch64_swp1_relax_preservemost(i8 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_swp2_acq(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_swp2_acq_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_acq_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_swp2_rel(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_rel_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare zeroext i16 @__aarch64_swp2_relax(i16 zeroext, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost zeroext i16 @__aarch64_swp2_relax_preservemost(i16 zeroext, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_swp4_acq(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_swp4_acq_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_acq_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_swp4_rel(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_rel_preservemost(i32, ptr)
; ANDROID-NEXT: declare i32 @__aarch64_swp4_relax(i32, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i32 @__aarch64_swp4_relax_preservemost(i32, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_swp8_acq(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_swp8_acq_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_acq_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_swp8_rel(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_rel_preservemost(i64, ptr)
; ANDROID-NEXT: declare i64 @__aarch64_swp8_relax(i64, ptr)
+; ANDROID-NEXT: declare aarch64_outline_atomics_preservemost i64 @__aarch64_swp8_relax_preservemost(i64, ptr)
; ANDROID-NEXT: declare double @__adddf3(double, double)
; ANDROID-NEXT: declare float @__addsf3(float, float)
; ANDROID-NEXT: declare fp128 @__addtf3(fp128, fp128)
>From db8d91e8d8fd2eff9450a6c34b3eede6b4b96112 Mon Sep 17 00:00:00 2001
From: Roland McGrath <mcgrathr at google.com>
Date: Tue, 10 Nov 2026 14:15:28 -0800
Subject: [PATCH 3/5] [AArch64][compiler-rt] Fix _preservemost Darwin mangling,
.variant_pcs, and SME
Address ABI and test gaps for `__aarch64_*_preservemost`:
- Mangle the alias symbol with `SYMBOL_NAME` in
`DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS` so Mach-O exports
`___aarch64_*_preservemost` matching `AsmPrinter`, and use
`FUNC_SYMBOL(target)` consistently in ELF `.size`.
- Mark ELF `__aarch64_*_preservemost` aliases `.variant_pcs`
(`STO_AARCH64_VARIANT_PCS`) per AAELF64 so dynamic PLT resolution
eagerly binds rather than entering `_dl_runtime_resolve`.
- Mark `CallingConv::AArch64_Outline_Atomics_PreserveMost` runtime
libcalls `SM_Compatible | ZA_State_Agnostic` in
`SMEAttrs::addKnownFunctionAttrs` so SME callers do not emit
`smstop`/`smstart` or `ZA`/`ZT0` saves around scalar helpers.
- Emit `-Woption-ignored` for `-m[no-]outline-atomics-preserve-most` on
`arm64ec` in the Clang driver, tighten `FileCheck` symbol/register
boundaries, and add Darwin, `+lse`, `-outline-atomics`, SME, and
`compiler-rt` builtins unit tests.
Assisted-by: Gemini
---
clang/lib/Driver/ToolChains/CommonArgs.cpp | 44 ++++++------
.../test/Driver/unsupported-outline-atomics.c | 8 +++
compiler-rt/lib/builtins/aarch64/lse.S | 3 +-
compiler-rt/lib/builtins/assembly.h | 31 +++++---
...arch64_outline_atomics_preservemost_test.c | 72 +++++++++++++++++++
.../Target/AArch64/AArch64SMEAttributes.cpp | 5 ++
.../outline-atomics-preserve-most-sme.ll | 22 ++++++
.../outline-atomics-preserve-most-sve.ll | 4 +-
.../AArch64/outline-atomics-preserve-most.ll | 32 +++++++--
9 files changed, 181 insertions(+), 40 deletions(-)
create mode 100644 compiler-rt/test/builtins/Unit/aarch64_outline_atomics_preservemost_test.c
create mode 100644 llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sme.ll
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index 6b48129c8d276..c3b1f6aac8e3b 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -1194,7 +1194,7 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
// LowerMatrixIntrinsicsPass, which is transitively called by
// buildThinLTODefaultPipeline under EnableMatrix.
if ((IsThinLTO || IsFatLTO || IsUnifiedLTO) &&
- Args.hasArg(options::OPT_fenable_matrix))
+ Args.hasArg(options::OPT_fenable_matrix))
CmdArgs.push_back(
Args.MakeArgString(Twine(PluginOptPrefix) + "-enable-matrix"));
@@ -1551,7 +1551,7 @@ bool tools::addOpenMPRuntime(const Compilation &C, ArgStringList &CmdArgs,
CmdArgs.push_back("-Bdynamic");
if (RTKind == Driver::OMPRT_GOMP && GompNeedsRT)
- CmdArgs.push_back("-lrt");
+ CmdArgs.push_back("-lrt");
if (IsOffloadingHost)
CmdArgs.push_back("-lomptarget");
@@ -1585,7 +1585,8 @@ static void addSanitizerRuntime(const ToolChain &TC, const ArgList &Args,
bool IsShared, bool IsWhole) {
// Wrap any static runtimes that must be forced into executable in
// whole-archive.
- if (IsWhole) CmdArgs.push_back("--whole-archive");
+ if (IsWhole)
+ CmdArgs.push_back("--whole-archive");
CmdArgs.push_back(TC.getCompilerRTArgString(
Args, Sanitizer, IsShared ? ToolChain::FT_Shared : ToolChain::FT_Static));
if (IsWhole)
@@ -1968,7 +1969,8 @@ bool tools::addSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
FuzzerNeedsSanitizerDeps;
}
-bool tools::addXRayRuntime(const ToolChain&TC, const ArgList &Args, ArgStringList &CmdArgs) {
+bool tools::addXRayRuntime(const ToolChain &TC, const ArgList &Args,
+ ArgStringList &CmdArgs) {
const XRayArgs &XRay = TC.getXRayArgs(Args);
if (Args.hasArg(options::OPT_shared)) {
if (XRay.needsXRayDSORt()) {
@@ -1998,8 +2000,7 @@ void tools::linkXRayRuntimeDeps(const ToolChain &TC,
CmdArgs.push_back("-lrt");
CmdArgs.push_back("-lm");
- if (!TC.getTriple().isOSFreeBSD() &&
- !TC.getTriple().isOSNetBSD() &&
+ if (!TC.getTriple().isOSFreeBSD() && !TC.getTriple().isOSNetBSD() &&
!TC.getTriple().isOSOpenBSD())
CmdArgs.push_back("-ldl");
}
@@ -2276,19 +2277,19 @@ tools::ParsePICArgs(const ToolChain &ToolChain, const ArgList &Args) {
bool EmbeddedPISupported;
switch (Triple.getArch()) {
- case llvm::Triple::arm:
- case llvm::Triple::armeb:
- case llvm::Triple::thumb:
- case llvm::Triple::thumbeb:
- EmbeddedPISupported = true;
- break;
- default:
- EmbeddedPISupported = false;
- break;
+ case llvm::Triple::arm:
+ case llvm::Triple::armeb:
+ case llvm::Triple::thumb:
+ case llvm::Triple::thumbeb:
+ EmbeddedPISupported = true;
+ break;
+ default:
+ EmbeddedPISupported = false;
+ break;
}
bool ROPI = false, RWPI = false;
- Arg* LastROPIArg = Args.getLastArg(options::OPT_fropi, options::OPT_fno_ropi);
+ Arg *LastROPIArg = Args.getLastArg(options::OPT_fropi, options::OPT_fno_ropi);
if (LastROPIArg && LastROPIArg->getOption().matches(options::OPT_fropi)) {
if (!EmbeddedPISupported)
ToolChain.getDriver().Diag(diag::err_drv_unsupported_opt_for_target)
@@ -2317,7 +2318,7 @@ tools::ParsePICArgs(const ToolChain &ToolChain, const ArgList &Args) {
if (ABIName == "n64")
PIC = true;
// When targettng MIPS with -mno-abicalls, it's always static.
- if(Args.hasArg(options::OPT_mno_abicalls))
+ if (Args.hasArg(options::OPT_mno_abicalls))
return std::make_tuple(llvm::Reloc::Static, 0U, false);
// Unlike other architectures, MIPS, even with -fPIC/-mxgot/multigot,
// does not use PIC level 2 for historical reasons.
@@ -2537,7 +2538,8 @@ enum class LibGccType { UnspecifiedLibGcc, StaticLibGcc, SharedLibGcc };
static LibGccType getLibGccType(const ToolChain &TC, const Driver &D,
const ArgList &Args) {
if (Args.hasArg(options::OPT_static_libgcc) ||
- Args.hasArg(options::OPT_static) || Args.hasArg(options::OPT_static_pie) ||
+ Args.hasArg(options::OPT_static) ||
+ Args.hasArg(options::OPT_static_pie) ||
// The Android NDK only provides libunwind.a, not libunwind.so.
TC.getTriple().isAndroid())
return LibGccType::StaticLibGcc;
@@ -2813,8 +2815,8 @@ void tools::addMachineOutlinerArgs(const Driver &D,
}
};
- if (Arg *A = Args.getLastArg(options::OPT_moutline,
- options::OPT_mno_outline)) {
+ if (Arg *A =
+ Args.getLastArg(options::OPT_moutline, options::OPT_mno_outline)) {
if (A->getOption().matches(options::OPT_moutline)) {
// We only support -moutline in AArch64, ARM, RISC-V and X86 targets right
// now. If we're compiling for these, add the proper mllvm flags.
@@ -3071,7 +3073,7 @@ void tools::addOutlineAtomicsArgs(const Driver &D, const ToolChain &TC,
if (Arg *A =
Args.getLastArg(options::OPT_moutline_atomics_preserve_most,
options::OPT_mno_outline_atomics_preserve_most)) {
- if (!Triple.isAArch64()) {
+ if (!Triple.isAArch64() || Triple.isWindowsArm64EC()) {
D.Diag(diag::warn_drv_moutline_atomics_unsupported_opt)
<< Triple.getArchName() << A->getOption().getName();
} else if (A->getOption().matches(
diff --git a/clang/test/Driver/unsupported-outline-atomics.c b/clang/test/Driver/unsupported-outline-atomics.c
index 2cba7fb0d1dfb..7389af7f4ba3c 100644
--- a/clang/test/Driver/unsupported-outline-atomics.c
+++ b/clang/test/Driver/unsupported-outline-atomics.c
@@ -21,3 +21,11 @@
// RUN: %clang -target x86_64 -mno-outline-atomics-preserve-most -S %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-NO-PRESERVE-MOST-X86
// CHECK-NO-PRESERVE-MOST-X86: warning: 'x86_64' does not support '-mno-outline-atomics-preserve-most'; flag ignored [-Woption-ignored]
// CHECK-NO-PRESERVE-MOST-X86-NOT: "-target-feature" "-outline-atomics-preserve-most"
+
+// RUN: %clang -target arm64ec-pc-windows-msvc -moutline-atomics-preserve-most -S %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-PRESERVE-MOST-ARM64EC
+// CHECK-PRESERVE-MOST-ARM64EC: warning: 'arm64ec' does not support '-moutline-atomics-preserve-most'; flag ignored [-Woption-ignored]
+// CHECK-PRESERVE-MOST-ARM64EC-NOT: "-target-feature" "+outline-atomics-preserve-most"
+
+// RUN: %clang -target arm64ec-pc-windows-msvc -mno-outline-atomics-preserve-most -S %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-NO-PRESERVE-MOST-ARM64EC
+// CHECK-NO-PRESERVE-MOST-ARM64EC: warning: 'arm64ec' does not support '-mno-outline-atomics-preserve-most'; flag ignored [-Woption-ignored]
+// CHECK-NO-PRESERVE-MOST-ARM64EC-NOT: "-target-feature" "-outline-atomics-preserve-most"
diff --git a/compiler-rt/lib/builtins/aarch64/lse.S b/compiler-rt/lib/builtins/aarch64/lse.S
index b224fa8b480c3..4f28ffafbb6f5 100644
--- a/compiler-rt/lib/builtins/aarch64/lse.S
+++ b/compiler-rt/lib/builtins/aarch64/lse.S
@@ -121,8 +121,7 @@ HIDDEN(___aarch64_have_lse_atomics)
// the AAPCS64 and aarch64_outline_atomics_preservemost calling conventions.
#define PRESERVEMOST_NAME(BASE) GLUE2(NAME(BASE), _preservemost)
#define DEFINE_PRESERVEMOST_ALIAS(BASE) \
- DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS_UNMANGLED(PRESERVEMOST_NAME(BASE), \
- NAME(BASE))
+ DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS(PRESERVEMOST_NAME(BASE), NAME(BASE))
#else
#define DEFINE_PRESERVEMOST_ALIAS(BASE)
#endif
diff --git a/compiler-rt/lib/builtins/assembly.h b/compiler-rt/lib/builtins/assembly.h
index f72efbcfbd89d..5f0c5100d3cb0 100644
--- a/compiler-rt/lib/builtins/assembly.h
+++ b/compiler-rt/lib/builtins/assembly.h
@@ -349,28 +349,37 @@
#define DEFINE_AEABI_FUNCTION_ALIAS(aeabi_name, name)
#endif
+// clang-format off
+#if (defined(__aarch64__) || defined(__arm64ec__)) && defined(__ELF__)
+#define VARIANT_PCS(name) .variant_pcs name SEPARATOR
+#else
+#define VARIANT_PCS(name)
+#endif
+
#ifdef __ELF__
#define END_COMPILERRT_FUNCTION(name) \
.size SYMBOL_NAME(name), . - SYMBOL_NAME(name)
#define END_COMPILERRT_OUTLINE_FUNCTION(name) \
CFI_END SEPARATOR \
.size SYMBOL_NAME(name), . - SYMBOL_NAME(name)
-#define DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS_UNMANGLED(name, target) \
- .globl FUNC_SYMBOL(name) SEPARATOR \
- SYMBOL_IS_FUNC(name) SEPARATOR \
- DECLARE_SYMBOL_VISIBILITY_UNMANGLED(FUNC_SYMBOL(name)) SEPARATOR \
- .set FUNC_SYMBOL(name), FUNC_SYMBOL(target) SEPARATOR \
- .size SYMBOL_NAME(name), . - SYMBOL_NAME(target)
+#define DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS(name, target) \
+ .globl FUNC_SYMBOL(SYMBOL_NAME(name)) SEPARATOR \
+ SYMBOL_IS_FUNC(SYMBOL_NAME(name)) SEPARATOR \
+ VARIANT_PCS(FUNC_SYMBOL(SYMBOL_NAME(name))) \
+ DECLARE_SYMBOL_VISIBILITY(name) \
+ .set FUNC_SYMBOL(SYMBOL_NAME(name)), FUNC_SYMBOL(target) SEPARATOR \
+ .size FUNC_SYMBOL(SYMBOL_NAME(name)), . - FUNC_SYMBOL(target)
#else
#define END_COMPILERRT_FUNCTION(name)
#define END_COMPILERRT_OUTLINE_FUNCTION(name) \
CFI_END
-#define DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS_UNMANGLED(name, target) \
- .globl FUNC_SYMBOL(name) SEPARATOR \
- SYMBOL_IS_FUNC(name) SEPARATOR \
- DECLARE_SYMBOL_VISIBILITY_UNMANGLED(FUNC_SYMBOL(name)) SEPARATOR \
- .set FUNC_SYMBOL(name), FUNC_SYMBOL(target)
+#define DEFINE_COMPILERRT_OUTLINE_FUNCTION_ALIAS(name, target) \
+ .globl FUNC_SYMBOL(SYMBOL_NAME(name)) SEPARATOR \
+ SYMBOL_IS_FUNC(SYMBOL_NAME(name)) SEPARATOR \
+ DECLARE_SYMBOL_VISIBILITY(name) \
+ .set FUNC_SYMBOL(SYMBOL_NAME(name)), FUNC_SYMBOL(target)
#endif
+// clang-format on
#ifdef __arm__
#include "int_endianness.h"
diff --git a/compiler-rt/test/builtins/Unit/aarch64_outline_atomics_preservemost_test.c b/compiler-rt/test/builtins/Unit/aarch64_outline_atomics_preservemost_test.c
new file mode 100644
index 0000000000000..482559c77d894
--- /dev/null
+++ b/compiler-rt/test/builtins/Unit/aarch64_outline_atomics_preservemost_test.c
@@ -0,0 +1,72 @@
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+// REQUIRES: aarch64-target-arch
+// REQUIRES: librt_has_aarch64
+// RUN: %clang_builtins %s %librt -o %t && %run %t
+
+#include <stdint.h>
+
+#if defined(__aarch64__) && !defined(__arm64ec__)
+
+# define STR1(x) #x
+# define STR(x) STR1(x)
+# define ASM_SYM(name) STR(__USER_LABEL_PREFIX__) #name
+
+static int check_ldadd4_preservemost(void) {
+ uint32_t mem = 40;
+ register uint64_t x0 __asm__("x0") = 2;
+ register uint32_t *x1 __asm__("x1") = &mem;
+ register uint64_t x5 __asm__("x5") = 0x0505050550505050ULL;
+ register uint64_t x8 __asm__("x8") = 0x0808080880808080ULL;
+ register uint64_t x13 __asm__("x13") = 0x0d0d0d0dd0d0d0d0ULL;
+ register uint64_t x14 __asm__("x14") = 0x0e0e0e0ee0e0e0e0ULL;
+ __asm__ volatile(
+ "bl " ASM_SYM(__aarch64_ldadd4_acq_rel_preservemost)
+ : "+r"(x0), "+r"(x1), "+r"(x5), "+r"(x8), "+r"(x13), "+r"(x14)
+ :
+ : "x2", "x3", "x4", "x15", "x16", "x17", "x30", "cc", "memory");
+ if ((uint32_t)x0 != 40 || mem != 42)
+ return 1;
+ if (x5 != 0x0505050550505050ULL || x8 != 0x0808080880808080ULL ||
+ x13 != 0x0d0d0d0dd0d0d0d0ULL || x14 != 0x0e0e0e0ee0e0e0e0ULL)
+ return 2;
+ return 0;
+}
+
+static int check_cas16_preservemost(void) {
+ __attribute__((aligned(16))) uint64_t pair[2] = {11, 22};
+ register uint64_t x0 __asm__("x0") = 11;
+ register uint64_t x1 __asm__("x1") = 22;
+ register uint64_t x2 __asm__("x2") = 33;
+ register uint64_t x3 __asm__("x3") = 44;
+ register uint64_t *x4 __asm__("x4") = pair;
+ register uint64_t x5 __asm__("x5") = 0x5555aaaa5555aaaaULL;
+ register uint64_t x13 __asm__("x13") = 0xdddd2222dddd2222ULL;
+ register uint64_t x14 __asm__("x14") = 0xeeee1111eeee1111ULL;
+ __asm__ volatile("bl " ASM_SYM(__aarch64_cas16_acq_rel_preservemost)
+ : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5),
+ "+r"(x13), "+r"(x14)
+ :
+ : "x15", "x16", "x17", "x30", "cc", "memory");
+ if (x0 != 11 || x1 != 22 || pair[0] != 33 || pair[1] != 44)
+ return 3;
+ if (x5 != 0x5555aaaa5555aaaaULL || x13 != 0xdddd2222dddd2222ULL ||
+ x14 != 0xeeee1111eeee1111ULL)
+ return 4;
+ return 0;
+}
+
+#endif
+
+int main(void) {
+#if defined(__aarch64__) && !defined(__arm64ec__)
+ int rc = check_ldadd4_preservemost();
+ if (rc != 0)
+ return rc;
+ return check_cas16_preservemost();
+#else
+ return 0;
+#endif
+}
diff --git a/llvm/lib/Target/AArch64/AArch64SMEAttributes.cpp b/llvm/lib/Target/AArch64/AArch64SMEAttributes.cpp
index fa4732bf3e8b5..c7c56630d5dff 100644
--- a/llvm/lib/Target/AArch64/AArch64SMEAttributes.cpp
+++ b/llvm/lib/Target/AArch64/AArch64SMEAttributes.cpp
@@ -78,6 +78,11 @@ void SMEAttrs::addKnownFunctionAttrs(StringRef FuncName,
RTLIB::LibcallImpl Impl = RTLCI.getSupportedLibcallImpl(FuncName);
if (Impl == RTLIB::Unsupported)
return;
+ if (RTLCI.getLibcallImplCallingConv(Impl) ==
+ CallingConv::AArch64_Outline_Atomics_PreserveMost) {
+ set(SMEAttrs::SM_Compatible | SMEAttrs::ZA_State_Agnostic);
+ return;
+ }
unsigned KnownAttrs = SMEAttrs::Normal;
RTLIB::Libcall LC = RTLCI.getLibcallFromImpl(Impl);
switch (LC) {
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sme.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sme.ll
new file mode 100644
index 0000000000000..c1d6e6ac2b001
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sme.ll
@@ -0,0 +1,22 @@
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+sme2,+outline-atomics -verify-machineinstrs < %s | FileCheck %s --check-prefix=DEFAULT
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+sme2,+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=PRESERVE
+
+; Verify that __aarch64_*_preservemost helpers are treated as streaming-compatible
+; and ZA/ZT0-agnostic, avoiding smstop/smstart and ZA/ZT0 saves in SME callers.
+define i64 @ldadd8_sme_streaming_za_zt0(ptr %p, i64 %v) "aarch64_pstate_sm_enabled" "aarch64_inout_za" "aarch64_inout_zt0" nounwind {
+; DEFAULT-LABEL: ldadd8_sme_streaming_za_zt0:
+; DEFAULT: smstop sm
+; DEFAULT: bl __aarch64_ldadd8_acq_rel{{$}}
+; DEFAULT: smstart sm
+;
+; PRESERVE-LABEL: ldadd8_sme_streaming_za_zt0:
+; PRESERVE-NOT: smstop
+; PRESERVE-NOT: __arm_tpidr2_save
+; PRESERVE-NOT: str zt0
+; PRESERVE: bl __aarch64_ldadd8_acq_rel_preservemost{{$}}
+; PRESERVE-NOT: smstart
+; PRESERVE-NOT: ldr zt0
+; PRESERVE: ret
+ %old = atomicrmw add ptr %p, i64 %v acq_rel
+ ret i64 %old
+}
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
index 3f65949c14cfa..e2ff611e83205 100644
--- a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most-sve.ll
@@ -9,13 +9,13 @@ define <vscale x 4 x i32> @ldset8_preserves_sve(ptr %p, i64 %v, <vscale x 4 x i3
; DEFAULT-LABEL: ldset8_preserves_sve:
; DEFAULT: addvl sp, sp,
; DEFAULT: str z{{[0-9]+}},
-; DEFAULT: bl __aarch64_ldset8_acq_rel
+; DEFAULT: bl __aarch64_ldset8_acq_rel{{$}}
;
; PRESERVE-LABEL: ldset8_preserves_sve:
; PRESERVE-NOT: addvl
; PRESERVE-NOT: str z
; PRESERVE-NOT: str p
-; PRESERVE: bl __aarch64_ldset8_acq_rel_preservemost
+; PRESERVE: bl __aarch64_ldset8_acq_rel_preservemost{{$}}
; PRESERVE-NOT: ldr z
; PRESERVE-NOT: ldr p
; PRESERVE: add z0.s, p0/m, z0.s, z1.s
diff --git a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
index 0cf8d67d62bd2..850410d275bf4 100644
--- a/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
+++ b/llvm/test/CodeGen/AArch64/outline-atomics-preserve-most.ll
@@ -13,7 +13,11 @@
; RUN: llc -mtriple=aarch64-unknown-fuchsia -O0 -global-isel=0 -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=O0
; RUN: llc -mtriple=aarch64-unknown-fuchsia -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=CSRMASK
; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=CSRMASK
+; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=DARWIN-ASM
; RUN: llc -mtriple=arm64ec-pc-windows-msvc -mattr=+outline-atomics,+outline-atomics-preserve-most -stop-after=finalize-isel < %s | FileCheck %s --check-prefix=ARM64EC
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+lse,+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=LSE
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=-outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=NO-OUTLINE
+; RUN: llc -mtriple=aarch64-unknown-linux-gnu -mattr=+outline-atomics,+outline-atomics-preserve-most -verify-machineinstrs < %s | FileCheck %s --check-prefix=NOX5X14
; Verify that live GPRs are held in preserved x5-x14 across 1/2/4/8-byte
; outline-atomic libcalls without callee-saved stack spills.
@@ -22,14 +26,29 @@ define i64 @ldadd4_preserves_gprs(ptr %p, i32 %v, i64 %a, i64 %b, i64 %c, i64 %d
; DEFAULT: stp x24, x23,
; DEFAULT: stp x22, x21,
; DEFAULT: stp x20, x19,
-; DEFAULT: bl __aarch64_ldadd4_acq_rel
+; DEFAULT: bl __aarch64_ldadd4_acq_rel{{$}}
;
; PRESERVE-LABEL: ldadd4_preserves_gprs:
; PRESERVE-NOT: stp x
-; PRESERVE: bl __aarch64_ldadd4_acq_rel_preservemost
+; PRESERVE: bl __aarch64_ldadd4_acq_rel_preservemost{{$}}
; PRESERVE-NOT: ldp x
; PRESERVE: ret
;
+; DARWIN-ASM-LABEL: _ldadd4_preserves_gprs:
+; DARWIN-ASM-NOT: x19
+; DARWIN-ASM-NOT: x20
+; DARWIN-ASM: bl ___aarch64_ldadd4_acq_rel_preservemost{{$}}
+; DARWIN-ASM: ret
+;
+; LSE-LABEL: ldadd4_preserves_gprs:
+; LSE-NOT: __aarch64_
+; LSE: ldaddal w{{[0-9]+}}, w{{[0-9]+}}, [x0]
+;
+; NO-OUTLINE-LABEL: ldadd4_preserves_gprs:
+; NO-OUTLINE-NOT: __aarch64_
+; NO-OUTLINE: ldaxr w{{[0-9]+}}, [x0]
+; NO-OUTLINE: stlxr w{{[0-9]+}}, w{{[0-9]+}}, [x0]
+;
; GISEL-LABEL: ldadd4_preserves_gprs:
; GISEL-NOT: stp x
; GISEL: bl __aarch64_ldadd4_acq_rel_preservemost
@@ -57,7 +76,7 @@ define i64 @ldadd4_preserves_gprs(ptr %p, i32 %v, i64 %a, i64 %b, i64 %c, i64 %d
define i128 @cas16_preserves_gprs(ptr %p, i128 %exp, i128 %des, i64 %extra) nounwind {
; DEFAULT-LABEL: cas16_preserves_gprs:
; DEFAULT: stp {{.*}}x19
-; DEFAULT: bl __aarch64_cas16_acq_rel
+; DEFAULT: bl __aarch64_cas16_acq_rel{{$}}
;
; PRESERVE-LABEL: cas16_preserves_gprs:
; PRESERVE-NOT: x19
@@ -88,7 +107,7 @@ define i128 @cas16_preserves_gprs(ptr %p, i128 %exp, i128 %des, i64 %extra) noun
define <4 x i32> @swp8_preserves_simd(ptr %p, i64 %v, <4 x i32> %vec0, <4 x i32> %vec1) nounwind {
; DEFAULT-LABEL: swp8_preserves_simd:
; DEFAULT: stp q{{[0-9]+}}, q{{[0-9]+}}, [sp
-; DEFAULT: bl __aarch64_swp8_rel
+; DEFAULT: bl __aarch64_swp8_rel{{$}}
;
; PRESERVE-LABEL: swp8_preserves_simd:
; PRESERVE-NOT: stp q
@@ -127,6 +146,11 @@ define i64 @exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary(ptr %p, i32 %v, i64 %x
;
; O0-LABEL: exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary:
; O0: bl __aarch64_ldclr4_relax_preservemost
+;
+; NOX5X14-LABEL: exact_x4_x5_x14_x15_x16_x17_and_nzcv_boundary:
+; NOX5X14-NOT: {{mov|str|stp|ldr|ldp}} {{.*}}x5{{[,\]\n]|$}}
+; NOX5X14-NOT: {{mov|str|stp|ldr|ldp}} {{.*}}x14{{[,\]\n]|$}}
+; NOX5X14: ret
%cmp = icmp eq i64 %x, %y
%r4 = call i64 asm sideeffect "", "={x4}"()
%r5 = call i64 asm sideeffect "", "={x5}"()
>From d554c39ac158323b74e30cad046a6302fd64b4f9 Mon Sep 17 00:00:00 2001
From: Roland McGrath <mcgrathr at google.com>
Date: Tue, 10 Nov 2026 17:43:08 -0800
Subject: [PATCH 4/5] [AArch64] Move ReleaseNotes entry to resolve merge
conflict
---
llvm/docs/ReleaseNotes.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index e4f940f8af9f3..a2a9b9318fdd3 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -284,6 +284,8 @@ Makes programs 10x faster by doing Special New Thing.
### Changes to the AArch64 Backend
+* Added the `+outline-atomics-preserve-most` subtarget feature (and `--aarch64-outline-atomics-preserve-most` backend option) to lower AArch64 out-of-line atomic helper calls to `__aarch64_*_preservemost` entry points using `aarch64_outline_atomics_preservemost`, preserving all registers except `x0`-`x4`, `x15`-`x17`, and `NZCV`.
+
* Added support for C2-Pro and C2-Ultra CPUs.
* Added support for hardening return address signing against PACMAN attacks.
Functions with the `"sign-return-address-harden"="load-return-address"`
@@ -303,8 +305,6 @@ Makes programs 10x faster by doing Special New Thing.
ARM64 ABI (calling convention, callee saved registers, stack probing and SEH
unwind information) and produces PE/COFF objects.
-* Added the `+outline-atomics-preserve-most` subtarget feature (and `--aarch64-outline-atomics-preserve-most` backend option) to lower AArch64 out-of-line atomic helper calls to `__aarch64_*_preservemost` entry points using `aarch64_outline_atomics_preservemost`, preserving all registers except `x0`-`x4`, `x15`-`x17`, and `NZCV`.
-
### Changes to the AMDGPU Backend
* Replaced `xnack` and `sramecc` target features with `amdgpu.xnack`
>From 57e0c77b04f18c995ddad0d742a338b270e0d900 Mon Sep 17 00:00:00 2001
From: Roland McGrath <mcgrathr at google.com>
Date: Thu, 12 Nov 2026 18:58:15 -0800
Subject: [PATCH 5/5] [AArch64][clang] Drop unrelated reformatting in
CommonArgs.cpp
Change-Id: I147ff9c9ffbc61d4b027f6d3445987ca03ed7671
---
clang/lib/Driver/ToolChains/CommonArgs.cpp | 42 +++++++++++-----------
1 file changed, 20 insertions(+), 22 deletions(-)
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index c3b1f6aac8e3b..353bc30870ec2 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -1194,7 +1194,7 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
// LowerMatrixIntrinsicsPass, which is transitively called by
// buildThinLTODefaultPipeline under EnableMatrix.
if ((IsThinLTO || IsFatLTO || IsUnifiedLTO) &&
- Args.hasArg(options::OPT_fenable_matrix))
+ Args.hasArg(options::OPT_fenable_matrix))
CmdArgs.push_back(
Args.MakeArgString(Twine(PluginOptPrefix) + "-enable-matrix"));
@@ -1551,7 +1551,7 @@ bool tools::addOpenMPRuntime(const Compilation &C, ArgStringList &CmdArgs,
CmdArgs.push_back("-Bdynamic");
if (RTKind == Driver::OMPRT_GOMP && GompNeedsRT)
- CmdArgs.push_back("-lrt");
+ CmdArgs.push_back("-lrt");
if (IsOffloadingHost)
CmdArgs.push_back("-lomptarget");
@@ -1585,8 +1585,7 @@ static void addSanitizerRuntime(const ToolChain &TC, const ArgList &Args,
bool IsShared, bool IsWhole) {
// Wrap any static runtimes that must be forced into executable in
// whole-archive.
- if (IsWhole)
- CmdArgs.push_back("--whole-archive");
+ if (IsWhole) CmdArgs.push_back("--whole-archive");
CmdArgs.push_back(TC.getCompilerRTArgString(
Args, Sanitizer, IsShared ? ToolChain::FT_Shared : ToolChain::FT_Static));
if (IsWhole)
@@ -1969,8 +1968,7 @@ bool tools::addSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
FuzzerNeedsSanitizerDeps;
}
-bool tools::addXRayRuntime(const ToolChain &TC, const ArgList &Args,
- ArgStringList &CmdArgs) {
+bool tools::addXRayRuntime(const ToolChain&TC, const ArgList &Args, ArgStringList &CmdArgs) {
const XRayArgs &XRay = TC.getXRayArgs(Args);
if (Args.hasArg(options::OPT_shared)) {
if (XRay.needsXRayDSORt()) {
@@ -2000,7 +1998,8 @@ void tools::linkXRayRuntimeDeps(const ToolChain &TC,
CmdArgs.push_back("-lrt");
CmdArgs.push_back("-lm");
- if (!TC.getTriple().isOSFreeBSD() && !TC.getTriple().isOSNetBSD() &&
+ if (!TC.getTriple().isOSFreeBSD() &&
+ !TC.getTriple().isOSNetBSD() &&
!TC.getTriple().isOSOpenBSD())
CmdArgs.push_back("-ldl");
}
@@ -2277,19 +2276,19 @@ tools::ParsePICArgs(const ToolChain &ToolChain, const ArgList &Args) {
bool EmbeddedPISupported;
switch (Triple.getArch()) {
- case llvm::Triple::arm:
- case llvm::Triple::armeb:
- case llvm::Triple::thumb:
- case llvm::Triple::thumbeb:
- EmbeddedPISupported = true;
- break;
- default:
- EmbeddedPISupported = false;
- break;
+ case llvm::Triple::arm:
+ case llvm::Triple::armeb:
+ case llvm::Triple::thumb:
+ case llvm::Triple::thumbeb:
+ EmbeddedPISupported = true;
+ break;
+ default:
+ EmbeddedPISupported = false;
+ break;
}
bool ROPI = false, RWPI = false;
- Arg *LastROPIArg = Args.getLastArg(options::OPT_fropi, options::OPT_fno_ropi);
+ Arg* LastROPIArg = Args.getLastArg(options::OPT_fropi, options::OPT_fno_ropi);
if (LastROPIArg && LastROPIArg->getOption().matches(options::OPT_fropi)) {
if (!EmbeddedPISupported)
ToolChain.getDriver().Diag(diag::err_drv_unsupported_opt_for_target)
@@ -2318,7 +2317,7 @@ tools::ParsePICArgs(const ToolChain &ToolChain, const ArgList &Args) {
if (ABIName == "n64")
PIC = true;
// When targettng MIPS with -mno-abicalls, it's always static.
- if (Args.hasArg(options::OPT_mno_abicalls))
+ if(Args.hasArg(options::OPT_mno_abicalls))
return std::make_tuple(llvm::Reloc::Static, 0U, false);
// Unlike other architectures, MIPS, even with -fPIC/-mxgot/multigot,
// does not use PIC level 2 for historical reasons.
@@ -2538,8 +2537,7 @@ enum class LibGccType { UnspecifiedLibGcc, StaticLibGcc, SharedLibGcc };
static LibGccType getLibGccType(const ToolChain &TC, const Driver &D,
const ArgList &Args) {
if (Args.hasArg(options::OPT_static_libgcc) ||
- Args.hasArg(options::OPT_static) ||
- Args.hasArg(options::OPT_static_pie) ||
+ Args.hasArg(options::OPT_static) || Args.hasArg(options::OPT_static_pie) ||
// The Android NDK only provides libunwind.a, not libunwind.so.
TC.getTriple().isAndroid())
return LibGccType::StaticLibGcc;
@@ -2815,8 +2813,8 @@ void tools::addMachineOutlinerArgs(const Driver &D,
}
};
- if (Arg *A =
- Args.getLastArg(options::OPT_moutline, options::OPT_mno_outline)) {
+ if (Arg *A = Args.getLastArg(options::OPT_moutline,
+ options::OPT_mno_outline)) {
if (A->getOption().matches(options::OPT_moutline)) {
// We only support -moutline in AArch64, ARM, RISC-V and X86 targets right
// now. If we're compiling for these, add the proper mllvm flags.
More information about the cfe-commits
mailing list