[llvm] 4de8521 - [MachineOutliner][AArch64] NFC: Split MBBs into "outlinable ranges"
Jessica Paquette via llvm-commits
llvm-commits at lists.llvm.org
Fri Feb 3 15:33:52 PST 2023
Author: Jessica Paquette
Date: 2023-02-03T15:33:37-08:00
New Revision: 4de8521bc528436abc47a250b2495f8b8fbc7798
URL: https://github.com/llvm/llvm-project/commit/4de8521bc528436abc47a250b2495f8b8fbc7798
DIFF: https://github.com/llvm/llvm-project/commit/4de8521bc528436abc47a250b2495f8b8fbc7798.diff
LOG: [MachineOutliner][AArch64] NFC: Split MBBs into "outlinable ranges"
Recommit with bug fixes + added testcases to the outliner. Also adds some
debug output.
We found a case in the Swift benchmarks where the MachineOutliner introduces
about a 20% compile time overhead in comparison to building without the
MachineOutliner.
The origin of this slowdown is that the benchmark has long blocks which incur
lots of LRU checks for lots of candidates.
Imagine a case like this:
```
bb:
i1
i2
i3
...
i123456
```
Now imagine that all of the outlining candidates appear early in the block, and
that something like, say, NZCV is defined at the end of the block.
The outliner has to check liveness for certain registers across all candidates,
because outlining from areas where those registers are used is unsafe at call
boundaries.
This is fairly wasteful because in the previously-described case, the outlining
candidates will never appear in an area where those registers are live.
To avoid this, precalculate areas where we will consider outlining from.
Anything outside of these areas is mapped to illegal and not included in the
outlining search space. This allows us to reduce the size of the outliner's
suffix tree as well, giving us a potential memory win.
By precalculating areas, we can also optimize other checks too, like whether
or not LR is live across an outlining candidate.
Doing all of this is about a 16% compile time improvement on the case.
This is likely useful for other targets (e.g. ARM + RISCV) as well, but for now,
this only implements the AArch64 path. The original "is the MBB safe" method
still works as before.
Added:
llvm/test/CodeGen/AArch64/machine-outliner-only-unsafe-ranges.mir
llvm/test/CodeGen/AArch64/machine-outliner-safe-range-in-middle.mir
llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-beginning.mir
llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-end.mir
Modified:
llvm/include/llvm/CodeGen/TargetInstrInfo.h
llvm/lib/CodeGen/MachineOutliner.cpp
llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
llvm/lib/Target/AArch64/AArch64InstrInfo.h
Removed:
################################################################################
diff --git a/llvm/include/llvm/CodeGen/TargetInstrInfo.h b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
index ee5d87e0ce2e2..a19259c0a9557 100644
--- a/llvm/include/llvm/CodeGen/TargetInstrInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
@@ -1987,6 +1987,25 @@ class TargetInstrInfo : public MCInstrInfo {
virtual bool isMBBSafeToOutlineFrom(MachineBasicBlock &MBB,
unsigned &Flags) const;
+ /// Optional target hook which partitions \p MBB into outlinable ranges for
+ /// instruction mapping purposes. Each range is defined by two iterators:
+ /// [start, end).
+ ///
+ /// Ranges are expected to be ordered top-down. That is, ranges closer to the
+ /// top of the block should come before ranges closer to the end of the block.
+ ///
+ /// Ranges cannot overlap.
+ ///
+ /// If an entire block is mappable, then its range is [MBB.begin(), MBB.end())
+ ///
+ /// All instructions not present in an outlinable range are considered
+ /// illegal.
+ virtual SmallVector<
+ std::pair<MachineBasicBlock::iterator, MachineBasicBlock::iterator>>
+ getOutlinableRanges(MachineBasicBlock &MBB, unsigned &Flags) const {
+ return {std::make_pair(MBB.begin(), MBB.end())};
+ }
+
/// Insert a custom frame for outlined functions.
virtual void buildOutlinedFrame(MachineBasicBlock &MBB, MachineFunction &MF,
const outliner::OutlinedFunction &OF) const {
diff --git a/llvm/lib/CodeGen/MachineOutliner.cpp b/llvm/lib/CodeGen/MachineOutliner.cpp
index c7ba66bd36782..a7422f6f8d396 100644
--- a/llvm/lib/CodeGen/MachineOutliner.cpp
+++ b/llvm/lib/CodeGen/MachineOutliner.cpp
@@ -254,12 +254,20 @@ struct InstructionMapper {
/// \param TII \p TargetInstrInfo for the function.
void convertToUnsignedVec(MachineBasicBlock &MBB,
const TargetInstrInfo &TII) {
+ LLVM_DEBUG(dbgs() << "*** Converting MBB '" << MBB.getName()
+ << "' to unsigned vector ***\n");
unsigned Flags = 0;
// Don't even map in this case.
if (!TII.isMBBSafeToOutlineFrom(MBB, Flags))
return;
+ auto OutlinableRanges = TII.getOutlinableRanges(MBB, Flags);
+ LLVM_DEBUG(dbgs() << MBB.getName() << ": " << OutlinableRanges.size()
+ << " outlinable range(s)\n");
+ if (OutlinableRanges.empty())
+ return;
+
// Store info for the MBB for later outlining.
MBBFlagsMap[&MBB] = Flags;
@@ -282,37 +290,68 @@ struct InstructionMapper {
std::vector<unsigned> UnsignedVecForMBB;
std::vector<MachineBasicBlock::iterator> InstrListForMBB;
- for (MachineBasicBlock::iterator Et = MBB.end(); It != Et; ++It) {
- // Keep track of where this instruction is in the module.
- switch (TII.getOutliningType(It, Flags)) {
- case InstrType::Illegal:
+ LLVM_DEBUG(dbgs() << "*** Mapping outlinable ranges ***\n");
+ for (auto &OutlinableRange : OutlinableRanges) {
+ auto OutlinableRangeBegin = OutlinableRange.first;
+ auto OutlinableRangeEnd = OutlinableRange.second;
+#ifndef NDEBUG
+ LLVM_DEBUG(
+ dbgs() << "Mapping "
+ << std::distance(OutlinableRangeBegin, OutlinableRangeEnd)
+ << " instruction range\n");
+ // Everything outside of an outlinable range is illegal.
+ unsigned NumSkippedInRange = 0;
+#endif
+ for (; It != OutlinableRangeBegin; ++It) {
+#ifndef NDEBUG
+ ++NumSkippedInRange;
+#endif
mapToIllegalUnsigned(It, CanOutlineWithPrevInstr, UnsignedVecForMBB,
InstrListForMBB);
- break;
-
- case InstrType::Legal:
- mapToLegalUnsigned(It, CanOutlineWithPrevInstr, HaveLegalRange,
- NumLegalInBlock, UnsignedVecForMBB, InstrListForMBB);
- break;
-
- case InstrType::LegalTerminator:
- mapToLegalUnsigned(It, CanOutlineWithPrevInstr, HaveLegalRange,
- NumLegalInBlock, UnsignedVecForMBB, InstrListForMBB);
- // The instruction also acts as a terminator, so we have to record that
- // in the string.
- mapToIllegalUnsigned(It, CanOutlineWithPrevInstr, UnsignedVecForMBB,
+ }
+#ifndef NDEBUG
+ LLVM_DEBUG(dbgs() << "Skipped " << NumSkippedInRange
+ << " instructions outside outlinable range\n");
+#endif
+ assert(It != MBB.end() && "Should still have instructions?");
+ // `It` is now positioned at the beginning of a range of instructions
+ // which may be outlinable. Check if each instruction is known to be safe.
+ for (; It != OutlinableRangeEnd; ++It) {
+ // Keep track of where this instruction is in the module.
+ switch (TII.getOutliningType(It, Flags)) {
+ case InstrType::Illegal:
+ mapToIllegalUnsigned(It, CanOutlineWithPrevInstr, UnsignedVecForMBB,
+ InstrListForMBB);
+ break;
+
+ case InstrType::Legal:
+ mapToLegalUnsigned(It, CanOutlineWithPrevInstr, HaveLegalRange,
+ NumLegalInBlock, UnsignedVecForMBB,
InstrListForMBB);
- break;
-
- case InstrType::Invisible:
- // Normally this is set by mapTo(Blah)Unsigned, but we just want to
- // skip this instruction. So, unset the flag here.
- ++NumInvisible;
- AddedIllegalLastTime = false;
- break;
+ break;
+
+ case InstrType::LegalTerminator:
+ mapToLegalUnsigned(It, CanOutlineWithPrevInstr, HaveLegalRange,
+ NumLegalInBlock, UnsignedVecForMBB,
+ InstrListForMBB);
+ // The instruction also acts as a terminator, so we have to record
+ // that in the string.
+ mapToIllegalUnsigned(It, CanOutlineWithPrevInstr, UnsignedVecForMBB,
+ InstrListForMBB);
+ break;
+
+ case InstrType::Invisible:
+ // Normally this is set by mapTo(Blah)Unsigned, but we just want to
+ // skip this instruction. So, unset the flag here.
+ ++NumInvisible;
+ AddedIllegalLastTime = false;
+ break;
+ }
}
}
+ LLVM_DEBUG(dbgs() << "HaveLegalRange = " << HaveLegalRange << "\n");
+
// Are there enough legal instructions in the block for outlining to be
// possible?
if (HaveLegalRange) {
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
index 6916e1ec57002..0b33d7c7dac85 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
@@ -7269,41 +7269,6 @@ outliner::OutlinedFunction AArch64InstrInfo::getOutliningCandidateInfo(
for (outliner::Candidate &C : RepeatedSequenceLocs)
FlagsSetInAll &= C.Flags;
- // According to the AArch64 Procedure Call Standard, the following are
- // undefined on entry/exit from a function call:
- //
- // * Registers x16, x17, (and thus w16, w17)
- // * Condition codes (and thus the NZCV register)
- //
- // Because if this, we can't outline any sequence of instructions where
- // one
- // of these registers is live into/across it. Thus, we need to delete
- // those
- // candidates.
- auto CantGuaranteeValueAcrossCall = [&TRI](outliner::Candidate &C) {
- // If the unsafe registers in this block are all dead, then we don't need
- // to compute liveness here.
- if (C.Flags & UnsafeRegsDead)
- return false;
- return C.isAnyUnavailableAcrossOrOutOfSeq(
- {AArch64::W16, AArch64::W17, AArch64::NZCV}, TRI);
- };
-
- // Are there any candidates where those registers are live?
- if (!(FlagsSetInAll & UnsafeRegsDead)) {
- // Erase every candidate that violates the restrictions above. (It could be
- // true that we have viable candidates, so it's not worth bailing out in
- // the case that, say, 1 out of 20 candidates violate the restructions.)
- llvm::erase_if(RepeatedSequenceLocs, CantGuaranteeValueAcrossCall);
-
- // If the sequence doesn't have enough candidates left, then we're done.
- if (RepeatedSequenceLocs.size() < 2)
- return outliner::OutlinedFunction();
- }
-
- // At this point, we have only "safe" candidates to outline. Figure out
- // frame + call instruction information.
-
unsigned LastInstrOpcode = RepeatedSequenceLocs[0].back()->getOpcode();
// Helper lambda which sets call information for every candidate.
@@ -7429,6 +7394,10 @@ outliner::OutlinedFunction AArch64InstrInfo::getOutliningCandidateInfo(
// Check if we have to save LR.
for (outliner::Candidate &C : RepeatedSequenceLocs) {
+ bool LRAvailable =
+ (C.Flags & MachineOutlinerMBBFlags::LRUnavailableSomewhere)
+ ? C.isAvailableAcrossAndOutOfSeq(AArch64::LR, TRI)
+ : true;
// If we have a noreturn caller, then we're going to be conservative and
// say that we have to save LR. If we don't have a ret at the end of the
// block, then we can't reason about liveness accurately.
@@ -7439,7 +7408,7 @@ outliner::OutlinedFunction AArch64InstrInfo::getOutliningCandidateInfo(
C.getMF()->getFunction().hasFnAttribute(Attribute::NoReturn);
// Is LR available? If so, we don't need a save.
- if (C.isAvailableAcrossAndOutOfSeq(AArch64::LR, TRI) && !IsNoReturn) {
+ if (LRAvailable && !IsNoReturn) {
NumBytesNoStackCalls += 4;
C.setCallInfo(MachineOutlinerNoLRSave, 4);
CandidatesWithoutStackFixups.push_back(C);
@@ -7611,72 +7580,118 @@ bool AArch64InstrInfo::isFunctionSafeToOutlineFrom(
return true;
}
-bool AArch64InstrInfo::isMBBSafeToOutlineFrom(MachineBasicBlock &MBB,
- unsigned &Flags) const {
- if (!TargetInstrInfo::isMBBSafeToOutlineFrom(MBB, Flags))
- return false;
- // Check if LR is available through all of the MBB. If it's not, then set
- // a flag.
+SmallVector<std::pair<MachineBasicBlock::iterator, MachineBasicBlock::iterator>>
+AArch64InstrInfo::getOutlinableRanges(MachineBasicBlock &MBB,
+ unsigned &Flags) const {
assert(MBB.getParent()->getRegInfo().tracksLiveness() &&
- "Suitable Machine Function for outlining must track liveness");
+ "Must track liveness!");
+ SmallVector<
+ std::pair<MachineBasicBlock::iterator, MachineBasicBlock::iterator>>
+ Ranges;
+ // According to the AArch64 Procedure Call Standard, the following are
+ // undefined on entry/exit from a function call:
+ //
+ // * Registers x16, x17, (and thus w16, w17)
+ // * Condition codes (and thus the NZCV register)
+ //
+ // If any of these registers are used inside or live across an outlined
+ // function, then they may be modified later, either by the compiler or
+ // some other tool (like the linker).
+ //
+ // To avoid outlining in these situations, partition each block into ranges
+ // where these registers are dead. We will only outline from those ranges.
LiveRegUnits LRU(getRegisterInfo());
+ auto AreAllUnsafeRegsDead = [&LRU]() {
+ return LRU.available(AArch64::W16) && LRU.available(AArch64::W17) &&
+ LRU.available(AArch64::NZCV);
+ };
- for (MachineInstr &MI : llvm::reverse(MBB))
- LRU.accumulate(MI);
-
- // Check if each of the unsafe registers are available...
- bool W16AvailableInBlock = LRU.available(AArch64::W16);
- bool W17AvailableInBlock = LRU.available(AArch64::W17);
- bool NZCVAvailableInBlock = LRU.available(AArch64::NZCV);
-
- // If all of these are dead (and not live out), we know we don't have to check
- // them later.
- if (W16AvailableInBlock && W17AvailableInBlock && NZCVAvailableInBlock)
- Flags |= MachineOutlinerMBBFlags::UnsafeRegsDead;
-
- // Now, add the live outs to the set.
+ // We need to know if LR is live across an outlining boundary later on in
+ // order to decide how we'll create the outlined call, frame, etc.
+ //
+ // It's pretty expensive to check this for *every candidate* within a block.
+ // That's some potentially n^2 behaviour, since in the worst case, we'd need
+ // to compute liveness from the end of the block for O(n) candidates within
+ // the block.
+ //
+ // So, to improve the average case, let's keep track of liveness from the end
+ // of the block to the beginning of *every outlinable range*. If we know that
+ // LR is available in every range we could outline from, then we know that
+ // we don't need to check liveness for any candidate within that range.
+ bool LRAvailableEverywhere = true;
+ // Compute liveness bottom-up.
LRU.addLiveOuts(MBB);
-
- // If any of these registers is available in the MBB, but also a live out of
- // the block, then we know outlining is unsafe.
- if (W16AvailableInBlock && !LRU.available(AArch64::W16))
- return false;
- if (W17AvailableInBlock && !LRU.available(AArch64::W17))
- return false;
- if (NZCVAvailableInBlock && !LRU.available(AArch64::NZCV))
- return false;
-
- // Check if there's a call inside this MachineBasicBlock. If there is, then
- // set a flag.
- if (any_of(MBB, [](MachineInstr &MI) { return MI.isCall(); }))
- Flags |= MachineOutlinerMBBFlags::HasCalls;
-
- MachineFunction *MF = MBB.getParent();
-
- // In the event that we outline, we may have to save LR. If there is an
- // available register in the MBB, then we'll always save LR there. Check if
- // this is true.
- bool CanSaveLR = false;
- const AArch64RegisterInfo *ARI = static_cast<const AArch64RegisterInfo *>(
- MF->getSubtarget().getRegisterInfo());
-
- // Check if there is an available register across the sequence that we can
- // use.
- for (unsigned Reg : AArch64::GPR64RegClass) {
- if (!ARI->isReservedReg(*MF, Reg) && Reg != AArch64::LR &&
- Reg != AArch64::X16 && Reg != AArch64::X17 && LRU.available(Reg)) {
- CanSaveLR = true;
+ // Update flags that require info about the entire MBB.
+ auto UpdateWholeMBBFlags = [&Flags](const MachineInstr &MI) {
+ if (MI.isCall() && !MI.isTerminator())
+ Flags |= MachineOutlinerMBBFlags::HasCalls;
+ };
+ // Range: [RangeBegin, RangeEnd)
+ MachineBasicBlock::instr_iterator RangeBegin, RangeEnd;
+ unsigned RangeLen;
+ auto CreateNewRangeStartingAt =
+ [&RangeBegin, &RangeEnd,
+ &RangeLen](MachineBasicBlock::instr_iterator NewBegin) {
+ RangeBegin = NewBegin;
+ RangeEnd = std::next(RangeBegin);
+ RangeLen = 0;
+ };
+ auto SaveRangeIfNonEmpty = [&RangeLen, &Ranges, &RangeBegin, &RangeEnd]() {
+ // At least one unsafe register is not dead. We do not want to outline at
+ // this point. If it is long enough to outline from, save the range
+ // [RangeBegin, RangeEnd).
+ if (RangeLen > 1)
+ Ranges.push_back(std::make_pair(RangeBegin, RangeEnd));
+ };
+ // Find the first point where all unsafe registers are dead.
+ // FIND: <safe instr> <-- end of first potential range
+ // SKIP: <unsafe def>
+ // SKIP: ... everything between ...
+ // SKIP: <unsafe use>
+ auto FirstPossibleEndPt = MBB.instr_rbegin();
+ for (; FirstPossibleEndPt != MBB.instr_rend(); ++FirstPossibleEndPt) {
+ LRU.stepBackward(*FirstPossibleEndPt);
+ // Update flags that impact how we outline across the entire block,
+ // regardless of safety.
+ UpdateWholeMBBFlags(*FirstPossibleEndPt);
+ if (AreAllUnsafeRegsDead())
break;
- }
}
-
- // Check if we have a register we can save LR to, and if LR was used
- // somewhere. If both of those things are true, then we need to evaluate the
- // safety of outlining stack instructions later.
- if (!CanSaveLR && !LRU.available(AArch64::LR))
+ // If we exhausted the entire block, we have no safe ranges to outline.
+ if (FirstPossibleEndPt == MBB.instr_rend())
+ return Ranges;
+ // Current range.
+ CreateNewRangeStartingAt(FirstPossibleEndPt->getIterator());
+ // StartPt points to the first place where all unsafe registers
+ // are dead (if there is any such point). Begin partitioning the MBB into
+ // ranges.
+ for (auto &MI : make_range(FirstPossibleEndPt, MBB.instr_rend())) {
+ LRU.stepBackward(MI);
+ UpdateWholeMBBFlags(MI);
+ if (!AreAllUnsafeRegsDead()) {
+ SaveRangeIfNonEmpty();
+ CreateNewRangeStartingAt(MI.getIterator());
+ continue;
+ }
+ LRAvailableEverywhere &= LRU.available(AArch64::LR);
+ RangeBegin = MI.getIterator();
+ ++RangeLen;
+ continue;
+ }
+ // Above loop misses the last (or only) range. If we are still safe, then
+ // let's save the range.
+ if (AreAllUnsafeRegsDead())
+ SaveRangeIfNonEmpty();
+ if (Ranges.empty())
+ return Ranges;
+ // We found the ranges bottom-up. Mapping expects the top-down. Reverse
+ // the order.
+ std::reverse(Ranges.begin(), Ranges.end());
+ // If there is at least one outlinable range where LR is unavailable
+ // somewhere, remember that.
+ if (!LRAvailableEverywhere)
Flags |= MachineOutlinerMBBFlags::LRUnavailableSomewhere;
-
- return true;
+ return Ranges;
}
outliner::InstrType
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.h b/llvm/lib/Target/AArch64/AArch64InstrInfo.h
index caf9421eb001e..5b2fb038423a6 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.h
@@ -291,10 +291,11 @@ class AArch64InstrInfo final : public AArch64GenInstrInfo {
bool OutlineFromLinkOnceODRs) const override;
outliner::OutlinedFunction getOutliningCandidateInfo(
std::vector<outliner::Candidate> &RepeatedSequenceLocs) const override;
- outliner::InstrType
- getOutliningType(MachineBasicBlock::iterator &MIT, unsigned Flags) const override;
- bool isMBBSafeToOutlineFrom(MachineBasicBlock &MBB,
- unsigned &Flags) const override;
+ outliner::InstrType getOutliningType(MachineBasicBlock::iterator &MIT,
+ unsigned Flags) const override;
+ SmallVector<
+ std::pair<MachineBasicBlock::iterator, MachineBasicBlock::iterator>>
+ getOutlinableRanges(MachineBasicBlock &MBB, unsigned &Flags) const override;
void buildOutlinedFrame(MachineBasicBlock &MBB, MachineFunction &MF,
const outliner::OutlinedFunction &OF) const override;
MachineBasicBlock::iterator
diff --git a/llvm/test/CodeGen/AArch64/machine-outliner-only-unsafe-ranges.mir b/llvm/test/CodeGen/AArch64/machine-outliner-only-unsafe-ranges.mir
new file mode 100644
index 0000000000000..c4e5895d8a491
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/machine-outliner-only-unsafe-ranges.mir
@@ -0,0 +1,24 @@
+# RUN: llc %s -mtriple aarch64 -run-pass=machine-outliner -o - | FileCheck %s
+# CHECK-NOT: OUTLINED
+
+...
+---
+name: no_safe_range
+tracksRegLiveness: true
+machineFunctionInfo:
+ hasRedZone: false
+body: |
+ bb.0.entry:
+ liveins: $x0, $x10, $x16
+ $x4 = ADDXri $x0, 0, 0
+ $x5 = ADDXri $x0, 1, 0
+ $x6 = ADDXri $x0, 2, 0
+ $x7 = ADDXri $x0, 3, 0
+
+ $x10 = ADDXri $x0, 10, 0
+
+ $x4 = ADDXri $x0, 0, 0
+ $x5 = ADDXri $x0, 1, 0
+ $x6 = ADDXri $x0, 2, 0
+ $x7 = ADDXri $x0, 3, 0
+ RET $x16
diff --git a/llvm/test/CodeGen/AArch64/machine-outliner-safe-range-in-middle.mir b/llvm/test/CodeGen/AArch64/machine-outliner-safe-range-in-middle.mir
new file mode 100644
index 0000000000000..23811425101fd
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/machine-outliner-safe-range-in-middle.mir
@@ -0,0 +1,42 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc %s -mtriple aarch64 -run-pass=machine-outliner -o - | FileCheck %s
+
+...
+---
+name: unsafe_range_in_middle
+tracksRegLiveness: true
+machineFunctionInfo:
+ hasRedZone: false
+body: |
+ bb.0:
+ liveins: $x0
+ ; Begin safe range of 3 instructions
+ ; CHECK-LABEL: name: unsafe_range_in_middle
+ ; CHECK: liveins: $x0
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp, implicit-def $lr, implicit-def $x0, implicit-def $x1, implicit-def $x2, implicit-def $x3, implicit-def $x16, implicit $x0, implicit $sp
+ ; CHECK-NEXT: $x9 = ADDXri $x16, 16, 0
+ ; CHECK-NEXT: $x16 = ADDXri killed $x16, 16, 0
+ ; CHECK-NEXT: BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp, implicit-def $lr, implicit-def $x0, implicit-def $x1, implicit-def $x2, implicit-def $x3, implicit-def $x16, implicit $x0, implicit $sp
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x16 = ADDXri killed $x16, 16, 0
+ ; CHECK-NEXT: RET undef $x9
+ $x0 = ADDXri $x0, 0, 0
+ $x1 = ADDXri $x0, 1, 0
+ $x2 = ADDXri $x0, 2, 0
+ $x3 = ADDXri $x0, 3, 0
+
+ ; End safe range
+ $x16 = ADDXri $x0, 16, 0
+ $x9 = ADDXri $x16, 16, 0
+ $x16 = ADDXri killed $x16, 16, 0
+
+ $x0 = ADDXri $x0, 0, 0
+ $x1 = ADDXri $x0, 1, 0
+ $x2 = ADDXri $x0, 2, 0
+ $x3 = ADDXri $x0, 3, 0
+ ; End safe range
+ $x16 = ADDXri $x0, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x16 = ADDXri killed $x16, 16, 0
+ RET undef $x9
diff --git a/llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-beginning.mir b/llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-beginning.mir
new file mode 100644
index 0000000000000..a5b2d493a8d04
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-beginning.mir
@@ -0,0 +1,52 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc %s -mtriple aarch64 -run-pass=machine-outliner -o - | FileCheck %s
+
+...
+---
+name: unsafe_range_at_beginning
+tracksRegLiveness: true
+machineFunctionInfo:
+ hasRedZone: false
+body: |
+ bb.0:
+ liveins: $x0, $x9
+ ; Don't outline any of this
+ ; CHECK-LABEL: name: unsafe_range_at_beginning
+ ; CHECK: liveins: $x0, $x9
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: $x16 = ADDXri $x0, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x16 = ADDXri killed $x16, 16, 0
+ ; CHECK-NEXT: BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp, implicit-def $lr, implicit-def $x0, implicit-def $x1, implicit-def $x2, implicit-def $x3, implicit $x0, implicit $sp
+ ; CHECK-NEXT: $x8 = ADDXri $x3, 3, 0
+ ; CHECK-NEXT: BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp, implicit-def $lr, implicit-def $x0, implicit-def $x1, implicit-def $x2, implicit-def $x3, implicit $x0, implicit $sp
+ ; CHECK-NEXT: RET undef $x9
+ $x16 = ADDXri $x0, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x16 = ADDXri killed $x16, 16, 0
+
+ ; Outline
+ $x0 = ADDXri $x0, 0, 0
+ $x1 = ADDXri $x0, 1, 0
+ $x2 = ADDXri $x0, 2, 0
+ $x3 = ADDXri $x0, 3, 0
+
+ ; Split here
+ $x8 = ADDXri $x3, 3, 0
+
+ ; Outline
+ $x0 = ADDXri $x0, 0, 0
+ $x1 = ADDXri $x0, 1, 0
+ $x2 = ADDXri $x0, 2, 0
+ $x3 = ADDXri $x0, 3, 0
+ RET undef $x9
diff --git a/llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-end.mir b/llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-end.mir
new file mode 100644
index 0000000000000..18c12ea3a8a34
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/machine-outliner-unsafe-range-at-end.mir
@@ -0,0 +1,54 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc %s -mtriple aarch64 -run-pass=machine-outliner -o - | FileCheck %s
+
+...
+---
+name: unsafe_range_at_end
+tracksRegLiveness: true
+machineFunctionInfo:
+ hasRedZone: false
+body: |
+ bb.0:
+ liveins: $x0, $x9
+ ; Begin safe range of 3 instructions
+
+ ; Outline
+ ; CHECK-LABEL: name: unsafe_range_at_end
+ ; CHECK: liveins: $x0, $x9
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp, implicit-def $lr, implicit-def $x0, implicit-def $x1, implicit-def $x2, implicit-def $x3, implicit $x0, implicit $sp
+ ; CHECK-NEXT: $x8 = ADDXri $x3, 3, 0
+ ; CHECK-NEXT: BL @OUTLINED_FUNCTION_0, implicit-def $lr, implicit $sp, implicit-def $lr, implicit-def $x0, implicit-def $x1, implicit-def $x2, implicit-def $x3, implicit $x0, implicit $sp
+ ; CHECK-NEXT: $x16 = ADDXri $x0, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x9 = ADDXri $x9, 16, 0
+ ; CHECK-NEXT: $x16 = ADDXri killed $x16, 16, 0
+ ; CHECK-NEXT: RET undef $x9
+ $x0 = ADDXri $x0, 0, 0
+ $x1 = ADDXri $x0, 1, 0
+ $x2 = ADDXri $x0, 2, 0
+ $x3 = ADDXri $x0, 3, 0
+
+ ; Split here
+ $x8 = ADDXri $x3, 3, 0
+
+ ; Outline
+ $x0 = ADDXri $x0, 0, 0
+ $x1 = ADDXri $x0, 1, 0
+ $x2 = ADDXri $x0, 2, 0
+ $x3 = ADDXri $x0, 3, 0
+
+ ; Don't outline any of this
+ $x16 = ADDXri $x0, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x9 = ADDXri $x9, 16, 0
+ $x16 = ADDXri killed $x16, 16, 0
+ RET undef $x9
More information about the llvm-commits
mailing list