[llvm] dc312f0 - [MISched] Introduce and use ResourceSegments.
Francesco Petrogalli via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 9 04:05:35 PDT 2023
Author: Francesco Petrogalli
Date: 2023-06-09T13:00:50+02:00
New Revision: dc312f0331309692e8d6e06e93b3492b6a40989f
URL: https://github.com/llvm/llvm-project/commit/dc312f0331309692e8d6e06e93b3492b6a40989f
DIFF: https://github.com/llvm/llvm-project/commit/dc312f0331309692e8d6e06e93b3492b6a40989f.diff
LOG: [MISched] Introduce and use ResourceSegments.
The class `ResourceSegments` is used to keep track of the intervals
that represent resource usage of a list of instructions that are
being scheduled by the machine scheduler.
The collection is made of intervals that are closed on the left and
open on the right (represented by the standard notation `[a, b)`).
These collections of intervals can be extended by `add`ing new
intervals accordingly while scheduling a basic block.
Unit tests are added to verify the possible configurations of
intervals, and the relative possibility of scheduling a new
instruction in these configurations. Specifically, the methods
`getFirstAvailableAtFromBottom` and `getFirstAvailableAtFromTop` are
tested to make sure that both bottom-up and top-down scheduling work
when tracking resource usage across the basic block with
`ResourceSegments`.
Note that the scheduler tracks resource usage with two methods:
1. counters (via `std::vector<unsigned> ReservedCycles;`);
2. intervals (via `std::map<unsigned, ResourceSegments> ReservedResourceSegments;`).
This patch can be considered a NFC test for existing scheduling models
because the tracking system that uses intervals is turned off by
default (field `bit EnableIntervals = false;` in the tablegen class
`SchedMachineModel`).
Reviewed By: andreadb
Differential Revision: https://reviews.llvm.org/D150312
Added:
llvm/unittests/CodeGen/SchedBoundary.cpp
Modified:
llvm/include/llvm/CodeGen/MachineScheduler.h
llvm/include/llvm/CodeGen/TargetSchedule.h
llvm/include/llvm/MC/MCSchedule.h
llvm/include/llvm/Target/TargetSchedule.td
llvm/lib/CodeGen/MachineScheduler.cpp
llvm/lib/MC/MCSchedule.cpp
llvm/unittests/CodeGen/CMakeLists.txt
llvm/utils/TableGen/SubtargetEmitter.cpp
Removed:
################################################################################
diff --git a/llvm/include/llvm/CodeGen/MachineScheduler.h b/llvm/include/llvm/CodeGen/MachineScheduler.h
index af7c57c950008..bd3885d0463ef 100644
--- a/llvm/include/llvm/CodeGen/MachineScheduler.h
+++ b/llvm/include/llvm/CodeGen/MachineScheduler.h
@@ -92,6 +92,7 @@
#include "llvm/Support/ErrorHandling.h"
#include <algorithm>
#include <cassert>
+#include <llvm/Support/raw_ostream.h>
#include <memory>
#include <string>
#include <vector>
@@ -610,6 +611,222 @@ struct SchedRemainder {
void init(ScheduleDAGMI *DAG, const TargetSchedModel *SchedModel);
};
+/// ResourceSegments are a collection of intervals closed on the
+/// left and opened on the right:
+///
+/// list{ [a1, b1), [a2, b2), ..., [a_N, b_N) }
+///
+/// The collection has the following properties:
+///
+/// 1. The list is ordered: a_i < b_i and b_i < a_(i+1)
+///
+/// 2. The intervals in the collection do not intersect each other.
+///
+/// A \ref ResourceSegments instance represents the cycle
+/// reservation history of the instance of and individual resource.
+class ResourceSegments {
+public:
+ /// Represents an interval of discrete integer values closed on
+ /// the left and open on the right: [a, b).
+ typedef std::pair<int64_t, int64_t> IntervalTy;
+
+ /// Adds an interval [a, b) to the collection of the instance.
+ ///
+ /// When adding [a, b[ to the collection, the operation merges the
+ /// adjacent intervals. For example
+ ///
+ /// 0 1 2 3 4 5 6 7 8 9 10
+ /// [-----) [--) [--)
+ /// + [--)
+ /// = [-----------) [--)
+ ///
+ /// To be able to debug duplicate resource usage, the function has
+ /// assertion that checks that no interval should be added if it
+ /// overlaps any of the intervals in the collection. We can
+ /// require this because by definition a \ref ResourceSegments is
+ /// attached only to an individual resource instance.
+ void add(IntervalTy A, const unsigned CutOff = 10);
+
+public:
+ /// Checks whether intervals intersect.
+ static bool intersects(IntervalTy A, IntervalTy B);
+
+ /// These function return the interval used by a resource in bottom and top
+ /// scheduling.
+ ///
+ /// Consider an instruction that uses resources X0, X1 and X2 as follows:
+ ///
+ /// X0 X1 X1 X2 +--------+------------+------+
+ /// |Resource|StartAtCycle|Cycles|
+ /// +--------+------------+------+
+ /// | X0 | 0 | 1 |
+ /// +--------+------------+------+
+ /// | X1 | 1 | 3 |
+ /// +--------+------------+------+
+ /// | X2 | 3 | 4 |
+ /// +--------+------------+------+
+ ///
+ /// If we can schedule the instruction at cycle C, we need to
+ /// compute the interval of the resource as follows:
+ ///
+ /// # TOP DOWN SCHEDULING
+ ///
+ /// Cycles scheduling flows to the _right_, in the same direction
+ /// of time.
+ ///
+ /// C 1 2 3 4 5 ...
+ /// ------|------|------|------|------|------|----->
+ /// X0 X1 X1 X2 ---> direction of time
+ /// X0 [C, C+1)
+ /// X1 [C+1, C+3)
+ /// X2 [C+3, C+4)
+ ///
+ /// Therefore, the formula to compute the interval for a resource
+ /// of an instruction that can be scheduled at cycle C in top-down
+ /// scheduling is:
+ ///
+ /// [C+StartAtCycle, C+Cycles)
+ ///
+ ///
+ /// # BOTTOM UP SCHEDULING
+ ///
+ /// Cycles scheduling flows to the _left_, in opposite direction
+ /// of time.
+ ///
+ /// In bottom up scheduling, the scheduling happens in opposite
+ /// direction to the execution of the cycles of the
+ /// instruction. When the instruction is scheduled at cycle `C`,
+ /// the resources are allocated in the past relative to `C`:
+ ///
+ /// 2 1 C -1 -2 -3 -4 -5 ...
+ /// <-----|------|------|------|------|------|------|------|---
+ /// X0 X1 X1 X2 ---> direction of time
+ /// X0 (C+1, C]
+ /// X1 (C, C-2]
+ /// X2 (C-2, C-3]
+ ///
+ /// Therefore, the formula to compute the interval for a resource
+ /// of an instruction that can be scheduled at cycle C in bottom-up
+ /// scheduling is:
+ ///
+ /// [C-Cycle+1, C-StartAtCycle+1)
+ ///
+ ///
+ /// NOTE: In both cases, the number of cycles booked by a
+ /// resources is the value (Cycle - StartAtCycles).
+ static IntervalTy getResourceIntervalBottom(unsigned C, unsigned StartAtCycle,
+ unsigned Cycle) {
+ return std::make_pair<long, long>((long)C - (long)Cycle + 1L,
+ (long)C - (long)StartAtCycle + 1L);
+ }
+ static IntervalTy getResourceIntervalTop(unsigned C, unsigned StartAtCycle,
+ unsigned Cycle) {
+ return std::make_pair<long, long>((long)C + (long)StartAtCycle,
+ (long)C + (long)Cycle);
+ }
+
+private:
+ /// Finds the first cycle in which a resource can be allocated.
+ ///
+ /// The function uses the \param IntervalBuider [*] to build a
+ /// resource interval [a, b[ out of the input parameters \param
+ /// CurrCycle, \param StartAtCycle and \param Cycle.
+ ///
+ /// The function then loops through the intervals in the ResourceSegments
+ /// and shifts the interval [a, b[ and the ReturnCycle to the
+ /// right until there is no intersection between the intervals of
+ /// the \ref ResourceSegments instance and the new shifted [a, b[. When
+ /// this condition is met, the ReturnCycle (which
+ /// correspond to the cycle in which the resource can be
+ /// allocated) is returned.
+ ///
+ /// c = CurrCycle in input
+ /// c 1 2 3 4 5 6 7 8 9 10 ... ---> (time
+ /// flow)
+ /// ResourceSegments... [---) [-------) [-----------)
+ /// c [1 3[ -> StartAtCycle=1, Cycles=3
+ /// ++c [1 3)
+ /// ++c [1 3)
+ /// ++c [1 3)
+ /// ++c [1 3)
+ /// ++c [1 3) ---> returns c
+ /// incremented by 5 (c+5)
+ ///
+ ///
+ /// Notice that for bottom-up scheduling the diagram is slightly
+ ///
diff erent because the current cycle c is always on the right
+ /// of the interval [a, b) (see \ref
+ /// `getResourceIntervalBottom`). This is because the cycle
+ /// increments for bottom-up scheduling moved in the direction
+ /// opposite to the direction of time:
+ ///
+ /// --------> direction of time.
+ /// XXYZZZ (resource usage)
+ /// --------> direction of top-down execution cycles.
+ /// <-------- direction of bottom-up execution cycles.
+ ///
+ /// Even though bottom-up scheduling moves against the flow of
+ /// time, the algorithm used to find the first free slot in between
+ /// intervals is the same as for top-down scheduling.
+ ///
+ /// [*] See \ref `getResourceIntervalTop` and
+ /// \ref `getResourceIntervalBottom` to see how such resource intervals
+ /// are built.
+ unsigned
+ getFirstAvailableAt(unsigned CurrCycle, unsigned StartAtCycle, unsigned Cycle,
+ std::function<IntervalTy(unsigned, unsigned, unsigned)>
+ IntervalBuilder) const;
+
+public:
+ /// getFirstAvailableAtFromBottom and getFirstAvailableAtFromTop
+ /// should be merged in a single function in which a function that
+ /// creates the `NewInterval` is passed as a parameter.
+ unsigned getFirstAvailableAtFromBottom(unsigned CurrCycle,
+ unsigned StartAtCycle,
+ unsigned Cycle) const {
+ return getFirstAvailableAt(CurrCycle, StartAtCycle, Cycle,
+ getResourceIntervalBottom);
+ }
+ unsigned getFirstAvailableAtFromTop(unsigned CurrCycle, unsigned StartAtCycle,
+ unsigned Cycle) const {
+ return getFirstAvailableAt(CurrCycle, StartAtCycle, Cycle,
+ getResourceIntervalTop);
+ }
+
+private:
+ std::list<IntervalTy> _Intervals;
+ /// Merge all adjacent intervals in the collection. For all pairs
+ /// of adjacient intervals, it performs [a, b) + [b, c) -> [a, c).
+ ///
+ /// Before performing the merge operation, the intervals are
+ /// sorted with \ref sort_predicate.
+ void sortAndMerge();
+
+public:
+ // constructor for empty set
+ explicit ResourceSegments(){};
+ bool empty() const { return _Intervals.empty(); }
+ explicit ResourceSegments(std::list<IntervalTy> Intervals)
+ : _Intervals(Intervals) {
+ sortAndMerge();
+ }
+
+ friend bool operator==(const ResourceSegments &c1,
+ const ResourceSegments &c2) {
+ return c1._Intervals == c2._Intervals;
+ }
+#ifndef NDEBUG
+ friend llvm::raw_ostream &operator<<(llvm::raw_ostream &os,
+ const ResourceSegments &Segments) {
+ os << "{ ";
+ for (auto p : Segments._Intervals)
+ os << "[" << p.first << ", " << p.second << "), ";
+ os << "}\n";
+ return os;
+ }
+#endif
+};
+
/// Each Scheduling boundary is associated with ready queues. It tracks the
/// current cycle in the direction of movement, and maintains the state
/// of "hazards" and other interlocks at the current cycle.
@@ -675,12 +892,14 @@ class SchedBoundary {
// Is the scheduled region resource limited vs. latency limited.
bool IsResourceLimited;
- // Record the highest cycle at which each resource has been reserved by a
- // scheduled instruction.
- SmallVector<unsigned, 16> ReservedCycles;
-
- /// For each PIdx, stores first index into ReservedCycles that corresponds to
- /// it.
+public:
+private:
+ /// Record how resources have been allocated across the cycles of
+ /// the execution.
+ std::map<unsigned, ResourceSegments> ReservedResourceSegments;
+ std::vector<unsigned> ReservedCycles;
+ /// For each PIdx, stores first index into ReservedResourceSegments that
+ /// corresponds to it.
///
/// For example, consider the following 3 resources (ResourceCount =
/// 3):
@@ -696,12 +915,14 @@ class SchedBoundary {
/// +------------+--------+
///
/// In this case, the total number of resource instances is 6. The
- /// vector \ref ReservedCycles will have a slot for each instance. The
- /// vector \ref ReservedCyclesIndex will track at what index the first
+ /// vector \ref ReservedResourceSegments will have a slot for each instance.
+ /// The vector \ref ReservedCyclesIndex will track at what index the first
/// instance of the resource is found in the vector of \ref
- /// ReservedCycles:
+ /// ReservedResourceSegments:
+ ///
+ /// Indexes of instances in
+ /// ReservedResourceSegments
///
- /// Indexes of instances in ReservedCycles
/// 0 1 2 3 4 5
/// ReservedCyclesIndex[0] = 0; [X0, X1,
/// ReservedCyclesIndex[1] = 2; Y0, Y1, Y2
@@ -787,11 +1008,13 @@ class SchedBoundary {
unsigned getLatencyStallCycles(SUnit *SU);
unsigned getNextResourceCycleByInstance(unsigned InstanceIndex,
- unsigned Cycles);
+ unsigned Cycles,
+ unsigned StartAtCycle);
std::pair<unsigned, unsigned> getNextResourceCycle(const MCSchedClassDesc *SC,
unsigned PIdx,
- unsigned Cycles);
+ unsigned Cycles,
+ unsigned StartAtCycle);
bool isUnbufferedGroup(unsigned PIdx) const {
return SchedModel->getProcResource(PIdx)->SubUnitsIdxBegin &&
@@ -820,7 +1043,8 @@ class SchedBoundary {
void incExecutedResources(unsigned PIdx, unsigned Count);
unsigned countResource(const MCSchedClassDesc *SC, unsigned PIdx,
- unsigned Cycles, unsigned ReadyCycle);
+ unsigned Cycles, unsigned ReadyCycle,
+ unsigned StartAtCycle);
void bumpNode(SUnit *SU);
diff --git a/llvm/include/llvm/CodeGen/TargetSchedule.h b/llvm/include/llvm/CodeGen/TargetSchedule.h
index 049ede89ab461..bfab9cb92a385 100644
--- a/llvm/include/llvm/CodeGen/TargetSchedule.h
+++ b/llvm/include/llvm/CodeGen/TargetSchedule.h
@@ -90,7 +90,7 @@ class TargetSchedModel {
bool hasInstrSchedModelOrItineraries() const {
return hasInstrSchedModel() || hasInstrItineraries();
}
-
+ bool enableIntervals() const { return SchedModel.EnableIntervals; }
/// Identify the processor corresponding to the current subtarget.
unsigned getProcessorID() const { return SchedModel.getProcessorID(); }
diff --git a/llvm/include/llvm/MC/MCSchedule.h b/llvm/include/llvm/MC/MCSchedule.h
index 98254fc23b225..e6cf27ce2d651 100644
--- a/llvm/include/llvm/MC/MCSchedule.h
+++ b/llvm/include/llvm/MC/MCSchedule.h
@@ -58,10 +58,17 @@ struct MCProcResourceDesc {
}
};
-/// Identify one of the processor resource kinds consumed by a particular
-/// scheduling class for the specified number of cycles.
+/// Identify one of the processor resource kinds consumed by a
+/// particular scheduling class for the specified number of cycles.
+/// TODO: consider renaming the field `StartAtCycle` and `Cycles` to
+/// `AcquireAtCycle` and `ReleaseAtCycle` respectively, to stress the
+/// fact that resource allocation is now represented as an interval,
+/// relatively to the issue cycle of the instruction.
struct MCWriteProcResEntry {
uint16_t ProcResourceIdx;
+ /// Cycle at which the resource will be released by an instruction,
+ /// relatively to the cycle in which the instruction is issued
+ /// (assuming no stalls inbetween).
uint16_t Cycles;
/// Cycle at which the resource will be grabbed by an instruction,
/// relatively to the cycle in which the instruction is issued
@@ -306,6 +313,11 @@ struct MCSchedModel {
bool CompleteModel;
+ // Tells the MachineScheduler whether or not to track resource usage
+ // using intervals via ResourceSegments (see
+ // llvm/include/llvm/CodeGen/MachineScheduler.h).
+ bool EnableIntervals;
+
unsigned ProcID;
const MCProcResourceDesc *ProcResourceTable;
const MCSchedClassDesc *SchedClassTable;
diff --git a/llvm/include/llvm/Target/TargetSchedule.td b/llvm/include/llvm/Target/TargetSchedule.td
index d0742c2adb781..f5aa105ec0cbb 100644
--- a/llvm/include/llvm/Target/TargetSchedule.td
+++ b/llvm/include/llvm/Target/TargetSchedule.td
@@ -117,6 +117,11 @@ class SchedMachineModel {
list<Predicate> UnsupportedFeatures = [];
bit NoModel = false; // Special tag to indicate missing machine model.
+
+ // Tells the MachineScheduler whether or not to track resource usage
+ // using intervals via ResourceSegments (see
+ // llvm/include/llvm/CodeGen/MachineScheduler.h).
+ bit EnableIntervals = false;
}
def NoSchedModel : SchedMachineModel {
diff --git a/llvm/lib/CodeGen/MachineScheduler.cpp b/llvm/lib/CodeGen/MachineScheduler.cpp
index b5b91802ef238..59248bdea1914 100644
--- a/llvm/lib/CodeGen/MachineScheduler.cpp
+++ b/llvm/lib/CodeGen/MachineScheduler.cpp
@@ -162,6 +162,10 @@ static cl::opt<unsigned>
cl::init(5));
#endif
+static cl::opt<unsigned>
+ MIResourceCutOff("misched-resource-cutoff", cl::Hidden,
+ cl::desc("Number of intervals to track"), cl::init(10));
+
// DAG subtrees must have at least this many nodes.
static const unsigned MinSubtreeSize = 8;
@@ -2168,6 +2172,7 @@ void SchedBoundary::reset() {
ZoneCritResIdx = 0;
IsResourceLimited = false;
ReservedCycles.clear();
+ ReservedResourceSegments.clear();
ReservedCyclesIndex.clear();
ResourceGroupSubUnitMasks.clear();
#if LLVM_ENABLE_ABI_BREAKING_CHECKS
@@ -2196,7 +2201,8 @@ init(ScheduleDAGMI *DAG, const TargetSchedModel *SchedModel) {
PE = SchedModel->getWriteProcResEnd(SC); PI != PE; ++PI) {
unsigned PIdx = PI->ProcResourceIdx;
unsigned Factor = SchedModel->getResourceFactor(PIdx);
- RemainingCounts[PIdx] += (Factor * PI->Cycles);
+ assert(PI->Cycles >= PI->StartAtCycle);
+ RemainingCounts[PIdx] += (Factor * (PI->Cycles - PI->StartAtCycle));
}
}
}
@@ -2249,7 +2255,17 @@ unsigned SchedBoundary::getLatencyStallCycles(SUnit *SU) {
/// Compute the next cycle at which the given processor resource unit
/// can be scheduled.
unsigned SchedBoundary::getNextResourceCycleByInstance(unsigned InstanceIdx,
- unsigned Cycles) {
+ unsigned Cycles,
+ unsigned StartAtCycle) {
+ if (SchedModel && SchedModel->enableIntervals()) {
+ if (isTop())
+ return ReservedResourceSegments[InstanceIdx].getFirstAvailableAtFromTop(
+ CurrCycle, StartAtCycle, Cycles);
+
+ return ReservedResourceSegments[InstanceIdx].getFirstAvailableAtFromBottom(
+ CurrCycle, StartAtCycle, Cycles);
+ }
+
unsigned NextUnreserved = ReservedCycles[InstanceIdx];
// If this resource has never been used, always return cycle zero.
if (NextUnreserved == InvalidCycle)
@@ -2265,7 +2281,7 @@ unsigned SchedBoundary::getNextResourceCycleByInstance(unsigned InstanceIdx,
/// instance in the reserved cycles vector.
std::pair<unsigned, unsigned>
SchedBoundary::getNextResourceCycle(const MCSchedClassDesc *SC, unsigned PIdx,
- unsigned Cycles) {
+ unsigned Cycles, unsigned StartAtCycle) {
unsigned MinNextUnreserved = InvalidCycle;
unsigned InstanceIdx = 0;
@@ -2294,7 +2310,7 @@ SchedBoundary::getNextResourceCycle(const MCSchedClassDesc *SC, unsigned PIdx,
for (unsigned I = 0, End = NumberOfInstances; I < End; ++I) {
unsigned NextUnreserved, NextInstanceIdx;
std::tie(NextUnreserved, NextInstanceIdx) =
- getNextResourceCycle(SC, SubUnits[I], Cycles);
+ getNextResourceCycle(SC, SubUnits[I], Cycles, StartAtCycle);
if (MinNextUnreserved > NextUnreserved) {
InstanceIdx = NextInstanceIdx;
MinNextUnreserved = NextUnreserved;
@@ -2305,7 +2321,8 @@ SchedBoundary::getNextResourceCycle(const MCSchedClassDesc *SC, unsigned PIdx,
for (unsigned I = StartIndex, End = StartIndex + NumberOfInstances; I < End;
++I) {
- unsigned NextUnreserved = getNextResourceCycleByInstance(I, Cycles);
+ unsigned NextUnreserved =
+ getNextResourceCycleByInstance(I, Cycles, StartAtCycle);
if (MinNextUnreserved > NextUnreserved) {
InstanceIdx = I;
MinNextUnreserved = NextUnreserved;
@@ -2355,8 +2372,10 @@ bool SchedBoundary::checkHazard(SUnit *SU) {
SchedModel->getWriteProcResEnd(SC))) {
unsigned ResIdx = PE.ProcResourceIdx;
unsigned Cycles = PE.Cycles;
+ unsigned StartAtCycle = PE.StartAtCycle;
unsigned NRCycle, InstanceIdx;
- std::tie(NRCycle, InstanceIdx) = getNextResourceCycle(SC, ResIdx, Cycles);
+ std::tie(NRCycle, InstanceIdx) =
+ getNextResourceCycle(SC, ResIdx, Cycles, StartAtCycle);
if (NRCycle > CurrCycle) {
#if LLVM_ENABLE_ABI_BREAKING_CHECKS
MaxObservedStall = std::max(Cycles, MaxObservedStall);
@@ -2507,9 +2526,10 @@ void SchedBoundary::incExecutedResources(unsigned PIdx, unsigned Count) {
/// \return the next cycle at which the instruction may execute without
/// oversubscribing resources.
unsigned SchedBoundary::countResource(const MCSchedClassDesc *SC, unsigned PIdx,
- unsigned Cycles, unsigned NextCycle) {
+ unsigned Cycles, unsigned NextCycle,
+ unsigned StartAtCycle) {
unsigned Factor = SchedModel->getResourceFactor(PIdx);
- unsigned Count = Factor * Cycles;
+ unsigned Count = Factor * (Cycles - StartAtCycle);
LLVM_DEBUG(dbgs() << " " << SchedModel->getResourceName(PIdx) << " +"
<< Cycles << "x" << Factor << "u\n");
@@ -2529,7 +2549,8 @@ unsigned SchedBoundary::countResource(const MCSchedClassDesc *SC, unsigned PIdx,
}
// For reserved resources, record the highest cycle using the resource.
unsigned NextAvailable, InstanceIdx;
- std::tie(NextAvailable, InstanceIdx) = getNextResourceCycle(SC, PIdx, Cycles);
+ std::tie(NextAvailable, InstanceIdx) =
+ getNextResourceCycle(SC, PIdx, Cycles, StartAtCycle);
if (NextAvailable > CurrCycle) {
LLVM_DEBUG(dbgs() << " Resource conflict: "
<< SchedModel->getResourceName(PIdx)
@@ -2608,8 +2629,8 @@ void SchedBoundary::bumpNode(SUnit *SU) {
for (TargetSchedModel::ProcResIter
PI = SchedModel->getWriteProcResBegin(SC),
PE = SchedModel->getWriteProcResEnd(SC); PI != PE; ++PI) {
- unsigned RCycle =
- countResource(SC, PI->ProcResourceIdx, PI->Cycles, NextCycle);
+ unsigned RCycle = countResource(SC, PI->ProcResourceIdx, PI->Cycles,
+ NextCycle, PI->StartAtCycle);
if (RCycle > NextCycle)
NextCycle = RCycle;
}
@@ -2623,14 +2644,33 @@ void SchedBoundary::bumpNode(SUnit *SU) {
PE = SchedModel->getWriteProcResEnd(SC); PI != PE; ++PI) {
unsigned PIdx = PI->ProcResourceIdx;
if (SchedModel->getProcResource(PIdx)->BufferSize == 0) {
- unsigned ReservedUntil, InstanceIdx;
- std::tie(ReservedUntil, InstanceIdx) =
- getNextResourceCycle(SC, PIdx, 0);
- if (isTop()) {
- ReservedCycles[InstanceIdx] =
- std::max(ReservedUntil, NextCycle + PI->Cycles);
- } else
- ReservedCycles[InstanceIdx] = NextCycle;
+
+ if (SchedModel && SchedModel->enableIntervals()) {
+ unsigned ReservedUntil, InstanceIdx;
+ std::tie(ReservedUntil, InstanceIdx) =
+ getNextResourceCycle(SC, PIdx, PI->Cycles, PI->StartAtCycle);
+ if (isTop()) {
+ ReservedResourceSegments[InstanceIdx].add(
+ ResourceSegments::getResourceIntervalTop(
+ NextCycle, PI->StartAtCycle, PI->Cycles),
+ MIResourceCutOff);
+ } else {
+ ReservedResourceSegments[InstanceIdx].add(
+ ResourceSegments::getResourceIntervalBottom(
+ NextCycle, PI->StartAtCycle, PI->Cycles),
+ MIResourceCutOff);
+ }
+ } else {
+
+ unsigned ReservedUntil, InstanceIdx;
+ std::tie(ReservedUntil, InstanceIdx) =
+ getNextResourceCycle(SC, PIdx, 0, PI->StartAtCycle);
+ if (isTop()) {
+ ReservedCycles[InstanceIdx] =
+ std::max(ReservedUntil, NextCycle + PI->Cycles);
+ } else
+ ReservedCycles[InstanceIdx] = NextCycle;
+ }
}
}
}
@@ -2770,8 +2810,14 @@ LLVM_DUMP_METHOD void SchedBoundary::dumpReservedCycles() const {
const unsigned NumUnits = SchedModel->getProcResource(ResIdx)->NumUnits;
std::string ResName = SchedModel->getResourceName(ResIdx);
for (unsigned UnitIdx = 0; UnitIdx < NumUnits; ++UnitIdx) {
- dbgs() << ResName << "(" << UnitIdx
- << ") = " << ReservedCycles[StartIdx + UnitIdx] << "\n";
+ dbgs() << ResName << "(" << UnitIdx << ") = ";
+ if (SchedModel && SchedModel->enableIntervals()) {
+ if (ReservedResourceSegments.count(StartIdx + UnitIdx))
+ dbgs() << ReservedResourceSegments.at(StartIdx + UnitIdx);
+ else
+ dbgs() << "{ }\n";
+ } else
+ dbgs() << ReservedCycles[StartIdx + UnitIdx] << "\n";
}
StartIdx += NumUnits;
}
@@ -4138,3 +4184,102 @@ void ScheduleDAGMI::viewGraph(const Twine &Name, const Twine &Title) {
void ScheduleDAGMI::viewGraph() {
viewGraph(getDAGName(), "Scheduling-Units Graph for " + getDAGName());
}
+
+/// Sort predicate for the intervals stored in an instance of
+/// ResourceSegments. Intervals are always disjoint (no intersection
+/// for any pairs of intervals), therefore we can sort the totality of
+/// the intervals by looking only at the left boundary.
+static bool sortIntervals(const ResourceSegments::IntervalTy &A,
+ const ResourceSegments::IntervalTy &B) {
+ return A.first < B.first;
+}
+
+unsigned ResourceSegments::getFirstAvailableAt(
+ unsigned CurrCycle, unsigned StartAtCycle, unsigned Cycle,
+ std::function<ResourceSegments::IntervalTy(unsigned, unsigned, unsigned)>
+ IntervalBuilder) const {
+ assert(std::is_sorted(std::begin(_Intervals), std::end(_Intervals),
+ sortIntervals) &&
+ "Cannot execute on an un-sorted set of intervals.");
+ unsigned RetCycle = CurrCycle;
+ ResourceSegments::IntervalTy NewInterval =
+ IntervalBuilder(RetCycle, StartAtCycle, Cycle);
+ for (auto &Interval : _Intervals) {
+ if (!intersects(NewInterval, Interval))
+ continue;
+
+ // Move the interval right next to the top of the one it
+ // intersects.
+ assert(Interval.second > NewInterval.first &&
+ "Invalid intervals configuration.");
+ RetCycle += (unsigned)Interval.second - (unsigned)NewInterval.first;
+ NewInterval = IntervalBuilder(RetCycle, StartAtCycle, Cycle);
+ }
+ return RetCycle;
+}
+
+void ResourceSegments::add(ResourceSegments::IntervalTy A,
+ const unsigned CutOff) {
+ using IntervalTy = ResourceSegments::IntervalTy;
+ assert(A.first < A.second && "Cannot add empty resource usage");
+ assert(CutOff > 0 && "0-size interval history has no use.");
+ assert(all_of(_Intervals,
+ [&A](const IntervalTy &Interval) -> bool {
+ return !intersects(A, Interval);
+ }) &&
+ "A resource is being overwritten");
+ _Intervals.push_back(A);
+
+ sortAndMerge();
+
+ // Do not keep the full history of the intervals, just the
+ // latest #CutOff.
+ while (_Intervals.size() > CutOff)
+ _Intervals.pop_front();
+}
+
+bool ResourceSegments::intersects(ResourceSegments::IntervalTy A,
+ ResourceSegments::IntervalTy B) {
+ assert(A.first <= A.second && "Invalid interval");
+ assert(B.first <= B.second && "Invalid interval");
+
+ // Share one boundary.
+ if ((A.first == B.first) || (A.second == B.second))
+ return true;
+
+ // full intersersect: [ *** ) B
+ // [***) A
+ if ((A.first > B.first) && (A.second < B.second))
+ return true;
+
+ // right intersect: [ ***) B
+ // [*** ) A
+ if ((A.first > B.first) && (A.first < B.second) && (A.second > B.second))
+ return true;
+
+ // left intersect: [*** ) B
+ // [ ***) A
+ if ((A.first < B.first) && (B.first < A.second) && (B.second > B.first))
+ return true;
+
+ return false;
+}
+
+void ResourceSegments::sortAndMerge() {
+ if (_Intervals.size() <= 1)
+ return;
+
+ // First sort the collection.
+ _Intervals.sort(sortIntervals);
+
+ // can use next because I have at least 2 elements in the list
+ auto next = std::next(std::begin(_Intervals));
+ auto E = std::end(_Intervals);
+ for (; next != E; ++next) {
+ if (std::prev(next)->second >= next->first) {
+ next->first = std::prev(next)->first;
+ _Intervals.erase(std::prev(next));
+ continue;
+ }
+ }
+}
diff --git a/llvm/lib/MC/MCSchedule.cpp b/llvm/lib/MC/MCSchedule.cpp
index 9f00064f501bb..5a893b803fd07 100644
--- a/llvm/lib/MC/MCSchedule.cpp
+++ b/llvm/lib/MC/MCSchedule.cpp
@@ -30,6 +30,7 @@ const MCSchedModel MCSchedModel::Default = {DefaultIssueWidth,
DefaultMispredictPenalty,
false,
true,
+ false /*EnableIntervals*/,
0,
nullptr,
nullptr,
diff --git a/llvm/unittests/CodeGen/CMakeLists.txt b/llvm/unittests/CodeGen/CMakeLists.txt
index f3a5b0f8e75db..dc47174b9513e 100644
--- a/llvm/unittests/CodeGen/CMakeLists.txt
+++ b/llvm/unittests/CodeGen/CMakeLists.txt
@@ -35,6 +35,7 @@ add_llvm_unittest(CodeGenTests
RegAllocScoreTest.cpp
PassManagerTest.cpp
ScalableVectorMVTsTest.cpp
+ SchedBoundary.cpp
SelectionDAGAddressAnalysisTest.cpp
TypeTraitsTest.cpp
TargetOptionsTest.cpp
diff --git a/llvm/unittests/CodeGen/SchedBoundary.cpp b/llvm/unittests/CodeGen/SchedBoundary.cpp
new file mode 100644
index 0000000000000..0eaa1b216f2cc
--- /dev/null
+++ b/llvm/unittests/CodeGen/SchedBoundary.cpp
@@ -0,0 +1,398 @@
+#include "llvm/CodeGen/MachineScheduler.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+#ifndef NDEBUG
+TEST(ResourceSegmentsDeath, OverwriteOnRight) {
+ auto X = ResourceSegments({{10, 20}});
+ EXPECT_DEATH(X.add({15, 30}), "A resource is being overwritten");
+}
+
+TEST(ResourceSegmentsDeath, OverwriteOnLeft) {
+ auto X = ResourceSegments({{10, 20}});
+ EXPECT_DEATH(X.add({5, 11}), "A resource is being overwritten");
+ ;
+}
+
+TEST(ResourceSegmentsDeath, FullOverwrite) {
+ auto X = ResourceSegments({{10, 20}});
+ EXPECT_DEATH(X.add({15, 18}), "A resource is being overwritten");
+}
+
+TEST(ResourceSegmentsDeath, ZeroSizeIntervalsNotAllowed) {
+ auto X = ResourceSegments({{10, 20}});
+ EXPECT_DEATH(X.add({20, 30}, 0), "0-size interval history has no use.");
+}
+#endif // NDEBUG
+
+TEST(ResourceSegments, ConsecutiveLeftNoOverlap) {
+ auto X = ResourceSegments({{10, 20}});
+ X.add({7, 9});
+ EXPECT_EQ(X, ResourceSegments({{7, 9}, {10, 20}}));
+}
+
+TEST(ResourceSegments, ConsecutiveLeftWithOverlap) {
+ auto X = ResourceSegments({{10, 20}});
+ X.add({7, 10});
+ EXPECT_EQ(X, ResourceSegments({{7, 20}}));
+}
+
+TEST(ResourceSegments, ConsecutiveRightNoOverlap) {
+ auto X = ResourceSegments({{10, 20}});
+ X.add({21, 22});
+ EXPECT_EQ(X, ResourceSegments({{10, 20}, {21, 22}}));
+}
+
+TEST(ResourceSegments, ConsecutiveRightWithOverlap) {
+ auto X = ResourceSegments({{10, 20}});
+ X.add({20, 22});
+ EXPECT_EQ(X, ResourceSegments({{10, 22}}));
+}
+
+TEST(ResourceSegments, Disjoint) {
+ auto X = ResourceSegments({{10, 20}});
+ X.add({22, 23});
+ EXPECT_EQ(X, ResourceSegments({{10, 20}, {22, 23}}));
+}
+
+TEST(ResourceSegments, SortAfterAdd) {
+ auto X = ResourceSegments({{10, 20}, {3, 4}});
+ X.add({6, 8});
+ EXPECT_EQ(X, ResourceSegments({{3, 4}, {6, 8}, {10, 20}}));
+}
+
+TEST(ResourceSegments, AddWithCutOff) {
+ auto X = ResourceSegments({{1, 2}, {3, 4}});
+ X.add({6, 8}, 2);
+ EXPECT_EQ(X, ResourceSegments({{3, 4}, {6, 8}}));
+}
+
+TEST(ResourceSegments, add_01) {
+ auto X = ResourceSegments({{10, 20}, {30, 40}});
+ X.add({21, 29});
+ EXPECT_EQ(X, ResourceSegments({{10, 20}, {21, 29}, {30, 40}}));
+}
+
+TEST(ResourceSegments, add_02) {
+ auto X = ResourceSegments({{10, 20}, {30, 40}});
+ X.add({22, 29});
+ EXPECT_EQ(X, ResourceSegments({{10, 20}, {22, 29}, {30, 40}}));
+ X.add({29, 30});
+ EXPECT_EQ(X, ResourceSegments({{10, 20}, {22, 40}}));
+}
+
+#ifndef NDEBUG
+TEST(ResourceSegmentsDeath, add_empty) {
+ auto X = ResourceSegments({{10, 20}, {30, 40}});
+ EXPECT_DEATH(X.add({22, 22}), "Cannot add empty resource usage");
+}
+#endif
+
+TEST(ResourceSegments, sort_two) {
+ EXPECT_EQ(ResourceSegments({{30, 40}, {10, 28}}),
+ ResourceSegments({{10, 28}, {30, 40}}));
+}
+
+TEST(ResourceSegments, sort_three) {
+ EXPECT_EQ(ResourceSegments({{30, 40}, {71, 200}, {10, 29}}),
+ ResourceSegments({{10, 29}, {30, 40}, {71, 200}}));
+}
+
+TEST(ResourceSegments, merge_two) {
+ EXPECT_EQ(ResourceSegments({{10, 33}, {30, 40}}),
+ ResourceSegments({{10, 40}}));
+ EXPECT_EQ(ResourceSegments({{10, 30}, {30, 40}}),
+ ResourceSegments({{10, 40}}));
+ // Cycle 29 is resource free, so the interval is disjoint.
+ EXPECT_EQ(ResourceSegments({{10, 29}, {30, 40}}),
+ ResourceSegments({{10, 29}, {30, 40}}));
+}
+
+TEST(ResourceSegments, merge_three) {
+ EXPECT_EQ(ResourceSegments({{10, 29}, {30, 40}, {71, 200}}),
+ ResourceSegments({{10, 29}, {30, 40}, {71, 200}}));
+ EXPECT_EQ(ResourceSegments({{10, 29}, {30, 40}, {41, 200}}),
+ ResourceSegments({{10, 29}, {30, 40}, {41, 200}}));
+ EXPECT_EQ(ResourceSegments({{10, 30}, {30, 40}, {40, 200}}),
+ ResourceSegments({{10, 200}}));
+ EXPECT_EQ(ResourceSegments({{10, 28}, {30, 71}, {71, 200}}),
+ ResourceSegments({{10, 28}, {30, 200}}));
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// Intersection
+TEST(ResourceSegments, intersects) {
+ // no intersect
+ EXPECT_FALSE(ResourceSegments::intersects({0, 1}, {3, 4}));
+ EXPECT_FALSE(ResourceSegments::intersects({3, 4}, {0, 1}));
+ EXPECT_FALSE(ResourceSegments::intersects({0, 3}, {3, 4}));
+ EXPECT_FALSE(ResourceSegments::intersects({3, 4}, {0, 3}));
+
+ // Share one boundary
+ EXPECT_TRUE(ResourceSegments::intersects({5, 6}, {5, 10}));
+ EXPECT_TRUE(ResourceSegments::intersects({5, 10}, {5, 6}));
+
+ // full intersect
+ EXPECT_TRUE(ResourceSegments::intersects({1, 2}, {0, 3}));
+ EXPECT_TRUE(ResourceSegments::intersects({1, 2}, {0, 2}));
+ EXPECT_TRUE(ResourceSegments::intersects({0, 3}, {1, 2}));
+ EXPECT_TRUE(ResourceSegments::intersects({0, 2}, {1, 2}));
+
+ // right intersect
+ EXPECT_TRUE(ResourceSegments::intersects({2, 4}, {0, 3}));
+ EXPECT_TRUE(ResourceSegments::intersects({0, 3}, {2, 4}));
+
+ // left intersect
+ EXPECT_TRUE(ResourceSegments::intersects({2, 4}, {3, 5}));
+ EXPECT_TRUE(ResourceSegments::intersects({3, 5}, {2, 4}));
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// TOP-DOWN getFirstAvailableAt
+TEST(ResourceSegments, getFirstAvailableAtFromTop_oneCycle) {
+ auto X = ResourceSegments({{2, 5}});
+ // 0 1 2 3 4 5 6 7
+ // Res X X X
+ // ...X...
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(0, 0, 1), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(1, 0, 1), 1U);
+ // Skip to five when hitting cycle 2
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 0, 1), 5U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromTop_twoCycles) {
+ auto X = ResourceSegments({{4, 5}});
+ // 0 1 2 3 4 5 6 7
+ // Res X
+ // ...X X....
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(0, 0, 2), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(1, 0, 2), 1U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 0, 2), 2U);
+ // Skip to cycle 5
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(3, 0, 2), 5U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromTop_twoCycles_Shifted) {
+ auto X = ResourceSegments({{4, 5}});
+ // 0 1 2 3 4 5 6 7
+ // Res X
+ // ...c X X...
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(0, 1, 3), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(1, 1, 3), 1U);
+ // Skip to cycle 4
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 1, 3), 4U);
+ // Stay con cycle 4
+ // 0 1 2 3 4 5 6 7
+ // Res X
+ // ...c X X...
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(3, 1, 3), 4U);
+ //
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(4, 1, 3), 4U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(5, 1, 3), 5U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromTop_twoCycles_Shifted_withGap) {
+ auto X = ResourceSegments({{4, 5}, {7, 9}});
+ // 0 1 2 3 4 5 6 7 8 9
+ // Res X X X
+ // c X X
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(1, 1, 3), 1U);
+ // 0 1 2 3 4 5 6 7 8 9
+ // Res X X X
+ // c X X --> moves to 4
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 1, 3), 4U);
+ // 0 1 2 3 4 5 6 7 8 9
+ // Res X X X
+ // c X X --> moves to 4
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(3, 1, 3), 4U);
+ // 0 1 2 3 4 5 6 7 8 9
+ // Res X X X
+ // c X X --> stays on 4
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(4, 1, 3), 4U);
+ // 0 1 2 3 4 5 6 7 8 9
+ // Res X X X
+ // c X X --> skips to 8
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(5, 1, 3), 8U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromTop_basic) {
+ auto X = ResourceSegments({{5, 10}, {30, 40}});
+
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(0, 3, 4), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(1, 3, 4), 1U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 3, 4), 7U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(3, 3, 4), 7U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(4, 3, 4), 7U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(5, 3, 4), 7U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(6, 3, 4), 7U);
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(7, 3, 4), 7U);
+ // Check the empty range between the two intervals of X.
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(15, 3, 4), 15U);
+ // Overlap the second interval.
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(28, 3, 4), 37U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromTop_advanced) {
+ auto X = ResourceSegments({{3, 6}, {7, 9}, {11, 14}, {30, 33}});
+
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 4, 5), 2U);
+
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(2, 3, 4), 3U);
+ // Can schedule at 7U because the interval [14, 19[ does not
+ // overlap any of the intervals in X.
+ EXPECT_EQ(X.getFirstAvailableAtFromTop(1, 7, 12), 7U);
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// BOTTOM-UP getFirstAvailableAt
+TEST(ResourceSegments, getFirstAvailableAtFromBottom) {
+ // Scheduling cycles move to the left...
+ //
+ // 41 40 39 ... 31 30 29 ... 21 20 19 ... 11 10 9 8 7 6 ... 1 0
+ // Res X X X X X X X X
+ // X X X X X X
+ // Time (relative to instruction execution) 0 1 2 3 4 5
+ auto X = ResourceSegments({{10, 20}, {30, 40}});
+ // .. but time (instruction cycle) moves to the right. Therefore, it
+ // is always possible to llocate a resource to the right of 0 if 0
+ // is not taken, because the right side of the scheduling cycles is
+ // empty.
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 1), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 9), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 10), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 20), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 21), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 22), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 29), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 30), 0U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromBottom_01) {
+ auto X = ResourceSegments({{3, 7}});
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // ...X... <- one cycle resource placement
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 1), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(1, 0, 1), 1U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(2, 0, 1), 2U);
+ // Skip to 7
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(3, 0, 1), 7U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromBottom_02) {
+ auto X = ResourceSegments({{3, 7}});
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // ...X X... <- two cycles resource placement
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 2), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(1, 0, 2), 1U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(2, 0, 2), 2U);
+ // skip to 8
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(3, 0, 2), 8U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromBottom_02_shifted) {
+ auto X = ResourceSegments({{3, 7}});
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // c X X <- two cycles resource placement but shifted by 1
+ // 0 1 2 <- cycles relative to the execution of the
+ // instruction
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 1, 3), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(1, 1, 3), 1U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(2, 1, 3), 2U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(3, 1, 3), 3U);
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // c X X -> skip to 9
+ // 0 1 2
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(4, 1, 3), 9U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(5, 1, 3), 9U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(6, 1, 3), 9U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(7, 1, 3), 9U);
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // c X X <- skip to 9
+ // 0 1 2
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(8, 1, 3), 9U);
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // c X X
+ // 0 1 2
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(9, 1, 3), 9U);
+ // 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X
+ // c X X
+ // 0 1 2
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(10, 1, 3), 10U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromBottom_03) {
+ auto X = ResourceSegments({{1, 2}, {3, 7}});
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X X
+ // X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 1), 0U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X X
+ // X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(1, 0, 1), 2U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X X
+ // X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(2, 0, 1), 2U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X X
+ // X X X X X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(2, 0, 5), 11U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(3, 0, 5), 11U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(5, 0, 5), 11U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(11, 0, 5), 11U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+ // X X X X X
+ // X X X X X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(12, 0, 5), 12U);
+}
+
+TEST(ResourceSegments, getFirstAvailableAtFromBottom_03_shifted) {
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ auto X = ResourceSegments({{-3, -1}, {1, 2}, {3, 7}, {9, 10}});
+
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 1, 2), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 2), 0U);
+
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ // X X X -> skip to cycle 12
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 3), 12U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ // X X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 1, 3), 1U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 1, 4), 13U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(12, 1, 4), 13U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ // c X X X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(13, 1, 4), 13U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ // X X
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(1, 1, 3), 1U);
+ // 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ // C X X 0 -> skip to cycle 9
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(2, 1, 3), 9U);
+ // 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3
+ // X X X X X X X X
+ // C C X X X X X -> skip to cycle 16
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(3, 2, 7), 16U);
+}
+TEST(ResourceSegments, getFirstAvailableAtFromBottom_empty) {
+ // Empty resource usage can accept schediling at any cycle
+ auto X = ResourceSegments({});
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(0, 0, 1), 0U);
+ EXPECT_EQ(X.getFirstAvailableAtFromBottom(17, 1, 22), 17U);
+}
diff --git a/llvm/utils/TableGen/SubtargetEmitter.cpp b/llvm/utils/TableGen/SubtargetEmitter.cpp
index 69884a050852e..bd96ea74829d9 100644
--- a/llvm/utils/TableGen/SubtargetEmitter.cpp
+++ b/llvm/utils/TableGen/SubtargetEmitter.cpp
@@ -1453,6 +1453,12 @@ void SubtargetEmitter::EmitProcessorModels(raw_ostream &OS) {
OS << " " << (CompleteModel ? "true" : "false") << ", // "
<< "CompleteModel\n";
+ bool EnableIntervals =
+ (PM.ModelDef ? PM.ModelDef->getValueAsBit("EnableIntervals") : false);
+
+ OS << " " << (EnableIntervals ? "true" : "false") << ", // "
+ << "EnableIntervals\n";
+
OS << " " << PM.Index << ", // Processor ID\n";
if (PM.hasInstrSchedModel())
OS << " " << PM.ModelName << "ProcResources" << ",\n"
More information about the llvm-commits
mailing list