[llvm] [DebugInfo][InstrRef] Index DebugVariables and some DILocations (PR #99318)
Jeremy Morse via llvm-commits
llvm-commits at lists.llvm.org
Wed Jul 17 05:55:57 PDT 2024
https://github.com/jmorse created https://github.com/llvm/llvm-project/pull/99318
A lot of time in LiveDebugValues is spent computing DenseMap keys for DebugVariables, and they're made up of three pointers, so are large. This patch installs an index for them: for the SSA and value-to-location mapping parts of InstrRefBasedLDV we don't need to access things like the variable declaration or the inlining site, so just use a uint32_t identifier for each variable fragment that's tracked. The compile-time performance improvements are substantial (almost 0.4% on the tracker).
About 80% of this patch is just replacing DebugVariable references with DebugVariableIDs instead, however there are some larger consequences. We spend lots of time fetching DILocations when emitting DBG_VALUE instructions, so index those with the DebugVariables: this means all DILocations on all new DBG_VALUE instructions will normalise to the first-seen DILocation for the variable (which should be fine).
We also used to keep an ordering of when each variable was seen first in a DBG_* instruction, in the AllVarsNumbering collection, so that we can emit new DBG_* instructions in a stable order. We can hang this off the DebugVariable index instead, so AllVarsNumbering is deleted.
As a consequence of deleting AllVarsNumbering, the order that DBG_* instructions are outputted in can change: we now produce a total order based on RPO exploration of the function, not exploring in normal block-order. That causes the DBG_* instructions in live-debug-values-fragments.mir to change order. These differences don't affect the meaning of the debug-info, but will cause slightly different ordered output, hence this isn't an NFC patch. Downstream tests that fail because of ordering changes /should/ be able to just update the order with no ill effects.
Finally, rather than ordering by AllVarsNumbering just before DBG_* instructions are linked into the output MIR, store instructions along with their DebugVariableID, so that they can be sorted by that instead.
(I thought about trying to split this patch up into something smaller, but there's no convenient half-way point without having to generate some dodgy code splicing bits and pieces together, sorry).
>From 7b425960c2ad9797330218aa6ed91bd3ee3a32ec Mon Sep 17 00:00:00 2001
From: Jeremy Morse <jeremy.morse at sony.com>
Date: Wed, 19 Jun 2024 13:15:18 +0000
Subject: [PATCH] [InstrRef] Index DebugVariables and some DILocations
A lot of time in LiveDebugValues is spent computing DenseMap keys for
DebugVariables, and they're made up of three pointers, so are large. This
patch installs an index for them: for the SSA and value-to-location mapping
parts of InstrRefBasedLDV we don't need to access things like the variable
declaration or the inlining site, so just use a uint32_t identifier for
each variable fragment that's tracked. The compile-time performance
improvements are substantial (almost 0.4% on the tracker).
About 80% of this patch is just replacing DebugVariable references with
DebugVariableIDs instead, however there are some larger consequences. We
spend lots of time fetching DILocations when emitting DBG_VALUE
instructions, so index those with the DebugVariables: this means all
DILocations on all DBG_VALUE instructions will normalise to the first-seen
DILocation for the variable (which should be fine).
We also used to keep an ordering of when each variable was seen first in a
DBG_* instruction, in the AllVarsNumbering collection, so that we can emit
new DBG_* instructions in a stable order. We can hang this off the
DebugVariable index instead, so AllVarsNumbering is deleted.
As a consequence of deleting AllVarsNumbering, the order that DBG_*
instructions are outputted in can change: we now produce a total order
based on RPO exploration of the function, not exploring in normal
block-order. That causes the DBG_* instructions in
live-debug-values-fragments.mir to change order. These differences don't
affect the meaning of the debug-info, but will cause slightly different
ordered output, hence this isn't an NFC patch. Downstream tests that fail
because of ordering changes /should/ be able to just update the order with
no ill effects.
Finally, rather than ordering by AllVarsNumbering just before DBG_*
instructions are linked into the output MIR, store instructions along with
their DebugVariableID, so that they can be sorted by that instead.
---
.../LiveDebugValues/InstrRefBasedImpl.cpp | 201 +++++++++---------
.../LiveDebugValues/InstrRefBasedImpl.h | 105 ++++++---
.../MIR/X86/live-debug-values-fragments.mir | 12 +-
llvm/unittests/CodeGen/InstrRefLDVTest.cpp | 3 +-
4 files changed, 189 insertions(+), 132 deletions(-)
diff --git a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
index 555cbb7a507f4..dbb21344c72d0 100644
--- a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
+++ b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
@@ -183,6 +183,7 @@ class TransferTracker {
/// information from it. (XXX make it const?)
MLocTracker *MTracker;
MachineFunction &MF;
+ const DebugVariableMap &DVMap;
bool ShouldEmitDebugEntryValues;
/// Record of all changes in variable locations at a block position. Awkwardly
@@ -191,7 +192,9 @@ class TransferTracker {
struct Transfer {
MachineBasicBlock::instr_iterator Pos; /// Position to insert DBG_VALUes
MachineBasicBlock *MBB; /// non-null if we should insert after.
- SmallVector<MachineInstr *, 4> Insts; /// Vector of DBG_VALUEs to insert.
+ /// Vector of DBG_VALUEs to insert. Store with their DebugVariableID so that
+ /// they can be sorted into a stable order for emission at a later time.
+ SmallVector<std::pair<DebugVariableID, MachineInstr *>, 4> Insts;
};
/// Stores the resolved operands (machine locations and constants) and
@@ -227,15 +230,15 @@ class TransferTracker {
/// Map from LocIdxes to which DebugVariables are based that location.
/// Mantained while stepping through the block. Not accurate if
/// VarLocs[Idx] != MTracker->LocIdxToIDNum[Idx].
- DenseMap<LocIdx, SmallSet<DebugVariable, 4>> ActiveMLocs;
+ DenseMap<LocIdx, SmallSet<DebugVariableID, 4>> ActiveMLocs;
/// Map from DebugVariable to it's current location and qualifying meta
/// information. To be used in conjunction with ActiveMLocs to construct
/// enough information for the DBG_VALUEs for a particular LocIdx.
- DenseMap<DebugVariable, ResolvedDbgValue> ActiveVLocs;
+ DenseMap<DebugVariableID, ResolvedDbgValue> ActiveVLocs;
/// Temporary cache of DBG_VALUEs to be entered into the Transfers collection.
- SmallVector<MachineInstr *, 4> PendingDbgValues;
+ SmallVector<std::pair<DebugVariableID, MachineInstr *>, 4> PendingDbgValues;
/// Record of a use-before-def: created when a value that's live-in to the
/// current block isn't available in any machine location, but it will be
@@ -244,12 +247,12 @@ class TransferTracker {
/// Value of this variable, def'd in block.
SmallVector<DbgOp> Values;
/// Identity of this variable.
- DebugVariable Var;
+ DebugVariableID VarID;
/// Additional variable properties.
DbgValueProperties Properties;
- UseBeforeDef(ArrayRef<DbgOp> Values, const DebugVariable &Var,
+ UseBeforeDef(ArrayRef<DbgOp> Values, DebugVariableID VarID,
const DbgValueProperties &Properties)
- : Values(Values.begin(), Values.end()), Var(Var),
+ : Values(Values.begin(), Values.end()), VarID(VarID),
Properties(Properties) {}
};
@@ -260,15 +263,16 @@ class TransferTracker {
/// The set of variables that are in UseBeforeDefs and can become a location
/// once the relevant value is defined. An element being erased from this
/// collection prevents the use-before-def materializing.
- DenseSet<DebugVariable> UseBeforeDefVariables;
+ DenseSet<DebugVariableID> UseBeforeDefVariables;
const TargetRegisterInfo &TRI;
const BitVector &CalleeSavedRegs;
TransferTracker(const TargetInstrInfo *TII, MLocTracker *MTracker,
- MachineFunction &MF, const TargetRegisterInfo &TRI,
+ MachineFunction &MF, const DebugVariableMap &DVMap,
+ const TargetRegisterInfo &TRI,
const BitVector &CalleeSavedRegs, const TargetPassConfig &TPC)
- : TII(TII), MTracker(MTracker), MF(MF), TRI(TRI),
+ : TII(TII), MTracker(MTracker), MF(MF), DVMap(DVMap), TRI(TRI),
CalleeSavedRegs(CalleeSavedRegs) {
TLI = MF.getSubtarget().getTargetLowering();
auto &TM = TPC.getTM<TargetMachine>();
@@ -345,7 +349,7 @@ class TransferTracker {
/// determine the values used by Value.
void loadVarInloc(MachineBasicBlock &MBB, DbgOpIDMap &DbgOpStore,
const DenseMap<ValueIDNum, LocationAndQuality> &ValueToLoc,
- DebugVariable Var, DbgValue Value) {
+ DebugVariableID VarID, DbgValue Value) {
SmallVector<DbgOp> DbgOps;
SmallVector<ResolvedDbgOp> ResolvedDbgOps;
bool IsValueValid = true;
@@ -386,7 +390,7 @@ class TransferTracker {
static_cast<unsigned>(Num.getInst()));
continue;
}
- recoverAsEntryValue(Var, Value.Properties, Num);
+ recoverAsEntryValue(VarID, Value.Properties, Num);
IsValueValid = false;
break;
}
@@ -404,8 +408,7 @@ class TransferTracker {
// Add UseBeforeDef entry for the last value to be defined in this block.
if (LastUseBeforeDef) {
- addUseBeforeDef(Var, Value.Properties, DbgOps,
- LastUseBeforeDef);
+ addUseBeforeDef(VarID, Value.Properties, DbgOps, LastUseBeforeDef);
return;
}
@@ -413,13 +416,15 @@ class TransferTracker {
// the transfer.
for (const ResolvedDbgOp &Op : ResolvedDbgOps)
if (!Op.IsConst)
- ActiveMLocs[Op.Loc].insert(Var);
+ ActiveMLocs[Op.Loc].insert(VarID);
auto NewValue = ResolvedDbgValue{ResolvedDbgOps, Value.Properties};
- auto Result = ActiveVLocs.insert(std::make_pair(Var, NewValue));
+ auto Result = ActiveVLocs.insert(std::make_pair(VarID, NewValue));
if (!Result.second)
Result.first->second = NewValue;
+ auto &[Var, DILoc] = DVMap.lookupDVID(VarID);
PendingDbgValues.push_back(
- MTracker->emitLoc(ResolvedDbgOps, Var, Value.Properties));
+ std::make_pair(VarID, &*MTracker->emitLoc(ResolvedDbgOps, Var, DILoc,
+ Value.Properties)));
}
/// Load object with live-in variable values. \p mlocs contains the live-in
@@ -430,7 +435,7 @@ class TransferTracker {
/// FIXME: could just examine mloctracker instead of passing in \p mlocs?
void
loadInlocs(MachineBasicBlock &MBB, ValueTable &MLocs, DbgOpIDMap &DbgOpStore,
- const SmallVectorImpl<std::pair<DebugVariable, DbgValue>> &VLocs,
+ const SmallVectorImpl<std::pair<DebugVariableID, DbgValue>> &VLocs,
unsigned NumLocs) {
ActiveMLocs.clear();
ActiveVLocs.clear();
@@ -486,11 +491,11 @@ class TransferTracker {
/// Record that \p Var has value \p ID, a value that becomes available
/// later in the function.
- void addUseBeforeDef(const DebugVariable &Var,
+ void addUseBeforeDef(DebugVariableID VarID,
const DbgValueProperties &Properties,
const SmallVectorImpl<DbgOp> &DbgOps, unsigned Inst) {
- UseBeforeDefs[Inst].emplace_back(DbgOps, Var, Properties);
- UseBeforeDefVariables.insert(Var);
+ UseBeforeDefs[Inst].emplace_back(DbgOps, VarID, Properties);
+ UseBeforeDefVariables.insert(VarID);
}
/// After the instruction at index \p Inst and position \p pos has been
@@ -509,7 +514,7 @@ class TransferTracker {
// Populate ValueToLoc with illegal default mappings for every value used by
// any UseBeforeDef variables for this instruction.
for (auto &Use : MIt->second) {
- if (!UseBeforeDefVariables.count(Use.Var))
+ if (!UseBeforeDefVariables.count(Use.VarID))
continue;
for (DbgOp &Op : Use.Values) {
@@ -548,7 +553,7 @@ class TransferTracker {
// Using the map of values to locations, produce a final set of values for
// this variable.
for (auto &Use : MIt->second) {
- if (!UseBeforeDefVariables.count(Use.Var))
+ if (!UseBeforeDefVariables.count(Use.VarID))
continue;
SmallVector<ResolvedDbgOp> DbgOps;
@@ -571,8 +576,9 @@ class TransferTracker {
continue;
// Otherwise, we're good to go.
- PendingDbgValues.push_back(
- MTracker->emitLoc(DbgOps, Use.Var, Use.Properties));
+ auto &[Var, DILoc] = DVMap.lookupDVID(Use.VarID);
+ PendingDbgValues.push_back(std::make_pair(
+ Use.VarID, MTracker->emitLoc(DbgOps, Var, DILoc, Use.Properties)));
}
flushDbgValues(pos, nullptr);
}
@@ -622,7 +628,7 @@ class TransferTracker {
return Reg != SP && Reg != FP;
}
- bool recoverAsEntryValue(const DebugVariable &Var,
+ bool recoverAsEntryValue(DebugVariableID VarID,
const DbgValueProperties &Prop,
const ValueIDNum &Num) {
// Is this variable location a candidate to be an entry value. First,
@@ -643,6 +649,8 @@ class TransferTracker {
DIExpr = *NonVariadicExpression;
}
+ auto &[Var, DILoc] = DVMap.lookupDVID(VarID);
+
// Is the variable appropriate for entry values (i.e., is a parameter).
if (!isEntryValueVariable(Var, DIExpr))
return false;
@@ -656,9 +664,8 @@ class TransferTracker {
DIExpression::prepend(DIExpr, DIExpression::EntryValue);
Register Reg = MTracker->LocIdxToLocID[Num.getLoc()];
MachineOperand MO = MachineOperand::CreateReg(Reg, false);
-
- PendingDbgValues.push_back(
- emitMOLoc(MO, Var, {NewExpr, Prop.Indirect, false}));
+ PendingDbgValues.push_back(std::make_pair(
+ VarID, &*emitMOLoc(MO, Var, {NewExpr, Prop.Indirect, false})));
return true;
}
@@ -667,19 +674,20 @@ class TransferTracker {
DebugVariable Var(MI.getDebugVariable(), MI.getDebugExpression(),
MI.getDebugLoc()->getInlinedAt());
DbgValueProperties Properties(MI);
+ DebugVariableID VarID = DVMap.getDVID(Var);
// Ignore non-register locations, we don't transfer those.
if (MI.isUndefDebugValue() ||
all_of(MI.debug_operands(),
[](const MachineOperand &MO) { return !MO.isReg(); })) {
- auto It = ActiveVLocs.find(Var);
+ auto It = ActiveVLocs.find(VarID);
if (It != ActiveVLocs.end()) {
for (LocIdx Loc : It->second.loc_indices())
- ActiveMLocs[Loc].erase(Var);
+ ActiveMLocs[Loc].erase(VarID);
ActiveVLocs.erase(It);
}
// Any use-before-defs no longer apply.
- UseBeforeDefVariables.erase(Var);
+ UseBeforeDefVariables.erase(VarID);
return;
}
@@ -705,14 +713,15 @@ class TransferTracker {
SmallVectorImpl<ResolvedDbgOp> &NewLocs) {
DebugVariable Var(MI.getDebugVariable(), MI.getDebugExpression(),
MI.getDebugLoc()->getInlinedAt());
+ DebugVariableID VarID = DVMap.getDVID(Var);
// Any use-before-defs no longer apply.
- UseBeforeDefVariables.erase(Var);
+ UseBeforeDefVariables.erase(VarID);
// Erase any previous location.
- auto It = ActiveVLocs.find(Var);
+ auto It = ActiveVLocs.find(VarID);
if (It != ActiveVLocs.end()) {
for (LocIdx Loc : It->second.loc_indices())
- ActiveMLocs[Loc].erase(Var);
+ ActiveMLocs[Loc].erase(VarID);
}
// If there _is_ no new location, all we had to do was erase.
@@ -722,7 +731,7 @@ class TransferTracker {
return;
}
- SmallVector<std::pair<LocIdx, DebugVariable>> LostMLocs;
+ SmallVector<std::pair<LocIdx, DebugVariableID>> LostMLocs;
for (ResolvedDbgOp &Op : NewLocs) {
if (Op.IsConst)
continue;
@@ -749,17 +758,17 @@ class TransferTracker {
for (const auto &LostMLoc : LostMLocs)
ActiveMLocs[LostMLoc.first].erase(LostMLoc.second);
LostMLocs.clear();
- It = ActiveVLocs.find(Var);
+ It = ActiveVLocs.find(VarID);
ActiveMLocs[NewLoc.asU64()].clear();
VarLocs[NewLoc.asU64()] = MTracker->readMLoc(NewLoc);
}
- ActiveMLocs[NewLoc].insert(Var);
+ ActiveMLocs[NewLoc].insert(VarID);
}
if (It == ActiveVLocs.end()) {
ActiveVLocs.insert(
- std::make_pair(Var, ResolvedDbgValue(NewLocs, Properties)));
+ std::make_pair(VarID, ResolvedDbgValue(NewLocs, Properties)));
} else {
It->second.Ops.assign(NewLocs);
It->second.Properties = Properties;
@@ -802,21 +811,21 @@ class TransferTracker {
// explicitly undef, then stop here.
if (!NewLoc && !MakeUndef) {
// Try and recover a few more locations with entry values.
- for (const auto &Var : ActiveMLocIt->second) {
- auto &Prop = ActiveVLocs.find(Var)->second.Properties;
- recoverAsEntryValue(Var, Prop, OldValue);
+ for (DebugVariableID VarID : ActiveMLocIt->second) {
+ auto &Prop = ActiveVLocs.find(VarID)->second.Properties;
+ recoverAsEntryValue(VarID, Prop, OldValue);
}
flushDbgValues(Pos, nullptr);
return;
}
// Examine all the variables based on this location.
- DenseSet<DebugVariable> NewMLocs;
+ DenseSet<DebugVariableID> NewMLocs;
// If no new location has been found, every variable that depends on this
// MLoc is dead, so end their existing MLoc->Var mappings as well.
- SmallVector<std::pair<LocIdx, DebugVariable>> LostMLocs;
- for (const auto &Var : ActiveMLocIt->second) {
- auto ActiveVLocIt = ActiveVLocs.find(Var);
+ SmallVector<std::pair<LocIdx, DebugVariableID>> LostMLocs;
+ for (DebugVariableID VarID : ActiveMLocIt->second) {
+ auto ActiveVLocIt = ActiveVLocs.find(VarID);
// Re-state the variable location: if there's no replacement then NewLoc
// is std::nullopt and a $noreg DBG_VALUE will be created. Otherwise, a
// DBG_VALUE identifying the alternative location will be emitted.
@@ -835,19 +844,21 @@ class TransferTracker {
replace_copy(ActiveVLocIt->second.Ops, DbgOps.begin(), OldOp, NewOp);
}
- PendingDbgValues.push_back(MTracker->emitLoc(DbgOps, Var, Properties));
+ auto &[Var, DILoc] = DVMap.lookupDVID(VarID);
+ PendingDbgValues.push_back(std::make_pair(
+ VarID, &*MTracker->emitLoc(DbgOps, Var, DILoc, Properties)));
// Update machine locations <=> variable locations maps. Defer updating
// ActiveMLocs to avoid invalidating the ActiveMLocIt iterator.
if (!NewLoc) {
for (LocIdx Loc : ActiveVLocIt->second.loc_indices()) {
if (Loc != MLoc)
- LostMLocs.emplace_back(Loc, Var);
+ LostMLocs.emplace_back(Loc, VarID);
}
ActiveVLocs.erase(ActiveVLocIt);
} else {
ActiveVLocIt->second.Ops = DbgOps;
- NewMLocs.insert(Var);
+ NewMLocs.insert(VarID);
}
}
@@ -871,8 +882,8 @@ class TransferTracker {
// Commit ActiveMLoc changes.
ActiveMLocIt->second.clear();
if (!NewMLocs.empty())
- for (auto &Var : NewMLocs)
- ActiveMLocs[*NewLoc].insert(Var);
+ for (DebugVariableID VarID : NewMLocs)
+ ActiveMLocs[*NewLoc].insert(VarID);
}
/// Transfer variables based on \p Src to be based on \p Dst. This handles
@@ -895,17 +906,18 @@ class TransferTracker {
// For each variable based on Src; create a location at Dst.
ResolvedDbgOp SrcOp(Src);
ResolvedDbgOp DstOp(Dst);
- for (const auto &Var : MovingVars) {
- auto ActiveVLocIt = ActiveVLocs.find(Var);
+ for (DebugVariableID VarID : MovingVars) {
+ auto ActiveVLocIt = ActiveVLocs.find(VarID);
assert(ActiveVLocIt != ActiveVLocs.end());
// Update all instances of Src in the variable's tracked values to Dst.
std::replace(ActiveVLocIt->second.Ops.begin(),
ActiveVLocIt->second.Ops.end(), SrcOp, DstOp);
- MachineInstr *MI = MTracker->emitLoc(ActiveVLocIt->second.Ops, Var,
+ auto &[Var, DILoc] = DVMap.lookupDVID(VarID);
+ MachineInstr *MI = MTracker->emitLoc(ActiveVLocIt->second.Ops, Var, DILoc,
ActiveVLocIt->second.Properties);
- PendingDbgValues.push_back(MI);
+ PendingDbgValues.push_back(std::make_pair(VarID, MI));
}
ActiveMLocs[Src].clear();
flushDbgValues(Pos, nullptr);
@@ -1156,11 +1168,9 @@ LLVM_DUMP_METHOD void MLocTracker::dump_mloc_map() {
MachineInstrBuilder
MLocTracker::emitLoc(const SmallVectorImpl<ResolvedDbgOp> &DbgOps,
- const DebugVariable &Var,
+ const DebugVariable &Var, const DILocation *DILoc,
const DbgValueProperties &Properties) {
- DebugLoc DL = DILocation::get(Var.getVariable()->getContext(), 0, 0,
- Var.getVariable()->getScope(),
- const_cast<DILocation *>(Var.getInlinedAt()));
+ DebugLoc DL = DebugLoc(DILoc);
const MCInstrDesc &Desc = Properties.IsVariadic
? TII.get(TargetOpcode::DBG_VALUE_LIST)
@@ -1706,7 +1716,8 @@ bool InstrRefBasedLDV::transferDebugInstrRef(MachineInstr &MI,
LastUseBeforeDef = std::max(LastUseBeforeDef, NewID.getInst());
}
if (IsValidUseBeforeDef) {
- TTracker->addUseBeforeDef(V, {MI.getDebugExpression(), false, true},
+ DebugVariableID VID = DVMap.insertDVID(V, MI.getDebugLoc().get());
+ TTracker->addUseBeforeDef(VID, {MI.getDebugExpression(), false, true},
DbgOps, LastUseBeforeDef);
}
}
@@ -1715,9 +1726,11 @@ bool InstrRefBasedLDV::transferDebugInstrRef(MachineInstr &MI,
// This DBG_VALUE is potentially a $noreg / undefined location, if
// FoundLoc is illegal.
// (XXX -- could morph the DBG_INSTR_REF in the future).
- MachineInstr *DbgMI = MTracker->emitLoc(NewLocs, V, Properties);
+ MachineInstr *DbgMI =
+ MTracker->emitLoc(NewLocs, V, MI.getDebugLoc().get(), Properties);
+ DebugVariableID ID = DVMap.getDVID(V);
- TTracker->PendingDbgValues.push_back(DbgMI);
+ TTracker->PendingDbgValues.push_back(std::make_pair(ID, DbgMI));
TTracker->flushDbgValues(MI.getIterator(), nullptr);
return true;
}
@@ -3092,7 +3105,8 @@ void InstrRefBasedLDV::getBlocksForScope(
}
void InstrRefBasedLDV::buildVLocValueMap(
- const DILocation *DILoc, const SmallSet<DebugVariable, 4> &VarsWeCareAbout,
+ const DILocation *DILoc,
+ const SmallSet<DebugVariableID, 4> &VarsWeCareAbout,
SmallPtrSetImpl<MachineBasicBlock *> &AssignBlocks, LiveInsT &Output,
FuncValueTable &MOutLocs, FuncValueTable &MInLocs,
SmallVectorImpl<VLocTracker> &AllTheVLocs) {
@@ -3167,7 +3181,7 @@ void InstrRefBasedLDV::buildVLocValueMap(
// between blocks. This keeps the locality of working on one lexical scope at
// at time, but avoids re-processing variable values because some other
// variable has been assigned.
- for (const auto &Var : VarsWeCareAbout) {
+ for (DebugVariableID VarID : VarsWeCareAbout) {
// Re-initialize live-ins and live-outs, to clear the remains of previous
// variables live-ins / live-outs.
for (unsigned int I = 0; I < NumBlocks; ++I) {
@@ -3181,7 +3195,7 @@ void InstrRefBasedLDV::buildVLocValueMap(
SmallPtrSet<MachineBasicBlock *, 32> DefBlocks;
for (const MachineBasicBlock *ExpMBB : BlocksToExplore) {
auto &TransferFunc = AllTheVLocs[ExpMBB->getNumber()].Vars;
- if (TransferFunc.contains(Var))
+ if (TransferFunc.contains(VarID))
DefBlocks.insert(const_cast<MachineBasicBlock *>(ExpMBB));
}
@@ -3191,7 +3205,7 @@ void InstrRefBasedLDV::buildVLocValueMap(
// only one value definition, things are very simple.
if (DefBlocks.size() == 1) {
placePHIsForSingleVarDefinition(MutBlocksToExplore, *DefBlocks.begin(),
- AllTheVLocs, Var, Output);
+ AllTheVLocs, VarID, Output);
continue;
}
@@ -3264,7 +3278,7 @@ void InstrRefBasedLDV::buildVLocValueMap(
// Do transfer function.
auto &VTracker = AllTheVLocs[MBB->getNumber()];
- auto TransferIt = VTracker.Vars.find(Var);
+ auto TransferIt = VTracker.Vars.find(VarID);
if (TransferIt != VTracker.Vars.end()) {
// Erase on empty transfer (DBG_VALUE $noreg).
if (TransferIt->second.Kind == DbgValue::Undef) {
@@ -3326,9 +3340,11 @@ void InstrRefBasedLDV::buildVLocValueMap(
continue;
if (BlockLiveIn->Kind == DbgValue::VPHI)
BlockLiveIn->Kind = DbgValue::Def;
+ auto &[Var, DILoc] = DVMap.lookupDVID(VarID);
assert(BlockLiveIn->Properties.DIExpr->getFragmentInfo() ==
- Var.getFragment() && "Fragment info missing during value prop");
- Output[MBB->getNumber()].push_back(std::make_pair(Var, *BlockLiveIn));
+ Var.getFragment() &&
+ "Fragment info missing during value prop");
+ Output[MBB->getNumber()].push_back(std::make_pair(VarID, *BlockLiveIn));
}
} // Per-variable loop.
@@ -3339,7 +3355,7 @@ void InstrRefBasedLDV::buildVLocValueMap(
void InstrRefBasedLDV::placePHIsForSingleVarDefinition(
const SmallPtrSetImpl<MachineBasicBlock *> &InScopeBlocks,
MachineBasicBlock *AssignMBB, SmallVectorImpl<VLocTracker> &AllTheVLocs,
- const DebugVariable &Var, LiveInsT &Output) {
+ DebugVariableID VarID, LiveInsT &Output) {
// If there is a single definition of the variable, then working out it's
// value everywhere is very simple: it's every block dominated by the
// definition. At the dominance frontier, the usual algorithm would:
@@ -3352,7 +3368,7 @@ void InstrRefBasedLDV::placePHIsForSingleVarDefinition(
// Pick out the variables value from the block transfer function.
VLocTracker &VLocs = AllTheVLocs[AssignMBB->getNumber()];
- auto ValueIt = VLocs.Vars.find(Var);
+ auto ValueIt = VLocs.Vars.find(VarID);
const DbgValue &Value = ValueIt->second;
// If it's an explicit assignment of "undef", that means there is no location
@@ -3367,7 +3383,7 @@ void InstrRefBasedLDV::placePHIsForSingleVarDefinition(
if (!DomTree->properlyDominates(AssignMBB, ScopeBlock))
continue;
- Output[ScopeBlock->getNumber()].push_back({Var, Value});
+ Output[ScopeBlock->getNumber()].push_back({VarID, Value});
}
// All blocks that aren't dominated have no live-in value, thus no variable
@@ -3486,9 +3502,9 @@ bool InstrRefBasedLDV::depthFirstVLocAndEmit(
const ScopeToVarsT &ScopeToVars, ScopeToAssignBlocksT &ScopeToAssignBlocks,
LiveInsT &Output, FuncValueTable &MOutLocs, FuncValueTable &MInLocs,
SmallVectorImpl<VLocTracker> &AllTheVLocs, MachineFunction &MF,
- DenseMap<DebugVariable, unsigned> &AllVarsNumbering,
const TargetPassConfig &TPC) {
- TTracker = new TransferTracker(TII, MTracker, MF, *TRI, CalleeSavedRegs, TPC);
+ TTracker =
+ new TransferTracker(TII, MTracker, MF, DVMap, *TRI, CalleeSavedRegs, TPC);
unsigned NumLocs = MTracker->getNumLocs();
VTracker = nullptr;
@@ -3593,31 +3609,24 @@ bool InstrRefBasedLDV::depthFirstVLocAndEmit(
if (MInLocs.hasTableFor(*MBB))
EjectBlock(*MBB);
- return emitTransfers(AllVarsNumbering);
+ return emitTransfers();
}
-bool InstrRefBasedLDV::emitTransfers(
- DenseMap<DebugVariable, unsigned> &AllVarsNumbering) {
+bool InstrRefBasedLDV::emitTransfers() {
// Go through all the transfers recorded in the TransferTracker -- this is
// both the live-ins to a block, and any movements of values that happen
// in the middle.
- for (const auto &P : TTracker->Transfers) {
+ for (auto &P : TTracker->Transfers) {
// We have to insert DBG_VALUEs in a consistent order, otherwise they
// appear in DWARF in different orders. Use the order that they appear
// when walking through each block / each instruction, stored in
- // AllVarsNumbering.
- SmallVector<std::pair<unsigned, MachineInstr *>> Insts;
- for (MachineInstr *MI : P.Insts) {
- DebugVariable Var(MI->getDebugVariable(), MI->getDebugExpression(),
- MI->getDebugLoc()->getInlinedAt());
- Insts.emplace_back(AllVarsNumbering.find(Var)->second, MI);
- }
- llvm::sort(Insts, llvm::less_first());
+ // DVMap.
+ llvm::sort(P.Insts, llvm::less_first());
// Insert either before or after the designated point...
if (P.MBB) {
MachineBasicBlock &MBB = *P.MBB;
- for (const auto &Pair : Insts)
+ for (const auto &Pair : P.Insts)
MBB.insert(P.Pos, Pair.second);
} else {
// Terminators, like tail calls, can clobber things. Don't try and place
@@ -3626,7 +3635,7 @@ bool InstrRefBasedLDV::emitTransfers(
continue;
MachineBasicBlock &MBB = *P.Pos->getParent();
- for (const auto &Pair : Insts)
+ for (const auto &Pair : P.Insts)
MBB.insertAfterBundle(P.Pos, Pair.second);
}
}
@@ -3681,7 +3690,7 @@ bool InstrRefBasedLDV::ExtendRanges(MachineFunction &MF,
initialSetup(MF);
MLocTransfer.resize(MaxNumBlocks);
- vlocs.resize(MaxNumBlocks, VLocTracker(OverlapFragments, EmptyExpr));
+ vlocs.resize(MaxNumBlocks, VLocTracker(DVMap, OverlapFragments, EmptyExpr));
SavedLiveIns.resize(MaxNumBlocks);
produceMLocTransferFunction(MF, MLocTransfer, MaxNumBlocks);
@@ -3738,10 +3747,6 @@ bool InstrRefBasedLDV::ExtendRanges(MachineFunction &MF,
MTracker->reset();
}
- // Number all variables in the order that they appear, to be used as a stable
- // insertion order later.
- DenseMap<DebugVariable, unsigned> AllVarsNumbering;
-
// Map from one LexicalScope to all the variables in that scope.
ScopeToVarsT ScopeToVars;
@@ -3760,16 +3765,15 @@ bool InstrRefBasedLDV::ExtendRanges(MachineFunction &MF,
auto *VTracker = &vlocs[MBB->getNumber()];
// Collect each variable with a DBG_VALUE in this block.
for (auto &idx : VTracker->Vars) {
- const auto &Var = idx.first;
- const DILocation *ScopeLoc = VTracker->Scopes[Var];
+ DebugVariableID VarID = idx.first;
+ const DILocation *ScopeLoc = VTracker->Scopes[VarID];
assert(ScopeLoc != nullptr);
auto *Scope = LS.findLexicalScope(ScopeLoc);
// No insts in scope -> shouldn't have been recorded.
assert(Scope != nullptr);
- AllVarsNumbering.insert(std::make_pair(Var, AllVarsNumbering.size()));
- ScopeToVars[Scope].insert(Var);
+ ScopeToVars[Scope].insert(VarID);
ScopeToAssignBlocks[Scope].insert(VTracker->MBB);
ScopeToDILocation[Scope] = ScopeLoc;
++VarAssignCount;
@@ -3793,7 +3797,7 @@ bool InstrRefBasedLDV::ExtendRanges(MachineFunction &MF,
// the "else" block of this condition.
Changed = depthFirstVLocAndEmit(
MaxNumBlocks, ScopeToDILocation, ScopeToVars, ScopeToAssignBlocks,
- SavedLiveIns, MOutLocs, MInLocs, vlocs, MF, AllVarsNumbering, *TPC);
+ SavedLiveIns, MOutLocs, MInLocs, vlocs, MF, *TPC);
}
delete MTracker;
@@ -3812,6 +3816,7 @@ bool InstrRefBasedLDV::ExtendRanges(MachineFunction &MF,
SeenFragments.clear();
SeenDbgPHIs.clear();
DbgOpStore.clear();
+ DVMap.clear();
return Changed;
}
diff --git a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h
index 6d77a6972f09b..5417e2a59cb25 100644
--- a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h
+++ b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h
@@ -35,6 +35,44 @@ class DbgOpIDMap;
using namespace llvm;
+using DebugVariableID = unsigned;
+using VarAndLoc = std::pair<DebugVariable, const DILocation *>;
+
+/// Mapping from DebugVariable to/from a unique identifying number. Each
+/// DebugVariable consists of three pointers, and after a small amount of
+/// work to identify overlapping fragments of variables we mostly only use
+/// DebugVariables as identities of variables. It's much more compile-time
+/// efficient to use an ID number instead, which this class provides.
+class DebugVariableMap {
+ DenseMap<DebugVariable, unsigned> VarToIdx;
+ SmallVector<VarAndLoc> IdxToVar;
+
+public:
+ DebugVariableID getDVID(const DebugVariable &Var) const {
+ auto It = VarToIdx.find(Var);
+ assert(It != VarToIdx.end());
+ return It->second;
+ }
+
+ DebugVariableID insertDVID(DebugVariable &Var, const DILocation *Loc) {
+ unsigned Size = VarToIdx.size();
+ auto ItPair = VarToIdx.insert({Var, Size});
+ if (ItPair.second) {
+ IdxToVar.push_back({Var, Loc});
+ return Size;
+ }
+
+ return ItPair.first->second;
+ }
+
+ const VarAndLoc &lookupDVID(DebugVariableID ID) const { return IdxToVar[ID]; }
+
+ void clear() {
+ VarToIdx.clear();
+ IdxToVar.clear();
+ }
+};
+
/// Handle-class for a particular "location". This value-type uniquely
/// symbolises a register or stack location, allowing manipulation of locations
/// without concern for where that location is. Practically, this allows us to
@@ -985,7 +1023,7 @@ class MLocTracker {
/// information in \pProperties, for variable Var. Don't insert it anywhere,
/// just return the builder for it.
MachineInstrBuilder emitLoc(const SmallVectorImpl<ResolvedDbgOp> &DbgOps,
- const DebugVariable &Var,
+ const DebugVariable &Var, const DILocation *DILoc,
const DbgValueProperties &Properties);
};
@@ -1003,38 +1041,44 @@ using OverlapMap =
/// identified.
class VLocTracker {
public:
+ /// Ref to function-wide map of DebugVariable <=> ID-numbers.
+ DebugVariableMap &DVMap;
/// Map DebugVariable to the latest Value it's defined to have.
/// Needs to be a MapVector because we determine order-in-the-input-MIR from
- /// the order in this container.
+ /// the order in this container. (FIXME: this is less true now).
/// We only retain the last DbgValue in each block for each variable, to
/// determine the blocks live-out variable value. The Vars container forms the
/// transfer function for this block, as part of the dataflow analysis. The
/// movement of values between locations inside of a block is handled at a
/// much later stage, in the TransferTracker class.
- MapVector<DebugVariable, DbgValue> Vars;
- SmallDenseMap<DebugVariable, const DILocation *, 8> Scopes;
+ MapVector<DebugVariableID, DbgValue> Vars;
+ SmallDenseMap<DebugVariableID, const DILocation *, 8> Scopes;
MachineBasicBlock *MBB = nullptr;
const OverlapMap &OverlappingFragments;
DbgValueProperties EmptyProperties;
public:
- VLocTracker(const OverlapMap &O, const DIExpression *EmptyExpr)
- : OverlappingFragments(O), EmptyProperties(EmptyExpr, false, false) {}
+ VLocTracker(DebugVariableMap &DVMap, const OverlapMap &O,
+ const DIExpression *EmptyExpr)
+ : DVMap(DVMap), OverlappingFragments(O),
+ EmptyProperties(EmptyExpr, false, false) {}
void defVar(const MachineInstr &MI, const DbgValueProperties &Properties,
const SmallVectorImpl<DbgOpID> &DebugOps) {
assert(MI.isDebugValueLike());
DebugVariable Var(MI.getDebugVariable(), MI.getDebugExpression(),
MI.getDebugLoc()->getInlinedAt());
+ // Either insert or fetch an ID number for this variable.
+ DebugVariableID VarID = DVMap.insertDVID(Var, MI.getDebugLoc().get());
DbgValue Rec = (DebugOps.size() > 0)
? DbgValue(DebugOps, Properties)
: DbgValue(Properties, DbgValue::Undef);
// Attempt insertion; overwrite if it's already mapped.
- auto Result = Vars.insert(std::make_pair(Var, Rec));
+ auto Result = Vars.insert(std::make_pair(VarID, Rec));
if (!Result.second)
Result.first->second = Rec;
- Scopes[Var] = MI.getDebugLoc().get();
+ Scopes[VarID] = MI.getDebugLoc().get();
considerOverlaps(Var, MI.getDebugLoc().get());
}
@@ -1056,13 +1100,15 @@ class VLocTracker {
DebugVariable Overlapped(Var.getVariable(), OptFragmentInfo,
Var.getInlinedAt());
+ // Produce an ID number for this overlapping fragment of a variable.
+ DebugVariableID OverlappedID = DVMap.insertDVID(Overlapped, Loc);
DbgValue Rec = DbgValue(EmptyProperties, DbgValue::Undef);
// Attempt insertion; overwrite if it's already mapped.
- auto Result = Vars.insert(std::make_pair(Overlapped, Rec));
+ auto Result = Vars.insert(std::make_pair(OverlappedID, Rec));
if (!Result.second)
Result.first->second = Rec;
- Scopes[Overlapped] = Loc;
+ Scopes[OverlappedID] = Loc;
}
}
@@ -1093,7 +1139,7 @@ class InstrRefBasedLDV : public LDVImpl {
/// variables to their values.
using LiveIdxT = DenseMap<const MachineBasicBlock *, DbgValue *>;
- using VarAndLoc = std::pair<DebugVariable, DbgValue>;
+ using VarAndLoc = std::pair<DebugVariableID, DbgValue>;
/// Type for a live-in value: the predecessor block, and its value.
using InValueT = std::pair<MachineBasicBlock *, DbgValue *>;
@@ -1106,7 +1152,8 @@ class InstrRefBasedLDV : public LDVImpl {
using ScopeToDILocT = DenseMap<const LexicalScope *, const DILocation *>;
/// Mapping from lexical scopes to variables in that scope.
- using ScopeToVarsT = DenseMap<const LexicalScope *, SmallSet<DebugVariable, 4>>;
+ using ScopeToVarsT =
+ DenseMap<const LexicalScope *, SmallSet<DebugVariableID, 4>>;
/// Mapping from lexical scopes to blocks where variables in that scope are
/// assigned. Such blocks aren't necessarily "in" the lexical scope, it's
@@ -1200,6 +1247,11 @@ class InstrRefBasedLDV : public LDVImpl {
DbgOpIDMap DbgOpStore;
+ /// Mapping between DebugVariables and unique ID numbers. This is a more
+ /// efficient way to represent the identity of a variable, versus a plain
+ /// DebugVariable.
+ DebugVariableMap DVMap;
+
/// True if we need to examine call instructions for stack clobbers. We
/// normally assume that they don't clobber SP, but stack probes on Windows
/// do.
@@ -1330,9 +1382,9 @@ class InstrRefBasedLDV : public LDVImpl {
/// performance as it doesn't have to find the dominance frontier between
/// different assignments.
void placePHIsForSingleVarDefinition(
- const SmallPtrSetImpl<MachineBasicBlock *> &InScopeBlocks,
- MachineBasicBlock *MBB, SmallVectorImpl<VLocTracker> &AllTheVLocs,
- const DebugVariable &Var, LiveInsT &Output);
+ const SmallPtrSetImpl<MachineBasicBlock *> &InScopeBlocks,
+ MachineBasicBlock *MBB, SmallVectorImpl<VLocTracker> &AllTheVLocs,
+ DebugVariableID Var, LiveInsT &Output);
/// Calculate the iterated-dominance-frontier for a set of defs, using the
/// existing LLVM facilities for this. Works for a single "value" or
@@ -1381,7 +1433,7 @@ class InstrRefBasedLDV : public LDVImpl {
/// scope, but which do contain DBG_VALUEs, which VarLocBasedImpl tracks
/// locations through.
void buildVLocValueMap(const DILocation *DILoc,
- const SmallSet<DebugVariable, 4> &VarsWeCareAbout,
+ const SmallSet<DebugVariableID, 4> &VarsWeCareAbout,
SmallPtrSetImpl<MachineBasicBlock *> &AssignBlocks,
LiveInsT &Output, FuncValueTable &MOutLocs,
FuncValueTable &MInLocs,
@@ -1414,10 +1466,8 @@ class InstrRefBasedLDV : public LDVImpl {
const SmallVectorImpl<const MachineBasicBlock *> &BlockOrders);
/// Take collections of DBG_VALUE instructions stored in TTracker, and
- /// install them into their output blocks. Preserves a stable order of
- /// DBG_VALUEs produced (which would otherwise cause nondeterminism) through
- /// the AllVarsNumbering order.
- bool emitTransfers(DenseMap<DebugVariable, unsigned> &AllVarsNumbering);
+ /// install them into their output blocks.
+ bool emitTransfers();
/// Boilerplate computation of some initial sets, artifical blocks and
/// RPOT block ordering.
@@ -1437,13 +1487,14 @@ class InstrRefBasedLDV : public LDVImpl {
/// block information can be fully computed before exploration finishes,
/// allowing us to emit it and free data structures earlier than otherwise.
/// It's also good for locality.
- bool depthFirstVLocAndEmit(
- unsigned MaxNumBlocks, const ScopeToDILocT &ScopeToDILocation,
- const ScopeToVarsT &ScopeToVars, ScopeToAssignBlocksT &ScopeToBlocks,
- LiveInsT &Output, FuncValueTable &MOutLocs, FuncValueTable &MInLocs,
- SmallVectorImpl<VLocTracker> &AllTheVLocs, MachineFunction &MF,
- DenseMap<DebugVariable, unsigned> &AllVarsNumbering,
- const TargetPassConfig &TPC);
+ bool depthFirstVLocAndEmit(unsigned MaxNumBlocks,
+ const ScopeToDILocT &ScopeToDILocation,
+ const ScopeToVarsT &ScopeToVars,
+ ScopeToAssignBlocksT &ScopeToBlocks,
+ LiveInsT &Output, FuncValueTable &MOutLocs,
+ FuncValueTable &MInLocs,
+ SmallVectorImpl<VLocTracker> &AllTheVLocs,
+ MachineFunction &MF, const TargetPassConfig &TPC);
bool ExtendRanges(MachineFunction &MF, MachineDominatorTree *DomTree,
TargetPassConfig *TPC, unsigned InputBBLimit,
diff --git a/llvm/test/DebugInfo/MIR/X86/live-debug-values-fragments.mir b/llvm/test/DebugInfo/MIR/X86/live-debug-values-fragments.mir
index b54c748ac9e84..637d11838df0e 100644
--- a/llvm/test/DebugInfo/MIR/X86/live-debug-values-fragments.mir
+++ b/llvm/test/DebugInfo/MIR/X86/live-debug-values-fragments.mir
@@ -15,16 +15,16 @@
#
# CHECK-LABEL: foo
# CHECK-LABEL: bb.3.bb3:
-# CHECK: DBG_VALUE $ecx, $noreg, !{{[0-9]+}},
-# CHECK-SAME: !DIExpression(DW_OP_LLVM_fragment, 0, 32)
-# CHECK-NEXT: DBG_VALUE_LIST !{{[0-9]+}},
-# CHECK-SAME: !DIExpression({{[^)]+}}, DW_OP_LLVM_fragment, 0, 32)
-# CHECK-SAME: $ecx, $r8d
-# CHECK-NEXT: DBG_VALUE $ebx, $noreg, !{{[0-9]+}},
+# CHECK: DBG_VALUE $ebx, $noreg, !{{[0-9]+}},
# CHECK-SAME: !DIExpression(DW_OP_LLVM_fragment, 32, 32)
# CHECK-NEXT: DBG_VALUE_LIST !{{[0-9]+}},
# CHECK-SAME: !DIExpression({{[^)]+}}, DW_OP_LLVM_fragment, 32, 32)
# CHECK-SAME: $ebx, $r10d
+# CHECK-NEXT: DBG_VALUE $ecx, $noreg, !{{[0-9]+}},
+# CHECK-SAME: !DIExpression(DW_OP_LLVM_fragment, 0, 32)
+# CHECK-NEXT: DBG_VALUE_LIST !{{[0-9]+}},
+# CHECK-SAME: !DIExpression({{[^)]+}}, DW_OP_LLVM_fragment, 0, 32)
+# CHECK-SAME: $ecx, $r8d
# CHECK-NEXT: XOR32rr
# CHECK-NEXT: RET64
#
diff --git a/llvm/unittests/CodeGen/InstrRefLDVTest.cpp b/llvm/unittests/CodeGen/InstrRefLDVTest.cpp
index 306a97c3149cc..50a8cb97ae061 100644
--- a/llvm/unittests/CodeGen/InstrRefLDVTest.cpp
+++ b/llvm/unittests/CodeGen/InstrRefLDVTest.cpp
@@ -55,6 +55,7 @@ class InstrRefLDVTest : public testing::Test {
DIBasicType *LongInt;
DIExpression *EmptyExpr;
LiveDebugValues::OverlapMap Overlaps;
+ LiveDebugValues::DebugVariableMap DVMap;
DebugLoc OutermostLoc, InBlockLoc, NotNestedBlockLoc, InlinedLoc;
@@ -176,7 +177,7 @@ class InstrRefLDVTest : public testing::Test {
void addVTracker() {
ASSERT_TRUE(LDV);
- VTracker = std::make_unique<VLocTracker>(Overlaps, EmptyExpr);
+ VTracker = std::make_unique<VLocTracker>(DVMap, Overlaps, EmptyExpr);
LDV->VTracker = &*VTracker;
}
More information about the llvm-commits
mailing list