[llvm] r348324 - [SelectionDAG] Split very large token factors for loads into 64k chunks.
Amara Emerson via llvm-commits
llvm-commits at lists.llvm.org
Tue Dec 4 16:41:30 PST 2018
Author: aemerson
Date: Tue Dec 4 16:41:30 2018
New Revision: 348324
URL: http://llvm.org/viewvc/llvm-project?rev=348324&view=rev
Log:
[SelectionDAG] Split very large token factors for loads into 64k chunks.
There's a 64k limit on the number of SDNode operands, and some very large
functions with 64k or more loads can cause crashes due to this limit being hit
when a TokenFactor with this many operands is created. To fix this, create
sub-tokenfactors if we've exceeded the limit.
No test case as it requires a very large function.
rdar://45196621
Differential Revision: https://reviews.llvm.org/D55073
Modified:
llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
Modified: llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAG.cpp?rev=348324&r1=348323&r2=348324&view=diff
==============================================================================
--- llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAG.cpp (original)
+++ llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Tue Dec 4 16:41:30 2018
@@ -9031,8 +9031,11 @@ SDNode *SelectionDAG::isConstantFPBuildV
void SelectionDAG::createOperands(SDNode *Node, ArrayRef<SDValue> Vals) {
assert(!Node->OperandList && "Node already has operands");
+ assert(std::numeric_limits<decltype(SDNode::NumOperands)>::max() >
+ Vals.size() &&
+ "too many operands to fit into SDNode");
SDUse *Ops = OperandRecycler.allocate(
- ArrayRecycler<SDUse>::Capacity::get(Vals.size()), OperandAllocator);
+ ArrayRecycler<SDUse>::Capacity::get(Vals.size()), OperandAllocator);
bool IsDivergent = false;
for (unsigned I = 0; I != Vals.size(); ++I) {
Modified: llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp?rev=348324&r1=348323&r2=348324&view=diff
==============================================================================
--- llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp (original)
+++ llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp Tue Dec 4 16:41:30 2018
@@ -1032,8 +1032,19 @@ SDValue SelectionDAGBuilder::getRoot() {
}
// Otherwise, we have to make a token factor node.
- SDValue Root = DAG.getNode(ISD::TokenFactor, getCurSDLoc(), MVT::Other,
- PendingLoads);
+ // If we have >= 2^16 loads then split across multiple token factors as
+ // there's a 64k limit on the number of SDNode operands.
+ SDValue Root;
+ size_t Limit = (1 << 16) - 1;
+ while (PendingLoads.size() > Limit) {
+ unsigned SliceIdx = PendingLoads.size() - Limit;
+ auto ExtractedTFs = ArrayRef<SDValue>(PendingLoads).slice(SliceIdx, Limit);
+ SDValue NewTF =
+ DAG.getNode(ISD::TokenFactor, getCurSDLoc(), MVT::Other, ExtractedTFs);
+ PendingLoads.erase(PendingLoads.begin() + SliceIdx, PendingLoads.end());
+ PendingLoads.emplace_back(NewTF);
+ }
+ Root = DAG.getNode(ISD::TokenFactor, getCurSDLoc(), MVT::Other, PendingLoads);
PendingLoads.clear();
DAG.setRoot(Root);
return Root;
More information about the llvm-commits
mailing list