[llvm] 655d857 - [SROA] `isVectorPromotionViable()`: avoid allowing overly large vectors

[Roman Lebedev via llvm-commits]

Author: Roman Lebedev
Date: 2022-11-23T03:23:08+03:00
New Revision: 655d85732536648803612814159c897c15469e47

URL: https://github.com/llvm/llvm-project/commit/655d85732536648803612814159c897c15469e47
DIFF: https://github.com/llvm/llvm-project/commit/655d85732536648803612814159c897c15469e47.diff

LOG: [SROA] `isVectorPromotionViable()`: avoid allowing overly large vectors

Otherwise, `compiler-rt/test/asan/TestCases/pr33372.cpp` fails with an assertion:
```
clang-16: /repositories/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:11988: void llvm::SelectionDAG::createOperands(llvm::SDNode *, ArrayRef<llvm::SDValue>): Assertion `SDNode::getMaxNumOperands() >= Vals.size() && "too many operands to fit into SDNode"' failed.
```
I'm not sure if this should be even more conservative,
or if we have a named constant for this in middle-end.

Added: 
    

Modified: 
    llvm/lib/Transforms/Scalar/SROA.cpp
    llvm/test/Transforms/SROA/basictest.ll

Removed: 
    


################################################################################
diff  --git a/llvm/lib/Transforms/Scalar/SROA.cpp b/llvm/lib/Transforms/Scalar/SROA.cpp
index 0fafa3af7636..09a445c236fa 100644
--- a/llvm/lib/Transforms/Scalar/SROA.cpp
+++ b/llvm/lib/Transforms/Scalar/SROA.cpp
@@ -1999,6 +1999,13 @@ static VectorType *isVectorPromotionViable(Partition &P, const DataLayout &DL) {
     CandidateTys.resize(1);
   }
 
+  // FIXME: hack. Do we have a named constant for this?
+  // SDAG SDNode can't have more than 65535 operands.
+  llvm::erase_if(CandidateTys, [](VectorType *VTy) {
+    return cast<FixedVectorType>(VTy)->getNumElements() >
+           std::numeric_limits<unsigned short>::max();
+  });
+
   for (VectorType *VTy : CandidateTys)
     if (checkVectorTypeForPromotion(P, VTy, DL))
       return VTy;

diff  --git a/llvm/test/Transforms/SROA/basictest.ll b/llvm/test/Transforms/SROA/basictest.ll
index 18874617e0b2..a95d84dc913b 100644
--- a/llvm/test/Transforms/SROA/basictest.ll
+++ b/llvm/test/Transforms/SROA/basictest.ll
@@ -1178,6 +1178,8 @@ define void @PR14465() {
 ; Ensure that we don't crash when analyzing a alloca larger than the maximum
 ; integer type width (MAX_INT_BITS) supported by llvm (1048576*32 > (1<<23)-1).
 ; CHECK-LABEL: @PR14465(
+; CHECK-NEXT:    [[STACK:%.*]] = alloca [1048576 x i32], align 16
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr align 16 [[STACK]], i8 -2, i64 4194304, i1 false)
 ; CHECK-NEXT:    ret void
 ;