[llvm] [InstCombine] Avoid Allocating Arrays Too Large For the Target (PR #70980)

[Qiongsi Wu via llvm-commits]

================
@@ -210,14 +210,23 @@ static Instruction *simplifyAllocaArraySize(InstCombinerImpl &IC,
   if (const ConstantInt *C = dyn_cast<ConstantInt>(AI.getArraySize())) {
     if (C->getValue().getActiveBits() <= 64) {
       Type *NewTy = ArrayType::get(AI.getAllocatedType(), C->getZExtValue());
+
+      // Make sure we do not create an array type larger than pointers on the
+      // target can index.
+      unsigned MaxArrSizeBitWidth =
+          IC.getDataLayout().getPointerTypeSizeInBits(AI.getType());
+      APInt ArrayAllocSize(64, IC.getDataLayout().getTypeAllocSize(NewTy));
+      if (ArrayAllocSize.getActiveBits() > MaxArrSizeBitWidth)
+        NewTy = ArrayType::get(AI.getAllocatedType(), 0);
----------------
qiongsiwu wrote:

@arsenm Logic is update to avoid clamping at 0. Such large allocations are probably undefined behaviours as mentioned here (https://github.com/llvm/llvm-project/pull/70980#issuecomment-1792654827). I agree that doing nothing is less unexpected. This will leave the tests look inconsistent, but that seems fine since we are probably in UB territory anyways. 

https://github.com/llvm/llvm-project/pull/70980