[llvm] [InstCombine] Avoid Allocating Arrays Too Large For the Target (PR #70980)

[Qiongsi Wu via llvm-commits]

================
@@ -210,14 +210,23 @@ static Instruction *simplifyAllocaArraySize(InstCombinerImpl &IC,
   if (const ConstantInt *C = dyn_cast<ConstantInt>(AI.getArraySize())) {
     if (C->getValue().getActiveBits() <= 64) {
       Type *NewTy = ArrayType::get(AI.getAllocatedType(), C->getZExtValue());
+
+      // Make sure we do not create an array type larger than pointers on the
+      // target can index.
+      unsigned MaxArrSizeBitWidth =
+          IC.getDataLayout().getPointerTypeSizeInBits(AI.getType());
+      APInt ArrayAllocSize(64, IC.getDataLayout().getTypeAllocSize(NewTy));
+      if (ArrayAllocSize.getActiveBits() > MaxArrSizeBitWidth)
+        NewTy = ArrayType::get(AI.getAllocatedType(), 0);
----------------
qiongsiwu wrote:

The clamping behaviour actually comes from here in the test case https://github.com/llvm/llvm-project/pull/70980/files#diff-0809d302e300182ab600334691baa08c038e2c15f6d457f48d25a49245892240R12. In 32 bit mode, this existing test case clamps `i864 -4294967296` to 0. My first attempt to fix this did left the alloca unchanged, but then I saw the existing test case's behaviour and changed my mind. 

Could you elaborate why "clamping to 0" is certainly incorrect? Thanks so much!

Als

https://github.com/llvm/llvm-project/pull/70980