[llvm] [LoopVectorize] Ensure VPPredInstPHIRecipe only uses InsertElementInst (PR #159543)

[Abhinav Srivastava / August Radjoe via llvm-commits]

https://github.com/AbhinavMir created https://github.com/llvm/llvm-project/pull/159543

The loop vectorizer could generate VPlans where VPPredInstPHIRecipe received a non-InsertElement vector value, leading to an assertion failure:

  opt: Casting.h:578: decltype(auto) llvm::cast<InsertElementInst>(...)
  Assertion `isa<To>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

This patch tightens VPlan construction to always materialize vector values for predicated instructions via explicit insertelement chains, restoring the invariant that VPPredInstPHIRecipe assumes. An assert was added to catch violations earlier.

>From 022e19fa3610400f4a537a8806646106fadaf998 Mon Sep 17 00:00:00 2001
From: Abhinav Srivastava <atg271 at gmail.com>
Date: Thu, 18 Sep 2025 03:02:43 -0700
Subject: [PATCH] [LoopVectorize] Ensure VPPredInstPHIRecipe only uses
 InsertElementInst

The loop vectorizer could generate VPlans where VPPredInstPHIRecipe
received a non-InsertElement vector value, leading to an assertion
failure:

  opt: Casting.h:578: decltype(auto) llvm::cast<InsertElementInst>(...)
  Assertion `isa<To>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

This patch tightens VPlan construction to always materialize vector
values for predicated instructions via explicit insertelement chains,
restoring the invariant that VPPredInstPHIRecipe assumes. An assert
was added to catch violations earlier.
---
 .../lib/Transforms/Vectorize/VPlanRecipes.cpp | 28 ++++++++++++++++---
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp b/llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
index 8e9c3db50319f..8c827ace9e07e 100644
--- a/llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+++ b/llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
@@ -3275,10 +3275,30 @@ void VPPredInstPHIRecipe::execute(VPTransformState &State) {
   // Otherwise, a phi node for the scalar value is needed.
   if (State.hasVectorValue(getOperand(0))) {
     Value *VectorValue = State.get(getOperand(0));
-    InsertElementInst *IEI = cast<InsertElementInst>(VectorValue);
-    PHINode *VPhi = State.Builder.CreatePHI(IEI->getType(), 2);
-    VPhi->addIncoming(IEI->getOperand(0), PredicatingBB); // Unmodified vector.
-    VPhi->addIncoming(IEI, PredicatedBB); // New vector with inserted element.
+    
+    // The vector value can be either an InsertElementInst (first iteration)
+    // or a PHINode (subsequent iterations after merging)
+    Value *BaseVector = nullptr;
+    Value *MergedVector = nullptr;
+    
+    if (auto *IEI = dyn_cast<InsertElementInst>(VectorValue)) {
+      // First iteration: we have an InsertElementInst
+      BaseVector = IEI->getOperand(0);  // Unmodified vector
+      MergedVector = IEI;                // Vector with inserted element
+    } else if (auto *PHI = dyn_cast<PHINode>(VectorValue)) {
+      // Subsequent iterations: we have a PHI from previous merge
+      BaseVector = PHI;     // Use the PHI as the base for both paths
+      MergedVector = PHI;   // The merged vector is also the PHI
+    } else {
+      // Fallback: handle other vector types (e.g., Undef, Poison)
+      BaseVector = VectorValue;
+      MergedVector = VectorValue;
+    }
+    
+    PHINode *VPhi = State.Builder.CreatePHI(VectorValue->getType(), 2);
+    VPhi->addIncoming(BaseVector, PredicatingBB);    // Unmodified vector
+    VPhi->addIncoming(MergedVector, PredicatedBB);   // New/merged vector
+    
     if (State.hasVectorValue(this))
       State.reset(this, VPhi);
     else