[llvm] AMDGPU: Preserve carry-out dead flag in V_ADD_CO_U32 custom inserter (PR #225736)

[Matt Arsenault via llvm-commits]

https://github.com/arsenm created https://github.com/llvm/llvm-project/pull/225736

EmitInstrWithCustomInserter rebuilds V_ADD/SUB_CO_U32_e32 but did
not carry over the dead flag. Preserve the flag to reduce implicitly
depending on dead flags recomputed by LiveVariables.

Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>

>From 2adec5a5592eb634c0852bf3a2f4b76593f97c4e Mon Sep 17 00:00:00 2001
From: Matt Arsenault <Matthew.Arsenault at amd.com>
Date: Wed, 23 Sep 2026 12:15:53 +0200
Subject: [PATCH] AMDGPU: Preserve carry-out dead flag in V_ADD_CO_U32 custom
 inserter

EmitInstrWithCustomInserter rebuilds V_ADD/SUB_CO_U32_e32 but did
not carry over the dead flag. Preserve the flag to reduce implicitly
depending on dead flags recomputed by LiveVariables.

Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
---
 llvm/lib/Target/AMDGPU/SIISelLowering.cpp     | 12 +++-
 .../finalize-isel-vadd-co-u32-dead-carry.mir  | 72 +++++++++++++++++++
 2 files changed, 81 insertions(+), 3 deletions(-)
 create mode 100644 llvm/test/CodeGen/AMDGPU/finalize-isel-vadd-co-u32-dead-carry.mir

diff --git a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
index f9a4ba4c1bd1d..43cb9580e80e9 100644
--- a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
@@ -7334,14 +7334,20 @@ SITargetLowering::EmitInstrWithCustomInserter(MachineInstr &MI,
       NeedClampOperand = true;
     }
 
+    bool VCCDead = MI.getOperand(3).isDead();
+
     auto I = BuildMI(*BB, MI, DL, TII->get(Opc), MI.getOperand(0).getReg());
-    if (TII->isVOP3(*I)) {
-      I.addReg(TRI->getVCC(), RegState::Define);
-    }
+    bool IsVOP3 = TII->isVOP3(*I);
+    if (IsVOP3)
+      I.addReg(TRI->getVCC(), RegState::Define | getDeadRegState(VCCDead));
+
     I.add(MI.getOperand(1)).add(MI.getOperand(2));
     if (NeedClampOperand)
       I.addImm(0); // clamp bit for e64 encoding
 
+    if (!IsVOP3 && VCCDead)
+      I.setOperandDead(3);
+
     TII->legalizeOperands(*I);
 
     MI.eraseFromParent();
diff --git a/llvm/test/CodeGen/AMDGPU/finalize-isel-vadd-co-u32-dead-carry.mir b/llvm/test/CodeGen/AMDGPU/finalize-isel-vadd-co-u32-dead-carry.mir
new file mode 100644
index 0000000000000..bf6a1fd777d94
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/finalize-isel-vadd-co-u32-dead-carry.mir
@@ -0,0 +1,72 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 6
+# RUN: llc -mtriple=amdgpu8.03 -passes=finalize-isel -o - %s | FileCheck %s
+
+# The V_ADD_CO/V_SUB_CO_U32_e32 custom inserter rebuilds the
+# instruction; the carry-out ($vcc) dead flag should be preserved
+
+---
+name: add_co_u32_e32_dead_vcc
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1
+    ; CHECK-LABEL: name: add_co_u32_e32_dead_vcc
+    ; CHECK: liveins: $vgpr0, $vgpr1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+    ; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr1
+    ; CHECK-NEXT: [[V_ADD_CO_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_CO_U32_e32 [[COPY]], [[COPY1]], implicit-def dead $vcc, implicit $exec
+    ; CHECK-NEXT: $vgpr0 = COPY [[V_ADD_CO_U32_e32_]]
+    ; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0
+    %0:vgpr_32 = COPY $vgpr0
+    %1:vgpr_32 = COPY $vgpr1
+    %2:vgpr_32 = V_ADD_CO_U32_e32 %0, %1, implicit-def dead $vcc, implicit $exec
+    $vgpr0 = COPY %2
+    SI_RETURN_TO_EPILOG $vgpr0
+...
+
+---
+name: sub_co_u32_e32_dead_vcc
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1
+    ; CHECK-LABEL: name: sub_co_u32_e32_dead_vcc
+    ; CHECK: liveins: $vgpr0, $vgpr1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+    ; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr1
+    ; CHECK-NEXT: [[V_SUB_CO_U32_e32_:%[0-9]+]]:vgpr_32 = V_SUB_CO_U32_e32 [[COPY]], [[COPY1]], implicit-def dead $vcc, implicit $exec
+    ; CHECK-NEXT: $vgpr0 = COPY [[V_SUB_CO_U32_e32_]]
+    ; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0
+    %0:vgpr_32 = COPY $vgpr0
+    %1:vgpr_32 = COPY $vgpr1
+    %2:vgpr_32 = V_SUB_CO_U32_e32 %0, %1, implicit-def dead $vcc, implicit $exec
+    $vgpr0 = COPY %2
+    SI_RETURN_TO_EPILOG $vgpr0
+...
+
+---
+name: add_co_u32_e32_live_vcc
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1
+    ; CHECK-LABEL: name: add_co_u32_e32_live_vcc
+    ; CHECK: liveins: $vgpr0, $vgpr1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+    ; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr1
+    ; CHECK-NEXT: [[V_ADD_CO_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_CO_U32_e32 [[COPY]], [[COPY1]], implicit-def $vcc, implicit $exec
+    ; CHECK-NEXT: [[V_ADDC_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADDC_U32_e32 [[COPY]], [[COPY1]], implicit-def dead $vcc, implicit $vcc, implicit $exec
+    ; CHECK-NEXT: $vgpr0 = COPY [[V_ADD_CO_U32_e32_]]
+    ; CHECK-NEXT: $vgpr1 = COPY [[V_ADDC_U32_e32_]]
+    ; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0, implicit $vgpr1
+    %0:vgpr_32 = COPY $vgpr0
+    %1:vgpr_32 = COPY $vgpr1
+    %2:vgpr_32 = V_ADD_CO_U32_e32 %0, %1, implicit-def $vcc, implicit $exec
+    %3:vgpr_32 = V_ADDC_U32_e32 %0, %1, implicit-def dead $vcc, implicit $vcc, implicit $exec
+    $vgpr0 = COPY %2
+    $vgpr1 = COPY %3
+    SI_RETURN_TO_EPILOG $vgpr0, implicit $vgpr1
+...