[llvm] 54cb405 - [X86][ISel][FMA] Get a handle on operand nodes when negating FMA (#130176)
via llvm-commits
llvm-commits at lists.llvm.org
Mon Mar 17 09:07:37 PDT 2025
Author: Vineet Kumar
Date: 2025-03-17T16:07:33Z
New Revision: 54cb4059da2751758b3ab6dc5630e40b228bcda5
URL: https://github.com/llvm/llvm-project/commit/54cb4059da2751758b3ab6dc5630e40b228bcda5
DIFF: https://github.com/llvm/llvm-project/commit/54cb4059da2751758b3ab6dc5630e40b228bcda5.diff
LOG: [X86][ISel][FMA] Get a handle on operand nodes when negating FMA (#130176)
When negating an FMA opcode, a new node created for a negated FMA
operand may be deleted while recursively negating another FMA operand.
This causes the following assertion to fail:
```
llc: /root/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:7686: llvm::SDValue llvm::SelectionDAG::getNode(unsigned int, const llvm::SDLoc&, llvm::EVT, llvm::SDValue, llvm::SDValue, llvm::SDValue, llvm::SDNodeFlags): Assertion `N1.getOpcode() != ISD::DELETED_NODE && N2.getOpcode() != ISD::DELETED_NODE && N3.getOpcode() != ISD::DELETED_NODE && "Operand is DELETED_NODE!"' failed.
```
This patch adds a temporary handle on the new negated nodes to prevent
them from being deleted.
For eg. see https://godbolt.org/z/Tq4PvnKM4 .
Co-authored-by: Vineet Kumar <vineetk at hpe.com>
Added:
llvm/test/CodeGen/X86/combine-fma-negate.ll
Modified:
llvm/lib/Target/X86/X86ISelLowering.cpp
Removed:
################################################################################
diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp
index da10a88d0111b..d3e6bff302600 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.cpp
+++ b/llvm/lib/Target/X86/X86ISelLowering.cpp
@@ -55694,7 +55694,12 @@ static SDValue combineFMA(SDNode *N, SelectionDAG &DAG,
// Do not convert the passthru input of scalar intrinsics.
// FIXME: We could allow negations of the lower element only.
bool NegA = invertIfNegative(A);
+ // Create a dummy use for A so that in the process of negating B or C
+ // recursively, it is not deleted.
+ HandleSDNode NegAHandle(A);
bool NegB = invertIfNegative(B);
+ // Similar to A, get a handle on B.
+ HandleSDNode NegBHandle(B);
bool NegC = invertIfNegative(C);
if (!NegA && !NegB && !NegC)
diff --git a/llvm/test/CodeGen/X86/combine-fma-negate.ll b/llvm/test/CodeGen/X86/combine-fma-negate.ll
new file mode 100644
index 0000000000000..945df32781464
--- /dev/null
+++ b/llvm/test/CodeGen/X86/combine-fma-negate.ll
@@ -0,0 +1,25 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -mattr=+avx512f -mattr=+fma | FileCheck %s
+
+define void @fma_neg(<8 x i1> %r280, ptr %pp1, ptr %pp2) {
+; CHECK-LABEL: fma_neg:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vpmovsxwq %xmm0, %zmm0
+; CHECK-NEXT: vpsllq $63, %zmm0, %zmm0
+; CHECK-NEXT: vptestmq %zmm0, %zmm0, %k1
+; CHECK-NEXT: vmovdqu64 (%rdi), %zmm0
+; CHECK-NEXT: vpxorq {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to8}, %zmm0, %zmm1 {%k1} {z}
+; CHECK-NEXT: vxorpd %xmm2, %xmm2, %xmm2
+; CHECK-NEXT: vfnmadd213pd {{.*#+}} zmm2 = -(zmm0 * zmm2) + zmm1
+; CHECK-NEXT: vmovupd %zmm2, (%rsi)
+; CHECK-NEXT: vzeroupper
+; CHECK-NEXT: retq
+ %r290 = load <8 x double>, ptr %pp1, align 8
+ %r307 = fneg <8 x double> %r290
+ %r309 = select <8 x i1> %r280, <8 x double> %r307, <8 x double> zeroinitializer
+ %r311 = tail call <8 x double> @llvm.x86.avx512.vfmadd.pd.512(<8 x double> %r307, <8 x double> zeroinitializer, <8 x double> %r309, i32 4)
+ store <8 x double> %r311, ptr %pp2, align 8
+ ret void
+}
+
+declare <8 x double> @llvm.x86.avx512.vfmadd.pd.512(<8 x double>, <8 x double>, <8 x double>, i32 immarg)
More information about the llvm-commits
mailing list