[llvm-branch-commits] [llvm] [DAGCombiner][GlobalISel] Prevent FMA contraction when fmul cannot be eliminated (FADD/FSUB pattern) (PR #188114)
via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Fri Apr 3 08:57:14 PDT 2026
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-llvm-globalisel
@llvm/pr-subscribers-backend-powerpc
Author: Adel Ejjeh (adelejjeh)
<details>
<summary>Changes</summary>
Supersedes #<!-- -->169735 (PR 2/5). Split into a stack of 5 PRs per reviewer request.
---
`fmul` nodes with multiple uses can currently be contracted into FMA operations even when the `fmul` itself cannot be eliminated, resulting in a redundant multiply (wasted power and compute). The existing guard `Aggressive || N0->hasOneUse()` allows contraction under Aggressive mode regardless of whether the multiply can be removed.
This patch tightens the guard to:
```
N0->hasOneUse() || (Aggressive && allMulUsesCanBeContracted(N0))
```
`allMulUsesCanBeContracted()` iterates all users of the multiply and returns true only if every use is itself contractable into an FMA. For this patch, only direct FADD and FSUB uses are recognized as contractable (FNEG, FPEXT, and FMA/FMAD patterns follow in subsequent patches).
Changes:
- **DAGCombiner.cpp**: Add `allMulUsesCanBeContracted()` that iterates multiply users and checks contractability. Update 4 fold site guards in `visitFADDForFMACombine` (2 sites) and `visitFSUBForFMACombine` (2 sites).
- **CombinerHelper.cpp**: Add equivalent `allMulUsesCanBeContracted()` for GlobalISel. Update 4 fold site guards in `matchCombineFAddFMulToFMadOrFMA` (2 sites) and `matchCombineFSubFMulToFMadOrFMA` (2 sites).
### Test Changes
#### `fma-multiple-uses-contraction.ll`
The test file switches RUN lines from `P0-` prefixed check prefixes (established in PR 1/5) to real prefixes. The "Direct FADD/FSUB" section is activated with CHECK lines reflecting the guard changes. Remaining sections (FNEG, FPEXT, FMA/FMAD) retain `P0-` prefixed baseline checks -- these patterns are not yet recognized by the guard and are activated by later PRs. Section headers include NOTE comments explaining this. `--allow-unused-prefixes` is added since some denorm-specific prefix variants only appear in the P0-prefixed sections.
#### Other lit tests
##### **Permanent changes**: fmul has a genuinely non-contractable use (fcmp, fdiv, store), so contraction is correctly blocked to avoid duplicating the multiply:
| Test | Functions | Non-contractable use |
| --- | --- | --- |
| `amdgpu-simplify-libcall-pow-codegen.ll` | `test_pow_fast_f32__integral_y`, `test_powr_fast_f32`, `test_pown_fast_f32`, `test_pown_fast_f32_known_even`, `test_pown_fast_f32_known_odd` | `fcmp` (pow range check) |
| `copysign-simplify-demanded-bits.ll` | `test_copysign_pow_fast_f32__integral_y` | `fcmp` (pow range check) |
| `fma.f16.ll` | `test_D139469_f16`, `test_D139469_v2f16` | `fcmp` |
| `fmul-2-combine-multi-use.ll` | `multiple_use_fadd_fmac_f32`, `multiple_use_fadd_fmad_f32` | `store` |
| `fma-aggr-FMF.ll` (PPC) | `no_fma_with_fewer_uses` | `fdiv` |
| `fma-precision.ll` (PPC) | `fma_multi_uses1`, `fma_multi_uses2` | `store` |
##### **Temporary changes**: fmul use is contractable but not yet recognized in this patch; subsequent patches restore the original contraction:
| Test | Functions | Use type | Restored by |
| --- | --- | --- | --- |
| `mad-combine.ll` | `combine_to_mad_fsub_2_f32_2uses_mul` | `fneg` | PR 3/5 (FNEG pattern) |
| `dagcombine-fma-fmad.ll` | `_amdgpu_ps_main` | fmul→FMA chain | PR 5/5 (FMA/FMAD pattern) |
| `dagcombine-fma-crash.ll` | `main` | fmul→FMA chain | PR 5/5 (FMA/FMAD pattern) |
---
_This PR was split from the original_ #<!-- -->169735 _with the assistance of_ [_Claude Code_](https://claude.ai/code)_._
---
Patch is 132.86 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/188114.diff
14 Files Affected:
- (modified) llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h (+4)
- (modified) llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp (+35-5)
- (modified) llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp (+32-4)
- (modified) llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow-codegen.ll (+25-25)
- (modified) llvm/test/CodeGen/AMDGPU/copysign-simplify-demanded-bits.ll (+5-5)
- (modified) llvm/test/CodeGen/AMDGPU/dagcombine-fma-crash.ll (+12-10)
- (modified) llvm/test/CodeGen/AMDGPU/dagcombine-fma-fmad.ll (+95-94)
- (modified) llvm/test/CodeGen/AMDGPU/fma-multiple-uses-contraction.ll (+597-659)
- (modified) llvm/test/CodeGen/AMDGPU/fma.f16.ll (+115-148)
- (modified) llvm/test/CodeGen/AMDGPU/fmul-2-combine-multi-use.ll (+2-2)
- (modified) llvm/test/CodeGen/AMDGPU/mad-combine.ll (+5-4)
- (modified) llvm/test/CodeGen/PowerPC/fma-aggr-FMF.ll (+3-3)
- (modified) llvm/test/CodeGen/PowerPC/fma-precision.ll (+12-13)
- (added) uv.lock (+3)
``````````diff
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
index 365bbaacfe055..09c827f71a34d 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
@@ -803,6 +803,10 @@ class CombinerHelper {
bool matchFsubToFneg(MachineInstr &MI, Register &MatchInfo) const;
void applyFsubToFneg(MachineInstr &MI, Register &MatchInfo) const;
+ /// Check if all uses of a multiply can be contracted into fma/fmad
+ /// operations, so that duplicating the multiply is acceptable.
+ bool allMulUsesCanBeContracted(const MachineInstr &MI) const;
+
bool canCombineFMadOrFMA(MachineInstr &MI, bool &AllowFusionGlobally,
bool &HasFMAD, bool &Aggressive,
bool CanReassociate = false) const;
diff --git a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
index f4d5bd5ee5745..5fdba8cd4ab99 100644
--- a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
@@ -6309,6 +6309,32 @@ static bool hasMoreUses(const MachineInstr &MI0, const MachineInstr &MI1,
MRI.use_instr_nodbg_end());
}
+/// Check if all uses of a multiply can be contracted into FMA operations.
+/// Returns true if all uses of the multiply are contractable, meaning the
+/// multiply can potentially be eliminated through FMA contraction.
+/// Returns false if any use cannot be contracted, which would mean contracting
+/// would duplicate the multiply without reducing the total number of
+/// operations.
+///
+/// Currently checks for the following pattern:
+/// - fmul --> fadd/fsub: Direct contraction
+bool CombinerHelper::allMulUsesCanBeContracted(const MachineInstr &MI) const {
+ Register MulReg = MI.getOperand(0).getReg();
+
+ for (const MachineInstr &UseMI : MRI.use_nodbg_instructions(MulReg)) {
+ unsigned Opcode = UseMI.getOpcode();
+
+ // Direct FADD/FSUB uses - contractable.
+ if (Opcode == TargetOpcode::G_FADD || Opcode == TargetOpcode::G_FSUB)
+ continue;
+
+ // Any other use type is not currently recognized as contractable.
+ return false;
+ }
+
+ return true; // All uses can be contracted.
+}
+
bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
bool &AllowFusionGlobally,
bool &HasFMAD, bool &Aggressive,
@@ -6366,7 +6392,8 @@ bool CombinerHelper::matchCombineFAddFMulToFMadOrFMA(
// fold (fadd (fmul x, y), z) -> (fma x, y, z)
if (isContractableFMul(*LHS.MI, AllowFusionGlobally) &&
- (Aggressive || MRI.hasOneNonDBGUse(LHS.Reg))) {
+ (MRI.hasOneNonDBGUse(LHS.Reg) ||
+ (Aggressive && allMulUsesCanBeContracted(*LHS.MI)))) {
MatchInfo = [=, &MI](MachineIRBuilder &B) {
B.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
{LHS.MI->getOperand(1).getReg(),
@@ -6377,7 +6404,8 @@ bool CombinerHelper::matchCombineFAddFMulToFMadOrFMA(
// fold (fadd x, (fmul y, z)) -> (fma y, z, x)
if (isContractableFMul(*RHS.MI, AllowFusionGlobally) &&
- (Aggressive || MRI.hasOneNonDBGUse(RHS.Reg))) {
+ (MRI.hasOneNonDBGUse(RHS.Reg) ||
+ (Aggressive && allMulUsesCanBeContracted(*RHS.MI)))) {
MatchInfo = [=, &MI](MachineIRBuilder &B) {
B.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
{RHS.MI->getOperand(1).getReg(),
@@ -6671,7 +6699,8 @@ bool CombinerHelper::matchCombineFSubFMulToFMadOrFMA(
// fold (fsub (fmul x, y), z) -> (fma x, y, -z)
if (FirstMulHasFewerUses &&
(isContractableFMul(*LHS.MI, AllowFusionGlobally) &&
- (Aggressive || MRI.hasOneNonDBGUse(LHS.Reg)))) {
+ (MRI.hasOneNonDBGUse(LHS.Reg) ||
+ (Aggressive && allMulUsesCanBeContracted(*LHS.MI))))) {
MatchInfo = [=, &MI](MachineIRBuilder &B) {
Register NegZ = B.buildFNeg(DstTy, RHS.Reg).getReg(0);
B.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
@@ -6681,8 +6710,9 @@ bool CombinerHelper::matchCombineFSubFMulToFMadOrFMA(
return true;
}
// fold (fsub x, (fmul y, z)) -> (fma -y, z, x)
- else if ((isContractableFMul(*RHS.MI, AllowFusionGlobally) &&
- (Aggressive || MRI.hasOneNonDBGUse(RHS.Reg)))) {
+ if (isContractableFMul(*RHS.MI, AllowFusionGlobally) &&
+ (MRI.hasOneNonDBGUse(RHS.Reg) ||
+ (Aggressive && allMulUsesCanBeContracted(*RHS.MI)))) {
MatchInfo = [=, &MI](MachineIRBuilder &B) {
Register NegY =
B.buildFNeg(DstTy, RHS.MI->getOperand(1).getReg()).getReg(0);
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index 1e1c184b9d119..38cacf0d52182 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -17678,6 +17678,30 @@ static bool isFusedOp(const MatchContextClass &Matcher, SDValue N) {
return Matcher.match(N, ISD::FMA) || Matcher.match(N, ISD::FMAD);
}
+/// Check if all uses of a multiply can be contracted into FMA operations.
+/// Returns true if all uses of the multiply are contractable, meaning the
+/// multiply can potentially be eliminated through FMA contraction.
+/// Returns false if any use cannot be contracted, which would mean contracting
+/// would duplicate the multiply without reducing the total number of
+/// operations.
+///
+/// Currently checks for the following pattern:
+/// - fmul --> fadd/fsub: Direct contraction
+static bool allMulUsesCanBeContracted(SDValue Mul) {
+ for (const auto *User : Mul->users()) {
+ unsigned Opcode = User->getOpcode();
+
+ // Direct FADD/FSUB - contractable.
+ if (Opcode == ISD::FADD || Opcode == ISD::FSUB)
+ continue;
+
+ // Any other use type is not currently recognized as contractable.
+ return false;
+ }
+
+ return true; // All uses can be contracted.
+}
+
/// Try to perform FMA combining on a given FADD node.
template <class MatchContextClass>
SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
@@ -17739,14 +17763,16 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
}
// fold (fadd (fmul x, y), z) -> (fma x, y, z)
- if (isContractableFMUL(N0) && (Aggressive || N0->hasOneUse())) {
+ if (isContractableFMUL(N0) &&
+ (N0->hasOneUse() || (Aggressive && allMulUsesCanBeContracted(N0)))) {
return matcher.getNode(PreferredFusedOpcode, SL, VT, N0.getOperand(0),
N0.getOperand(1), N1);
}
// fold (fadd x, (fmul y, z)) -> (fma y, z, x)
// Note: Commutes FADD operands.
- if (isContractableFMUL(N1) && (Aggressive || N1->hasOneUse())) {
+ if (isContractableFMUL(N1) &&
+ (N1->hasOneUse() || (Aggressive && allMulUsesCanBeContracted(N1)))) {
return matcher.getNode(PreferredFusedOpcode, SL, VT, N1.getOperand(0),
N1.getOperand(1), N0);
}
@@ -17962,7 +17988,8 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
// fold (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
auto tryToFoldXYSubZ = [&](SDValue XY, SDValue Z) {
- if (isContractableFMUL(XY) && (Aggressive || XY->hasOneUse())) {
+ if (isContractableFMUL(XY) &&
+ (XY->hasOneUse() || (Aggressive && allMulUsesCanBeContracted(XY)))) {
return matcher.getNode(PreferredFusedOpcode, SL, VT, XY.getOperand(0),
XY.getOperand(1),
matcher.getNode(ISD::FNEG, SL, VT, Z));
@@ -17973,7 +18000,8 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
// fold (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
// Note: Commutes FSUB operands.
auto tryToFoldXSubYZ = [&](SDValue X, SDValue YZ) {
- if (isContractableFMUL(YZ) && (Aggressive || YZ->hasOneUse())) {
+ if (isContractableFMUL(YZ) &&
+ (YZ->hasOneUse() || (Aggressive && allMulUsesCanBeContracted(YZ)))) {
return matcher.getNode(
PreferredFusedOpcode, SL, VT,
matcher.getNode(ISD::FNEG, SL, VT, YZ.getOperand(0)),
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow-codegen.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow-codegen.ll
index afe0971088bc1..056ec8215e6b4 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow-codegen.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-simplify-libcall-pow-codegen.ll
@@ -90,12 +90,12 @@ define float @test_pow_fast_f32__integral_y(float %x, i32 %y.i) {
; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
; CHECK-NEXT: v_sub_f32_e32 v2, v3, v2
; CHECK-NEXT: v_trunc_f32_e32 v3, v1
-; CHECK-NEXT: v_mul_f32_e32 v4, v2, v3
+; CHECK-NEXT: v_mul_f32_e32 v2, v2, v3
; CHECK-NEXT: s_mov_b32 s4, 0xc2fc0000
-; CHECK-NEXT: v_mov_b32_e32 v5, 0x42800000
-; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v4
-; CHECK-NEXT: v_cndmask_b32_e32 v4, 0, v5, vcc
-; CHECK-NEXT: v_fma_f32 v2, v2, v3, v4
+; CHECK-NEXT: v_mov_b32_e32 v3, 0x42800000
+; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v2
+; CHECK-NEXT: v_cndmask_b32_e32 v3, 0, v3, vcc
+; CHECK-NEXT: v_add_f32_e32 v2, v2, v3
; CHECK-NEXT: v_exp_f32_e32 v2, v2
; CHECK-NEXT: v_cvt_i32_f32_e32 v1, v1
; CHECK-NEXT: v_not_b32_e32 v3, 63
@@ -235,11 +235,11 @@ define float @test_powr_fast_f32(float %x, float %y) {
; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
; CHECK-NEXT: s_mov_b32 s4, 0xc2fc0000
; CHECK-NEXT: v_sub_f32_e32 v0, v0, v2
-; CHECK-NEXT: v_mul_f32_e32 v2, v1, v0
-; CHECK-NEXT: v_mov_b32_e32 v3, 0x42800000
-; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v2
-; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v3, vcc
-; CHECK-NEXT: v_fma_f32 v0, v1, v0, v2
+; CHECK-NEXT: v_mul_f32_e32 v0, v1, v0
+; CHECK-NEXT: v_mov_b32_e32 v1, 0x42800000
+; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v0
+; CHECK-NEXT: v_cndmask_b32_e32 v1, 0, v1, vcc
+; CHECK-NEXT: v_add_f32_e32 v0, v0, v1
; CHECK-NEXT: v_exp_f32_e32 v0, v0
; CHECK-NEXT: v_not_b32_e32 v1, 63
; CHECK-NEXT: v_cndmask_b32_e32 v1, 0, v1, vcc
@@ -374,12 +374,12 @@ define float @test_pown_fast_f32(float %x, i32 %y) {
; CHECK-NEXT: v_mov_b32_e32 v2, 0x42000000
; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
; CHECK-NEXT: v_sub_f32_e32 v2, v3, v2
-; CHECK-NEXT: v_mul_f32_e32 v3, v2, v4
+; CHECK-NEXT: v_mul_f32_e32 v2, v2, v4
; CHECK-NEXT: s_mov_b32 s4, 0xc2fc0000
-; CHECK-NEXT: v_mov_b32_e32 v5, 0x42800000
-; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v3
-; CHECK-NEXT: v_cndmask_b32_e32 v3, 0, v5, vcc
-; CHECK-NEXT: v_fma_f32 v2, v2, v4, v3
+; CHECK-NEXT: v_mov_b32_e32 v3, 0x42800000
+; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v2
+; CHECK-NEXT: v_cndmask_b32_e32 v3, 0, v3, vcc
+; CHECK-NEXT: v_add_f32_e32 v2, v2, v3
; CHECK-NEXT: v_exp_f32_e32 v2, v2
; CHECK-NEXT: v_not_b32_e32 v3, 63
; CHECK-NEXT: v_cndmask_b32_e32 v3, 0, v3, vcc
@@ -517,12 +517,12 @@ define float @test_pown_fast_f32_known_even(float %x, i32 %y.arg) {
; CHECK-NEXT: v_mov_b32_e32 v2, 0x42000000
; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
; CHECK-NEXT: v_sub_f32_e32 v0, v0, v2
-; CHECK-NEXT: v_mul_f32_e32 v2, v0, v1
+; CHECK-NEXT: v_mul_f32_e32 v0, v0, v1
; CHECK-NEXT: s_mov_b32 s4, 0xc2fc0000
-; CHECK-NEXT: v_mov_b32_e32 v3, 0x42800000
-; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v2
-; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v3, vcc
-; CHECK-NEXT: v_fma_f32 v0, v0, v1, v2
+; CHECK-NEXT: v_mov_b32_e32 v1, 0x42800000
+; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v0
+; CHECK-NEXT: v_cndmask_b32_e32 v1, 0, v1, vcc
+; CHECK-NEXT: v_add_f32_e32 v0, v0, v1
; CHECK-NEXT: v_exp_f32_e32 v0, v0
; CHECK-NEXT: v_not_b32_e32 v1, 63
; CHECK-NEXT: v_cndmask_b32_e32 v1, 0, v1, vcc
@@ -656,12 +656,12 @@ define float @test_pown_fast_f32_known_odd(float %x, i32 %y.arg) {
; CHECK-NEXT: v_mov_b32_e32 v2, 0x42000000
; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
; CHECK-NEXT: v_sub_f32_e32 v2, v3, v2
-; CHECK-NEXT: v_mul_f32_e32 v3, v2, v1
+; CHECK-NEXT: v_mul_f32_e32 v1, v2, v1
; CHECK-NEXT: s_mov_b32 s4, 0xc2fc0000
-; CHECK-NEXT: v_mov_b32_e32 v4, 0x42800000
-; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v3
-; CHECK-NEXT: v_cndmask_b32_e32 v3, 0, v4, vcc
-; CHECK-NEXT: v_fma_f32 v1, v2, v1, v3
+; CHECK-NEXT: v_mov_b32_e32 v2, 0x42800000
+; CHECK-NEXT: v_cmp_gt_f32_e32 vcc, s4, v1
+; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
+; CHECK-NEXT: v_add_f32_e32 v1, v1, v2
; CHECK-NEXT: v_exp_f32_e32 v1, v1
; CHECK-NEXT: v_not_b32_e32 v2, 63
; CHECK-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
diff --git a/llvm/test/CodeGen/AMDGPU/copysign-simplify-demanded-bits.ll b/llvm/test/CodeGen/AMDGPU/copysign-simplify-demanded-bits.ll
index ef676ddc8070e..3c67255ebdc5b 100644
--- a/llvm/test/CodeGen/AMDGPU/copysign-simplify-demanded-bits.ll
+++ b/llvm/test/CodeGen/AMDGPU/copysign-simplify-demanded-bits.ll
@@ -339,12 +339,12 @@ define float @test_copysign_pow_fast_f32__integral_y(float %x, i32 %y.i) {
; GFX9-NEXT: v_mov_b32_e32 v2, 0x42000000
; GFX9-NEXT: v_cndmask_b32_e32 v2, 0, v2, vcc
; GFX9-NEXT: v_sub_f32_e32 v2, v3, v2
-; GFX9-NEXT: v_mul_f32_e32 v3, v2, v1
+; GFX9-NEXT: v_mul_f32_e32 v2, v2, v1
; GFX9-NEXT: s_mov_b32 s4, 0xc2fc0000
-; GFX9-NEXT: v_mov_b32_e32 v4, 0x42800000
-; GFX9-NEXT: v_cmp_gt_f32_e32 vcc, s4, v3
-; GFX9-NEXT: v_cndmask_b32_e32 v3, 0, v4, vcc
-; GFX9-NEXT: v_fma_f32 v2, v2, v1, v3
+; GFX9-NEXT: v_mov_b32_e32 v3, 0x42800000
+; GFX9-NEXT: v_cmp_gt_f32_e32 vcc, s4, v2
+; GFX9-NEXT: v_cndmask_b32_e32 v3, 0, v3, vcc
+; GFX9-NEXT: v_add_f32_e32 v2, v2, v3
; GFX9-NEXT: v_exp_f32_e32 v2, v2
; GFX9-NEXT: v_cvt_i32_f32_e32 v1, v1
; GFX9-NEXT: v_not_b32_e32 v3, 63
diff --git a/llvm/test/CodeGen/AMDGPU/dagcombine-fma-crash.ll b/llvm/test/CodeGen/AMDGPU/dagcombine-fma-crash.ll
index 142494a803755..57070e763e79b 100644
--- a/llvm/test/CodeGen/AMDGPU/dagcombine-fma-crash.ll
+++ b/llvm/test/CodeGen/AMDGPU/dagcombine-fma-crash.ll
@@ -20,22 +20,24 @@ define void @main(float %arg) {
; CHECK-NEXT: bb.1.bb2:
; CHECK-NEXT: successors: %bb.2(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 1065353216, implicit $exec
- ; CHECK-NEXT: [[V_FMAC_F32_e64_:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_FMAC_F32_e64 0, [[S_MOV_B32_]], 0, [[S_MOV_B32_]], 0, [[V_MOV_B32_e32_]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_FMAC_F32_e64_1:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_FMAC_F32_e64 0, [[COPY]], 0, [[COPY]], 0, [[V_FMAC_F32_e64_]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[V_ADD_F32_e64_:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_ADD_F32_e64 0, [[V_FMAC_F32_e64_1]], 0, [[V_MOV_B32_e32_]], 0, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sreg_32 = S_MOV_B32 0
+ ; CHECK-NEXT: [[V_MUL_F32_e64_:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_MUL_F32_e64 0, [[S_MOV_B32_]], 0, [[S_MOV_B32_]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sgpr_32 = S_MOV_B32 1065353216
+ ; CHECK-NEXT: [[V_ADD_F32_e64_:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_ADD_F32_e64 0, [[V_MUL_F32_e64_]], 0, [[S_MOV_B32_2]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_FMAC_F32_e64_:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_FMAC_F32_e64 0, [[COPY]], 0, [[COPY]], 0, [[V_ADD_F32_e64_]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_FMAC_F32_e64_1:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_FMAC_F32_e64 0, [[COPY]], 0, [[COPY]], 0, [[V_MUL_F32_e64_]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[V_ADD_F32_e64_1:%[0-9]+]]:vgpr_32 = contract reassoc nofpexcept V_ADD_F32_e64 0, killed [[V_FMAC_F32_e64_1]], 0, [[S_MOV_B32_2]], 0, 0, implicit $mode, implicit $exec
+ ; CHECK-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sreg_32 = S_MOV_B32 0
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2.bb11:
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.4(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[PHI:%[0-9]+]]:vgpr_32 = PHI [[DEF]], %bb.0, [[V_FMAC_F32_e64_1]], %bb.1
- ; CHECK-NEXT: [[PHI1:%[0-9]+]]:vgpr_32 = PHI [[DEF]], %bb.0, [[V_ADD_F32_e64_]], %bb.1
- ; CHECK-NEXT: [[PHI2:%[0-9]+]]:sreg_32_xm0_xexec = PHI [[S_MOV_B32_1]], %bb.0, [[S_MOV_B32_2]], %bb.1
+ ; CHECK-NEXT: [[PHI:%[0-9]+]]:vgpr_32 = PHI [[DEF]], %bb.0, [[V_FMAC_F32_e64_]], %bb.1
+ ; CHECK-NEXT: [[PHI1:%[0-9]+]]:vgpr_32 = PHI [[DEF]], %bb.0, [[V_ADD_F32_e64_1]], %bb.1
+ ; CHECK-NEXT: [[PHI2:%[0-9]+]]:sreg_32_xm0_xexec = PHI [[S_MOV_B32_1]], %bb.0, [[S_MOV_B32_3]], %bb.1
; CHECK-NEXT: [[V_CNDMASK_B32_e64_:%[0-9]+]]:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, [[PHI2]], implicit $exec
- ; CHECK-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sreg_32 = S_MOV_B32 1
+ ; CHECK-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sreg_32 = S_MOV_B32 1
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sreg_32 = COPY [[V_CNDMASK_B32_e64_]]
- ; CHECK-NEXT: S_CMP_LG_U32 killed [[COPY1]], killed [[S_MOV_B32_3]], implicit-def $scc
+ ; CHECK-NEXT: S_CMP_LG_U32 killed [[COPY1]], killed [[S_MOV_B32_4]], implicit-def $scc
; CHECK-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $scc
; CHECK-NEXT: [[S_AND_B32_1:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, killed [[COPY2]], implicit-def dead $scc
; CHECK-NEXT: $vcc_lo = COPY [[S_AND_B32_1]]
diff --git a/llvm/test/CodeGen/AMDGPU/dagcombine-fma-fmad.ll b/llvm/test/CodeGen/AMDGPU/dagcombine-fma-fmad.ll
index 28a18ec3845e0..e95f19a19d134 100644
--- a/llvm/test/CodeGen/AMDGPU/dagcombine-fma-fmad.ll
+++ b/llvm/test/CodeGen/AMDGPU/dagcombine-fma-fmad.ll
@@ -12,64 +12,64 @@ define amdgpu_ps float @_amdgpu_ps_main() #0 {
; GFX10-NEXT: s_and_b32 exec_lo, exec_lo, s0
; GFX10-NEXT: s_waitcnt vmcnt(0)
; GFX10-NEXT: s_clause 0x1
-; GFX10-NEXT: image_sample v2, v[0:1], s[0:7], s[0:3] dmask:0x4 dim:SQ_RSRC_IMG_2D
-; GFX10-NEXT: image_sample v3, v[0:1], s[0:7], s[0:3] dmask:0x1 dim:SQ_RSRC_IMG_2D
+; GFX10-NEXT: image_sample v2, v[0:1], s[0:7], s[0:3] dmask:0x1 dim:SQ_RSRC_IMG_2D
+; GFX10-NEXT: image_sample v3, v[0:1], s[0:7], s[0:3] dmask:0x4 dim:SQ_RSRC_IMG_2D
; GFX10-NEXT: v_mov_b32_e32 v4, 0
; GFX10-NEXT: s_waitcnt vmcnt(0)
; GFX10-NEXT: image_load_mip v4, v[2:4], s[0:7] dmask:0x4 dim:SQ_RSRC_IMG_2D unorm
-; GFX10-NEXT: s_clause 0x3
+; GFX10-NEXT: s_clause 0x2
; GFX10-NEXT: s_buffer_load_dword s24, s[0:3], 0x5c
-; GFX10-NEXT: s_buffer_load_dword s28, s[0:3], 0x7c
-; GFX10-NEXT: s_buffer_load_dword s29, s[0:3], 0xc0
+; GFX10-NEXT: s_buffer_load_dword s25, s[0:3], 0x7c
; GFX10-NEXT: s_waitcnt_depctr depctr_vm_vsrc(0)
; GFX10-NEXT: s_nop 0
; GFX10-NEXT: s_buffer_load_dwordx4 s[0:3], s[0:3], 0x40
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
-; GFX10-NEXT: s_clause 0x1
+; GFX10-NEXT: s_clause 0x2
+; GFX10-NEXT: s_buffer_load_dword s26, s[0:3], 0xc0
; GFX10-NEXT: s_buffer_load_dwordx4 s[4:7], s[0:3], 0x50
-; GFX10-NEXT: s_nop 0
-; GFX10-NEXT: s_buffer_load_dword s0, s[0:3], 0x2c
-; GFX10-NEXT: v_sub_f32_e64 v5, s24, s28
-; GFX10-NEXT: s_waitcnt lgkmcnt(0)
-; GFX10-NEXT: s_clause 0x4
; GFX10-NEXT: s_buffer_load_dwordx4 s[8:11], s[0:3], 0x60
-; GFX10-NEXT: s_buffer_load_dwordx4 s[12:15], s[0:3], 0x20
-; GFX10-NEXT: s_buffer_load_dwordx4 s[16:19], s[0:3], 0x0
-; GFX10-NEXT: s_buffer_load_dwordx4 s[20:23], s[0:3], 0x70
-; GFX10-NEXT: s_buffer_load_dwordx4 s[24:27], s[0:3], 0x10
-; GFX10-NEXT: v_fma_f32 v1, v1, v5, s28
-; GFX10-NEXT: v_max_f32_e64 v6, s0, s0 clamp
-; GFX10-NEXT: v_add_f32_e64 v5, s29, -1.0
-; GFX10-NEXT: v_sub_f32_e32 v8, s0, v1
-; GFX10-NEXT: v_fma_f32 v7, -s2, v6, s6
-; GFX10-NEXT: v_fma_f32 v5, v6, v5, 1.0
-; GFX10-NEXT: v_mad_f32 v10, s2, v6, v2
-; GFX10-NEXT: s_mov_b32 s0, 0x3c23d70a
-; GFX10-NEXT: v_fmac_f32_e32 v1, v6, v8
-; GFX10-NEXT: v_fmac_f32_e32 v10, v7, v6
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
-; GFX10-NEXT: v_mul_f32_e32 v9, s10, v0
-; GFX10-NEXT: v_fma_f32 v0, -v0, s10, s14
-; GFX10-NEXT: v_mul_f32_e32 v8, s18, v2
-; GFX10-NEXT: v_mul_f32_e32 v3, s22, v3
-; GFX10-NEXT: v_fmac_f32_e32 v9, v0, v6
-; GFX10-NEXT: v_sub_f32_e32 v0, v1, v5
-; GFX10-NEXT: v_mul_f32_e32 v1, v8, v6
-; GFX10-NEXT: v_mul_f32_e32 v7, v6, v3
-; GFX10-NEXT: v_fma_f32...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/188114
More information about the llvm-branch-commits
mailing list