[llvm] [LoopUnroll] UnrollRuntimeMultiExit takes precedence over TTI. (PR #134259)

[Florian Hahn via llvm-commits]

https://github.com/fhahn created https://github.com/llvm/llvm-project/pull/134259

Update UnrollRuntimeLoopRemainder to always give priority to the
UnrollRuntimeMultiExit option, if provided.

After https://github.com/llvm/llvm-project/commit/ad9da92cf6f735747ef04fd56937e1d76819e503 / https://github.com/llvm/llvm-project/pull/124462, we would ignore the option if the backend
indicates multi-exit is profitable. This means it cannot be used to
disable runtime unrolling.

To be consistent with canProfitablyRuntimeUnrollMultiExitLoop, always
respect the option.

This surfaced while discussing https://github.com/llvm/llvm-project/pull/131998.

>From 8d5bddfae3c2624e27b7c4052d3d89cc9690aefe Mon Sep 17 00:00:00 2001
From: Florian Hahn <flo at fhahn.com>
Date: Thu, 3 Apr 2025 13:35:53 +0100
Subject: [PATCH 1/2] [LoopUnroll] Add RUN line with unroll-runtime-multi-exit
 disabled.

---
 .../Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll  | 1 +
 1 file changed, 1 insertion(+)

diff --git a/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll b/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll
index 31b23eae0f866..8f1d12d2a4130 100644
--- a/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll
+++ b/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll
@@ -3,6 +3,7 @@
 ; RUN: opt -p loop-unroll -mcpu=apple-m2 -S %s | FileCheck --check-prefix=APPLE %s
 ; RUN: opt -p loop-unroll -mcpu=apple-m3 -S %s | FileCheck --check-prefix=APPLE %s
 ; RUN: opt -p loop-unroll -mcpu=apple-m4 -S %s | FileCheck --check-prefix=APPLE %s
+; RUN: opt -p loop-unroll -mcpu=apple-m1 -unroll-runtime-multi-exit=false -S %s | FileCheck --check-prefix=APPLE %s
 ; RUN: opt -p loop-unroll -mcpu=cortex-a57 -S %s | FileCheck --check-prefix=OTHER %s
 
 target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-n32:64-S128-Fn32"

>From 111a78e8648f1c93a1e0db3003743ce1589d886d Mon Sep 17 00:00:00 2001
From: Florian Hahn <flo at fhahn.com>
Date: Thu, 3 Apr 2025 13:51:50 +0100
Subject: [PATCH 2/2] [LoopUnroll] UnrollRuntimeMultiExit takes precedence over
 TTI.

Update UnrollRuntimeLoopRemainder to always give priority to the
UnrollRuntimeMultiExit option, if provided.

After ad9da92cf6f7357, we would ignore the option if the backend
indicates multi-exit is profitable. This means it cannot be used to
disable runtime unrolling.

To be consistent with canProfitablyRuntimeUnrollMultiExitLoop, always
repsect the option.

This surfaced while discussing https://github.com/llvm/llvm-project/pull/131998.
---
 .../Transforms/Utils/LoopUnrollRuntime.cpp    | 27 ++++++++++---------
 .../AArch64/apple-unrolling-multi-exit.ll     |  2 +-
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp b/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp
index 524b268aee2f3..b91f4f6c67408 100644
--- a/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp
+++ b/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp
@@ -465,10 +465,6 @@ static bool canProfitablyRuntimeUnrollMultiExitLoop(
     Loop *L, SmallVectorImpl<BasicBlock *> &OtherExits, BasicBlock *LatchExit,
     bool UseEpilogRemainder) {
 
-  // Priority goes to UnrollRuntimeMultiExit if it's supplied.
-  if (UnrollRuntimeMultiExit.getNumOccurrences())
-    return UnrollRuntimeMultiExit;
-
   // The main pain point with multi-exit loop unrolling is that once unrolled,
   // we will not be able to merge all blocks into a straight line code.
   // There are branches within the unrolled loop that go to the OtherExits.
@@ -633,14 +629,21 @@ bool llvm::UnrollRuntimeLoopRemainder(
     if (!PreserveLCSSA)
       return false;
 
-    if (!RuntimeUnrollMultiExit &&
-        !canProfitablyRuntimeUnrollMultiExitLoop(L, OtherExits, LatchExit,
-                                                 UseEpilogRemainder)) {
-      LLVM_DEBUG(
-          dbgs()
-          << "Multiple exit/exiting blocks in loop and multi-exit unrolling not "
-             "enabled!\n");
-      return false;
+    // Priority goes to UnrollRuntimeMultiExit if it's supplied.
+    if (UnrollRuntimeMultiExit.getNumOccurrences()) {
+      if (!UnrollRuntimeMultiExit)
+        return false;
+    } else {
+      // Otherwise perform multi-exit unrolling, if either the target indicates
+      // it is profitable or the general profitability heuristics apply.
+      if (!RuntimeUnrollMultiExit &&
+          !canProfitablyRuntimeUnrollMultiExitLoop(L, OtherExits, LatchExit,
+                                                   UseEpilogRemainder)) {
+        LLVM_DEBUG(dbgs() << "Multiple exit/exiting blocks in loop and "
+                             "multi-exit unrolling not "
+                             "enabled!\n");
+        return false;
+      }
     }
   }
   // Use Scalar Evolution to compute the trip count. This allows more loops to
diff --git a/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll b/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll
index 8f1d12d2a4130..23771de4c8938 100644
--- a/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll
+++ b/llvm/test/Transforms/LoopUnroll/AArch64/apple-unrolling-multi-exit.ll
@@ -3,7 +3,7 @@
 ; RUN: opt -p loop-unroll -mcpu=apple-m2 -S %s | FileCheck --check-prefix=APPLE %s
 ; RUN: opt -p loop-unroll -mcpu=apple-m3 -S %s | FileCheck --check-prefix=APPLE %s
 ; RUN: opt -p loop-unroll -mcpu=apple-m4 -S %s | FileCheck --check-prefix=APPLE %s
-; RUN: opt -p loop-unroll -mcpu=apple-m1 -unroll-runtime-multi-exit=false -S %s | FileCheck --check-prefix=APPLE %s
+; RUN: opt -p loop-unroll -mcpu=apple-m1 -unroll-runtime-multi-exit=false -S %s | FileCheck --check-prefix=OTHER %s
 ; RUN: opt -p loop-unroll -mcpu=cortex-a57 -S %s | FileCheck --check-prefix=OTHER %s
 
 target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-n32:64-S128-Fn32"