[llvm] [DFAJumpThreading] Prevent pass from using too much memory. (PR #145482)

[Bushev Dmitry via llvm-commits]

https://github.com/dybv-sc created https://github.com/llvm/llvm-project/pull/145482

The limit 'dfa-max-num-paths' that is used to control number of enumerated paths was not checked against inside getPathsFromStateDefMap. It may lead to large memory consumption for complex enough switch statements.

>From e89a0860f4da654390014dd4b8d3beaf5808d5af Mon Sep 17 00:00:00 2001
From: Dmitry Bushev <dmitry.bushev at syntacore.com>
Date: Mon, 23 Jun 2025 08:54:56 +0000
Subject: [PATCH] [DFAJumpThreading] Prevent pass from using too much memory.

The limit 'dfa-max-num-paths' that is used to control number of enumerated paths
was not checked against inside getPathsFromStateDefMap. It may lead to large memory consumption for
complex enough switch statements.
---
 llvm/lib/Transforms/Scalar/DFAJumpThreading.cpp | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/llvm/lib/Transforms/Scalar/DFAJumpThreading.cpp b/llvm/lib/Transforms/Scalar/DFAJumpThreading.cpp
index 938aab5879044..ac7af712bcf12 100644
--- a/llvm/lib/Transforms/Scalar/DFAJumpThreading.cpp
+++ b/llvm/lib/Transforms/Scalar/DFAJumpThreading.cpp
@@ -618,6 +618,8 @@ struct AllSwitchPaths {
 
     VisitedBlocks UniqueBlocks;
     for (auto *IncomingBB : Phi->blocks()) {
+      if(Res.size() >= MaxNumPaths)
+        break;
       if (!UniqueBlocks.insert(IncomingBB).second)
         continue;
       if (!SwitchOuterLoop->contains(IncomingBB))
@@ -657,6 +659,8 @@ struct AllSwitchPaths {
             getPathsFromStateDefMap(StateDef, IncomingPhi, VB);
         for (ThreadingPath &Path : PredPaths) {
           Path.push_back(PhiBB);
+          if(Res.size() >= MaxNumPaths)
+            break;
           Res.push_back(std::move(Path));
         }
         continue;
@@ -679,6 +683,10 @@ struct AllSwitchPaths {
           ThreadingPath NewPath(Path);
           NewPath.appendExcludingFirst(IPath);
           NewPath.push_back(PhiBB);
+          if(Res.size() >= MaxNumPaths) {
+            VB.erase(PhiBB);
+            return Res;
+	  }
           Res.push_back(NewPath);
         }
       }