[llvm] [IPO] Fix infinite recursive inlining in ModuleInliner (PR #195471)

[via llvm-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-transforms

Author: Kazu Hirata (kazutakahirata)

<details>
<summary>Changes</summary>

The ModuleInliner currently lacks inline history tracking.  Without
it, the inliner can get stuck in an infinite loop when mutually
recursive functions are involved.

This patch enables inline history tracking in the ModuleInliner to
address this issue.

The minsize attribute in the test case lowers the threshold for the
mutually recursive functions, ensuring the bug reproduces in pass
isolation.


---
Full diff: https://github.com/llvm/llvm-project/pull/195471.diff


2 Files Affected:

- (modified) llvm/lib/Transforms/IPO/ModuleInliner.cpp (+3-1) 
- (added) llvm/test/Transforms/Inline/module-inliner-cycle.ll (+34) 


``````````diff
diff --git a/llvm/lib/Transforms/IPO/ModuleInliner.cpp b/llvm/lib/Transforms/IPO/ModuleInliner.cpp
index 79f545a9b3283..d5aa63970de83 100644
--- a/llvm/lib/Transforms/IPO/ModuleInliner.cpp
+++ b/llvm/lib/Transforms/IPO/ModuleInliner.cpp
@@ -206,7 +206,9 @@ PreservedAnalyses ModuleInlinerPass::run(Module &M,
 
     InlineResult IR =
         InlineFunction(*CB, IFI, CtxProf, /*MergeAttributes=*/true,
-                       &FAM.getResult<AAManager>(*CB->getCaller()));
+                       &FAM.getResult<AAManager>(*CB->getCaller()),
+                       /*InsertLifetime=*/true,
+                       /*TrackInlineHistory=*/true);
     if (!IR.isSuccess()) {
       Advice->recordUnsuccessfulInlining(IR);
       continue;
diff --git a/llvm/test/Transforms/Inline/module-inliner-cycle.ll b/llvm/test/Transforms/Inline/module-inliner-cycle.ll
new file mode 100644
index 0000000000000..f8198ddeae1f9
--- /dev/null
+++ b/llvm/test/Transforms/Inline/module-inliner-cycle.ll
@@ -0,0 +1,34 @@
+; RUN: opt < %s -passes='module-inline' -S | FileCheck %s
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-pc-linux-gnu"
+
+define ptr @foo() {
+; CHECK: define ptr @foo()
+; CHECK: %{{.*}} = call ptr @bar(), !inline_history ![[HIST:[0-9]+]]
+entry:
+  %0 = call ptr @bar()
+  ret ptr %0
+}
+
+; Function Attrs: minsize
+define ptr @bar() #0 {
+entry:
+  %0 = load i64, ptr null, align 8
+  call void @baz()
+  ret ptr null
+}
+
+; Function Attrs: minsize
+define void @baz() #0 {
+entry:
+  br label %loop
+
+loop:                                             ; preds = %loop, %entry
+  %0 = call ptr @bar()
+  br label %loop
+}
+
+attributes #0 = { minsize }
+
+; CHECK: ![[HIST]] = !{ptr @{{baz|bar}}, ptr @{{baz|bar}}}

``````````

</details>


https://github.com/llvm/llvm-project/pull/195471