[llvm] [FuncAttrs] Relax norecurse attribute inference (PR #139943)

[Usha Gupta via llvm-commits]

================
@@ -2322,8 +2343,39 @@ PreservedAnalyses PostOrderFunctionAttrsPass::run(LazyCallGraph::SCC &C,
     Functions.push_back(&N.getFunction());
   }
 
-  auto ChangedFunctions =
-      deriveAttrsInPostOrder(Functions, AARGetter, ArgAttrsOnly);
+  bool NoFunctionsAddressIsTaken = false;
+  // Check if any function in the whole program has its address taken or has
+  // potentially external linkage.
+  // We use this information when inferring norecurse attribute: If there is
+  // no function whose address is taken and all functions have internal
+  // linkage, there is no path for a callback to any user function.
+  if (IsLTOPostLink || ForceLTOFuncAttrs) {
+    bool AnyFunctionsAddressIsTaken = false;
+    // Get the parent Module of the Function
+    Module &M = *C.begin()->getFunction().getParent();
+    for (Function &F : M) {
+      // We only care about functions defined in user program whose addresses
+      // escape, making them potential callback targets.
+      if (F.isDeclaration())
+        continue;
+
+      // If the function is already marked as norecurse, this should not block
+      // norecurse inference even though it may have external linkage.
+      // For ex: main() in C++.
+      if (F.doesNotRecurse())
+        continue;
----------------
usha1830 wrote:

@nikic Yes. I think relying on existing `norecurse `on callers/callees is not correct.

In my initial commit, I implemented a graph walk to detect unknown external calls throughout the call chain of the current Node `(callsUnknownExternal(LazyCallGraph::Node *)).`

Perhaps, we could store this information at the Node level in the LazyCallGraph, and reuse it here?

I noticed the _populateSlow_() function in [LazyCallGraph.cpp](https://github.com/llvm/llvm-project/blob/main/llvm/lib/Analysis/LazyCallGraph.cpp#L72). It walks through all instructions in a Function and builds the graph edges. We could add a new member **unknownExternalCall** to the Node class and populate it at this stage.

We might also consider validating the _LazyCallGraph_ after module merging during LTO post-link. (This might already be happening, but I’m not certain.)

By adding a check for unknown external calls in both the PO and RPO function attribute passes, we can prevent the `_norecurse_ `attribute from being applied to function `a` (tested for the two examples in discussion). However, this also prevents `_norecurse_ `from being added to `norecurse_fn()`, which is otherwise eligible (as it calls `a(0)`)


https://github.com/llvm/llvm-project/pull/139943