[llvm] [Inline][WinEH] Fix try scopes leaking to caller on inline (PR #164170)

[via llvm-commits]

================
@@ -2210,6 +2211,135 @@ inlineRetainOrClaimRVCalls(CallBase &CB, objcarc::ARCInstKind RVCallKind,
   }
 }
 
+// Determine SEH try scopes and order them by dominance.
+static SmallVector<llvm::InvokeInst *, 1>
+GetOrderedSehTryScopes(Function *Func, DominatorTree &DT) {
+  SmallVector<llvm::InvokeInst *, 1> Scopes{};
+  bool DTCalculated = false;
+
+  for (auto &BB : *Func) {
+    auto *TI = BB.getTerminator();
+    if (auto *II = dyn_cast<InvokeInst>(TI)) {
+      auto *Call = cast<CallBase>(II);
+      const auto *Fn = Call->getCalledFunction();
+      if (!Fn || !Fn->isIntrinsic())
+        continue;
+
+      if (Fn->getIntrinsicID() != Intrinsic::seh_try_begin)
+        continue;
+
+      if (!DTCalculated) {
+        DT.recalculate(*Func);
+        DTCalculated = true;
+      }
+
+      auto InsertIt = Scopes.end();
+      if (!Scopes.empty())
+        for (auto ScopeIt = Scopes.begin(); ScopeIt != Scopes.end(); ++ScopeIt)
+          if (DT.dominates((*ScopeIt)->getParent(), &BB))
+            InsertIt = std::next(ScopeIt);
+
+      Scopes.insert(InsertIt, II);
+    }
+  }
+
+  return Scopes;
+}
+
+// Find, if present, the outermost unterminated try scope for the input block.
+static llvm::InvokeInst *GetOutermostUnterminatedTryScopeBegin(
+    llvm::BasicBlock *ReturnBlock, SmallVector<llvm::InvokeInst *, 1> &Scopes,
+    DominatorTree &DT) {
+  llvm::InvokeInst *UnterminatedScope{nullptr};
+
+  for (auto ScopeIt = Scopes.rbegin(); ScopeIt != Scopes.rend(); ++ScopeIt) {
+    auto *Invoke = *ScopeIt;
+
+    // Return might not be in a try scope.
+    if (!DT.dominates(Invoke->getParent(), ReturnBlock))
+      continue;
+
+    // If there is a catch which connects to the return, the try scope is
+    // considered to be terminated.
----------------
MuellerMP wrote:

You are right that I missed the seh.try.end here.
The seh.try.end is actually only emitted for SEH (__try /__finally / __except) and not Cpp EH (try/catch).
This will also only be emitted for non cxx unwind personalities.
But since I didn't explicitly check for a CXX personality the algorithm is currently wrong.
Since this fix also works for SEH and non cpp personalities (as one can see with this sample https://godbolt.org/z/Eco55qYMv), this should also handle seh.try.end's correctly.

As to when seh.try.end's are emitted: usually for conditionally executed ctors inside of a __try scope.

Now regarding the catch connection:
So the `calculateCXXStateForAsynchEH` function in [WinEHPrepare](https://github.com/llvm/llvm-project/blob/main/llvm/lib/CodeGen/WinEHPrepare.cpp#L264) tells us that cleanupreturns and catch returns can also be an explicit scope end. (the `ToState` represents the parent state - as seen in `calculateCXXStateNumbers`)

After a catchret or cleanupret there might be a non trivial connection back to the return block, thats why i chose the isPotentiallyReachable here.



https://github.com/llvm/llvm-project/pull/164170