[llvm] [X86] Reuse reserved call frames in Win64 exception funclets (PR #226340)

[via llvm-commits]

https://github.com/kingakasa created https://github.com/llvm/llvm-project/pull/226340

Win64 exception funclets already reserve outgoing argument space in their prologues, including when the parent function has a variable-sized allocation. However, call-frame pseudo elimination uses the parent function's reserved-frame decision and can emit additional per-call stack adjustments inside the funclet. These adjustments disagree with its unwind information.

Use EH-scope membership to recognize funclet blocks and reuse their reserved call frames. Add a regression with a dynamic alloca and a catch funclet making a call with stack arguments.

Validation:
- The new regression fails on the baseline at `-O0` because of the extra stack adjustment; the patched compiler passes at `-O0` and `-O2`.
- Full X86 CodeGen suite: 5,668 passed, 15 expected failures, 3 unsupported; no unexpected failures.
- CodeGen unit tests: 274 passed, 319 internally skipped.

Related: #191616 addresses the wider Win64 dynamic-allocation/call-frame problem. This change is limited to eliminating call-frame pseudos within exception funclets; it does not change parent-frame dynamic-allocation lowering.

Tested independently on Windows x64 using a fresh Release LLVM/Clang build with assertions enabled and a matching Release ASan runtime, based on `6dc4a4ba2850369afd537001377fb9d404653d26`.

Changed-line clang-format and git diff --check pass.

Assisted-by: OpenAI Codex


>From fa4533198eea106c180722bf31b0d32bb1ca4bee Mon Sep 17 00:00:00 2001
From: Kinga Kasa <kasakinga14 at gmail.com>
Date: Thu, 24 Sep 2026 22:50:02 -0400
Subject: [PATCH] [X86] Reuse reserved call frames in Win64 exception funclets

Win64 exception funclets already reserve outgoing argument space in their prologues, including when the parent function has a variable-sized allocation. However, call-frame pseudo elimination uses the parent function's reserved-frame decision and can emit additional per-call stack adjustments inside the funclet. These adjustments disagree with its unwind information.

Use EH-scope membership to recognize funclet blocks and reuse their reserved call frames. Add a regression with a dynamic alloca and a catch funclet making a call with stack arguments.

Validation:
- The new regression fails on the baseline at `-O0` because of the extra stack adjustment; the patched compiler passes at `-O0` and `-O2`.
- Full X86 CodeGen suite: 5,668 passed, 15 expected failures, 3 unsupported; no unexpected failures.
- CodeGen unit tests: 274 passed, 319 internally skipped.

Related: #191616 addresses the wider Win64 dynamic-allocation/call-frame problem. This change is limited to eliminating call-frame pseudos within exception funclets; it does not change parent-frame dynamic-allocation lowering.

Tested independently on Windows x64 using a fresh Release LLVM/Clang build with assertions enabled and a matching Release ASan runtime, based on `6dc4a4ba2850369afd537001377fb9d404653d26`.

Changed-line clang-format and git diff --check pass.

Assisted-by: OpenAI Codex
---
 llvm/lib/Target/X86/X86FrameLowering.cpp      | 11 ++++++++
 .../X86/win64-funclet-dynamic-call-frame.ll   | 26 +++++++++++++++++++
 2 files changed, 37 insertions(+)
 create mode 100644 llvm/test/CodeGen/X86/win64-funclet-dynamic-call-frame.ll

diff --git a/llvm/lib/Target/X86/X86FrameLowering.cpp b/llvm/lib/Target/X86/X86FrameLowering.cpp
index a25aba6d0afe0..9798a2b69f2bd 100644
--- a/llvm/lib/Target/X86/X86FrameLowering.cpp
+++ b/llvm/lib/Target/X86/X86FrameLowering.cpp
@@ -19,6 +19,7 @@
 #include "X86Subtarget.h"
 #include "X86TargetMachine.h"
 #include "llvm/ADT/Statistic.h"
+#include "llvm/CodeGen/Analysis.h"
 #include "llvm/CodeGen/LivePhysRegs.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/MachineFunction.h"
@@ -4035,6 +4036,16 @@ MachineBasicBlock::iterator X86FrameLowering::eliminateCallFramePseudoInstr(
     MachineFunction &MF, MachineBasicBlock &MBB,
     MachineBasicBlock::iterator I) const {
   bool reserveCallFrame = hasReservedCallFrame(MF);
+  // Win64 funclet prologues reserve their outgoing argument space even when
+  // the parent frame has a variable-sized allocation. Their frame pointer
+  // addresses the parent frame, so additional stack adjustments cannot be
+  // described by the funclet's unwind information.
+  if (!reserveCallFrame && STI.isTargetWin64() && MF.hasEHFunclets()) {
+    auto Membership = getEHScopeMembership(MF);
+    auto Scope = Membership.find(&MBB);
+    if (Scope != Membership.end() && Scope->second != MF.front().getNumber())
+      reserveCallFrame = true;
+  }
   unsigned Opcode = I->getOpcode();
   bool isDestroy = Opcode == TII.getCallFrameDestroyOpcode();
   DebugLoc DL = I->getDebugLoc(); // copy DebugLoc as I will be erased.
diff --git a/llvm/test/CodeGen/X86/win64-funclet-dynamic-call-frame.ll b/llvm/test/CodeGen/X86/win64-funclet-dynamic-call-frame.ll
new file mode 100644
index 0000000000000..6dd92a00297a2
--- /dev/null
+++ b/llvm/test/CodeGen/X86/win64-funclet-dynamic-call-frame.ll
@@ -0,0 +1,26 @@
+; RUN: llc -mtriple=x86_64-pc-windows-msvc -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-pc-windows-msvc -O2 < %s | FileCheck %s
+; Funclets already reserve outgoing argument space in their prologues, even
+; when the parent has a variable-sized stack allocation.
+declare i32 @__CxxFrameHandler3(...)
+declare void @use(ptr)
+declare void @may_throw(i64, i64, i64, i64, i64, i64)
+define void @dynamic_catch(i64 %n) personality ptr @__CxxFrameHandler3 {
+entry:
+  %a = alloca i8, i64 %n, align 32
+  invoke void @use(ptr %a) to label %done unwind label %dispatch
+dispatch:
+  %switch = catchswitch within none [label %catch] unwind to caller
+catch:
+  %pad = catchpad within %switch [ptr null, i32 64, ptr null]
+  call void @may_throw(i64 1, i64 2, i64 3, i64 4, i64 5, i64 6) [ "funclet"(token %pad) ]
+  catchret from %pad to label %done
+done:
+  ret void
+}
+; CHECK-LABEL: "?catch$
+; CHECK: .seh_endprologue
+; CHECK-NOT: subq {{.*}}%rsp
+; CHECK: callq may_throw
+; CHECK-NOT: addq {{.*}}%rsp
+; CHECK: .seh_startepilogue