[llvm] [WebAssembly] Fix unwind mismatches in new EH (PR #114361)
Heejin Ahn via llvm-commits
llvm-commits at lists.llvm.org
Mon Nov 4 20:27:26 PST 2024
https://github.com/aheejin updated https://github.com/llvm/llvm-project/pull/114361
>From b8d03287b20565c17f3b0167f7dd47391ab0fd53 Mon Sep 17 00:00:00 2001
From: Heejin Ahn <aheejin at gmail.com>
Date: Thu, 31 Oct 2024 05:04:26 +0000
Subject: [PATCH 1/4] [WebAssembly] Fix unwind mismatches in new EH
This fixes unwind mismatches for the new EH spec.
The main flow is similar to that of the legacy EH's unwind mismatch
fixing. The new EH shared `fixCallUnwindMismatches` and
`fixCatchUnwindMismatches` functions, which gather the range of
instructions we need to fix their unwind destination for, with the
legacy EH. But unlike the legacy EH that uses `try`-`delegate`s to fix
them, the new EH wrap those instructions with nested
`try_table`-`end_try_table`s that jump to a "trampoline" BB, where we
rethrow (using a `throw_ref`) the exception to the correct `try_table`.
For a simple example of a call unwind mismatch, suppose if `call foo`
should unwind to the outer `try_table` but is wrapped in another
`try_table` (not shown here):
```wast
try_table
...
call foo ;; Unwind mismatch. Should unwind to the outer try_table
...
end_try_table
```
Then we wrap the call with a new nested `try_table`-`end_try_table`, add
a `block` / `end_block` right inside the target `try_table`, and make
the nested `try_table` jump to it using a `catch_all_ref` clause, and
rethrow the exception using a `throw_ref`:
```wast
try_table
block $l0 exnref
...
try_table (catch_all_ref $l0)
call foo
end_try_table
...
end_block ;; Trampoline BB
throw_ref
end_try_table
```
---
This fixes two existing bugs. These are not easy to test independently
without the unwind mismatch fixing. The first one is how we calculate
`ScopeTops`. Turns out, we should do it in the same way as in the legacy
EH even though there is no `end_try` at the end of `catch` block
anymore. `nested_try` in `cfg-stackify-eh.ll` tests this case.
The second bug is in `rewriteDepthImmediates`. `try_table`'s immediates
should be computed without the `try_table` itself, meaning
```wast
block
try_table (catch ... 0)
end_try_table
end_block
```
Here 0 should target not `end_try_table` but `end_block`. This bug
didn't crash the program because `placeTryTableMarker` generated only
the simple form of `try_table` that has a single catch clause and an
`end_block` follows right after the `end_try_table` in the same BB, so
jumping to an `end_try_table` is the same as jumping to the `end_block`.
But now we generate `catch` clauses with depths greater than 0 with when
fixing unwind mismatches, which uncovered this bug.
---
One case that needs a special treatment was when `end_loop` precedes an
`end_try_table` within a BB and this BB is a (true) unwind destination
when fixing unwind mismatches. In this case we need to split this
`end_loop` into a predecessor BB. This case is tested in
`unwind_mismatches_with_loop` in `cfg-stackify-eh.ll`.
---
`cfg-stackify-eh.ll` contains mostly the same set of tests with the
existing `cfg-stackify-eh-legacy.ll` with the updated FileCheck
expectations. As in `cfg-stackify-eh-legacy.ll`, the FileCheck lines
mostly only contain control flow instructions and calls for
readability.
- `nested_try` and `unwind_mismatches_with_loop` are added to test newly
found bugs in the new EH.
- Some tests in `cfg-stackify-eh-legacy.ll` about the legacy-EH-specific
asepcts have not been added to `cfg-stackify-eh.ll`.
(`remove_unnecessary_instrs`, `remove_unnecessary_br`,
`fix_function_end_return_type_with_try_catch`, and
`branch_remapping_after_fixing_unwind_mismatches_0/1`)
---
.../MCTargetDesc/WebAssemblyMCTargetDesc.h | 30 +
.../WebAssembly/WebAssemblyCFGStackify.cpp | 653 ++++++-
.../CodeGen/WebAssembly/cfg-stackify-eh.ll | 1555 +++++++++++++++++
3 files changed, 2205 insertions(+), 33 deletions(-)
create mode 100644 llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll
diff --git a/llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h b/llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h
index e3a60fa4812d8f..3900d4a0aa7044 100644
--- a/llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h
+++ b/llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h
@@ -478,6 +478,22 @@ inline bool isMarker(unsigned Opc) {
}
}
+inline bool isEndMarker(unsigned Opc) {
+ switch (Opc) {
+ case WebAssembly::END_BLOCK:
+ case WebAssembly::END_BLOCK_S:
+ case WebAssembly::END_LOOP:
+ case WebAssembly::END_LOOP_S:
+ case WebAssembly::END_TRY:
+ case WebAssembly::END_TRY_S:
+ case WebAssembly::END_TRY_TABLE:
+ case WebAssembly::END_TRY_TABLE_S:
+ return true;
+ default:
+ return false;
+ }
+}
+
inline bool isTry(unsigned Opc) {
switch (Opc) {
case WebAssembly::TRY:
@@ -510,6 +526,20 @@ inline bool isCatch(unsigned Opc) {
}
}
+inline bool isCatchAll(unsigned Opc) {
+ switch (Opc) {
+ case WebAssembly::CATCH_ALL_LEGACY:
+ case WebAssembly::CATCH_ALL_LEGACY_S:
+ case WebAssembly::CATCH_ALL:
+ case WebAssembly::CATCH_ALL_S:
+ case WebAssembly::CATCH_ALL_REF:
+ case WebAssembly::CATCH_ALL_REF_S:
+ return true;
+ default:
+ return false;
+ }
+}
+
inline bool isLocalGet(unsigned Opc) {
switch (Opc) {
case WebAssembly::LOCAL_GET_I32:
diff --git a/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp b/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
index a5f73fabca3542..6800fb614300fa 100644
--- a/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
+++ b/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
@@ -78,13 +78,19 @@ class WebAssemblyCFGStackify final : public MachineFunctionPass {
void placeTryMarker(MachineBasicBlock &MBB);
void placeTryTableMarker(MachineBasicBlock &MBB);
- // Exception handling related functions
+ // Unwind mismatch fixing for exception handling
+ // - Common functions
bool fixCallUnwindMismatches(MachineFunction &MF);
bool fixCatchUnwindMismatches(MachineFunction &MF);
+ void recalculateScopeTops(MachineFunction &MF);
+ // - Legacy EH
void addNestedTryDelegate(MachineInstr *RangeBegin, MachineInstr *RangeEnd,
MachineBasicBlock *UnwindDest);
- void recalculateScopeTops(MachineFunction &MF);
void removeUnnecessaryInstrs(MachineFunction &MF);
+ // - Standard EH (exnref)
+ void addNestedTryTable(MachineInstr *RangeBegin, MachineInstr *RangeEnd,
+ MachineBasicBlock *UnwindDest);
+ MachineBasicBlock *getTrampolineBlock(MachineBasicBlock *UnwindDest);
// Wrap-up
using EndMarkerInfo =
@@ -111,6 +117,9 @@ class WebAssemblyCFGStackify final : public MachineFunctionPass {
// <EH pad, TRY marker> map
DenseMap<const MachineBasicBlock *, MachineInstr *> EHPadToTry;
+ DenseMap<const MachineBasicBlock *, MachineBasicBlock *>
+ UnwindDestToTrampoline;
+
// We need an appendix block to place 'end_loop' or 'end_try' marker when the
// loop / exception bottom block is the last block in a function
MachineBasicBlock *AppendixBB = nullptr;
@@ -119,11 +128,27 @@ class WebAssemblyCFGStackify final : public MachineFunctionPass {
AppendixBB = MF.CreateMachineBasicBlock();
// Give it a fake predecessor so that AsmPrinter prints its label.
AppendixBB->addSuccessor(AppendixBB);
- MF.push_back(AppendixBB);
+ // If the caller trampoline BB exists, insert the appendix BB before it.
+ // Otherwise insert it at the end of the function.
+ if (CallerTrampolineBB)
+ MF.insert(CallerTrampolineBB->getIterator(), AppendixBB);
+ else
+ MF.push_back(AppendixBB);
}
return AppendixBB;
}
+ // Create a caller-dedicated trampoline BB to be used for fixing unwind
+ // mismatches where the unwind destination is the caller.
+ MachineBasicBlock *CallerTrampolineBB = nullptr;
+ MachineBasicBlock *getCallerTrampolineBlock(MachineFunction &MF) {
+ if (!CallerTrampolineBB) {
+ CallerTrampolineBB = MF.CreateMachineBasicBlock();
+ MF.push_back(CallerTrampolineBB);
+ }
+ return CallerTrampolineBB;
+ }
+
// Before running rewriteDepthImmediates function, 'delegate' has a BB as its
// destination operand. getFakeCallerBlock() returns a fake BB that will be
// used for the operand when 'delegate' needs to rethrow to the caller. This
@@ -691,12 +716,20 @@ void WebAssemblyCFGStackify::placeTryTableMarker(MachineBasicBlock &MBB) {
if (!Header)
return;
- assert(&MBB != &MF.front() && "Header blocks shouldn't have predecessors");
- MachineBasicBlock *LayoutPred = MBB.getPrevNode();
+ // Unlike the end_try marker, we don't place an end marker at the end of
+ // exception bottom, i.e., at the end of the old 'catch' block. But we still
+ // consider the try-catch part as a scope when computing ScopeTops.
+ WebAssemblyException *WE = WEI.getExceptionFor(&MBB);
+ assert(WE);
+ MachineBasicBlock *Bottom = SRI.getBottom(WE);
+ auto Iter = std::next(Bottom->getIterator());
+ if (Iter == MF.end())
+ Iter--;
+ MachineBasicBlock *Cont = &*Iter;
// If the nearest common dominator is inside a more deeply nested context,
// walk out to the nearest scope which isn't more deeply nested.
- for (MachineFunction::iterator I(LayoutPred), E(Header); I != E; --I) {
+ for (MachineFunction::iterator I(Bottom), E(Header); I != E; --I) {
if (MachineBasicBlock *ScopeTop = ScopeTops[I->getNumber()]) {
if (ScopeTop->getNumber() > Header->getNumber()) {
// Skip over an intervening scope.
@@ -905,14 +938,52 @@ void WebAssemblyCFGStackify::placeTryTableMarker(MachineBasicBlock &MBB) {
BuildMI(MBB, InsertPos, MBB.findPrevDebugLoc(InsertPos),
TII.get(WebAssembly::END_BLOCK));
registerScope(Block, EndBlock);
+
// Track the farthest-spanning scope that ends at this point.
- updateScopeTops(Header, &MBB);
+ // Unlike the end_try, even if we don't put a end marker at the end of catch
+ // block, we still have to create two mappings: (BB with 'end_try_table' -> BB
+ // with 'try_table') and (BB after the (conceptual) catch block -> BB with
+ // 'try_table').
+ //
+ // This is what can happen if we don't create the latter mapping:
+ //
+ // Suppoe in the legacy EH we have this code:
+ // try
+ // try
+ // code1
+ // catch (a)
+ // end_try
+ // code2
+ // catch (b)
+ // end_try
+ //
+ // If we don't create the latter mapping, try_table markers would be placed
+ // like this:
+ // try_table
+ // code1
+ // end_try_table (a)
+ // try_table
+ // code2
+ // end_try_table (b)
+ //
+ // This does not reflect the original structure, and more important problem
+ // is, in case 'code1' has an unwind mismatch and should unwind to
+ // 'end_try_table (b)' rather than 'end_try_table (a)', we don't have a way to
+ // make it jump after 'end_try_table (b)' without creating another block. So
+ // even if we don't place 'end_try' marker at the end of 'catch' block
+ // anymore, we create ScopeTops mapping the same way as the legacy exception,
+ // so the resulting code will look like:
+ // try_table
+ // try_table
+ // code1
+ // end_try_table (a)
+ // code2
+ // end_try_table (b)
+ for (auto *End : {&MBB, Cont})
+ updateScopeTops(Header, End);
}
void WebAssemblyCFGStackify::removeUnnecessaryInstrs(MachineFunction &MF) {
- if (WebAssembly::WasmEnableExnref)
- return;
-
const auto &TII = *MF.getSubtarget<WebAssemblySubtarget>().getInstrInfo();
// When there is an unconditional branch right before a catch instruction and
@@ -1215,7 +1286,291 @@ void WebAssemblyCFGStackify::addNestedTryDelegate(
registerTryScope(Try, Delegate, nullptr);
}
+// Given an unwind destination, return a trampoline BB. A trampoline BB is a
+// destination of a nested try_table inserted to fix an unwind mismatch. It
+// contains an end_block, which is the target of the try_table, and a throw_ref,
+// to rethrow the exception to the right try_table.
+// try_table (catch ... )
+// block exnref
+// ...
+// try_table (catch_all_ref N)
+// some code
+// end_try_table
+// ...
+// end_block ;; Trampoline BB
+// throw_fef
+// end_try_table
+MachineBasicBlock *
+WebAssemblyCFGStackify::getTrampolineBlock(MachineBasicBlock *UnwindDest) {
+ // We need one trampoline BB per an unwind destination, even though there are
+ // multiple try_tables target the same unwind destination. If we have already
+ // created one for the given UnwindDest, return it.
+ auto It = UnwindDestToTrampoline.find(UnwindDest);
+ if (It != UnwindDestToTrampoline.end())
+ return It->second;
+
+ auto &MF = *UnwindDest->getParent();
+ auto &MRI = MF.getRegInfo();
+ const auto &TII = *MF.getSubtarget<WebAssemblySubtarget>().getInstrInfo();
+
+ MachineInstr *Block = nullptr;
+ MachineBasicBlock *TrampolineBB = nullptr;
+ DebugLoc EndDebugLoc;
+
+ if (UnwindDest == getFakeCallerBlock(MF)) {
+ // If the unwind destination is the caller, create a caller-dedicated
+ // trampoline BB at the end of the function and wrap the whole function with
+ // a block.
+ auto BeginPos = MF.begin()->begin();
+ while (WebAssembly::isArgument(BeginPos->getOpcode()))
+ BeginPos++;
+ Block = BuildMI(*MF.begin(), BeginPos, MF.begin()->begin()->getDebugLoc(),
+ TII.get(WebAssembly::BLOCK))
+ .addImm(int64_t(WebAssembly::BlockType::Exnref));
+ TrampolineBB = getCallerTrampolineBlock(MF);
+ MachineBasicBlock *PrevBB = &*std::prev(CallerTrampolineBB->getIterator());
+ EndDebugLoc = PrevBB->findPrevDebugLoc(PrevBB->end());
+ } else {
+ // If the unwind destination is another EH pad, create a trampoline BB for
+ // the unwind dest and insert a block instruction right after the target
+ // try_table.
+ auto *TargetBeginTry = EHPadToTry[UnwindDest];
+ auto *TargetEndTry = BeginToEnd[TargetBeginTry];
+ auto *TargetBeginBB = TargetBeginTry->getParent();
+ auto *TargetEndBB = TargetEndTry->getParent();
+
+ Block = BuildMI(*TargetBeginBB, std::next(TargetBeginTry->getIterator()),
+ TargetBeginTry->getDebugLoc(), TII.get(WebAssembly::BLOCK))
+ .addImm(int64_t(WebAssembly::BlockType::Exnref));
+ TrampolineBB = MF.CreateMachineBasicBlock();
+ EndDebugLoc = TargetEndTry->getDebugLoc();
+ MF.insert(TargetEndBB->getIterator(), TrampolineBB);
+ TrampolineBB->addSuccessor(UnwindDest);
+ }
+
+ // Insert an end_block, catch_all_ref (pseudo instruction), and throw_ref
+ // instructions in the trampoline BB.
+ MachineInstr *EndBlock =
+ BuildMI(TrampolineBB, EndDebugLoc, TII.get(WebAssembly::END_BLOCK));
+ auto ExnReg = MRI.createVirtualRegister(&WebAssembly::EXNREFRegClass);
+ BuildMI(TrampolineBB, EndDebugLoc, TII.get(WebAssembly::CATCH_ALL_REF))
+ .addDef(ExnReg);
+ BuildMI(TrampolineBB, EndDebugLoc, TII.get(WebAssembly::THROW_REF))
+ .addReg(ExnReg);
+
+ registerScope(Block, EndBlock);
+ UnwindDestToTrampoline[UnwindDest] = TrampolineBB;
+ return TrampolineBB;
+}
+
+// Wrap the given range of instructions with a try_table-end_try_table that
+// targets 'UnwindDest'. RangeBegin and RangeEnd are inclusive.
+void WebAssemblyCFGStackify::addNestedTryTable(MachineInstr *RangeBegin,
+ MachineInstr *RangeEnd,
+ MachineBasicBlock *UnwindDest) {
+ auto *BeginBB = RangeBegin->getParent();
+ auto *EndBB = RangeEnd->getParent();
+
+ MachineFunction &MF = *BeginBB->getParent();
+ const auto &MFI = *MF.getInfo<WebAssemblyFunctionInfo>();
+ const auto &TII = *MF.getSubtarget<WebAssemblySubtarget>().getInstrInfo();
+
+ // Get the trampoline BB that the new try_table will unwind to.
+ auto *TrampolineBB = getTrampolineBlock(UnwindDest);
+
+ // Local expression tree before the first call of this range should go
+ // after the nested TRY_TABLE.
+ SmallPtrSet<const MachineInstr *, 4> AfterSet;
+ AfterSet.insert(RangeBegin);
+ for (auto I = MachineBasicBlock::iterator(RangeBegin), E = BeginBB->begin();
+ I != E; --I) {
+ if (std::prev(I)->isDebugInstr() || std::prev(I)->isPosition())
+ continue;
+ if (WebAssembly::isChild(*std::prev(I), MFI))
+ AfterSet.insert(&*std::prev(I));
+ else
+ break;
+ }
+
+ // Create the nested try_table instruction.
+ auto TryTablePos = getLatestInsertPos(
+ BeginBB, SmallPtrSet<const MachineInstr *, 4>(), AfterSet);
+ MachineInstr *TryTable =
+ BuildMI(*BeginBB, TryTablePos, RangeBegin->getDebugLoc(),
+ TII.get(WebAssembly::TRY_TABLE))
+ .addImm(int64_t(WebAssembly::BlockType::Void))
+ .addImm(1) // # of catch clauses
+ .addImm(wasm::WASM_OPCODE_CATCH_ALL_REF)
+ .addMBB(TrampolineBB);
+
+ // Create a BB to insert the 'end_try_table' instruction.
+ MachineBasicBlock *EndTryTableBB = MF.CreateMachineBasicBlock();
+ EndTryTableBB->addSuccessor(TrampolineBB);
+
+ auto SplitPos = std::next(RangeEnd->getIterator());
+ if (SplitPos == EndBB->end()) {
+ // If the range's end instruction is at the end of the BB, insert the new
+ // end_try_table BB after the current BB.
+ MF.insert(std::next(EndBB->getIterator()), EndTryTableBB);
+ EndBB->addSuccessor(EndTryTableBB);
+
+ } else {
+ // When the split pos is in the middle of a BB, we split the BB into two and
+ // put the 'end_try_table' BB in between. We normally create a split BB and
+ // make it a successor of the original BB (CatchAfterSplit == false), but in
+ // case the BB is an EH pad and there is a 'catch' after split pos
+ // (CatchAfterSplit == true), we should preserve the BB's property,
+ // including that it is an EH pad, in the later part of the BB, where the
+ // 'catch' is.
+ bool CatchAfterSplit = false;
+ if (EndBB->isEHPad()) {
+ for (auto I = MachineBasicBlock::iterator(SplitPos), E = EndBB->end();
+ I != E; ++I) {
+ if (WebAssembly::isCatch(I->getOpcode())) {
+ CatchAfterSplit = true;
+ break;
+ }
+ }
+ }
+
+ MachineBasicBlock *PreBB = nullptr, *PostBB = nullptr;
+ if (!CatchAfterSplit) {
+ // If the range's end instruction is in the middle of the BB, we split the
+ // BB into two and insert the end_try_table BB in between.
+ // - Before:
+ // bb:
+ // range_end
+ // other_insts
+ //
+ // - After:
+ // pre_bb: (previous 'bb')
+ // range_end
+ // end_try_table_bb: (new)
+ // end_try_table
+ // post_bb: (new)
+ // other_insts
+ PreBB = EndBB;
+ PostBB = MF.CreateMachineBasicBlock();
+ MF.insert(std::next(PreBB->getIterator()), PostBB);
+ MF.insert(std::next(PreBB->getIterator()), EndTryTableBB);
+ PostBB->splice(PostBB->end(), PreBB, SplitPos, PreBB->end());
+ PostBB->transferSuccessors(PreBB);
+ } else {
+ // - Before:
+ // ehpad:
+ // range_end
+ // catch
+ // ...
+ //
+ // - After:
+ // pre_bb: (new)
+ // range_end
+ // end_try_table: (new)
+ // end_try_table
+ // post_bb: (previous 'ehpad')
+ // catch
+ // ...
+ assert(EndBB->isEHPad());
+ PreBB = MF.CreateMachineBasicBlock();
+ PostBB = EndBB;
+ MF.insert(PostBB->getIterator(), PreBB);
+ MF.insert(PostBB->getIterator(), EndTryTableBB);
+ PreBB->splice(PreBB->end(), PostBB, PostBB->begin(), SplitPos);
+ // We don't need to transfer predecessors of the EH pad to 'PreBB',
+ // because an EH pad's predecessors are all through unwind edges and they
+ // should still unwind to the EH pad, not PreBB.
+ }
+ unstackifyVRegsUsedInSplitBB(*PreBB, *PostBB);
+ PreBB->addSuccessor(EndTryTableBB);
+ PreBB->addSuccessor(PostBB);
+ }
+
+ // Add a 'try_table' instruction in the delegate BB created above.
+ MachineInstr *EndTryTable = BuildMI(EndTryTableBB, RangeEnd->getDebugLoc(),
+ TII.get(WebAssembly::END_TRY_TABLE));
+ registerTryScope(TryTable, EndTryTable, nullptr);
+}
+
+// In the standard (exnref) EH, we fix unwind mismatches by adding a new
+// block~end_block inside of the unwind destination try_table~end_try_table:
+// try_table ...
+// block exnref ;; (new)
+// ...
+// try_table (catch_all_ref N) ;; (new) to trampoline BB
+// code
+// end_try_table ;; (new)
+// ...
+// end_block ;; (new) trampoline BB
+// throw_ref ;; (new)
+// end_try_table
+//
+// To do this, we will create a new BB that will contain the new 'end_block' and
+// 'throw_ref' and insert it before the 'end_try_table' BB.
+//
+// But there are cases when there are 'end_loop'(s) before the 'end_try_table'
+// in the same BB. (There can't be 'end_block' before 'end_try_table' in the
+// same BB because EH pads can't be directly branched to.) Then after fixing
+// unwind mismatches this will create the mismatching markers like below:
+// bb0:
+// try_table
+// block exnref
+// ...
+// loop
+// ...
+// new_bb:
+// end_block
+// end_try_table_bb:
+// end_loop
+// end_try_table
+//
+// So if the unwind dest BB has a end_loop before an end_try_table, we split the
+// BB with the end_loop as a separate BB before the end_try_table BB, so that
+// after we fix the unwind mismatch, the code will be like:
+// bb0:
+// try_table
+// block exnref
+// ...
+// loop
+// ...
+// end_loop_bb:
+// end_loop
+// new_bb:
+// end_block
+// end_try_table_bb:
+// end_try_table
+static void splitEndLoopBB(MachineBasicBlock *UnwindDest) {
+ auto &MF = *UnwindDest->getParent();
+ MachineInstr *EndTryTable = nullptr, *EndLoop = nullptr;
+ for (auto &MI : reverse(*UnwindDest)) {
+ if (MI.getOpcode() == WebAssembly::END_TRY_TABLE) {
+ EndTryTable = &MI;
+ continue;
+ }
+ if (EndTryTable && MI.getOpcode() == WebAssembly::END_LOOP) {
+ EndLoop = &MI;
+ break;
+ }
+ }
+ if (!EndLoop)
+ return;
+
+ auto *EndLoopBB = MF.CreateMachineBasicBlock();
+ MF.insert(UnwindDest->getIterator(), EndLoopBB);
+ auto SplitPos = std::next(EndLoop->getIterator());
+ EndLoopBB->splice(EndLoopBB->end(), UnwindDest, UnwindDest->begin(),
+ SplitPos);
+ EndLoopBB->addSuccessor(UnwindDest);
+}
+
bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
+ // This function is used for both the legacy EH and the standard (exnref) EH,
+ // and the reason we have unwind mismatches is the same for the both of them,
+ // but the code examples in the comments are going to be different. To make
+ // the description less confusing, we write the basically same comments twice,
+ // once for the legacy EH and the standard EH.
+ //
+ // -- Legacy EH --------------------------------------------------------------
+ //
// Linearizing the control flow by placing TRY / END_TRY markers can create
// mismatches in unwind destinations for throwing instructions, such as calls.
//
@@ -1334,12 +1689,128 @@ bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
// couldn't happen, because may-throwing instruction there had an unwind
// destination, i.e., it was an invoke before, and there could be only one
// invoke within a BB.)
+ //
+ // -- Standard EH ------------------------------------------------------------
+ //
+ // Linearizing the control flow by placing TRY / END_TRY_TABLE markers can
+ // create mismatches in unwind destinations for throwing instructions, such as
+ // calls.
+ //
+ // We use the a nested 'try_table'~'end_try_table' instruction to fix the
+ // unwind mismatches. try_table's catch clauses take an immediate argument
+ // that specifics which block we should branch to.
+ //
+ // 1. When an instruction may throw, but the EH pad it will unwind to can be
+ // different from the original CFG.
+ //
+ // Example: we have the following CFG:
+ // bb0:
+ // call @foo ; if it throws, unwind to bb2
+ // bb1:
+ // call @bar ; if it throws, unwind to bb3
+ // bb2 (ehpad):
+ // catch
+ // ...
+ // bb3 (ehpad)
+ // catch
+ // ...
+ //
+ // And the CFG is sorted in this order. Then after placing TRY_TABLE markers
+ // (and BLOCK markers for the TRY_TABLE's destinations), it will look like:
+ // (BB markers are omitted)
+ // block
+ // try_table (catch ... 0)
+ // block
+ // try_table (catch ... 0)
+ // call @foo
+ // call @bar ;; if it throws, unwind to bb3
+ // end_try_table
+ // end_block ;; ehpad (bb2)
+ // ...
+ // end_try_table
+ // end_block ;; ehpad (bb3)
+ // ...
+ //
+ // Now if bar() throws, it is going to end up in bb2, not bb3, where it is
+ // supposed to end up. We solve this problem by wrapping the mismatching call
+ // with an inner try_table~end_try_table that sends the exception to the the
+ // 'trampoline' block, which rethrows, or 'bounces' it to the right
+ // end_try_table:
+ // block
+ // try_table (catch ... 0)
+ // block exnref ;; (new)
+ // block
+ // try_table (catch ... 0)
+ // call @foo
+ // try_table (catch_all_ref 2) ;; (new) to trampoline BB
+ // call @bar
+ // end_try_table ;; (new)
+ // end_try_table
+ // end_block ;; ehpad (bb2)
+ // ...
+ // end_block ;; (new) trampoline BB
+ // throw_ref ;; (new)
+ // end_try_table
+ // end_block ;; ehpad (bb3)
+ //
+ // ---
+ // 2. The same as 1, but in this case an instruction unwinds to a caller
+ // function and not another EH pad.
+ //
+ // Example: we have the following CFG:
+ // bb0:
+ // call @foo ; if it throws, unwind to bb2
+ // bb1:
+ // call @bar ; if it throws, unwind to caller
+ // bb2 (ehpad):
+ // catch
+ // ...
+ //
+ // And the CFG is sorted in this order. Then after placing TRY_TABLE markers
+ // (and BLOCK markers for the TRY_TABLE's destinations), it will look like:
+ // block
+ // try_table (catch ... 0)
+ // call @foo
+ // call @bar ;; if it throws, unwind to caller
+ // end_try_table
+ // end_block ;; ehpad (bb2)
+ // ...
+ //
+ // Now if bar() throws, it is going to end up in bb2, when it is supposed
+ // throw up to the caller. We solve this problem in the same way, but in this
+ // case 'delegate's immediate argument is the number of block depths + 1,
+ // which means it rethrows to the caller.
+ // block exnref ;; (new)
+ // block
+ // try_table (catch ... 0)
+ // call @foo
+ // try_table (catch_all_ref 2) ;; (new) to trampoline BB
+ // call @bar
+ // end_try_table ;; (new)
+ // end_try_table
+ // end_block ;; ehpad (bb2)
+ // ...
+ // end_block ;; (new) caller trampoline BB
+ // throw_ref ;; (new) throw to the caller
+ //
+ // Before rewriteDepthImmediates, try_table's catch clauses' argument is a
+ // trampoline BB from which we throw_ref the exception to the right
+ // end_try_table. In case of the caller, it will take a new caller-dedicated
+ // trampoline BB generated by getCallerTrampolineBlock(), which throws the
+ // exception to the caller.
+ //
+ // In case there are multiple calls in a BB that may throw to the caller, they
+ // can be wrapped together in one nested try_table-end_try_table scope. (In 1,
+ // this couldn't happen, because may-throwing instruction there had an unwind
+ // destination, i.e., it was an invoke before, and there could be only one
+ // invoke within a BB.)
SmallVector<const MachineBasicBlock *, 8> EHPadStack;
- // Range of intructions to be wrapped in a new nested try/catch. A range
- // exists in a single BB and does not span multiple BBs.
+ // Range of intructions to be wrapped in a new nested try~delegate or
+ // try_table~end_try_table. A range exists in a single BB and does not span
+ // multiple BBs.
using TryRange = std::pair<MachineInstr *, MachineInstr *>;
- // In original CFG, <unwind destination BB, a vector of try ranges>
+ // In original CFG, <unwind destination BB, a vector of try/try_table ranges>
DenseMap<MachineBasicBlock *, SmallVector<TryRange, 4>> UnwindDestToTryRanges;
// Gather possibly throwing calls (i.e., previously invokes) whose current
@@ -1348,7 +1819,7 @@ bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
for (auto &MBB : reverse(MF)) {
bool SeenThrowableInstInBB = false;
for (auto &MI : reverse(MBB)) {
- if (MI.getOpcode() == WebAssembly::TRY)
+ if (WebAssembly::isTry(MI.getOpcode()))
EHPadStack.pop_back();
else if (WebAssembly::isCatch(MI.getOpcode()))
EHPadStack.push_back(MI.getParent());
@@ -1453,7 +1924,7 @@ bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
}
// Update EHPadStack.
- if (MI.getOpcode() == WebAssembly::TRY)
+ if (WebAssembly::isTry(MI.getOpcode()))
EHPadStack.pop_back();
else if (WebAssembly::isCatch(MI.getOpcode()))
EHPadStack.push_back(MI.getParent());
@@ -1469,6 +1940,12 @@ bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
if (UnwindDestToTryRanges.empty())
return false;
+ // When end_loop is before end_try_table within the same BB in unwind
+ // destinations, we should split the end_loop into another BB.
+ if (WebAssembly::WasmEnableExnref)
+ for (auto &[UnwindDest, _] : UnwindDestToTryRanges)
+ splitEndLoopBB(UnwindDest);
+
// Now we fix the mismatches by wrapping calls with inner try-delegates.
for (auto &P : UnwindDestToTryRanges) {
NumCallUnwindMismatches += P.second.size();
@@ -1482,9 +1959,10 @@ bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
// If this BB has an EH pad successor, i.e., ends with an 'invoke', and if
// the current range contains the invoke, now we are going to wrap the
- // invoke with try-delegate, making the 'delegate' BB the new successor
- // instead, so remove the EH pad succesor here. The BB may not have an EH
- // pad successor if calls in this BB throw to the caller.
+ // invoke with try-delegate or try_table-end_try_table, making the
+ // 'delegate' or 'end_try_table' BB the new successor instead, so remove
+ // the EH pad succesor here. The BB may not have an EH pad successor if
+ // calls in this BB throw to the caller.
if (UnwindDest != getFakeCallerBlock(MF)) {
MachineBasicBlock *EHPad = nullptr;
for (auto *Succ : MBB->successors()) {
@@ -1497,14 +1975,43 @@ bool WebAssemblyCFGStackify::fixCallUnwindMismatches(MachineFunction &MF) {
MBB->removeSuccessor(EHPad);
}
- addNestedTryDelegate(RangeBegin, RangeEnd, UnwindDest);
+ if (WebAssembly::WasmEnableExnref)
+ addNestedTryTable(RangeBegin, RangeEnd, UnwindDest);
+ else
+ addNestedTryDelegate(RangeBegin, RangeEnd, UnwindDest);
}
}
return true;
}
+// Returns the single destination of try_table, if there is one. All try_table
+// we generate in this pass has a single destination, i.e., a single catch
+// clause.
+static MachineBasicBlock *getSingleUnwindDest(const MachineInstr *TryTable) {
+ if (TryTable->getOperand(1).getImm() != 1)
+ return nullptr;
+ switch (TryTable->getOperand(2).getImm()) {
+ case wasm::WASM_OPCODE_CATCH:
+ case wasm::WASM_OPCODE_CATCH_REF:
+ return TryTable->getOperand(4).getMBB();
+ case wasm::WASM_OPCODE_CATCH_ALL:
+ case wasm::WASM_OPCODE_CATCH_ALL_REF:
+ return TryTable->getOperand(3).getMBB();
+ default:
+ assert(false);
+ }
+}
+
bool WebAssemblyCFGStackify::fixCatchUnwindMismatches(MachineFunction &MF) {
+ // This function is used for both the legacy EH and the standard (exnref) EH,
+ // and the reason we have unwind mismatches is the same for the both of them,
+ // but the code examples in the comments are going to be different. To make
+ // the description less confusing, we write the basically same comments twice,
+ // once for the legacy EH and the standard EH.
+ //
+ // -- Legacy EH --------------------------------------------------------------
+ //
// There is another kind of unwind destination mismatches besides call unwind
// mismatches, which we will call "catch unwind mismatches". See this example
// after the marker placement:
@@ -1541,6 +2048,60 @@ bool WebAssemblyCFGStackify::fixCatchUnwindMismatches(MachineFunction &MF) {
// catch_all ;; ehpad B
// ...
// end_try
+ //
+ // The right destination may be another EH pad or the caller. (The example
+ // here shows the case it is the caller.)
+ //
+ // -- Standard EH ------------------------------------------------------------
+ //
+ // There is another kind of unwind destination mismatches besides call unwind
+ // mismatches, which we will call "catch unwind mismatches". See this example
+ // after the marker placement:
+ // block
+ // try_table (catch_all_ref 0)
+ // block
+ // try_table (catch ... 0)
+ // call @foo
+ // end_try_table
+ // end_block ;; ehpad A (next unwind dest: caller)
+ // ...
+ // end_try_table
+ // end_block ;; ehpad B
+ // ...
+ //
+ // 'call @foo's unwind destination is the ehpad A. But suppose 'call @foo'
+ // throws a foreign exception that is not caught by ehpad A, and its next
+ // destination should be the caller. But after control flow linearization,
+ // another EH pad can be placed in between (e.g. ehpad B here), making the
+ // next unwind destination incorrect. In this case, the foreign exception will
+ // instead go to ehpad B and will be caught there instead. In this example the
+ // correct next unwind destination is the caller, but it can be another outer
+ // catch in other cases.
+ //
+ // There is no specific 'call' or 'throw' instruction to wrap with an inner
+ // try_table-end_try_table, so we wrap the whole try_table-end_try_table with
+ // an inner try_table-end_try_table that sends the exception to a trampoline
+ // BB. We rethrow the sent exception using a throw_ref to the right
+ // destination, which is the caller in the example below:
+ // block exnref
+ // block
+ // try_table (catch_all_ref 0)
+ // try_table (catch_all_ref 2) ;; (new) to trampoline
+ // block
+ // try_table (catch ... 0)
+ // call @foo
+ // end_try_table
+ // end_block ;; ehpad A (next unwind dest: caller)
+ // end_try_table ;; (new)
+ // ...
+ // end_try_table
+ // end_block ;; ehpad B
+ // ...
+ // end_block ;; (new) caller trampoline BB
+ // throw_ref ;; (new) throw to the caller
+ //
+ // The right destination may be another EH pad or the caller. (The example
+ // here shows the case it is the caller.)
const auto *EHInfo = MF.getWasmEHFuncInfo();
assert(EHInfo);
@@ -1553,14 +2114,26 @@ bool WebAssemblyCFGStackify::fixCatchUnwindMismatches(MachineFunction &MF) {
for (auto &MI : reverse(MBB)) {
if (MI.getOpcode() == WebAssembly::TRY)
EHPadStack.pop_back();
- else if (MI.getOpcode() == WebAssembly::DELEGATE)
+ else if (MI.getOpcode() == WebAssembly::TRY_TABLE) {
+ // We want to exclude try_tables created in fixCallUnwindMismatches.
+ // Check if the try_table's unwind destination matches the EH pad stack
+ // top. If it is created in fixCallUnwindMismatches, it wouldn't.
+ if (getSingleUnwindDest(&MI) == EHPadStack.back())
+ EHPadStack.pop_back();
+ } else if (MI.getOpcode() == WebAssembly::DELEGATE)
EHPadStack.push_back(&MBB);
else if (WebAssembly::isCatch(MI.getOpcode())) {
auto *EHPad = &MBB;
+ // If the BB has a catch pseudo instruction but is not marked as an EH
+ // pad, it's a trampoline BB we created in fixCallUnwindMismatches. Skip
+ // it.
+ if (!EHPad->isEHPad())
+ continue;
+
// catch_all always catches an exception, so we don't need to do
// anything
- if (MI.getOpcode() == WebAssembly::CATCH_ALL_LEGACY) {
+ if (WebAssembly::isCatchAll(MI.getOpcode())) {
}
// This can happen when the unwind dest was removed during the
@@ -1602,16 +2175,29 @@ bool WebAssemblyCFGStackify::fixCatchUnwindMismatches(MachineFunction &MF) {
assert(EHPadStack.empty());
if (EHPadToUnwindDest.empty())
return false;
+
+ // When end_loop is before end_try_table within the same BB in unwind
+ // destinations, we should split the end_loop into another BB.
+ for (auto &[_, UnwindDest] : EHPadToUnwindDest)
+ splitEndLoopBB(UnwindDest);
+
NumCatchUnwindMismatches += EHPadToUnwindDest.size();
SmallPtrSet<MachineBasicBlock *, 4> NewEndTryBBs;
for (auto &[EHPad, UnwindDest] : EHPadToUnwindDest) {
MachineInstr *Try = EHPadToTry[EHPad];
MachineInstr *EndTry = BeginToEnd[Try];
- addNestedTryDelegate(Try, EndTry, UnwindDest);
- NewEndTryBBs.insert(EndTry->getParent());
+ if (WebAssembly::WasmEnableExnref) {
+ addNestedTryTable(Try, EndTry, UnwindDest);
+ } else {
+ addNestedTryDelegate(Try, EndTry, UnwindDest);
+ NewEndTryBBs.insert(EndTry->getParent());
+ }
}
+ if (WebAssembly::WasmEnableExnref)
+ return true;
+
// Adding a try-delegate wrapping an existing try-catch-end can make existing
// branch destination BBs invalid. For example,
//
@@ -1811,11 +2397,6 @@ void WebAssemblyCFGStackify::placeMarkers(MachineFunction &MF) {
}
}
- // FIXME We return here temporarily until we implement fixing unwind
- // mismatches for the new exnref proposal.
- if (WebAssembly::WasmEnableExnref)
- return;
-
// Fix mismatches in unwind destinations induced by linearizing the code.
if (MCAI->getExceptionHandlingType() == ExceptionHandling::Wasm &&
MF.getFunction().hasPersonalityFn()) {
@@ -1935,9 +2516,6 @@ void WebAssemblyCFGStackify::rewriteDepthImmediates(MachineFunction &MF) {
for (auto &MBB : reverse(MF)) {
for (MachineInstr &MI : llvm::reverse(MBB)) {
switch (MI.getOpcode()) {
- case WebAssembly::TRY_TABLE:
- RewriteOperands(MI);
- [[fallthrough]];
case WebAssembly::BLOCK:
case WebAssembly::TRY:
assert(ScopeTops[Stack.back().first->getNumber()]->getNumber() <=
@@ -1946,6 +2524,14 @@ void WebAssemblyCFGStackify::rewriteDepthImmediates(MachineFunction &MF) {
Stack.pop_back();
break;
+ case WebAssembly::TRY_TABLE:
+ assert(ScopeTops[Stack.back().first->getNumber()]->getNumber() <=
+ MBB.getNumber() &&
+ "Block/try/try_table marker should be balanced");
+ Stack.pop_back();
+ RewriteOperands(MI);
+ break;
+
case WebAssembly::LOOP:
assert(Stack.back().first == &MBB && "Loop top should be balanced");
Stack.pop_back();
@@ -1992,7 +2578,7 @@ void WebAssemblyCFGStackify::rewriteDepthImmediates(MachineFunction &MF) {
void WebAssemblyCFGStackify::cleanupFunctionData(MachineFunction &MF) {
if (FakeCallerBB)
MF.deleteMachineBasicBlock(FakeCallerBB);
- AppendixBB = FakeCallerBB = nullptr;
+ AppendixBB = FakeCallerBB = CallerTrampolineBB = nullptr;
}
void WebAssemblyCFGStackify::releaseMemory() {
@@ -2001,6 +2587,7 @@ void WebAssemblyCFGStackify::releaseMemory() {
EndToBegin.clear();
TryToEHPad.clear();
EHPadToTry.clear();
+ UnwindDestToTrampoline.clear();
}
bool WebAssemblyCFGStackify::runOnMachineFunction(MachineFunction &MF) {
@@ -2021,7 +2608,7 @@ bool WebAssemblyCFGStackify::runOnMachineFunction(MachineFunction &MF) {
// Remove unnecessary instructions possibly introduced by try/end_trys.
if (MCAI->getExceptionHandlingType() == ExceptionHandling::Wasm &&
- MF.getFunction().hasPersonalityFn())
+ MF.getFunction().hasPersonalityFn() && !WebAssembly::WasmEnableExnref)
removeUnnecessaryInstrs(MF);
// Convert MBB operands in terminators to relative depth immediates.
diff --git a/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll b/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll
new file mode 100644
index 00000000000000..c7542bd6eef77f
--- /dev/null
+++ b/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll
@@ -0,0 +1,1555 @@
+; REQUIRES: asserts
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling | FileCheck %s
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling
+; RUN: llc < %s -O0 -disable-wasm-fallthrough-return-opt -verify-machineinstrs -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling | FileCheck %s --check-prefix=NOOPT
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling -wasm-disable-ehpad-sort -stats 2>&1 | FileCheck %s --check-prefix=NOSORT
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling -wasm-disable-ehpad-sort | FileCheck %s --check-prefix=NOSORT-LOCALS
+
+target triple = "wasm32-unknown-unknown"
+
+ at _ZTIi = external constant ptr
+ at _ZTId = external constant ptr
+
+%class.Object = type { i8 }
+%class.MyClass = type { i32 }
+
+; Simple test case with two catch clauses
+;
+; void foo();
+; void two_catches() {
+; try {
+; foo();
+; } catch (int) {
+; } catch (double) {
+; }
+; }
+
+; CHECK-LABEL: two_catches:
+; CHECK: block
+; CHECK: block () -> (i32, exnref)
+; CHECK: try_table (catch_ref __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 2 # 2: down to label[[L1:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L0]]:
+; CHECK: local.set 2
+; CHECK: local.set 1
+; CHECK: local.get 0
+; CHECK: call _Unwind_CallPersonality
+; CHECK: block
+; CHECK: br_if 0 # 0: down to label[[L2:[0-9]+]]
+; CHECK: call __cxa_begin_catch
+; CHECK: call __cxa_end_catch
+; CHECK: br 1 # 1: down to label[[L1]]
+; CHECK: end_block # label[[L2]]:
+; CHECK: block
+; CHECK: br_if 0 # 0: down to label[[L3:[0-9]+]]
+; CHECK: call __cxa_begin_catch
+; CHECK: call __cxa_end_catch
+; CHECK: br 1 # 1: down to label[[L1]]
+; CHECK: end_block # label[[L3]]:
+; CHECK: throw_ref
+; CHECK: end_block # label[[L1]]:
+define void @two_catches() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ invoke void @foo()
+ to label %try.cont unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr @_ZTIi, ptr @_ZTId]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi)
+ %matches = icmp eq i32 %3, %4
+ br i1 %matches, label %catch2, label %catch.fallthrough
+
+catch2: ; preds = %catch.start
+ %5 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont
+
+catch.fallthrough: ; preds = %catch.start
+ %6 = call i32 @llvm.eh.typeid.for(ptr @_ZTId)
+ %matches1 = icmp eq i32 %3, %6
+ br i1 %matches1, label %catch, label %rethrow
+
+catch: ; preds = %catch.fallthrough
+ %7 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont
+
+rethrow: ; preds = %catch.fallthrough
+ call void @llvm.wasm.rethrow() [ "funclet"(token %1) ]
+ unreachable
+
+try.cont: ; preds = %catch, %catch2, %entry
+ ret void
+}
+
+; Nested try-catches within a catch
+; void nested_catch() {
+; try {
+; foo();
+; } catch (int) {
+; try {
+; foo();
+; } catch (int) {
+; foo();
+; }
+; }
+; }
+
+; CHECK-LABEL: nested_catch:
+; CHECK: block exnref
+; CHECK: block
+; CHECK: block () -> (i32, exnref)
+; CHECK: try_table (catch_ref __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 2 # 2: down to label[[L1:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L0]]:
+; CHECK: call _Unwind_CallPersonality
+; CHECK: block
+; CHECK: block
+; CHECK: br_if 0 # 0: down to label[[L2:[0-9]+]]
+; CHECK: call __cxa_begin_catch
+; CHECK: block exnref
+; CHECK: try_table (catch_all_ref 0) # 0: down to label[[L3:[0-9]+]]
+; CHECK: block () -> (i32, exnref)
+; CHECK: try_table (catch_ref __cpp_exception 0) # 0: down to label[[L4:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 5 # 5: down to label[[L5:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L4]]:
+; CHECK: call _Unwind_CallPersonality
+; CHECK: block
+; CHECK: block
+; CHECK: br_if 0 # 0: down to label[[L6:[0-9]+]]
+; CHECK: call __cxa_begin_catch
+; CHECK: block exnref
+; CHECK: try_table (catch_all_ref 0) # 0: down to label[[L7:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 3 # 3: down to label[[L8:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L7]]:
+; CHECK: try_table (catch_all_ref 7) # 7: down to label[[L9:[0-9]+]]
+; CHECK: call __cxa_end_catch
+; CHECK: end_try_table
+; CHECK: throw_ref
+; CHECK: end_block # label[[L6]]:
+; CHECK: throw_ref
+; CHECK: end_block # label[[L8]]:
+; CHECK: try_table (catch_all_ref 5) # 5: down to label[[L9]]
+; CHECK: call __cxa_end_catch
+; CHECK: end_try_table
+; CHECK: br 3 # 3: down to label[[L5]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L3]]:
+; CHECK: call __cxa_end_catch
+; CHECK: throw_ref
+; CHECK: end_block # label[[L2]]:
+; CHECK: throw_ref
+; CHECK: end_block # label[[L5]]:
+; CHECK: call __cxa_end_catch
+; CHECK: end_block # label[[L1]]:
+; CHECK: return
+; CHECK: end_block # label[[L9]]:
+; CHECK: throw_ref
+define void @nested_catch() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ invoke void @foo()
+ to label %try.cont11 unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr @_ZTIi]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi)
+ %matches = icmp eq i32 %3, %4
+ br i1 %matches, label %catch, label %rethrow
+
+catch: ; preds = %catch.start
+ %5 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ %6 = load i32, ptr %5, align 4
+ invoke void @foo() [ "funclet"(token %1) ]
+ to label %try.cont unwind label %catch.dispatch2
+
+catch.dispatch2: ; preds = %catch
+ %7 = catchswitch within %1 [label %catch.start3] unwind label %ehcleanup9
+
+catch.start3: ; preds = %catch.dispatch2
+ %8 = catchpad within %7 [ptr @_ZTIi]
+ %9 = call ptr @llvm.wasm.get.exception(token %8)
+ %10 = call i32 @llvm.wasm.get.ehselector(token %8)
+ %11 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi)
+ %matches4 = icmp eq i32 %10, %11
+ br i1 %matches4, label %catch6, label %rethrow5
+
+catch6: ; preds = %catch.start3
+ %12 = call ptr @__cxa_begin_catch(ptr %9) [ "funclet"(token %8) ]
+ %13 = load i32, ptr %12, align 4
+ invoke void @foo() [ "funclet"(token %8) ]
+ to label %invoke.cont8 unwind label %ehcleanup
+
+invoke.cont8: ; preds = %catch6
+ call void @__cxa_end_catch() [ "funclet"(token %8) ]
+ catchret from %8 to label %try.cont
+
+rethrow5: ; preds = %catch.start3
+ invoke void @llvm.wasm.rethrow() [ "funclet"(token %8) ]
+ to label %unreachable unwind label %ehcleanup9
+
+try.cont: ; preds = %invoke.cont8, %catch
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont11
+
+rethrow: ; preds = %catch.start
+ call void @llvm.wasm.rethrow() [ "funclet"(token %1) ]
+ unreachable
+
+try.cont11: ; preds = %try.cont, %entry
+ ret void
+
+ehcleanup: ; preds = %catch6
+ %14 = cleanuppad within %8 []
+ call void @__cxa_end_catch() [ "funclet"(token %14) ]
+ cleanupret from %14 unwind label %ehcleanup9
+
+ehcleanup9: ; preds = %ehcleanup, %rethrow5, %catch.dispatch2
+ %15 = cleanuppad within %1 []
+ call void @__cxa_end_catch() [ "funclet"(token %15) ]
+ cleanupret from %15 unwind to caller
+
+unreachable: ; preds = %rethrow5
+ unreachable
+}
+
+; Nested try-catches within a try
+; void nested_try() {
+; try {
+; try {
+; foo();
+; } catch (...) {
+; }
+; } catch (...) {
+; }
+; }
+
+; CHECK-LABEL: nested_try:
+; CHECK: block
+; CHECK: block i32
+; CHECK: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; CHECK: block i32
+; CHECK: try_table (catch __cpp_exception 0) # 0: down to label[[L1:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 4 # 4: down to label[[L2:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L1]]:
+; CHECK: call __cxa_begin_catch
+; CHECK: call __cxa_end_catch
+; CHECK: br 2 # 2: down to label[[L2]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L0]]:
+; CHECK: call __cxa_begin_catch
+; CHECK: call __cxa_end_catch
+; CHECK: end_block # label[[L2]]:
+define void @nested_try() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ invoke void @foo()
+ to label %try.cont7 unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind label %catch.dispatch2
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ invoke void @__cxa_end_catch() [ "funclet"(token %1) ]
+ to label %invoke.cont1 unwind label %catch.dispatch2
+
+catch.dispatch2: ; preds = %catch.start, %catch.dispatch
+ %5 = catchswitch within none [label %catch.start3] unwind to caller
+
+catch.start3: ; preds = %catch.dispatch2
+ %6 = catchpad within %5 [ptr null]
+ %7 = call ptr @llvm.wasm.get.exception(token %6)
+ %8 = call i32 @llvm.wasm.get.ehselector(token %6)
+ %9 = call ptr @__cxa_begin_catch(ptr %7) [ "funclet"(token %6) ]
+ call void @__cxa_end_catch() [ "funclet"(token %6) ]
+ catchret from %6 to label %try.cont7
+
+try.cont7: ; preds = %entry, %invoke.cont1, %catch.start3
+ ret void
+
+invoke.cont1: ; preds = %catch.start
+ catchret from %1 to label %try.cont7
+}
+
+
+; CHECK-LABEL: loop_within_catch:
+; CHECK: block
+; CHECK: block i32
+; CHECK: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 2 # 2: down to label[[L1:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L0]]:
+; CHECK: call __cxa_begin_catch
+; CHECK: loop # label[[L2:[0-9]+]]:
+; CHECK: block
+; CHECK: block
+; CHECK: br_if 0 # 0: down to label[[L3:[0-9]+]]
+; CHECK: block exnref
+; CHECK: try_table (catch_all_ref 0) # 0: down to label[[L4:[0-9]+]]
+; CHECK: call foo
+; CHECK: br 3 # 3: down to label[[L5:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L4]]:
+; CHECK: block
+; CHECK: block
+; CHECK: try_table (catch_all 0) # 0: down to label[[L6:[0-9]+]]
+; CHECK: call __cxa_end_catch
+; CHECK: br 2 # 2: down to label[[L7:[0-9]+]]
+; CHECK: end_try_table
+; CHECK: end_block # label[[L6]]:
+; CHECK: call _ZSt9terminatev
+; CHECK: unreachable
+; CHECK: end_block # label[[L7]]:
+; CHECK: throw_ref
+; CHECK: end_block # label[[L3]]:
+; CHECK: call __cxa_end_catch
+; CHECK: br 2 # 2: down to label[[L1]]
+; CHECK: end_block # label[[L5]]:
+; CHECK: br 0 # 0: up to label[[L2]]
+; CHECK: end_loop
+; CHECK: end_block # label[[L1]]:
+define void @loop_within_catch() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ invoke void @foo()
+ to label %try.cont unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ br label %for.cond
+
+for.cond: ; preds = %for.inc, %catch.start
+ %i.0 = phi i32 [ 0, %catch.start ], [ %inc, %for.inc ]
+ %cmp = icmp slt i32 %i.0, 50
+ br i1 %cmp, label %for.body, label %for.end
+
+for.body: ; preds = %for.cond
+ invoke void @foo() [ "funclet"(token %1) ]
+ to label %for.inc unwind label %ehcleanup
+
+for.inc: ; preds = %for.body
+ %inc = add nsw i32 %i.0, 1
+ br label %for.cond
+
+for.end: ; preds = %for.cond
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont
+
+try.cont: ; preds = %for.end, %entry
+ ret void
+
+ehcleanup: ; preds = %for.body
+ %5 = cleanuppad within %1 []
+ invoke void @__cxa_end_catch() [ "funclet"(token %5) ]
+ to label %invoke.cont2 unwind label %terminate
+
+invoke.cont2: ; preds = %ehcleanup
+ cleanupret from %5 unwind to caller
+
+terminate: ; preds = %ehcleanup
+ %6 = cleanuppad within %5 []
+ call void @_ZSt9terminatev() [ "funclet"(token %6) ]
+ unreachable
+}
+
+; Tests if block and try_table markers are correctly placed. Even if two
+; predecessors of the EH pad are bb2 and bb3 and their nearest common dominator
+; is bb1, the TRY_TABLE marker should be placed at bb0 because there's a branch
+; from bb0 to bb2, and scopes cannot be interleaved.
+; NOOPT-LABEL: block_try_table_markers:
+; NOOPT: block
+; NOOPT: block i32
+; NOOPT: try_table (catch __cpp_exception 0)
+; NOOPT: block
+; NOOPT: block
+; NOOPT: block
+; NOOPT: end_block
+; NOOPT: end_block
+; NOOPT: call foo
+; NOOPT: end_block
+; NOOPT: call bar
+; NOOPT: end_try_table
+; NOOPT: end_block
+; NOOPT: end_block
+define void @block_try_table_markers() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ br i1 undef, label %bb1, label %bb2
+
+bb1: ; preds = %bb0
+ br i1 undef, label %bb3, label %bb4
+
+bb2: ; preds = %bb0
+ br label %try.cont
+
+bb3: ; preds = %bb1
+ invoke void @foo()
+ to label %try.cont unwind label %catch.dispatch
+
+bb4: ; preds = %bb1
+ invoke void @bar()
+ to label %try.cont unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %bb4, %bb3
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %try.cont
+
+try.cont: ; preds = %catch.start, %bb4, %bb3, %bb2
+ ret void
+}
+
+; Tests if try_table/end_try_table markers are placed correctly wrt
+; loop/end_loop markers, when try_table and loop markers are in the same BB and
+; end_try_table and end_loop are in another BB.
+; CHECK-LABEL: loop_try_table_markers:
+; CHECK: loop
+; CHECK: block i32
+; CHECK: try_table (catch __cpp_exception 0)
+; CHECK: call foo
+; CHECK: end_try_table
+; CHECK: end_block
+; CHECK: end_loop
+define void @loop_try_table_markers(ptr %p) personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ store volatile i32 0, ptr %p
+ br label %loop
+
+loop: ; preds = %try.cont, %entry
+ store volatile i32 1, ptr %p
+ invoke void @foo()
+ to label %try.cont unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %loop
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %try.cont
+
+try.cont: ; preds = %catch.start, %loop
+ br label %loop
+}
+
+; Some of test cases below are hand-tweaked by deleting some library calls to
+; simplify tests and changing the order of basic blocks to cause unwind
+; destination mismatches. And we use -wasm-disable-ehpad-sort to create maximum
+; number of mismatches in several tests below.
+
+; - Call unwind mismatch
+; 'call bar''s original unwind destination was 'C0', but after control flow
+; linearization, its unwind destination incorrectly becomes 'C1'. We fix this by
+; wrapping the call with a nested try_table-end_try_table that targets 'C0'.
+; - Catch unwind mismatch
+; If 'call foo' throws a foreign exception, it will not be caught by C1, and
+; should be rethrown to the caller. But after control flow linearization, it
+; will instead unwind to C0, an incorrect next EH pad. We wrap the whole
+; try_table-end_try_table with another try_table-end_try_table that jumps to a
+; trampoline BB, from which we rethrow the exception to the caller to fix this.
+
+; NOSORT-LABEL: unwind_mismatches_0:
+; NOSORT: block exnref
+; NOSORT: block
+; NOSORT: block i32
+; NOSORT: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; NOSORT: block exnref
+; NOSORT: block i32
+; --- nested try_table-end_try_table starts (catch unwind mismatch)
+; NOSORT: try_table (catch_all_ref 5) # 5: down to label[[L1:[0-9]+]]
+; NOSORT: try_table (catch __cpp_exception 1) # 1: down to label[[L2:[0-9]+]]
+; NOSORT: call foo
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 3) # 3: down to label[[L3:[0-9]+]]
+; NOSORT: call bar
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: end_try_table
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (catch unwind mismatch)
+; NOSORT: end_block # label[[L2]]:
+; NOSORT: end_block # label[[L3]]:
+; NOSORT: throw_ref
+; NOSORT: end_try_table
+; NOSORT: end_block # label[[L0]]:
+; NOSORT: end_block
+; NOSORT: return
+; NOSORT: end_block # label[[L1]]:
+; NOSORT: throw_ref
+define void @unwind_mismatches_0() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ invoke void @bar()
+ to label %try.cont unwind label %catch.dispatch1
+
+catch.dispatch0: ; preds = %bb0
+ %0 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %try.cont
+
+catch.dispatch1: ; preds = %bb1
+ %4 = catchswitch within none [label %catch.start1] unwind to caller
+
+catch.start1: ; preds = %catch.dispatch1
+ %5 = catchpad within %4 [ptr null]
+ %6 = call ptr @llvm.wasm.get.exception(token %5)
+ %7 = call i32 @llvm.wasm.get.ehselector(token %5)
+ catchret from %5 to label %try.cont
+
+try.cont: ; preds = %catch.start1, %catch.start0, %bb1
+ ret void
+}
+
+; 'call bar' and 'call baz''s original unwind destination was the caller, but
+; after control flow linearization, their unwind destination incorrectly becomes
+; 'C0'. We fix this by wrapping the calls with a nested try_table-end_try_table
+; that jumps to a trampoline BB where we rethrow the exception to the caller.
+
+; And the return value of 'baz' should NOT be stackified because the BB is split
+; during fixing unwind mismatches.
+
+; NOSORT-LABEL: unwind_mismatches_1:
+; NOSORT: block exnref
+; NOSORT: block i32
+; NOSORT: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; NOSORT: call foo
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 2) # 2: down to label[[L1:[0-9]+]]
+; NOSORT: call bar
+; NOSORT: call baz
+; NOSORT: local.set [[LOCAL:[0-9]+]]
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: local.get [[LOCAL]]
+; NOSORT: call nothrow
+; NOSORT: return
+; NOSORT: end_try_table
+; NOSORT: end_block # label[[L0]]:
+; NOSORT: return
+; NOSORT: end_block # label[[L1]]:
+; NOSORT: throw_ref
+define void @unwind_mismatches_1() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ call void @bar()
+ %call = call i32 @baz()
+ call void @nothrow(i32 %call) #0
+ ret void
+
+catch.dispatch0: ; preds = %bb0
+ %0 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %try.cont
+
+try.cont: ; preds = %catch.start0
+ ret void
+}
+
+; The same as unwind_mismatches_0, but we have one more call 'call @foo' in bb1
+; which unwinds to the caller. IN this case bb1 has two call unwind mismatches:
+; 'call @foo' unwinds to the caller and 'call @bar' unwinds to catch C0.
+
+; NOSORT-LABEL: unwind_mismatches_2:
+; NOSORT: block exnref
+; NOSORT: block
+; NOSORT: block i32
+; NOSORT: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; NOSORT: block exnref
+; NOSORT: block i32
+; --- nested try_table-end_try_table starts (catch unwind mismatch)
+; NOSORT: try_table (catch_all_ref 5) # 5: down to label[[L1:[0-9]+]]
+; NOSORT: try_table (catch __cpp_exception 1) # 1: down to label[[L2:[0-9]+]]
+; NOSORT: call foo
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 7) # 7: down to label[[L1]]
+; NOSORT: call foo
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 3) # 3: down to label[[L3:[0-9]+]]
+; NOSORT: call bar
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: end_try_table
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (catch unwind mismatch)
+; NOSORT: end_block # label[[L2]]:
+; NOSORT: end_block # label[[L3]]:
+; NOSORT: throw_ref
+; NOSORT: end_try_table
+; NOSORT: end_block # label[[L0]]:
+; NOSORT: end_block
+; NOSORT: return
+; NOSORT: end_block # label[[L1]]:
+; NOSORT: throw_ref
+define void @unwind_mismatches_2() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ call void @foo()
+ invoke void @bar()
+ to label %try.cont unwind label %catch.dispatch1
+
+catch.dispatch0: ; preds = %bb0
+ %0 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %try.cont
+
+catch.dispatch1: ; preds = %bb1
+ %4 = catchswitch within none [label %catch.start1] unwind to caller
+
+catch.start1: ; preds = %catch.dispatch1
+ %5 = catchpad within %4 [ptr null]
+ %6 = call ptr @llvm.wasm.get.exception(token %5)
+ %7 = call i32 @llvm.wasm.get.ehselector(token %5)
+ catchret from %5 to label %try.cont
+
+try.cont: ; preds = %catch.start1, %catch.start0, %bb1
+ ret void
+}
+
+; Similar situation as @unwind_mismatches_1. Here 'call @qux''s original unwind
+; destination was the caller, but after control flow linearization, their unwind
+; destination incorrectly becomes 'C0' within the function. We fix this by
+; wrapping the call with a nested try_table-end_try_table that jumps to a
+; trampoline BB where rethrow the exception to the caller.
+
+; Because 'call @qux' pops an argument pushed by 'i32.const 5' from stack, the
+; nested 'try_table' should be placed before `i32.const 5', not between
+; 'i32.const 5' and 'call @qux'.
+
+; NOSORT-LABEL: unwind_mismatches_3:
+; NOSORT: block exnref
+; NOSORT: block i32
+; NOSORT: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; NOSORT: call foo
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 2) # 2: down to label[[L1:[0-9]+]]
+; NOSORT: i32.const 5
+; NOSORT: call qux
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: return
+; NOSORT: end_try_table
+; NOSORT: end_block # label[[L0]]:
+; NOSORT: return
+; NOSORT: end_block # label[[L1]]:
+; NOSORT: throw_ref
+define i32 @unwind_mismatches_3() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ %0 = call i32 @qux(i32 5)
+ ret i32 %0
+
+catch.dispatch0: ; preds = %bb0
+ %1 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %2 = catchpad within %1 [ptr null]
+ %3 = call ptr @llvm.wasm.get.exception(token %2)
+ %j = call i32 @llvm.wasm.get.ehselector(token %2)
+ catchret from %2 to label %try.cont
+
+try.cont: ; preds = %catch.start0
+ ret i32 0
+}
+
+; We have two call unwind unwind mismatches:
+; - A may-throw instruction unwinds to an incorrect EH pad after linearizing the
+; CFG, when it is supposed to unwind to another EH pad.
+; - A may-throw instruction unwinds to an incorrect EH pad after linearizing the
+; CFG, when it is supposed to unwind to the caller.
+; We also have a catch unwind mismatch: If an exception is not caught by the
+; first catch because it is a non-C++ exception, it shouldn't unwind to the next
+; catch, but it should unwind to the caller.
+
+; NOSORT-LABEL: unwind_mismatches_4:
+; NOSORT: block exnref
+; NOSORT: block
+; NOSORT: block i32
+; NOSORT: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; NOSORT: block exnref
+; NOSORT: block i32
+; --- nested try_table-end_try_table starts (catch unwind mismatch)
+; NOSORT: try_table (catch_all_ref 5) # 5: down to label[[L1:[0-9]+]]
+; NOSORT: try_table (catch __cpp_exception 1) # 1: down to label[[L2:[0-9]+]]
+; NOSORT: call foo
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 3) # 3: down to label[[L3:[0-9]+]]
+; NOSORT: call bar
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: end_try_table
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (catch unwind mismatch)
+; NOSORT: end_block # label[[L2]]:
+; NOSORT: call __cxa_begin_catch
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 4) # 4: down to label[[L1]]
+; NOSORT: call __cxa_end_catch
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: end_block # label[[L3]]:
+; NOSORT: throw_ref
+; NOSORT: end_try_table
+; NOSORT: end_block # label[[L0]]:
+; NOSORT: call __cxa_begin_catch
+; NOSORT: call __cxa_end_catch
+; NOSORT: end_block # label74:
+; NOSORT: return
+; NOSORT: end_block # label[[L1]]:
+; NOSORT: throw_ref
+define void @unwind_mismatches_4() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ invoke void @bar()
+ to label %try.cont unwind label %catch.dispatch1
+
+catch.dispatch0: ; preds = %bb0
+ %0 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont
+
+catch.dispatch1: ; preds = %bb1
+ %5 = catchswitch within none [label %catch.start1] unwind to caller
+
+catch.start1: ; preds = %catch.dispatch1
+ %6 = catchpad within %5 [ptr null]
+ %7 = call ptr @llvm.wasm.get.exception(token %6)
+ %8 = call i32 @llvm.wasm.get.ehselector(token %6)
+ %9 = call ptr @__cxa_begin_catch(ptr %7) [ "funclet"(token %6) ]
+ call void @__cxa_end_catch() [ "funclet"(token %6) ]
+ catchret from %6 to label %try.cont
+
+try.cont: ; preds = %catch.start1, %catch.start0, %bb1
+ ret void
+}
+
+; This crashed when updating EHPadStack within fixCallUniwindMismatch had a bug.
+; This should not crash and a nested try_table-end_try_table has to be created
+; around 'call @baz', because the initial TRY_TABLE placement for 'call @quux'
+; was done before 'call @baz' because 'call @baz''s return value is stackified.
+
+; CHECK-LABEL: unwind_mismatches_5:
+; CHECK: block exnref
+; CHECK: block
+; CHECK: block exnref
+; CHECK: try_table (catch_all_ref 0) # 0: down to label[[L0:[0-9]+]]
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; CHECK: try_table (catch_all_ref 3) # 3: down to label[[L1:[0-9]+]]
+; CHECK: call baz
+; CHECK: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; CHECK: call quux
+; CHECK: end_try_table
+; CHECK: end_block # label[[L0]]:
+; CHECK: throw_ref
+; CHECK: end_block
+; CHECK: unreachable
+; CHECK: end_block # label[[L1]]:
+; CHECK: throw_ref
+define void @unwind_mismatches_5() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %call = call i32 @baz()
+ invoke void @quux(i32 %call)
+ to label %invoke.cont unwind label %ehcleanup
+
+ehcleanup: ; preds = %entry
+ %0 = cleanuppad within none []
+ cleanupret from %0 unwind to caller
+
+invoke.cont: ; preds = %entry
+ unreachable
+}
+
+; The structure is similar to unwind_mismatches_0, where the call to 'bar''s
+; original unwind destination is catch.dispatch1 but after placing markers it
+; unwinds to catch.dispatch0, which we fix. This additionally has a loop before
+; the real unwind destination (catch.dispatch1). This makes sure the code
+; generation works when the unwind destination has an end_loop before
+; end_try_table before the mismatch fixing.
+
+; NOSORT-LABEL: unwind_mismatches_with_loop:
+; NOSORT: block exnref
+; NOSORT: block i32
+; NOSORT: try_table (catch __cpp_exception 0) # 0: down to label[[L0:[0-9]+]]
+; NOSORT: block exnref
+; NOSORT: block
+; NOSORT: block i32
+; --- nested try_table-end_try_table starts (catch unwind mismatch)
+; NOSORT: try_table (catch_all_ref 5) # 5: down to label[[L1:[0-9]+]]
+; NOSORT: try_table (catch __cpp_exception 1) # 1: down to label[[L2:[0-9]+]]
+; NOSORT: call foo
+; --- nested try_table-end_try_table starts (call unwind mismatch)
+; NOSORT: try_table (catch_all_ref 4) # 4: down to label[[L3:[0-9]+]]
+; NOSORT: call bar
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (call unwind mismatch)
+; NOSORT: end_try_table
+; NOSORT: end_try_table
+; --- nested try_table-end_try_table ends (catch unwind mismatch)
+; NOSORT: end_block # label[[L2]]:
+; NOSORT: end_block
+; NOSORT: loop
+; NOSORT: call foo
+; NOSORT: end_loop
+; NOSORT: end_block # label[[L3]]:
+; NOSORT: throw_ref
+; NOSORT: end_try_table
+; NOSORT: end_block # label[[L0]]:
+; NOSORT: return
+; NOSORT: end_block # label[[L1]]:
+; NOSORT: throw_ref
+define void @unwind_mismatches_with_loop() personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ invoke void @bar()
+ to label %bb2 unwind label %catch.dispatch1
+
+catch.dispatch0: ; preds = %bb0
+ %0 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %bb2
+
+bb2:
+ invoke void @foo()
+ to label %bb3 unwind label %catch.dispatch1
+
+bb3: ; preds = %bb14
+ br label %bb2
+
+catch.dispatch1: ; preds = %bb1
+ %4 = catchswitch within none [label %catch.start1] unwind to caller
+
+catch.start1: ; preds = %catch.dispatch1
+ %5 = catchpad within %4 [ptr null]
+ %6 = call ptr @llvm.wasm.get.exception(token %5)
+ %7 = call i32 @llvm.wasm.get.ehselector(token %5)
+ catchret from %5 to label %try.cont
+
+try.cont: ; preds = %catch.start1, %catch.start0, %bb1
+ ret void
+}
+
+; Tests the case when TEE stackifies a register in RegStackify but it gets
+; unstackified in fixCallUnwindMismatches in CFGStackify.
+
+; NOSORT-LOCALS-LABEL: unstackify_when_fixing_unwind_mismatch:
+define void @unstackify_when_fixing_unwind_mismatch(i32 %x) personality ptr @__gxx_wasm_personality_v0 {
+bb0:
+ invoke void @foo()
+ to label %bb1 unwind label %catch.dispatch0
+
+bb1: ; preds = %bb0
+ %t = add i32 %x, 4
+ ; This %addr is used in multiple places, so tee is introduced in RegStackify,
+ ; which stackifies the use of %addr in store instruction. A tee has two dest
+ ; registers, the first of which is stackified and the second is not.
+ ; But when we introduce a nested try_table-end_try_table in
+ ; fixCallUnwindMismatches in CFGStackify, we end up unstackifying the first
+ ; dest register. In that case, we convert that tee into a copy.
+ %addr = inttoptr i32 %t to ptr
+ %load = load i32, ptr %addr
+ %call = call i32 @baz()
+ %add = add i32 %load, %call
+ store i32 %add, ptr %addr
+ ret void
+; NOSORT-LOCALS: i32.add
+; NOSORT-LOCALS-NOT: local.tee
+; NOSORT-LOCALS-NEXT: local.set
+
+catch.dispatch0: ; preds = %bb0
+ %0 = catchswitch within none [label %catch.start0] unwind to caller
+
+catch.start0: ; preds = %catch.dispatch0
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ catchret from %1 to label %try.cont
+
+try.cont: ; preds = %catch.start0
+ ret void
+}
+
+; In CFGSort, EH pads should be sorted as soon as it is available and
+; 'Preferred' queue and should NOT be entered into 'Ready' queue unless we are
+; in the middle of sorting another region that does not contain the EH pad. In
+; this example, 'catch.start' should be sorted right after 'if.then' is sorted
+; (before 'cont' is sorted) and there should not be any unwind destination
+; mismatches in CFGStackify.
+
+; NOOPT-LABEL: cfg_sort_order:
+; NOOPT: block
+; NOOPT: block
+; NOOPT: block i32
+; NOOPT: try_table (catch __cpp_exception 0)
+; NOOPT: call foo
+; NOOPT: end_try_table
+; NOOPT: end_block
+; NOOPT: call __cxa_begin_catch
+; NOOPT: call __cxa_end_catch
+; NOOPT: end_block
+; NOOPT: call foo
+; NOOPT: end_block
+; NOOPT: return
+define void @cfg_sort_order(i32 %arg) personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %tobool = icmp ne i32 %arg, 0
+ br i1 %tobool, label %if.then, label %if.end
+
+catch.dispatch: ; preds = %if.then
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %if.end
+
+if.then: ; preds = %entry
+ invoke void @foo()
+ to label %cont unwind label %catch.dispatch
+
+cont: ; preds = %if.then
+ call void @foo()
+ br label %if.end
+
+if.end: ; preds = %cont, %catch.start, %entry
+ ret void
+}
+
+; Intrinsics like memcpy, memmove, and memset don't throw and are lowered into
+; calls to external symbols (not global addresses) in instruction selection,
+; which will be eventually lowered to library function calls.
+; Because this test runs with -wasm-disable-ehpad-sort, these library calls in
+; invoke.cont BB fall within try~end_try, but they shouldn't cause crashes or
+; unwinding destination mismatches in CFGStackify.
+
+; NOSORT-LABEL: mem_intrinsics:
+; NOSORT: block exnref
+; NOSORT: try_table (catch_all_ref 0)
+; NOSORT: call foo
+; NOSORT: call memcpy
+; NOSORT: call memmove
+; NOSORT: call memset
+; NOSORT: return
+; NOSORT: end_try_table
+; NOSORT: end_block
+; NOSORT: throw_ref
+define void @mem_intrinsics(ptr %a, ptr %b) personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %o = alloca %class.Object, align 1
+ invoke void @foo()
+ to label %invoke.cont unwind label %ehcleanup
+
+invoke.cont: ; preds = %entry
+ call void @llvm.memcpy.p0.p0.i32(ptr %a, ptr %b, i32 100, i1 false)
+ call void @llvm.memmove.p0.p0.i32(ptr %a, ptr %b, i32 100, i1 false)
+ call void @llvm.memset.p0.i32(ptr %a, i8 0, i32 100, i1 false)
+ %call = call ptr @_ZN6ObjectD2Ev(ptr %o)
+ ret void
+
+ehcleanup: ; preds = %entry
+ %0 = cleanuppad within none []
+ %call2 = call ptr @_ZN6ObjectD2Ev(ptr %o) [ "funclet"(token %0) ]
+ cleanupret from %0 unwind to caller
+}
+
+; Tests if 'try_table' marker is placed correctly. In this test, 'try_table'
+; should be placed before the call to 'nothrow_i32' and not between the call to
+; 'nothrow_i32' and 'fun', because the return value of 'nothrow_i32' is
+; stackified and pushed onto the stack to be consumed by the call to 'fun'.
+
+; CHECK-LABEL: try_table_marker_with_stackified_input:
+; CHECK: try_table (catch_all 0)
+; CHECK: call nothrow_i32
+; CHECK: call fun
+define void @try_table_marker_with_stackified_input() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %call = call i32 @nothrow_i32()
+ invoke void @fun(i32 %call)
+ to label %invoke.cont unwind label %terminate
+
+invoke.cont: ; preds = %entry
+ ret void
+
+terminate: ; preds = %entry
+ %0 = cleanuppad within none []
+ call void @_ZSt9terminatev() [ "funclet"(token %0) ]
+ unreachable
+}
+
+; This crashed on debug mode (= when NDEBUG is not defined) when the logic for
+; computing the innermost region was not correct, in which a loop region
+; contains an exception region. This should pass CFGSort without crashing.
+define void @loop_exception_region() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %e = alloca %class.MyClass, align 4
+ br label %for.cond
+
+for.cond: ; preds = %for.inc, %entry
+ %i.0 = phi i32 [ 0, %entry ], [ %inc, %for.inc ]
+ %cmp = icmp slt i32 %i.0, 9
+ br i1 %cmp, label %for.body, label %for.end
+
+for.body: ; preds = %for.cond
+ invoke void @quux(i32 %i.0)
+ to label %for.inc unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %for.body
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr @_ZTI7MyClass]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call i32 @llvm.eh.typeid.for(ptr @_ZTI7MyClass)
+ %matches = icmp eq i32 %3, %4
+ br i1 %matches, label %catch, label %rethrow
+
+catch: ; preds = %catch.start
+ %5 = call ptr @__cxa_get_exception_ptr(ptr %2) [ "funclet"(token %1) ]
+ %call = call ptr @_ZN7MyClassC2ERKS_(ptr %e, ptr dereferenceable(4) %5) [ "funclet"(token %1) ]
+ %6 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ %7 = load i32, ptr %e, align 4
+ invoke void @quux(i32 %7) [ "funclet"(token %1) ]
+ to label %invoke.cont2 unwind label %ehcleanup
+
+invoke.cont2: ; preds = %catch
+ %call3 = call ptr @_ZN7MyClassD2Ev(ptr %e) [ "funclet"(token %1) ]
+ call void @__cxa_end_catch() [ "funclet"(token %1) ]
+ catchret from %1 to label %for.inc
+
+rethrow: ; preds = %catch.start
+ call void @llvm.wasm.rethrow() [ "funclet"(token %1) ]
+ unreachable
+
+for.inc: ; preds = %invoke.cont2, %for.body
+ %inc = add nsw i32 %i.0, 1
+ br label %for.cond
+
+ehcleanup: ; preds = %catch
+ %8 = cleanuppad within %1 []
+ %call4 = call ptr @_ZN7MyClassD2Ev(ptr %e) [ "funclet"(token %8) ]
+ invoke void @__cxa_end_catch() [ "funclet"(token %8) ]
+ to label %invoke.cont6 unwind label %terminate7
+
+invoke.cont6: ; preds = %ehcleanup
+ cleanupret from %8 unwind to caller
+
+for.end: ; preds = %for.cond
+ ret void
+
+terminate7: ; preds = %ehcleanup
+ %9 = cleanuppad within %8 []
+ call void @_ZSt9terminatev() [ "funclet"(token %9) ]
+ unreachable
+}
+
+; Here exceptions are semantically contained in a loop. 'ehcleanup' BB belongs
+; to the exception, but does not belong to the loop (because it does not have a
+; path back to the loop header), and is placed after the loop latch block
+; 'invoke.cont' intentionally. This tests if 'end_loop' marker is placed
+; correctly not right after 'invoke.cont' part but after 'ehcleanup' part.
+; NOSORT-LABEL: loop_contains_exception:
+; NOSORT: loop
+; NOSORT: try_table (catch __cpp_exception 0)
+; NOSORT: end_try_table
+; NOSORT: try_table (catch_all 0)
+; NOSORT: end_try_table
+; NOSORT: end_loop
+define void @loop_contains_exception(i32 %n) personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ br label %while.cond
+
+while.cond: ; preds = %invoke.cont, %entry
+ %n.addr.0 = phi i32 [ %n, %entry ], [ %dec, %invoke.cont ]
+ %tobool = icmp ne i32 %n.addr.0, 0
+ br i1 %tobool, label %while.body, label %while.end
+
+while.body: ; preds = %while.cond
+ %dec = add nsw i32 %n.addr.0, -1
+ invoke void @foo()
+ to label %while.end unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %while.body
+ %0 = catchswitch within none [label %catch.start] unwind to caller
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr null]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call ptr @__cxa_begin_catch(ptr %2) [ "funclet"(token %1) ]
+ invoke void @__cxa_end_catch() [ "funclet"(token %1) ]
+ to label %invoke.cont unwind label %ehcleanup
+
+invoke.cont: ; preds = %catch.start
+ catchret from %1 to label %while.cond
+
+ehcleanup: ; preds = %catch.start
+ %5 = cleanuppad within %1 []
+ call void @_ZSt9terminatev() [ "funclet"(token %5) ]
+ unreachable
+
+while.end: ; preds = %while.body, %while.cond
+ ret void
+}
+
+; Regression test for WasmEHFuncInfo's reverse mapping bug. 'UnwindDestToSrc'
+; should return a vector and not a single BB, which was incorrect.
+; This was reduced by bugpoint and should not crash in CFGStackify.
+define void @wasm_eh_func_info_regression_test() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ invoke void @foo()
+ to label %invoke.cont unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind label %ehcleanup22
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr @_ZTIi]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ invoke void @__cxa_throw(ptr null, ptr null, ptr null) #1 [ "funclet"(token %1) ]
+ to label %unreachable unwind label %catch.dispatch2
+
+catch.dispatch2: ; preds = %catch.start
+ %4 = catchswitch within %1 [label %catch.start3] unwind label %ehcleanup
+
+catch.start3: ; preds = %catch.dispatch2
+ %5 = catchpad within %4 [ptr @_ZTIi]
+ %6 = call ptr @llvm.wasm.get.exception(token %5)
+ %7 = call i32 @llvm.wasm.get.ehselector(token %5)
+ catchret from %5 to label %try.cont
+
+try.cont: ; preds = %catch.start3
+ invoke void @foo() [ "funclet"(token %1) ]
+ to label %invoke.cont8 unwind label %ehcleanup
+
+invoke.cont8: ; preds = %try.cont
+ invoke void @__cxa_throw(ptr null, ptr null, ptr null) #1 [ "funclet"(token %1) ]
+ to label %unreachable unwind label %catch.dispatch11
+
+catch.dispatch11: ; preds = %invoke.cont8
+ %8 = catchswitch within %1 [label %catch.start12] unwind label %ehcleanup
+
+catch.start12: ; preds = %catch.dispatch11
+ %9 = catchpad within %8 [ptr @_ZTIi]
+ %10 = call ptr @llvm.wasm.get.exception(token %9)
+ %11 = call i32 @llvm.wasm.get.ehselector(token %9)
+ unreachable
+
+invoke.cont: ; preds = %entry
+ unreachable
+
+ehcleanup: ; preds = %catch.dispatch11, %try.cont, %catch.dispatch2
+ %12 = cleanuppad within %1 []
+ cleanupret from %12 unwind label %ehcleanup22
+
+ehcleanup22: ; preds = %ehcleanup, %catch.dispatch
+ %13 = cleanuppad within none []
+ cleanupret from %13 unwind to caller
+
+unreachable: ; preds = %invoke.cont8, %catch.start
+ unreachable
+}
+
+; void exception_grouping_0() {
+; try {
+; try {
+; throw 0;
+; } catch (int) {
+; }
+; } catch (int) {
+; }
+; }
+;
+; Regression test for a WebAssemblyException grouping bug. After catchswitches
+; are removed, EH pad catch.start2 is dominated by catch.start, but because
+; catch.start2 is the unwind destination of catch.start, it should not be
+; included in catch.start's exception. Also, after we take catch.start2's
+; exception out of catch.start's exception, we have to take out try.cont8 out of
+; catch.start's exception, because it has a predecessor in catch.start2.
+define void @exception_grouping_0() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %exception = call ptr @__cxa_allocate_exception(i32 4) #0
+ store i32 0, ptr %exception, align 16
+ invoke void @__cxa_throw(ptr %exception, ptr @_ZTIi, ptr null) #1
+ to label %unreachable unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind label %catch.dispatch1
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr @_ZTIi]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #0
+ %matches = icmp eq i32 %3, %4
+ br i1 %matches, label %catch, label %rethrow
+
+catch: ; preds = %catch.start
+ %5 = call ptr @__cxa_begin_catch(ptr %2) #0 [ "funclet"(token %1) ]
+ %6 = load i32, ptr %5, align 4
+ call void @__cxa_end_catch() #0 [ "funclet"(token %1) ]
+ catchret from %1 to label %catchret.dest
+
+catchret.dest: ; preds = %catch
+ br label %try.cont
+
+rethrow: ; preds = %catch.start
+ invoke void @llvm.wasm.rethrow() #1 [ "funclet"(token %1) ]
+ to label %unreachable unwind label %catch.dispatch1
+
+catch.dispatch1: ; preds = %rethrow, %catch.dispatch
+ %7 = catchswitch within none [label %catch.start2] unwind to caller
+
+catch.start2: ; preds = %catch.dispatch1
+ %8 = catchpad within %7 [ptr @_ZTIi]
+ %9 = call ptr @llvm.wasm.get.exception(token %8)
+ %10 = call i32 @llvm.wasm.get.ehselector(token %8)
+ %11 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #0
+ %matches3 = icmp eq i32 %10, %11
+ br i1 %matches3, label %catch5, label %rethrow4
+
+catch5: ; preds = %catch.start2
+ %12 = call ptr @__cxa_begin_catch(ptr %9) #0 [ "funclet"(token %8) ]
+ %13 = load i32, ptr %12, align 4
+ call void @__cxa_end_catch() #0 [ "funclet"(token %8) ]
+ catchret from %8 to label %catchret.dest7
+
+catchret.dest7: ; preds = %catch5
+ br label %try.cont8
+
+rethrow4: ; preds = %catch.start2
+ call void @llvm.wasm.rethrow() #1 [ "funclet"(token %8) ]
+ unreachable
+
+try.cont8: ; preds = %try.cont, %catchret.dest7
+ ret void
+
+try.cont: ; preds = %catchret.dest
+ br label %try.cont8
+
+unreachable: ; preds = %rethrow, %entry
+ unreachable
+}
+
+; Test for WebAssemblyException grouping. This test is hand-modified to generate
+; this structure:
+; catch.start dominates catch.start4 and catch.start4 dominates catch.start12,
+; so the after dominator-based grouping, we end up with:
+; catch.start's exception > catch4.start's exception > catch12.start's exception
+; (> here represents subexception relationship)
+;
+; But the unwind destination chain is catch.start -> catch.start4 ->
+; catch.start12. So all these subexception relationship should be deconstructed.
+; We have to make sure to take out catch.start4's exception out of catch.start's
+; exception first, before taking out catch.start12's exception out of
+; catch.start4's exception; otherwise we end up with an incorrect relationship
+; of catch.start's exception > catch.start12's exception.
+define void @exception_grouping_1() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ invoke void @foo()
+ to label %invoke.cont unwind label %catch.dispatch
+
+invoke.cont: ; preds = %entry
+ invoke void @foo()
+ to label %invoke.cont1 unwind label %catch.dispatch
+
+invoke.cont1: ; preds = %invoke.cont
+ invoke void @foo()
+ to label %try.cont18 unwind label %catch.dispatch
+
+catch.dispatch11: ; preds = %rethrow6, %catch.dispatch3
+ %0 = catchswitch within none [label %catch.start12] unwind to caller
+
+catch.start12: ; preds = %catch.dispatch11
+ %1 = catchpad within %0 [ptr @_ZTIi]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #0
+ %matches13 = icmp eq i32 %3, %4
+ br i1 %matches13, label %catch15, label %rethrow14
+
+catch15: ; preds = %catch.start12
+ %5 = call ptr @__cxa_begin_catch(ptr %2) #0 [ "funclet"(token %1) ]
+ %6 = load i32, ptr %5, align 4
+ call void @__cxa_end_catch() #0 [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont18
+
+rethrow14: ; preds = %catch.start12
+ call void @llvm.wasm.rethrow() #1 [ "funclet"(token %1) ]
+ unreachable
+
+catch.dispatch3: ; preds = %rethrow, %catch.dispatch
+ %7 = catchswitch within none [label %catch.start4] unwind label %catch.dispatch11
+
+catch.start4: ; preds = %catch.dispatch3
+ %8 = catchpad within %7 [ptr @_ZTIi]
+ %9 = call ptr @llvm.wasm.get.exception(token %8)
+ %10 = call i32 @llvm.wasm.get.ehselector(token %8)
+ %11 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #0
+ %matches5 = icmp eq i32 %10, %11
+ br i1 %matches5, label %catch7, label %rethrow6
+
+catch7: ; preds = %catch.start4
+ %12 = call ptr @__cxa_begin_catch(ptr %9) #0 [ "funclet"(token %8) ]
+ %13 = load i32, ptr %12, align 4
+ call void @__cxa_end_catch() #0 [ "funclet"(token %8) ]
+ catchret from %8 to label %try.cont18
+
+rethrow6: ; preds = %catch.start4
+ invoke void @llvm.wasm.rethrow() #1 [ "funclet"(token %8) ]
+ to label %unreachable unwind label %catch.dispatch11
+
+catch.dispatch: ; preds = %invoke.cont1, %invoke.cont, %entry
+ %14 = catchswitch within none [label %catch.start] unwind label %catch.dispatch3
+
+catch.start: ; preds = %catch.dispatch
+ %15 = catchpad within %14 [ptr @_ZTIi]
+ %16 = call ptr @llvm.wasm.get.exception(token %15)
+ %17 = call i32 @llvm.wasm.get.ehselector(token %15)
+ %18 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #0
+ %matches = icmp eq i32 %17, %18
+ br i1 %matches, label %catch, label %rethrow
+
+catch: ; preds = %catch.start
+ %19 = call ptr @__cxa_begin_catch(ptr %16) #0 [ "funclet"(token %15) ]
+ %20 = load i32, ptr %19, align 4
+ call void @__cxa_end_catch() #0 [ "funclet"(token %15) ]
+ catchret from %15 to label %try.cont18
+
+rethrow: ; preds = %catch.start
+ invoke void @llvm.wasm.rethrow() #1 [ "funclet"(token %15) ]
+ to label %unreachable unwind label %catch.dispatch3
+
+try.cont18: ; preds = %catch, %catch7, %catch15, %invoke.cont1
+ ret void
+
+unreachable: ; preds = %rethrow, %rethrow6
+ unreachable
+}
+
+; void exception_grouping_2() {
+; try {
+; try {
+; throw 0;
+; } catch (int) { // (a)
+; }
+; } catch (int) { // (b)
+; }
+; try {
+; foo();
+; } catch (int) { // (c)
+; }
+; }
+;
+; Regression test for an ExceptionInfo grouping bug. Because the first (inner)
+; try always throws, both EH pads (b) (catch.start2) and (c) (catch.start10) are
+; dominated by EH pad (a) (catch.start), even though they are not semantically
+; contained in (a)'s exception. Because (a)'s unwind destination is (b), (b)'s
+; exception is taken out of (a)'s. But because (c) is reachable from (b), we
+; should make sure to take out (c)'s exception out of (a)'s exception too.
+define void @exception_grouping_2() personality ptr @__gxx_wasm_personality_v0 {
+entry:
+ %exception = call ptr @__cxa_allocate_exception(i32 4) #1
+ store i32 0, ptr %exception, align 16
+ invoke void @__cxa_throw(ptr %exception, ptr @_ZTIi, ptr null) #3
+ to label %unreachable unwind label %catch.dispatch
+
+catch.dispatch: ; preds = %entry
+ %0 = catchswitch within none [label %catch.start] unwind label %catch.dispatch1
+
+catch.start: ; preds = %catch.dispatch
+ %1 = catchpad within %0 [ptr @_ZTIi]
+ %2 = call ptr @llvm.wasm.get.exception(token %1)
+ %3 = call i32 @llvm.wasm.get.ehselector(token %1)
+ %4 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #1
+ %matches = icmp eq i32 %3, %4
+ br i1 %matches, label %catch, label %rethrow
+
+catch: ; preds = %catch.start
+ %5 = call ptr @__cxa_begin_catch(ptr %2) #1 [ "funclet"(token %1) ]
+ %6 = load i32, ptr %5, align 4
+ call void @__cxa_end_catch() #1 [ "funclet"(token %1) ]
+ catchret from %1 to label %try.cont8
+
+rethrow: ; preds = %catch.start
+ invoke void @llvm.wasm.rethrow() #3 [ "funclet"(token %1) ]
+ to label %unreachable unwind label %catch.dispatch1
+
+catch.dispatch1: ; preds = %rethrow, %catch.dispatch
+ %7 = catchswitch within none [label %catch.start2] unwind to caller
+
+catch.start2: ; preds = %catch.dispatch1
+ %8 = catchpad within %7 [ptr @_ZTIi]
+ %9 = call ptr @llvm.wasm.get.exception(token %8)
+ %10 = call i32 @llvm.wasm.get.ehselector(token %8)
+ %11 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #1
+ %matches3 = icmp eq i32 %10, %11
+ br i1 %matches3, label %catch5, label %rethrow4
+
+catch5: ; preds = %catch.start2
+ %12 = call ptr @__cxa_begin_catch(ptr %9) #1 [ "funclet"(token %8) ]
+ %13 = load i32, ptr %12, align 4
+ call void @__cxa_end_catch() #1 [ "funclet"(token %8) ]
+ catchret from %8 to label %try.cont8
+
+rethrow4: ; preds = %catch.start2
+ call void @llvm.wasm.rethrow() #3 [ "funclet"(token %8) ]
+ unreachable
+
+try.cont8: ; preds = %catch, %catch5
+ invoke void @foo()
+ to label %try.cont16 unwind label %catch.dispatch9
+
+catch.dispatch9: ; preds = %try.cont8
+ %14 = catchswitch within none [label %catch.start10] unwind to caller
+
+catch.start10: ; preds = %catch.dispatch9
+ %15 = catchpad within %14 [ptr @_ZTIi]
+ %16 = call ptr @llvm.wasm.get.exception(token %15)
+ %17 = call i32 @llvm.wasm.get.ehselector(token %15)
+ %18 = call i32 @llvm.eh.typeid.for(ptr @_ZTIi) #1
+ %matches11 = icmp eq i32 %17, %18
+ br i1 %matches11, label %catch13, label %rethrow12
+
+catch13: ; preds = %catch.start10
+ %19 = call ptr @__cxa_begin_catch(ptr %16) #1 [ "funclet"(token %15) ]
+ %20 = load i32, ptr %19, align 4
+ call void @__cxa_end_catch() #1 [ "funclet"(token %15) ]
+ catchret from %15 to label %try.cont16
+
+rethrow12: ; preds = %catch.start10
+ call void @llvm.wasm.rethrow() #3 [ "funclet"(token %15) ]
+ unreachable
+
+try.cont16: ; preds = %try.cont8, %catch13
+ ret void
+
+unreachable: ; preds = %rethrow, %entry
+ unreachable
+}
+
+; Check if the unwind destination mismatch stats are correct
+; NOSORT: 23 wasm-cfg-stackify - Number of call unwind mismatches found
+; NOSORT: 4 wasm-cfg-stackify - Number of catch unwind mismatches found
+
+declare void @foo()
+declare void @bar()
+declare i32 @baz()
+declare i32 @qux(i32)
+declare void @quux(i32)
+declare void @fun(i32)
+; Function Attrs: nounwind
+declare void @nothrow(i32) #0
+; Function Attrs: nounwind
+declare i32 @nothrow_i32() #0
+
+; Function Attrs: nounwind
+declare ptr @_ZN6ObjectD2Ev(ptr returned) #0
+ at _ZTI7MyClass = external constant { ptr, ptr }, align 4
+; Function Attrs: nounwind
+declare ptr @_ZN7MyClassD2Ev(ptr returned) #0
+; Function Attrs: nounwind
+declare ptr @_ZN7MyClassC2ERKS_(ptr returned, ptr dereferenceable(4)) #0
+
+declare i32 @__gxx_wasm_personality_v0(...)
+; Function Attrs: nounwind
+declare ptr @llvm.wasm.get.exception(token) #0
+; Function Attrs: nounwind
+declare i32 @llvm.wasm.get.ehselector(token) #0
+declare ptr @__cxa_allocate_exception(i32) #0
+declare void @__cxa_throw(ptr, ptr, ptr)
+; Function Attrs: noreturn
+declare void @llvm.wasm.rethrow() #1
+; Function Attrs: nounwind
+declare i32 @llvm.eh.typeid.for(ptr) #0
+
+declare ptr @__cxa_begin_catch(ptr)
+declare void @__cxa_end_catch()
+declare ptr @__cxa_get_exception_ptr(ptr)
+declare void @_ZSt9terminatev()
+; Function Attrs: nounwind
+declare void @llvm.memcpy.p0.p0.i32(ptr noalias nocapture writeonly, ptr noalias nocapture readonly, i32, i1 immarg) #0
+; Function Attrs: nounwind
+declare void @llvm.memmove.p0.p0.i32(ptr nocapture, ptr nocapture readonly, i32, i1 immarg) #0
+; Function Attrs: nounwind
+declare void @llvm.memset.p0.i32(ptr nocapture writeonly, i8, i32, i1 immarg) #0
+
+attributes #0 = { nounwind }
+attributes #1 = { noreturn }
>From 993b987a8ff47e3adc6a362b3e9443c0d8309a61 Mon Sep 17 00:00:00 2001
From: Heejin Ahn <aheejin at gmail.com>
Date: Mon, 4 Nov 2024 16:19:40 -0800
Subject: [PATCH 2/4] Apply suggestions from code review
Co-authored-by: Derek Schuff <dschuff at chromium.org>
---
llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp b/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
index 715ad1c5f74478..ae25ca0981ed30 100644
--- a/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
+++ b/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
@@ -1299,11 +1299,11 @@ void WebAssemblyCFGStackify::addNestedTryDelegate(
// end_try_table
// ...
// end_block ;; Trampoline BB
-// throw_fef
+// throw_ref
// end_try_table
MachineBasicBlock *
WebAssemblyCFGStackify::getTrampolineBlock(MachineBasicBlock *UnwindDest) {
- // We need one trampoline BB per an unwind destination, even though there are
+ // We need one trampoline BB per unwind destination, even though there are
// multiple try_tables target the same unwind destination. If we have already
// created one for the given UnwindDest, return it.
auto It = UnwindDestToTrampoline.find(UnwindDest);
@@ -1486,7 +1486,7 @@ void WebAssemblyCFGStackify::addNestedTryTable(MachineInstr *RangeBegin,
PreBB->addSuccessor(PostBB);
}
- // Add a 'try_table' instruction in the delegate BB created above.
+ // Add a 'end_try_table' instruction in the EndTryTable BB created above.
MachineInstr *EndTryTable = BuildMI(EndTryTableBB, RangeEnd->getDebugLoc(),
TII.get(WebAssembly::END_TRY_TABLE));
registerTryScope(TryTable, EndTryTable, nullptr);
>From 2345a7e6c797974f251ed9321b25654db7f40f5c Mon Sep 17 00:00:00 2001
From: Heejin Ahn <aheejin at gmail.com>
Date: Tue, 5 Nov 2024 00:21:08 +0000
Subject: [PATCH 3/4] assert -> llvm_unreachable
---
llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp b/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
index ae25ca0981ed30..6c13beb5feaa88 100644
--- a/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
+++ b/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp
@@ -2000,7 +2000,7 @@ static MachineBasicBlock *getSingleUnwindDest(const MachineInstr *TryTable) {
case wasm::WASM_OPCODE_CATCH_ALL_REF:
return TryTable->getOperand(3).getMBB();
default:
- assert(false);
+ llvm_unreachable("try_table: Invalid catch clause\n");
}
}
>From 289e75c211979c53ef7583f7d0d0961c5af14653 Mon Sep 17 00:00:00 2001
From: Heejin Ahn <aheejin at gmail.com>
Date: Tue, 5 Nov 2024 04:26:25 +0000
Subject: [PATCH 4/4] Add -mattr=+-bulk-memory to tests to match #112049
---
llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll b/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll
index c7542bd6eef77f..6df626df08883f 100644
--- a/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll
+++ b/llvm/test/CodeGen/WebAssembly/cfg-stackify-eh.ll
@@ -1,9 +1,9 @@
; REQUIRES: asserts
-; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling | FileCheck %s
-; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling
-; RUN: llc < %s -O0 -disable-wasm-fallthrough-return-opt -verify-machineinstrs -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling | FileCheck %s --check-prefix=NOOPT
-; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling -wasm-disable-ehpad-sort -stats 2>&1 | FileCheck %s --check-prefix=NOSORT
-; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling -wasm-disable-ehpad-sort | FileCheck %s --check-prefix=NOSORT-LOCALS
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling,bulk-memory | FileCheck %s
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling,bulk-memory
+; RUN: llc < %s -O0 -disable-wasm-fallthrough-return-opt -verify-machineinstrs -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling,-bulk-memory | FileCheck %s --check-prefix=NOOPT
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling,-bulk-memory -wasm-disable-ehpad-sort -stats 2>&1 | FileCheck %s --check-prefix=NOSORT
+; RUN: llc < %s -disable-wasm-fallthrough-return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false -machine-sink-split-probability-threshold=0 -cgp-freq-ratio-to-skip-merge=1000 -wasm-enable-eh -wasm-enable-exnref -exception-model=wasm -mattr=+exception-handling,-bulk-memory -wasm-disable-ehpad-sort | FileCheck %s --check-prefix=NOSORT-LOCALS
target triple = "wasm32-unknown-unknown"
More information about the llvm-commits
mailing list