[lld] [lld][WebAssembly] Split __wasm_apply_data_relocs to respect the maximum function body size (PR #129007)
David Johnson via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 25 09:49:43 PDT 2026
https://github.com/dmjio updated https://github.com/llvm/llvm-project/pull/129007
>From eb41ae124347593a3c2d25a0c671838f5161afc6 Mon Sep 17 00:00:00 2001
From: dmjio <code at dmj.io>
Date: Fri, 25 Sep 2026 11:27:57 -0500
Subject: [PATCH] [lld][WebAssembly] Split __wasm_apply_data_relocs to respect
the maximum function body size
The JS API limits a function body, including its locals declarations, to
7,654,321 bytes (https://www.w3.org/TR/wasm-js-api-2/#limits). In a
-shared/-pie link the relocation code for the data segments all goes into
__wasm_apply_data_relocs, and a large enough program pushes that single
function over the limit, at which point browsers refuse to compile the
module. GHC's WebAssembly backend hits this on ordinary programs, since
every cross-module reference in its generated code is a pointer in a data
segment.
When the code does not fit in one function, emit it as a sequence of
local helper functions, __wasm_apply_data_relocs_<n>, and make
__wasm_apply_data_relocs a sequence of calls to them. Each relocation's
code is generated separately so the decision to start a new function is
exact: a function is started when appending the relocation, plus the
final END, would exceed the limit. Below the limit the output is
unchanged.
The helpers are created after assignIndexes, once function indexes
exist: the calls need the helpers' indexes, and the TLS relocation code
can only be generated then, since it uses the index of __tls_base, a
defined global. Functions can still be added to the function section
at that point, which is what createCommandExportWrappers already does,
so both __wasm_apply_data_relocs and __wasm_apply_tls_relocs are
finished in one place, finalizeRelocationFunction. (The data path keeps
generating its code early, as that decides whether the function exists
at all, and stores it until then.)
The helpers are locally bound. They are only ever called by index, and
hidden visibility is not enough to keep them private: --export-all
exports hidden symbols, which would put __wasm_apply_data_relocs_<n>
into a shared library's exports and make a later link that consumes it
collide with its own helpers.
The limit is exposed as the hidden option --max-function-body-size= so
the splitting can be tested with a small input; the default is the
spec's limit.
This change has been carried by the GHC toolchain's fork of LLVM since
February 2025.
Co-Authored-By: Cheng Shao <terrorjack at type.dance>
---
lld/docs/ReleaseNotes.md | 5 +
lld/test/wasm/apply-data-relocs-split.s | 206 ++++++++++++++++++++++++
lld/test/wasm/apply-tls-relocs-split.s | 64 ++++++++
lld/wasm/Config.h | 4 +
lld/wasm/Driver.cpp | 9 ++
lld/wasm/InputChunks.cpp | 26 ++-
lld/wasm/InputChunks.h | 5 +-
lld/wasm/Options.td | 4 +
lld/wasm/Writer.cpp | 115 +++++++++----
9 files changed, 403 insertions(+), 35 deletions(-)
create mode 100644 lld/test/wasm/apply-data-relocs-split.s
create mode 100644 lld/test/wasm/apply-tls-relocs-split.s
diff --git a/lld/docs/ReleaseNotes.md b/lld/docs/ReleaseNotes.md
index 827d09bec5776..401682d82ac2c 100644
--- a/lld/docs/ReleaseNotes.md
+++ b/lld/docs/ReleaseNotes.md
@@ -52,5 +52,10 @@ from the [LLVM releases web site](https://llvm.org/releases/).
into .bss.common in executable/shared module links, or merging them with max
size/alignment in relocatable -r links). See
https://github.com/WebAssembly/tool-conventions/pull/267
+* `__wasm_apply_data_relocs` (and `__wasm_apply_tls_relocs`) are now split into
+ several functions when the body
+ would otherwise exceed the JS API's limit of 7,654,321 bytes per function
+ body, so that very large `-shared`/`-pie` modules can be instantiated.
+ (https://github.com/llvm/llvm-project/pull/129007)
#### Fixes
diff --git a/lld/test/wasm/apply-data-relocs-split.s b/lld/test/wasm/apply-data-relocs-split.s
new file mode 100644
index 0000000000000..3c229673e0293
--- /dev/null
+++ b/lld/test/wasm/apply-data-relocs-split.s
@@ -0,0 +1,206 @@
+## The runtime relocation code that goes into __wasm_apply_data_relocs is split
+## across several functions when it would exceed the maximum function body
+## size. --max-function-body-size= lowers the limit so that a small input is
+## enough to see it happen.
+
+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown -o %t.o %s
+
+## Each of the five relocations below turns into 10 bytes of code; with the
+## locals declaration and the final END they fit exactly in a 52-byte body.
+
+## Default limit, and a limit they fit in exactly: one function, no helpers.
+# RUN: wasm-ld -shared -o %t.wasm %t.o
+# RUN: obj2yaml %t.wasm | FileCheck %s --check-prefix=ONE
+# RUN: wasm-ld -shared --max-function-body-size=52 -o %t.52.wasm %t.o
+# RUN: obj2yaml %t.52.wasm | FileCheck %s --check-prefix=ONE
+
+## One byte less: four relocations in the first helper and one in the second.
+# RUN: wasm-ld -shared --max-function-body-size=51 -o %t.51.wasm %t.o
+# RUN: obj2yaml %t.51.wasm | FileCheck %s --check-prefix=SPLIT51
+
+## Room for two relocations per function (1 + 2 * 10 + 1 = 22) but not three.
+# RUN: wasm-ld -shared --max-function-body-size=22 -o %t.22.wasm %t.o
+# RUN: obj2yaml %t.22.wasm | FileCheck %s --check-prefix=SPLIT22
+# RUN: llvm-objdump -d --no-show-raw-insn --no-leading-addr %t.22.wasm | FileCheck %s --check-prefix=DIS
+
+## One byte less again: one relocation per function.
+# RUN: wasm-ld -shared --max-function-body-size=21 -o %t.21.wasm %t.o
+# RUN: obj2yaml %t.21.wasm | FileCheck %s --check-prefix=SPLIT21
+
+## The helpers are local: --export-all must not export them (it does export
+## hidden symbols), and a link that consumes a split shared library must not
+## collide with them when it splits its own relocation code.
+# RUN: wasm-ld -shared --export-all --max-function-body-size=22 -o %t.all.wasm %t.o
+# RUN: obj2yaml %t.all.wasm | FileCheck %s --check-prefix=EXPORTALL
+# RUN: wasm-ld -shared --max-function-body-size=22 -o %t.consumer.wasm %t.o %t.all.wasm
+
+## The limit has to be positive.
+# RUN: not wasm-ld -shared --max-function-body-size=0 -o %t.zero.wasm %t.o 2>&1 | FileCheck %s --check-prefix=ZERO
+# RUN: not wasm-ld -shared --max-function-body-size=-1 -o %t.neg.wasm %t.o 2>&1 | FileCheck %s --check-prefix=ZERO
+
+.section .data.ptrs,"",@
+.globl ptrs
+.p2align 2
+ptrs:
+ .int32 target
+ .int32 target
+ .int32 target
+ .int32 target
+ .int32 target
+.size ptrs, 20
+
+.section .data.target,"",@
+.globl target
+.p2align 2
+target:
+ .int32 42
+.size target, 4
+
+# ONE: - Type: EXPORT
+# ONE: Name: __wasm_apply_data_relocs
+# ONE-NEXT: Kind: FUNCTION
+# ONE-NEXT: Index: 1
+# ONE-NOT: __wasm_apply_data_relocs_
+# ONE: - Type: CODE
+# ONE-NEXT: Functions:
+# ONE-NEXT: - Index: 0
+# ONE-NEXT: Locals: []
+# ONE-NEXT: Body: 0B
+# ONE-NEXT: - Index: 1
+# ONE-NEXT: Locals: []
+# ONE-NEXT: Body: 410023006A2302360200410423006A2302360200410823006A2302360200410C23006A2302360200411023006A23023602000B
+# ONE-NEXT: - Type: DATA
+# ONE-NOT: __wasm_apply_data_relocs_
+
+# SPLIT51: - Type: EXPORT
+# SPLIT51: Name: __wasm_apply_data_relocs
+# SPLIT51-NEXT: Kind: FUNCTION
+# SPLIT51-NEXT: Index: 1
+# SPLIT51-NOT: __wasm_apply_data_relocs_
+# SPLIT51: - Type: CODE
+# SPLIT51-NEXT: Functions:
+# SPLIT51-NEXT: - Index: 0
+# SPLIT51-NEXT: Locals: []
+# SPLIT51-NEXT: Body: 0B
+# SPLIT51-NEXT: - Index: 1
+# SPLIT51-NEXT: Locals: []
+# SPLIT51-NEXT: Body: 100210030B
+# SPLIT51-NEXT: - Index: 2
+# SPLIT51-NEXT: Locals: []
+# SPLIT51-NEXT: Body: 410023006A2302360200410423006A2302360200410823006A2302360200410C23006A23023602000B
+# SPLIT51-NEXT: - Index: 3
+# SPLIT51-NEXT: Locals: []
+# SPLIT51-NEXT: Body: 411023006A23023602000B
+# SPLIT51-NEXT: - Type: DATA
+# SPLIT51: FunctionNames:
+# SPLIT51: Name: __wasm_apply_data_relocs_0
+# SPLIT51-NEXT: - Index: 3
+# SPLIT51-NEXT: Name: __wasm_apply_data_relocs_1
+# SPLIT51-NEXT: GlobalNames:
+
+# SPLIT22: - Type: EXPORT
+# SPLIT22: Name: __wasm_apply_data_relocs
+# SPLIT22-NEXT: Kind: FUNCTION
+# SPLIT22-NEXT: Index: 1
+# SPLIT22-NOT: __wasm_apply_data_relocs_
+# SPLIT22: - Type: CODE
+# SPLIT22-NEXT: Functions:
+# SPLIT22-NEXT: - Index: 0
+# SPLIT22-NEXT: Locals: []
+# SPLIT22-NEXT: Body: 0B
+# SPLIT22-NEXT: - Index: 1
+# SPLIT22-NEXT: Locals: []
+# SPLIT22-NEXT: Body: 1002100310040B
+# SPLIT22-NEXT: - Index: 2
+# SPLIT22-NEXT: Locals: []
+# SPLIT22-NEXT: Body: 410023006A2302360200410423006A23023602000B
+# SPLIT22-NEXT: - Index: 3
+# SPLIT22-NEXT: Locals: []
+# SPLIT22-NEXT: Body: 410823006A2302360200410C23006A23023602000B
+# SPLIT22-NEXT: - Index: 4
+# SPLIT22-NEXT: Locals: []
+# SPLIT22-NEXT: Body: 411023006A23023602000B
+# SPLIT22-NEXT: - Type: DATA
+# SPLIT22: FunctionNames:
+# SPLIT22-NEXT: - Index: 0
+# SPLIT22-NEXT: Name: __wasm_call_ctors
+# SPLIT22-NEXT: - Index: 1
+# SPLIT22-NEXT: Name: __wasm_apply_data_relocs
+# SPLIT22-NEXT: - Index: 2
+# SPLIT22-NEXT: Name: __wasm_apply_data_relocs_0
+# SPLIT22-NEXT: - Index: 3
+# SPLIT22-NEXT: Name: __wasm_apply_data_relocs_1
+# SPLIT22-NEXT: - Index: 4
+# SPLIT22-NEXT: Name: __wasm_apply_data_relocs_2
+# SPLIT22-NEXT: GlobalNames:
+
+# DIS: <__wasm_apply_data_relocs>:
+# DIS-EMPTY:
+# DIS-NEXT: call 2
+# DIS-NEXT: call 3
+# DIS-NEXT: call 4
+# DIS-NEXT: end
+# DIS: <__wasm_apply_data_relocs_0>:
+# DIS-EMPTY:
+# DIS-NEXT: i32.const 0
+# DIS-NEXT: global.get 0
+# DIS-NEXT: i32.add
+# DIS-NEXT: global.get 2
+# DIS-NEXT: i32.store 0
+# DIS-NEXT: i32.const 4
+# DIS-NEXT: global.get 0
+# DIS-NEXT: i32.add
+# DIS-NEXT: global.get 2
+# DIS-NEXT: i32.store 0
+# DIS-NEXT: end
+# DIS: <__wasm_apply_data_relocs_1>:
+# DIS-EMPTY:
+# DIS-NEXT: i32.const 8
+# DIS: i32.const 12
+# DIS: end
+# DIS: <__wasm_apply_data_relocs_2>:
+# DIS-EMPTY:
+# DIS-NEXT: i32.const 16
+# DIS-NEXT: global.get 0
+# DIS-NEXT: i32.add
+# DIS-NEXT: global.get 2
+# DIS-NEXT: i32.store 0
+# DIS-NEXT: end
+
+# SPLIT21: - Type: CODE
+# SPLIT21-NEXT: Functions:
+# SPLIT21-NEXT: - Index: 0
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 0B
+# SPLIT21-NEXT: - Index: 1
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 100210031004100510060B
+# SPLIT21-NEXT: - Index: 2
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 410023006A23023602000B
+# SPLIT21-NEXT: - Index: 3
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 410423006A23023602000B
+# SPLIT21-NEXT: - Index: 4
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 410823006A23023602000B
+# SPLIT21-NEXT: - Index: 5
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 410C23006A23023602000B
+# SPLIT21-NEXT: - Index: 6
+# SPLIT21-NEXT: Locals: []
+# SPLIT21-NEXT: Body: 411023006A23023602000B
+# SPLIT21-NEXT: - Type: DATA
+# SPLIT21: Name: __wasm_apply_data_relocs_4
+# SPLIT21-NEXT: GlobalNames:
+
+# EXPORTALL: - Type: EXPORT
+# EXPORTALL: Name: __wasm_apply_data_relocs
+# EXPORTALL-NOT: __wasm_apply_data_relocs_
+# EXPORTALL: - Type: CODE
+# EXPORTALL: FunctionNames:
+# EXPORTALL: Name: __wasm_apply_data_relocs_0
+# EXPORTALL: Name: __wasm_apply_data_relocs_1
+# EXPORTALL: Name: __wasm_apply_data_relocs_2
+
+# ZERO: error: --max-function-body-size=N must be greater than 0
diff --git a/lld/test/wasm/apply-tls-relocs-split.s b/lld/test/wasm/apply-tls-relocs-split.s
new file mode 100644
index 0000000000000..e4b73a1d65afb
--- /dev/null
+++ b/lld/test/wasm/apply-tls-relocs-split.s
@@ -0,0 +1,64 @@
+## __wasm_apply_tls_relocs is split across helper functions in the same way as
+## __wasm_apply_data_relocs when its body would exceed the maximum function
+## body size.
+
+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown -o %t.o %s
+
+## Each of the five relocations below turns into 13 bytes of code; 28 bytes
+## hold two per function (1 + 2 * 13 + 1) but not three.
+
+# RUN: wasm-ld -pie -no-gc-sections --shared-memory --no-entry -o %t.wasm %t.o
+# RUN: obj2yaml %t.wasm | FileCheck %s --check-prefix=ONE
+
+# RUN: wasm-ld -pie -no-gc-sections --shared-memory --no-entry --max-function-body-size=28 -o %t.28.wasm %t.o
+# RUN: obj2yaml %t.28.wasm | FileCheck %s --check-prefix=SPLIT
+
+.section .tdata,"T",@
+.globl tls_sym
+.p2align 2
+tls_sym:
+ .int32 tls_sym
+ .int32 tls_sym
+ .int32 tls_sym
+ .int32 tls_sym
+ .int32 tls_sym
+.size tls_sym, 20
+
+.section .custom_section.target_features,"",@
+ .int8 2
+ .int8 43
+ .int8 7
+ .ascii "atomics"
+ .int8 43
+ .int8 11
+ .ascii "bulk-memory"
+
+# ONE: FunctionNames:
+# ONE: Name: __wasm_apply_tls_relocs
+# ONE-NOT: __wasm_apply_tls_relocs_
+
+## The helpers are not exported, and __wasm_apply_tls_relocs only calls them.
+# SPLIT: - Type: EXPORT
+# SPLIT-NOT: __wasm_apply_tls_relocs
+# SPLIT: - Type: CODE
+# SPLIT: - Index: 3
+# SPLIT-NEXT: Locals: []
+# SPLIT-NEXT: Body: 1004100510060B
+# SPLIT-NEXT: - Index: 4
+# SPLIT-NEXT: Locals: []
+# SPLIT-NEXT: Body: 410023036A230341006A360200410423036A230341006A3602000B
+# SPLIT-NEXT: - Index: 5
+# SPLIT-NEXT: Locals: []
+# SPLIT-NEXT: Body: 410823036A230341006A360200410C23036A230341006A3602000B
+# SPLIT-NEXT: - Index: 6
+# SPLIT-NEXT: Locals: []
+# SPLIT-NEXT: Body: 411023036A230341006A3602000B
+# SPLIT: FunctionNames:
+# SPLIT: Name: __wasm_apply_tls_relocs
+# SPLIT-NEXT: - Index: 4
+# SPLIT-NEXT: Name: __wasm_apply_tls_relocs_0
+# SPLIT-NEXT: - Index: 5
+# SPLIT-NEXT: Name: __wasm_apply_tls_relocs_1
+# SPLIT-NEXT: - Index: 6
+# SPLIT-NEXT: Name: __wasm_apply_tls_relocs_2
+# SPLIT-NEXT: GlobalNames:
diff --git a/lld/wasm/Config.h b/lld/wasm/Config.h
index 260df6f9b8e24..4933bc25eb8da 100644
--- a/lld/wasm/Config.h
+++ b/lld/wasm/Config.h
@@ -97,6 +97,10 @@ struct Config {
uint64_t initialHeap;
uint64_t initialMemory;
uint64_t maxMemory;
+ // Size above which the relocation code the linker synthesizes is split
+ // across several functions. Defaults to the limit set by the JS API; the
+ // option that lowers it exists for testing.
+ uint64_t maxFunctionBodySize;
bool noGrowableMemory;
// The table offset at which to place function addresses. We reserve zero
// for the null function pointer. This gets set to 1 for executables and 0
diff --git a/lld/wasm/Driver.cpp b/lld/wasm/Driver.cpp
index 1de9851d5ed3e..4982b21d2b520 100644
--- a/lld/wasm/Driver.cpp
+++ b/lld/wasm/Driver.cpp
@@ -606,6 +606,15 @@ static void readConfigs(opt::InputArgList &args) {
ctx.arg.initialHeap = args::getInteger(args, OPT_initial_heap, 0);
ctx.arg.initialMemory = args::getInteger(args, OPT_initial_memory, 0);
ctx.arg.maxMemory = args::getInteger(args, OPT_max_memory, 0);
+ // The JS API limits the size of a function body, including its locals
+ // declarations, to 7,654,321 bytes:
+ // https://www.w3.org/TR/wasm-js-api-2/#limits
+ constexpr uint64_t wasmMaxFunctionBodySize = 7654321;
+ int64_t maxFunctionBodySize = args::getInteger(
+ args, OPT_max_function_body_size, wasmMaxFunctionBodySize);
+ if (maxFunctionBodySize <= 0)
+ error("--max-function-body-size=N must be greater than 0");
+ ctx.arg.maxFunctionBodySize = maxFunctionBodySize;
ctx.arg.noGrowableMemory = args.hasArg(OPT_no_growable_memory);
ctx.arg.zStackSize =
args::getZOptionValue(args, OPT_z, "stack-size", WasmDefaultPageSize);
diff --git a/lld/wasm/InputChunks.cpp b/lld/wasm/InputChunks.cpp
index 605d49f08fde7..a11ab970f59f9 100644
--- a/lld/wasm/InputChunks.cpp
+++ b/lld/wasm/InputChunks.cpp
@@ -12,6 +12,7 @@
#include "WriterUtils.h"
#include "lld/Common/ErrorHandler.h"
#include "lld/Common/LLVM.h"
+#include "llvm/ADT/SmallString.h"
#include "llvm/Support/LEB128.h"
#include "llvm/Support/xxhash.h"
#include <algorithm>
@@ -437,12 +438,11 @@ bool isValidRuntimeRelocation(WasmRelocType type) {
// Generate code to apply relocations to the data section at runtime.
// This is only called when generating shared libraries (PIC) where address are
// not known at static link time.
-bool InputChunk::generateRelocationCode(raw_ostream &os) const {
+void InputChunk::generateRelocationCode(std::vector<std::string> &funcs) const {
LLVM_DEBUG(dbgs() << "generating runtime relocations: " << name
<< " count=" << relocations.size() << "\n");
bool is64 = ctx.arg.is64.value_or(false);
- bool generated = false;
unsigned opcode_ptr_add = is64 ? WASM_OPCODE_I64_ADD : WASM_OPCODE_I32_ADD;
uint64_t tombstone = getTombstone();
@@ -463,6 +463,12 @@ bool InputChunk::generateRelocationCode(raw_ostream &os) const {
}
uint64_t offset = getVA(rel.Offset) - getInputSectionOffset();
+
+ // Generate the code for this relocation on its own, so that the check
+ // against the maximum function body size below is exact.
+ SmallString<32> code;
+ raw_svector_ostream os(code);
+
LLVM_DEBUG(dbgs() << "gen reloc: type=" << relocTypeToString(rel.Type)
<< " addend=" << rel.Addend << " index=" << rel.Index
<< " output offset=" << offset << "\n");
@@ -513,9 +519,21 @@ bool InputChunk::generateRelocationCode(raw_ostream &os) const {
writeU8(os, opcode_reloc_store, "I32_STORE");
writeUleb128(os, 2, "align");
writeUleb128(os, 0, "offset");
- generated = true;
+
+ // The writer wraps each entry of `funcs` into a function body, adding a
+ // one-byte locals declaration in front and END behind (hence the 2), and
+ // a body may not exceed ctx.arg.maxFunctionBodySize bytes, so start a
+ // new entry when this relocation would push the current one over. A
+ // relocation whose code is itself larger than the limit (only possible
+ // with an artificially small limit) gets an entry of its own, over it.
+ //
+ // Splitting keeps the unrolled code within the limit; the TODO above (a
+ // relocation table applied by a loop) is what would shrink it.
+ if (funcs.empty() ||
+ funcs.back().size() + code.size() + 2 > ctx.arg.maxFunctionBodySize)
+ funcs.emplace_back();
+ funcs.back().append(code.data(), code.size());
}
- return generated;
}
// Split WASM_SEG_FLAG_STRINGS section. Such a section is a sequence of
diff --git a/lld/wasm/InputChunks.h b/lld/wasm/InputChunks.h
index 0e4b8882c90a3..61ad870d1ce24 100644
--- a/lld/wasm/InputChunks.h
+++ b/lld/wasm/InputChunks.h
@@ -80,7 +80,10 @@ class InputChunk {
size_t getNumRelocations() const { return relocations.size(); }
size_t getNumLiveRelocations() const;
void writeRelocations(llvm::raw_ostream &os) const;
- bool generateRelocationCode(raw_ostream &os) const;
+ // Append this chunk's runtime relocation code to `funcs`, starting a new
+ // entry whenever the current one, once wrapped as a function body, would
+ // exceed ctx.arg.maxFunctionBodySize.
+ void generateRelocationCode(std::vector<std::string> &funcs) const;
bool isTLS() const { return flags & llvm::wasm::WASM_SEG_FLAG_TLS; }
bool isRetained() const { return flags & llvm::wasm::WASM_SEG_FLAG_RETAIN; }
diff --git a/lld/wasm/Options.td b/lld/wasm/Options.td
index a54979527b186..75326be286c11 100644
--- a/lld/wasm/Options.td
+++ b/lld/wasm/Options.td
@@ -244,6 +244,10 @@ def cooperative_threading: FF<"cooperative-threading">,
def max_memory: JJ<"max-memory=">,
HelpText<"Maximum size of the linear memory">;
+def max_function_body_size: JJ<"max-function-body-size=">,
+ HelpText<"Maximum size of the relocation functions the linker synthesizes (for testing)">,
+ Flags<[HelpHidden]>;
+
def no_growable_memory: FF<"no-growable-memory">,
HelpText<"Set maximum size of the linear memory to its initial size">;
diff --git a/lld/wasm/Writer.cpp b/lld/wasm/Writer.cpp
index c7a7ef035ecbf..6529e341fcfcd 100644
--- a/lld/wasm/Writer.cpp
+++ b/lld/wasm/Writer.cpp
@@ -62,6 +62,9 @@ class Writer {
void createInitMemoryFunction();
void createStartFunction();
void createApplyDataRelocationsFunction();
+ void finalizeRelocationFunction(DefinedFunction *entry,
+ std::vector<std::string> &bodies,
+ StringRef helperPrefix);
void createApplyGlobalRelocationsFunction();
void createApplyTLSRelocationsFunction();
void createApplyGlobalTLSRelocationsFunction();
@@ -106,6 +109,12 @@ class Writer {
void writeSections();
void writeBuildId();
+ // __wasm_apply_data_relocs and its relocation code, generated early (to
+ // know whether the function is needed) and turned into bodies late (once
+ // function indexes exist), see finalizeRelocationFunction.
+ DefinedFunction *applyDataRelocs = nullptr;
+ std::vector<std::string> applyDataRelocsBodies;
+
uint64_t fileSize = 0;
std::vector<WasmInitEntry> initFunctions;
@@ -1619,53 +1628,96 @@ void Writer::createStartFunction() {
// apply any relocations to the data segments on startup. This function is
// called `__wasm_apply_data_relocs` and is expected to be called before
// any user code (i.e. before `__wasm_call_ctors`).
-void Writer::createApplyDataRelocationsFunction() {
- LLVM_DEBUG(dbgs() << "createApplyDataRelocationsFunction\n");
- // First write the body's contents to a string.
- std::string bodyContent;
+
+// A function body for the code section: no locals, the code, then END.
+static std::string wrapFunctionBody(StringRef code) {
+ std::string body;
{
- raw_string_ostream os(bodyContent);
+ raw_string_ostream os(body);
writeUleb128(os, 0, "num locals");
- bool generated = false;
- for (const OutputSegment *seg : segments)
- if (!ctx.arg.isMultithreaded() || !seg->isTLS())
- for (const InputChunk *inSeg : seg->inputSegments)
- generated |= inSeg->generateRelocationCode(os);
-
- if (!generated) {
- LLVM_DEBUG(dbgs() << "skipping empty __wasm_apply_data_relocs\n");
- return;
- }
+ os << code;
writeU8(os, WASM_OPCODE_END, "END");
}
+ return body;
+}
+
+void Writer::createApplyDataRelocationsFunction() {
+ LLVM_DEBUG(dbgs() << "createApplyDataRelocationsFunction\n");
+ // Generate the relocation code now, to know whether the function is needed
+ // at all. It only becomes function bodies in finalizeRelocationFunction,
+ // once function indexes are assigned; see there for why.
+ for (const OutputSegment *seg : segments)
+ if (!ctx.arg.isMultithreaded() || !seg->isTLS())
+ for (const InputChunk *inSeg : seg->inputSegments)
+ inSeg->generateRelocationCode(applyDataRelocsBodies);
+
+ if (applyDataRelocsBodies.empty()) {
+ LLVM_DEBUG(dbgs() << "skipping empty __wasm_apply_data_relocs\n");
+ return;
+ }
// __wasm_apply_data_relocs
// Function that applies relocations to data segment post-instantiation.
static WasmSignature nullSignature = {{}, {}};
- auto def = symtab->addSyntheticFunction(
+ applyDataRelocs = symtab->addSyntheticFunction(
"__wasm_apply_data_relocs",
WASM_SYMBOL_VISIBILITY_DEFAULT | WASM_SYMBOL_EXPORTED,
make<SyntheticFunction>(nullSignature, "__wasm_apply_data_relocs"));
- def->markLive();
-
- createFunction(def, bodyContent);
+ applyDataRelocs->markLive();
}
-void Writer::createApplyTLSRelocationsFunction() {
- LLVM_DEBUG(dbgs() << "createApplyTLSRelocationsFunction\n");
- std::string bodyContent;
+// Give `entry` the relocation code in `bodies`. A single body goes into
+// `entry` itself. More than one means the code exceeded the maximum function
+// body size: each body then becomes a local helper function <helperPrefix><n>
+// and `entry` calls them in turn.
+//
+// This runs after assignIndexes: the TLS relocation code can only be
+// generated then (it uses the index of __tls_base, a defined global), and
+// the calls need the helpers' function indexes. So the helpers are placed
+// in the function section directly, as createCommandExportWrappers does for
+// the functions it adds at this point.
+void Writer::finalizeRelocationFunction(DefinedFunction *entry,
+ std::vector<std::string> &bodies,
+ StringRef helperPrefix) {
+ if (bodies.size() <= 1) {
+ StringRef code = bodies.empty() ? StringRef() : StringRef(bodies.front());
+ createFunction(entry, wrapFunctionBody(code));
+ return;
+ }
+
+ static WasmSignature nullSignature = {{}, {}};
+ std::string calls;
{
- raw_string_ostream os(bodyContent);
- writeUleb128(os, 0, "num locals");
- for (const OutputSegment *seg : segments)
- if (seg->isTLS())
- for (const InputChunk *inSeg : seg->inputSegments)
- inSeg->generateRelocationCode(os);
+ raw_string_ostream os(calls);
+ for (size_t i = 0; i < bodies.size(); ++i) {
+ StringRef name = saver().save(Twine(helperPrefix) + Twine(i));
+ auto *func = make<SyntheticFunction>(nullSignature, name);
+ // Local: the helpers are only ever called by index, so they must
+ // neither be exported (which --export-all does even to hidden symbols)
+ // nor be visible to a later link that consumes this module as a shared
+ // library.
+ DefinedFunction *helper =
+ symtab->addSyntheticFunction(name, WASM_SYMBOL_BINDING_LOCAL, func);
+ helper->markLive();
+ out.functionSec->addFunction(func);
+ createFunction(helper, wrapFunctionBody(bodies[i]));
- writeU8(os, WASM_OPCODE_END, "END");
+ writeU8(os, WASM_OPCODE_CALL, "CALL");
+ writeUleb128(os, helper->getFunctionIndex(), "function index");
+ }
}
+ createFunction(entry, wrapFunctionBody(calls));
+}
- createFunction(ctx.sym.applyTLSRelocs, bodyContent);
+void Writer::createApplyTLSRelocationsFunction() {
+ LLVM_DEBUG(dbgs() << "createApplyTLSRelocationsFunction\n");
+ std::vector<std::string> bodies;
+ for (const OutputSegment *seg : segments)
+ if (seg->isTLS())
+ for (const InputChunk *inSeg : seg->inputSegments)
+ inSeg->generateRelocationCode(bodies);
+ finalizeRelocationFunction(ctx.sym.applyTLSRelocs, bodies,
+ "__wasm_apply_tls_relocs_");
}
// Similar to createApplyDataRelocationsFunction but generates relocation code
@@ -1942,6 +1994,9 @@ void Writer::run() {
if (!ctx.arg.relocatable) {
// Create linker synthesized functions
+ if (applyDataRelocs)
+ finalizeRelocationFunction(applyDataRelocs, applyDataRelocsBodies,
+ "__wasm_apply_data_relocs_");
if (ctx.sym.applyGlobalRelocs) {
createApplyGlobalRelocationsFunction();
}
More information about the llvm-commits
mailing list