[lld] [lld][WebAssembly] Fix spurious signature mismatch under LTO (PR #136197)

[Daniel Bertalan via llvm-commits]

https://github.com/BertalanD created https://github.com/llvm/llvm-project/pull/136197

When generating C++ vtables, Clang declares virtual functions as `void(void)` when their signature is not known (e.g.parameter types are forward-declared). As WASM type checks imports, this would conflict with the real definition during linking. Commit 59f959ff introduced a workaround for this by deferring signature assignment until a definition or direct call is seen.

When performing LTO, LLD first scans the bitcode files and creates `DefinedFunction` symbol table entries for their contents. After LTO codegen, they are replaced with `UndefinedFunction`s (so that the definitions will be pulled in from the native LTO-d files when they are added). At this point, if a function is only referenced in bitcode, its signature remains `nullptr`.

>From here, it should have behaved like in the non-LTO case: the first direct call sets the signature. However, as the `isCalledDirectly` flag was set to true, the missing signature was filled in by the type of the first reference to the function, which could be a `void(void)` vtable entry, which would then conflict with the real definition.

This commit sets `isCalledDirectly` to false so that the signature will only be populated when a direct call is found.

Potentially fixes godotengine/godot#104497

>From 76a10ecf9d080f73c398eea403b5d05463906bd6 Mon Sep 17 00:00:00 2001
From: Daniel Bertalan <dani at danielbertalan.dev>
Date: Thu, 17 Apr 2025 22:14:46 +0200
Subject: [PATCH] [lld][WebAssembly] Fix spurious signature mismatch under LTO

When generating C++ vtables, Clang declares virtual functions as
`void(void)` when their signature is not known (e.g.parameter types are
forward-declared). As WASM type checks imports, this would conflict with
the real definition during linking. Commit 59f959ff introduced a
workaround for this by deferring signature assignment until a definition
or direct call is seen.

When performing LTO, LLD first scans the bitcode files and creates
`DefinedFunction` symbol table entries for their contents. After LTO
codegen, they are replaced with `UndefinedFunction`s (so that the
definitions will be pulled in from the native LTO-d files when they are
added). At this point, if a function is only referenced in bitcode, its
signature remains `nullptr`.

>From here, it should have behaved like in the non-LTO case: the first
direct call sets the signature. However, as the `isCalledDirectly` flag
was set to true, the missing signature was filled in by the type of the
first reference to the function, which could be a `void(void)` vtable
entry, which would then conflict with the real definition.

This commit sets `isCalledDirectly` to false so that the signature will
only be populated when a direct call is found.

Potentially fixes godotengine/godot#104497
---
 .../lto/thinlto-signature-mismatch-unknown.ll | 45 +++++++++++++++++++
 lld/wasm/LTO.cpp                              |  3 +-
 2 files changed, 47 insertions(+), 1 deletion(-)
 create mode 100644 lld/test/wasm/lto/thinlto-signature-mismatch-unknown.ll

diff --git a/lld/test/wasm/lto/thinlto-signature-mismatch-unknown.ll b/lld/test/wasm/lto/thinlto-signature-mismatch-unknown.ll
new file mode 100644
index 0000000000000..2f216e5853ff8
--- /dev/null
+++ b/lld/test/wasm/lto/thinlto-signature-mismatch-unknown.ll
@@ -0,0 +1,45 @@
+; RUN: rm -rf %t && split-file %s %t && cd %t
+; RUN: opt -thinlto-bc a.ll -o a.o
+; RUN: opt -thinlto-bc b.ll -o b.o
+; RUN: llvm-ar rcs b.a b.o
+; RUN: opt -thinlto-bc c.ll -o c.o
+
+;; Taking the address of the incorrectly declared @foo should not generate a warning.
+; RUN: wasm-ld --fatal-warnings --no-entry --export-all a.o b.a -o a.out \
+; RUN:         | FileCheck %s --implicit-check-not 'warning' --allow-empty
+
+;; But we should still warn if we call the function with the wrong signature.
+; RUN: not wasm-ld --fatal-warnings --no-entry --export-all a.o b.a c.o -o b.out 2>&1 \
+; RUN:         | FileCheck %s --check-prefix=INVALID
+
+; INVALID: error: function signature mismatch: foo
+; INVALID: >>> defined as () -> void
+; INVALID: >>> defined as () -> i32
+
+;--- a.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+ at ptr = constant ptr @foo
+declare void @foo()
+
+;--- b.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+define i32 @foo() noinline {
+entry:
+  ret i32 42
+}
+
+;--- c.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+declare void @foo()
+
+define void @invalid() {
+entry:
+    call void @foo()
+    ret void
+}
diff --git a/lld/wasm/LTO.cpp b/lld/wasm/LTO.cpp
index b9bd48acd6dc1..ab63281012eae 100644
--- a/lld/wasm/LTO.cpp
+++ b/lld/wasm/LTO.cpp
@@ -108,9 +108,10 @@ BitcodeCompiler::~BitcodeCompiler() = default;
 
 static void undefine(Symbol *s) {
   if (auto f = dyn_cast<DefinedFunction>(s))
+    // If the signature is null, there were no calls from non-bitcode objects.
     replaceSymbol<UndefinedFunction>(f, f->getName(), std::nullopt,
                                      std::nullopt, 0, f->getFile(),
-                                     f->signature);
+                                     f->signature, f->signature != nullptr);
   else if (isa<DefinedData>(s))
     replaceSymbol<UndefinedData>(s, s->getName(), 0, s->getFile());
   else