[llvm] [SelectionDAG] Respect no-builtins attribute in SMULO expansion (PR #191020)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-backend-webassembly

Author: Cameron Taggart (cataggar)

<details>
<summary>Changes</summary>

When expanding `ISD::SMULO` for types that require a libcall (e.g. i128 on WebAssembly), the legalizer now checks for the `no-builtins` function attribute. If present, it expands inline instead of emitting a call to `__muloti4`.

This prevents infinite recursion when a custom implementation of `__muloti4` (under a different name) is compiled with `no-builtins`, as the overflow intrinsic introduced by instcombine would otherwise be lowered back to a call to `__muloti4`.

Fixes https://github.com/llvm/llvm-project/issues/189173

---

**Disclosure (per [LLVM AI Tool Use Policy](https://llvm.org/docs/AIToolPolicy.html)):** This patch was drafted with assistance from GitHub Copilot. It was reviewed, tested, and validated locally by @<!-- -->cataggar, who takes full responsibility for the submission and is available to answer reviewer questions.

---
Full diff: https://github.com/llvm/llvm-project/pull/191020.diff


2 Files Affected:

- (modified) llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp (+6-3) 
- (added) llvm/test/CodeGen/WebAssembly/muloti4-no-builtins.ll (+21) 


``````````diff
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp
index 35e61b78e99a2..eed88eb3cdd47 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp
@@ -5301,11 +5301,14 @@ void DAGTypeLegalizer::ExpandIntRes_XMULO(SDNode *N,
   RTLIB::Libcall LC = RTLIB::getMULO(VT);
   RTLIB::LibcallImpl LCImpl = DAG.getLibcalls().getLibcallImpl(LC);
 
-  // If we don't have the libcall or if the function we are compiling is the
-  // implementation of the expected libcall (avoid inf-loop), expand inline.
+  // If we don't have the libcall, if the function we are compiling is the
+  // implementation of the expected libcall (avoid inf-loop), or if the
+  // function has the no-builtins attribute (it may be a custom implementation
+  // of the libcall under a different name), expand inline.
   if (LCImpl == RTLIB::Unsupported ||
       RTLIB::RuntimeLibcallsInfo::getLibcallImplName(LCImpl) ==
-          DAG.getMachineFunction().getName()) {
+          DAG.getMachineFunction().getName() ||
+      DAG.getMachineFunction().getFunction().hasFnAttribute("no-builtins")) {
     // FIXME: This is not an optimal expansion, but better than crashing.
     SDValue MulLo, MulHi;
     TLI.forceExpandWideMUL(DAG, dl, /*Signed=*/true, N->getOperand(0),
diff --git a/llvm/test/CodeGen/WebAssembly/muloti4-no-builtins.ll b/llvm/test/CodeGen/WebAssembly/muloti4-no-builtins.ll
new file mode 100644
index 0000000000000..b75bf4d16fb58
--- /dev/null
+++ b/llvm/test/CodeGen/WebAssembly/muloti4-no-builtins.ll
@@ -0,0 +1,21 @@
+; RUN: llc -asm-verbose=false < %s -wasm-keep-registers | FileCheck %s
+
+; Test that 128-bit smul.with.overflow does not emit a call to __muloti4
+; when the function has the "no-builtins" attribute. This avoids infinite
+; recursion when the function is a custom implementation of __muloti4.
+; See https://github.com/llvm/llvm-project/issues/189173
+
+target triple = "wasm32-unknown-unknown"
+
+define i128 @custom_muloti4(i128 %a, i128 %b) nounwind "no-builtins" {
+entry:
+  %smul = tail call { i128, i1 } @llvm.smul.with.overflow.i128(i128 %a, i128 %b)
+  %cmp = extractvalue { i128, i1 } %smul, 1
+  %smul.result = extractvalue { i128, i1 } %smul, 0
+  %X = select i1 %cmp, i128 %smul.result, i128 42
+  ret i128 %X
+}
+
+; CHECK-NOT: call __muloti4
+
+declare { i128, i1 } @llvm.smul.with.overflow.i128(i128, i128) nounwind readnone

``````````

</details>


https://github.com/llvm/llvm-project/pull/191020