[compiler-rt] [compiler-rt] Add missing carry to 128x128->256 wide multiply (PR #97257)

[via llvm-commits]

https://github.com/beetrees created https://github.com/llvm/llvm-project/pull/97257

The addition `r0 + (r1 << 64)` when calculating `lo` can overflow. This PR makes sure the overflow is carried into `hi`.

Credit to @marthadev for the Rust version of this fix in https://github.com/rust-lang/compiler-builtins/pull/587#issuecomment-2076009499.

I've tested the fix works locally. I've also added a test to this PR for the fix, although I can't run it locally as the `__multf3` test is only run on platforms where `long double` is f128.

(Note to reviewer: I don't have commit access)

Fixes #90009
Fixes #91840

>From 192e36876a5e2123bd1470920dc2745c71de29c8 Mon Sep 17 00:00:00 2001
From: beetrees <b at beetr.ee>
Date: Mon, 1 Jul 2024 06:58:34 +0100
Subject: [PATCH] [compiler-rt] Add missing carry to 128x128->256 wide multiply

---
 compiler-rt/lib/builtins/fp_lib.h            | 5 ++++-
 compiler-rt/test/builtins/Unit/multf3_test.c | 6 ++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/compiler-rt/lib/builtins/fp_lib.h b/compiler-rt/lib/builtins/fp_lib.h
index 8404d98c93508..5658f80cfd0ef 100644
--- a/compiler-rt/lib/builtins/fp_lib.h
+++ b/compiler-rt/lib/builtins/fp_lib.h
@@ -180,8 +180,11 @@ static __inline void wideMultiply(rep_t a, rep_t b, rep_t *hi, rep_t *lo) {
                          (sum2 & Word_FullMask) + ((sum3 << 32) & Word_HiMask);
 
   *lo = r0 + (r1 << 64);
+  // The addition above can overflow, in which case `*lo` will be less than
+  // `r0`. Carry any overflow into `hi`.
+  const bool carry = *lo < r0;
   *hi = (r1 >> 64) + (sum1 >> 96) + (sum2 >> 64) + (sum3 >> 32) + sum4 +
-        (sum5 << 32) + (sum6 << 64);
+        (sum5 << 32) + (sum6 << 64) + carry;
 }
 #undef Word_1
 #undef Word_2
diff --git a/compiler-rt/test/builtins/Unit/multf3_test.c b/compiler-rt/test/builtins/Unit/multf3_test.c
index 543b55899ce82..ac56dc535b883 100644
--- a/compiler-rt/test/builtins/Unit/multf3_test.c
+++ b/compiler-rt/test/builtins/Unit/multf3_test.c
@@ -77,6 +77,12 @@ int main()
                      UINT64_C(0x0),
                      UINT64_C(0x0)))
         return 1;
+    // test carry between lo and hi in widening multiply
+    if (test__multf3(0x0.7fffffffffffffffffffffffffffp-16382L,
+                     0x1.7fffffffffffffffffffffffffffp+1L,
+                     UINT64_C(0x00017fffffffffff),
+                     UINT64_C(0xfffffffffffffffc)))
+        return 1;
 
 #else
     printf("skipped\n");