[compiler-rt] 831b509 - [builtins] Fix signed integer overflows in fp_fixint_impl.inc

[Karl-Johan Karlsson via llvm-commits]

Author: Karl-Johan Karlsson
Date: 2023-08-31T10:08:15+02:00
New Revision: 831b509d5f73a4e8f86cf01de41c6e96f33de013

URL: https://github.com/llvm/llvm-project/commit/831b509d5f73a4e8f86cf01de41c6e96f33de013
DIFF: https://github.com/llvm/llvm-project/commit/831b509d5f73a4e8f86cf01de41c6e96f33de013.diff

LOG: [builtins] Fix signed integer overflows in fp_fixint_impl.inc

When compiling the builtins with the undefined behavior sanitizer and running
testcases you end up with the following warning:

UBSan: fp_fixint_impl.inc:39:42: left shift of 8388608 by 40 places cannot be represented in type 'fixint_t' (aka 'long long')
UBSan: fp_fixint_impl.inc:39:17: signed integer overflow: -1 * -9223372036854775808 cannot be represented in type 'fixint_t' (aka 'long long')

This can be avoided by doing the shift and the multiplication in a matching
unsigned variant of the type.

The added test only trigger the intended signed overflow case when the builtins
are built with -D__SOFTFP__.

This was found in an out of tree target.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D159069

Added: 
    

Modified: 
    compiler-rt/lib/builtins/fp_fixint_impl.inc
    compiler-rt/test/builtins/Unit/fixsfdi_test.c

Removed: 
    


################################################################################
diff  --git a/compiler-rt/lib/builtins/fp_fixint_impl.inc b/compiler-rt/lib/builtins/fp_fixint_impl.inc
index 2196d712f05f6d..3556bad9990b2f 100644
--- a/compiler-rt/lib/builtins/fp_fixint_impl.inc
+++ b/compiler-rt/lib/builtins/fp_fixint_impl.inc
@@ -36,5 +36,5 @@ static __inline fixint_t __fixint(fp_t a) {
   if (exponent < significandBits)
     return sign * (significand >> (significandBits - exponent));
   else
-    return sign * ((fixint_t)significand << (exponent - significandBits));
+    return sign * ((fixuint_t)significand << (exponent - significandBits));
 }

diff  --git a/compiler-rt/test/builtins/Unit/fixsfdi_test.c b/compiler-rt/test/builtins/Unit/fixsfdi_test.c
index 811c5ff4264415..7dd28a25fd5ac4 100644
--- a/compiler-rt/test/builtins/Unit/fixsfdi_test.c
+++ b/compiler-rt/test/builtins/Unit/fixsfdi_test.c
@@ -70,5 +70,8 @@ int main()
     if (test__fixsfdi(-0x1.FFFFFCp+62F, 0x8000010000000000LL))
         return 1;
 
-   return 0;
+    if (test__fixsfdi(-0x8000000000000000.0p+0F, 0x8000000000000000LL))
+        return 1;
+
+    return 0;
 }