[PATCH] D159069: [builtins] Fix signed integer overflows in fp_fixint_impl.inc

[Karl-Johan Karlsson via Phabricator via llvm-commits]

Ka-Ka created this revision.
Ka-Ka added reviewers: MaskRay, phosek.
Herald added a subscriber: Enna1.
Herald added a project: All.
Ka-Ka requested review of this revision.
Herald added a project: Sanitizers.
Herald added a subscriber: Sanitizers.

When compiling the builtins with the undefined behavior sanitizer and running
testcases you end up with the following warning:

UBSan: fp_fixint_impl.inc:39:42: left shift of 8388608 by 40 places cannot be represented in type 'fixint_t' (aka 'long long')
UBSan: fp_fixint_impl.inc:39:17: signed integer overflow: -1 * -9223372036854775808 cannot be represented in type 'fixint_t' (aka 'long long')

This can be avoided by doing the shift and the multiplication in a matching
unsigned variant of the type.

This was found in an out of tree target.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D159069

Files:
  compiler-rt/lib/builtins/fp_fixint_impl.inc
  compiler-rt/test/builtins/Unit/fixsfdi_test.c


Index: compiler-rt/test/builtins/Unit/fixsfdi_test.c
===================================================================
--- compiler-rt/test/builtins/Unit/fixsfdi_test.c
+++ compiler-rt/test/builtins/Unit/fixsfdi_test.c
@@ -70,5 +70,8 @@
     if (test__fixsfdi(-0x1.FFFFFCp+62F, 0x8000010000000000LL))
         return 1;
 
-   return 0;
+    if (test__fixsfdi(-0x8000000000000000.0p+0F, 0x8000000000000000LL))
+      return 1;
+
+    return 0;
 }
Index: compiler-rt/lib/builtins/fp_fixint_impl.inc
===================================================================
--- compiler-rt/lib/builtins/fp_fixint_impl.inc
+++ compiler-rt/lib/builtins/fp_fixint_impl.inc
@@ -36,5 +36,5 @@
   if (exponent < significandBits)
     return sign * (significand >> (significandBits - exponent));
   else
-    return sign * ((fixint_t)significand << (exponent - significandBits));
+    return sign * ((fixuint_t)significand << (exponent - significandBits));
 }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D159069.554198.patch
Type: text/x-patch
Size: 946 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230829/e7bd393a/attachment.bin>