[libc-commits] [libc] [llvm] [libc][math][c23] Implement canonicalize functions (PR #85940)

[Joshua Cranmer via libc-commits]

================
@@ -178,6 +181,66 @@ LIBC_INLINE T fdim(T x, T y) {
   return (x > y ? x - y : 0);
 }
 
+template <typename T, cpp::enable_if_t<cpp::is_floating_point_v<T>, int> = 0>
+LIBC_INLINE int canonicalize(T &cx, const T &x) {
+  FPBits<T> sx(x);
+  if constexpr (get_fp_type<T>() == FPType::X86_Binary80) {
+    // All the pseudo and unnormal numbers are not canonical.
+    // More precisely :
+    // Exponent   |       Significand      | Meaning
+    //            | Bits 63-62 | Bits 61-0 |
+    // All Ones   |     00     |    Zero   | Pseudo Infinity, Value = Infinty
----------------
jcranmer-intel wrote:

This is not correct.

The x86 manual states:
> Specifically, the categories of encodings formerly known as pseudo-NaNs, pseudo-infinities, and un-normal
numbers are not supported and should not be used as operand values. The Intel 387 math coprocessor and later
IA-32 processors generate an invalid-operation exception when these encodings are encountered as operands.

So pseudo-infinity and pseudo-denormal should be canonicalized to sNaN, not infinity or finite value (respectively).

https://github.com/llvm/llvm-project/pull/85940