[llvm] r186409 - [APFloat] PR16573: Avoid losing mantissa bits in ppc_fp128 to double truncation

[Ulrich Weigand]

Author: uweigand
Date: Tue Jul 16 08:03:25 2013
New Revision: 186409

URL: http://llvm.org/viewvc/llvm-project?rev=186409&view=rev
Log:

[APFloat] PR16573: Avoid losing mantissa bits in ppc_fp128 to double truncation

When truncating to a format with fewer mantissa bits, APFloat::convert
will perform a right shift of the mantissa by the difference of the
precision of the two formats.  Usually, this will result in just the
mantissa bits needed for the target format.

One special situation is if the input number is denormal.  In this case,
the right shift may discard significant bits.  This is usually not a
problem, since truncating a denormal usually results in zero (underflow)
after normalization anyway, since the result format's exponent range is
usually smaller than the target format's.

However, there is one case where the latter property does not hold:
when truncating from ppc_fp128 to double.  In particular, truncating
a ppc_fp128 whose first double of the pair is denormal should result
in just that first double, not zero.  The current code however
performs an excessive right shift, resulting in lost result bits.
This is then caught in the APFloat::normalize call performed by
APFloat::convert and causes an assertion failure.

This patch checks for the scenario of truncating a denormal, and
attempts to (possibly partially) replace the initial mantissa
right shift by decrementing the exponent, if doing so will still
result in a valid *target format* exponent.


Index: test/CodeGen/PowerPC/pr16573.ll
===================================================================
--- test/CodeGen/PowerPC/pr16573.ll	(revision 0)
+++ test/CodeGen/PowerPC/pr16573.ll	(revision 0)
@@ -0,0 +1,11 @@
+; RUN: llc < %s | FileCheck %s
+
+target triple = "powerpc64-unknown-linux-gnu"
+
+define double @test() {
+  %1 = fptrunc ppc_fp128 0xM818F2887B9295809800000000032D000 to double
+  ret double %1
+}
+
+; CHECK: .quad -9111018957755033591
+
Index: lib/Support/APFloat.cpp
===================================================================
--- lib/Support/APFloat.cpp	(revision 185817)
+++ lib/Support/APFloat.cpp	(working copy)
@@ -1956,6 +1956,23 @@
     X86SpecialNan = true;
   }
 
+  // If this is a truncation of a denormal number, and the target semantics
+  // has larger exponent range than the source semantics (this can happen
+  // when truncating from PowerPC double-double to double format), the
+  // right shift could lose result mantissa bits.  Adjust exponent instead
+  // of performing excessive shift.
+  if (shift < 0 && isFiniteNonZero()) {
+    int exponentChange = significandMSB() + 1 - fromSemantics.precision;
+    if (exponent + exponentChange < toSemantics.minExponent)
+      exponentChange = toSemantics.minExponent - exponent;
+    if (exponentChange < shift)
+      exponentChange = shift;
+    if (exponentChange < 0) {
+      shift -= exponentChange;
+      exponent += exponentChange;
+    }
+  }
+
   // If this is a truncation, perform the shift before we narrow the storage.
   if (shift < 0 && (isFiniteNonZero() || category==fcNaN))
     lostFraction = shiftRight(significandParts(), oldPartCount, -shift);

Added:
    llvm/trunk/test/CodeGen/PowerPC/pr16573.ll
Modified:
    llvm/trunk/lib/Support/APFloat.cpp

Modified: llvm/trunk/lib/Support/APFloat.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/APFloat.cpp?rev=186409&r1=186408&r2=186409&view=diff
==============================================================================
--- llvm/trunk/lib/Support/APFloat.cpp (original)
+++ llvm/trunk/lib/Support/APFloat.cpp Tue Jul 16 08:03:25 2013
@@ -1956,6 +1956,23 @@ APFloat::convert(const fltSemantics &toS
     X86SpecialNan = true;
   }
 
+  // If this is a truncation of a denormal number, and the target semantics
+  // has larger exponent range than the source semantics (this can happen
+  // when truncating from PowerPC double-double to double format), the
+  // right shift could lose result mantissa bits.  Adjust exponent instead
+  // of performing excessive shift.
+  if (shift < 0 && isFiniteNonZero()) {
+    int exponentChange = significandMSB() + 1 - fromSemantics.precision;
+    if (exponent + exponentChange < toSemantics.minExponent)
+      exponentChange = toSemantics.minExponent - exponent;
+    if (exponentChange < shift)
+      exponentChange = shift;
+    if (exponentChange < 0) {
+      shift -= exponentChange;
+      exponent += exponentChange;
+    }
+  }
+
   // If this is a truncation, perform the shift before we narrow the storage.
   if (shift < 0 && (isFiniteNonZero() || category==fcNaN))
     lostFraction = shiftRight(significandParts(), oldPartCount, -shift);

Added: llvm/trunk/test/CodeGen/PowerPC/pr16573.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/PowerPC/pr16573.ll?rev=186409&view=auto
==============================================================================
--- llvm/trunk/test/CodeGen/PowerPC/pr16573.ll (added)
+++ llvm/trunk/test/CodeGen/PowerPC/pr16573.ll Tue Jul 16 08:03:25 2013
@@ -0,0 +1,11 @@
+; RUN: llc < %s | FileCheck %s
+
+target triple = "powerpc64-unknown-linux-gnu"
+
+define double @test() {
+  %1 = fptrunc ppc_fp128 0xM818F2887B9295809800000000032D000 to double
+  ret double %1
+}
+
+; CHECK: .quad -9111018957755033591
+