[PATCH] D68257: [Support] Add mathematical constants

Wed Oct 9 12:24:00 PDT 2019

evandro marked 2 inline comments as done.
evandro added inline comments.

================
Comment at: llvm/include/llvm/Support/MathExtras.h:66
+                 inv_sqrtpi = 0.5641895835477563, // https://oeis.org/A087197
+                 sqrt2      = 1.414213562373095, // https://oeis.org/A002193
+                 inv_sqrt2  = 0.7071067811865475,
----------------
efriedma wrote:
> evandro wrote:
> > efriedma wrote:
> > > evandro wrote:
> > > > efriedma wrote:
> > > > > The correct value of sqrt(2) in double-precision is 1.4142135623730951.
> > > > > 
> > > > > And now I don't trust any of the other values...
> > > > `double` has a precision of 15 or 16 significant digits.  I don't understand why are you suggesting 17 significant digits when you asked to trim the precision down.
> > > > 
> > > > Besides, the reference I provided states that this value is 1.41421356237309505.  Whether it's rounded to 1.4142135623730950 or 1.4142135623730951 is a bit moot, IMO.
> > > I asked for "the smallest number of digits required to produce the correct double-precision result".  This is what you get if, for example, you ask Python 2.7 or later to convert the value to a string with `repr()` (`printf "import math\nprint(repr(math.sqrt(2)))" | python`).  `1.414213562373095` produces a value that's different by one ulp.
> > > 
> > > Yes, a one ulp difference is unlikely to matter for most uses, but if we're going to take the time to define these, we should define them correctly.
> > You're assuming that Python is correct.  `bc` says 1.41421356237309504880.  glibc's `math.h` says 1.41421356237309504880 as well.  And none of these is the same as your 1.4142135623730951.  
> > 
> > As I said, the precision of `double` is 15 to 16 digits and of `float`, 6 to 7 digits.  `math.h` defines them with 20 digits, which is probably an agreeable precision, yes?  But I believe that we call all live with a difference of ±1ulp.
> 1.4142135623730951 is the shortest decimal representation that produces the same double-precision number as  0x1.6a09e667f3bcdP+0.  It isn't "correct" in any other sense, sure.
> 
> A few extra digits is okay, I guess.
Indeed, for some numbers more digits were necessary than for others.  For uniformity's sake, I used the maximum from the number of digits required.

Thank you.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D68257/new/

https://reviews.llvm.org/D68257