[libc-commits] [PATCH] D118157: [libc] Improve hypotf performance with different algorithm correctly rounded to all rounding modes.

[Tue Ly via Phabricator via libc-commits]

lntue marked an inline comment as not done.
lntue added a comment.

In D118157#3278638 <https://reviews.llvm.org/D118157#3278638>, @zimmermann6 wrote:

> I'm still running semi-exhaustive tests, it takes some time. I wonder whether a full exhaustive test is possible, by comparing the LLVM implementation with the code from Alexei at https://core-math.gitlabpages.inria.fr/. On a 64-core machine (Intel Xeon Gold 6130 @ 2.10GHz), it takes 4.6s to check 2^33 pairs (x,y). If one tests only positive x,y and x>=y, as exhaustive comparison would have to check 2^61 pairs for each rounding mode, which would take less than 1.5 month using 10000 such machines. This would not be a proof, but the probability that both codes are wrong for the same inputs and give exactly the same wrong answer is quite small.

Or if you don't mind to be slower, you can compare it with the shift-and-add algorithm implemented in the LLVM-libc that this one is trying to speed up, since that can be proved mathematically to be correct.

Another option is that since the idea of this algorithm is scalable, we can have a version of it for half precision (essentially just change the data types and masks/constants), where we it can be tested exhaustively?  That should at least increase the confidence with single, and maybe double precision later?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D118157/new/

https://reviews.llvm.org/D118157