[PATCH] D142178: [X86] Change precision control to FP80 during u64->fp32 conversion on Windows.

Mon Feb 6 07:33:53 PST 2023

craig.topper added a comment.

In D142178#4104248 <https://reviews.llvm.org/D142178#4104248>, @icedrocket wrote:

> I wrote some code to verify that u64 to f64 conversion using FILD+FADD with 53-bit precision is accurate. I tested it with 10^12 cases and the results are: u64 to f32 conversion failed frequently, but u64 to f64 conversion does not failed.
>
> https://reviews.llvm.org/F26366621
> https://reviews.llvm.org/F26366622

The u64->f64 conversion can only fail if PC is set to single precision. The original bug we are fixing is that with PC set to double precision we round from fp80 to fp64 then to fp32. If PC is set to single precision we have a lot more problems with f64.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142178/new/

https://reviews.llvm.org/D142178