[PATCH] D134725: [ADT] Add support for more formats in APFixedPoint

[Mikael Holmén via Phabricator via llvm-commits]

uabelho added a comment.

In D134725#3842813 <https://reviews.llvm.org/D134725#3842813>, @uabelho wrote:

> In D134725#3842654 <https://reviews.llvm.org/D134725#3842654>, @uabelho wrote:
>
>> Hi,
>> Just a heads-up that we see miscompiles with this patch for our out-of-tree target.
>> It might very well be something in our code that is broken, just thought I'd mention it in case someone else have problems too.
>> I'll try to debug our problems and see what's up.
>
> I don't have any details about how, but it's a cast from a negative fixed point number to an integer that goes wrong now.
> So -0.1r is turned into -1 instead of the expected value 0.

I still really don't know how this should work and what's happening but I added some printouts to getIntPart() and got

  getIntPart: APFixedPoint(-0.0999755859375, {width=16, scale=15, msb=0, lsb=-15, IsSigned=1, HasUnsignedPadding=0, IsSaturated=0})getMsbWeight(): 0
  getLsbWeight(): -15
  ExtVal: -3276
  Val: -3276
  res: -1

So to me it looks like it's converting -0.1 to -1 there, which is wrong i think?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134725/new/

https://reviews.llvm.org/D134725