[llvm-dev] Incorrect return values for APFloat::convertFromString?

[Mueller-Roemer, Johannes Sebastian via llvm-dev]

The mantissa size is specified at initialization of the APFloat and the rounding mode shouldn't matter in determining *if* a value was rounded. The strings themselves are exact representations of the same value.

--
Johannes S. Mueller-Roemer, MSc
Wiss. Mitarbeiter - Interactive Engineering Technologies (IET)

Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
Tel +49 6151 155-606  |  Fax +49 6151 155-139
johannes.mueller-roemer at igd.fraunhofer.de  |  www.igd.fraunhofer.de


-----Original Message-----
From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Joerg Sonnenberger via llvm-dev
Sent: Wednesday, March 02, 2016 17:03
To: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] Incorrect return values for APFloat::convertFromString?

On Wed, Mar 02, 2016 at 03:47:43PM +0000, Mueller-Roemer, Johannes Sebastian via llvm-dev wrote:
> 1.       If I pass the hex representation of the closest value to 0.1
> (0x19999Ap-24), everything is fine and opOk is returned. However, if I 
> pass the same value as a decimal string (0.10000002384185791015625), 
> opInexact is set.

Well, they are not the same value. The decimal string depends on the mantissa size and the rounding mode. For the hex version, only the mantissa size has to be large enough to not require truncation/rounding.

Joerg
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev