Bug 366 – Adding trailing zeros to a real literal makes it smaller!

I know that conversions of decimal literals to floating point is always a bit dodgy -- but this particular behaviour is hard to understand. Happens with lots of numbers. static assert( 0.6600_0112_2408_4800_3227e-8L <= 0.6600_0112_2408_4800_32270000e-8L);

Conversion from decimal to binary is done using "round to nearest", not "truncate towards zero". Round to nearest will 50% of the time result in larger numbers than truncating towards 0. Adding 0's to the literal is the same as truncate towards zero.

[email protected] wrote: > http://d.puremagic.com/issues/show_bug.cgi?id=366 > > > [email protected] changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > Status|NEW |RESOLVED > Resolution| |INVALID > > > > > ------- Comment #1 from [email protected] 2006-09-25 13:40 ------- > Conversion from decimal to binary is done using "round to nearest", not > "truncate towards zero". Round to nearest will 50% of the time result in larger > numbers than truncating towards 0. Adding 0's to the literal is the same as > truncate towards zero. In fact, adding ANY digits to the literal makes it smaller. This one also fails: static assert( 0.6600_0112_2408_4800_3227e-8L <= 0.6600_0112_2408_4800_3227999999999999999999999999999999e-8L);

Walter Bright wrote: > Don Clugston wrote: >>> Conversion from decimal to binary is done using "round to nearest", not >>> "truncate towards zero". Round to nearest will 50% of the time result >>> in larger >>> numbers than truncating towards 0. Adding 0's to the literal is the >>> same as >>> truncate towards zero. >> >> In fact, adding ANY digits to the literal makes it smaller. This one >> also fails: >> >> static assert( >> 0.6600_0112_2408_4800_3227e-8L <= >> 0.6600_0112_2408_4800_3227999999999999999999999999999999e-8L); > > Those extra digits are beyond the precision of the floating point > format. Try printing them out in %a format to see the bit pattern. The > behavior is governed by C's strtold, and it does the same thing in gcc. > > I can send you the source to strtold if you want to examine it. Don't worry, I'm a constant user of the %a format! I've been reading a recent draft of the IEEE 754R standard. It states: (here μ = real.dig, but could be quadruple.dig if it were expanded). ----------- If more than μ digits are given for decimal-string to internal conversions, the result of the conversion shall be as if it were carried out in two steps: First round the given number to μ decimal digits, and then convert the resulting μ-digit number to the target format, in both cases rounding correctly according to the prevailing rounding mode. As a consequence of the foregoing, conversions shall be monotonic: increasing the value of an internal floating-point number shall not decrease its value after conversion to an external character sequence, and increasing the value of a external character sequence shall not decrease its value after conversion to an internal floating-point number. ------------ It's the first step (decimal rounding) that appears to be missing. I don't know how much of a change this is from IEEE 754. Without monotonicity, you need to be very careful about increasing the precision of constants.