Re: [Full-disclosure] Firefox 2.0.0.7 has a very serious calculation bug

On Fri, 28 Sep 2007, carl hardwick wrote:

javascript:5.2-0.1
Firefox 2.0.0.7 result: 5.1000000000000005 (WRONG!)

This is a proper behavior of IEEE 754 64-bit double float, which, IIRC, is
precisely what ECMA standard mandates.

You will get the same from any C-style 'double' arithmetics.

Internet Explorer 7 result: 5.1 (OK)

They use a marginally higher precision. Now try 5.002-.001 - chances are,
you will get 5.00999...

Neither is a "very serious calculation bug". Javascript does not guarantee
- and nowhere actually delivers - arbitrary GMP-style precision.

/mz

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/

Relevant Pages

  • Re: small integer powers -- use pow() or explicit multiplication?
    ... students evaluate the potential loss of accuracy by the implicit ... conversionto float or double. ... but double is not required to be better than IEEE ... IEEE 754 32-bit floating-point does not meet the range and precision ...
    (comp.lang.c)
  • Re: gfortran prints incorrect arithmetic results
    ... 64-bit IEEE has (iirc) 53 bits of mantissa so the ... As Craig says, you apparently are confusing range w/ precision. ...
    (comp.lang.fortran)
  • Re: Losing Precision from FLOAT in DBD::Informix
    ... The versions of CSDK and IDS are also not ... STSM, Informix Database Engineering, IBM Data Management ... >> are declared FLOAT. ... >> - where and why is the precision lost? ...
    (perl.dbi.users)
  • Re: Ruby cant subtract ?
    ... IEEE 754 does an inadequate job of representing arbitrary rational ... The problem is that the 754 representation has finite precision. ... provides only binary floating point representations, ... implementations of IEEE 754 existed before it was a standard, ...
    (comp.lang.ruby)
  • Re: converting float to double
    ... >> insists that I read the stock prices as float. ... >> Since everywhere else the system uses double to hold these prices, ... A 64 bit integer will correctly model currency to 18 digits (with ... calcuations in 110 digits of precision (so that things like interest ...
    (comp.lang.c)