Floating point bum-biting

G'day:
I'll get back to bitching about ColdFusion 9's JSON shortcomings tomorrow. In the mean time, here's some "surprising" floating point inaccuracy weirdness. Well: it's predictable rather than weird, but I did go "WTF?" for a few min before I twigged.

There's a new bug in the bug tracker: "3587263 - MOD operator producing incorrect results". The gist of it is this:

<cfset theValue = 1.15>
<cfset theMultiplier = 100>
<cfset theDivisor = 5>
<cfset theProduct = theValue * theMultiplier>
<cfoutput>
#theProduct# % #theDivisor# = #theProduct % theDivisor#<br>
#theProduct# / #theDivisor# = #theProduct / theDivisor#<br>
#theProduct# \ #theDivisor# = #theProduct \ theDivisor#<br>
</cfoutput>

And the output:

115 % 5 = 4
115 / 5 = 23
115 \ 5 = 22

Huh? Clearly the answers ought to be 0, 23, 23.

I figured it was some floating point dodginess with "1.15" not being easy to represent in binary, so did some precisionEvaluate()-ion (!) instead:

<cfset thePreciseProduct = precisionEvaluate(theValue * theMultiplier)>
<cfoutput>
#thePreciseProduct# % #theDivisor# = #thePreciseProduct % theDivisor#<br>
#thePreciseProduct# / #theDivisor# = #thePreciseProduct / theDivisor#<br>
#thePreciseProduct# \ #theDivisor# = #thePreciseProduct \ theDivisor#<br>
</cfoutput>

Result:

115.00 % 5 = 0
115.00 / 5 = 23
115.00 \ 5 = 23

Hurrah. It's correct.

But what's the story? Let's see what Java has to say about 1.15*100:

<cfset theJavaVersion = createObject("java", "java.lang.Double").init(theValue*theMultiplier)>
<cfoutput>
Output: #theJavaVersion#<br>
toString(): #theJavaVersion.toString()#<br>
</cfoutput>

Answer:

Output: 115
toString(): 114.99999999999999

I'm not that impressed that when simply asked out output the number, CF decides I must've meant "actually, dolly it up and give me the wrong value. Just make it look nice: that's the main thing". All it should be doing is calling the toString() method, as per the second example. Oh well: this is no real surprise, I guess.

Oh for good measure... "What would Javascript do?"

<script>
    theValue = 1.15;
    theMultiplier = 100;
    theDivisor = 5;
    theProduct = theValue * theMultiplier;

    document.write(theProduct + " % " + theDivisor + " = " + (theProduct % theDivisor) + "<br>");
    document.write(theProduct + " / " + theDivisor + " = " + (theProduct / theDivisor) + "<br>");
</script>

It would do this:

114.99999999999999 % 5 = 4.999999999999986
114.99999999999999 / 5 = 22.999999999999996

Really... that's the sort of thing that it'd be good for ColdFusion to do. Just do what we ask. Not what it decides we'd be better off having it do.

Anyway, bottom line? Remember that computers don't handle decimal fractions very well at all, so if you seem to be getting odd / incorrect results sometimes... it's probably a floating point accuracy issue.

Back to Friday night TV.

--
Adam