Tuesday 19 January 2016

Floating point arithmetic with decimals

As a human... what is the value of z, after you process this pseudocode with your wetware:

x = 17.76
y = 100
z = x * y

Hopefully you'd say "1776". It was not a trick question.

And that's an integer, right? Correct.


Now... try this CFML code:

x = 17.76;
y  = 100;
z = x*y;


1776 So far so good.

But what about this:

writeOutput(isValid("integer", z));

You might think "YES" (or true if yer on Lucee), however it's "NO".

And this is where young players fall into the trap. They get all annoyed with isValid() getting it wrong, etc. Which, to be fair, is a reasonable assumption with isValid(), but it's not correct in this instance. It's the young player who is mistaken.

If we now do this:


We get: java.lang.Double

OK, but 1776 can be a Double, sure. But CFML should still consider a Double 1776 as a valid integer, as it should be able to be treated like one. So why doesn't it? What if we circumvent CFML, and go straight to Java:



Boom. Floating point arithmetic inaccuracy.

Never ever ever forget, everyone... when you multiply floating point numbers with decimals... you will get "unexpected" (but you should pretty much expect it!) floating point accuracy issues. This is for the perennial reason that what's easy for us to express in decimal is actually quite hard for a computer to translate into binary accurately.

Aside: we were chatting about all this on the CFML Slack channel this morning, and one person asked "OK, so how come 17.75 x 100 works and 17.76 x 100 does not?". This is because a computer can represent 0.75 in binary exactly (2-1 + 2-2), whereas 0.76 can only be approximated, hence causing the "issue".

The problem really is that CFML should simply output 1776.0000000000002 when we ask it, and it should not try to be clever and hide this stuff. Because it's significant information. Then when the young player output the value, they'd go "oh yeah, better round that" or whatever they need to do before proceeding. CFML is not helping here.

This is pretty ubiquitous in programming. Let's have a trawl through the various languages I can write the simplest of code in:


x = 17.76;
y = 100;
z = x * y


>node jsVersion.js


JS just does what it's told. Unsurprisingly.


x = 17.76
y = 100
z = x * y
println "x * y: " + z

println "x: " + x.getClass().getName()
println "y: " + y.getClass().getName()
println "z: " + z.getClass().getName()
println "z: " + z.toString()

>groovy32 groovyVersion.groovy

x * y: 1776.00
x: java.math.BigDecimal
y: java.lang.Integer
z: java.math.BigDecimal
z: 1776.00

This is interesting. Whilst Groovy keeps the result as a float (specifically a BigDecimal) - which is correct - it truncates it to the total number of decimal places expressed in its factors. That's how I was taught to do it in Physics at school, so I like this. This second example makes it more clear:

x = 3.30
y = 7.70
z = x * y
println "x * y: " + z

println "x: " + x.getClass().getName()
println "y: " + y.getClass().getName()
println "z: " + z.getClass().getName()
println "z: " + z.toString()

>groovy32 more.groovy
x * y: 25.4100
x: java.math.BigDecimal
y: java.math.BigDecimal
z: java.math.BigDecimal
z: 25.4100

In 3.30 and 7.70 there are four decimal places expressed (ie: two for each factor), so Groovy maintains that accuracy. Nice!


import java.math.BigDecimal;

class JavaVersion {

    public static void main(String[] args){
        double x = 17.76;
        int y = 100;
        BigDecimal x2 = new BigDecimal(17.76);
        BigDecimal y2 = new BigDecimal(100);

Here I added a different variation because I was trying to see why the Groovy code behaved the way it did, but it didn't answer my question. I suspected that perhaps it was a BigDecimal thing how it decided on the accuracy of the result, but it wasn't:

>java JavaVersion


This is a good demonstration of how a simply base-10 decimal fraction is actually an irrational number in binary.


function reportResult($z, $label){
    echo "Result with " . $label . ": " . $z . ", integer check: [" . (is_int($z) ? "true" : "false") . "]" . PHP_EOL;

$x = 17.76;
$y = 100;

$z = $x * $y;
reportResult($z, "basic arithmetic");

$z = bcmul($x, $y, 10);
reportResult($z, "Binary Calculator");

Here PHP behaves more like CF, it does away with the 0.00000002 bit, and it is equally unhelpful as to whether the result is an integer:

>php phpVersion.php
Result with basic arithmetic: 1776, integer check: [false]
Result with Binary Calculator: 1776.00, integer check: [false]


I mean... fair enough I guess that 1776.00 might not be considered an integer, but 1776 should be!


Just does what it's told:

x = 17.76
y = 100
z = x * y

puts z

>ruby rubyVersion.rb



As does Python:

x = 17.76
y = 100
z = x * y


>python pythonVersion.py


And Go...

package main

import "fmt"

func main() {
    var x = 17.76
    var y = float64(100)
    var z = x * y

>go run goVersion.go



And - and this is the last one, sorry about the repetition - Clojure is the same:

(def x 17.76)
(def y 100)
(def z (* x y))

(println z)

>lein repl

user=> (load-file "clojureVersion.clj")

I mostly included all those different language variations just to see how different languages did the same (-ish) thing. It's interesting. Well: a bit. Innit?

But anyway... I reckon the answer should be 1776.0000000000002!!! You're not being helpful here, CFML.

Back to CFML again

Wanna no something even more annoying? Check this:

v = val(z);
writeOutput(v & "<br>");
writeOutput(v.getClass().getName() & "<br>");
writeOutput(v.toString() & "<br>");
writeOutput(isValid("integer", v) & "<br>");

This yields:


So the act of calling val() on a number that is 1776.0000000000002 results in it being converted to 1776.0... which is now, apparently, a valid integer. So val() has killed some precision here. It's not called roundAndVal(), so this is the wrong thing to do.

You really dunno what yer doing, do you CFML? Sigh.