Wednesday, 4 May 2022

CFML: updates to my TinyTestFramework

G'day:

Just to pass the time / avoid other things I really ought to be doing instead, over the last few evenings I'm been messing around with my TinyTestFramework. I first created this as an exercise in doing some "real world" TDD for a blog article: "TDD: writing a micro testing framework, using the framework to test itself as I build it". The other intent of this work is so I can run actual tests in my code on trycf.com. This is useful when I'm both asking and answering CFML questions I encounter on the CFML Slack and other places.

The first iteration of the framework was pretty minimal. It was just this:

void function describe(required string label, required function testGroup) {
    try {
        writeOutput("#label#<br>")
        testGroup()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}
void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#: ")
        implementation()
        writeOutput("OK<br>")
    } catch (TestFailedException e) {
        writeOutput("Failed<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TestFailedException")
    }}
}

That's it.

But it let me write tests in a Jasmine/TestBox sort of way, right there in trycf.com:

describe("describe", () => {
    it("it is a test", () => {
        expect(true).toBe(true)
    })
}) 

And this would output:

describe
it is a test: OK

That's cool. That was a good MVP. And I actually use it on trycf.com.

However I quickly felt that only having the one toBe matcher was limiting, and made my tests less clear than they could be. Especially when I wanted to expect null or an exception. So… I messed around some more.

I'm not going to take you through the full TDD exercise of writing all this, but I assure you I TDDed almost all of it (I forgot with a coupla small tweaks, I have to admit. I'm not perfect).

But here's the code (also as a gist), for those that are interested:

<style>
    .tinyTest {background-color: black; color:white; font-family:monospace}
    .tinyTest div {margin-left: 1em}
    .tinyTest .pass {color:green;}
    .tinyTest .fail {color:red;}
    .tinyTest .error {background-color:red; color:black}
</style>
<cfscript>
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TinyTest.TestFailedException")
    }}
}

tinyTest = {
    describe = (string label, function testGroup) => {
        tinyTest.inDiv(() => {
            try {
                writeOutput("#label#<br>")
                testGroup()
            } catch (any e) {
                writeOutput("Error: #e.message#<br>")
            }
        })
    },
    it = (string label, function implementation) => {
        tinyTest.inDiv(() => {
            try {
                writeOutput("It #label#: ")
                implementation()
                tinyTest.showPass("OK<br>")
            } catch (TinyTest e) {
                tinyTest.showFail("Failed<br>")
            } catch (any e) {
                tinyTest.showError("Error: #e.message#<br>")
            }
        })
    },
    expect = (any actual) => {
        var proxy.actual = arguments?.actual
    
        return {
            toBe = (expected) => tinyTest.matchers.toBe(expected, actual),
            toBeTrue = () => tinyTest.matchers.toBe(true, actual),
            toBeFalse = () => tinyTest.matchers.toBe(false, actual),
            toBeNull = () => tinyTest.matchers.toBeNull(proxy?.actual),
            toThrow = () => tinyTest.matchers.toThrow(actual),
            toInclude = (needle) => tinyTest.matchers.toInclude(needle, actual),
            notToBe = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(expected, actual)),
            notToBeTrue = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(true, actual)),
            notToBeFalse = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(false, actual)),
            notToBeNull = () => tinyTest.matchers.not(() => tinyTest.matchers.toBeNull(proxy?.actual)),
            notToThrow = () => tinyTest.matchers.not(() => tinyTest.matchers.toThrow(actual)),
            notToInclude = (needle) => tinyTest.matchers.not((needle) => tinyTest.matchers.toInclude(needle, actual))
            
        }
    },
    fail = () => {
        throw(type="TinyTest.FailureException")
    },
    matchers = {},
    inDiv = (callback)  => {
        writeOutput("<div>")
        callBack()
        writeOutput("</div>")
    },
    showPass = (message) => {
        writeOutput('<span class="pass">#message#</span>')
    },
    showFail = (message) => {
        writeOutput('<span class="fail"><em>#message#</em></span>')
    },
    showError = (message) => {
        writeOutput('<span class="error"><strong>#message#</strong></span>')
    }
    
}
tinyTest.matchers.toBe = (expected, actual) => {
    if (actual.equals(expected)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
}
tinyTest.matchers.toBeNull = (actual) => {
    if (isNull(actual)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
}
tinyTest.matchers.toThrow = (callback) => {
    try {
        callback()
        throw(type="TinyTest.TestFailedException")
    } catch (TinyTest.TestFailedException e) {
        rethrow
    } catch (any e) {
        return true
    }
}
tinyTest.matchers.toInclude = (needle, haystack) => tinyTest.matchers.toBe(true, haystack.findNoCase(needle) > 0)

tinyTest.matchers.not = (callback) => {
    try {
        callback()
        throw(type="TinyTest.NotTestFailedException")
    } catch (TinyTest.NotTestFailedException e) {
        throw(type="TinyTest.TestFailedException")
    } catch (TinyTest.TestFailedException e) {
        return true
    } catch (any e) {
        rethrow
    }
}


describe = tinyTest.describe
it = tinyTest.it
expect = tinyTest.expect
fail = tinyTest.fail
</cfscript>

120 lines now.

What functionality has all this added? Well here's the thing with BDD-style tests. I have documentation and proof that it does what it says it does:

Tests of TinyTestFramework
Tests of it
It prefixes its message with it: OK
Tests of expect
It exists: OK
It returns a struct with keys for matcher callbacks: OK
Tests of fail
It fails a test: OK
Test of test result visualisations
It specifies that a pass should have positive emphasis: OK
It specifies that a fail should have negative emphasis: OK
It specifies that an error should have more emphasis than a fail: OK
Tests of matchers
Tests of toBe
It passes if the actual and expected values are equal: OK
It fails if the actual and expected values are not equal: OK
It expects java.lang.String to work with toBe: OK
It expects java.lang.Double to work with toBe: OK
It expects java.lang.Double to work with toBe: OK
It expects java.lang.Boolean to work with toBe: OK
It expects lucee.runtime.type.ArrayImpl to work with toBe: OK
It expects lucee.runtime.type.StructImpl to work with toBe: OK
It expects lucee.runtime.type.QueryImpl to work with toBe: OK
Tests of notToBe
It passes if the actual and expected values are not equal: OK
It fails if the actual and expected values are equal: OK
It expects java.lang.String to work with notToBe: OK
It expects java.lang.Double to work with notToBe: OK
It expects java.lang.Double to work with notToBe: OK
It expects java.lang.Boolean to work with notToBe: OK
It expects lucee.runtime.type.ArrayImpl to work with notToBe: OK
It expects lucee.runtime.type.StructImpl to work with notToBe: OK
It expects lucee.runtime.type.QueryImpl to work with notToBe: OK
Tests of toBeTrue
It passes if the value is true: OK
It fails if the value is false: OK
Tests of notToBeTrue
It passes if the value is not true: OK
It fails if the value is true: OK
Tests of toBeFalse
It passes if the value is false: OK
It fails if the value is true: OK
Tests of notToBeFalse
It passes if the value is not false: OK
It fails if the value is false: OK
Tests of toThrow
It expects an exception to be thrown from its callback argument: OK
It fails if an exception is not thrown from its callback argument: OK
Tests of notToThrow
It passes if the callback does not throw an exception: OK
It fails if the callback does throw an exception: OK
Tests of toBeNull
It passes if the value is null: OK
It fails if the value is not null: OK
Tests of notToBeNull
It passes if the value is not null: OK
It fails if the value is null: OK
Tests of toInclude
It passes if the haystack contains the needle: OK
It passes if the haystack and needle exactly match: OK
It ignores case: OK
It fails if the haystack does not contain the needle: OK
Tests of notToInclude
It passes if a haystack does not contains the needle: OK
It fails if the haystack contains the needle: OK

All nicely indented and emphasised and shit.

So what have I added for this new version?

  • Tidied up the output to make it easier to read
  • Added these matchers:
    • toBeTrue
    • toBeFalse
    • toBeNull
    • toThrow
    • toInclude
  • And not versions of each of those: notToBe, notToThrow, etc
  • Tests for everything: adamcameron/testTinyTestFramework.cfm

And it's tested using itself, obviously. Interestingly / predictably, there >300 lines of test code there. The ratio is 1:2.5 code:tests

What am I gonna do next? I want to improve that toInclude matcher to work on more than just strings. I also want to have a toBeInstanceOf matcher. Also at some point I better do something with checking structs and arrays and that sorta jazz. I've not needed to actually do that stuff yet, so have not bothered to implement them. But I intend to.

Oh… and you can use this yerself in trycf.com via this URL: https://trycf.com/gist/c631c1f47c8addb2d9aa4d7dacad114f/lucee5?setupCodeGistId=816ce84fd991c2682df612dbaf1cad11&theme=monokai.

Righto.

--
Adam