Tuesday, 17 May 2022

If yer a CFML dev, you should consider financially supporting trycf.com

G'day:

I'll keep this on-point today.

If you are a CFML developer, you will be aware and likely use trycf.com. Whenever I have an issue with some CFML that needs to be demonstrated to someone else; eg: I'm asking for help on Slack or Stack Overflow, or demonstrating an answer to someone else's question: I create a portable / repeatable repro case on trycf.com. I use it to demonstrate bugs and behavioural differences to Adobe or Lucee when both vendors don't give the same result from the same code. I use it every day.

I believe trycf.com is the handiest resource available to CFML developers.

Until now Abram has done all the work to create and maintain trycf.com at his own expense, both financially and with his own time. I reached out to him a coupla days ago wanting to see if I could help-out financially, and he has set up a Patreon page for this very reason. His blurb is:

And the Patreon link is: https://www.patreon.com/trycf/posts.

If you don't want to do a monthly payment, there's also an option to flick him some dosh on a one off basis: https://buy.stripe.com/28o8y6dzHh1bezucMM

Both those links are on the trycf.com homepage.

Go give him some money. If you're the purse-string holder at a company: consider giving him some money on behalf of your company too, to make yourselves better CFML Community participants.

Cheers.


Whilst I'm talking about money, don't forget you perhaps also ought to be supporting Lucee. I wrote about this before: If your company (or yourself) makes money using Lucee… you should throw them a bone. Consider supporting them as well if you haven't already.

Righto.

--
Adam

Sunday, 15 May 2022

CFML: fixing a coupla bugs in my recent work on TinyTestFramework

G'day:

Last week I did some more work on my TinyTestFramework:

On Saturday, I found a bug in each of those. Same bug, basically, surfacing in two different ways. Here's an example:

describe("Demonstrating afterEach bug", () => {
    afterEach(() => {
        writeOutput("<br><br>This should be displayed<br><br>")
    })

    describe("Control", () => {
        it("is a passing test, to demonstrate expected behaviour", () => {
            expect(true).toBeTrue()
        })        
    })

    describe("Demonstrating bug", () => {
        it("should run even if the test fails", () => {
            expect(true).toBeFalse()
        })
    })
})

Output:

Demonstrating afterEach bug
Control
It is a passing test, to demonstrate expected behaviour:

This should be displayed

OK
Demonstrating bug
It should run even if the test fails: Failed
Results: [Pass: 1] [Fail: 1] [Error: 0] [Total: 2]

Note how the second This should be displayed is not being displayed. Why's this? It's because, internally, a failing test throws an exception:

toBeTrue = () => tinyTest.matchers.toBe(true, actual),

// ...

toBe = (expected, actual) => {
    if (actual.equals(expected)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
},

And in the implementation of it, an exception is caught before the code handling afterEach has a chance to run:

it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts
                .filter((context) => context.keyExists("beforeEachHandler"))
                .each((context) => context.beforeEachHandler())

            decoratedImplementation = tinyTest.contexts
                .filter((context) => context.keyExists("aroundEachHandler"))
                .reduce((reversed, context) => reversed.prepend(context), [])
                .reduce((decorated, context) => () => context.aroundEachHandler(decorated), implementation)
            decoratedImplementation()

            tinyTest.contexts
                .filter((context) => context.keyExists("afterEachHandler"))
                .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
                .each((context) => context.afterEachHandler())

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

To explain:

  • implementation is the callback from the it in the test suite. The actual test.
  • All that filter / reduce stuff is just handling aroundEach: don't worry about that.
  • After the decoration we run the test.
  • If it fails, it is caught down here.
  • Meaning the handling of the afterEach callback is never run.

This seems fairly easy to sort out:

try {
    decoratedImplementation()
} finally {
    tinyTest.contexts
        .filter((context) => context.keyExists("afterEachHandler"))
        .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
        .each((context) => context.afterEachHandler())
}

Now even if the test fails, the afterEachHandler handler will still be run. And as I'm not catching the exception, it'll still do what it was supposed to. Rerunning the tests demonstrates this bug is fixed:

Demonstrating afterEach bug
Control
It is a passing test, to demonstrate excpected behaviour:

This should be displayed

OK
Demonstrating bug
It should run even if the test fails:

This should be displayed

Failed
Results: [Pass: 1] [Fail: 1] [Error: 0] [Total: 2]

I also ran all the rest of the tests as well, and they all still pass, so I'm pretty confident my fix has had no repercussions.


I've got the same problem with aroundEach: the bit of the handler after the call to run the test was not being run, for the same reason we had with afterEach: a failing or erroring test throws an exception, and the exception is caught before the rest of the aroundEach handler can be run. This seems slightly trickier to handle, as the code to call the test is within the callback the tester provides:

aroundEach((test) => {
    // top bit before calling the test. No problem with this

    test()

    // bottom bit. This is not getting run after a failed / erroring test
})

I can't expect the testing dev to stick handling of the test failure in there. I need to do this within the framework.

How to do this flummoxed me a bit, but I wrote some tests in the mean time to give my brain some time to think about things:

describe("Demonstrating aroundEach bug", () => {
    aroundEach((test) => {
        writeOutput("<br><br>Before the call to the test: this should be displayed<br>")
        test()
        writeOutput("<br>After the call to the test:This should be displayed<br><br>")
    })

    describe("Control", () => {
        it("is a passing test, to demonstrate expected behaviour", () => {
            expect(true).toBeTrue()
        })        
    })

    describe("Demonstrating bug", () => {
        it("should display the 'bottom' message even if the test fails", () => {
            expect(true).toBeFalse()
        })
    })
})

Results:

Demonstrating aroundEach bug
Control
It is a passing test, to demonstrate expected behaviour:

Before the call to the test: this should be displayed

After the call to the test: This should be displayed

OK
Demonstrating bug
It should display the 'bottom' message even if the test fails:

Before the call to the test: this should be displayed
Failed
Results: [Pass: 1] [Fail: 1] [Error: 0] [Total: 2]

See how the second test isn't outputting After the call to the test: This should be displayed.

it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts
                .filter((context) => context.keyExists("beforeEachHandler"))
                .each((context) => context.beforeEachHandler())

            decoratedImplementation = tinyTest.contexts
                .filter((context) => context.keyExists("aroundEachHandler"))
                .reduce((reversed, context) => reversed.prepend(context), [])
                .reduce((decorated, context) => () => context.aroundEachHandler(decorated), implementation)

        try {    
            decoratedImplementation()
        } finally {
            tinyTest.contexts
                .filter((context) => context.keyExists("afterEachHandler"))
                .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
                .each((context) => context.afterEachHandler())
        }

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

The culmination of that deocration code is how any aroundEach handler is called around the test implementation. Somehow I need to do the equivalent of that try / finally here. But it's not so straight forward as I basically need to prevent that call to implementation from erroring until after we bubble out of all the aroundEach handlers. Bear in mind there can be any number of aroundEach handlers to run: one for each level of describe in the tests:

describe("Test of a CFC", () => {

    aroundEach((test) => {
        // something before
        test()
        // something afterwards
    })

    describe("Test of a method", () => {

        aroundEach((test) => {
            // something before
            test()
            // something afterwards
        })

        describe("Test of a specific part of the method's behaviour", () => {

            aroundEach((test) => {
                // something before
                test()
                // something afterwards
            })
            
            it("will have all three of those `aroundEach` handlers run around it",  () => {
                // test stuff
            })
        })
    })
})

OK so I need to put a try / catch around the call to implementation so i can stop it erroring-out too soon. That's easy enough:

decoratedImplementation = tinyTest.contexts
    .filter((context) => context.keyExists("aroundEachHandler"))
    .reduce((reversed, context) => reversed.prepend(context), [])
    .reduce((decorated, context) => () => context.aroundEachHandler(decorated), () => {
        try {
            implementation()
        } catch (any e) {
            // ???
        }                            
    })
    
try {    
    decoratedImplementation()
    // ???
} finally {
    tinyTest.contexts
        .filter((context) => context.keyExists("afterEachHandler"))
        .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
        .each((context) => context.afterEachHandler())
}

But I still need to know about that exception after we finish calling the aroundEach handlers and the test implementation.

I'm not sure I like this implementation, but this is what I have done:

decoratedImplementation = tinyTest.contexts
    .filter((context) => context.keyExists("aroundEachHandler"))
    .reduce((reversed, context) => reversed.prepend(context), [])
    .reduce((decorated, context) => () => context.aroundEachHandler(decorated), () => {
        try {
            implementation()
            tinyTest.testResult = true
        } catch (any e) {
            tinyTest.testResult = e
        }                            
    })
    
try {    
    decoratedImplementation()
    if (!tinyTest.testResult.equals(true)) {
        throw(object=tinyTest.testResult)
    }
} finally {

I set a variable in the calling code either flagging the test worked, or if not: how it failed (or errored). The if it failed, I throw the exception I originally caught.

This works, and both those tests I wrote above, and all the rest of the test suite still passes too. Bug fixed.

I'm still thinking about this though. I feel I have nailed the red/green part of this process, but I still possibly have some refactoring to do. But obvs now I'm safe to do so because everything is tested. Well: except any other bugs I haven't noticed yet :-)

Righto.

--
Adam

Friday, 13 May 2022

CFML: adding aroundEach to TinyTestFramework was way easier than I expected

G'day:

I'm still pottering around with my TinyTestFramework. Last night I added beforeEach and afterEach handlers, but then thought about how the hell I could easily implement aroundEach support, and I could only see about 50% of it, so I decided to sleep on it.

After a night's sleep I spent about 30min before work doing a quick spike (read: no tests, just "will this even work?"), and surprisingly it did work. First time. Well except for a coupla typos, but I nailed the logic first time. I'm sorta halfway chuffed by this, sorta halfway worried that even though what I decided would probably work - and it did - I haven't quite got my head around how it works, or even quite what it's doing. So let's blog about that.

This evening after work I ignored my spike, and wrote some tests. This is another TDD lesson: it's OK to do a spike in yer code without tests and stuff to just prove a proof of concept, but then once yer good with that, put it to once side and go back to writing tests. My spike code is in a completely different environment from the environment I'm writing the tests in.

For the tests, I consider a lot of the testing of the hierarchical mechanics of how a describe / it testing framework works has already been tested extensively by the beforeEach tests (see "CFML: Adding beforeEach handlers to my TinyTestFramework. Another exercise in TDD"), and I don't need to restest that. This is the same reason I tested the afterEach stuff quite lightly too ("CFML: for the sake of completeness, here's the afterEach treatment"). In those afterEach tests I just tested the "afterness" of the way it works, which is the only difference in the implementation of afterEach compared to beforeEach. The chief difference being that when there is a hierarchy of afterEach handlers, they are called in the reverse order from equivalent beforeEach handlers.

For aroundEach what I am testing is the "aroundness" of how it works.

Here are the tests:

describe("Tests of aroundEach", () => {
    describe("Tests hierarchical sequencing", () => {
        result = []
        aroundEach((test) => {
            result.append("aroundEach top level before test")
            test()
            result.append("aroundEach top level after test")
        })
        describe("Tests hierarchical sequencing (second level: no aroundEach in this one)", () => {
            describe("Tests hierarchical sequencing (third level)", () => {
                aroundEach((test) => {
                    result.append("aroundEach third level before test")
                    test()
                    result.append("aroundEach third level after test")
                })
                describe("Tests hierarchical sequencing (inner level)", () => {
                    aroundEach((test) => {
                        result.append("aroundEach inner before test")
                        test()
                        result.append("aroundEach inner after test")
                    })
                    
                    it("is the baseline test", ()=> {
                        expect(true).toBeTrue()
                    })

                    it("tests the aroundEach handlers are called in the correct order", ()=> {
                        result.append("tests the aroundEach handlers are called in the correct order")

                        expect(result).toBe([
                            "aroundEach top level before test",
                            "aroundEach third level before test",
                            "aroundEach inner before test",
                            "aroundEach inner after test",
                            "aroundEach third level after test",
                            "aroundEach top level after test",
                            "aroundEach top level before test",
                            "aroundEach third level before test",
                            "aroundEach inner before test",
                            "tests the aroundEach handlers are called in the correct order"
                        ])
                    })
                })
            })
        })
    })

    describe("Tests with beforeEach and afterEach", () => {
        result = []

        afterEach(() => {
            result.append("set by afterEach")
        })

        beforeEach(() => {
            result.append("set by beforeEach")
        })

        aroundEach((test) => {
            result.append("set by aroundEach before test")
            test()
            result.append("set by aroundEach after test")
        })

        it("is the baseline test", ()=> {
            expect(true).toBeTrue()
        })

        it("tests the aroundEach handlers are called in the correct order", ()=> {
            result.append("tests the aroundEach handlers are called in the correct order")

            expect(result).toBe([
                "set by beforeEach",
                "set by aroundEach before test",
                "set by aroundEach after test",
                "set by afterEach",
                "set by beforeEach",
                "set by aroundEach before test",
                "tests the aroundEach handlers are called in the correct order"
            ])
        })
    })
})

So we have:

  • Test how a collection of hierarchical aroundEach handlers work. It looks like a lot of code, but it's just three nested describe blocks each with an aroundEach handler, before finally a test to observe, and a test that does the observation and tests some expectations on same.
  • Test that aroundEach plays nice with beforeEach and afterEach. Again a few lines of code, but pretty simple stuff, and the expectations are pretty clear and straight forward.

Um. OK. So.

The implementation bit is going to be a bit of an anti-climax after that lot. Firstly there's some scaffolding stuff that's obviously needed:

beforeEach = (callback) => {
    tinyTest.contexts.last().beforeEachHandler = callback
},
afterEach = (callback) => {
    tinyTest.contexts.last().afterEachHandler = callback
},
aroundEach = (callback) => {
    tinyTest.contexts.last().aroundEachHandler = callback
},
// ...
beforeEach = tinyTest.beforeEach
afterEach = tinyTest.afterEach
aroundEach = tinyTest.aroundEach

And the implementation is changing this:

it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts.each((context) => {
                context.keyExists("beforeEachHandler") ? context.beforeEachHandler() : false
            })

            implementation()

            tinyTest.contexts.reduce((reversedContexts, context) => reversedContexts.prepend(context), []).each((context) => {
                context.keyExists("afterEachHandler") ? context.afterEachHandler() : false
            })

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

To be this:

decoratedImplementation = tinyTest.contexts
    .filter((context) => context.keyExists("aroundEachHandler"))
    .reduce((reversed, context) => reversed.prepend(context), [])
    .reduce((decorated, context) => () => context.aroundEachHandler(decorated), implementation)

decoratedImplementation()

That's not much code, but it's a wee bit dense, even with putting each step on a separate line. What are we doing:

  • Taking the context collection.
  • Filtering out all the context objects with no aroundEachHandler element. We don't care about those.
  • Flipping the remainder around, as we need to apply them from the one nearest the test to the furthest.
  • Then, starting with the test implementation callback, sequentially pass it to the preceding callback in the hierarchy, if that makes sense. It's kind of a partial-application of each aroundEach callback I guess.
  • Once we've done that, call the returned function.

In effect what I think I am doing is calling a function that calls each aroundEachHandler callback from outermost to innermost, passing it the next handler down as its argument, all the way until the test implementation gets called at the end.

Once that was working, I realised that the filtering step could be applied to the beforeEach and afterEach calls too:

tinyTest.contexts.each((context) => {
    context.keyExists("beforeEachHandler") ? context.beforeEachHandler() : false
})

Becomes:

tinyTest.contexts
    .filter((context) => context.keyExists("beforeEachHandler"))
    .each((context) => context.beforeEachHandler())

And equivalently with afterEach:

tinyTest.contexts.reduce((reversedContexts, context) => reversedContexts.prepend(context), []).each((context) => {
    context.keyExists("afterEachHandler") ? context.afterEachHandler() : false
})

Becomes:

tinyTest.contexts
    .filter((context) => context.keyExists("afterEachHandler"))
    .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
    .each((context) => context.afterEachHandler())

The usage of ?: didn't sit well with me before, and I like this approach. Of course YMMV.

To copy and paste from last night's article: that's it. I'll add these to the source code:

You know what? I'm quite pleased that the aroundEach handling was so easily solved with just four chained method calls. It seems amazing to me for some reason. I don't mean my code is amazing: I mean the technique is.

Righto.

--
Adam

Thursday, 12 May 2022

CFML: for the sake of completeness, here's the afterEach treatment

G'day:

This immediately follows on from "CFML: Adding beforeEach handlers to my TinyTestFramework. Another exercise in TDD".

Having done the beforeEach implementation for my TinyTestFramework, I reckoned afterEach would be super easy: barely an inconvenience. And indeed it was. Took me about 15min, given most of the logic is the same as for beforeEach.

Here are the tests:

describe("Tests of afterEach", () => {
    it("will not break if an afterEach is not specified for a given describe block", () => {
        expect(true).toBeTrue()
    })
    
    describe("Baseline", () => {
        result = []
        afterEach(() => { 
            result.append("set in afterEach")
        })
        
        it("runs after a test (setup)", () => {
            expect(result).toBe([])
        })
        
        it("runs after a test (test)", () => {
            expect(result).toBe(["set in afterEach"])
        })
    })
    
    describe("Works in a hierarchy (top)", () => {
        result = []

        afterEach(() => { 
            result.append("set in afterEach in outer describe")
        })
        
        describe("Works in a hierarchy (middle)", () => {
            afterEach(() => { 
                result.append("set in afterEach in middle describe")
            })
            
            describe("Works in a hierarchy (inner)", () => {
                afterEach(() => { 
                    result.append("set in afterEach in inner describe")
                })
                
                it("runs all afterEach handlers, from innermost to outermost (setup)", () => {
                    expect(result).toBe([])
                })
                
                it("runs all afterEach handlers, from innermost to outermost (test)", () => {
                    expect(result).toBe([
                        "set in afterEach in inner describe",
                        "set in afterEach in middle describe",
                        "set in afterEach in outer describe"
                    ])
                })
            })
        })
    })
    
    describe("Tests with beforeEach as well", () => {
        result = []
        
        afterEach(() => {
            result.append("set by afterEach")
        })

        beforeEach(() => {
            result.append("set by beforeEach")
        })

        it("is the setup test", () => {
            expect(true).toBeTrue()
        })
        it("tests that both beforeEach and afterEach got run", () => {
            result.append("testing that the preceding setup test had its afterEach called")

            expect(result).toBe([
                "set by beforeEach", // setup test
                "set by afterEach", // setup test
                "set by beforeEach", // this test
                "testing that the preceding setup test had its afterEach called"
            ])
        })
    })
})

These are more superficial than the beforeEach ones because most of it is already tested in those tests. I just test that it's called, works in a hierarchy (no reason why it won't given the implemntation requirements, but it's a belt-n-braces sort of test), and works with a beforeEach in play before. One thing to note is that I need to run a stub/control/setup test before my test of afterEach, because obviously it runs after the test's code, so we can't test what it does with a single test. Hopefully you see what I mean there. That's the chief difference.

The implementation is simple:

beforeEach = (callback) => {
    tinyTest.contexts.last().beforeEachHandler = callback
},
afterEach = (callback) => {
    tinyTest.contexts.last().afterEachHandler = callback
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts.each((context) => {
                context.keyExists("beforeEachHandler") ? context.beforeEachHandler() : false
            })

            implementation()

            tinyTest.contexts.reduce((reversedContexts, context) => reversedContexts.prepend(context), []).each((context) => {
                context.keyExists("afterEachHandler") ? context.afterEachHandler() : false
            })

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},
// ...
afterEach = tinyTest.afterEach

That is literally it.

The only non-obvious thing is this:

tinyTest.contexts.reduce((reversedContexts, context) => reversedContexts.prepend(context), []).each((context) => {
    context.keyExists("afterEachHandler") ? context.afterEachHandler() : false
})

Where the beforeEach handlers are run from the outermost to the innermost context, the afterEach ones are run in the reverse order.

That's it. I'll add these to the source code:

Righto.

--
Adam

CFML: Adding beforeEach handlers to my TinyTestFramework. Another exercise in TDD

G'day:

I have to admit I'm not sure where I'm going with this one yet. I dunno how to implement what I'm needing to do, but I'm gonna start with a test and see where I go from there.

Context: I've been messing around with this TinyTestFramework thing for a bit… it's intended to be a test framework one can run in trycf.com, so I need to squeeze it all into one include file, and at the same time make it not seem too rubbish in the coding dept. The current state of affairs is here: tinyTestFramework.cfm, and its tests: testTinyTestFramework.cfm. Runnable here: on trycf.com

The next thing that has piqued my interest for this is to add beforeEach and afterEach handlers in there too. This will be more of a challenge than the recent "add another matcher" carry on I've done.

First test:

describe("Tests of beforeEach", () => {
    result = ""
    beforeEach(() => {
        result = "set in beforeEach handler"
    })
    
    it("was called before the first test in the set", () => {
        expect(result).toBe("set in beforeEach handler")
    })
})

Right and the first implementation doesn't need to be clever. Just make it pass:

tinyTest = {
    // ...
    beforeEach = (callback) => {
        callback()
    }
}

// ...
beforeEach = tinyTest.beforeEach

This passes. Cool.

That's fine but it's a bit daft. My next test needs to check that beforeEach is called before subsequent tests too. To test this, simply setting a string and checking it's set won't be any use: it'll still be set in the second test too. Well: either set or reset… no way to tell. So I'll make things more intelligent (just a bit):

describe("Tests of beforeEach", () => {
    result = []
    beforeEach(() => {
        result.append("beforeEach")
    })
    
    it("was called before the first test in the set", () => {
        result.append("first test")
        
        expect(result).toBe([
            "beforeEach",
            "first test"
        ])
    })
    
    it("was called before the second test in the set", () => {
        result.append("second test")
        
        expect(result).toBe([
            "beforeEach",
            "first test",
            "beforeEach",
            "second test"
        ])
    })
})

Now each time beforeEach is called it will cumulatively affect the result, so we can test that it's being called for each test. Which of course it is not, currently, so the second test fails.

Note: it's important to consider that in the real world having beforeEach cumulatively change data, and having the sequence the tests are being run be significant - eg: we need the first test to be run before the second test for either test to pass - is really bad form. beforeEach should be idempotent. But given it's what we're actually testing here, this is a reasonable way of testing its behaviour, I think.

Right so currently we are running the beforeEach callback straight away:

beforeEach = (callback) => {
    callback()
}

It needs to be cleverer than that, and only be called when the test is run, which occurs inside it:

it = (string label, function implementation) => {
    tinyTest.inDiv(() => 
        try {
            writeOutput("It #label#: ")
            implementation()
            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

The beforeEach call just has to stick the callback somewhere for later. Hrm. OK:

beforeEachHandler = false,
beforeEach = (callback) => {
    tinyTest.beforeEachHandler = callback
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.beforeEachHandler()
            
            implementation()

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

That works. Although it's dangerously fragile, as that's gonna collapse in a heap if I don't have a beforeEach handler set. I've put this test before those other ones:

describe("Tests without beforeEach", () => {
    it("was called before the first test in the set", () => {
        expect(true).toBe(true)
    })
})

And I get:

Tests of TinyTestFramework
Tests without beforeEach
It was called before the first test in the set: Error: [The function [beforeEachHandler] does not exist in the Struct, only the following functions are available: [append, clear, copy, count, delete, duplicate, each, every, filter, find, findKey, findValue, insert, isEmpty, keyArray, keyExists, keyList, keyTranslate, len, map, reduce, some, sort, toJson, update, valueArray].][]
Tests of beforeEach
It was called before the first test in the set: OK
It was called before the second test in the set: OK
Results: [Pass: 2] [Fail: 0] [Error: 1] [Total: 3]

I need a guard statement around the call to the beforeEach handler:

if (isCustomFunction(tinyTest.beforeEachHandler)) {
    tinyTest.beforeEachHandler()
}

That fixed it.

Next I need to check that the beforeEach handler cascades into nested describe blocks. I've a strong feeling this will "just work":

describe("Tests of beforeEach", () => {
    describe("Testing first level implementation", () => {
        // (tests that were already in place now in here)
    })
    describe("Testing cascade from ancestor", () => {
        result = []
        beforeEach(() => {
            result.append("beforeEach in ancestor")
        })
        describe("Child of parent", () => {
            it("was called even though it is in an ancestor describe block", () => {
                result.append("test in descendant")
                
                expect(result).toBe([
                    "beforeEach in ancestor",
                    "test in descendant"
                ])
            })
        })
    })
})

Note that I have shunted the first lot of tests into their own block now. Also: yeah, this already passes, but I think it's a case of coincidence rather than good design. I'll add another test to demonstrate this:

describe("Tests without beforeEach (bottom)", () => {
    result = []
    it("was called after all other tests", () => {
        result.append("test after any beforeEach implementation")
        
        expect(result).toBe([
            "test after any beforeEach implementation"
        ])
    })
})

This code is right at the bottom of the test suite. If I put a writeDump(result) in there, we'll see why:

implentation (sic) error

After I pressed send on this, I noticed the typo in the test and in the dump above. I fixed the test, but can't be arsed fixing the screen cap. Oops.

You might not have noticed, but I had not VARed that result variable: it's being used by all the tests. This was by design so I could test for leakage, and here we have some: tinyTest.beforeEachHandler has been set in the previous describe block, and it's still set in the following one. We can't be having that: we need to contextualise the handlers to only be in-context within their original describe blocks, and its descendants.

I think all I need to do is to get rid of the handler at the end of the describe implementation:

describe = (string label, function testGroup) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("#label#<br>")
            testGroup()
            tinyTest.beforeEachHandler = false
        } catch (any e) {
            writeOutput("Error: #e.message#<br>")
        }
    })
},

The really seemed easier than I expected it to be. I have a feeling this next step is gonna be trickier though: I need to be able to support multiple sequential handlers, like this:

describe("Multiple sequential handlers", () => {
    beforeEach(() => {
    	result = []
        result.append("beforeEach in outer")
    })
    describe("first descendant of ancestor", () => {
        beforeEach(() => {
            result.append("beforeEach in middle")
        })
        describe("inner descendant of ancestor", () => {
            beforeEach(() => {
                result.append("beforeEach in inner")
            })
            it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
                result.append("test in innermost descendant")
                
                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "test in innermost descendant"
                ])
            })
        })
    })
})

Here we have three nested beforeEach handlers. This fails because we're only storing one, which we can see if we do a dump in the test:

I guess we need to chuck these things into an array instead:

describe = (string label, function testGroup) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("#label#<br>")
            testGroup()
           
        } catch (any e) {
            writeOutput("Error: #e.message#<br>")
        } finally {
            tinyTest.beforeEachHandlers = []
        }
    })
},
beforeEachHandlers = [],
beforeEach = (callback) => {
    tinyTest.beforeEachHandlers.append(callback)
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.beforeEachHandlers.each((handler) => {
                handler()
            })

            implementation()

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

This makes the tests pass, but I know this bit is wrong:

tinyTest.beforeEachHandlers = []

If I have a second test anywhere in that hierarchy, the handlers will have been blown away, and won't run:

describe("Multiple sequential handlers", () => {
    beforeEach(() => {
        result = []
        result.append("beforeEach in outer")
    })
    describe("first descendant of ancestor", () => {
        beforeEach(() => {
            result.append("beforeEach in middle")
        })

        describe("inner descendant of ancestor", () => {
            beforeEach(() => {
                result.append("beforeEach in inner")
            })
            it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
                result.append("test in innermost descendant")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "test in innermost descendant"
                ])
            })
        })

        it("is a test in the middle of the hierarchy, after the inner describe", () => {
            result.append("test after the inner describe")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach in middle",
                "after the inner describe"
            ])
        
        })
        
    })
})

This fails, and a dump shows why:

So I've got no handlers at all (which is correct given my current implementation), but it should still have the "beforeEach in outer" and "beforeEach in middle" handlers for this test. I've deleted too much. Initially I was puzzled why I still had all that stuff in the result still, but then it occurs to me that was the stuff created for the previous test, just with my last "after the inner describe" appended. So that's predictable/"correct" for there being no beforeEach handlers running at all.

I had to think about this a bit. Initially I thought I'd need to concoct some sort of hierarchical data structure to contain the "array" of handlers, but after some thought I think an array is right, it's just that I only need to pop off the last handler, and only if it's the one set in that describe block. Not sure how I'm gonna work that out, but give me a bit…

OK, I think I've got it:

contexts = [],
describe = (string label, function testGroup) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("#label#<br>")
            tinyTest.contexts.push({})
            testGroup()
           
        } catch (any e) {
            writeOutput("Error: #e.message#<br>")
        } finally {
            tinyTest.contexts.pop()
        }
    })
},
beforeEach = (callback) => {
    tinyTest.contexts.last().beforeEachHandler = callback
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts.each((context) => {
                context.keyExists("beforeEachHandler") ? context.beforeEachHandler() : false
            })

            implementation()

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},
  • I maintain an array of contexts.
  • At the beginning of each describe handler I create a context for it - which is just a struct, and push it onto the contexts array.
  • A beforeEach call sticks its handler into the last context struct, which will be the one for the describe that the beforeEach call was made in.
  • When it runs, it iterates over contexts.
  • And if there's a beforeEach handler in a context, then its run.
  • The last thing describe does is to remove its context from the context array.

This means that as each describe block in a hierarchy is run, it "knows" about all the beforeEach handlers created in its ancestors, and during its own run, it adds its own context to that stack. All tests immediately within it, and within any descendant describe blocks will have all the beforeEach handlers down it and including itself. Once it's done, it tidies up after itself, so any subsequently adjacent describe blocks start with on the the their ancestor contexts.

Hopefully one of the code itself, the bulleted list or the narrative paragraph explained what I mean.

As well as the tests I had before this implementation, I added tests for another few scenarios too. Basically any combination / ordering / nesting of describe / it calls I could think of, testing the correct hierarchical sequence of beforeEach handlers was called in the correct order, for the correct test, without interfering with any other test.

describe("Multiple sequential handlers", () => {
    beforeEach(() => {
        result = []
        result.append("beforeEach in outer")
    })
    
    it("is at the top of the hierarchy before any describe", () => {
        result.append("at the top of the hierarchy before any describe")
        
        expect(result).toBe([
            "beforeEach in outer",
            "at the top of the hierarchy before any describe"
        ])
    })
    
    describe("first descendant of ancestor", () => {
        beforeEach(() => {
            result.append("beforeEach in middle")
        })

        it("is a test in the middle of the hierarchy, before the inner describe", () => {
            result.append("test before the inner describe")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach in middle",
                "test before the inner describe"
            ])
        })

        describe("inner descendant of ancestor", () => {
            it("is a test in the bottom of the hierarchy, before the inner beforeEach", () => {
                result.append("in the bottom of the hierarchy, before the inner beforeEach")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "in the bottom of the hierarchy, before the inner beforeEach"
                ])
            })
            beforeEach(() => {
                result.append("beforeEach in inner")
            })
            it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
                result.append("test in innermost descendant")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "test in innermost descendant"
                ])
            })
            it("is another innermost test", () => {
                result.append("is another innermost test")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "is another innermost test"
                ])
            })
        })

        it("is a test in the middle of the hierarchy, after the inner describe", () => {
            result.append("test after the inner describe")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach in middle",
                "test after the inner describe"
            ])
        })
    })
    
    describe("A second describe in the middle tier of the hierarchy", () => {
        beforeEach(() => {
            result.append("beforeEach second middle describe")
        })

        it("is a test in the second describe in the middle tier of the hierarchy", () => {
            result.append("in the second describe in the middle tier of the hierarchy")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach second middle describe",
                "in the second describe in the middle tier of the hierarchy"
            ])
        })
    })
    
    it("is at the top of the hierarchy after any describe", () => {
        result.append("at the top of the hierarchy after any describe")
        
        expect(result).toBe([
            "beforeEach in outer",
            "at the top of the hierarchy after any describe"
        ])
    })
})

All are green, and all the other tests are still green as well. Yay for the testing safety-net that TDD provides for one. I think I have implemented beforeEach now. Implementing afterEach is next, but this should be easy, and just really the same as I have done here, with similar tests.

However I will do that separate to this, and I am gonna press "send" on this, have a beer first.

Code:

Oh: the code:test ratio is now 179:713, or around 1:4.

Righto.

--
Adam

Saturday, 7 May 2022

Running CFML code on trycf.com via a remote HTTP request

G'day:

This ended up being more of a rabbit hole than I expected it to be. But in the process I've learned a bit more about curl, PHP, Python, JS (client). And actually CFML too I guess.

I can't even remember why I needed to do this, but it was something to do with testing that TinyTestFramework I've been blathering about recently.

Anyhow, I decided I needed to run some code locally on my PC which would send some code off to trycf.com, run it, and send me back the response. I figured it'd be doable if I worked out what Abram was doing when I click the "Run Code" button on the trycf.com UI. As it turns out it's just an HTTP post, and I could could re-run the curl captured from my browser easily enough:

Yeah I don't care

Thanks, but before you take time to mention it: I know the code blows out to the right. It doesn't matter, no-one's expecting you to read it really, and the blow-out is just just cosmetic shite.

curl 'https://lucee5-sbx.trycf.com/lucee5/getremote.cfm' \
  -H 'authority: lucee5-sbx.trycf.com' \
  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'cache-control: max-age=0' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundaryaYm5TBPgCaT5n3TK' \
  -H 'cookie: _ga=GA1.2.771352503.1647902596; _gid=GA1.2.1730098774.1651605323; _gat_gtag_UA_35934323_2=1' \
  -H 'dnt: 1' \
  -H 'origin: https://trycf.com' \
  -H 'referer: https://trycf.com/' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="100", "Google Chrome";v="100"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'sec-fetch-dest: iframe' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-site: same-site' \
  -H 'sec-fetch-user: ?1' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36' \
  --data-raw $'------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="setupcode"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="zoom"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="code"\r\n\r\n<cfscript>\r\nwriteOutput("hi")\r\n\r\n</cfscript>\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="postcode"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="key"\r\n\r\nmain1651930152982-9b67997d-643d-3ddf-0b9f-6154282cfcf4\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="asserts"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK--\r\n' \
  --compressed

I can run that in bash and it works fine. BTW, the actual code I'm running is buried in the middle there, There's quite a chunk of overhead to execute that code, and I reckoned the browser was probably being a bit belt-n-braces about all the headers it was sending, and I'd not need most of them. I whittled it down to this:

curl 'https://lucee5-sbx.trycf.com/lucee5/getremote.cfm' \
  -H 'content-type: multipart/form-data; boundary=----__trycf__' \
  --data-raw $'------__trycf__\r\nContent-Disposition: form-data; name="setupcode"\r\n\r\n\r\n------__trycf__\r\nContent-Disposition: form-data; name="code"\r\n\r\n<cfscript>\r\nwriteOutput("hi")\r\n</cfscript>\r\n------__trycf__\r\nContent-Disposition: form-data; name="asserts"\r\n\r\n\r\n------__trycf__--\r\n' \
  --compressed

There's a handy site that converts curls to language-specific implementations, and CFML is one of the options: https://curlconverter.com/#cfml. This was handy in theory, but the HTTP service call it created didn't work. Not its fault: it should have, but it seems CFHTTP can't handle that boundary syntax in the curl. Note that PHP's version also struggled, but the JS (fetch), Java and Python versions all worked fine.

This threw me for a while cos I'm not really that au fait with building HTTP requests by hand, but eventually I cracked it. Don't hand-crank the multipart boundary stuff: let CFML do it for you. So I came up with this proof of concept:

<cfset code = fileRead(expandPath("./code.cfm"))>

<cfhttp method="post" url="https://acf14-sbx.trycf.com/getremote.cfm" result="httpResponse">
    <cfhttpparam type="formField" name="code" value=#code#>
    <cfhttpparam type="formField" name="asserts" value="">
</cfhttp>

<cfif httpResponse.statusCode EQ "200 OK">
    <cfoutput>#httpResponse.fileContent#</cfoutput>
<cfelse>
    <cfdump var="#httpResponse#">
</cfif>

That's pretty simple. And my test code for this is just:

<cfscript>
    name = "Scott Steinbeck"
    writeOutput("G'day #name#")
</cfscript>

Results:

G'day Scott Steinbeck

(It was Scott that asked me to write this up).

That's where I've got to so far. I also want to see how I can include some set-up code. Hang on a sec whilst I watch some more HTTP traffic.

[…]

Oh OK, that was easy:

<cfset framework = fileRead(expandPath("./tinyTestFramework.cfm"))>
<cfset tests = fileRead(expandPath("./tests.cfm"))>

<cfhttp method="post" url="https://acf14-sbx.trycf.com/getremote.cfm" result="httpResponse">
    <cfhttpparam type="formField" name="setupcode" value="#framework#">
    <cfhttpparam type="formField" name="code" value="#tests#">
    <cfhttpparam type="formField" name="asserts" value="">
</cfhttp>

<cfif httpResponse.statusCode EQ "200 OK">
    <cfoutput>#httpResponse.fileContent#</cfoutput>
<cfelse>
    <cfdump var="#httpResponse#">
</cfif>

(code on github)

That example just runs the test for my test framework up on trycf instead of here on my own server. Because I can. At least now I remember why I wanted to do this in the first place, but that will be in a later article.

All these test so far only run on ColdFusion 2021, because that was what I was wanting to do. The other hosts are easy enough to glean just by watching the request when clicking "run code". The Lucee (latest) one is https://lucee5-sbx.trycf.com/lucee5/getremote.cfm

Anyway, not a terribly exciting one this (not like how terribly exciting the shit I write here usually is, eh? EH??), but problem solved and hopefully this will help Scott.

Righto.

--
Adam

ColdFusion: probable bug with the implementation of the rest operator

G'day:

ColdFusion 2021 added the spread and rest operators. These are implemented as two different usages of .... In this article I am going to be making an observation about how the implementation of the rest operator is incomplete and faulty. I first raised this in the CFML Slack channel, but I've now wittered on enough about it to copy it to here.

What does the rest operator do? The docs say:

[The] Rest Operator operator is similar to [the] Spread Operator but behaves in [the] opposite way, while Spread syntax expands the iterables into individual element[s], the Rest syntax collects and condenses them into a single element.

ibid.

That's not so useful out of the context of a discussion on the spread operator, that said. So a test should clarify:

function testRest(first, ...rest) {
    return arguments
}

function run() {
    describe("Testing CF's rest operator, eg: function testRest(first, ...rest)", () => {
        it("combines all latter argument values into one parameter value", () => {
            actual = testRest("first", "second", "third", "fourth")
            expect(actual).toBe([
                first = "first",
                rest = ["second", "third", "fourth"]
            ])
            
             writeDump(var=actual, label="actual/expected")
        })
    })
}

tinyTest.runTests()
  • The rest operator is ... as a prefix to the last parameter in a function signature.
  • Any arguments passed from that position on are combined into that one parameter's value.

So far so good. So what's the problem? I had a use case where I needed this to work for named arguments, not positional ones:

it("combines all latter argument values into one parameter value when using named arguments", () => {
    actual = testRest(first="first", two="second", three="third", four="fourth")
    expect(actual.first).toBe("first")
    expect(actual).toHaveKey("rest")
    expect(actual.rest).toBe({two="second", three="third", four="fourth"})
})

Same as before, just the arguments have names now. This test fails. Why? Because CF completely ignores the rest operator when one uses named arguments. This "passing" test shows the actual (wrong) behaviour:

it("doesn't work at all with named arguments", () => {
    actual = testRest(first="first", two="second", three="third", four="fourth")
    expect(actual.first).toBe("first")
    expect(actual?.rest).toBeNull()
    expect(actual.keyList().listSort("TEXTNOCASE")).toBe("FIRST,four,REST,three,two")
    
     writeDump(var=actual, label="actual")
     writeDump(var={first="first", rest={two="second", three="third", four="fourth"}}, label="expected")
})

As I said, I raised this on the CFML Slack channel. I got one useful response:

I think it's unusual to use rest with named parameters, but, CF supports named parameters as well as positional so I would expect it to work. I'd settle for it being fully documented though as only working with positional.

John Whish

Fair point, and as I said in reply:

My reasoning was remarkably similar. I started thinking "it's a bit unorthodox to use named arguments here", but then I stopped to think… why? And my conclusion was "because in other languages I use this operation there's no such thing as named arguments, so I've not been used to thinking about it", and that was the only reason I could come up with for me to think that. So I binned that thought (other than deciding to ask about it here).

The thing is there's no good reason I can think of that named arguments should not work. One cannot mix named and positional arguments which would be one wrinkle, so it's 100% reliable to take a set of named arguments, and match the argument names to the parameter names in the method signature. There is no ambiguity: any args that have the same name as a param are assigned as that parameter value. All the rest - instead of being passed in as ad-hoc arguments - are handled by the ... operation.

I cannot see a way that there's any ambiguity either. It's 100% "match the named ones, pass all others in the rest param".

What happens if the method call actually specifies a named argument that matched the name of the "rest" param? Same as if one specifies a positional argument in the position of the "rest" param: it doesn't matter. all arguments that don't match other named params are passed in the rest argument value.

I also think that if for some reason named arguments are not supported for use on function using the rest operator, then an exception should be thrown; not simply the code being ignored.

And whatever the behaviour is needs to be documented.

However one spins it, there are at least two bugs here:

  • Either it should work as I'd expect (or some variation thereof, if I have not thought of something), or it should throw an exception.
  • The behaviour should be documented.

I have not raised tickets for these as I'm not really a user of CF any more, so I don't care so much. Enough to raise it with them; not enough to care what they do about it. But CFers probably should care.

NB: I did not realise Lucee did not support the spread and rest operators at all, so I had to take a different approach to my requirement anyhow. I've not decided on the best way as yet.

There is a ticket for them to be implemented in Lucee: LDEV-2201.

The tests I wrote for this can be run on trycf.com.

Righto.

--
Adam

Wednesday, 4 May 2022

CFML: updates to my TinyTestFramework

G'day:

Just to pass the time / avoid other things I really ought to be doing instead, over the last few evenings I'm been messing around with my TinyTestFramework. I first created this as an exercise in doing some "real world" TDD for a blog article: "TDD: writing a micro testing framework, using the framework to test itself as I build it". The other intent of this work is so I can run actual tests in my code on trycf.com. This is useful when I'm both asking and answering CFML questions I encounter on the CFML Slack and other places.

The first iteration of the framework was pretty minimal. It was just this:

void function describe(required string label, required function testGroup) {
    try {
        writeOutput("#label#<br>")
        testGroup()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}
void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#: ")
        implementation()
        writeOutput("OK<br>")
    } catch (TestFailedException e) {
        writeOutput("Failed<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TestFailedException")
    }}
}

That's it.

But it let me write tests in a Jasmine/TestBox sort of way, right there in trycf.com:

describe("describe", () => {
    it("it is a test", () => {
        expect(true).toBe(true)
    })
}) 

And this would output:

describe
it is a test: OK

That's cool. That was a good MVP. And I actually use it on trycf.com.

However I quickly felt that only having the one toBe matcher was limiting, and made my tests less clear than they could be. Especially when I wanted to expect null or an exception. So… I messed around some more.

I'm not going to take you through the full TDD exercise of writing all this, but I assure you I TDDed almost all of it (I forgot with a coupla small tweaks, I have to admit. I'm not perfect).

But here's the code (also as a gist), for those that are interested:

<style>
    .tinyTest {background-color: black; color:white; font-family:monospace}
    .tinyTest div {margin-left: 1em}
    .tinyTest .pass {color:green;}
    .tinyTest .fail {color:red;}
    .tinyTest .error {background-color:red; color:black}
</style>
<cfscript>
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TinyTest.TestFailedException")
    }}
}

tinyTest = {
    describe = (string label, function testGroup) => {
        tinyTest.inDiv(() => {
            try {
                writeOutput("#label#<br>")
                testGroup()
            } catch (any e) {
                writeOutput("Error: #e.message#<br>")
            }
        })
    },
    it = (string label, function implementation) => {
        tinyTest.inDiv(() => {
            try {
                writeOutput("It #label#: ")
                implementation()
                tinyTest.showPass("OK<br>")
            } catch (TinyTest e) {
                tinyTest.showFail("Failed<br>")
            } catch (any e) {
                tinyTest.showError("Error: #e.message#<br>")
            }
        })
    },
    expect = (any actual) => {
        var proxy.actual = arguments?.actual
    
        return {
            toBe = (expected) => tinyTest.matchers.toBe(expected, actual),
            toBeTrue = () => tinyTest.matchers.toBe(true, actual),
            toBeFalse = () => tinyTest.matchers.toBe(false, actual),
            toBeNull = () => tinyTest.matchers.toBeNull(proxy?.actual),
            toThrow = () => tinyTest.matchers.toThrow(actual),
            toInclude = (needle) => tinyTest.matchers.toInclude(needle, actual),
            notToBe = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(expected, actual)),
            notToBeTrue = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(true, actual)),
            notToBeFalse = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(false, actual)),
            notToBeNull = () => tinyTest.matchers.not(() => tinyTest.matchers.toBeNull(proxy?.actual)),
            notToThrow = () => tinyTest.matchers.not(() => tinyTest.matchers.toThrow(actual)),
            notToInclude = (needle) => tinyTest.matchers.not((needle) => tinyTest.matchers.toInclude(needle, actual))
            
        }
    },
    fail = () => {
        throw(type="TinyTest.FailureException")
    },
    matchers = {},
    inDiv = (callback)  => {
        writeOutput("<div>")
        callBack()
        writeOutput("</div>")
    },
    showPass = (message) => {
        writeOutput('<span class="pass">#message#</span>')
    },
    showFail = (message) => {
        writeOutput('<span class="fail"><em>#message#</em></span>')
    },
    showError = (message) => {
        writeOutput('<span class="error"><strong>#message#</strong></span>')
    }
    
}
tinyTest.matchers.toBe = (expected, actual) => {
    if (actual.equals(expected)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
}
tinyTest.matchers.toBeNull = (actual) => {
    if (isNull(actual)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
}
tinyTest.matchers.toThrow = (callback) => {
    try {
        callback()
        throw(type="TinyTest.TestFailedException")
    } catch (TinyTest.TestFailedException e) {
        rethrow
    } catch (any e) {
        return true
    }
}
tinyTest.matchers.toInclude = (needle, haystack) => tinyTest.matchers.toBe(true, haystack.findNoCase(needle) > 0)

tinyTest.matchers.not = (callback) => {
    try {
        callback()
        throw(type="TinyTest.NotTestFailedException")
    } catch (TinyTest.NotTestFailedException e) {
        throw(type="TinyTest.TestFailedException")
    } catch (TinyTest.TestFailedException e) {
        return true
    } catch (any e) {
        rethrow
    }
}


describe = tinyTest.describe
it = tinyTest.it
expect = tinyTest.expect
fail = tinyTest.fail
</cfscript>

120 lines now.

What functionality has all this added? Well here's the thing with BDD-style tests. I have documentation and proof that it does what it says it does:

Tests of TinyTestFramework
Tests of it
It prefixes its message with it: OK
Tests of expect
It exists: OK
It returns a struct with keys for matcher callbacks: OK
Tests of fail
It fails a test: OK
Test of test result visualisations
It specifies that a pass should have positive emphasis: OK
It specifies that a fail should have negative emphasis: OK
It specifies that an error should have more emphasis than a fail: OK
Tests of matchers
Tests of toBe
It passes if the actual and expected values are equal: OK
It fails if the actual and expected values are not equal: OK
It expects java.lang.String to work with toBe: OK
It expects java.lang.Double to work with toBe: OK
It expects java.lang.Double to work with toBe: OK
It expects java.lang.Boolean to work with toBe: OK
It expects lucee.runtime.type.ArrayImpl to work with toBe: OK
It expects lucee.runtime.type.StructImpl to work with toBe: OK
It expects lucee.runtime.type.QueryImpl to work with toBe: OK
Tests of notToBe
It passes if the actual and expected values are not equal: OK
It fails if the actual and expected values are equal: OK
It expects java.lang.String to work with notToBe: OK
It expects java.lang.Double to work with notToBe: OK
It expects java.lang.Double to work with notToBe: OK
It expects java.lang.Boolean to work with notToBe: OK
It expects lucee.runtime.type.ArrayImpl to work with notToBe: OK
It expects lucee.runtime.type.StructImpl to work with notToBe: OK
It expects lucee.runtime.type.QueryImpl to work with notToBe: OK
Tests of toBeTrue
It passes if the value is true: OK
It fails if the value is false: OK
Tests of notToBeTrue
It passes if the value is not true: OK
It fails if the value is true: OK
Tests of toBeFalse
It passes if the value is false: OK
It fails if the value is true: OK
Tests of notToBeFalse
It passes if the value is not false: OK
It fails if the value is false: OK
Tests of toThrow
It expects an exception to be thrown from its callback argument: OK
It fails if an exception is not thrown from its callback argument: OK
Tests of notToThrow
It passes if the callback does not throw an exception: OK
It fails if the callback does throw an exception: OK
Tests of toBeNull
It passes if the value is null: OK
It fails if the value is not null: OK
Tests of notToBeNull
It passes if the value is not null: OK
It fails if the value is null: OK
Tests of toInclude
It passes if the haystack contains the needle: OK
It passes if the haystack and needle exactly match: OK
It ignores case: OK
It fails if the haystack does not contain the needle: OK
Tests of notToInclude
It passes if a haystack does not contains the needle: OK
It fails if the haystack contains the needle: OK

All nicely indented and emphasised and shit.

So what have I added for this new version?

  • Tidied up the output to make it easier to read
  • Added these matchers:
    • toBeTrue
    • toBeFalse
    • toBeNull
    • toThrow
    • toInclude
  • And not versions of each of those: notToBe, notToThrow, etc
  • Tests for everything: adamcameron/testTinyTestFramework.cfm

And it's tested using itself, obviously. Interestingly / predictably, there >300 lines of test code there. The ratio is 1:2.5 code:tests

What am I gonna do next? I want to improve that toInclude matcher to work on more than just strings. I also want to have a toBeInstanceOf matcher. Also at some point I better do something with checking structs and arrays and that sorta jazz. I've not needed to actually do that stuff yet, so have not bothered to implement them. But I intend to.

Oh… and you can use this yerself in trycf.com via this URL: https://trycf.com/gist/c631c1f47c8addb2d9aa4d7dacad114f/lucee5?setupCodeGistId=816ce84fd991c2682df612dbaf1cad11&theme=monokai.

Righto.

--
Adam

Tuesday, 3 May 2022

Work with me here - revisited

G'day

Back in Jan I posted "Work with me here". Once again we are growing our team, and the equivalent job has opened again (this is in addition to the successful hire we made after the January round, not "instead of ~" ;-).

So if you're in the UK, and fancy leveraging your existing strong (strong) CFML skills to get into a position where we'll migrate your capabilities to Kotlin, this could be a good opportunity.

The narrative detail remains the same as in that last article, so go give it a read.

The formal job spec is at https://www.easydirectdebits.co.uk/job/senior-application-developer-uk/, and you can apply from there too.

Or ping me here if you need further info. Any actual applications should use the link above though.

Righto.

--
Adam

Monday, 2 May 2022

CFML: with Lucee, true isn't necessarily the same as true

G'day:

Yesterday I decided to improve my "Tiny Test Framework". I wrote an article about developing the first iterationof this as a TDD exercise six or so months ago: "TDD: writing a micro testing framework, using the framework to test itself as I build it". I use this framework on trycf.com, so I can include tests in my code samples. The first iteration of this only had the one toBe matcher, and yesterday I decided to add in toBeTrue, toBeFalse and toThrow: just to make my sample code on trycf a bit clearer.

Whilst implementing the toBeTrue matcher, I had a brain fart and ended up with this:

if (actual === true) {
    return true
}

This was the wrong approach to start with, and I'm too explicit in my check. I don't wanna be using the === operator there, this should be sufficient:

if (actual) {
    return true
}

IN CFML "truthy" should be enough for this matcher, as that's idiotmatic-CFML.

But whilst running my tests, I noticed something weird in Lucee. I had a test along these lines:

it("a passing test", () => {
    x = "something"
    expect(x.equals(x)).toBeTrue()
})

On ColdFusion, this passed fine.

But on Lucee, it failed. It seemed that apparently the same string doesn't equal itself. Although the more I dug, the weirder it got. I distilled a repro case down to:

s = "there can be only one"    
writeOutput(s.equals(s) === true) // Lucee: false; ColdFusion: true

This completely flummoxed me, and I posted something on the CFML Slack channel about it. Jonas Eriksson pointed me in the direction of an article on the topic of === that Ben had written a coupla years ago("Exploring The Triple Equals (===) Operator In Lucee CFML 5.3.4.77"). The article itself didn't tell me anything I didn't already "know" (ahem), but Brad's comment reminded me that Lucee's implementation of === is a bit shit. There's a Jira ticket for it here: "LDEV-1282: Fix === operator to check for type equality", and another one "LDEV-3132: ACF2021 - Identity/Not Identity Operator" that I have even participated in. Just a week ago!!.

The issue is that Lucee has decided to make the === operator to be an identity operator. IE: it does not do an equality and type check like ColdFusion does (full disclosure: Lucee implemented this operator before ColdFusion did), it just checks that the two operands are exactly the same object (ie: in memory).

How is this causing my problem? Well it's because not all trues are created equal, in Lucee. This gist demonstrates it:

system=createObject("java", "java.lang.System")    
    
s = "there can be only one"

e = s.equals(s)

t = true

writeDump([
    "e" = e, // true
    "t" = t, // true
    "==" = e == t, // true
    "===" = e === t, // LUCEE: FALSE; ColdFusion: true
    "e hashcode" = system.identityHashCode(e), // one thing, eg 712659136
    "t hashcode" = system.identityHashCode(t) // a different thing, eg: 558076530
])

Notice how the identity hashes for e and t are different. Therefore e === t is false on Lucee. This, I'm afraid, makes the === operator on Lucee about as useful as a chocolate teapot. In CFML it's really not important that a value occupies the same memory; but sometimes it's handy - because CFML is a loosely- & dynamically-typed language - to be able to check a value is the same and that it's actually the same type too (eg:"1" is equal to 1, but is not the same type). And this lack of joined-up-thinking in Lucee is demonstrated pretty clearly in that two instances of true are not considered the same as each other. That's just daft.

Out of interest I decided to push the boat out on my largely non-existent Java skills, check what Java does here. I came up with this:

class TestIdentiyEquality {

    public static void main(String[] args) {
        Boolean b1 = new Boolean(true); // deprecated approach to getting a Boolean
        Boolean b2 = new Boolean(true);

        System.out.println(String.format("via constructor, using equals: %b", b1.equals(b2))); // true 
        System.out.println(String.format("via constructor, using ==: %b", b1 == b2)); // false

        Boolean b3 = Boolean.parseBoolean("true");
        Boolean b4 = Boolean.valueOf("true");

        System.out.println(String.format("via parseBoolean/valueOf, using equals: %b", b3.equals(b4))); // true 
        System.out.println(String.format("via parseBoolean/valueOf, using ==: %b", b3 == b4)); // true
    }

}

It's interesting that in Java, creating a new Boolean object via its constructor is deprecated these days, in favour of using static methods that return the same Boolean each time. I guess this is Lucee's problem too: its true isn't the same object as the one the JVM is using (I presume internally Java uses the same object all over, but can't be arsed thoroughly checking this). Either way, Lucee have claimed that they have followed Java's route for the identity operator, which I think is odd given Java doesn't distinguish between value equality and identity equality via operators; it uses the equals method for checking value, and it only has the == operator other than that, and that does do an identity check. Also Java has no need to differentiate between "value equality" and "value and type equality" because it's a strongly typed language. It's not routine in Java to need to do myStringContaining1 == myNumericContaining1 like one would in a loosely typed language like CFML.

I did a check of some other languages that operate in a similar space to CFML (this is being a bit kind to CFML, but hey). My findings were:

  • Lucee, Python, Groovy, Kotlin treat === as reference equality check
  • CF, PHP, JS, Ruby treat it as a value & type check (in Ruby even == checks type too)

The code for the comparisons is in this gist.

I think CFML more closely fits with PHP, JS and - to a lesser extent - Ruby here, and I think ColdFusion has got the implementation of === right. Lucee should sort theirs out to work the same.


Oh, and where did I get to with my improvements to my tiny testing framework? This Twitter message sums-up my afternoon:

I spent so much time looking at various bugs in Lucee and spilling over into ColdFusion as I checked my code for the Lucee problem and found other bugs on ColdFusion too that I lost the taste for doing CFML, and just drank beer instead. It's a better use of my time, I think. The same could be applied to writing daft blog articles, so I'm off to find a beer instead.

Righto.

--
Adam


Update

OKOK, I did check what Java does about comparing two different true expressions:

class TestEquals {

    public static void main(String[] args) {
        String s1 = "there can be only one";
        String s2 = "maybe there can be another one, actually";
        Boolean b1 = s1.equals(s1);
        Boolean b2 = s2.equals(s2);

        System.out.println(String.format("b1: %b; b2: %b; b1 == b2: %b", b1, b2, b1 == b2)); // b1: true; b2: true; b1 == b2: true
    }

}

The equivalent in CFML is:

s1 = "there can be only one"
s2 = "maybe there can be another one, actually"
b1 = s1.equals(s1)
b2 = s2.equals(s2)

writeOutput("b1: #b1#; b2: #b2#; b1 == b2: #b1 === b2#") // b1: true; b2: true; b1 == b2: false (true on ColdFusion)

Even in the context of booleans "we do it how Java does it" doesn't stand-up to too much scrunity with Lucee. I think its implementation is just wrong.