Showing posts with label Code Examples. Show all posts
Showing posts with label Code Examples. Show all posts

Sunday, 15 May 2022

CFML: fixing a coupla bugs in my recent work on TinyTestFramework

G'day:

Last week I did some more work on my TinyTestFramework:

On Saturday, I found a bug in each of those. Same bug, basically, surfacing in two different ways. Here's an example:

describe("Demonstrating afterEach bug", () => {
    afterEach(() => {
        writeOutput("<br><br>This should be displayed<br><br>")
    })

    describe("Control", () => {
        it("is a passing test, to demonstrate expected behaviour", () => {
            expect(true).toBeTrue()
        })        
    })

    describe("Demonstrating bug", () => {
        it("should run even if the test fails", () => {
            expect(true).toBeFalse()
        })
    })
})

Output:

Demonstrating afterEach bug
Control
It is a passing test, to demonstrate expected behaviour:

This should be displayed

OK
Demonstrating bug
It should run even if the test fails: Failed
Results: [Pass: 1] [Fail: 1] [Error: 0] [Total: 2]

Note how the second This should be displayed is not being displayed. Why's this? It's because, internally, a failing test throws an exception:

toBeTrue = () => tinyTest.matchers.toBe(true, actual),

// ...

toBe = (expected, actual) => {
    if (actual.equals(expected)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
},

And in the implementation of it, an exception is caught before the code handling afterEach has a chance to run:

it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts
                .filter((context) => context.keyExists("beforeEachHandler"))
                .each((context) => context.beforeEachHandler())

            decoratedImplementation = tinyTest.contexts
                .filter((context) => context.keyExists("aroundEachHandler"))
                .reduce((reversed, context) => reversed.prepend(context), [])
                .reduce((decorated, context) => () => context.aroundEachHandler(decorated), implementation)
            decoratedImplementation()

            tinyTest.contexts
                .filter((context) => context.keyExists("afterEachHandler"))
                .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
                .each((context) => context.afterEachHandler())

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

To explain:

  • implementation is the callback from the it in the test suite. The actual test.
  • All that filter / reduce stuff is just handling aroundEach: don't worry about that.
  • After the decoration we run the test.
  • If it fails, it is caught down here.
  • Meaning the handling of the afterEach callback is never run.

This seems fairly easy to sort out:

try {
    decoratedImplementation()
} finally {
    tinyTest.contexts
        .filter((context) => context.keyExists("afterEachHandler"))
        .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
        .each((context) => context.afterEachHandler())
}

Now even if the test fails, the afterEachHandler handler will still be run. And as I'm not catching the exception, it'll still do what it was supposed to. Rerunning the tests demonstrates this bug is fixed:

Demonstrating afterEach bug
Control
It is a passing test, to demonstrate excpected behaviour:

This should be displayed

OK
Demonstrating bug
It should run even if the test fails:

This should be displayed

Failed
Results: [Pass: 1] [Fail: 1] [Error: 0] [Total: 2]

I also ran all the rest of the tests as well, and they all still pass, so I'm pretty confident my fix has had no repercussions.


I've got the same problem with aroundEach: the bit of the handler after the call to run the test was not being run, for the same reason we had with afterEach: a failing or erroring test throws an exception, and the exception is caught before the rest of the aroundEach handler can be run. This seems slightly trickier to handle, as the code to call the test is within the callback the tester provides:

aroundEach((test) => {
    // top bit before calling the test. No problem with this

    test()

    // bottom bit. This is not getting run after a failed / erroring test
})

I can't expect the testing dev to stick handling of the test failure in there. I need to do this within the framework.

How to do this flummoxed me a bit, but I wrote some tests in the mean time to give my brain some time to think about things:

describe("Demonstrating aroundEach bug", () => {
    aroundEach((test) => {
        writeOutput("<br><br>Before the call to the test: this should be displayed<br>")
        test()
        writeOutput("<br>After the call to the test:This should be displayed<br><br>")
    })

    describe("Control", () => {
        it("is a passing test, to demonstrate expected behaviour", () => {
            expect(true).toBeTrue()
        })        
    })

    describe("Demonstrating bug", () => {
        it("should display the 'bottom' message even if the test fails", () => {
            expect(true).toBeFalse()
        })
    })
})

Results:

Demonstrating aroundEach bug
Control
It is a passing test, to demonstrate expected behaviour:

Before the call to the test: this should be displayed

After the call to the test: This should be displayed

OK
Demonstrating bug
It should display the 'bottom' message even if the test fails:

Before the call to the test: this should be displayed
Failed
Results: [Pass: 1] [Fail: 1] [Error: 0] [Total: 2]

See how the second test isn't outputting After the call to the test: This should be displayed.

it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts
                .filter((context) => context.keyExists("beforeEachHandler"))
                .each((context) => context.beforeEachHandler())

            decoratedImplementation = tinyTest.contexts
                .filter((context) => context.keyExists("aroundEachHandler"))
                .reduce((reversed, context) => reversed.prepend(context), [])
                .reduce((decorated, context) => () => context.aroundEachHandler(decorated), implementation)

        try {    
            decoratedImplementation()
        } finally {
            tinyTest.contexts
                .filter((context) => context.keyExists("afterEachHandler"))
                .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
                .each((context) => context.afterEachHandler())
        }

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

The culmination of that deocration code is how any aroundEach handler is called around the test implementation. Somehow I need to do the equivalent of that try / finally here. But it's not so straight forward as I basically need to prevent that call to implementation from erroring until after we bubble out of all the aroundEach handlers. Bear in mind there can be any number of aroundEach handlers to run: one for each level of describe in the tests:

describe("Test of a CFC", () => {

    aroundEach((test) => {
        // something before
        test()
        // something afterwards
    })

    describe("Test of a method", () => {

        aroundEach((test) => {
            // something before
            test()
            // something afterwards
        })

        describe("Test of a specific part of the method's behaviour", () => {

            aroundEach((test) => {
                // something before
                test()
                // something afterwards
            })
            
            it("will have all three of those `aroundEach` handlers run around it",  () => {
                // test stuff
            })
        })
    })
})

OK so I need to put a try / catch around the call to implementation so i can stop it erroring-out too soon. That's easy enough:

decoratedImplementation = tinyTest.contexts
    .filter((context) => context.keyExists("aroundEachHandler"))
    .reduce((reversed, context) => reversed.prepend(context), [])
    .reduce((decorated, context) => () => context.aroundEachHandler(decorated), () => {
        try {
            implementation()
        } catch (any e) {
            // ???
        }                            
    })
    
try {    
    decoratedImplementation()
    // ???
} finally {
    tinyTest.contexts
        .filter((context) => context.keyExists("afterEachHandler"))
        .reduce((reversedContexts, context) => reversedContexts.prepend(context), [])
        .each((context) => context.afterEachHandler())
}

But I still need to know about that exception after we finish calling the aroundEach handlers and the test implementation.

I'm not sure I like this implementation, but this is what I have done:

decoratedImplementation = tinyTest.contexts
    .filter((context) => context.keyExists("aroundEachHandler"))
    .reduce((reversed, context) => reversed.prepend(context), [])
    .reduce((decorated, context) => () => context.aroundEachHandler(decorated), () => {
        try {
            implementation()
            tinyTest.testResult = true
        } catch (any e) {
            tinyTest.testResult = e
        }                            
    })
    
try {    
    decoratedImplementation()
    if (!tinyTest.testResult.equals(true)) {
        throw(object=tinyTest.testResult)
    }
} finally {

I set a variable in the calling code either flagging the test worked, or if not: how it failed (or errored). The if it failed, I throw the exception I originally caught.

This works, and both those tests I wrote above, and all the rest of the test suite still passes too. Bug fixed.

I'm still thinking about this though. I feel I have nailed the red/green part of this process, but I still possibly have some refactoring to do. But obvs now I'm safe to do so because everything is tested. Well: except any other bugs I haven't noticed yet :-)

Righto.

--
Adam

Thursday, 12 May 2022

CFML: Adding beforeEach handlers to my TinyTestFramework. Another exercise in TDD

G'day:

I have to admit I'm not sure where I'm going with this one yet. I dunno how to implement what I'm needing to do, but I'm gonna start with a test and see where I go from there.

Context: I've been messing around with this TinyTestFramework thing for a bit… it's intended to be a test framework one can run in trycf.com, so I need to squeeze it all into one include file, and at the same time make it not seem too rubbish in the coding dept. The current state of affairs is here: tinyTestFramework.cfm, and its tests: testTinyTestFramework.cfm. Runnable here: on trycf.com

The next thing that has piqued my interest for this is to add beforeEach and afterEach handlers in there too. This will be more of a challenge than the recent "add another matcher" carry on I've done.

First test:

describe("Tests of beforeEach", () => {
    result = ""
    beforeEach(() => {
        result = "set in beforeEach handler"
    })
    
    it("was called before the first test in the set", () => {
        expect(result).toBe("set in beforeEach handler")
    })
})

Right and the first implementation doesn't need to be clever. Just make it pass:

tinyTest = {
    // ...
    beforeEach = (callback) => {
        callback()
    }
}

// ...
beforeEach = tinyTest.beforeEach

This passes. Cool.

That's fine but it's a bit daft. My next test needs to check that beforeEach is called before subsequent tests too. To test this, simply setting a string and checking it's set won't be any use: it'll still be set in the second test too. Well: either set or reset… no way to tell. So I'll make things more intelligent (just a bit):

describe("Tests of beforeEach", () => {
    result = []
    beforeEach(() => {
        result.append("beforeEach")
    })
    
    it("was called before the first test in the set", () => {
        result.append("first test")
        
        expect(result).toBe([
            "beforeEach",
            "first test"
        ])
    })
    
    it("was called before the second test in the set", () => {
        result.append("second test")
        
        expect(result).toBe([
            "beforeEach",
            "first test",
            "beforeEach",
            "second test"
        ])
    })
})

Now each time beforeEach is called it will cumulatively affect the result, so we can test that it's being called for each test. Which of course it is not, currently, so the second test fails.

Note: it's important to consider that in the real world having beforeEach cumulatively change data, and having the sequence the tests are being run be significant - eg: we need the first test to be run before the second test for either test to pass - is really bad form. beforeEach should be idempotent. But given it's what we're actually testing here, this is a reasonable way of testing its behaviour, I think.

Right so currently we are running the beforeEach callback straight away:

beforeEach = (callback) => {
    callback()
}

It needs to be cleverer than that, and only be called when the test is run, which occurs inside it:

it = (string label, function implementation) => {
    tinyTest.inDiv(() => 
        try {
            writeOutput("It #label#: ")
            implementation()
            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

The beforeEach call just has to stick the callback somewhere for later. Hrm. OK:

beforeEachHandler = false,
beforeEach = (callback) => {
    tinyTest.beforeEachHandler = callback
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.beforeEachHandler()
            
            implementation()

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

That works. Although it's dangerously fragile, as that's gonna collapse in a heap if I don't have a beforeEach handler set. I've put this test before those other ones:

describe("Tests without beforeEach", () => {
    it("was called before the first test in the set", () => {
        expect(true).toBe(true)
    })
})

And I get:

Tests of TinyTestFramework
Tests without beforeEach
It was called before the first test in the set: Error: [The function [beforeEachHandler] does not exist in the Struct, only the following functions are available: [append, clear, copy, count, delete, duplicate, each, every, filter, find, findKey, findValue, insert, isEmpty, keyArray, keyExists, keyList, keyTranslate, len, map, reduce, some, sort, toJson, update, valueArray].][]
Tests of beforeEach
It was called before the first test in the set: OK
It was called before the second test in the set: OK
Results: [Pass: 2] [Fail: 0] [Error: 1] [Total: 3]

I need a guard statement around the call to the beforeEach handler:

if (isCustomFunction(tinyTest.beforeEachHandler)) {
    tinyTest.beforeEachHandler()
}

That fixed it.

Next I need to check that the beforeEach handler cascades into nested describe blocks. I've a strong feeling this will "just work":

describe("Tests of beforeEach", () => {
    describe("Testing first level implementation", () => {
        // (tests that were already in place now in here)
    })
    describe("Testing cascade from ancestor", () => {
        result = []
        beforeEach(() => {
            result.append("beforeEach in ancestor")
        })
        describe("Child of parent", () => {
            it("was called even though it is in an ancestor describe block", () => {
                result.append("test in descendant")
                
                expect(result).toBe([
                    "beforeEach in ancestor",
                    "test in descendant"
                ])
            })
        })
    })
})

Note that I have shunted the first lot of tests into their own block now. Also: yeah, this already passes, but I think it's a case of coincidence rather than good design. I'll add another test to demonstrate this:

describe("Tests without beforeEach (bottom)", () => {
    result = []
    it("was called after all other tests", () => {
        result.append("test after any beforeEach implementation")
        
        expect(result).toBe([
            "test after any beforeEach implementation"
        ])
    })
})

This code is right at the bottom of the test suite. If I put a writeDump(result) in there, we'll see why:

implentation (sic) error

After I pressed send on this, I noticed the typo in the test and in the dump above. I fixed the test, but can't be arsed fixing the screen cap. Oops.

You might not have noticed, but I had not VARed that result variable: it's being used by all the tests. This was by design so I could test for leakage, and here we have some: tinyTest.beforeEachHandler has been set in the previous describe block, and it's still set in the following one. We can't be having that: we need to contextualise the handlers to only be in-context within their original describe blocks, and its descendants.

I think all I need to do is to get rid of the handler at the end of the describe implementation:

describe = (string label, function testGroup) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("#label#<br>")
            testGroup()
            tinyTest.beforeEachHandler = false
        } catch (any e) {
            writeOutput("Error: #e.message#<br>")
        }
    })
},

The really seemed easier than I expected it to be. I have a feeling this next step is gonna be trickier though: I need to be able to support multiple sequential handlers, like this:

describe("Multiple sequential handlers", () => {
    beforeEach(() => {
    	result = []
        result.append("beforeEach in outer")
    })
    describe("first descendant of ancestor", () => {
        beforeEach(() => {
            result.append("beforeEach in middle")
        })
        describe("inner descendant of ancestor", () => {
            beforeEach(() => {
                result.append("beforeEach in inner")
            })
            it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
                result.append("test in innermost descendant")
                
                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "test in innermost descendant"
                ])
            })
        })
    })
})

Here we have three nested beforeEach handlers. This fails because we're only storing one, which we can see if we do a dump in the test:

I guess we need to chuck these things into an array instead:

describe = (string label, function testGroup) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("#label#<br>")
            testGroup()
           
        } catch (any e) {
            writeOutput("Error: #e.message#<br>")
        } finally {
            tinyTest.beforeEachHandlers = []
        }
    })
},
beforeEachHandlers = [],
beforeEach = (callback) => {
    tinyTest.beforeEachHandlers.append(callback)
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.beforeEachHandlers.each((handler) => {
                handler()
            })

            implementation()

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},

This makes the tests pass, but I know this bit is wrong:

tinyTest.beforeEachHandlers = []

If I have a second test anywhere in that hierarchy, the handlers will have been blown away, and won't run:

describe("Multiple sequential handlers", () => {
    beforeEach(() => {
        result = []
        result.append("beforeEach in outer")
    })
    describe("first descendant of ancestor", () => {
        beforeEach(() => {
            result.append("beforeEach in middle")
        })

        describe("inner descendant of ancestor", () => {
            beforeEach(() => {
                result.append("beforeEach in inner")
            })
            it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
                result.append("test in innermost descendant")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "test in innermost descendant"
                ])
            })
        })

        it("is a test in the middle of the hierarchy, after the inner describe", () => {
            result.append("test after the inner describe")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach in middle",
                "after the inner describe"
            ])
        
        })
        
    })
})

This fails, and a dump shows why:

So I've got no handlers at all (which is correct given my current implementation), but it should still have the "beforeEach in outer" and "beforeEach in middle" handlers for this test. I've deleted too much. Initially I was puzzled why I still had all that stuff in the result still, but then it occurs to me that was the stuff created for the previous test, just with my last "after the inner describe" appended. So that's predictable/"correct" for there being no beforeEach handlers running at all.

I had to think about this a bit. Initially I thought I'd need to concoct some sort of hierarchical data structure to contain the "array" of handlers, but after some thought I think an array is right, it's just that I only need to pop off the last handler, and only if it's the one set in that describe block. Not sure how I'm gonna work that out, but give me a bit…

OK, I think I've got it:

contexts = [],
describe = (string label, function testGroup) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("#label#<br>")
            tinyTest.contexts.push({})
            testGroup()
           
        } catch (any e) {
            writeOutput("Error: #e.message#<br>")
        } finally {
            tinyTest.contexts.pop()
        }
    })
},
beforeEach = (callback) => {
    tinyTest.contexts.last().beforeEachHandler = callback
},
it = (string label, function implementation) => {
    tinyTest.inDiv(() => {
        try {
            writeOutput("It #label#: ")

            tinyTest.contexts.each((context) => {
                context.keyExists("beforeEachHandler") ? context.beforeEachHandler() : false
            })

            implementation()

            tinyTest.handlePass()
        } catch (TinyTest e) {
            tinyTest.handleFail()
        } catch (any e) {
            tinyTest.handleError(e)
        }
    })
},
  • I maintain an array of contexts.
  • At the beginning of each describe handler I create a context for it - which is just a struct, and push it onto the contexts array.
  • A beforeEach call sticks its handler into the last context struct, which will be the one for the describe that the beforeEach call was made in.
  • When it runs, it iterates over contexts.
  • And if there's a beforeEach handler in a context, then its run.
  • The last thing describe does is to remove its context from the context array.

This means that as each describe block in a hierarchy is run, it "knows" about all the beforeEach handlers created in its ancestors, and during its own run, it adds its own context to that stack. All tests immediately within it, and within any descendant describe blocks will have all the beforeEach handlers down it and including itself. Once it's done, it tidies up after itself, so any subsequently adjacent describe blocks start with on the the their ancestor contexts.

Hopefully one of the code itself, the bulleted list or the narrative paragraph explained what I mean.

As well as the tests I had before this implementation, I added tests for another few scenarios too. Basically any combination / ordering / nesting of describe / it calls I could think of, testing the correct hierarchical sequence of beforeEach handlers was called in the correct order, for the correct test, without interfering with any other test.

describe("Multiple sequential handlers", () => {
    beforeEach(() => {
        result = []
        result.append("beforeEach in outer")
    })
    
    it("is at the top of the hierarchy before any describe", () => {
        result.append("at the top of the hierarchy before any describe")
        
        expect(result).toBe([
            "beforeEach in outer",
            "at the top of the hierarchy before any describe"
        ])
    })
    
    describe("first descendant of ancestor", () => {
        beforeEach(() => {
            result.append("beforeEach in middle")
        })

        it("is a test in the middle of the hierarchy, before the inner describe", () => {
            result.append("test before the inner describe")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach in middle",
                "test before the inner describe"
            ])
        })

        describe("inner descendant of ancestor", () => {
            it("is a test in the bottom of the hierarchy, before the inner beforeEach", () => {
                result.append("in the bottom of the hierarchy, before the inner beforeEach")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "in the bottom of the hierarchy, before the inner beforeEach"
                ])
            })
            beforeEach(() => {
                result.append("beforeEach in inner")
            })
            it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
                result.append("test in innermost descendant")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "test in innermost descendant"
                ])
            })
            it("is another innermost test", () => {
                result.append("is another innermost test")

                expect(result).toBe([
                    "beforeEach in outer",
                    "beforeEach in middle",
                    "beforeEach in inner",
                    "is another innermost test"
                ])
            })
        })

        it("is a test in the middle of the hierarchy, after the inner describe", () => {
            result.append("test after the inner describe")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach in middle",
                "test after the inner describe"
            ])
        })
    })
    
    describe("A second describe in the middle tier of the hierarchy", () => {
        beforeEach(() => {
            result.append("beforeEach second middle describe")
        })

        it("is a test in the second describe in the middle tier of the hierarchy", () => {
            result.append("in the second describe in the middle tier of the hierarchy")

            expect(result).toBe([
                "beforeEach in outer",
                "beforeEach second middle describe",
                "in the second describe in the middle tier of the hierarchy"
            ])
        })
    })
    
    it("is at the top of the hierarchy after any describe", () => {
        result.append("at the top of the hierarchy after any describe")
        
        expect(result).toBe([
            "beforeEach in outer",
            "at the top of the hierarchy after any describe"
        ])
    })
})

All are green, and all the other tests are still green as well. Yay for the testing safety-net that TDD provides for one. I think I have implemented beforeEach now. Implementing afterEach is next, but this should be easy, and just really the same as I have done here, with similar tests.

However I will do that separate to this, and I am gonna press "send" on this, have a beer first.

Code:

Oh: the code:test ratio is now 179:713, or around 1:4.

Righto.

--
Adam

Saturday, 7 May 2022

Running CFML code on trycf.com via a remote HTTP request

G'day:

This ended up being more of a rabbit hole than I expected it to be. But in the process I've learned a bit more about curl, PHP, Python, JS (client). And actually CFML too I guess.

I can't even remember why I needed to do this, but it was something to do with testing that TinyTestFramework I've been blathering about recently.

Anyhow, I decided I needed to run some code locally on my PC which would send some code off to trycf.com, run it, and send me back the response. I figured it'd be doable if I worked out what Abram was doing when I click the "Run Code" button on the trycf.com UI. As it turns out it's just an HTTP post, and I could could re-run the curl captured from my browser easily enough:

Yeah I don't care

Thanks, but before you take time to mention it: I know the code blows out to the right. It doesn't matter, no-one's expecting you to read it really, and the blow-out is just just cosmetic shite.

curl 'https://lucee5-sbx.trycf.com/lucee5/getremote.cfm' \
  -H 'authority: lucee5-sbx.trycf.com' \
  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'cache-control: max-age=0' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundaryaYm5TBPgCaT5n3TK' \
  -H 'cookie: _ga=GA1.2.771352503.1647902596; _gid=GA1.2.1730098774.1651605323; _gat_gtag_UA_35934323_2=1' \
  -H 'dnt: 1' \
  -H 'origin: https://trycf.com' \
  -H 'referer: https://trycf.com/' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="100", "Google Chrome";v="100"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'sec-fetch-dest: iframe' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-site: same-site' \
  -H 'sec-fetch-user: ?1' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36' \
  --data-raw $'------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="setupcode"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="zoom"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="code"\r\n\r\n<cfscript>\r\nwriteOutput("hi")\r\n\r\n</cfscript>\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="postcode"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="key"\r\n\r\nmain1651930152982-9b67997d-643d-3ddf-0b9f-6154282cfcf4\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK\r\nContent-Disposition: form-data; name="asserts"\r\n\r\n\r\n------WebKitFormBoundaryaYm5TBPgCaT5n3TK--\r\n' \
  --compressed

I can run that in bash and it works fine. BTW, the actual code I'm running is buried in the middle there, There's quite a chunk of overhead to execute that code, and I reckoned the browser was probably being a bit belt-n-braces about all the headers it was sending, and I'd not need most of them. I whittled it down to this:

curl 'https://lucee5-sbx.trycf.com/lucee5/getremote.cfm' \
  -H 'content-type: multipart/form-data; boundary=----__trycf__' \
  --data-raw $'------__trycf__\r\nContent-Disposition: form-data; name="setupcode"\r\n\r\n\r\n------__trycf__\r\nContent-Disposition: form-data; name="code"\r\n\r\n<cfscript>\r\nwriteOutput("hi")\r\n</cfscript>\r\n------__trycf__\r\nContent-Disposition: form-data; name="asserts"\r\n\r\n\r\n------__trycf__--\r\n' \
  --compressed

There's a handy site that converts curls to language-specific implementations, and CFML is one of the options: https://curlconverter.com/#cfml. This was handy in theory, but the HTTP service call it created didn't work. Not its fault: it should have, but it seems CFHTTP can't handle that boundary syntax in the curl. Note that PHP's version also struggled, but the JS (fetch), Java and Python versions all worked fine.

This threw me for a while cos I'm not really that au fait with building HTTP requests by hand, but eventually I cracked it. Don't hand-crank the multipart boundary stuff: let CFML do it for you. So I came up with this proof of concept:

<cfset code = fileRead(expandPath("./code.cfm"))>

<cfhttp method="post" url="https://acf14-sbx.trycf.com/getremote.cfm" result="httpResponse">
    <cfhttpparam type="formField" name="code" value=#code#>
    <cfhttpparam type="formField" name="asserts" value="">
</cfhttp>

<cfif httpResponse.statusCode EQ "200 OK">
    <cfoutput>#httpResponse.fileContent#</cfoutput>
<cfelse>
    <cfdump var="#httpResponse#">
</cfif>

That's pretty simple. And my test code for this is just:

<cfscript>
    name = "Scott Steinbeck"
    writeOutput("G'day #name#")
</cfscript>

Results:

G'day Scott Steinbeck

(It was Scott that asked me to write this up).

That's where I've got to so far. I also want to see how I can include some set-up code. Hang on a sec whilst I watch some more HTTP traffic.

[…]

Oh OK, that was easy:

<cfset framework = fileRead(expandPath("./tinyTestFramework.cfm"))>
<cfset tests = fileRead(expandPath("./tests.cfm"))>

<cfhttp method="post" url="https://acf14-sbx.trycf.com/getremote.cfm" result="httpResponse">
    <cfhttpparam type="formField" name="setupcode" value="#framework#">
    <cfhttpparam type="formField" name="code" value="#tests#">
    <cfhttpparam type="formField" name="asserts" value="">
</cfhttp>

<cfif httpResponse.statusCode EQ "200 OK">
    <cfoutput>#httpResponse.fileContent#</cfoutput>
<cfelse>
    <cfdump var="#httpResponse#">
</cfif>

(code on github)

That example just runs the test for my test framework up on trycf instead of here on my own server. Because I can. At least now I remember why I wanted to do this in the first place, but that will be in a later article.

All these test so far only run on ColdFusion 2021, because that was what I was wanting to do. The other hosts are easy enough to glean just by watching the request when clicking "run code". The Lucee (latest) one is https://lucee5-sbx.trycf.com/lucee5/getremote.cfm

Anyway, not a terribly exciting one this (not like how terribly exciting the shit I write here usually is, eh? EH??), but problem solved and hopefully this will help Scott.

Righto.

--
Adam

Wednesday, 4 May 2022

CFML: updates to my TinyTestFramework

G'day:

Just to pass the time / avoid other things I really ought to be doing instead, over the last few evenings I'm been messing around with my TinyTestFramework. I first created this as an exercise in doing some "real world" TDD for a blog article: "TDD: writing a micro testing framework, using the framework to test itself as I build it". The other intent of this work is so I can run actual tests in my code on trycf.com. This is useful when I'm both asking and answering CFML questions I encounter on the CFML Slack and other places.

The first iteration of the framework was pretty minimal. It was just this:

void function describe(required string label, required function testGroup) {
    try {
        writeOutput("#label#<br>")
        testGroup()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}
void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#: ")
        implementation()
        writeOutput("OK<br>")
    } catch (TestFailedException e) {
        writeOutput("Failed<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TestFailedException")
    }}
}

That's it.

But it let me write tests in a Jasmine/TestBox sort of way, right there in trycf.com:

describe("describe", () => {
    it("it is a test", () => {
        expect(true).toBe(true)
    })
}) 

And this would output:

describe
it is a test: OK

That's cool. That was a good MVP. And I actually use it on trycf.com.

However I quickly felt that only having the one toBe matcher was limiting, and made my tests less clear than they could be. Especially when I wanted to expect null or an exception. So… I messed around some more.

I'm not going to take you through the full TDD exercise of writing all this, but I assure you I TDDed almost all of it (I forgot with a coupla small tweaks, I have to admit. I'm not perfect).

But here's the code (also as a gist), for those that are interested:

<style>
    .tinyTest {background-color: black; color:white; font-family:monospace}
    .tinyTest div {margin-left: 1em}
    .tinyTest .pass {color:green;}
    .tinyTest .fail {color:red;}
    .tinyTest .error {background-color:red; color:black}
</style>
<cfscript>
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TinyTest.TestFailedException")
    }}
}

tinyTest = {
    describe = (string label, function testGroup) => {
        tinyTest.inDiv(() => {
            try {
                writeOutput("#label#<br>")
                testGroup()
            } catch (any e) {
                writeOutput("Error: #e.message#<br>")
            }
        })
    },
    it = (string label, function implementation) => {
        tinyTest.inDiv(() => {
            try {
                writeOutput("It #label#: ")
                implementation()
                tinyTest.showPass("OK<br>")
            } catch (TinyTest e) {
                tinyTest.showFail("Failed<br>")
            } catch (any e) {
                tinyTest.showError("Error: #e.message#<br>")
            }
        })
    },
    expect = (any actual) => {
        var proxy.actual = arguments?.actual
    
        return {
            toBe = (expected) => tinyTest.matchers.toBe(expected, actual),
            toBeTrue = () => tinyTest.matchers.toBe(true, actual),
            toBeFalse = () => tinyTest.matchers.toBe(false, actual),
            toBeNull = () => tinyTest.matchers.toBeNull(proxy?.actual),
            toThrow = () => tinyTest.matchers.toThrow(actual),
            toInclude = (needle) => tinyTest.matchers.toInclude(needle, actual),
            notToBe = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(expected, actual)),
            notToBeTrue = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(true, actual)),
            notToBeFalse = (expected) => tinyTest.matchers.not((expected) => tinyTest.matchers.toBe(false, actual)),
            notToBeNull = () => tinyTest.matchers.not(() => tinyTest.matchers.toBeNull(proxy?.actual)),
            notToThrow = () => tinyTest.matchers.not(() => tinyTest.matchers.toThrow(actual)),
            notToInclude = (needle) => tinyTest.matchers.not((needle) => tinyTest.matchers.toInclude(needle, actual))
            
        }
    },
    fail = () => {
        throw(type="TinyTest.FailureException")
    },
    matchers = {},
    inDiv = (callback)  => {
        writeOutput("<div>")
        callBack()
        writeOutput("</div>")
    },
    showPass = (message) => {
        writeOutput('<span class="pass">#message#</span>')
    },
    showFail = (message) => {
        writeOutput('<span class="fail"><em>#message#</em></span>')
    },
    showError = (message) => {
        writeOutput('<span class="error"><strong>#message#</strong></span>')
    }
    
}
tinyTest.matchers.toBe = (expected, actual) => {
    if (actual.equals(expected)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
}
tinyTest.matchers.toBeNull = (actual) => {
    if (isNull(actual)) {
        return true
    }
    throw(type="TinyTest.TestFailedException")
}
tinyTest.matchers.toThrow = (callback) => {
    try {
        callback()
        throw(type="TinyTest.TestFailedException")
    } catch (TinyTest.TestFailedException e) {
        rethrow
    } catch (any e) {
        return true
    }
}
tinyTest.matchers.toInclude = (needle, haystack) => tinyTest.matchers.toBe(true, haystack.findNoCase(needle) > 0)

tinyTest.matchers.not = (callback) => {
    try {
        callback()
        throw(type="TinyTest.NotTestFailedException")
    } catch (TinyTest.NotTestFailedException e) {
        throw(type="TinyTest.TestFailedException")
    } catch (TinyTest.TestFailedException e) {
        return true
    } catch (any e) {
        rethrow
    }
}


describe = tinyTest.describe
it = tinyTest.it
expect = tinyTest.expect
fail = tinyTest.fail
</cfscript>

120 lines now.

What functionality has all this added? Well here's the thing with BDD-style tests. I have documentation and proof that it does what it says it does:

Tests of TinyTestFramework
Tests of it
It prefixes its message with it: OK
Tests of expect
It exists: OK
It returns a struct with keys for matcher callbacks: OK
Tests of fail
It fails a test: OK
Test of test result visualisations
It specifies that a pass should have positive emphasis: OK
It specifies that a fail should have negative emphasis: OK
It specifies that an error should have more emphasis than a fail: OK
Tests of matchers
Tests of toBe
It passes if the actual and expected values are equal: OK
It fails if the actual and expected values are not equal: OK
It expects java.lang.String to work with toBe: OK
It expects java.lang.Double to work with toBe: OK
It expects java.lang.Double to work with toBe: OK
It expects java.lang.Boolean to work with toBe: OK
It expects lucee.runtime.type.ArrayImpl to work with toBe: OK
It expects lucee.runtime.type.StructImpl to work with toBe: OK
It expects lucee.runtime.type.QueryImpl to work with toBe: OK
Tests of notToBe
It passes if the actual and expected values are not equal: OK
It fails if the actual and expected values are equal: OK
It expects java.lang.String to work with notToBe: OK
It expects java.lang.Double to work with notToBe: OK
It expects java.lang.Double to work with notToBe: OK
It expects java.lang.Boolean to work with notToBe: OK
It expects lucee.runtime.type.ArrayImpl to work with notToBe: OK
It expects lucee.runtime.type.StructImpl to work with notToBe: OK
It expects lucee.runtime.type.QueryImpl to work with notToBe: OK
Tests of toBeTrue
It passes if the value is true: OK
It fails if the value is false: OK
Tests of notToBeTrue
It passes if the value is not true: OK
It fails if the value is true: OK
Tests of toBeFalse
It passes if the value is false: OK
It fails if the value is true: OK
Tests of notToBeFalse
It passes if the value is not false: OK
It fails if the value is false: OK
Tests of toThrow
It expects an exception to be thrown from its callback argument: OK
It fails if an exception is not thrown from its callback argument: OK
Tests of notToThrow
It passes if the callback does not throw an exception: OK
It fails if the callback does throw an exception: OK
Tests of toBeNull
It passes if the value is null: OK
It fails if the value is not null: OK
Tests of notToBeNull
It passes if the value is not null: OK
It fails if the value is null: OK
Tests of toInclude
It passes if the haystack contains the needle: OK
It passes if the haystack and needle exactly match: OK
It ignores case: OK
It fails if the haystack does not contain the needle: OK
Tests of notToInclude
It passes if a haystack does not contains the needle: OK
It fails if the haystack contains the needle: OK

All nicely indented and emphasised and shit.

So what have I added for this new version?

  • Tidied up the output to make it easier to read
  • Added these matchers:
    • toBeTrue
    • toBeFalse
    • toBeNull
    • toThrow
    • toInclude
  • And not versions of each of those: notToBe, notToThrow, etc
  • Tests for everything: adamcameron/testTinyTestFramework.cfm

And it's tested using itself, obviously. Interestingly / predictably, there >300 lines of test code there. The ratio is 1:2.5 code:tests

What am I gonna do next? I want to improve that toInclude matcher to work on more than just strings. I also want to have a toBeInstanceOf matcher. Also at some point I better do something with checking structs and arrays and that sorta jazz. I've not needed to actually do that stuff yet, so have not bothered to implement them. But I intend to.

Oh… and you can use this yerself in trycf.com via this URL: https://trycf.com/gist/c631c1f47c8addb2d9aa4d7dacad114f/lucee5?setupCodeGistId=816ce84fd991c2682df612dbaf1cad11&theme=monokai.

Righto.

--
Adam

Saturday, 30 October 2021

TDD: writing a micro testing framework, using the framework to test itself as I build it

G'day:

Being back in CFML land, I spend a lot of time using trycf.com to write sample code. Sometimes I just want to see how ColdFusion and Lucee behave differently in a given situation. Sometimes I'm "helping" someone on the CFML Slack channel, and want to give them a runnable example of what I think they might want to be doing. trycf.com is cool, but I usually want to write my example code as a series of tests demonstrating variants of its behaviour. I also generally want to TDD even sample code that I write, so I know I'm staying on-point and the code works without any surprises. trycf.com is a bit basic, one can only really have one CFML script and run that. One cannot import other libraries like TestBox so I can write my code the way I want.

Quite often I write a little stub tester to help me, and I find myself doing this over and over again, and I never think to save these stubs across examples. Plus having to include the stubs along with the sample code I'm writing clutters things a bit, and also means my example code doesn't really stick to the Single Responsibility Principle.

I found out from Abram - owner of trycf.com - that one can specify a setupCodeGistId in the trycf.com URL, and it will load a CFML script from that and pre-include that before the scratch-pad code. This is bloody handy, and armed with this knowledge I decided I was gonna knock together a minimal single-file testing framework which I could include transparently in with a scratch-pad file, so that scratch-pad file could focus on the sample code I'm writing, and the testing of the same.

A slightly hare-brained idea I have had when doing this is to TDD the whole exercise… and using the test framework to test itself as it evolves. Obviously to start with the test will need to be separate pieces of code before the framework is usable, but beyond a point it should work enough for me to be able to use it to test the latter stages of its development. Well: we'll see, anyhow.

Another challenge I am setting for myself is that I'm gonna mimic the "syntax" of TestBox, so that the tests I write in a scratch-pad should be able to be lifted-out as-is and dumped into a TestBox test spec CFC. Obviously I'm not going to reimplement all of TestBox, just a subset of stuff to be able to do simple tests. I think I will need to implement the following functions:

  • void run() - well, like in TestBox all my tests will be implemented in a function called run, and then one runs that to execute the tests. This is just convention rather than needing any development.
  • void describe(required string label, required function testGroup) - this will output its label and call its testGroup.
  • void it(required string label, required function implementation) - this will output its label and call its implementation. I'm pretty sure the implementation of describe and it will be identical. I mean like I will use two references to the same function to implement this.
  • struct expect(required any actual) - this will take the actual value being tested, and return a struct containing a matcher to that actual value.
  • boolean toBe(required any expected) - this will take the value that the actual value passed to expect should be. This will be as a key in the struct returned by expect (this will emulate the function-chaining TestBox uses with expect(x).toBe(y).

If I create that minimalist implementation, then I will be able to write this much of a test suite in a trycf.com scratch pad:

function myFunctionToTest(x) {
    // etc
}

function run() {
    describe("Tests for myFunctionToTest" ,() => {
        it("tests some variant", () => {
            expect(myFunctionToTest("variant a")).toBe("something")
        })

        it("tests some other variant", () => {
            expect(myFunctionToTest("variant b")).toBe("something else")
        })
    })
}

run()

And everything in that run function will be compatible with TestBox.

I am going to show every single iteration of the process here, to demonstrate TDD in action. This means this article will be long, and it will be code-heavy. And it will have a lot of repetition. I'll try to keep my verbiage minimal, so if I think the iteration of the code speaks for itself, I possibly won't have anything to add.

Let's get on with it.


It runs the tests via a function "run"

<cfscript>
// tests    
try {
    run()
    writeOutput("OK")
} catch (any e) {
    writeOutput("run function not found")
}
</cfscript>

<cfscript>
// implementation    
    
</cfscript>

I am going to do this entire implementation in a trycf.com scratch-pad file. As per above, I have two code blocks: tests and implementation. For each step I will show you the test (or updates to existing tests), and then I will show you the implementation. We can take it as a given that the test will fail in the way I expect (which will be obvious from the code), unless I state otherwise. As a convention a test will output "OK" if it passed, or "Failure: " and some error explanation if not. Later these will be baked into the framework code, but for now, it's hand-cranked. This shows that you don't need any framework to design your code via TDD: it's a practice, it's not a piece of software.

When I run the code above, I get "Failure: run function not found". This is obviously because the run function doesn't even exist yet. Let's address that.

// implementation    
void function run(){
}

Results: OK

We have completed one iteration of red/green. There is nothing to refactor yet. Onto the next iteration.


It has a function describe

We already have some of our framework operational. We can put tests into that run function, and they'll be run: that's about all that function needs to do. Hey I didn't say this framework was gonna be complicated. In fact the aim is for it to be the exact opposite of complicated.

// tests
// ...

void function run(){
    try {
        describe()
        writeOutput("OK")
    } catch (any e) {
        writeOutput("Failure: describe function not found")
    }
}
// implementation
void function describe(){
}

The tests already helped me here actually. In my initial effort at implementing describe, I misspelt it as "desribe". Took me a few seconds to spot why the test failed. I presume, like me, you have found a function with a spelling mistake in it that has escaped into production. I was saved that here, by having to manually type the name of the function into the test before I did the first implementation.


describe takes a string parameter label which is displayed on a line by itself as a label for the test grouping

// tests
// ...
savecontent variable="testOutput" {
    describe("TEST_DESCRIPTION")
};
if (testOutput == "TEST_DESCRIPTION<br>") {
    writeOutput("OK<br>")
    return
}
writeOutput("Failure: label not output<br>")
// implementation
void function describe(required string label){
    writeOutput("#label#<br>")
}

This implementation passes its own test (good), but it makes the previous test break:

try {
    describe()
    writeOutput("OK")
} catch (any e) {
    writeOutput("Failure: describe function not found")
}

We require describe to take an argument now, so we need to update that test slightly:

try {
    describe("NOT_TESTED")
    writeOutput("OK")
} catch (any e) {
    writeOutput("Failure: describe function not found")
}

I make a point of being very clear when arguments etc I need to use are not part of the test.

It's also worth noting that the test output is a bit of a mess at the moment:

OKNOT_TESTED
OKOK

For the sake of cosmetics, I've gone through and put <br> tags on all the test messages I'm outputting, and I've also slapped a cfsilent around that first describe test, as its output is just clutter. The full implementation is currently:

// tests    
try {
    cfsilent() {
        run()
    }
    writeOutput("OK<br>")
} catch (any e) {
    writeOutput("Failure: run function not found<br>")
}

void function run(){
    try {
        cfsilent(){describe("NOT_TESTED")}
        writeOutput("OK<br>")
    } catch (any e) {
        writeOutput("Failure: describe function not found<br>")
    }

    savecontent variable="testOutput" {
        describe("TEST_DESCRIPTION")
    };
    if (testOutput == "TEST_DESCRIPTION<br>") {
        writeOutput("OK<br>")
    }else{
    	writeOutput("Failure: label not output<br>")
    }
}

run()
</cfscript>

<cfscript>
// implementation
void function describe(required string label){
    writeOutput("#label#<br>")
}

And now the output is tidier:

OK
OK
OK

I actually did a double-take here, wondering why the TEST_DESCRIPTION message was not displaying for that second test: the one where I'm actually testing that that message displays. Of course it's because I've got the savecontent around it, so I'm capturing the output, not letting it output. Duh.


describe takes a callback parameter testGroup which is is executed after the description is displayed

savecontent variable="testOutput" {
    describe("TEST_DESCRIPTION", () => {
        writeOutput("DESCRIBE_GROUP_OUTPUT")
    })
};
if (testOutput == "TEST_DESCRIPTION<br>DESCRIBE_GROUP_OUTPUT") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: testGroup not executed<br>")
}
void function describe(required string label, required function testGroup){
    writeOutput("#label#<br>")
    testGroup()
}

This implementation change broke earlier tests that called describe, because they were not passing a testGroup argument. I've updated those to just slung an empty callback into those calls, eg:

describe("TEST_DESCRIPTION", () => {})

That's all I really need from describe, really. But I'll just do one last test to confirm that it can be nested OK (there's no reason why it couldn't be, but I'm gonna check anyways.


describe calls can be nested

savecontent variable="testOutput" {
    describe("OUTER_TEST_DESCRIPTION", () => {
        describe("INNER_TEST_DESCRIPTION", () => {
            writeOutput("DESCRIBE_GROUP_OUTPUT")
        })
    })
};
if (testOutput == "OUTER_TEST_DESCRIPTION<br>INNER_TEST_DESCRIPTION<br>DESCRIBE_GROUP_OUTPUT") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: describe nested did not work<br>")
}

And this passes without any adjustment to the implementation, as I expected. Note that it's OK to write a test that just passes, if there's not then a need to make an implementation change to make it pass. One only needs a failing test before one makes an implementation change. Remember the tests are to test that the implementation change does what it's supposed to. This test here is just a belt and braces thing, and to be declarative about some functionality the system has.


The it function behaves the same way as the describe function

it will end up having more required functionality than describe needs, but there's no reason for them to not just be aliases of each other for the purposes of this micro-test-framework. As long as the describe alias still passes all its tests, it doesn't matter what extra functionality I put into it to accommodate the requirements of it.

savecontent variable="testOutput" {
    describe("TEST_DESCRIPTION", () => {
        it("TEST_CASE_DESCRIPTION", () => {
            writeOutput("TEST_CASE_RESULT")
        })
    })
};
if (testOutput == "TEST_DESCRIPTION<br>TEST_CASE_DESCRIPTION<br>TEST_CASE_RESULT") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: the it function did not work<br>")
}
it = describe

it will not error-out if an exception occurs in its callback, instead reporting an error result, with the exception's message

Tests are intrinsically the sort of thing that might break, so we can't have the test run stopping just cos an exception occurs.

savecontent variable="testOutput" {
    describe("NOT_TESTED", () => {
        it("tests an exception", () => {
            throw "EXCEPTION_MESSAGE";
        })
    })
};
if (testOutput CONTAINS "tests an exception<br>Error: EXCEPTION_MESSAGE<br>") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: the it function did not correctly report the test error<br>")
}
void function describe(required string label, required function testGroup) {
    try {
        writeOutput("#label#<br>")
        testGroup()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

At this point I decided I didn't like how I was just using describe to implement it, so I refactored:

void function runLabelledCallback(required string label, required function callback) {
    try {
        writeOutput("#label#<br>")
        callback()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

void function describe(required string label, required function testGroup) {
    runLabelledCallback(label, testGroup)
}

void function it(required string label, required function implementation) {
    runLabelledCallback(label, implementation)
}

Because everything is test-covered, I am completely safe to just make that change. Now I have the correct function signature on describe and it, and an appropriately "general" function to do the actual execution. And the tests all still pass, so that's cool. Reminder: refactoring doesn't need to start with a failing test. Intrinsically it's an activity that mustn't impact the behaviour of the code being refactored, otherwise it's not a refactor. Also note that one does not ever alter implementation and refactor at the same time. Follow red / green / refactor as separate steps.


it outputs "OK" if the test ran correctly

// <samp>it</samp> outputs OK if the test ran correctly
savecontent variable="testOutput" {
    describe("NOT_TESTED", () => {
        it("outputs OK if the test ran correctly", () => {
            // some test here... this actually starts to demonstrate an issue with the implementation, but we'll get to that
        })
    })
};
if (testOutput CONTAINS "outputs OK if the test ran correctly<br>OK<br>") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: the it function did not correctly report the test error<br>")
}

The implementation of this demonstrated that describe and it can't share an implementation. I don't want describe calls outputting "OK" when their callback runs OK, and this was what started happening when I did my first pass of the implementation for this:

void function runLabelledCallback(required string label, required function callback) {
    try {
        writeOutput("#label#<br>")
        callback()
        writeOutput("OK<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

Only it is a test, and only it is supposed to say "OK". This is an example of premature refactoring, and probably going overboard with DRY. The describe function wasn't complex, so we were gaining nothing by de-duping it from it, especially when it's implementation will have more complexity still to come.

I backed out my implementation and my failing test for a moment, and did another refactor to separate-out the two functions completely. I know I'm good when all my tests still pass.

void function describe(required string label, required function testGroup) {
    writeOutput("#label#<br>")
    testGroup()
}

void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#<br>")
        implementation()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

I also needed to update the baseline it test to expect the "OK<br>". After that everything is green again, and I can bring my new test back (failure as expected), and implementation (all green again/still):

void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#<br>")
        implementation()
        writeOutput("OK<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

describe will not error-out if an exception occurs in its callback, instead reporting an error result, with the exception's message

Back to describe again. It too needs to not error-out if there's an issue with its callback, so we're going to put in some handling for that again too. This test is largely copy and paste from the equivalent it test:

savecontent variable="testOutput" {
    describe("TEST_DESCRIPTION", () => {
        throw "EXCEPTION_MESSAGE";
    })
};
if (testOutput CONTAINS "TEST_DESCRIPTION<br>Error: EXCEPTION_MESSAGE<br>") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: the it function did not correctly report the test error<br>")
}
void function describe(required string label, required function testGroup) {
    try {
        writeOutput("#label#<br>")
        testGroup()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

it outputs "Failed" if the test failed

This took some thought. How do I know a test "failed"? In TestBox when something like expect(true).toBeFalse() runs, it exits from the test immediately, and does not run any further expectations the test might have. Clearly it's throwing an exception from toBeFalse if the actual value (the one passed to expect) isn't false. So this is what I need to do. I have not written any assertions or expectations yet, so for now I'll just test with an actual exception. It can't be any old exception, because the test could break for any other reason too, so I need to differentiate between a test fail exception (the test has failed), and any other sort of exception (the test errored). I'll use a TestFailedException!

savecontent variable="testOutput" {
    describe("NOT_TESTED", () => {
        it("outputs failed if the test failed", () => {
            throw(type="TestFailedException");
        })
    })
};
if (testOutput.reFind("Failed<br>$")) {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: the it function did not correctly report the test failure<br>")
}
void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#<br>")
        implementation()
        writeOutput("OK<br>")
    } catch (TestFailedException e) {
        writeOutput("Failed<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

This is probably enough now to a) start using the framework for its own tests; b) start working on expect and toBe.


has a function expect

it("has a function expect", () => {
    expect()
})

Check. It. Out. I'm actually able to use the framework now. I run this and I get:

has a function expect
Error: Variable EXPECT is undefined.

And when I do the implementation:

function expect() {
}
has a function expect
OK

I think once I'm tidying up, I might look to move the test result to be on the same line as the label. We'll see. For now though: functionality.


expect returns a struct with a key toBe which is a function

it("expect returns a struct with a key toBe which is a function", () => {
    var result = expect()
    if (isNull(result) || isNull(result.toBe) || !isCustomFunction(local.result.toBe)) {
        throw(type="TestFailedException")
    }
})

I'd like to be able to just use !isCustomFunction(local?.result?.toBe) in that if statement there, but Lucee has a bug that prevents ?. to be used in a function expression (I am not making this up, go look at LDEV-3020). Anyway, the implementation for this for now is easy:

function expect() {
    return {toBe = () => {}}
}

toBe returns true if the actual and expected values are equal

it("toBe returns true if the actual and expected values are equal", () => {
    var actual = "TEST_VALUE"
    var expected = "TEST_VALUE"

    result = expect(actual).toBe(expected)
    if (isNull(result) || !result) {
        throw(type="TestFailedException")
    }
})

The implementation for this bit is deceptively easy:

function expect(required any actual) {
    return {toBe = (expected) => {
        return actual.equals(expected)
    }}
}

toBe throws a TestFailedException if the actual and expected values are not equal

it("toBe throws a TestFailedException if the actual and expected values are not equal", () => {
    var actual = "ACTUAL_VALUE"
    var expected = "EXPECTED_VALUE"

    try {
        expect(actual).toBe(expected)
    } catch (TestFailedException e) {
        return
    }
    throw(type="TestFailedException")
})
function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TestFailedException")
    }}
}

And with that… I have a very basic testing framework. Obviously it could be improved to have more expectations (toBeTrue, toBeFalse, toBe{Type}), and could have some nice messages on the toBe function so a failure is more clear as to what went on, but for a minimum viable project, this is fine. I'm going to do a couple more tests / tweaks though.


toBe works with a variety of data types

var types = ["string", 0, 0.0, true, ["array"], {struct="struct"}, queryNew(""), xmlNew()]
types.each((type) => {
    it("works with #type.getClass().getName()#", (type) => {
        expect(type).toBe(type)
    })
})

The results here differ between Lucee:

works with java.lang.String
OK
works with java.lang.Double
OK
works with java.lang.Double
OK
works with java.lang.Boolean
OK
works with lucee.runtime.type.ArrayImpl
OK
works with lucee.runtime.type.StructImpl
OK
works with lucee.runtime.type.QueryImpl
OK
works with lucee.runtime.text.xml.struct.XMLDocumentStruct
Failed

And ColdFusion:

works with java.lang.String
OK
works with java.lang.Integer
OK
works with coldfusion.runtime.CFDouble
Failed
works with coldfusion.runtime.CFBoolean
OK
works with coldfusion.runtime.Array
OK
works with coldfusion.runtime.Struct
OK
works with coldfusion.sql.QueryTable
OK
works with org.apache.xerces.dom.DocumentImpl
Failed

But the thing is the tests actually work on both platforms. If you compare the objects outside the context of the test framework, the results are the same. Apparently in ColdFusion 0.0 does not equal itself. I will be taking this up with Adobe, I think.

It's good to know that it works OK for structs, arrays and queries though.


The it function puts the test result on the same line as the test label, separated by a colon

As I mentioned above, I'd gonna take the <br> out from between the test message and the result, instead just colon-separating them:

// The <samp>it</samp> function puts the test result on the same line as the test label, separated by a colon
savecontent variable="testOutput" {
    describe("TEST_DESCRIPTION", () => {
        it("TEST_CASE_DESCRIPTION", () => {
            writeOutput("TEST_CASE_RESULT")
        })
    })
};
if (testOutput == "TEST_DESCRIPTION<br>TEST_CASE_DESCRIPTION: TEST_CASE_RESULTOK<br>") {
    writeOutput("OK<br>")
}else{
    writeOutput("Failure: the it function did not work<br>")
}
void function it(required string label, required function implementation) {
    try {
    	writeOutput("#label#<br>")
        writeOutput("#label#: ")
        implementation()
        writeOutput("OK<br>")
    } catch (TestFailedException e) {
        writeOutput("Failed<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

That's just a change to that line before calling implementation()


Putting it to use

I can now save the implementation part of this to a gist, save another gist with my actual tests in it, and load that into trycf.com with the setupCodeGistId param pointing to my test framework Gist: https://trycf.com/gist/c631c1f47c8addb2d9aa4d7dacad114f/lucee5?setupCodeGistId=816ce84fd991c2682df612dbaf1cad11&theme=monokai. And… (sigh of relief, cos I'd not tried this until now) it all works.

Update 2022-05-06

That gist points to a newer version of this work. I made a breaking change to how it works today, but wanted the link here to still work.


Outro

This was a long one, but a lot of it was really simple code snippets, so hopefully it doesn't overflow the brain to read it. If you'd like me to clarify anything or find any bugs or I've messed something up or whatever, let me know. Also note that whilst it'll take you a while to read, and it took me bloody ages to write, if I was just doing the actual TDD exercise it's only about an hour's effort. The red/green/refactor cycle is very short.

Oh! Speaking of implementations. I never showed the final product. It's just this:

void function describe(required string label, required function testGroup) {
    try {
        writeOutput("#label#<br>")
        testGroup()
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

void function it(required string label, required function implementation) {
    try {
        writeOutput("#label#: ")
        implementation()
        writeOutput("OK<br>")
    } catch (TestFailedException e) {
        writeOutput("Failed<br>")
    } catch (any e) {
        writeOutput("Error: #e.message#<br>")
    }
}

struct function expect(required any actual) {
    return {toBe = (expected) => {
        if (actual.equals(expected)) {
            return true
        }
        throw(type="TestFailedException")
    }}
}

That's a working testing framework. I quite like this :-)

All the code for the entire exercise - including all the test code - can be checked-out on trycf.com.

Righto.

--
Adam

Sunday, 11 July 2021

What logic should be in a controller? (and a wee bit of testing commentary)

G'day:

This topic has come up for me twice from different directtions in the last week or so, so I'm gonna dump some thoughts. I've actually discussed this before in "I actively consider what I think ought to go into a controller", and the conclusion I came to doesn't quite fit with how I'm writing code now so I'm gonna revise it a bit.

To start with though, I'll reiterate this question: "should this go in the controller?", and I'll repeat Mingo's answer that is still pithy but spot on:

Isn't the answer always "No"?
~ mjhagen 2017

This is a good starting point. 95% of the time if yer asking yerself that question, Mingo has answered it for you there.

The example of what I'd put in a controller from that article is along these lines:

class ContentController {
    function handleGet(rawArgs){
        args = validationService.validate(rawArgs)
        
        content = contentService.getById(args.id)
        
        response = {
            articles = content.published,
            socialContent = {
                twitter = content.twitter,
                facebook = content.facebook
            }
        }
        return new Response(response)
    }
}

It's not the worst code I've written, but I now think this is wrong. The problem lies with how I had a habit of abusing the Anaemic Domain Model pattern in the past, where I had a bunch fo really skinny service classes, and used them to apply behaviour to behaviourless model objects that were just bags of data. Not great.

Looking at that code now, I see these lines:

args = validationService.validate(rawArgs)
content = contentService.getById(args.id)

And I think "nah, those don't belong there. They belong in the model too". I'm doing too much "asking" when I should be "telling" (see "TellDontAsk" by Martin Fowler).

Basically the model here should know what it is to be "valid", so just give it the raw data and let it crack on with it.

My generic controller method these days would be formed along these lines:

function handleRequest(rawRequestValues) {
    try {
        dataFromModel = someModel.getSomeDataFromThisLot(rawRequestValues)
        
        renderedResponse = viewService.renderView("someView", dataFromModel)
        
        return new HtmlResponse(renderedResponse)
        
    } catch (ClientException e) {
        return new ClientErrorResponse(e)
    }
}

Here we clearly have a separation of controller, model and view. It's the controller's job to marshall getting values to a model, and getting the values from that to a view, deal with any error responses that might arise due to those two, or return what came back from the view tier as the response. That's it.

There's an assumption that the framework will deal with any unhandled exceptions there as a controlled 5xx type response. Also there could well be more catch statements, if different types of exception could bubble out of the model, for instance a ValidationException which returns details of validation failures in its response; or a 404 response being returned if a UserNotFoundException came back from some business-logic validation or whatever. But that's the pattern.

The key here is that the only time I'm using a value created by the model is to pass it to the view. I do not pass it to anything else in the interim, like some other model call. That action is not controller logic. It's business logic that we get an x and then pass it to a y. It should be encapsulated in the model.

On the other hand if there was more than one piece of view data to be derived directly from the incoming request values, then that would to me still possibly be legit to be in the controller, eg this is OK:

dataFromModel = someModel.getSomeDataFromThisLot(rawRequestValues)
moreDataFromDifferentModel = someModelOther.getSomeDifferentDataFromThisLot(rawRequestValues)

This would not be OK:

dataFromModel = someModel.getSomeDataFromThisLot(rawRequestValues)
moreDataFromDifferentModel = someModelOther.getSomeDifferentDataFromThisLot(dataFromModel.someValue)

It's a small distinction. But the thing to focus on more than that small example is just to be thinking "no" when you ask yerself "does this belong in the controller?". You're more likely to be right than wrong.


How do we apply that pattern to the example in the old article? Like this I think:

// UserContentController.cfc
component {

    function init(ViewService viewService, UserContentFactory userContentFactory) {
        variables.viewService = arguments.viewService
        variables.userContentFactory = arguments.userContentFactory
    }

    function getContent(rawArgs) {
        try {
            userContent = userContentFactory.getUserContent().loadContentByFilters(rawArgs)
            
            renderedResponse = viewService.renderView("userContentView", userContent)
            
            return new HtmlResponse(renderedResponse)
            
        } catch (ValidationException, e) {
            return new ClientErrorResponse(400, e)
        } catch (UserNotFoundException e) {
            return new ClientErrorResponse(404, e)
        }
    }
}

(I've changed what the controller is returning so as to still integrate the view tier into the example).

I've done away with the controller handling the validation itself, and left that to the model. If things don't pan out: the model will let the controller know. That's it's job. And it's just the controller's job to do something about it. Note that in this case I don't really need both catches. I could just group the exceptions into one ClientException, probably. But I wanted to demonstrate two potential failures from the logic in loadContentByFilters.


What's with this factory I'm using? It's just one of my idiosyncrasies. I like my models' constructors to take actual valid property values, like this:

// UserContent.cfc
component accessors=true invokeImplicitAccessor=true {

    property publishedContent;
    property twitterContent;
    property facebookContent;

    function init(publishedContent, twitterContent, facebookContent) {
        variables.publishedContent = arguments.publishedContent
        variables.twitterContent = arguments.twitterContent
        variables.facebookContent = arguments.facebookContent
    }

Our UserContent represents the data that are those content items. However we've not been given the content items, we've just been given a means to get them. So we can't just create a new object in our controller and slap the incoming values into them. We need to have another method on the UserContent model that works with what the controller can pass it:

function loadContentByFilters(required struct filters) {
    validFilters = validationService.validate(filters, getValidationRules()) // @throws ValidationException
    
    user = userFactory.getById(validFilters.id) // @throws UserNotFoundException
    
    variables.publishedContent = contentService.getUserContent(validFilters)
    variables.twitterContent = twitterService.getUserContent(validFilters)
    variables.facebookContent = facebookService.getUserContent(validFilters)
}

And this demonstrates that to do that work, UserContent needs a bunch of dependencies.

I'm not going to pass these in the constructor because they aren't 100% needed for the operation of a UserContent object, and I want the constructor focusing on its data. So instead these need to be injected as properties:

// UserContent.cfc
component accessors=true invokeImplicitAccessor=true {

    property publishedContent;
    property twitterContent;
    property facebookContent;

    function init(publishedContent, twitterContent, facebookContent) {
        variables.publishedContent = arguments.publishedContent
        variables.twitterContent = arguments.twitterContent
        variables.facebookContent = arguments.facebookContent
    }
    
    function setValidationService(ValidationService validationService) {
        variables.validationService = arguments.validationService
    }
    
    function setUserFactory(UserFactory userFactory) {
        variables.userFactory = arguments.userFactory
    }
    
    function setContentService(UserContentService contentService) {
        variables.contentService = arguments.contentService
    }
    
    function setTwitterService(TwitterService twitterService) {
        variables.twitterService = arguments.twitterService
    }
    
    function setFacebookService(FacebookService facebookService) {
        variables.facebookService = arguments.facebookService
    }

That's all a bit of a mouthful every time we want a UserContent object that needs to use alternative loading methods to get its data, so we hide all that away in our dependency injection set-up, and use a factory to create the object, set its properties, and then return the object:

// UserContentFactory.cfc
component {

    function init(
        ValidationService validationService,
        UserFactory userFactory,
        UserContentService contentService,
        TwitterService twitterService,
        FacebookService facebookService
    ) {
        variables.validationService = arguments.validationService
        variables.userFactory = arguments.userFactory
        variables.contentService = arguments.contentService
        variables.twitterService = arguments.twitterService
        variables.facebookService = arguments.facebookService
    }

    function getUserContent() {
        userContent = new UserContent()
        userContent.setValidationService(validationService)
        userContent.setUserFactory(userFactory)
        userContent.setContentService(contentService)
        userContent.setTwitterService(twitterService)
        userContent.setFacebookService(facebookService)
        
        return userContent
    }
}

The controller just needs to be able to ask the factory for a UserContent object, and then call the method it needs, passing its raw values:

userContent = userContentFactory.getUserContent().loadContentByFilters(rawArgs)

You'll noticed I kept the validation separate from the UserContent model:

function loadContentByFilters(required struct filters) {
    validFilters = validationService.validate(filters, getValidationRules()) // @throws ValidationException

(And then there's also this private method with the rules):

private function getValidationRules() {
    return {
        id = [
            {required = true},
            {type = "integer"}
        ],
        startDate = [
            {required = true},
            {type = "date"},
            {
                range = {
                    max = now()
                }
            }
        ],
        endDate = [
            {required = true},
            {type = "date"},
            {
                range = {
                    max = now()
                }
            }
        ],
        collection = [
            {callback = (collection) => collection.startDate.compare(collection.endDate) < 0}
        ]
    }
}

Validation is fiddly and needs to be accurate, so I don't believe how to validate some values is the job of the UserContent class. I believe it's just perhaps its job to know "what it is to be valid". Hence that separation of concerns. I could see a case for that private method to be its own class, eg UserContentValidationRules or something. But for here, just a private method is OK. Wherever those rules are homed, and whatever the syntax of defining them is, we then pass those and the data to be validated to a specialist validation service that does the business. In this example the validation service itself throws an exception if the validation fails. In reality it'd more likely return a collection of rules violations, and it'd be up to the model making the call to throw the exception. That's implementation detail not so relevant to the code here.


There's probably more off-piste code in this an on-~, but I think it shows how to keep yer domain / business logic out of your controllers, which should be very very light, and simply marshall the incoming request values to the places that need them to be able to come up with a response. That's all a controller ought to do.


Oh before I go. There's an attitude from some testing quarters that one doesn't test one's controllers. I don't actually agree with that, but even if I did: that whole notion is predicated on controllers being very very simple, like I show above. If you pile all (or any of ~) yer logic into yer controller methods: you do actually need to test them! Even in this case I'd still be testing the flow control around the try/catch stuff. If I didn't have that, I'd probably almost be OK if someone didn't test it. Almost.

Righto.

--
Adam