Time for me to fall on my sword.
Yesterday I started looking at TestBox ("Unit Testing / TDD - switching off MXUnit, switching on TestBox"), and whilst messing around, I came across a coupla bugs (some legit, I hasten to add), one of which Luis asked me to clarify on the ColdBox mailing list. Which I did: "[Testbox v1.0.0] "Variable ISEXPECTEDEXCEPTION is undefined" on erroring test".
The gist of this is as follows:
// Tests.cfc
component {
public void function beforeTests(){
variables.beforeTestsRun = true;
}
public void function setup(){
variables.setupRun = true;
}
public void function testInitFunctions(){
assert(structKeyExists(variables, "beforeTestsRun"));
assert(structKeyExists(variables, "setupRun"));
}
public void function testThatErrors(){
throw(type="TestException", message="This is a test exception", detail="Note that it is NOT being caught / tested for. This is by design");
}
public void function testThatFails(){
fail("This test should fail");
}
}
This is a distillation of my situation, not actual code. I am testing TestBox's MXUnit compatibility "mode", so here's a test that checks that beforeTests() and setup() run, plus how it deals with an erroring test, and a failing test.
When I was running this on ColdFusion, I got this output:
TestBox v1.0.0.00062
Global Stats (86 ms)
[ Bundles/Suites/Specs: 1/1/3 ] [ Pass: 1 ] [ Failures: 0 ] [ Errors: 2 ] [ Skipped: 0 ] [ Reset ]At this point... can you see what I've done wrong here?
Here's the thing. For TestBox test CFCs, there is no need to extend anything. I guess TestBox just uses mix-ins to insert all its bits and pieces. So my CFC doesn't extend anything.
However with MXUnit, one does need to extend mxunit.framework.TestCase.
Obviously on all my "normal" MXUnit test CFCs I do have the correct inheritance in place. But this particular test CFC started out with me testing TestBox tests, not MXUnit tests, so I neglected to put the inheritance in when I switched over to running it with MXUnit compatibility "mode". My bad.
Once I sorted that out, my tests run fine:
TestBox v1.0.0.00062
Global Stats (82 ms)
[ Bundles/Suites/Specs: 1/1/3 ] [ Pass: 1 ] [ Failures: 1 ] [ Errors: 1 ] [ Skipped: 0 ] [ Reset ]I fed this back to Luis, so that's case closed on that one. I also editorialised a bit, saying this:
It's a pity your test case CFCs don't need to be of a certain type, as that would have clarified that error from the outset. It would also help mitigate a failing MXUnit always had in my opinion: that it determines whether a CFC is a test CFC by its NAME rather than its TYPE (eg: it's a subclass of TestCase). This name-based approach isn't great OO, especially if the capability to do it "properly" is right there.
This has always annoyed me about MXUnit. And it'll annoy me about TestBox. It doesn't annoy me much, but that MXUnit always relied on a file-naming scheme to identify test CFCs always seemed a bit amateurish to me. [gallic shrug]... this is a very minor gripe, and I do not forget that MXUnit has still been one of the most useful CFML tools out there, and TestBox is almost certainly gonna be an appropriate successor.
And Luis... very awesome you followed up my mailing list question so promptly. Much appreciated. I need to venture out into an uncharacteristically windy London for coffee and food, but I shall be writing more tests today. Let's see how it goes...
--
Adam
