G'day:
I have to admit I'm not sure where I'm going with this one yet. I dunno how to implement what I'm needing to do, but I'm gonna start with a test and see where I go from there.
Context: I've been messing around with this TinyTestFramework thing for a bit… it's intended to be a test framework one can run in trycf.com, so I need to squeeze it all into one include file, and at the same time make it not seem too rubbish in the coding dept. The current state of affairs is here: tinyTestFramework.cfm, and its tests: testTinyTestFramework.cfm. Runnable here: on trycf.com
The next thing that has piqued my interest for this is to add beforeEach and afterEach handlers in there too. This will be more of a challenge than the recent "add another matcher" carry on I've done.
First test:
describe("Tests of beforeEach", () => {
result = ""
beforeEach(() => {
result = "set in beforeEach handler"
})
it("was called before the first test in the set", () => {
expect(result).toBe("set in beforeEach handler")
})
})
Right and the first implementation doesn't need to be clever. Just make it pass:
tinyTest = {
// ...
beforeEach = (callback) => {
callback()
}
}
// ...
beforeEach = tinyTest.beforeEach
This passes. Cool.
That's fine but it's a bit daft. My next test needs to check that beforeEach is called before subsequent tests too. To test this, simply setting a string and checking it's set won't be any use: it'll still be set in the second test too. Well: either set or reset… no way to tell. So I'll make things more intelligent (just a bit):
describe("Tests of beforeEach", () => {
result = []
beforeEach(() => {
result.append("beforeEach")
})
it("was called before the first test in the set", () => {
result.append("first test")
expect(result).toBe([
"beforeEach",
"first test"
])
})
it("was called before the second test in the set", () => {
result.append("second test")
expect(result).toBe([
"beforeEach",
"first test",
"beforeEach",
"second test"
])
})
})
Now each time beforeEach is called it will cumulatively affect the result, so we can test that it's being called for each test. Which of course it is not, currently, so the second test fails.
Note: it's important to consider that in the real world having beforeEach cumulatively change data, and having the sequence the tests are being run be significant - eg: we need the first test to be run before the second test for either test to pass - is really bad form. beforeEach should be idempotent. But given it's what we're actually testing here, this is a reasonable way of testing its behaviour, I think.
Right so currently we are running the beforeEach callback straight away:
beforeEach = (callback) => {
callback()
}
It needs to be cleverer than that, and only be called when the test is run, which occurs inside it:
it = (string label, function implementation) => {
tinyTest.inDiv(() =>
try {
writeOutput("It #label#: ")
implementation()
tinyTest.handlePass()
} catch (TinyTest e) {
tinyTest.handleFail()
} catch (any e) {
tinyTest.handleError(e)
}
})
},
The beforeEach call just has to stick the callback somewhere for later. Hrm. OK:
beforeEachHandler = false,
beforeEach = (callback) => {
tinyTest.beforeEachHandler = callback
},
it = (string label, function implementation) => {
tinyTest.inDiv(() => {
try {
writeOutput("It #label#: ")
tinyTest.beforeEachHandler()
implementation()
tinyTest.handlePass()
} catch (TinyTest e) {
tinyTest.handleFail()
} catch (any e) {
tinyTest.handleError(e)
}
})
},
That works. Although it's dangerously fragile, as that's gonna collapse in a heap if I don't have a beforeEach handler set. I've put this test before those other ones:
describe("Tests without beforeEach", () => {
it("was called before the first test in the set", () => {
expect(true).toBe(true)
})
})
And I get:
I need a guard statement around the call to the beforeEach handler:
if (isCustomFunction(tinyTest.beforeEachHandler)) {
tinyTest.beforeEachHandler()
}
That fixed it.
Next I need to check that the beforeEach handler cascades into nested describe blocks. I've a strong feeling this will "just work":
describe("Tests of beforeEach", () => {
describe("Testing first level implementation", () => {
// (tests that were already in place now in here)
})
describe("Testing cascade from ancestor", () => {
result = []
beforeEach(() => {
result.append("beforeEach in ancestor")
})
describe("Child of parent", () => {
it("was called even though it is in an ancestor describe block", () => {
result.append("test in descendant")
expect(result).toBe([
"beforeEach in ancestor",
"test in descendant"
])
})
})
})
})
Note that I have shunted the first lot of tests into their own block now. Also: yeah, this already passes, but I think it's a case of coincidence rather than good design. I'll add another test to demonstrate this:
describe("Tests without beforeEach (bottom)", () => {
result = []
it("was called after all other tests", () => {
result.append("test after any beforeEach implementation")
expect(result).toBe([
"test after any beforeEach implementation"
])
})
})
This code is right at the bottom of the test suite. If I put a writeDump(result) in there, we'll see why:
implentation (sic) error
After I pressed send on this, I noticed the typo in the test and in the dump above. I fixed the test, but can't be arsed fixing the screen cap. Oops.
You might not have noticed, but I had not VARed that result variable: it's being used by all the tests. This was by design so I could test for leakage, and here we have some: tinyTest.beforeEachHandler has been set in the previous describe block, and it's still set in the following one. We can't be having that: we need to contextualise the handlers to only be in-context within their original describe blocks, and its descendants.
I think all I need to do is to get rid of the handler at the end of the describe implementation:
describe = (string label, function testGroup) => {
tinyTest.inDiv(() => {
try {
writeOutput("#label#<br>")
testGroup()
tinyTest.beforeEachHandler = false
} catch (any e) {
writeOutput("Error: #e.message#<br>")
}
})
},
The really seemed easier than I expected it to be. I have a feeling this next step is gonna be trickier though: I need to be able to support multiple sequential handlers, like this:
describe("Multiple sequential handlers", () => {
beforeEach(() => {
result = []
result.append("beforeEach in outer")
})
describe("first descendant of ancestor", () => {
beforeEach(() => {
result.append("beforeEach in middle")
})
describe("inner descendant of ancestor", () => {
beforeEach(() => {
result.append("beforeEach in inner")
})
it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
result.append("test in innermost descendant")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"beforeEach in inner",
"test in innermost descendant"
])
})
})
})
})
Here we have three nested beforeEach handlers. This fails because we're only storing one, which we can see if we do a dump in the test:
I guess we need to chuck these things into an array instead:
describe = (string label, function testGroup) => {
tinyTest.inDiv(() => {
try {
writeOutput("#label#<br>")
testGroup()
} catch (any e) {
writeOutput("Error: #e.message#<br>")
} finally {
tinyTest.beforeEachHandlers = []
}
})
},
beforeEachHandlers = [],
beforeEach = (callback) => {
tinyTest.beforeEachHandlers.append(callback)
},
it = (string label, function implementation) => {
tinyTest.inDiv(() => {
try {
writeOutput("It #label#: ")
tinyTest.beforeEachHandlers.each((handler) => {
handler()
})
implementation()
tinyTest.handlePass()
} catch (TinyTest e) {
tinyTest.handleFail()
} catch (any e) {
tinyTest.handleError(e)
}
})
},
This makes the tests pass, but I know this bit is wrong:
tinyTest.beforeEachHandlers = []
If I have a second test anywhere in that hierarchy, the handlers will have been blown away, and won't run:
describe("Multiple sequential handlers", () => {
beforeEach(() => {
result = []
result.append("beforeEach in outer")
})
describe("first descendant of ancestor", () => {
beforeEach(() => {
result.append("beforeEach in middle")
})
describe("inner descendant of ancestor", () => {
beforeEach(() => {
result.append("beforeEach in inner")
})
it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
result.append("test in innermost descendant")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"beforeEach in inner",
"test in innermost descendant"
])
})
})
it("is a test in the middle of the hierarchy, after the inner describe", () => {
result.append("test after the inner describe")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"after the inner describe"
])
})
})
})
This fails, and a dump shows why:
So I've got no handlers at all (which is correct given my current implementation), but it should still have the "beforeEach in outer" and "beforeEach in middle" handlers for this test. I've deleted too much. Initially I was puzzled why I still had all that stuff in the result still, but then it occurs to me that was the stuff created for the previous test, just with my last "after the inner describe" appended. So that's predictable/"correct" for there being no beforeEach handlers running at all.
I had to think about this a bit. Initially I thought I'd need to concoct some sort of hierarchical data structure to contain the "array" of handlers, but after some thought I think an array is right, it's just that I only need to pop off the last handler, and only if it's the one set in that describe block. Not sure how I'm gonna work that out, but give me a bit…
OK, I think I've got it:
contexts = [],
describe = (string label, function testGroup) => {
tinyTest.inDiv(() => {
try {
writeOutput("#label#<br>")
tinyTest.contexts.push({})
testGroup()
} catch (any e) {
writeOutput("Error: #e.message#<br>")
} finally {
tinyTest.contexts.pop()
}
})
},
beforeEach = (callback) => {
tinyTest.contexts.last().beforeEachHandler = callback
},
it = (string label, function implementation) => {
tinyTest.inDiv(() => {
try {
writeOutput("It #label#: ")
tinyTest.contexts.each((context) => {
context.keyExists("beforeEachHandler") ? context.beforeEachHandler() : false
})
implementation()
tinyTest.handlePass()
} catch (TinyTest e) {
tinyTest.handleFail()
} catch (any e) {
tinyTest.handleError(e)
}
})
},
- I maintain an array of contexts.
- At the beginning of each describe handler I create a context for it - which is just a struct, and push it onto the contexts array.
- A beforeEach call sticks its handler into the last context struct, which will be the one for the describe that the beforeEach call was made in.
- When it runs, it iterates over contexts.
- And if there's a beforeEach handler in a context, then its run.
- The last thing describe does is to remove its context from the context array.
This means that as each describe block in a hierarchy is run, it "knows" about all the beforeEach handlers created in its ancestors, and during its own run, it adds its own context to that stack. All tests immediately within it, and within any descendant describe blocks will have all the beforeEach handlers down it and including itself. Once it's done, it tidies up after itself, so any subsequently adjacent describe blocks start with on the the their ancestor contexts.
Hopefully one of the code itself, the bulleted list or the narrative paragraph explained what I mean.
As well as the tests I had before this implementation, I added tests for another few scenarios too. Basically any combination / ordering / nesting of describe / it calls I could think of, testing the correct hierarchical sequence of beforeEach handlers was called in the correct order, for the correct test, without interfering with any other test.
describe("Multiple sequential handlers", () => {
beforeEach(() => {
result = []
result.append("beforeEach in outer")
})
it("is at the top of the hierarchy before any describe", () => {
result.append("at the top of the hierarchy before any describe")
expect(result).toBe([
"beforeEach in outer",
"at the top of the hierarchy before any describe"
])
})
describe("first descendant of ancestor", () => {
beforeEach(() => {
result.append("beforeEach in middle")
})
it("is a test in the middle of the hierarchy, before the inner describe", () => {
result.append("test before the inner describe")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"test before the inner describe"
])
})
describe("inner descendant of ancestor", () => {
it("is a test in the bottom of the hierarchy, before the inner beforeEach", () => {
result.append("in the bottom of the hierarchy, before the inner beforeEach")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"in the bottom of the hierarchy, before the inner beforeEach"
])
})
beforeEach(() => {
result.append("beforeEach in inner")
})
it("calls each beforeEach handler in the hierarchy, from outermost to innermost", () => {
result.append("test in innermost descendant")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"beforeEach in inner",
"test in innermost descendant"
])
})
it("is another innermost test", () => {
result.append("is another innermost test")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"beforeEach in inner",
"is another innermost test"
])
})
})
it("is a test in the middle of the hierarchy, after the inner describe", () => {
result.append("test after the inner describe")
expect(result).toBe([
"beforeEach in outer",
"beforeEach in middle",
"test after the inner describe"
])
})
})
describe("A second describe in the middle tier of the hierarchy", () => {
beforeEach(() => {
result.append("beforeEach second middle describe")
})
it("is a test in the second describe in the middle tier of the hierarchy", () => {
result.append("in the second describe in the middle tier of the hierarchy")
expect(result).toBe([
"beforeEach in outer",
"beforeEach second middle describe",
"in the second describe in the middle tier of the hierarchy"
])
})
})
it("is at the top of the hierarchy after any describe", () => {
result.append("at the top of the hierarchy after any describe")
expect(result).toBe([
"beforeEach in outer",
"at the top of the hierarchy after any describe"
])
})
})
All are green, and all the other tests are still green as well. Yay for the testing safety-net that TDD provides for one. I think I have implemented beforeEach now. Implementing afterEach is next, but this should be easy, and just really the same as I have done here, with similar tests.
However I will do that separate to this, and I am gonna press "send" on this, have a beer first.
Code:
- Gist with combined framework and tests.
- Just the framwork.
- Just the tests.
- Jtrycf.com runner with TinyTestFramework loaded.
Oh: the code:test ratio is now 179:713, or around 1:4.
Righto.
--
Adam
