Showing posts with label Unit Testing. Show all posts
Showing posts with label Unit Testing. Show all posts

Saturday 21 December 2013

So what did I do today? (Nothing interesting. Here are the details)


G'day:
I faffed around on the computer - with ColdFusion and Railo and TestBox - all day today, and have concluded I have nothing interesting to show for it. Still... this is a log of what I've been up to with CFML, so you're gonna hear all about it anyhow. And what's "better"... I'm gonna spin it out to be two articles. I am the Peter Jackson of CFML blogs!

OK, so I got wind of something to investigate the other day, and to do the metrics, I needed to time some stuff. Normally I'd just use a getTickCount() before and after the code, but I thought I might need something more comprehensive this time, so I figured a wee stopwatch UDF was in order. I've also just installed TestBox and am in the process of seeing if I can migrate away from MXUnit, so decided to do another TDD exercise with it. Note: there is no great exposition in this article, or really much new about TDD etc. It's simply what I did today.

Note: I abandoned TesxtBox's MXUnit compatibility mode today, because of a show-stopper bug: "ADDASSERTDECORATOR not implemented". This is the function that allows the importing of custom assertions, which I am using here. On the basis of that, I decided to just go with TestBox syntax instead.

Let's have a quick look at the function, and the tests.

//makeStopwatch.cfm
struct function makeStopwatch(){
    var timeline        = [];

    var lap = function(string message=""){
        var ticksNow    = getTickCount();
        var lapCount    = arrayLen(timeline);
        var lap            = {
            currentClock    = ticksNow,
            lapDuration        = lapCount > 0 ? ticksNow - timeLine[lapCount].currentClock : 0,
            totalDuration    = lapCount > 0 ? ticksNow - timeLine[1].currentClock : 0,
            message            = message
        };
        arrayAppend(timeline, lap);
        return lap;
    };

    return {
        start        = function(string message="start"){
            return lap(message);
        },
        lap            = function(string message="lap"){
            return lap(message);
        },
        stop        = function(string message="stop"){
            return lap(message);
        },
        getTimeline    = function(){
            return timeLine;
        }
    };
};

Not much interesting here. I enjoyed finding another reason / excuse to use function expressions and a wee bit of closure around the timeline which "sticks" with the start() / lap() / stop() / getTimeline() functions.

What this function does is to log a struct at start, each lap, and another on stop. Here's it in action:

// useStopwatch.cfm
include "makeStopwatch.cfm";
stopwatch = makeStopwatch();

stopwatch.start("Begin timing");
sleep(500);
stopwatch.lap();
sleep(1500);
secondLap = stopwatch.lap("after another 1500ms");
writeDump(var=secondLap, label="secondLap");
sleep(2000);
stopwatch.start("Stop timing");
writeDump(var=stopWatch.getTimeline());

Output:

secondLap - struct
CURRENTCLOCK1387658080067
LAPDURATION1500
MESSAGEafter another 1500ms
TOTALDURATION2000
array
1
struct
CURRENTCLOCK1387658078067
LAPDURATION0
MESSAGEBegin timing
TOTALDURATION0
2
struct
CURRENTCLOCK1387658078567
LAPDURATION500
MESSAGElap
TOTALDURATION500
3
struct
CURRENTCLOCK1387658080067
LAPDURATION1500
MESSAGEafter another 1500ms
TOTALDURATION2000
4
struct
CURRENTCLOCK1387658082117
LAPDURATION2050
MESSAGEStop timing
TOTALDURATION4050

This is the bumpf I usually need when I'm timing stuff. Well: some or all of it. It'll be quite handy, I reckon.

One interesting facet here is that initially I thought I'd need three separate functions for the start() / lap() / stop() functions, I was doing TDD (seriously, I knew you'd wonder, so everything I write on this blog uses full TDD now), and having knocked out the first few tests to verify the method signatures for the returned functions, it occurred to me that stop() was actually redundant. It doesn't do anything that lap() wouldn't already do. I mean all this "stopwatch" does is take time marks... there's no clock starting or stopping really (getTickCount() keeps on ticking, we simply start paying attention and then stop paying attention at some point).

So, anyway, I decided before I starting messing around with a redesign, I'd get the test coverage done, get it working, and then refactor. This is something one of my colleagues (Brian, I mean you) has been drumming into me recently: don't get halfway through something, decide to do it differently, start again, refactor, waste time, and not have anything to show for it if I get interrupted (or, in a work situation, we get to the end of the sprint and need to release). So I banged out the rest of the tests, got everything working, and looked at my code some more.

Here are the tests:

CFML: Follow-up to last night's article about a TestBox "bug". PEBCAK strikes again!

G'day:
Time for me to fall on my sword.

Yesterday I started looking at TestBox ("Unit Testing / TDD - switching off MXUnit, switching on TestBox"), and whilst messing around, I came across a coupla bugs (some legit, I hasten to add), one of which Luis asked me to clarify on the ColdBox mailing list. Which I did: "[Testbox v1.0.0] "Variable ISEXPECTEDEXCEPTION is undefined" on erroring test".

The gist of this is as follows:

// Tests.cfc
component {

    public void function beforeTests(){
        variables.beforeTestsRun = true;
    }

    public void function setup(){
        variables.setupRun = true;
    }

    public void function testInitFunctions(){
        assert(structKeyExists(variables, "beforeTestsRun"));
        assert(structKeyExists(variables, "setupRun"));
    }

    public void function testThatErrors(){
        throw(type="TestException", message="This is a test exception", detail="Note that it is NOT being caught / tested for. This is by design");
    }

    public void function testThatFails(){
        fail("This test should fail");
    }

}

This is a distillation of my situation, not actual code. I am testing TestBox's MXUnit compatibility "mode", so here's a test that checks that beforeTests() and setup() run, plus how it deals with an erroring test, and a failing test.

When I was running this on ColdFusion, I got this output:

TestBox v1.0.0.00062
 

Global Stats (86 ms)

[ Bundles/Suites/Specs: 1/1/3 ] [ Pass: 1 ] [ Failures: 0 ] [ Errors: 2 ] [ Skipped: 0 ] [ Reset ] 

shared.git.blogExamples.unittests.testbox.Tests (18 ms)

[ Suites/Specs: 1/3 ] [ Pass: 1 ] [ Failures: 0 ] [ Errors: 2 ] [ Skipped: 0 ] [ Reset ]
Note how the erroring and failing tests were reporting as erroring, but with an error internal to TestBox, not with the expected output MXUnit would give. So basically when a test failed or errored, TestBox itself was failing.

At this point... can you see what I've done wrong here?

Here's the thing. For TestBox test CFCs, there is no need to extend anything. I guess TestBox just uses mix-ins to insert all its bits and pieces. So my CFC doesn't extend anything.

However with MXUnit, one does need to extend mxunit.framework.TestCase.

Obviously on all my "normal" MXUnit test CFCs I do have the correct inheritance in place. But this particular test CFC started out with me testing TestBox tests, not MXUnit tests, so I neglected to put the inheritance in when I switched over to running it with MXUnit compatibility "mode". My bad.

Once I sorted that out, my tests run fine:

TestBox v1.0.0.00062
 

Global Stats (82 ms)

[ Bundles/Suites/Specs: 1/1/3 ] [ Pass: 1 ] [ Failures: 1 ] [ Errors: 1 ] [ Skipped: 0 ] [ Reset ] 

shared.git.blogExamples.unittests.testbox.Tests (15 ms)

[ Suites/Specs: 1/3 ] [ Pass: 1 ] [ Failures: 1 ] [ Errors: 1 ] [ Skipped: 0 ] [ Reset ]

I fed this back to Luis, so that's case closed on that one. I also editorialised a bit, saying this:

It's a pity your test case CFCs don't need to be of a certain type, as that would have clarified that error from the outset. It would also help mitigate a failing MXUnit always had in my opinion: that it determines whether a CFC is a test CFC by its NAME rather than its TYPE (eg: it's a subclass of TestCase). This name-based approach isn't great OO, especially if the capability to do it "properly" is right there.

This has always annoyed me about MXUnit. And it'll annoy me about TestBox. It doesn't annoy me much, but that MXUnit always relied on a file-naming scheme to identify test CFCs always seemed a bit amateurish to me. [gallic shrug]... this is a very minor gripe, and I do not forget that MXUnit has still been one of the most useful CFML tools out there, and TestBox is almost certainly gonna be an appropriate successor.

And Luis... very awesome you followed up my mailing list question so promptly. Much appreciated. I need to venture out into an uncharacteristically windy London for coffee and food, but I shall be writing more tests today. Let's see how it goes...

--
Adam

Friday 20 December 2013

Unit Testing / TDD - switching off MXUnit, switching on TestBox

G'day:
This article is more an infrastructure discussion, rather than examining more actual testing stuff. The ever-growing *Box empire has recently borged into yet another part of the CFML community: testing. They're released another box... TestBox. TestBox is interesting to me as it has a different approach to testing than MXUnit has... rather than xUnit style assertion-based testing, instead favouring a BDD approach. I've not done a lick of BDD, but people keep banging on about it, so I shall be looking at it soon. -ish. First I need to switch to TestBox.

One appealing thing I had heard about TestBox is that it's backwards compatible with MXUnit, so this should mean that I can just do the switch and continue with my current approach to testing, and ease my way into BDD as I learn more about it. So the first thing I decided to examine is how well this stands up, and how many changes I need to make to my existing tests to get them to run. Realistically, nothing is every completely backwards compatible... not even say between different versions of the same software (ColdFusion 9 to ColdFusion 10), let along a second system emulating another system (eg: Railo and ColdFusion). This is fine. I don't expect this migration to be seamless.

Here's what I worked through this morning to get up and running (spoilers: kinda running) on TestBox.

I preface this with the fact that I have always found Ortus's documentation to be a bit impenetrable (there's too much of it, it waffles too much), so I was hesitant about how long this would all take.

Locating, downloading and installing

Finding it

I googled "testbox", and the first link was the ColdBox Platform Wiki - TestBox. This is promising. Within a paragraph (and a to-the-point paragraph which just intros the product, so maybe the docs have got some improved focus: cool) there were download links. TestBox requires ColdFusion 10 / Railo 4.1, btw. I presume it uses closure or something? I'm not sure. But that's cool, I use CF10 and Railo [latest] for my work for this blog. It does preclude me from really giving it a test our with our 3000 unit tests at work though (which is a shame), because we're still on CF9 and will be for the foreseeable future.

Installing it

The installation instructions threw me a bit. The default suggestion is to put the testbox dir into the web root, but that's poor advice: only files specifically intended to be web browseable should ever be in your web root. Fortunately the also mention one can stick 'em anywhere, and map them in with a /testbox mapping. I wish this was their default suggestion. In fact I wish it was their only suggestion. They should not encourage poor practice.

There's a caveat with this though (and this is where I had problems), is that TestBox does have some web assets which need to be web browseable, so it does actually need a web mapping, not just a CF mapping. They do caveat this further down the page.

The first pitfall I had was which directory they're actually talking about. The zipfile has this baseline structure:

/testbox_1.0.0/
    testbox-1.0.0.00062-201312171237
    apidocs/
    browser/
    runner/
    runner-template/
    samples/
    testbox/
    license.txt
    mockbox.txt
    testbox.txt

So I homed this lot in my CF root (not web root, CF root) as /frameworks/testbox/1.0.0/, and added a /testbox CF mapping to that location.

WARNING (if you're following along and doing this at the same time): this is not the correct thing to do. Keep reading...

I then had a look around for which directory I needed to add a web server virtual directory for, and found web-servable assets in the following locations:

/apidocs/
/browser/
/runner/
/samples/
/testbox/system/testing/reports/assets/

(I searched for images, JS, CSS, HTML and index.cfm files; not perfect, but will give me an idea).

OK, so I figured he apidocs and samples are separate from the TestBox app, but that still leaves three disconnected (and laterally displaced) directories which need to be web browseable. This ain't great. So basically it looks like I need to make the entire /testbox dir web browseable. That's a bit shit, and a bit how we might have set up our CFML-driven websites... ten years ago. Oh well.

Configuring Tomcat

Here's a challenge (cue: Sean to get grumpy). I have no idea how to set up a virtual directory on Tomcat's built-in web server. Fortunately that's what Google is for, so I googled "tomcat web server virtual directories", and the very first link is a ColdFusion-10-specific document: "Getting Started with Tomcat in ColdFusion 10". I shuddered slightly that this is just in the ColdFusion Blog, rather than in the CF docs where it belongs, but it'll do. Fortunately the info in there is accurate, which is good.

Basically there's a file server.xml located at <ColdFusion_Home>/runtime/conf/server.xml, where <ColdFusion_Home> is the cfusion dir in your ColdFusion install directory. For me the conf dir is at: C:\apps\adobe\ColdFusion\10\cfusion\runtime\conf.

In there there's an XML note like this:

<Context
    path    = "/"
    docBase    = "<cf_home>\wwwroot"
    WorkDir    = "<cf_home>\runtime\conf\Catalina\localhost\tmp"
>
</Context>

It's commented out by default. All the instructions one needs are in the file itself, but basically it's uncomment it, put actual paths in, and add an aliases attribute:

<Context
    path    = "/"
    docBase    = "C:\apps\adobe\ColdFusion\10\cfusion\wwwroot"
    WorkDir    = "C:\apps\adobe\ColdFusion\10\cfusion\runtime\conf\Catalina\localhost\tmp"
    aliases    = "/testbox=C:\webroots\frameworks\testbox\1.0.0"
>
</Context>

I restarted CF and browsed to http://localhost:8500/testbox, and I got the files in my C:\webroots\frameworks\testbox\1.0.0 directory listing, so that worked. Good to know. I'll now forget about server.xml and aliases and stuff as I won't need to do it again for another six months. Shrug.

ColdFusion config

I put a mapping to the same place in my test app's Application.cfc:

// Application.cfc
component {

    this.mappings            = {
        "/cflib"    = getDirectoryFromPath(getCurrentTemplatePath()),
        "/testbox"    = expandPath("/testbox") // CF will use the virtual directory to resolve that. This is slightly cheating, but hey
    };

}

Monday 9 December 2013

I should pay attention to what my readers tell me! expectException()

G'day:
Sometimes I don't pay attention very well, or my attention span wanders...

... what was I saying?

Oh yeah, MXUnit. And me not paying attention to good advice people give me. I've just adjusted a bunch of my tests from previous articles in my unit testing / TDD series, to handle exceptions better.

Friday 6 December 2013

Unit Testing / TDD - refactoring existing code

G'day:
I've just had a new submission in the CFLib queue, and as it's an easy one to test and release, I'm gonna jump on it and get it out there today. However I want to test it first, plus I want to refactor it slightly, so I'm gonna use TDD to do so. And document what I'm doing as I go.

This continues a series on unit testing and TDD that I'm doing, the rest of which are tagged with either "Unit Testing" or "TDD" or both, so you can look them up via those links (there's too many to list now, so I won't bother).

The code for this article will be here: https://github.com/daccfml/scratch/tree/master/cflib/dayOfWeekAsInt (only the baseline files are there at the moment, as I haven't written the code yet ;-).


Sunday 1 December 2013

Unit Testing / TDD - passing data to on() and trigger()

G'day:
Yesterday I plugged through more test/code and got the code to the point that it would bind and trigger event handlers A-OK. However we still need to update the code so that we can pass data at both bind-time and trigger-time to the handler's execution.

Before I continue, here's the index to the articles in this series I've already written:

Friday 29 November 2013

Unit Testing / TDD - continuing the tests for on() and trigger()

G'day:
I was getting into a rhythm with my TDD cycle this afternoon... test... refine... test... refine... here's what I was doing. Well: after the obligatory recap links (/SEO bait):

Do you know what? I am actually enjoying writing this code. And I'm now past the bits I already knew I had to write (and had the code pretty much already written in my head), so I'm doing really really TDD now. On with the show...

Unit Testing / TDD - getting stuck on how / what to test (part 2/2)

G'day:
Today I'm resuming where I left off yesterday, but first the obligatory links to the rest of the series so far:
Yesterday I questioned how thoroughly I should test return values from functions. Today I am completely flummoxed as to how I test something at all. Spoiler warning: I never worked it out.

Wednesday 20 November 2013

Unit Testing / TDD - getting stuck on how / what to test (part 1/2)

G'day:
Today I'm gonna go over some head-scratching (some of which is still bugging me) I did on the weekend. I kinda got stuck with how I should be testing something, and this raised questions as to whether I even should be testing something. It's a reasonable exercise to go through, so I will document it here. First: recap.

This is an ongoing series of articles about unit testing and TDD (and will diversify from there as I progress), with - so far - five previous entries:

Monday 18 November 2013

Unit Testing / TDD - more tests, more development

G'day:
So enough prattling about <cfclient> for the time being (and, no, I haven't finished with that topic, but Adobe have demonstrated themselves up to the task of ridiculing themselves without my help at the moment, so I'm just leaving them to it). Back to something more useful: TDD / unit testing.

So far I've written four articles in this series:

Only the third of which had any code in it: the rest was just general discussion. Today I'm going to continue developing this createEventObject() idea I've had, using TDD.

Sunday 3 November 2013

Unit Testing / TDD - why you should bother

G'day:
OK, I actually showed some code in the last entry in this series, but now we're back to me spouting rhetoric (read: waffling on). This stems from Andrew Scott making a very good observation about a benefit of unit testing (not so much TDD) in a comment in my last article:
Adam, can I add another under estimated use for TDD. Apart from the bit you touched on about refactoring. But there comes a point when one will need to migrate to a newer version of ColdFusion or Railo, and this would help identify migration problems as well.

And Andrew is dead right.

Saturday 2 November 2013

Unit Testing / TDD - MXUnit and test scenario

G'day:
OK, so I've prattled on a coupla times about TDD and unit testing from a theoretical point of view:

I think today I'll actually get around to writing some code.

This assignment I'm setting myself could all end in tears, as I'm really putting my money where my mouth is... not only am I going to take a straight TDD approach to writing these tests, I'm also actually doing the design of the function I'm testing as I go too. I have just an idea of how something might be able to work, but I don't actually know if it will be viable yet: I've gone over it in my head, but I've not written any code yet. Other than the test rig.

Similar to my code deferral function from a coupla weeks ago ("Threads, callbacks, closure and a pub"), which handles calling blocks of code in an asnyc fashion, running callbacks at various milestones in execution, I started to wonder if I could write an event-driven system in a single function too. Only a proof-of-concept sort of thing, and most as an exercise in leveraging closure in a real world way (most closure examples I've seen don't actually leverage closure, so I'm all for examples that do), and just to see if the thought exercise can pan out into real code.

My theory is I should be able to have a function which returns a coupla other functions which handle event triggering (trigger()) and listening (on()), and because they'll implement closure, they'll remember the main function's variable context, so will both be able to communicate with an central event registry.

Saturday 26 October 2013

Unit Testing / TDD - why you shouldn't bother

G'day:
Here's the second in my intended series covering unit testing and TDD and anything else that springs to mind. Earlier in the week I started the series intended to get straight into code, but my intended "intro paragraph" ended up being an entire article, which was then retitled "Unit Testing - initial rhetoric".

So I'm not looking at code today, instead I'm gonna look at why you oughtn't bother doing unit tests.

Huh? Yeah, that's an odd thing to suggest in a series about unit tests, innit? Well let me be abundantly clear now that I'm below the fold, and you're reading the article and not just reading the intro: there are no bloody reasons why you shouldn't unit test. And you're a muppet if you think there are. I should add that I'm not having a go at people who thusfar haven't been doing unit testing etc for whatever reason, but those people generally know they should have been doing it, but haven't for [some reason which they probably concede is invalid]. I'm only having a go at people who try to justify their position as being a valid one.

Wednesday 23 October 2013

Unit Testing / TDD - initial rhetoric

G'day:
I was talking to a mate/colleague the other day @ CFCamp, and the topic of unit tests came around, and that they... well... didn't do any. They were aware this was not good, but had a sorta mental block of how to get going with TDD and the whole idea of test-first code-second. I had to admit I only started doing unit testing in my current role, and I have our lead architect, Simon Baynes to thank for this. For the preceding decade or so of my development career my approach to testing was the standard cowboy approach of if something didn't seem to break it was probably OK. A lot of people take this approach. It's a shit approach. Don't do it.

In my own defence (it's a lame defence, I hasten to add), I can explain away my lack of unit testing practice for the first few years of my career as I was one of these typical CFML developers who wasn't aware of the wider industry around me, so having not thought to investigate stuff like good coding practices and to check how other people did stuff. Also when new to dev I was in a very jack-the-lad cowboy-ish environment in which doing a decent job of anything was shunned in favour of doing the quickest job possible. Not ideal.

However ignorance is not a defence, and certainly for a few years after my start I was well aware of unit testing as a concept, but never really looked into it. I was like "yeah, I know I should look at this stuff, but it'll just end up meaning I have to write a whole lot of boring test code, and who wants to do that?" This is a very immature, lacking-in-self-disciple attitude, which I now regret.

Tuesday 23 July 2013

Two things: I'm thick; and one cannot use a Mockbox mocked method as a callback

G'day:
And to think someone said on Twitter the other day that I'm one of those people who are never wrong (the subtext being "actually you are, but just will never admit it"). Well here's an example of me being wrong, happily admitting it, and admitting to being a bit stupid to boot.

Thursday 11 July 2013

Weird issue with MockBox and interfaces: issue identified

G'day:
A coupla days back I wrote a vague article "Weird issue with Mockbox and interfaces", which detailed some weirdness I was seeing with using Mockbox to mock methods in CFCs which implement an interface.  The short version is that on the first use of a method mocked this way, I was getting this error:

coldfusion.runtime.InterfaceRuntimeExceptions$ArgumentsMistmatchException: Function argument mismatch.[etc]

However if one just re-ran the test, the problem went away. In fact it was only the first usage of the mocked method after ColdFusion was started that caused the problem. Subsequent runs: all good. Restart CF: problem. This is on ColdFusion 9 & 10, but not Railo.

Tuesday 9 July 2013

Weird issue with Mockbox and interfaces

G'day:
This is not gonna be a very well-realised article, as it's posing a question that I've not really been able to flesh out yet.

Thursday 4 July 2013

Repro case for "contains" pseudo-reserved-word interfering with Mockbox

G'day:
This is mostly for Brad Wood, but it might be of passable interest to others, so I'll plonk it here.

Yesterday's article discussed how contains is kind of a reserved word, but kind of not in ColdFusion (it's just not in Railo). I observed to Brad that this actually bites us on the bum with Mockbox, and he asked for more info, so here it is.

Basically we use a caching system which has a method "contains" which checks to see if there's an item with a given key already in the cache, before trying to fetch it. We've actually since revised this approach, but we have some legacy code still using methods called "contains". So we need to unit test them, and indeed their responses play a part in other methods we test. When testing these other methods which use the caching system, we mock-out the cache, and the methods within it, and we use Mockbox to do this. Mockbox is cool, btw. You should be using it if yer not already.

We're still using Mockbox 1.3 (more about why further down), and it is impossible to directly mock a method called contains using Mockbox. We've worked around this, but it took a bloody long time to work out what the hell was going on, and that there was working-around to do.