Wednesday 19 February 2014

TestBox, BDD-style tests, Railo member functions and bugs therein

G'day:
One of the promised features of ColdFusion 11 is to bring "member functions" to ColdFusion's inbuilt data types. Railo's already had a good go at doing this, and has reasonably good coverage. See "Member Functions" for details.

One concern I have is whether ColdFusion 11 will implement these the same way as Railo already has. I mean... they should do, there's not much wriggle room, but who knows. If there's a way to do it wrong, I'm sure Adobe can find it. With that in mind, I want to be able to run through some regression/transgression tests on both systems once ColdFusion 11 goes beta, so in prep for that, today I knocked out some unit tests for all the struct member functions.

What I did is to go into the ColdFusion 10 docs, go to the "Structure functions" section (to remind me what all the functions were), and write unit tests for the whole lot. I used the ColdFusion docs rather than the Railo ones because the CF10 docs are more comprehensive than Railo's (this is an indictment of Railo's docs, not a recommendation for the ColdFusion ones, btw!), plus I wanted to make sure I covered any functions Railo might have overlooked.

To make this more interesting (because, let's face it, it's not an interesting task; either to undertake, write up, or for you to read about!), I decided to have a look at TestBox's BDD-inspired testing syntax. The short version of this side of things is that I rather like this new syntax.  I don't think it's got anything to do with BDD (which is an approach to documentation and test design, not a syntax model), but it's interesting anyhow.

Anyway, stand-by for a raft of code (there's a Gist of this too: Struct.cfc):



// Struct.cfc
component {
    
    pageEncoding "utf-8"; // there's a ß, further down

    function run(){
        describe("append() tests", function(){
            it("appends two empty structs", function(){
                var initialStruct = {};
                initialStruct.append({});
                expect(initialStruct).toBe({});
            });
            it("appends a new key to an empty struct", function(){
                var initialStruct = {};
                var structWithNewKey = {a="b"};
                initialStruct.append(structWithNewKey);
                expect(initialStruct).toBe(structWithNewKey);
            });
            it("appends a new key to a struct with an existing key", function(){
                var initialStruct = {a="b"};
                var structWithNewKey = {c="d"};
                initialStruct.append(structWithNewKey);
                expect(initialStruct).toBe({a="b",c="d"});
            });
            it("by default overwrites a key's value in a struct with a new value from the appended struct", function(){
                var initialStruct = {a="be"};
                var structWithNewKey = {a="bee"};
                initialStruct.append(structWithNewKey);
                expect(initialStruct).toBe({a="bee"});
            });
            it("will not overwrite a key's value in a struct with a new value from the appended struct if the overwriteFlag is set to false", function(){
                var initialStruct = {a="be"};
                var structWithNewKey = {a="bee"};
                initialStruct.append(structWithNewKey, false);
                expect(initialStruct).toBe({a="be"});
            });
        });

        describe("clear() tests", function(){
            it("clears a struct", function(){
                var initialStruct = {a="b"};
                initialStruct.clear();
                expect(initialStruct).toBe({});
            });
            it("returns false if the struct was not cleared", function(){
                var initialStruct = CGI;
                var result = initialStruct.clear();
                expect(result).toBeTrue();
            });
        });

        describe("copy() tests", function(){
            it("copies a struct", function(){
                var initialStruct = {a="b"};
                var newStruct = initialStruct.copy();
                expect(newStruct).toBe(initialStruct);
            });
            it("copies deep structs by reference", function(){
                var initialStruct = {a="b", c={d={e="f"}}};
                var result = initialStruct.copy();

                initialStruct.c.d.e = "g";
                expect(result).toBe(initialStruct);
            });
        });

        describe("count() tests", function(){
            it("counts keys in a struct", function(){
                var testStruct = {a="b", c="d"};
                var count = testStruct.count();
                expect(count).toBe(2);
            });
        });

        describe("delete() tests", function(){
            it("deletes a key from a struct", function(){
                var testStruct = {a="b", c="d"};
                testStruct.delete("c");
                expect(testStruct).toBe({a="b"});
            });
            it("returns true if the indicatenotexisting argument is default", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.delete("c");
                expect(result).toBeTrue();

                result = testStruct.delete("d");
                expect(result).toBeTrue();
            });
            it("returns true if the indicatenotexisting argument is true and the key existed", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.delete("c", true);
                expect(result).toBeTrue();
            });
            it("returns false if the indicatenotexisting argument is true and the key didn't exist", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.delete("e", true);
                expect(result).toBeFalse();
            });
        });

        describe("find() tests", function(){
            it("returns the value of the key found in a struct", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.find("c");
                expect(result).toBe("d");
            });
            it("returns the value of the key found in a struct", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.find("e");
                expect(result).toBeEmpty();
            });
        });

        describe("findKey() tests", function(){
            it("returns an empty array for a key that doesn't exist in the struct", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.findKey("e");
                expect(result).toBeArray();
                expect(result).toHaveLength(0);
            });
            it("returns an array with a match of a key that exists in the struct", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.findKey("c");
                expect(result).toBeArray();
                expect(result).toHaveLength(1);
            });
            it("returns an array with a single match of a key that exists in the struct multiple times", function(){
                var testStruct = {a="b", c="d", e={c="f"}};
                var result = testStruct.findKey("c");
                expect(result).toBeArray();
                expect(result).toHaveLength(1);
            });
            it("returns an array with a multiple matches of a key that exists in the struct multiple times, when the scope is ALL", function(){
                var testStruct = {a="b", c="d", e={c="f"}};
                var result = testStruct.findKey("c", "ALL");
                expect(result).toBeArray();
                expect(result).toHaveLength(2);
            });
        });

        describe("findValue() tests", function(){
            it("returns an empty array for a value that doesn't exist in the struct", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.findValue("e");
                expect(result).toBeArray();
                expect(result).toHaveLength(0);
            });
            it("returns an array with a match of a value that exists in the struct", function(){
                var testStruct = {a="b", c="d"};
                var result = testStruct.findValue("d");
                expect(result).toBeArray();
                expect(result).toHaveLength(1);
            });
            it("returns an array with a single match of a value that exists in the struct multiple times", function(){
                var testStruct = {a="b", c="d", e={f="d"}};
                var result = testStruct.findValue("d");
                expect(result).toBeArray();
                expect(result).toHaveLength(1);
            });
            it("returns an array with a multiple matches of a value that exists in the struct multiple times, when the scope is ALL", function(){
                var testStruct = {a="b", c="d", e={f="d"}};
                var result = testStruct.findValue("d", "all");
                expect(result).toBeArray();
                expect(result).toHaveLength(2);
            });
        });

        describe("get() tests", function(){
            it("creates the substruct path specified when it doesn't exist in an empty struct", function(){
                var testStruct = {};
                var result = testStruct.get("testStruct.a.b.c");
                expect(result).toBeStruct();
                expect(result).toBe({});
                expect(testStruct).toBe({a={b={c={}}}});
            });
            it("creates the substruct path specified when it doesn't exist in an empty struct (includes array references)", function(){
                var testStruct = {};
                var result = testStruct.get("testStruct.a[1].b[2].c[3]");
                expect(result).toBeStruct();
                expect(result).toBe({});

                var expected = {a=[{b=[]}]};
                expected.a[1].b[2]={c=[]};
                expected.a[1].b[2].c[3] = {};
                expect(testStruct).toBe(expected);
            });
            it("completes the substruct path specified when it doesn't exist in a struct", function(){
                var testStruct = {a={}};
                var result = testStruct.get("testStruct.a.b.c");
                expect(result).toBeStruct();
                expect(result).toBe({c={}});
                expect(testStruct).toBe({a={b={c={}}}});
            });
            it("returns a substruct if it is found in the struct", function(){
                var testStruct = {a={b={c={}}}};
                var result = testStruct.get("testStruct.a.b");
                expect(result).toBeStruct();
                expect(result).toBe({c={}});
                expect(testStruct).toBe({a={b={c={}}}});
            });
            it("destroys existing subkeys if they are not themselves structs", function(){
                var testStruct = {a={b={c={d="NOT A STRUCT"}}}};
                var result = testStruct.get("testStruct.a.b.c.d.e");
                expect(result).toBeStruct();
                expect(result).toBe({});
                expect(testStruct).toBe({a={b={c={d={e={}}}}}});
            });
        });

        describe("insert() tests", function(){
            it("new key is inserted into empty struct", function(){
                var testStruct = {};
                testStruct.insert("a", "b");
                expect(testStruct).toBe({a="b"});
            });
            it("throws an exception if the key already exists", function(){
                expect(function(){
                    var testStruct = {a="b"};
                    testStruct.insert("a", "c");
                }).toThrow("expression");
            });
            it("overwrites existing key in struct when allowoverwrite is true", function(){
                var testStruct = {a="b"};
                testStruct.insert("a", "c", true);
                expect(testStruct).toBe({a="c"});
            });
        });

        describe("isEmpty() tests", function(){
            it("is true for an empty struct", function(){
                var testStruct = {};
                var result = testStruct.isEmpty();
                expect(result).toBeTrue();
            });
            it("is false for an empty struct", function(){
                var testStruct = {a="b"};
                var result = testStruct.isEmpty();
                expect(result).toBeFalse();
            });
        });

        describe("keyArray() tests", function(){
            it("returns an array of its keys", function(){
                var testStruct = {a="b", c="d", e="f"};
                var result = testStruct.keyArray();
                result.sort("TEXTNOCASE");
                expect(result).toBe(["a","c","e"]);
            });
        });

        describe("keyList() tests", function(){
            it("returns a list of its keys", function(){
                var testStruct = {a="b", c="d", e="f"};
                var result = testStruct.keyList();
                result = listSort(result, "TEXTNOCASE");
                expect(result).toBe("a,c,e");
            });
        });

        describe("new() tests", function(){
            it("has not been implemented", function(){
                expect(function(){
                    var testStruct = {a="b", c="d", e="f"};
                    var result = testStruct.new();
                }).toThrow("expression");
            });
        });

        describe("sort() tests", function(){
            it("returns an array of keys sorted by string value", function(){
                var testStruct = {a="z", b="y", c="x"};
                var result = testStruct.sort("TEXTNOCASE");
                expect(result).toBe(["c","b","a"]);
            });
            it("returns an array of keys reverse-sorted by string value", function(){
                var testStruct = {a="x", b="y", c="z"};
                var result = testStruct.sort("TEXTNOCASE","DESC");
                expect(result).toBe(["c","b","a"]);
            });
            it("returns an array of keys sorted by subkey value", function(){
                var testStruct = {a={sortKey="z"}, b={sortKey="y"}, c={sortKey="x"}};
                var result = testStruct.sort("TEXTNOCASE","ASC", "sortKey");
                expect(result).toBe(["c","b","a"]);
            });
            if (!structKeyExists(server, "Railo")){
                include "structSortWithEncoding.cfm";
            }
        });

        describe("update() tests", function(){
            it("new key is inserted into empty struct", function(){
                expect(function(){
                    var testStruct = {};
                    testStruct.update("a", "b");
                }).toThrow("expression");
            });
            it("overwrites existing key in struct", function(){
                var testStruct = {a="b"};
                testStruct.update("a", "c");
                expect(testStruct).toBe({a="c"});
            });
        });

        describe("each() tests", function(){
            it("processes each key & value", function(){
                var testStruct = {a="b",c="d"};
                var result = "";
                testStruct.each(function(k,v){
                    result = listAppend(result, "#k#:#v#");
                });
                expect(result).toBe("a:b,c:d");
            });
        });

        describe("filter() tests", function(){
            it("filters each key/value pair as per callback", function(){
                var testStruct = {a="b",c="d",e="f",g="h"};
                var result = testStruct.filter(function(k,v){
                    return k == "a" || v == "h";
                });
                expect(result).toBe({a="b",g="h"});
            });
        });

        describe("keyTranslate() tests", function(){
            it("translates a dotted key name into a deep struct", function(){
                var testStruct = {"a.b.c" = "d"};
                testStruct.keyTranslate();
                expect(testStruct).toBe({a={b={c="d"}}});
            });
            it("does not translate a dotted key name when it is beyond the first level", function(){
                var testStruct = {"a.b.c"="d",e={"f.g"="h"}};
                testStruct.keyTranslate(false);
                expect(testStruct).toBe({a={b={c="d"}},e={"f.g"="h"}});
            });
            it("translates a dotted key name into a deep struct, preserving the original key as well", function(){
                var testStruct = {"a.b.c" = "d"};
                testStruct.keyTranslate(true, true);
                expect(testStruct).toBe({"a.b.c"="d", a={b={c="d"}}});
            });
        });

    }
}

Sorry for all of that. Most of it you can breeze over. I've linked back to the TestBox docs for each new test method I use, and have highlighted a coupla discussion points, which I'll review now.

The basic gist of this style of testing is that we run() a batch of tests, within which there's groupings of tests which we describe(), and each test within that declares what it() does, has some code (ie: the stuff we're actually testing), and when we say what we expect() it toBe() (toBe() is just one of many expectations we can have. toBe() is the equavalent of assertEquals() in xUnit tests, really).


For each describe() block, I perform all the tests for a given member function. IE: the first describe() is the grouping for all .append() tests. In the case of .append(), I have these tests:

  • appending an empty struct to another empty struct;
  • appending a populated struct to an empty struct;
  • overwriting a key in the the base struct with one in the struct being appended;
  • unless the overWrite flag is false.
For each member function, I look at the parameters that are possible to pass, and write a test for each of them. This way of using describe() and it() is quite a nice way of grouping things.

One random thing to note: having looked up that there was a toBeArray() expectation, I guessed there was a toBeStruct() one. Indeed there is one, but it's not in the docs. I'll let Brad/Luis know about that (guys: now you know).

Another thing that I quite liked is how expected exceptions are handled. Instead of passing a "dumb" value to the expect(), one can specify a callback. And this is ultimately run in an error-trapped way which then means we can just chain a .toThrow() the expect() call, and it all just works. Very logical.

Lastly in the highlighted stuff is some factoring-out I had to do to allow this file to compile on Railo. Railo hasn't implemented the localeSensitive argument to the structSort() function (or .sort() method), and the test of that causes a parser error on Railo. So in putting it into a separate file, Railo never looks at it. This locale-sensitive functionality doesn't even work on ColdFusion, btw, so no great loss there.

Oh! One last one... Railo has a structKeyTranslate() / keyTranslate() function / method. It took me a while to google-up what that was all about (the docs are completely unhelpful), but finally found some notes here: "Struct addressing in Railo". Basically it seems its for converting keys which have dots embedded in them to a hierarchy of keys. That is a hell of an edge-case function. [Boggle].

OK, so I cannot run those test on ColdFusion yet, but I can run them on Railo. And there was some surprises in the results:

There are a few failures there! I've gone over my tests and they all look legit to me. But there are some problems:

  • clear() (and structClear()) don't work as advertised. It should not error if the struct can't be cleared, it should simply return false. This is the documented behaviour.
  • count() doesn't seem to have been implemented. No great loss, but for the sake of completeness, it should have been.
  • find() (and structFind()) both error if the key being sought doesn't exist in the struct. This is not documented behaviour, and is a bit daft. However it's what ColdFusion does as well. It should return null, I think.
  • get() seems completely broken. Or its got some not-obvious syntactical vagary.
  • in testing the issue above, I noticed structGet() also doesn't work quite right when it comes to getting a reference which has array notation in it. It's supposed to create the arrays if needed. Railo creates them as structs. The correct behaviour (creating arrays) is specifically documented in the ColdFusion docs. So unless there's a good reason to act differently, this is a cross-compat bug in Railo. TBH, I think it's stupid functionality, but that's beside the point.
Everything else is fine, and ready to go for comparative tests once ColdFusion 11's beta shows up. Now... when will that be?

That was a bit of a random article, sorry. But the BDD-inspired TestBox syntax is quite cool, innit? It'll certainly get you practising yer inline function expressions! And writing all that OO code was really a pleasure. So much better than the anachronistic procedural-oriented approach CFML - under the helm of Adobe - has had to contend with until now. It's excellent that CFML is moving forward here.

I still wanna do tests for arrays etc too. I'll do that tomorrow. I'll gist the code, but I'll spare you an article on it. But if anything interesting presents itself, I'll let you know.

I've also got a bunch of bugs to raise in Railo and in ColdFusion too here. But I can't be arsed just now, so that can wait until tomorrow. I'll cross-ref them back here when I'm done.

Righto.

--
Adam