Thursday 15 November 2012

Custom tags: nesting

G'day:
I meant to write this one ages and ages ago, back when I did the previous article on looping with custom tags. As I said in that one, I think custom tags are a great concept, but have been a bit eclipsed by all the non-UI-centric things that have been added to ColdFusion in recent years, and they've fallen out of fashion a bit. I think they still have a place in CFML's arsenal, and do things in a way that is more elegant that other more recent alternatives, when used appropriately.


If people use custom tags, I think they use them mostly as one-off sort of things: wrapping some text or mark-up and doing some processing on the captured result. Much like how one might do with <cfsavecontent> and some string operations.  But one can do some pretty tricky things with custom tags, and this is understated.  Hardly anyone out there had mentioned looping in custom tags, and I don't seem much about nesting custom tags either, so I thought I'd write about that.  I think these are two rather powerful text manipulation tools.

I wanted to use an example that everyone would grasp from the outset, so I've decided to mock-up <cfquery> and <cfqueryparam> as custom tags.  The examples I've got are just proof of concept, all the functionality not specifically related to custom-tagginess is stubbed (so the validation is faked, no DB call is actually made etc).  What it does do is process the enclosed SQL and parameter tags into an SQL string suitable for sending to the DB, and an array of param values.

Here's my test file:

<!--- test.cfm --->
<cf_query name="people" datasource="myDsn" result="result">
    SELECT    *
    FROM    person
    WHERE    name    LIKE    <cf_queryparam value="%Zachary%" cfsqltype="CF_SQL_VARCHAR" maxlength="50">
    AND        dob        >        <cf_queryparam value="#createDate(2011,1,1)#" cfsqltype="CF_SQL_DATE">
</cf_query>
<cfdump var="#people#" label="query result">
<cfdump var="#result#" label="query metadata">

This should look familar: it's like a <cfquery> call.  It outputs the following:

query result - query
FIRSTNAMEIDLASTNAME
1Zachary1Cameron Lynch
query metadata - struct
PARAMS
query metadata - array
1
query metadata - struct
CFSQLTYPECF_SQL_VARCHAR
MAXLENGTH50
VALUE%Zachary%
2
query metadata - struct
CFSQLTYPECF_SQL_DATE
VALUE{ts '2011-01-01 00:00:00'}
SQLSELECT * FROM person WHERE name LIKE ? AND dob > ?

So it returns a query into the people variable, and also returns a result struct of the SQL string it passed to the DB, along with the parameters.  Note that the parameters are not in the format any DB would actually accept, but it demonstrates the result of processing the nested tags.

There are two key files in implementing this: query.cfm and queryparam.cfm, plus I have a file helpers.cfm which has some mocked functions in it, which I've done just so the code runs, even if it doesn't do much.  Anyway, here's query.cfm:

<cfscript>
// make sure the code is well-formed: must have an end tag & we must have a datasource
if (!thistag.hasEndtag){
    throw(type="UnmatchedStartTagException" message="Context validation error for the query tag" detail="The start tag must have a matching end tag.");
}
if (!structKeyExists(attributes, "datasource")){
    throw(type="RequiredAttributesException" message="Attribute validation error for tag QUERY" detail="It requires the attribute(s): DATASOURCE.");
}


if (thisTag.executionMode == "START"){
    include "helpers.cfm";    // this just abstracts out some mocked functionality to keep this file to-the-point
    dbConnector = getDbConnector(attributes.datasource);    // (UDF)
}else{
    // by the time we're here the queryparams have been processed, so the generatedContent is a value SQL string, and we have a collection of parameters
    
    result = dbConnector.execute(sql=thisTag.generatedContent, parameters=thisTag.queryparams);    // (UDF)
    
    // send some stuff back to the calling code
    if (structKeyExists(attributes, "name")){
        caller[attributes.name] = result; 
    }

    param name="attributes.result" default="cfquery";
    caller[attributes.result] = {
        sql        = thisTag.generatedContent,
        params    = thisTag.queryparams
    };
    thisTag.generatedContent = "";
}
</cfscript>

I've commented the code, but I'll go through it too.

Firstly there's some housekeeping: validate the syntax of the code. A <cf_query> call must have an end tag, and must also - for the purposes of this example - have a datasource. I realise <cfquery> doesn't need one these days, but I wanted to use the data source in the code, so I'm demonstrating validating that it's there.  If one was mirroring <cfquery> accurately, there'd need to be a bunch of rules about the data types of the arguments (datasource needs to be a string, for example), and if - again, for example - datasource is used then dbtype can't be, etc. You get the point.

Next the actual functionality of the tag.  We need to set some stuff up before we get to the bulk of the operations. First we include helpers.cfm, which is that mockery file I mentioned before. The code for that is further down. Then I use the datasource value to get a dbConnector. This would be a DB-specific object to help act as a bridge between CFML and JDBC. I've just mocked it up here as a proof of concept. We need these two things in place before any of the <cf_queryparam> tags are processed, hence doing them in the START tag.

Before going to the END tag bit, the <cf_queryparam> tags are processed. Let's look at that.

<cfscript>
// make sure the code is well-formed: no closing tag, and nested within a query tag
if (thistag.hasEndtag){    
    throw(type="UnmatchedEndTagException" message="Context validation error for the queryparam tag" detail="The queryparam tag does not support an end tag.");
}
if (listLast(getBasetagList()) != "cf_query"){
    throw(type="UnmatchedEndTagException" message="Context validation error for the queryparam tag" detail="The tag must be nested inside a query tag.");
}

// if we're here, we've got a legit queryparam tag
if (thisTag.executionMode == "START"){
    include "helpers.cfm";    // this just abstracts out some mocked functionality to keep this file to-the-point

    validateParam(attributes);    // (UDF) will raise an exception if everything ain't legit for this param
    
    // the query tag will have worked out which DB we're dealing with, so get a DB-specific object to help "translate" the param from CF-speak to JDBC-speak
    dbConnector = getBaseTagData("cf_query").dbConnector;
    jdbcParam = dbConnector.createJdbcParam(attributes);    // (UDF)

    // CFASSOCIATE is not as granular as it could be, so get rid of ALL the attributes and just pop back in the one we actually want to give back to the calling code
    attributes = jdbcParam; 
    cfassociate(basetag="cf_query", datacollection="queryparams");    // (UDF)

    // put in a parameter placeholder in place of the tag 
    writeOutput("?");
}
// there is no closing tag, so no "ELSE"
</cfscript>

This starts with more validation: this time no end tag is allowed, plus the tags must be nested within a <cf_query> tagset. Note that this is achieved via looking at what getBaseTagList() returns: in this case it returns "cf_queryparam,cf_query". I was slightly worried about this, because what if I was calling the tags via <cfmodule> or via <cfimport>? Then they'd not be using the "cf_" syntax. Fortunately Allaire thought about this, and irrespective of the calling syntax, getBaseTagList() returns it as per above.

Processing the <cf_queryparam> tag is all done in the START tag (as there is no END tag). We still need to check for this because some muppet might try to close the tag. Obviously I've coded this so this will result in an error, but it's good to always differentiate between when processing takes place in a custom tag too.  Note I've included helpers.cfm again here: the previous inclusion was not in the same compilation context as this file, so it doesn't "count".

First we validate the attributes of the tag to make sure they're legit to pass to the DB. This would be stuff like checking the data type is a valid one, checking the lengths of CF_SQL_VARCHAR values; ensuring the value of a CF_SQL_INTEGER parameter actually is an integer, etc.  Basically making sure what ColdFusion has been passed is fit for passing to JDBC. Note I have marked this as a "(UDF)" in the comment. This just means it's one of my faked helper methods. It's just there to demonstrate the point.


Next we delve back into the <cf_query> tag's code and grab that dbConnector we set up, and use this to create a proper parameter object to pass to JDBC. We use the getBaseTagData() function to do this. Note I'm just getting the immediate parent tag's data (it's the variables scope of the query.cfm code, as it stands at the end of the START block of processing), but I could equally specify to grab the data from a grandparent or any other ancestor tag too (see the docs: basically it takes a second param that is how many ancestor-levels to go back, as an integer).

There's a slight quirk here.  I use <cfassociate> to pass this tag's data back to the parent, but there's no control over what gets passed back: it's the entire attributes scope (and why the attributes scope, I dunno: surely that stuff is for inbound values, not outbound?). I don't want all that stuff going back to the parent, so I just pass back the resolved parameter "object".  Also note I've used a UDF version of <cfassociate> here, otherwise I'd have to write this whole thing in tags, which would be a mess, and not appropriate for code that doesn't involve any mark-up.

Lastly I output a question mark. This is the marker for the parameter in the SQL string.  This seems odd at first, but that's all one has to do.  The <cf_query> tag remembers all its generated content, and "forgets" any CF tags (or custom tags) within it, so if I simply output the question mark, that's all I need to do to resolve the parameter in the SQL string (the DB will be expecting question marks as placeholders for parameters).

Bear in mind that all that code just processed one of the <cf_queryparam> tags, it'll all get called a second time for the next one.

Back to query.cfm, we're in the END block. Here it is again to save you scrolling up and down:

// by the time we're here the queryparams have been processed, so the generatedContent is a value SQL string, and we have a collection of parameters

result = dbConnector.execute(sql=thisTag.generatedContent, parameters=thisTag.queryparams);    // (UDF)

// send some stuff back to the calling code
if (structKeyExists(attributes, "name")){
    caller[attributes.name] = result; 
}

param name="attributes.result" default="cfquery";
caller[attributes.result] = {
    sql        = thisTag.generatedContent,
    params    = thisTag.queryparams
};
thisTag.generatedContent = "";

By the time we get to the END block, all the bits between the tags have been processed, so we have the following:
  1. thisTag.generatedContent holds the SQL string, sans <cf_queryparam> tags, but with ? placeholders in their place;
  2. the actual queryparam object that we passed back from queryparam.cfm. Remember we "associated" the sub-tag's attributes scope with the "queryparams" variable in the parent.  This is an array, one entry for each subtag.  As coincidence would have it, that's exactly what we'd need to pass to the DB: an array of parameters. Cool.
So what we need to do with the SQL string and the parameters array is pass it to the DB and get the result. Before doing this we could have dealt with caching: you know, like how <cfquery> has cachedwithin and cachedafter, but I've omitted that as it's not really demonstrating anything further about custom tags than I already have. What one would do though is to serialise the param array and concatentate that to the SQL string, hash the lot and use that as a key to look-up the cache. If found and the timing is appropriate: "return" it. Otherwise continue...

We then use a technique to pass this value back to the calling code.  The calling code's variables scope is availed to a custom tag as the caller scope.  We could just stick the returned query into any old variable in the caller scope, but it's much tidier to let the calling code tell the custom tag which variable to use. In this case we've passed in a "name" attribute - following the approach <cfquery> takes - so we set caller[attributes.name] to take the result. The value of attributes.name is "people", so we end up with variables.people holding the result in the calling code.  We do exactly the same thing with the "results" metadata struct, except this time we mimic a different behaviour used in CFML: if no specific variable to use is specified, we use a variable named the same as the tag itself. In this case we use "cfquery" (rather than "cf_query") because I'm mimicking the behaviour of <cfquery> itself.

The last thing we do is to wipe the generatedContent variable, otherwise it'll be output on the screen, which we don't want in this case.

All that was pretty easy, eh?  Just a few unloved bits of functionality: getBaseTagList(), getBaseTagData() and <cfassociate> facilitate the interchange of data between nested custom tags.

Oh. For completeness, here's helpers.cfm:

<cfscript>
    // this is just to factor some stuff out of the main tag files

    // fake function to resolve which kind of DB we're on, based on the DSN name
    function getDbConnector(string datasource){
        return     {
            createJdbcParam = createJdbcParam,
            execute            = execute
        };
    }
    
    // these are the "methods" of the DbConnector "object"
    struct function createJdbcParam(){
        // this would create a Java object suitable to pass to JDBC, based on the CFML values 
        return duplicate(arguments[1]);
    }
    
    query function execute(required string SQL, array parameters){
        // this would pass the SQL and the parameters to the DB, and wait for the result
        var result = queryNew("");
        queryAddColumn(result, "id", [1]);
        queryAddColumn(result, "firstName", ["Zachary"]);
        queryAddColumn(result, "lastName", ["Cameron Lynch"]);
        return result;
    }

    
    boolean function validateParam(attributes){
        /*    
            this would validate that:
            - the appropriate attribute combos have been passed,
            - the value conforms to the type constraints 
            - etc
        */
        var isValid = true;
        if (isValid){
            return true;
        }else{
            throw(type="SomeAppropriateException");
        };
    }
</cfscript>
<!--- just so I can do the whole thing in CFScript --->
<cffunction name="cfassociate">
    <cfassociate attributecollection="#arguments#">
</cffunction>

As you can see, it's just mocking-up functions (and an "object") so as to simplify the code I was actually demonstrating: focusing on techniques rather than specific implementation.

I like custom tags, and wish I found more uses for them. Do you use them?  If so, what for? What are your thoughts on them?

Righto.

--
Adam