CFML: Threads, callbacks, closure and a pub

G'day:
I'm over in Ireland this weekend, and am doing my normal Sunday afternoon activity of sitting at Shannon Airport drinking Guinness, having finished my business early, so have a 4h wait in departures before my flight even considers boarding. Today's excitement here was the 500-odd US Army squaddies in transit to or from what I presume was Afghanistan. It makes a change from being the only person in the place, which was the case two weeks ago (other than shop staff, that is). Last night I killed some time in the pub too, during which I knocked out some quick code, to solve a problem I invented for myself to kill time. Here's the result.

A week or so ago Luis started an interesting thread on the Railo Google Group: "Add closure support to blocking operations to allow them to be non-blocking operations.". The summary of which is:

[...]I propose adding closure callback support to the following functions/operations so they can be processed natively in the background (non-blocking) and then call back when done.

FileRead
FileWrite
[...]
etc

Basically any operation that can block a network or local resource.  Imagine doing this:

fileRead( file, function(contents){
  process file here in the background once it is read.
});
[...]

The bits I omitted ([...]) are simply for abbreviation, and preserve the gist of his suggestion. This is a reasonable idea in general, but - as I go on to say in the thread - I don't think the suggested approach is very good for a coupla reasons. It's not bad, but it just seems a bit "embryonic" to me. But this is fine, this is what discussion forums are for.

My suggested approach would be more all-encompassing. Why limit this functionality to a subset of predetermined built-in functions? What about other built-in operations that might be slow, but otherwise doesn't need to interact with the rest of the code subsequent to it in the request? What about custom code that is similar? It makes little sense to me to "hard-code" this sort of behaviour to specific operations, to me.

A better solution would be to provide a general mechanism that can be used by any code to background-thread its execution, and fire callbacks on completion, failure etc, so as to kick off the next process, or flag-up that the process has completed.

Then it occurred to me that I thought I could knock a solution out to this using a single function. Not a very complete solution, but a solution nevertheless.

So sitting at the bar last night, over the course of four Guinnesseses, I came up with this lot:

public struct function defer(required function job, function onSuccess, function onFailure, function onError, function onTerminate){
    var deferThread = "";

    try {
        cfthread.status = "Running";
        thread name="deferThread" action="run" attributecollection=arguments {
            try {
                successData.result = job();
                cfthread.status = "Completed";
                if (structKeyExists(attributes, "onSuccess")){
                    onSuccess(successData);
                }
            } catch (any e){
                cfthread.status = "Failed";
                if (structKeyExists(attributes, "onFailure")){
                    onFailure(e);
                }else{
                    rethrow;
                }
            }
        }
    } catch (any e){
        cfthread.status = "Errored";
        if (structKeyExists(attributes, "onError")){
            onError(e);
        }else{
            rethrow;
        }
    }
    return {
        getStatus = function(){
            return cfthread.status;
        },
        terminate = function(){
            if (cfthread.status == "Running"){
                thread name="deferThread" action="terminate";
                cfthread.status = "Terminated";
                if (isDefined("onTerminate")){
                    onTerminate();
                }
            }
        }
    };
}

What this does is take a block of code - job - which will get fired off in a different thread. After it's done an optional onSuccess handler will fire; if it doesn't finish an onFailure handler fires (or onError, depending on the severity). It also returns two functions which can be run by the calling code: getStatus() - which does returns whether job is still running or has finished - and terminate() which kills the job.

This would be called like this:

bigFileToProcess = "path/to/file";

deferredJob = defer(
    job        = function(){
        var someResult = "";
        var fileHandle = fileOpen(bigFileToProcess, "read");
        while (!fileIsEof(fileHandle)){
            var line = fileReadLine(fileHandle);
            // etc
        }
        fileClose(fileHandle); 
        return someResult;
    },
    onSuccess    = function(result){
        // will receive someResult, so do something with it
    },
    onFailure    = function(e){
        writeLog(file="someLogFile", text="It didnae work because of #e.message#");
    }
);

This leverages ColdFusion 10's closures to bind the reference to bigFileToProcess to the calling code's reference, which is quite nice. This demonstrates that in effect job() just contains a chunk of the mainline code... one doesn't need to really think about it as a function, indeed it's good not to, rather just referring to the mainline code's variables and let closure look after the references for you.

It also leverages closure in the functions passed back, hence getStatus() knows about the cfthread.status variable in the defer() function, rather than look in the context it's called in. Similarly with terminate() accessing the same variable (and indeed the thread).

I dunno if this is really much use beyond a proof of concept. And as I just said to Adam Tuttle on IRC, I really mostly did it to try to get a solution to job-deferment implemented in a single function, instead of a framework / package / saga. I am surprised how much one can get done with how little code in CFML sometimes.

What would make it more handy is if ColdFusion had an inbuilt event-driven framework, allowing the callbacks to do stuff like trigger() other events, which other code is listening for. This could have huge possibilities, I think. I wrote-up my thoughts on this a while back: "Native event framework for ColdFusion 11 ?". And I have a follow-up to it to come.

I'm gonna pop it up on Code Review ("Quick 'n' dirty job deferment in CFML") to solicit sanity-checking / improvements, and also submit it to CFLib too (defer()). In case it is useful to anyone.

---

That planeload of squaddies is gone, and now another few hundred have shown up. Hopefully they're going home, not going out to fight somewhere :-S

Good luck either way, guys (and girls, obviously). And I hope "luck" never has to enter into it.

-- 
Adam