REST & nouns & verbs: analysing the right problem

G'day:
I've recently changed teams at work, and have shifted from "safe" code bases that I've either been working on daily for over a year, or new applications that I've had a hand in from the ground up. Now I'm doing maintenance and adding features to existing code-bases written by different teams for different requirements, developed to different coding guidelines. So that's all a bit of a rude awakening that I am still settling into. I'm definitely out of my comfort-zone, and having to ask different questions than I usually might. That said: it's a good move and I kinda like that sort of thing, so cool!

For better of for worse, we use a lot of REST web services even for our internal processes. I personally believe this is architecturally questionable, but as no-one thought to ask me the question, I didn't get to partake in that particular discussion. To be fair to the situation, it was a decision made before I was part of the department; and - equally - I'm not entirely convinced of my position that it's a daft idea. Just mostly convinced ;-) Anyway, as the over-used truism declares: we are where we are.

Anyway, I find myself doing some maintenance work on one of our web services. This particular service handles the back-end processing for our web applications which make image file uploads. As part of the file upload we need to do a few things:

• upload the file and stick it in the appropriate place;
• store some user-provided data about the file (name, description, etc);
• create a few different resizings of the image for various purposes;
• contrive some metadata on all the file variations based on [stuff].
• distribute all the files across our CDN.

The first two of those steps are fast and can be done in real time. The file operations (resizing and distribution) take some time, and obviously the metadata extraction cannot be peformed until the files actually exist.

We have these split into two processes: one that always occurs immediately that uploads the master file and stores it and its user-enter data; then a second process which handles all the slow stuff.  This is just so we can provide fast feedback to the UI. Note that all this is still an atomic operation: both processes need to run.

On the first iteration of the implementation of this the two processes were fired off by the UI. Basically the upload form was submitted and the main request would handle the upload and the data-write, and then a second request was fired off by AJAX. This is not a robust solution as it isn't atomic. It's possible that the first call can be made but the second one doesn't for some reason. From an architectural point of view it's just not up to the view layer to make this call either; the versioning and metadata processing is an adjunct to the main file-handling process on the server: it's more the second half of the same process, not a separate process in its own right. Realistically the two are only separated at all because of performance considerations. From the perspective of the consumer of the web service: it should be one call to the API.

This was driven home to us by a second web app needing to do the same operations with the same web service, and the bods on that team knew about the first call but not the second.

So my task was to remediate this.

The solution we adopted was to have the front-end web app only call the one process, POSTing to http://api.example.com/file, passing along the file-upload form information as the POST body: this is what the UI-driven process was already doing. I simply augmented that to put an async job into our queuing system. The queue job receives the ID of the created file, and then it simply makes a second REST call, also POSTing to another end point: http://api.example.com/process. All good.

A first observation here is that "process" really isn't a great name for a REST end point. An end point should have meaning unto itself, and "process" is meaningless unless one knows it's tightly coupled to an initial end point relating to a /file: "Ah: so it's processing the file". One shouldn't need this sort of context when it comes to a web service end point. Sadly this end point is now out in the wild. Well: here "the wild" is just internal apps, but we can't simply change the URL to be something better without scheduling updates to those apps too. I'll be back-logging work to get that sorted out.

Part of my work here was to boyscout-rule the controllers of the "process" end point to move business logic out of the controller (yeah, don't ask), and into a service tier, which called a repository tier etc.

Some of the business logic I moved was to extract the decision as to which request parameter to use for the ID. Apparently we have some legacy code somewhere which passes the file ID as another argument name - I do not have the code in front of me right now so I can't remember what it is - so there was some logic to determine whether to use the legacy argument or the new one. Arguably this is perhaps still the job of the controller - it's dealing with stuff from the request object, which I can sell to myself as being controlller logic - but we're pretty stringent that we have as little logic in the controller as possible. And as I already needed a service method to do "Other Stuff", so I pushed this back into helper in the service too: something like extractDistributionCriteria(array $allCriteria). I don't want to be passing the entire request object into the service, so I extracted the POST params and passed those, using Silex's $request->request->all() method, and that returns the POST params as an array. Other than that, I was just shuffling logic around, so everything should be grand.

Book review: REST Web APIs: The Book (win a copy here)

G'day:
If you're a CFML user, you probably already know that one of the cornerstone members of the CFML community - Adam Tuttle - has recently written a book "REST Web APIs: The Book". I had the privilege of being one of the pre-release reviewers - from a content and language perspective - and Adam has now asked me if I could flesh out my perceptions a bit, as a book review. I've never been asked to do a book review before. Blimey.

ColdFusion REST services and restSetResponse() revisited

G'day:
Ages ago I wrote an article lamenting the way restSetResponse() has been implemented: "restSetResponse() requires the method to be returntype void. What?". At the time I was looking at how it was instrumental in how "ColdFusion takes something that should be easy and makes it hard" in the context of exception handling. That's slightly edge-case-y, I'll admit it.

But I think I've encountered a standard-operating-procedure situation today which demonstrates the implementation of restSetResponse() is not fit for purpose. Literally: it's not fit for the purpose it has been implemented for.

Today has been a frickin' frustrating day. I sat down to do another backbone.js tutorial ("Backbone.js Beginner Video Tutorial"), having finished "Anatomy of Backbone.js" and "Anatomy of Backbone.js Part 2" from CodeSchool yesterday. I started watching the video @ around 10am, and was inspecting the code on Github at 10:20am. At that point in time I figured I had better knock together the server-side code the tutorial will need: basically some RESTful web services for get-all, get, create, update and delete. Easy. I set out to implement this code using Railo, and had it all operational by 11:30am. At that juncture I started reading up on exactly what I should be returning for the less-obvious situations: the response for a GET is obvious: return the object(s) concerned. But what do I return for a POST? And a DELETE? So I started reading the HTTP spec ("Hypertext Transfer Protocol -- HTTP/1.1 - 9 Method Definitions"), which explained it all clearly.

One interesting thing I read on Stack Overflow which had me going "oh yeah! (duh)", was in answer to this question:

However I am wondering what should be the HTTP status code is the request sent by the client is valid (DELETE mySite/entity/123) and the entity to delete does not exist.

Because I was facing the same question (and with GET operations too). The answer is the very obvious:

In that case, the service should return an HTTP 404. Strictly speaking, a DELETE or a GET request for a resource that does not exist is not a "valid" request - ie. the client should not re-attempt that request because it will never succeed... The HTTP protocol defines 2 categories of problems - those with a 4xx status code, where the client must modify the request before retrying it, and those with a 5xx status code, which indicate that the service ran into trouble and the client should/could retry the same exact request without changing it.

As I'm perpetually wont to say to people who marvel that REST is some kind of wonderous thing... it's not. It's just "making HTTP requests". We do this every day in our browser. REST requests are no different. So if a resource doesn't exist... 404 the request.

Help me understand how a CFML REST request should not cause onCfcRequest() to fire

G'day:
I have to admit I am not very au fait with the idiosyncrasies of REST requests. To me a REST request is just... a request. The difference being the mechanism at the other end is more likely to respect verbs other than GET and POST like normal browser-based requests make, and they return "data" rather than mark-up. But other than that: there's nothing really to them. It's more a concept / strategy than anything else.

In this light, a while back I noticed that a REST request I was making didn't fire the associated onCfcRequest() event handler, as documented here: "REST requests don't seem to correctly use Application.cfc either". I duly raised a ticket: 3590745.

Rupesh has come back with this:

OnCFCRequest is meant for the requests which are made directly to the CFC - as AJAX request or directly over HTTP. For Rest request or web service or web socket requests, CFC happens to be an end point to a request (where the CFC is not explicitly invoked) and therefore OnCFCRequest is not called for these three types of requests.

I understand what he's saying, but to me - as I say in my follow-up - what he says seems like an example of post hoc ergo propter hoc. Basically the conclusion drawn is not implied by the supporting information. Here's what I said in response:

CFML: REST requests don't seem to correctly use Application.cfc either

G'day:
The emphasis in the title is an allusion to web socket requests also not respecting them.

God knows what this article will read like. I headed to the pub to catch the last quarter of the Aussie v Lions match, and I've been sitting here since. I'm over in Portumna visiting my son, and other than my 4h appointment doing that, I've kinda got nothing better to do here, so I'm still in the pub (which has wireless), amusing myself and drinking Guinness. Am just finishing my third pint. And will order a fourth. It's 3pm ;-)


Anyway, someone posted a question on Stack Overflow about how erroring REST requests don't fire onError() calls. This is a known issue: 3506757. But it was enough for me to furrow my brow and decide to have a look into it.

Now I'm no REST expert (neither in the context of ColdFusion, nor in general), so it always takes me a bit to get a REST CFC / function working, but I've come up with this test environment:

// Application.cfc
component {

    this.name    = "testappCfcIssue09";

    public void function onApplicationStart(){
        restInitApplication(expandPath("../api"), "api");
    }

    public void function onError(){
        writeLog(file=this.name, text="#GetFunctionCalledName()# called");
        writeDump(var=local, label=this.name);
        abort;
    }

}

This is a fairly vanilla Application.cfc. The onApplicationStart() stuff is just to work around some lazy programming on the part of Adobe, in that when one changes a REST-aware CFC, it will error until one restarts the REST engine. The onError() is just there in case REST calls call this onError() instead of the one in the directory the REST CFCs are in (I came into this not knowing how this stuff was handled at all, so just making sure).

<!--- testException.cfm --->
<cfhttp method="get" url="http://#CGI.http_host#/rest/api/Messaging/forceError" result="response">
</cfhttp>
<cfdump var="#response#" label="HTTP Response">

[I have decided against Guinness for the fourth: it's cider. It's a cracking hot day here today in Portumna, so not really Guinness weather]

This calls my REST method, and dumps the response. Now, over in the API dir, I have this lot:

//Messaging.cfc
component rest=true {

    remote string function gday(required string name restargsource="query") httpmethod="get"  {
        return "G'day #name#";
    }

    remote string function forceError() httpmethod="get" restpath="forceError"  {
        throw(type="ForcedException", message="Exception forced", detail="Because you asked me to");
    }
} 

The first method is just me testing that stuff was actually working properly with a vanilla method. The second method is one that forces an error, specifically so I can see what happens when an exception occurs.

And I have this Application.cfc in this directory as well, to log everything which goes on during REST requests:

// Application.cfc
component {

    this.name                = "restRequestLogging02";
    this.sessionManagement    = true;

    public void function onApplicationStart(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
    }

    public void function onApplicationEnd(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
    }

    public void function onSessionStart(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
    }

    public void function onSessionEnd(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
    }

    public void function onRequestStart(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
    }

    public void function onRequestEnd(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
    }

    public void function onRequest(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
        include arguments[1];
    }

    public any function onCfcRequest(required string cfc, required string method, required struct args){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
        var o = createObject(arguments.cfc);
        var metadata = getMetadata(o[method]); 
        
        if (structKeyExists(metadata, "access") && metadata.access == "remote"){
            return invoke(o, method, args);
        }else{
            throw(type="InvalidMethodException", message="Invalid method called", detail="The method #method# does not exists or is inaccessible remotely");
        }
    }

    public void function onError(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
        writeDump(local);
        abort;
    }

    public void function onMissingTemplate(){
        writeLog(file=this.name, text="#getFunctionCalledName()# called");
        writeeDump(local);
        abort;
    }

}

The only non-obvious method in this is the onCfcRequest() one. The code here is reflective of the unexpected "security issue" onCfcRequest() could effect in an application that I blogged about a while ago. Any onCfcRequest() method should do this lot as a minimum. That consideration aside, basically every event handler call is logged. That's the crux of things.

Is ColdFusion's REST implementation more verbose than it needs to be?

G'day:
I've been using some REST web services at work recently (CFML calling .NET ones; I'm just working on the CFML side of things, unfortunately), and also pottering around with ColdFusion 10's REST stuff at home occasionally. And then I wrote my "things I ain't researching" article, and one of the entries in that was to look more at CF's REST implementation more thoroughly, as well as Taffy and Relaxation. That in turn reminded me of a thread on Twitter in which Sean was denigrating ColdFusion 10's approach to REST, instead espousing a by-convention approach to things, or just in general not having the REST config alongside the code itself. I don't necessarily agree with him on the latter (I don't necessarily disagree, either!), but it's thought-provoking.

One thing that persisted in my mind is that it seems like there's a lot of horsing about making a method REST-callable, and after discussing a conventions-based approach with both Sean and separately with Adam Tuttle, I turned my mind to that.

God my writing is awful today, sorry. Slightly hungover. Bear with me.

OK, so here's a CFC and a method:

component {

    public string function greet(required string name) {
        return "G'day #arguments.name#!";
    }
}

The minimum I seem to be able to get away with to "RESTify" this is:

component rest=true {

    remote string function greet(required string name restargsource="query") httpmethod="get" {
        return "G'day #arguments.name#!";
    }
}

Passing a complex object to a ColdFusion RESTful web service

G'day:
I googled everywhere for how to do this, and drew a complete blank. I had to fall on the mercy of a friendly Adobe engineer to help me get it working, so I thought I would knock together a quick article so if anyone else ever needs to do this, there's something for Google to find.

Firstly, I gotta say that on the whole I probably would not create a RESTful web service which accepts a complex value... it seems to make more sense to me to work with simply types, and perhaps reflect any complex nature using collections of simple-value fields, or JSON or something. Still: I had this specific requirement which was out of my hands, so I needed to make it work.

ColdFusion REST: is this a bug (in either CF or just my understanding)?

G'day:
(And, no, this is not a case of Betteridge's Law... I'm actually asking a question 'cos I don't know the answer).

I was having a mess around REST stuff this afternoon, half-heartedly trying to work out what the question actually was that someone asked on Twitter, and came across some puzzling behaviour. I don't really know REST from Adam (as it were) so this is definitely a question, not a statement.

More bugs that annoy me: CFHEADER & restSetResponse() (and CFCONTENT, whilst I'm about it)

G'day:
This came up whilst I was writing the code for the last article. In my Application.cfc, I had this onRequest() interceptor:

public void function onRequest(required string requestedFile){
    writelog(file="requests",text=arguments.requestedFile);
    if (arguments.requestedFile does not contain "restricted"){
        include arguments.requestedFile;
        writelog(file="requests", text="#arguments.requestedFile# Completed OK");
    }else{
        writelog(file="requests", text="#arguments.requestedFile# Blocked", type="warning");
        throw(type="InvalidFileException");
    }
}

Note how I'm raising an exception if the template is a "restricted" one. This sends a "500 Internal Server Error" back to the client. This is incorrect, as the server hasn't had one of those. What I should be returning is a "403 Forbidden", because that best describes the response.

But how am I to do this in a script-based CFC?

I can't use the script equivalent of <cfheader> because there isn't one (bug 3350715). I recalled there is a function restSetResponse(), but that only works in REST responses (and even then, it hardly works in a useful fashion). I have since raised a bug relating to this: 3546046.

Thinking about it further... why is it restSetResponse()? Why is it not just setResponse()? If there's a rationale for this functionality in the response from a REST request, then the same rationale exists for any other sort of request response. This function deals with HTTP, it's nothing specific to REST. I've raised another bug for this too: 3546047 (it's a bug, because it's a stupid inplementation, even if it's by design).

And while I'm about it: there's no script equivalent of <cfcontent> either. And that pisses me off too (bug 3133316). Come on Adobe... you need to get all this stuff over into CFScript quick smart.

--
Adam

restSetResponse() requires the method to be returntype void. What?

G'day:
I have to say I didn't even know the restSetResponse() function even existed until I read about it on Stackoverflow this afternoon. The Stackoverflow question drew my attention to this snippet in the docs:

You must set the returntype attribute of the function to void when a custom response is returned using the function restSetResponse.