Friday 19 October 2012

The CFML request/response process

Every month or so I see someone asking questions along these lines:
  • I've set a variable in JavaScript, but my CFML code cannot see it.
  • I've got some CFML code in a variable, but when I output it it just appears on the screen, it doesn't execute.
These appear on the Adobe ColdFusion forums fairly frequently, or less often on StackOverflow.

In response to these I usually trot out a description of processes taking place when a request is made, and "the CFML server runs on the server and JavaScript runs on the client" and all that sort of rigmarole. And I type it in again each time, and it has either a lot of detail, or bugger-all detail depending on my mood at the time.

I've not had one of these for a while, so I'm about due.

Then the day before yesterday this kind of thing came up again when I was reading about a bug that was fixed in ColdFusion 10.0.3:

3227975: Pages with multiple frames load the incorrect pages when updating frames, intermittently.
I was thinking... "What the hell?  the CFML server doesn't know anything about frames and stuff like that?!  How can that be possible?  Have I been misunderstanding something all along?" (no, as it happens... I'll get to that).

So this is all in the back of my mind. A dangerous place for things to lurk and fester.

Recently when answering questions on either of those forums I've handily been able to point back to articles I've written here, so I've decided to write a decent / thorough rendition of my answer to the above questions once, and do it here. Thereafter I can link back to it.

Firstly, I have a special diagram. I don't have Visio or anything like that, so I ferreted around on Google and found this Gliffy website that lets one create diagrams online, and for free (well the 30-day trial is free; I dunno what happens after that). It's a cool site!

Here's the diagram. I'll explain it in narrative below.

Right, so a user is sitting at their browser (1), and they request a document (by clicking on a link, submitting a form, or making an AJAX request or whatever). This goes out via the 'net (2) to a web server which - as far as the client browser knows - will be just returning the requested document.

However if the request is for a CFM file, then the request is passed-off (3) to the CFML server. The server looks for an already-compiled response to the request (4), and if it finds it (5), processes it, ready to return the response to the web server.

However if the CFML server doesn't have an already-compiled response, it fetches the source code (the CFML) to service the request and generate the response, and compiles it (4a), then responds with that (5, again). NB: when I say "already-compiled response", this comprises a whole swag of compiled class files: one for every CFM or CFC file, and one for every UDF involved in servicing the request. This is the only time CFML plays a part in this process. So take note: for almost every request, CFML is not involved. What gets executed is the compiled code.

OK, so the CFML server executes the compiled code, and passes the response to the request back to the web server (6). At this point it is just a stream of text, quite possibly mark-up (although it could be JSON, XML, JS source code, etc). This is the end of the CFML server's involvement in the request / response.

The web server then sends the response mark-up (presuming this is an HTML document request) back to the client PC (7), which passes it to the browser (8).

The browser then processes the mark-up and any inline CSS and JavaScript and what-have you (9), and renders it along with the mark-up, and the results are the web page that was initially requested. As part of that, the HTML document might require other assets such as JavaScript files, CSS files, images and the like. At that point these files are requested in a similar fashion (but they are separate, completely discrete requests).

And that's the end of the request/response.

So there's a few things to note here:

Client PC/browser and CFML server

The client browser / computer at no point communicates with the CFML server. It communicates solely with the web server. The client doesn't know about the CFML server, and the CFML server doesn't know about the client.

The CFML server only talks to the web server. It neither knows nor cares why it's being asked to do something (or whether it's a full HTML document request or an AJAX request or any other sort of request), it just takes a request from the web server, processes it, and returns a response. That's it.

Often the CFML server process might be running on the same hardware as the web server process, but they're still completely distinct processes.

It is vitally important to understand that.

CFML does not get executed

CFML does not get executed, it gets compiled. What gets executed is the result of the compilation. To this end, one cannot use CFML to write other CFML, or use CFML to "output" CFML.

Some people get confused because they can do this:

<cfset isChecked = ' checked="checked"'>
<input type="checkbox" name="cb1"#isChecked# />

Which ultimately renders this:

<input type="checkbox" name="cb1" checked="checked" />

But they cannot do this:

<cfset isChecked = ' checked="checked"'>
<cfinput type="checkbox" name="cb1"#isChecked# />

The reason here is that all the CFML server is doing in the first case is spewing out text (which happens to be an <input> tag).

In the second case, the code is trying to build a CFML tag with CFML. Remember that the CFML has to be compiled (compile time) before it can execute (runtime). However variables only exist at runtime. Or taking this a different way: they don't exist at compile time. So there's the first hitch: isChecked doesn't exist at compile time, so how is the compiler supposed to deal with that? This is a show-stopper to start with. But before this even becomes relevant, the CFML compiler needs to syntax-check the CFML, so the CFML needs to be well-formed. And that <cfinput> example is not well-formed CFML.

This is why one cannot build CFML with other CFML.

Well one can. But what one needs to do is to generate the final CFML, then write it to a file, then call the file somehow. Then the ColdFusion compiler will compile it, and it'll be fine. But one cannot do it all within the same file like that.

Outputting CFML does not cause it to be executed

This seems so obvious that I'm surprised I need to say it, but some people think doing this should work:

<cfquery name="code">
    SELECT    cfml
    FROM    someTable
    // etc

Seriously: why the hell would anyone expect that to work (where "work" means "the code will actually execute"). In all situations, <cfoutput> outputs stuff. That's all it does (yeah, OK, it loops too. Not relevant here). Why would anyone expect it suddenly to mean "compile any code within the <cfoutput> tag as CFML and execute it"? This doesn't work.

If one has code stored in the DB (please don't), then the only way to get the CFML server to compile and execute it is to write it to a file, and then call the file:

<cfquery name="code">
    SELECT    cfml
    FROM    someTable
    // etc
<cffile action="write" file="someFile.cfm">
<cfinclude template="someFile.cfm">

That's it. That's the only way to do it. The CFML server will only process CFML when it comes from a file. Lucee (as of version 5) has a function called render() or something like that which'll execute a string containing CFML.

CFML code and JavaScript run in different environments

As per the diagram, CFML code is executed on the CFML server, then there's an amount of time that passes before the results get to the browser, then JavaScript code executes. There is no overlap, and the two systems do not interact. JavaScript does not know about CFML variables, and CFML code does not know about JavaScript variables. Or code. Or anything about each other.

What one can do is use CFML code to render JavaScript, which is then sent to the browser, but that JS is not executed until it's back at the browser (some time after the CFML server has completed its job and moved on).

So one can do this:

    <cfset cfMessage = "Hello World">
    jsMessage= "#cfMessage#";

Superficially this looks like CFML and JS interacting.  But it's not. All this means is that the CFML server generates this text as part of its response:

    jsMessage= "Hello World";

And sends that back to the browser. Which then executes it, and the alert displays the message. There is no interaction. CFML and JS run on completely separate systems, which are unaware of each other, and are separated by time and space (ie: the CFML server completes its job long before the browser even gets the JS returned to it, before it can execute it).

So that's what happens during a request/response, and how the key players interact. Or don't interact.

So what's the story with that bug then? How was the ColdFusion server involved in the wrong responses going to the wrong frames, when the ColdFusion server doesn't know about frames. Or even web browsers in general? This seemed so weird!

But no, it's not.

What was (/is) apparently happening is that somehow the connector between ColdFusion and the web server is screwed, and which response from ColdFusion goes with which request the web server passed to it is getting fouled up. So like the web server is sending requests for frames 1, 2, and 3 to ColdFusion, and ColdFusion is processing the three requests and sending the responses, but it's saying here's responses 2, 3, 1 (or something), and the web server isn't to know any better, so it just sends the wrong responses back for the wrong requests. And I can see how that could happen.

Anyway, that's that. Hopefully the diagram and the narrative can be recycled on the forums when people ask these questions...