PHP: the first few tutorials, and getting to grips with a new (to me) language

G'day:
I'm back into the work swing of things now - having been remote for the last month - so have the usual 1h before work starts to knock out some blog bumpf. I'll not get this article finished in that time, but maybe at lunchtime. We'll see.

Anyway, a week or so ago I decided to start learning PHP, as you will know if you follow this blog with any sort of regularity. I promised to blog my progress, and it really seems like there hasn't been much yet. However I did dive in straight away and do some tutorials, I've just not had a chance (or motivation) to write anything about my findings yet.

But here I am now.

PHP: built-in web server

G'day:
This is a real quick one as I have only 20min before I'm due to start work. And I need to go to the supermarket yet to get my breakfast.

So far I'm messed around with running PHP on Apache and IIS, and I think I lamented that PHP didn't seem to have a built-in web server option (although whether this made it into an article I'm not sure: I can't find it). Well whilst reading some docs during my KL to London flight last week, I saw mention of a built-in web server, so I decided I'd better look into this at some stage.

ColdFusion REST: is this a bug (in either CF or just my understanding)?

G'day:
(And, no, this is not a case of Betteridge's Law... I'm actually asking a question 'cos I don't know the answer).

I was having a mess around REST stuff this afternoon, half-heartedly trying to work out what the question actually was that someone asked on Twitter, and came across some puzzling behaviour. I don't really know REST from Adam (as it were) so this is definitely a question, not a statement.

CFML compilation into Java .class files

G'day:
Once again, Stack Overflow is my muse. Today a person called David Mulder is asking a question the answer to which relates to how CFML source code is processed into a Java .class file before the class file is then executed.

David's question was basically why this code doesn't behave how one might hope:

<!---test.cfm --->
<cfset msg = "G'day World">
<cfoutput>
<cfinclude template="incNoOutput.cfm">    
</cfoutput>
<cfinclude template="incWithOutput.cfm">

<!--- incNoOutput.cfm --->
#msg#<br>

<!--- incWithOutput.cfm --->
<cfoutput>#msg#</cfoutput><br>

Completely useless information about a quirk in ColdFusion's CFML

G'day:
I'm just preparing to return to the UK on Tuesday evening (I'm currently in NZ), and as I do not intend to set foot back on these shores for a long time, there's been a lot of beer-drinking with NZ-based mates for the last few days. I was at the pub from 3pm until [I have no recollection of leaving, really] yesterday, and am about to head off to another pub to drink again with some different mates (and some of the same mates again). We're mustering at 1pm. I'm still feeling the rigors of yesterday's effort, and am trying to steel myself to start again. I have about an hour before I need to head out, so this is a very quick and silly article.

More bugs that annoy me: CFHEADER & restSetResponse() (and CFCONTENT, whilst I'm about it)

G'day:
This came up whilst I was writing the code for the last article. In my Application.cfc, I had this onRequest() interceptor:

public void function onRequest(required string requestedFile){
    writelog(file="requests",text=arguments.requestedFile);
    if (arguments.requestedFile does not contain "restricted"){
        include arguments.requestedFile;
        writelog(file="requests", text="#arguments.requestedFile# Completed OK");
    }else{
        writelog(file="requests", text="#arguments.requestedFile# Blocked", type="warning");
        throw(type="InvalidFileException");
    }
}

Note how I'm raising an exception if the template is a "restricted" one. This sends a "500 Internal Server Error" back to the client. This is incorrect, as the server hasn't had one of those. What I should be returning is a "403 Forbidden", because that best describes the response.

But how am I to do this in a script-based CFC?

I can't use the script equivalent of <cfheader> because there isn't one (bug 3350715). I recalled there is a function restSetResponse(), but that only works in REST responses (and even then, it hardly works in a useful fashion). I have since raised a bug relating to this: 3546046.

Thinking about it further... why is it restSetResponse()? Why is it not just setResponse()? If there's a rationale for this functionality in the response from a REST request, then the same rationale exists for any other sort of request response. This function deals with HTTP, it's nothing specific to REST. I've raised another bug for this too: 3546047 (it's a bug, because it's a stupid inplementation, even if it's by design).

And while I'm about it: there's no script equivalent of <cfcontent> either. And that pisses me off too (bug 3133316). Come on Adobe... you need to get all this stuff over into CFScript quick smart.

--
Adam

It's easy to create a security hole in your application with onCfcRequest()

G'day:
Man... PHP continues to take a back seat with me, as interesting ColdFusion stuff keeps cropping up. Well: my definition of "interesting", anyhow.

Today on Stackoverflow I was reminded of a potential "gotcha" that onCfcRequest() brings to the table. One that has bitten me on the bum in the past, has done so with the person on S/O too, and has probably bitten other people too. And the "gotcha" creates quite a gaping security hole in the application if one doesn't deal with it.


I've not seen much coverage of this, so I'll detail it here.

First some background.

When Application.cfc was implemented back in CFMX7 (?), one of the methods added was a request interceptor, onRequest(). Instead of the requested file being run, onRequest() was run instead. This gives one's application a chance to do [something] regarding the requested template before, after and/or instead of running the template. An example would be this:

// Application.cfc
component {

    public void function onRequest(required string requestedFile){
        writelog(file="requests",text=arguments.requestedFile);
        if (arguments.requestedFile does not contain "restricted"){
            include arguments.requestedFile;
            writelog(file="requests", text="#arguments.requestedFile# Completed OK");
        }else{
            writelog(file="requests", text="#arguments.requestedFile# Blocked", type="warning");
            throw(type="InvalidFileException");
        }
    }
    
}

<!---publicFile.cfm --->
This is the public file

<!--- restrictedFile.cfm --->
Should not be able to access this!

If I browse to each of publicFile.cfm and restrictedFile.cfm, I get the expected results on the screen, and this in the log:

/shared/git/blogExamples/onRequest/publicFile.cfm
/shared/git/blogExamples/onRequest/publicFile.cfm Completed OK
/shared/git/blogExamples/onRequest/restrictedFile.cfm
/shared/git/blogExamples/onRequest/restrictedFile.cfm Blocked

So that's quite cool.

However note how the requested file is included. This approach won't be suitable for remote method requests, so for those, there is a method onCfcRequest(). This works the same way, but invokes the CFC & method, passing in the specified arguments:

public any function onCfcRequest(required string cfc, required string method, required struct args){
    return invoke(cfc, method, args);
}

(Obviously the same logic as per the onRequest() method could be used here too, but I've omitted it for the sake af clarity).

OK, great. What's the problem?

Well consider this CFC:

component {

    remote string function remoteMethod(){
        return "From remoteMethod()";
    }

    public string function publicMethod(){
        return "From publicMethod()";
    }

    package string function packageMethod(){
        return "From packageMethod()";
    }

    private string function privateMethod(){
        return "From rivateMethod()";
    }

}

Which methods are intended to be called here? Just the remote one. But here's the thing... a method call is only "remote" if it's being called from outwith the application. And when using onCfcRequest(), where is the method actually being called from? From this statement:

return invoke(cfc, method, args);

Which is inside the application. It's not registered as a remote call, because it's not being called remotely! This means that as well as all remote methods, one can also actually access all public methods too (package and private ones are still secure). This is something people don't necessarily realise, so one might inadvertently expose a helluva lot more of one's API than is the intent. YIKES.

So what's the mitigation here? Well one needs to do something similar to the logic we employed in onRequest() to block restricted files, but the problem is tricky to work around than that. One needs to do this:

public any function onCfcRequest(required string cfc, required string method, required struct args){
    var o = createObject(arguments.cfc);
    var metadata = getMetadata(o[method]); 
    
    if (structKeyExists(metadata, "access") && metadata.access == "remote"){
        return invoke(o, method, args);
    }else{
        throw(type="InvalidMethodException", message="Invalid method called", detail="The method #method# does not exists or is inaccessible remotely");
    }
}

Basically one needs to look at the object's metadata and check to see if the method being called is remote. If so: all good; if not: block it.

So make sure you do this in your onCfcRequest(). Otherwise you could be leaving yourself a bit open.

--
Adam

Question: when to lock scopes

G'day:
I was chatting with some other CFML developers over the last few days about when one should / should not use <cflock> to lock scopes.  My position is "hardly ever", and "certainly not as often as people tend to, and not for the reasons they think they have to".

What follows is my understanding of the situation, but it occurs to me that my understanding comes from anecdote, blog articles and inference. Nothing concrete. That said, I am 95% confident that I understand things correctly, but I'm suddenly annoyed about possibly missing that last 5%. So this article is to articulate my understanding, and solicit input from other people if I've missed anything, or got anything wrong.

PHP: jumping in and out of code is easy

G'day:
Now I will get back to discussing those PHP tutorials I did at some point soon, but I have a fairly fractured day today, so I don't have heaps of spare time for the blog. So here's a quicky.

In CFML, a lot of people have traditionally kept away from CFScript, kinda screwing up their noses at it, and preferring tags for all their code. One of the points that was made in the past that there was a lot of pretty essential CFML functionality that wasn't possible to use in CFScript because it was all wrapped up in a tag-only solution. ColdFusion 9 & 10 (and of course Railo too) have pretty much dealt with probably 95% of the stuff one would want to do in CFML in CFScript now, which makes that reasoning a bit obsolete.

More ColdFusion 5... CFML.EXE

G'day.
Do you know what's really cool about CF5? CFML.EXE, which will execute CFML from the command-line, eg:

Here I have run my gdayWorld.cfm directly from the command line, and it outputs the result. Nice. CFML really was quit a nice shell-scripting language back in the day, on Windows environments.


It's such a shame this was taken out of later versions of ColdFusion.

--
Adam

Installing ColdFusion 5 (yeah, that's not a typo) on Windows 7 64-bit

G'day:
This is not exactly what I was intending to do this morning, but - as is often the case - a question on Stackoverflow piqued my interest. In this case, a person needs to get CF5 running on Windows Server 2008 R2, and - seemingly - was having some problems, supposedly with the 64-bitness of their OS.

I have the CF5 installer lying around here somewhere, so I decided to see if I could get it to install on my Windows 7 64-bit laptop. I reckon if it installs on Windows 7, there's a good chance of it installing on on Windows Server too.

PHP: I've done some basic tutorials

G'day:
First of all I am very hungover today, and looking at the screen hurts so I am not going to vouch for the quality of this article. Or, indeed, whether it'll even see the light of day.

Straight after doing my PHP install & "Hello World" exercise the other day, I went to the Codeacademy site and worked through their PHP exercises. They were all pretty easy, but were a good reintroduction to PHP. I distilled a coupla things along the way, which I'll observe here. Just a note: this will be of no interest to anyone who already knows PHP, as it's really superficial stuff. I don't know enough about PHP yet to be interesting about it.

Do you have your code reviewed?

G'day:
I've just been chatting to Chris Weller this afternoon about various odds 'n' sods, and the topic of code review came up. I recalled that Stack Exchange has a code review sub-site, but the CFML presence on it is pretty limp: 12 questions in two years. I think it's a pretty good idea though. I've added it to my RSS feed, and see what people post. And I'll put my oar in as needs must ;-)

Apropos of nothing: readership stats

G'day:
Vainly, I track how many unique visitors I get to this blog each day, and chart a weekly average. The readership is really low - since its inception in July last year, the average is just over 100 people a day - but it's steadily climbing.

This year the average has been around the 120-150 mark, but last week spiked to an average of 243. On Monday I had 447 unique visitors according to Google Analytics. My previous highest was 285: that's quite a spike. Obviously that 447 had a big impact on the rest of the week's average: the other days were each just above 200, except for Sunday which is traditionally a very quiet day, with just 80-odd.

Still: it seems I'm getting more traffic, which is good.

PHP: IDE

G'day:
That "Hello World" code from yesterday (dammit... I missed a trick there... it shoulda been "G'day World", for this blog, shouldn't it!) was written using Notepad++, as I didn't have an IDE installed at that point in time.

My initial instincts for an IDE was to just use Zend Studio, as my understanding was that it was considered where it's at as far as PHP dev environments go. Then Andrew Myers recommended Netbeans, and I also googled around to see what else was on offer.

"Request functionality" doesn't not work with CFThread

G'day:
That was an intentional double negative.

This is a follow-up to my article yesterday, questioning why Adobe / Shilpi / Rupesh have said not to use "request-related functionality" within CFThread, and specifically choose to mention this when releasing a security update for ColdFusion 9-10. This makes me concerned as to why they are telling us this. I am moderately confident it was not simply a CFML tip for the community (because that'd be a first, surely, if nothing else ;-), there's a problem they're not telling us about.

The guidance was a bit vague, so I've looked into it more.

PHP: from zero to... Hello World

G'day:
Right, so I'm commencing this process of getting my brain around PHP. First up, I need to get the thing installed on this machine. I had it installed and gathering dust before,  but I completely rebuilt this thing recently when I migrated to Windows 7 (from Vista. Go on: laugh), so I need to start again.

I am typing this as I investigate.

Request related functionality should not be used with CFThread. Or so we're told

G'day:
Hopefully you know by now a new security fix has been released for ColdFusion version's  9 and 10.

There's a note on the Adobe ColdFusion Blog about it, and lots of other people have announced the same too.

For what it's worth, I've installed it on my CF10 and 9.0.2 instances and it installed fine.

So... aah... PHP then

G'day:
It's funny how things work out. For various reasons (which I shall not be disclosing) I am going to start shifting my spare time focus from ColdFusion to PHP. ColdFusion will still continue to be my primary focus for the foreseeable future, however I want to get up to speed with PHP as fast as I can. Starting today (Saturday in NZ).

One of my colleagues is a published author

G'day:
Another community announcement. One that I mean to make a week or two ago, but it kept falling out of my brain.

Anyway, one of my mates from work - Marcos Placona - has had some of his work actually publishing in a real book: "Instant jQuery Drag-and-Drop Grids How-to". I reckon that's pretty impressive.

I haven't read it yet: it all happened whilst I was over here in NZ, and I was waiting for the hard-copy to be released (which it has been now, and I've just bought one... It'll be on my desk when I get back to London). My plan is to get him to sign it, and when he shuffles off this mortal coil, I'll be able to sell it for millions! As I said to him "this time next year, I'll be rich!" ;-)

Anyway, support your fellow community members and give it a read. Congrats, Marcos.

--
Adam

A new (to me) ColdFusion blog appears on my radar

G'day:
This might be old news to everyone, but thanks to a tip from one of the CF people I "follow" (how stalky that sounds. Probably more so now that I point it out) on Twitter - Carol Hamilton -  I "discovered" a ColdFusion blog I was previously unaware of. It's by Summer S. Wilson, and called ColdFusion Beyond. I've read a coupla articles, and have found it pretty interesting and with an engaging writing style. Plus she mentioned a curious bug that has piqued my interest, except I've not investigated yet.

Check it out!

And... err... that's it.

--
Adam

onApplicationEnd() usage survey results

G'day:
I've had a reasonable number of results on my survey asking whether or not you use onApplicationEnd(), and if so: how so. It's telling a predictable story: the answer is "no, not really".

I've had 42 responses, and 38 of them are "no".


Four were "yes", with the following comments:

Current application uses onApplicationEnd to write to a log file that the application has shut down.

---

for some projects I work on we have to do audit records almost everything. So that's one of the things that's triggered on Applicationend. Other stuff I've used it for was for general cleanup of data.

---

I've used this to clear a shopping cart for a reservation app. I'm not convinced it was the best solution. I also used it for a large streaming app with thousands of users to save the last channel they were listening to when their app timed out. So at their next login it remembered. It does delete the structures & I had a lot of issues with blowing up memory with session variables.

---

Record page views for cart tracking / checkout progress.

I think the logging situations make sense, although perhaps it would be more robust to put it in onApplicationStart(), because an application will always start, but it might not end: for example if the box itself crashes, or the JVM gets tangled-up (sorry about the technical jargon there). So there's a chance of missing some metrics doing it at the end. I understand, though, in an auditing situation "what makes best sense" is not necessarily one of the requirements. And also there's merit to be had in a "belt 'n' braces" approach.

I'm not sure about the shopping cart ones. Aren't shopping carts generally session-centric, not app centric? Also an application can time out without a session timing out. If the application times out and restarts, it will re-acquire any sessions that were still around when it timed out. Obviously it'd be unusual in the normal sequence of events for the application to time out before its sessions, but it's possible if say one calls applicationStop().

Still: I will not base any sort of judgement from a quick comment in a survey, obviously I know stuff-all about the environments / codebases in question.

There were a coupla good comments on the survey article too:

Abram Adams said:

I've never used onApplicationEnd (and indicated so in the survey), but that got me thinking of what I could use it for. Perhaps as a mechanism to keep an application alive and fresh? One of our larger projects uses a custom framework that does some data/object caching, etc... The first request (if app has expired) can take around 5-10 seconds to complete while subsequent requests take <1-2 seconds. Perhaps I could hook into the onAppEnd and re-init the framework so that it doesn't become stagnant, but is always ready...

I've prototyped the idea in my dev environment and it does seems to work nicely, though there may be hidden gotchas that I'm not aware of. Any thoughts to that approach?

And Jose Galdamez said:

I've rarely had a need for OnApplicationEnd. The server should be handling garbage collection of application-scoped variables, so one shouldn't have to write that sort of logic. When I see OnApplicationEnd in lessons I usually see examples where applications log information such as the date/time when it ended. Personally, I can't think of anything useful to log when the application ends. If you really want to keep track of the application lifecycle you could always log that within OnApplicationStart and leave it at that. I'm curious to see what people say in your survey!

Both good observations.

I think with the application timeout thing - I've had similar thoughts in the past - the easiest way to mitigate it is to have a really long application timeout. This removes the issue. Bear in mind that by the time onApplicationEnd() runs, the application has already timed out, so I think if one is after continuity, then making sure it never times out, rather than re-init-ing it if it does time out is a better approach.

Also if an application must be up, then there should be some sort of "are you up?" check going on to alert [someone] if the answer is "no". And the act of asking the question will reset the time out timer anyhow.

Unfortunately there were no responses with some really excellent thing we can all take away thinking "hey, that's cool". But I think we all probably suspected that would be the case anyhow.

Thanks for filling out the survey, to all those that did.

Righto.

--
Adam

Plutarch (via Andrew Myers) teaches me something about struct keys

G'day
I chat to Andrew Myers a bit on Twitter (OK, well it goes out to the entire Twitter universe, but you know what I mean). And a few weeks ago he flicked through some interesting code to me. I've been meaning to write this up for three weeks now, but other stuff seems to keep cropping up that I "need" to write up first. Anyway, I've just looked at his repro case and it's an interesting one. And the code speaks for itself (thus making this a very easy article to write):

CFML: Quick survey: do you actually use onApplicationEnd()?

G'day:
I'm in rapid fire mode today.

When talking about all this onApplicationEnd() stuff recently, Bruce asked a good question: what does one use it for? I mean in real life? Well: I don't.

Do you? If you do, and fancy sharing, pls fill out this survey. It's one question: just a bit of typing.

Update:
The survey is closed. The results are here.

NB: as Sean has just pointed out... my wording was slightly vague before, but I'm also keen on you answering even if you don't use onApplicationEnd(). There's a "no" option in there too. It'll be good to see the split of do/don't use it.

I'll report back with what people say when when the frequency of submissions flattens out.

Cheers.

--
Adam

CFML: Preventing onApplicationStart from kicking off whilst onApplicationEnd is still running

G'day:
This is a follow-up from yesterday's article discussing that onApplicationStart() will run quite happily whilst onApplicationEnd() is still working. The person on Stackoverflow - rhinds -  was trying to use locking to force the processes to queue-up, but was having issues. I volunteered to look into it further. And I have.


Here's a rough facsimile of what I was expecting the person was trying to do:

// Application.cfc
component {

    this.name = "lockApplicationScope4";
    this.applicationTimeout        = createTimespan(0,0,2,0);
    variables.sleepDuration        = 3000;
    sleep(variables.sleepDuration);


    function onApplicationStart(){
        logIt("Begin onApplicationStart()");
        logIt("Before lock onApplicationStart()");
        lock scope="application" type="exclusive" timeout=variables.sleepDuration/100 {
            logIt("Top of lock onApplicationStart()");
            sleep(variables.sleepDuration);
            logIt("Mid onApplicationStart()");
            sleep(variables.sleepDuration);
            logIt("Bottom of lock onApplicationStart()");
        }
        logIt("After lock onApplicationStart()");
        logIt("End onApplicationStart()");
    }

    function onRequestStart(){
        logIt("Begin onRequestStart()");
    }

    function onRequest(){
        include arguments[1];
    }

    function onRequestEnd(){
        sleep(variables.sleepDuration);
        logIt("End onRequestEnd()");
    }

    function onApplicationEnd(){
        logIt("Begin onApplicationEnd()");
        logIt("Before lock onApplicationEnd()");

        lock scope="application" type="exclusive" timeout=variables.sleepDuration/100 {
            logIt("Top of lock onApplicationEnd()");
            sleep(variables.sleepDuration * 3); // need some time to react
            logIt("Mid onApplicationEnd()");
            sleep(variables.sleepDuration * 3); // need some time to react
            logIt("Bottom of lock onApplicationEnd()");
        }
        logIt("End onApplicationEnd()");
    }
    
    function logIt(required String message){
        writeLog(file=this.name, text="#message# from @ #listLast(CGI.script_name, '/\')#");
    }

}

A lot of that is just telemetry-gathering so I can watch what's happening as it happens. The key bits are highlighted: I'm putting an exclusive application-scope lock on the code in onApplicationStart() and onApplicationEnd() so that the two won't run simultaneously.

My test code was much the same as yesterday:

<!--- normalRequest.cfm --->
<cfset logIt("Start requested template")>
<cfoutput>#now()#</cfoutput>

<!--- shutdownRequest.cfm --->
<cfset logIt("Start requested template")>
<cfoutput>#now()#</cfoutput>
<cfset applicationStop()>
<cfset logIt("End requested template")>

The test is run as follows:

• hit normalRequest.cfm to "warm" the application. I let this run to conclusion.
• Hit shutdownRequest.cfm, and after a few seconds whilst it's still running...
• hit normalRequest.cfm again.

What happens here is that normalRequest.cfm will start the application up; then the call to shutdownRequest.cfm will call applicationStop() to commence application shutdown, but whilst that's happening, a concurrent call to normalRequest.cfm comes in and commences application startup again. Yesterday we saw that onApplicationStart() will start running whilst onApplicationEnd() is still running, which could cause "unexpected results". We're trying to mitigate this.

Here's the log from a test run:

Begin onApplicationStart() from @ normalRequest.cfm
Before lock onApplicationStart() from @ normalRequest.cfm
Top of lock onApplicationStart() from @ normalRequest.cfm
Mid onApplicationStart() from @ normalRequest.cfm
Bottom of lock onApplicationStart() from @ normalRequest.cfm
After lock onApplicationStart() from @ normalRequest.cfm
End onApplicationStart() from @ normalRequest.cfm
Begin onRequestStart() from @ normalRequest.cfm
Start requested template from @ normalRequest.cfm
End onRequestEnd() from @ normalRequest.cfm
Begin onRequestStart() from @ shutdownRequest.cfm
Start requested template from @ shutdownRequest.cfm
Begin onApplicationEnd() from @ shutdownRequest.cfm
Before lock onApplicationEnd() from @ shutdownRequest.cfm
End requested template from @ shutdownRequest.cfm
Begin onApplicationStart() from @ normalRequest.cfm
Before lock onApplicationStart() from @ normalRequest.cfm
Top of lock onApplicationStart() from @ normalRequest.cfm
End onRequestEnd() from @ shutdownRequest.cfm
Mid onApplicationStart() from @ normalRequest.cfm
Bottom of lock onApplicationStart() from @ normalRequest.cfm
After lock onApplicationStart() from @ normalRequest.cfm
End onApplicationStart() from @ normalRequest.cfm
Begin onRequestStart() from @ normalRequest.cfm
Start requested template from @ normalRequest.cfm
End onRequestEnd() from @ normalRequest.cfm

Something's up here. onApplicationEnd() is never getting into the lock. It looked to me like it was erroring but there was nothing in the log.

So I hastily changed onApplicationEnd() to do my own logging:

function onApplicationEnd(){
    logIt("Begin onApplicationEnd()");
    logIt("Before lock onApplicationEnd()");
    try {
        lock scope="application" type="exclusive" timeout=variables.sleepDuration/100 {
            logIt("Top of lock onApplicationEnd()");
            sleep(variables.sleepDuration * 3); // need some time to react
            logIt("Mid onApplicationEnd()");
            sleep(variables.sleepDuration * 3); // need some time to react
            logIt("Bottom of lock onApplicationEnd()");
        }
    }
    catch (any e){
        logIt("Exception in onApplicationEnd(): #e.message# #e.detail#");
    }
    logIt("End onApplicationEnd()");
}

And repeating the experiment, we see this (just the relevant bit):

Begin onApplicationEnd() from @ shutdownRequest.cfm
Before lock onApplicationEnd() from @ shutdownRequest.cfm
Exception in onApplicationEnd(): Cannot lock application scope. Cannot use cflock to lock the application or session shared scopes without these scopes  using the cfapplication tag. To use the session  scope, you must enable session management. Application and/or Session variables  must also be enabled in the ColdFusion Administrator. from @ shutdownRequest.cfm
End onApplicationEnd() from @ shutdownRequest.cfm
So that explains that, and it kinda makes sense. Bear in mind that this is running in onApplicationEnd(), so the application scope is gone by the time this code runs.

So to do what rhinds wanted to effect, instead of using a scope lock, we can use a named lock quite safely here. IE:

function onApplicationStart(){
    logIt("Begin onApplicationStart()");
    logIt("Before lock onApplicationStart()");
    lock name="applicationEventHandlers" type="exclusive" timeout=variables.sleepDuration/100 {
        logIt("Top of lock onApplicationStart()");
        sleep(variables.sleepDuration);
        logIt("Mid onApplicationStart()");
        sleep(variables.sleepDuration);
        logIt("Bottom of lock onApplicationStart()");
    }
    logIt("After lock onApplicationStart()");
    logIt("End onApplicationStart()");
}

// ...

function onApplicationEnd(){
    logIt("Begin onApplicationEnd()");
    logIt("Before lock onApplicationEnd()");
    try {
        lock name="applicationEventHandlers" type="exclusive" timeout=variables.sleepDuration/100 {
            logIt("Top of lock onApplicationEnd()");
            sleep(variables.sleepDuration * 3); // need some time to react
            logIt("Mid onApplicationEnd()");
            sleep(variables.sleepDuration * 3); // need some time to react
            logIt("Bottom of lock onApplicationEnd()");
        }
    }
    catch (any e){
        logIt("Exception in onApplicationEnd(): #e.message# #e.detail#");
    }
    logIt("End onApplicationEnd()");
}

And now, running this, we get what we want:

Begin onRequestStart() from @ shutdownRequest.cfm
Start requested template from @ shutdownRequest.cfm
Begin onApplicationEnd() from @ shutdownRequest.cfm
Before lock onApplicationEnd() from @ shutdownRequest.cfm
Top of lock onApplicationEnd() from @ shutdownRequest.cfm
Begin onApplicationStart() from @ normalRequest.cfm
Before lock onApplicationStart() from @ normalRequest.cfm
Mid onApplicationEnd() from @ shutdownRequest.cfm
Bottom of lock onApplicationEnd() from @ shutdownRequest.cfm
End onApplicationEnd() from @ shutdownRequest.cfm
Top of lock onApplicationStart() from @ normalRequest.cfm
End requested template from @ shutdownRequest.cfm
End onRequestEnd() from @ shutdownRequest.cfm
Mid onApplicationStart() from @ normalRequest.cfm
Bottom of lock onApplicationStart() from @ normalRequest.cfm
After lock onApplicationStart() from @ normalRequest.cfm
End onApplicationStart() from @ normalRequest.cfm
Begin onRequestStart() from @ normalRequest.cfm
Start requested template from @ normalRequest.cfm
End onRequestEnd() from @ normalRequest.cfm
This was hard to colour-code sensibly but the green is the shutdown request, and the orange the following "restart" request. Within that, we can see the red text shows all the internals of onApplicationEnd() being run before the blue text that is onApplicationStart(). it all runs in series, so that's the result we want. Sorted.

So the work-around for this issue with onApplicationStart() and onApplicationEnd() running concurrently is to use a named lock.

---

Oh. And why did the error that we got when trying to lock the application scope in onApplicationEnd() not show up in the error log? because of this: 3041747. I even knew about that. It'd slipped my mind though. It'd be nice if Adobe - who apparently "didn't have time" to fix it - cracked on and found the time to sort that out.

--
Adam

Running ColdFusion Builder as a plug-in on 64-bit Eclipse is faster than CFB standalone

G'day:
Actually the title pretty much sums-up this article. But you know me: I'll pad it out to be a few hundred words at least ;-)

CFML: onApplicationStart will run whilst onApplicationEnd is still underway

G'day:
This is another article which comes on the back of research stemming from not knowing the answer to a Stackoverflow question. In this case the questioner had noticed that onApplicationStart() seems to re-run (so the application is restarting) whilst onApplicationEnd() from a previous application shutdown is still underway.


I figured this would be unfortunately if this was true, so decided to find out.

I knocked out this code:

// Application.cfc
component {

    this.name = "shutdownSequenceTest12";
    logIt("Begin Pseudoconstructor");
    this.applicationTimeout        = createTimespan(0,0,2,0);
    variables.sleepDuration        = 3000;
    sleep(variables.sleepDuration);
    logIt("End Pseudoconstructor");


    function onApplicationStart(){
        logIt("Begin onApplicationStart()");
        sleep(variables.sleepDuration);
        logIt("Mid onApplicationStart()");
        sleep(variables.sleepDuration);
        logIt("End onApplicationStart()");
    }

    function onRequestStart(){
        logIt("Begin onRequestStart()");
    }

    function onRequest(){
        include arguments[1];
    }

    function onRequestEnd(){
        sleep(variables.sleepDuration);
        logIt("End onRequestEnd()");
    }

    function onApplicationEnd(){
        logIt("Begin onApplicationEnd()");
        sleep(variables.sleepDuration * 3); // need some time to react
        logIt("Mid onApplicationEnd()");
        sleep(variables.sleepDuration * 3); // need some time to react
        logIt("End onApplicationEnd()");
    }
    
    function logIt(required String message){
        writeLog(file=this.name, text="#message# from @ #listLast(CGI.script_name, '/\')#");
    }

}

This basically emulates an app that takes some time to execute its various stages, so I can interleave requests and see how they run.

I then created two CFM templates:

<!--- normalRequest.cfm --->
<cfset logIt("Start requested template")>
<cfoutput>#now()#</cfoutput>

<!--- shutdownRequest.cfm --->
<cfset logIt("Start requested template")>
<cfoutput>#now()#</cfoutput>
<cfset applicationStop()>
<cfset logIt("End requested template")>

I use the former to "warm" the application, then use the latter to shut it down. Whilst the shutdown is in progress, I re-run the normalRequest.cfm, to see what happens.

Here's the resultant log file:

Begin Pseudoconstructor from @ normalRequest.cfm
End Pseudoconstructor from @ normalRequest.cfm
Begin onApplicationStart() from @ normalRequest.cfm
Mid onApplicationStart() from @ normalRequest.cfm
End onApplicationStart() from @ normalRequest.cfm
Begin onRequestStart() from @ normalRequest.cfm
Start requested template from @ normalRequest.cfm
End onRequestEnd() from @ normalRequest.cfm
Begin Pseudoconstructor from @ shutdownRequest.cfm
End Pseudoconstructor from @ shutdownRequest.cfm
Begin onRequestStart() from @ shutdownRequest.cfm
Start requested template from @ shutdownRequest.cfm
Begin onApplicationEnd() from @ shutdownRequest.cfm
Begin Pseudoconstructor from @ normalRequest.cfm
End Pseudoconstructor from @ normalRequest.cfm
Begin onApplicationStart() from @ normalRequest.cfm
Mid onApplicationStart() from @ normalRequest.cfm
Mid onApplicationEnd() from @ shutdownRequest.cfm
End onApplicationStart() from @ normalRequest.cfm
Begin onRequestStart() from @ normalRequest.cfm
Start requested template from @ normalRequest.cfm
End onRequestEnd() from @ normalRequest.cfm
End onApplicationEnd() from @ shutdownRequest.cfm
End requested template from @ shutdownRequest.cfm
End onRequestEnd() from @ shutdownRequest.cfm


Here the initial warming request is greyed out.

The next request is me requesting a shutdown, and whilst that's still running, I perform another normal request, which commences the application starting again. However the application is still in the process of being shut down from last time. Note that the stop/start event handler processing are intertwined there: the re-start actually finishes before the shutdown of the previous application is complete.

I think this is something to be aware of, and it makes a kind of sense.

Bear in mind that onApplicationStart() and onApplication() end are event handlers, not the events themselves. They do not represent the application starting or the application stopping, they are simply something that ColdFusion will fire off for us when it handles the start/stop process.

So if we were being dogmatic about things, then this is correct behaviour. However in reality, I think it's legitimate for a running onApplicationStop() process should block the next onApplicationStart() process, just like one won't get two almost concurrent requests both trying to run onApplicationStart() each: only the first one runs it, and the second one waits until the first one is done before proceeding (and by then the application is started, so that second request doesn't even try to run onApplicationStart() itself). I think there's a good case for onApplicationStop() to be synchronised with onApplicationStart().

I have not run this on Railo yet, but I will after I walk down to the park to get some coffee.

Thoughts?

Update: I've written a follow-up to this article, which shows how to resolve this issue.

--
Adam

Now with comments on the mobile version

G'day:
Whilst trying to coerce Google / Blogger / Disqus to cooperate and get the comments for this blog indexed, I discovered how to enable comments on the mobile view of the site. Cool.

Moving House... Mango Blog?

G'day:
I'm vexed that the comments on this blog are placed via Javascript, so Google doesn't pick them up. Some of the stuff in the comments is really good, and worthy of reread if Google would only index them. The Disqus people maintain Google does pick up stuff brought in via AJAX calls these days, but my brief research into seeing it in action on a Blogspot blog has drawn a blank. And certainly none of my comments have been spotted by Google (they've done a good job of the rest of the blog though!).

A reader asks me: "CF10 for a new start up?". Answer: no

G'day:
Goodness: someone asking me for advice. How risky is that? Anyway, it's interesting I guess, and the person said I could use this as a blog article topic, so that's precisely what I'm doing. What I reproduce below is the completely unexpurgated, other than removing a couple of references which identify the correspondent/client/project. At the person's request, I have fixed a couple of typos from the original too. NB: the links below are from the original email, not something I added. I can't / won't vouch for their merits (one way or the other).

Overlapping requests will not cause a race condition in a CFM's variables scope

G'day:
This is mostly a follow-up from a Stackoverflow question. The question is based around some confusion arising from a blog article which offers some misinformation about how functions can cause race conditions across two requests simply by not VARing variables, or accessing the variables scope explicitly instead of using a function-local variable. To be very clear from the outset: the blog article is mostly wrong.

abort

G'day (again):
Can anyone point me to where in the ColdFusion CFML docs that it tells us that the CFScript syntax for this:

<cfabort showerror="My error message">

Is this:
abort "My error message";

Cool Regex Tool!

G'day:
A real quick one. One of my cronies - Simon Baynes - just pointed me to this cool regex visualising tool. Here's a screen shot of how it analyses a regex:

OpenBD's attitude disappoints me sometimes

G'day:
This is not the article I was thinking of writing today. But this is just too daft not to repeat.

The other day there was some discussion on the Railo Google group about some weirdness with JSON validation, which is worth a quick read. In summary, it seemed to the original poster that there was some false positives coming through. Being a literalist I observed that none of the examples were actually JSON according to JSON's own spec, as well as the RFC. Micha and I agreed to disagree - kinda... I'm sure Micha doesn't care about my opinion ;-) - as to whether Railo should stick to the spec at the expense of breaking existing code. He suggested the more moderate route.

"If you don't ask, you don't get"

G'day:
I don't often get a chance to quote Gandhi, but there you go. It was gonna be "If you tolerate this...", but that sounded a bit egregious in the context I am going to use it. Plus Gandhi has a bit more cred than the Manics. Sorry Wales.

... however arrayContains() works much faster than arrayFind()

G'day:
This is a quick follow-up to that thing I mentioned the other day about the relative performance comparison of arrayFind() and listFind().

ColdFusion bugs I'd like to see dealt with: param (first in what will be a series)

G'day:
With the upcoming ColdFusion 11 pre-release, I'm going to start banging about some ColdFusion bugs that piss me off. I'll mention them as I encounter them in my day-to-day work.

First up: param. There's a bug with param detailed in bug 3364510. Basically it's not been completed.

Integrating CF ORM into FW/1 and Stackoverflow bullying (mostly the latter)

G'day:
OK, so I sound a bit like a broken record with Stackoverflow, but they've rankled me even more than usual today. Someone's raised a pretty poorly-worded question (I am moderately confident English is not their first language though, so this can't be helped), but it was decipherable and one of the ColdFusion community's chief stalwart's - Scott Stroz - was helping the person out with his problems. Scott had posted an answer, and they were working back & forth to clarify the requirement, and making good headway.

And then one of the Stackoverflow moderators took it upon themselves to both close the question, as also delete Scott's answer.

Regular expressions in CFML (part 10: Java support for regular expressions (2/3))

G'day:

Mon 18 March 2013
Once again - for reasons better described over a few pints and the pub rather than in a blog article - I find myself winging southwards from London to Auckland. This time not for a holiday stint of a coupla weeks, but for over a month whilst I finalise some paperwork which requires me to be in NZ. Or more to the point: require me to be not in the UK. I left the safety of London today having read in the news this morning that Auckland had two earthquakes yesterday. Only baby ones, but still... I think NZers are a bit hesitant about their nation living up to its nickname of the Shaky Isles these days after Christchurch was flattened a coupla years ago. And Auckland doesn't traditionally have earthquakes, so to get any - even if small ones - is a bit concerning. In the back of my mind I am (melodramatically, and unnecessarily ~) concerned there's still something to land on when I get there. So, yeah... in the back of my mind I am hoping these earthquakes in Auckland stay around the 3 / 4 level on the MMS.

None of this has anything to do with regular expressions in Java, sorry.