Things I'd never looked at before: ColdFusion 10 instance clustering

G'day:

This might well be one of those "stating the obvious" kinda posts, but this one's all about me deciding to have a look at what the "Cluster Manager" option in CFAdmin in ColdFusion 10 is all about.  I've never looked at it before, so I'm pig-ignorant about it.  Perfect for the topic of a blog article then, eh?  Heh.

Clarification:
As Sean points out: this is not a feature specifically of ColdFusion 10.  It's just I was using CF10 for this as the next step in this mission is how the new scheduled tasking in CF10 play with the clustering.  Sorry for the misdirection there.

This decision stemmed from a "conversation" (on Google Chat) with Dave Ferguson - of CFHour fame - the other night about clustering scheduled tasks in CF10 (a topic for another "things I have never looked at before" article in the next few days, once I actually look at them). I couldn't help him with his problem, but I felt guilty afterward about never having looked at this clustering malarky.  So I decided to set up a cluster, to see what was involved.

There's some docs about "Configure a cluster of server instances....", but they seem unhelpfully brief. So brief I'll replicate them here:

Manage clusters using the ColdFusion Administrator.

1. In the ColdFusion Administrator, click Enterprise Manager > Cluster Manager.
2. Enter a cluster name and then click Add.
3. Click the cluster name and move the servers to the cluster based on the requirement.
4. (If required) Edit the multicast port.
   Multicast port is used to group the cluster members together. Default value of multicast port is 45564. After you create a cluster, the port is added in the cfroot\cfusion\config\cluster.xml file.
   For more information on multicast port, see http://tomcat.apache.org/tomcat-6.0-doc/config/cluster-membership.html.
5. Specify if you need sticky session.
   Sticky session ensures that after a session is established on an instance, all future requests from the client are mapped to that instance.
6. Click Submit.

What.  That's it?  Well no.  All that achieves is having some instances listed under a "cluster" on the Cluster Manager screen.  There's nothing there about how to then use this cluster. That bites (from a documentation point of view).

I didn't want to horse around setting up an Apache or IIS website for this stuff, I just wanted to test it with the Tomcat web server.  But I saw no indication of what I was supposed to browse to to talk to the cluster, rather than just the individual instances.  Previously I'd used the instance manager to create two new instances: clusterNode1 and clusterNode2, and these both worked fine - individually - on ports 8502 and 8503: they were the third and fourth CF10 instance on this box, and the ports default to incrementing from the default instance's 8500.  But I should be able to browse to a single domain/port that hits the cluster, and the cluster managing layer picks one of the CF instances to service that request.  It wasn't that multicast port they mentioned: that's for the cluster nodes to talk to each other via (as far as I can tell).

I googled further (and there really is bugger-all in the CF10 docs about clustering), and landed on a PDF "Installing Adobe ColdFusion 10" which under the bit about WSConfig mentioned clustering, so I decided to bite the bullet and set up an IIS website and see if I could somehow coerce it to talk to the cluster rather than a specific node.  I set a vanilla site up, pointing to a new directory, popped a helloWorld.html file in there and tested the site served HTML.  Yes.

So then I ran WSConfig for one of the cluster's nodes, and clicked "Add", and saw something I had never noticed before:

(Apologies for my crude image manipulation skills there).

Honestly... I have used WSConfig hundreds and hundreds of times over the years. And I had never noticed one can select a cluster there.  So I basically selected MyCluster, pointed it to the new website, and... that was it.

I created a helloWorld.cfm file and hit that via the new website, and it ran fine. However a CFM file that outputs a variable saying "Hello World" on the screen does not really demonstrate that there's a cluster at play.  The easiest thing I found was to dump the SERVER scope out in a file, and that outputs server.coldfusion.rootdir, and that is instance-specific.  So running that repeatedly demonstrated I was flip-flopping between clusterNode1, and clusterNode2.  Cool!  It really was that easy.  It was harder finding the docs to tell me what to do than actually doing it.

---

That was yesterday evening (no: last night was the pub - still not allowed to drink alcohol, but I'll have alcohol-free beer - and catching up with Homeland season 2... it must've been Tues evening).  I can't do CF10 stuff at work as we've only got CF9, and as nice as I asked our sysadmins, they said they'd not let me install CF10 for experimentation in my own time.  Which is fair enough I guess, if inconvenient.

Fast forward to this evening and I saw a status update on Twitter by Dave saying he's seeing some weirdness with sessions on his cluster rig.  I'd not looked at sessions yet, so decided to see what was going on in my environment.

There's an option in the cluster set up to enable sticky sessions - so a given browser session will land on a cluster node, and then continue to use that node for the rest of the session - but I didn't want to do that: the sessions should replicate between the cluster nodes without being sticky.  My test for this was a couple of files:

// Application.cfc
component {

    this.name                = "testSessionCluster";
    this.sessionManagement    = true; 
}

<!---setSession.cfm --->
<cfset ts = now()>
<cfoutput>#ts#</cfoutput>
<cfset session.ts = ts>

<!---inspectSession.cfm --->
<cfdump var="#server#" label="server">
<cfdump var="#session#" label="session">

So the plan here is to hit setSession.cfm on one instance, and then I'd hit inspectSession.cfm on the other instance. By default the CF clustering flip-flops between instances every other request: so my request to setSession.cfm would hit clusterNode1, and the next request hitting inspectSession.cfm would hit clusterNode2. And if the session variables were being replicated, I should see the variable set on clusterNode1 on the request to clusterNode2.  No, I didn't.  Hmmm.  OK, I figured I had forgotten something about enabling sessions so I hit each node separately (via the Tomcat web server on ports 8502 and 8503), and the sessions were working fine.

Next I went back to the clustered site, and tried again, and noticed that the session.sessionId was changing every request. So not only were my sessions not replicating, they weren't actually even sticking from one request to the next.  Now note I don't mean that the sessionId was different between clusterNode1 and clusterNode2, I mean that every second request - so the ones to one node or the other - the sessions were changing every request.  So basically sessions weren't working.

On a whim I went into Cluster Manager and turned sticky sessions on, and restarted the instances (this is a necessity when reconfiguring the cluster), and repeated the experiment whilst "stuck" on a given node.  At least then sessions actually worked. But not the way they should.  I switched sticky sessions back off again, restarted the nodes, and they stopped working again. So - despite my experience with ColdFusion clustering being an amassed time of about 3h now - I am thinking my expectations are correct (sessions ought to be replicating between nodes) - and this represents a bug.

This, incidentally, is exactly what Dave is seeing (or so he says on Twitter to my 140-char description of the above).

Has anyone else had experience with ColdFusion 10 clusters, and know what's going on here?  I'm more than happy to be shown-up here, and find out I'm doing something wrong.  Or conversely, I'll get a bug logged.

The bottom line is though, I rather like how easy setting-up a cluster was!  I had always looked at that part of CFAdmin and screwed up my face and went "bleah... looks too much like server admin... yawn", and skipped it. I'm pleased Dave's mention of it all encouraged me to look at this.  Cheers Dave! :-)

Now that I have this cluster, I'm gonna look at clustered scheduled tasks.  And, indeed, the scheduled tasks in ColdFusion 10 from top to bottom: they've changed a lot since CF9, and there's some handy stuff in here.  Almost all of which I know nothing about.  At the moment.

Update:
I've compared the ColdFusion 10 clustering to ColdFusion 9's.  CF9's works better, but not perfectly.

Righto.

--
Adam