Thursday, 6 May 2021

Lucee: what now for goodness sake

G'day:

I'm just writing this here because it's too long to put in a message in the CFML/Lucee Slack channel, and so I can get some eyes on it.

Consider this code (test.cfm):

<cfoutput>
<cfset relativeFilePath = "../getCanonicalPathIssue/targetfile.cfm">
Relative file path: [#relativeFilePath#]<br>
Does relative path exist: [#fileExists(relativeFilePath)#]<br>
<br>

<cfset expandedPath = expandPath(relativeFilePath)>
Expanded file path: [#expandedPath#]<br>
Does expanded path exist: [#fileExists(expandedPath)#]<br>
<br>

<cfset canonicalPathFromRelativeFilePath = getCanonicalPath(relativeFilePath)>
Canonical path from relative path: [#canonicalPathFromRelativeFilePath#]<br>
Does canonical pathh from relative path exist: [#fileExists(canonicalPathFromRelativeFilePath)#]<br>
<br>

<cfset canonicalPathFromExpandedPath = getCanonicalPath(expandedPath)>
Canonical path from expanded path: [#canonicalPathFromExpandedPath#]<br>
Does canonical path from expanded path exist: [#fileExists(canonicalPathFromExpandedPath)#]<br>
<br>

<cfset directory = getDirectoryFromPath(canonicalPathFromRelativeFilePath)>
Directory: #directory#<br>
Directory contents and do they exist:<br>
#directoryList(directory).reduce((buffer="", filePath) => buffer & "#filePath#: #fileExists(filePath)#<br>")#
</cfoutput>

And this is its output:

Relative file path: [../getCanonicalPathIssue/targetfile.cfm]
Does relative path exist: [false]

Expanded file path: [/var/www/public/nonWheelsTests/getCanonicalPathIssue/targetfile.cfm]
Does expanded path exist: [true]

Canonical path from relative path: [/var/www/public/nonWheelsTests/getCanonicalPathIssue/targetFile.cfm]
Does canonical path from relative path exist: [false]

Canonical path from expanded path: [/var/www/public/nonWheelsTests/getCanonicalPathIssue/targetfile.cfm]
Does canonical path from expanded path exist: [true]

Directory: /var/www/public/nonWheelsTests/getCanonicalPathIssue/
Directory contents and do they exist:
/var/www/public/nonWheelsTests/getCanonicalPathIssue/test.cfm: true
/var/www/public/nonWheelsTests/getCanonicalPathIssue/targetfile.cfm: true

Basically I've got a second file (targetfile.cfm) in the same directory as I'm running this code from (test.cfm), and I'm giving Lucee a (valid) relative path to it, and asking some questions. Everything goes OK until I get to the result of getCanonicalPath(relativeFilePath): it's upper-cased part of the file name!??! Also note that expandPath(relativeFilePath) gets it right. And that getCanonicalPath(expandedPath) also gets it right.

The last bit of the code just shows what's def in the directory concerned.

I actually know what's causing the problem. A coupla hours ago I renamed the file from targetFile.cfm to targetfile.cfm. note the change in capitalisation, and how the old version matches the bad value getCanonicalPath is coming up with. So Lucee is caching that for some reason. I also found where to uncache it, but am buggered if I know why this setting caches file paths:

That's supposed to be about how often the contents of the file are checked; nothing about file paths. But if I set it to "Always (Bad)", then the problem goes away. It's clearly this lesser used definition of "bad", that means "actually works".

So that's an hour or so of my life I'll never get back. Thanks.


For shits and giggles I decided to run this on ColdFusion:

I'd give ColdFusion a pass here for choking on a relative path if not for two things:

  • it's specifically documented as dealing with them: "Absolute or relative path of a directory or to a file."
  • The path is a valid relative path. There's no trickery with the casing going on here, or caching or anything like that. The relative path I'm providing is the path to that file, relative to where the path is being used.

Oh well. I suppose I'll check with ppl to see if there's a reason why this behaviour on each platform, isn't wrong in the way I claim it is… and then raise some bugs. I'll cross ref once I've done that, but it'll be tomorrow now.

Righto.

--
Adam

Tuesday, 4 May 2021

Short version: getting CFWheels working outside the context of a web-browsable directory

G'day:

This is all extracted from my earlier article "Installing and running CFWheels in my Lucee Docker container" from a few days ago. Why am I repeating it? Because it kinda got lost in the morass of my other witterings, but it's probably reasonable information to have as a stand-alone guide.

This is also a living document. The steps here work for the bits of CFWheels I've used so far, but perhaps need tweaking as I delve deeper into it. I'll keep it up to date wth my findings. If you spot anything additional I need to do, let me know. I suspect there might be some config settings in the last section that might need adjustment. The code that uses those settings is pretty impenetrable, so I figured I'd discover by experimentation, not wading through the CFWheels codebase.

This still looks like a chunk of instructions, but it boils down to this:

  1. point your web root at the correct directory on both the CFML server and web server;
  2. get the CFWheels code;
  3. move it around a bit;
  4. do some source control;
  5. add some server mappings;
  6. add some application mappings;
  7. change some CFWheels config.

It's easy.

What?

CFWheels official installation guidance is to put all its files in a web browsable directory. I would never generally install a web application like this, and nor should you.

Why?

There are, conventionally, three components to the code in a web app:

  • Elements that need to be web browsable. Primarily images, JS, CSS and other assets the web server needs to serve to present the website. And secondary to that, the entry point to the web application. In the case of a CFML app this is a stub index.cfm and Application.cfc (both of which point back to the application code, and don't do much else other than that).
  • The web application. This is your CFML code. The stuff in source control, and the the stuff all your tests hit.
  • Third-party code. Stuff like the frameworks you use, and other libraries you might need along the way. You simply use this code, you do not maintain it.
  • OK, so a possible fourth: tests. These are usually separate from the application source code.

These are three different things, and they belong in three different places, for both security and management reasons. They should not be munged together, and it's not a good approach to suggest that they ought to be.

Where?

There's no standard in the CFML world that I am aware of, so in this case I am borrowing from how composer handles it in PHP applications. This puts third party code in a vendor directory, categorised by the vendor name within that (and the specific project from that vendor within that). I have this (very summarised):

/var/www/
├── public/
│  ├── images/
│  ├── javascripts/
│  ├── stylesheets/
│  ├── Application.cfc
│  └── index.cfm
├── src/
│  ├── controllers/
│  ├── models/
│  ├── views/
│  └── Application.cfc
├── test/
│  ├── functional/
│  ├── integration/
│  ├── unit/
│  └── Application.cfc
└── vendor/
  ├── cfwheels/
  └── testbox/
  • public is the web site's webroot. The other directories listed here are not web accessible. In fact they're not even present on the web server.
  • I did not see the need to have the vendor-company directory within vendor, so have just gone straight to the app level with cfwheels and testbox
  • It doesn't matter where you put the third-party stuff, as long as it's not public, and not muddled in with your own code. You could use org/cfwheels/cfwheels and com/ortussolutions/testbox if you wanted to. Or some other sensible schema.

How?

Step 1: ensure only the public directory is web browsable

I'm using Lucee via their official Docker container here, but the reconfiguration requirements (if not exact implementation) will apply on ColdFusion too, as well as Lucee installed via other mechanisms.

The general configuration of a CFML server is to create some directory (in my case /var/www), and say "this is the directory that's web browsable, stick you code in there". This is bad default advice, and encourages bad and insecure practice as that default. We only want the public directory of our app to be web browsable; not the whole codebase! Fortunately thisis reasonably easily remedied. The web-browsable directory is set in $TOMCAT_HOME/config/server.xml. For me this is /usr/local/tomcat/conf/server.xml. In that file there is this section:

<Host name="127.0.0.1"  appBase="webapps" unpackWARs="true" autoDeploy="true">
    <Valve className="org.apache.catalina.valves.RemoteIpValve"
         remoteIpHeader="X-Forwarded-For"
         requestAttributesEnabled="true" />
    <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
         prefix="localhost_access_log" suffix=".txt"
         pattern="%h %l %u %t &quot;%r&quot; %s %b" />

    <Context path="" docBase="/var/www">
        <JarScanner scanClassPath="false"/>
    </Context>
</Host>

Change the /var/www to /var/www/public.

I've handled this for my Docker container by grabbing the file, saving it in my build context, and copying it across when I'm building the container (docker/lucee/Dockerfile):

COPY ./server.xml /usr/local/tomcat/conf/server.xml

Configuring your web server is outwith the remit of this article, but your website's doc root should be the public directory too, obviously.

This is also not related to this exercise, but whilst we're mentioning web servers, also follow the CFWheels docs when it comes to configuring rewrites from the CFWheels end of things. But I basically needed to add this to the bottom of config/settings.cfm:

set(URLRewriting="On")
set(rewriteFile="index.cfm")
Step 2: getting the CFWheels code

I used CommandBox to install CFWheels from Forgebox. This is my box.json:

{
    "dependencies":{
        "cfwheels":"^2.2.0"
    },
    "devDependencies":{
        "testbox":"^4.2.1+400"
    },
    "installPaths":{
        "testbox":"vendor/testbox/",
        "cfwheels":"vendor/cfwheels/"
    },
    "testbox":{
        "runner":"http://localhost:8888/test/runTests.cfm"
    }
}
Step 3: separating the CFWheels application implementation stubs from the framework application

All the CFWheels code is now in vendor/cfwheels. Some of this code is the basis for our application which we need to change; some of it is the CFWheels framework code. We need to copy some files to different more appropriate locations. All source file references below are relative to vendor/cfwheels. Destination locations are relative to the /var/www/ directory shown above.

  • index.cfm and rewrite.cfm need to be copied to public/
  • Application.cfc contains only an include for /wheels/functions.cfm. Copy only this include statement to be the last statement of src/Application.cfc.
  • tests/ should be copied to /var/www (adjacent to src/). These need to be source controlled, as they will contain your test code.
  • All the other subdirectories other than wheels/ of vendor/cfwheels should be copied to src/. These need to be source controlled, as they will contain your application code.
Step 4: omit src/wheels from source control
# .gitignore
/src/wheels/
Step 5: copy vendor/cfwheels/wheels to src/ during deployment

I'm using Docker for this, but as some separate step when deploying your code, you need to put the un-source-controlled vendor/cfwheels/wheels into src/. This is clunky and this directory does not belong in the middle of your application code, but there are a couple of hard-coded touch points to your source code within the wheels codebase, and try as I might, I could not work out how to override them.

WORKDIR  /var/www

RUN git clone git@github.com:adamcameron/cfml-in-docker.git .

RUN mkdir -p vendor
RUN box install

RUN cp -R /var/www/vendor/cfwheels/wheels /var/www/src/wheels
Step 6: Server mappings

Add some mappings that need to be at server level (not in Application.cfc), because Application.cfc files need them internally. I have done this with CFConfig, but you could do it via the admin UI:

{
    "CFMappings":{
        "/cfmlInDocker":{
            "inspectTemplate":"once",
            "physical":"/var/www/src",
            "primary":"physical"
        },
        "/test":{
            "inspectTemplate":"once",
            "physical":"/var/www/test",
            "primary":"physical"
        },
        "/wheels":{
            "inspectTemplate":"once",
            "physical":"/var/www/src/wheels",
            "primary":"physical"
        }
    }
}

My application's code is referenced with the mapping prefix /cfmlInDocker.

public/Application.cfc needs to extend cfmlInDocker.Application (ie: the one in the src/ directory). It can/should be otherwise empty.

We need the /wheels mapping to accommodate an include to a file on that path that we needed to put in src/Application.cfc.

TestBox's test runners all work via HTTP requests, so unfortuately the /test directory needs to be web accessible (these mappings are for code location, and also make the resources web-accessible on the internal CFML web server). They should not be exposed on your actual web server, so make sure they are blocked there.

Step 7: add application mappings to src/Application.cfc

The tail of your src/Application.cfc should end like this:

    thisDirectory = getDirectoryFromPath(getCurrentTemplatePath())

    this.mappings["/public/wheels"] =  getCanonicalPath("#thisDirectory#wheels")
    this.mappings["/app/wheels"] = getCanonicalPath("#thisDirectory#wheels")

    this.mappings["/app/controllers"] = getCanonicalPath("#thisDirectory#controllers")
    this.mappings["/app/models"] = getCanonicalPath("#thisDirectory#models")
    this.mappings["/app/events"] = getCanonicalPath("#thisDirectory#events")
    this.mappings["/app/files"] = getCanonicalPath("#thisDirectory#files")
    this.mappings["/app/plugins"] = getCanonicalPath("#thisDirectory#plugins")
    this.mappings["/app/views"] = getCanonicalPath("#thisDirectory#views")

    testDirectory = getCanonicalPath("#thisDirectory#../tests")
    this.mappings["/public/tests"] = testDirectory
    this.mappings["/app/tests"] = testDirectory
    this.mappings["/tests"] = testDirectory

    include "/wheels/functions.cfm";
}

This is so Wheels's internal CFC references can find your application code. You might wonder why we need all those individual "sub"-mappings of /app, instead of just one mapping to /app. This is because somewhere in the bowels of /wheels/functions.cfm that mapping is created, and given hard-coded assumptions about the codebase layout, it ends up pointing to the wrong place. At the same time, that code also used the paths I map above. Fortunately /app is never used by itself; it's always with one of those suffixes above.

Step 8: remap some include paths in src/config/settings.cfm

Add this lot to the bottom of src/config/settings.cfm:

set(eventPath = "../src/events")
set(filePath = "../src/files")
set(modelPath = "../src/models")
set(modelComponentPath = "../src/models")
set(pluginPath = "../src/plugins")
set(pluginComponentPath = "../src/plugins")
set(viewPath = "../src/views")

set(controllerPath = "/app/controllers")

set(webPath = "")
set(imagePath = "images")
set(javascriptPath = "javascripts")
set(stylesheetPath = "stylesheets")

set(wheelsComponentPath = "cfmlInDocker.wheels")

Values that "worked" here were all set via trial and error, because the CFWheels code is very… opaque, and there's no documentation I can find on them. Certainly I needed these values for the controllers and views to be located (that's as much as I've looked at with CFWheels so far). I expect the component paths need to be dotted rather than a file-system path: I'll try to find out this evening.

Conclusion

Ideally at some juncture CFWheels will be reconfigured to not assume where the application implementation code is, and use a mapping (that it is provided at bootstrap) to find it instead. And similarly path all its own code on a /wheels mapping, then the framework could reside anywhere. It should just need to work like this:

// my Application.cfc
component extends=wheels.Application {

    this.appMapping = "/myApp"
}

And in the setup docs say "make sure to create these two server mappings". It's just easy. Which is what CFWheels is supposed to be all about, right?

Righto.

--
Adam

abort! abort;!

G'day

What do you (CFMLers, sorry) make of this?

<cfscript>
function f(){
    writeOutput("in f<br>")
    abort
}

writeOutput("before f<br>")
f()
writeOutput("after f<br>")
</cfscript>

What would you expect that to output? The foolhardly money would go on:

before f
in f

Because, like, it's obvious: after f is never called because we're aborting in the function. Right?

Well yer partly right. On ColdFusion, that's exacly what happens (I tested on CF2021). On Lucee, however, I get this:

before f
in f
after f

Um… kewl.

The thing is that if one adds a semi-colon after the abort, Lucee starts behaving.

<cfscript>
function f(){
    writeOutput("in f<br>")
    abort;
}

writeOutput("before f<br>")
f()
writeOutput("after f<br>")
</cfscript>

I had some abort-confusion the other day on the CFML / Lucee Slack channel because this had behaviour I did not expect:

But that's actually correct behaviour. abort takes an optional string parameter showError, and writeOutput returns true (don't ask), which can be coerced into a string. If a statement has only one parameter, its name can be omitted, so Lucee is interpretting that as a multi-line abort statement - with a showError value of true - that doesn't end until the explicit semi-colon.

This is not the same though, as far as I can tell. The } of the function block is an explicit end-of-statement token just like a semi-colon is. Or at least it ought to be.


In other news, this sample code won't run correctly on trycf.com, which you can check out at https://trycf.com/gist/759318ea257c7ca7130c1f12c3ee72f8: Lucee doesn't work as expected whether or not the semi-colon is there; and CF only works as expected when the semi-colon is not there. This is at odds to how it works on (my) actual servers. I'd be interested in what behaviour you get on your servers?

Am still wondering a bit if I'm missing something here; especially given the different varieties of behaviour…

Righto.

--
Adam

Monday, 3 May 2021

Testing: A Horror Story

G'day:

My adventures in Lucee / CFWheels / Docker continues. This time I'm looking at what CFWheels offers by way of testing.

2021-05-04 - editorial update

In this article I am very - but I feel justifiably - harsh about RocketUnit (you need to read down a bit before you get to this). I'm also harsh towards the CFWHeels team's historical decision to include it in CFWheels. I really do think that was a poor decision. But so be it, it's done: other than mentioning it, there's no point me dwelling on it beyond that.

Some of the tone and wording of this article has be read as an indictment of CFWheel and/or its team. Beyond that initial poor decision, all my opprobrium here is directed at RocketUnit, not CFWheels, and not the team. I can see how people would think otherwise though.

To be clear: this article is largely about RocketUnit. CFWheels was just the path via which I came to know about it.

I'm going to be brief, because it's so appalling I don't want to waste too much time on it. It can be summarised thus:

Are you f***ing joking?

Firstly I'm gonna quote the CFWheels docs (Testing Your Application › The Test Framework):

Like everything else in CFWheels, the testing framework is very simple, yet powerful. You don't need to remember a hundred different functions because CFWheels' testing framework contains only a handful.

(my emphasis there)

Sorry but I see wording like this implying something about CFML developers' capabilities all the time, and it really annoys me. To me that statement is patronising as well as disingenuous. It plays on this perception that somehow CFML devs aren't capable of doing anything unless it's reeeeally easy, and that that is just how things should be. Can we please not? Can we please not normalise CFML devs as somehow not being able to cope with a small amount of complexity if it will ultimately assist them growing as developers? It's also a bit disingenuous because no testing framework requires one to learn any more than a handful of methods, and learning how to write code is our job. It suggests testing tooling is somehow usually hard (which it just isn't).

Next I read further into the docs:

Evaluation

assert(): This is the main method that you will be using when developing tests. To use, all you have to do is provide a quoted expression. The power of this is that ANY 'truthy' expression can be used.

An example test that checks that two values equal each other:

function testActualEqualsExpected() {
    actual = true;
    expected = true;
    assert("actual eq expected");
}

I'm sorry, come again? You. Give. It. A. Quoted. Expression. It's using evaluate(), isn't it? Now I'm not actually fundamentally against evaluate as a thing. Not like some of the CFML community groupthink. It has its place, just that that place is seldom "in your code": one hardly ever needs to use it. I can't wait to see why it's being used here. Other than, like, cos for some reason the assert function expects a string, so it needs to be evaluated to even work.

And "[the] power of this is that ANY 'truthy' expression can be used"? So: just like any other implementation of assert that any other testing framework ever written. Except none of those need to pass a string to the assert function. This is not a MSP of this testing framework, and the implementation we're being shown is inferior to any other testing framework I've seen. I won't look at the actual code for this just yet, as there's still more horror to see in the guidance docs first.

I'm going to wind back up the docs a bit now:

Do not var-scope [any] variables used in your tests. In order for the testing framework to access the variables within the tests that you're writing, all variables need to be within the component's variables scope.

When I first read that I was like "oh yer having a laugh, right?" and it was not until I got to the "boolean expression as a string" and "evaluate" that it suddenly made "sense". Because the string expression needs to be evaluated within the implementation of assert, of course the variables in the "expression" can't be function-local to your test: assert's code won't be able to see those. So now because the implementation has chosen to pass the test expression as a string, it's forcing us to write bad, flaky, fragile test code, with variables bleeding all over the place by design. I can't really see how this framework could ever be used seriously outside a proof of concept or other very superficial environment. It'd drive the devs batty.

And now I hasten to add, the rest of the CFWHeels docs on testing are actually pretty helpful when it focuses-away from the test framework, and back to CFWheels stuff.

So what's with this test code? What is going on with this assert function? Oh yeah, btw: assert is the only assertion this framework offers. So there's goes your elegant, expressive, easy to understand test code. It's just a bunch of vanilla assert calls now. This is a telling stark contrast every other test framework out there that has decided to implement a lot of different assertions, and sees this as a good thing. I mean obviously at the end of the day they are all asserting if something is true; if you look at the internal implementations, generally all the facade assertions end up doing just that: calling a central "assert-true" assert method. The situation-specific assertions are there to make your test code easier and simpler to understand, which is vital in testing. I cannot understand why there was a perception that it was a good thing for the framework to have only one assertion.

Right. The code. Which is all in wheels/test/functions.cfm (nice file name there, CFWheels Team :-|. And I also have yet to work out why CFWheels is designed such that all the code for its .cfc files are implemented in .cfm files: this is a question for another day).

This is interesting:

Copyright 2007 RocketBoots Pty Limited - http://www.rocketboots.com.au

OK so here's where I clock that CFWheels have just bundled this "RocketUnit" thing into the framework. OK so in defence of the CFWheels team, this is possibly just a really poor decision to include this, rather than the team actively writing this… this… thing. I note that CFWheels (in 2021) still bundles only v1.0 of RocketUnit (from something like 2007).

And here we go:

public void function assert(required string expression) {
    // other stuff snipped
    if (!Evaluate(arguments.expression)) {

And why is it doing this? This is hilarious. The way this code has been implemented, and the reason that one needs to pass a string to assert is because if the assertion fails, then the code basically picks through the string, does a primitive tokenisation effort to find dynamic expressions, and then evaluates them again to get values to return in the assertion-failed message. EG; if your expression is "x == 3" and variables.x is 4, it'll re-evaluate each element of the string so it can say something like "uh-oh x was 4 not 3". And the entire reason it needs to do this with an equality-check is it's shot itself in the foot by only having assert, instead of taking the obvious route of having an equality assertion that takes two values to compare. Neither of which need to be in a global scope; neither of which need to be re-evaluated strings, because the assertion was passed their values. It could be called, I dunno, assertEquals or something.

It actually gets better (read: "worse"). In v2.x of RocketUnit, there's no need for the quoted string any more, because what the assertion implementation does when the assertion fails is it thows a fake exception to generate a callstack, and then crawls its way up that to try to find the line of code that called assert, and extract the expression from that. Seriously, have a look:

try {
    throw(errorCode=DUMMY_ERRCODE);
} catch(any) {
    // assert is one stack frame up from this function [1], therefore [2]
    source = cfcatch.tagContext[2].codePrintPlain;
    startLineNumber = lineNumber = cfcatch.tagContext[2].line;
}

If you look further down, you can see how the implementation itself knows how flaky its own approach even is, given the various places the code that tries to extract the expression needs to bail out.

Argh!

When discussing this with another CFML community member, they sent me this:

* see footnote regarding usage of this image

I think this is something that should have set-off alarm bells when the RocketUnit project first started… evolving.


Here's how it would seem that one needs to safely write tests with this shambles:

component extends="app.tests.Test" {

    function testBasicRoutingWorks() {
        variables.response = processRequest(
            params = {
                controller = "testroute",
                action = "debug"
            },
            returnAs = "struct"
        )
        try {
            assert("variables.response.status eq 200")
            assert("variables.response.body contains 'EXPECTED_CONTENT'")
        } finally {
            structDelete(variables, "response")
        }
    }
}

One can't just leave variables-scoped variables lying around the place in test code, so you need to get rid. Test code needs to be well encapsulated and robust and not prone to interference from unexpected avenues. Or one could just have one test per CFC. Or one could painstakingly make sure that tests don't accidentally share variables. Or hey just suck it and see (I suspect this is what RocketUnit expects). It's just too easy to accidentally not reinitialise a variable and be using an uncontrolled value from another test in a subsequent one.

Do not var-scope [any] variables

I have to wonder why - at the point one realises one needed to document something that contravenes standard good coding practice that languages go out of their way to accommodate - the person didn't go "ah now lads we've f***ed this one", and rethink things.


Do me a favour if you're using CFWheels? Don't use its in-built testing framework. It's shite. Use TestBox instead. It's dead easy to use, still supported, still being actively developed on a daily basis, and makes for really nice easy to read, easy to develop test code. There's a strong and willing community out there to help you if you get stuck with anything. It'll also be a cinch to test yer CFWheels work with, as it's completely situation-agnostic.

And to the CFWheels team: please consider just ripping this out of CFWheels wholesale. This is really letting your framework down, I think.

Righto.

--
Adam

* I am not the copyright holder of this image. I'm hoping I'm covered by "fair use" here, but if you are the copyright holder and disagree: let me know, and I'll replace it with something else.

Installing and running CFWheels in my Lucee Docker container

G'day:

2021-05-05

Some of the way I've needed to configure things to get CFWheels working across the board have been tweaked/augmented from what I have in this article. A stripped down version of the steps I currently have is at "Short version: getting CFWheels working outside the context of a web-browsable directory". I can't be arsed rewriting this every time I find something new I need to tweak, but I'll keep that other article up to date.

If you've been following along in the previous articles in this series (Lucee / CFWheels / Docker), You'll know I have a set of Docker containers running Nginx, MariaDB and Lucee. I have been configuring this lot using a TDD approach, so I have a suite of tests that demonstrate everything is integrated nicely. Now it's time to install CFWheels as well, and have a look at it. I will be continuing my TDD approach to doing this work, so will list the cases I'm addressing at the heading of each step. Update, two weeks later: no I won't. This took so bloody long and was such a pain in the arse, I'm not gonna bother with that.

Overall aim: "It serves the CFWheels welcome page"

Reading ahead through the CFWheels › Installation › Test it docs, I see I should expect a welcome page when I browse to the site, all going well. So I will write a test to expect this (CFWheelsTest.cfc):

component extends=testbox.system.BaseSpec {

    function run() {
        describe("Tests CFWheels app installation", () => {
            it("serves the CFWheels welcome page after installation", () => {
                http url="http://cfml-in-docker.frontend/" result="response";

                expect(response.status_code).toBe( 200, "HTTP status code incorrect")
                expect(response.fileContent.trim()).toBe( "NOT SURE YET", "Response body incorrect")
            })
        })
    }
}

I dunno what form this welcome will take as yet, so I've just got some text that definitely won't be right, and will start with that. I'd improve this later when I know what to check for.

Attempting (spoilers: and failing) to install using CFWheels CLI

Looking at the CFWheels installation docs, I should use CommandBox to do the installation from ForgeBox, so I'll give that a go. I've already got CommandBox installed to run my tests. One thing I've noticed from reading ahead, the default location the files for CFWheels get put in a manual install is just slapped in the web root. That's… erm… "less good than it could be", and I won't be doing this. I've already gone through this with TestBox, so you can read why in that earlier article: "Adding TestBox, some tests and CFConfig into my Lucee container › It needs to only expose public stuff to the public"

I can tell CommandBox where to install things easily enough:

root@efe37f109b87:/var/www# box install id=cfwheels-cli directory=vendor
√ | Installing package [forgebox:cfwheels-cli]
root@efe37f109b87:/var/www#

And this has updated my box.json for me:

root@efe37f109b87:/var/www# cat box.json
{
    "dependencies":{
        "cfwheels-cli":"^0.4.0"
    },
    "devDependencies":{
        "testbox":"^4.2.1+400"
    },
    "installPaths":{
        "cfwheels-cli":"vendor/cfwheels-cli/",
        "testbox":"vendor/testbox/"
    },
    "testbox":{
        "runner":"http://localhost:8888/test/runTests.cfm"
    }
}
root@efe37f109b87:/var/www#

So far… so good.

The way the docs are recommending I set up my new CFWheels app is through this CLI thing I just installed, by going box wheels new. It sounds to me like this is going to assume it can do whatever it likes in its install directory, so to start with I'm just going to do the install in a temp directory and have a look at what it does, and whether it will collide with my existing work. It's nae bother if it does, I'll move my stuff around, but I just want to know what I'm getting myself into first.

root@efe37f109b87:/var/tmp/cfwheels# pwd
/var/tmp/cfwheels
root@efe37f109b87:/var/tmp/cfwheels# box wheels new
ERROR (5.2.1+00295)

Command "wheels new" cannot be resolved.

Please type "help" for assistance.

root@efe37f109b87:/var/tmp/cfwheels#

OK. Not off to a good start. I gulped and wondered whether this needs to be run from within the directory I did the install, so I switched over to /var/www/vendor/cfwheels-cli and tried there: same result. I also tried doing it from within CommandBox's special wee shell, but the results were the same.

I tried to see what help was on offer for CFWheels in general:

CommandBox> wheels help

**************************************************
* CommandBox Help for wheels
**************************************************


Command [wheels] not found.

Fine. OK, look: I don't know what's going on here; and to be completely honest, I don't care. The manual installation process is "stick the files in a directory", and I can pretty much manage that by myself (cough: don't speak too soon, Cameron). I don't think I need the Wheels CLI to do this for me. Or, as is the case here: not do it for me. De-installing that.

Installing just CFWheels

I'll still use ForgeBox to install the app though:

root@efe37f109b87:/var/www# box install id=cfwheels directory=vendor/cfwheels
√ | Installing package [forgebox:cfwheels]
root@efe37f109b87:/var/www#

One thing to note here is that I needed to specify the installation subdirectory for CFWheels too, ie: directory=vendor/cfwheels. When I installed Testbox, I only had to specify the base directory, ie: directory=vendor, and the testbox subdirectory was created for me within that. Now I don't mind whether I need to not specify the directory the installation actually goes in, or whether I don't need to specify it. But it should be one or the other. I'm not sure what's going on here.

Anyway, having done this, I see all the files in a subdirectory how one might want. Kinda:

root@efe37f109b87:/var/www# ll vendor/cfwheels/
total 88
drwxr-xr-x 16 root root 4096 Apr 25 18:35 ./
drwxr-xr-x  1 root root 4096 Apr 25 18:35 ../
-rwxr--r--  1 root root   67 Nov 22 09:57 Application.cfc*
-rwxr--r--  1 root root  975 Apr 25 18:10 box.json*
drwxr-xr-x  6 root root 4096 Apr 25 18:35 config/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 controllers/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 events/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 files/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 global/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 images/
-rwxr--r--  1 root root   52 Nov 22 09:57 index.cfm*
drwxr-xr-x  2 root root 4096 Apr 25 18:35 javascripts/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 miscellaneous/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 models/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 plugins/
-rwxr--r--  1 root root   52 Nov 22 09:57 rewrite.cfm*
-rwxr--r--  1 root root  162 Nov 22 09:29 root.cfm*
drwxr-xr-x  2 root root 4096 Apr 25 18:35 stylesheets/
drwxr-xr-x  4 root root 4096 Apr 25 18:35 tests/
drwxr-xr-x  2 root root 4096 Apr 25 18:35 views/
drwxr-xr-x 14 root root 4096 Apr 25 18:35 wheels/
root@efe37f109b87:/var/www#

Most of that stuff is stub files for my Wheels application; except for the wheels directory which is the actual CFWheels application. I expected this, because the default approach to installing this thing is - remember - "slap it all in the web root". I'll deal with this next, but first I see a problem with box.json:

root@efe37f109b87:/var/www# cat box.json
{
    "dependencies":{
        "cfwheels":"^2.2.0"
    },
    "devDependencies":{
        "testbox":"^4.2.1+400"
    },
    "installPaths":{
        "testbox":"vendor/testbox/"
    },
    "testbox":{
        "runner":"http://localhost:8888/test/runTests.cfm"
    }
}
root@efe37f109b87:/var/www#

It's not put the installPath in for CFWheels. This is no good when I come to use this for a Dockerised deployment. I will quickly manually edit this file and do a rebuild and see what happens. Back in a few min.

(Oh I also note that CommandBox has written "vendor/testbox/" in there, despite me saying just "vendor". I mean… fine…: but why did it not do the same for CFWheels?)

Cool so I did the rebuild, and having put the installation path for CFWheels into box.json manually, it all seemed to work:

root@78cb5ad3bf2c:/var/www# ll vendor
total 16
drwxr-xr-x  1 root root 4096 Apr 25 18:47 ./
drwxr-xr-x  1 root root 4096 Apr 25 18:47 ../
drwxr-xr-x 16 root root 4096 Apr 25 18:47 cfwheels/
drwxr-xr-x  9 root root 4096 Apr 25 18:47 testbox/
root@78cb5ad3bf2c:/var/www# ll vendor/cfwheels/
total 84
drwxr-xr-x 16 root root 4096 Apr 25 18:47 ./
drwxr-xr-x  1 root root 4096 Apr 25 18:47 ../
-rwxr--r--  1 root root   67 Nov 22 09:57 Application.cfc*
-rwxr--r--  1 root root  975 Apr 25 18:47 box.json*
drwxr-xr-x  6 root root 4096 Apr 25 18:47 config/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 controllers/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 events/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 files/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 global/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 images/
-rwxr--r--  1 root root   52 Nov 22 09:57 index.cfm*
drwxr-xr-x  2 root root 4096 Apr 25 18:47 javascripts/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 miscellaneous/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 models/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 plugins/
-rwxr--r--  1 root root   52 Nov 22 09:57 rewrite.cfm*
-rwxr--r--  1 root root  162 Nov 22 09:29 root.cfm*
drwxr-xr-x  2 root root 4096 Apr 25 18:47 stylesheets/
drwxr-xr-x  4 root root 4096 Apr 25 18:47 tests/
drwxr-xr-x  2 root root 4096 Apr 25 18:47 views/
drwxr-xr-x 14 root root 4096 Apr 25 18:47 wheels/
root@78cb5ad3bf2c:/var/www#

Now my challenges start.

Reorganising the file structure

I have to take that listing above, and reorganise it as follows:

adam@DESKTOP-QV1A45U:/mnt/c/src/cfml-in-docker$ tree . --dirsfirst
.
├── public
│   ├── images
│   ├── javascripts
│   ├── stylesheets
│   ├── Application.cfc
│   ├── index.cfm
│   └── rewrite.cfm
├── src
│   ├── config
│   ├── controllers
│   ├── events
│   ├── files
│   ├── global
│   ├── miscellaneous
│   ├── models
│   ├── plugins
│   ├── views
│   ├── Application.cfc
│   └── root.cfm
├── tests
│   ├── functions
│   │   └── Example.cfc
│   ├── requests
│   │   └── Example.cfc
│   └── Test.cfc
└── vendor
   └── cfwheels
       └── wheels


Getting CFWheels to work with that file structure

A couple of week has passed since I wrote the preceding paragraph. In the interim I've written a couple of articles about some side challenges I have had as I progressed this work. They weren't directly-related enough to what I'm talking about here to include them, hence I've published them separately:

Along the way I've enhanced the testing I've had to do to ensure my tinkerings worked and didn't break anything. As it currently stands I have a CFWheels-driven site "working": I have created some test endpoints that behave as I expect, and I can run its internal test suite. That's a pretty light definition of "working", I know.

I'll continue to detail the CFWheels config changes I needed to make to get everything working…

After doing the work for that "How TDD and automated testing helped me…" article above, I had a decent battery of tests that acted as a safety net for expectations I had for my site URLs:

I'm showing you the status as they are now, after I've done the work; obviously as the work was under way they weren't all a) there; b) passing. I just can't be bothered rolling my code back to show them that way, and in a way this is a more useful listing anyhow.

Oh and ignore the skipped ones. There's a bug in CFWheels that prevents those from working properly: the 404 page in dev more returns a 200-OK. I've left the tests skipped as a reminder I need to revisit them once the bug is fixed. I actually note it's been fixed (nice work!), but the version it's in ain't up on ForgeBox yet.

So… how did I get the thing to work?

Well. Firstly. How did I not get it to work. My expectations were that any well-designed third-party app would not make assumptions about where it was in the file system; or where any files implementing it happened to be. All I expected to have to do here is to set a mapping for CFWheels, and a mapping for my app / implementation code, tell my code where the CFWheels application is and then it'd just work. However I'm afraid CFWheels is not a very well-designed application, so that was a non-starter.

Core CFWheels code has been implemented so that it will only work if the wheels directory is right slap-bang in the middle of my implementation code. Primarily this line of code in wheels/events/onapplicationstart.cfm:

$include(template = "config/settings.cfm");

And that weirdo (name and implementation) $include function does this to it:

<cfinclude template="../../#LCase(arguments.template)#">

Neat. Let's dispense with our carefully-cased file names, cos we'll enforce lower-case on them all. That's definitely the job of a function called "include". Sorry "$include". And we'll just force the all to be two driectories up from wheels/global because isn't that where everybody puts their files?

I'm going to need to put that wheels directory into the middle of my source-controlled application code files, are't I? Sigh. I'm not checking it in to source control though: that would be too far beyond the pale. I'm installing it to where it belongs: vendor, and then getting Docker to copy it into my source-controlled directory once it's there:

RUN git clone git@github.com:adamcameron/cfml-in-docker.git .

# irrelevant stuff snipped

RUN mkdir -p vendor
RUN box install

RUN cp -R /var/www/vendor/cfwheels/wheels /var/www/src/wheels

I'm also .gitignore-ing /src/wheels/ so it doesn't accidentally end up in source control.

I know this is a really jerry-built, but… well: my hands are kind of tied.

The next step can be sorted out via config settings CFWheels exposes via config/settings.cfm:

srcPrefix = "../src"

set(eventPath = "#srcPrefix#/events")
set(filePath = "#srcPrefix#/files")
set(modelPath = "#srcPrefix#/models")
set(modelComponentPath = "#srcPrefix#/models")
set(pluginPath = "#srcPrefix#/plugins")
set(pluginComponentPath = "#srcPrefix#/plugins")
set(viewPath = "#srcPrefix#/views")
set(controllerPath = "#srcPrefix#/controllers")

set(webPath = "")
set(imagePath = "images")
set(javascriptPath = "javascripts")
set(stylesheetPath = "stylesheets")

set(wheelsComponentPath = "cfmlInDocker.wheels")

This is almost OK. Except for the headless-function use of set, which appears in here as if by magic; and for some unfathomable reason that it's implemented in a .CFM file not in a method of maybe a Settings.cfc file or something a bit more OOP. The fact that this .CFM file is ultimately included into a .CFC file makes the situation worse, not better, IMO. This is endemic in the application design of CFWheels btw.

Getting the testing framework to work

The other thing I needed to do is to add some mappings so the inbuilt CFWheels testing "framework" can actually run tests. This is all web-driven, btw. These had to go into src/Application.cfc

thisDirectory = getDirectoryFromPath(getCurrentTemplatePath())
this.mappings["/public/wheels"] =  getCanonicalPath("#thisDirectory#wheels")
this.mappings["/public/tests"] = getCanonicalPath("#thisDirectory#../tests")
this.mappings["/app/tests"] = getCanonicalPath("#thisDirectory#../tests")

You might as why I need to put these into my application's Application.cfc, and I couldn't put them in a tests/Application.cfc that happened to extend the app's one. Yes, this was my expectation as well: this crazy notion of OOP and separation of concerns. However CFWheels apparently runs its tests via a controller in your web app, then there isn't really a tests/Application.cfc that I can leverage.

After that the UI in the test framework was still breaking, creating links incorrectly. This ended up being because I needed to tell CFWheels that I am rewriting URLs. Not sure why, but there you go (Tom King set me straight on this, on the CFWheels subchannel of the CFML Slack channel: he is a star and has been a big help to me in all of this). The settings are also in config/settings.cfm:

set(URLRewriting="On")
set(rewriteFile="index.cfm")

And after doing that lot: all my TestBox tests were green! And there's even a test in there that tests that CFWheels' tests are working, which they are. I say "all of them", that just means the coupla stub tests they include as examples, and two wee tests I stuck in as proof-of-working. I'm not doing anything major in there. But it's green:

I'll not go into the implementation of those tests here. I'll look at that particular horror story in a separate article.

And the site

Tests are one thing, but I better demonstrate to you that the CFWheels-driven site is up and running too:



Conclusion

Detailing how to get this working took a lot less time to write (and for you to read) than it did for me to work out. It was not helped by my various Nginx-config-shortcomings; but I did have to dive into CFWheels's code more than I would have liked, and that was an exercise in frustration in itself as the way it's been implemented is… "obstructive". As I mentioned above Tom King patiently answered a bunch of questions for me, which was also a big help.

However here I am. I've got my app working. It would not surprise me if there were some other config tweaks I need to make down the track, but I can start doing some exercises of adding some of me own routes, controller, models, views in here now. I've worked through the CFWHeels tutorials and a bunch of their screencasts, and it seems easy enough to throw simple things together. I have my suspicions that as soon as requirements are no longer simplistic then the way CFWheels is designed / implemented might start getting in its own way, but I'll give it the benefit of the doubt for now.

Righto.

--
Adam

 

 

 

 

But wait! There's more!

Just after pressing "Publish" on this, I foolishly clicked about on the UI of the site home page. And I clicked on "Docs":

And the thanks I got for doing that was this:

I looked through the CFWheels code to see what it was ass-u-me`ing, and it's here in wheels/functions.cfm:

this.wheels.rootPath = GetDirectoryFromPath(GetBaseTemplatePath());

// ...

// Add mapping to the root of the site (e.g. C:\inetpub\wwwroot\, C:\inetpub\wwwroot\appfolder\).
// This is useful when extending controllers and models in parent folders (e.g. extends="app.controllers.Controller").
this.mappings["/app"] = this.wheels.rootPath;

And in this case, GetBaseTemplatePath() is the index.cfm file in the public, not the app root. So that won't do. I thought this might be a show stopper because I can't override that because of the way CFWheels is architected. However all was not lost because as far as I can tell that base /app mapping is never used by itself, it's only ever used as the base for app.controllers, app.models, and app.wheels. I can just make the correct mappings to those, in my Application.cfc. But first a quick test:

it("serves the Docs page", () => {
    http url="http://cfml-in-docker.frontend/wheels/docs" result="response";

    expect(response.status_code).toBe(200, "HTTP status code incorrect")
    expect(response.fileContent).toInclude("<title>Docs | CFWheels</title>")
})

Now we can fix it:

thisDirectory = getDirectoryFromPath(getCurrentTemplatePath())
this.mappings["/public/wheels"] =  getCanonicalPath("#thisDirectory#wheels")
this.mappings["/public/tests"] = getCanonicalPath("#thisDirectory#../tests")
this.mappings["/app/tests"] = getCanonicalPath("#thisDirectory#../tests")

this.mappings["/app/controllers"] = getCanonicalPath("#thisDirectory#controllers")
this.mappings["/app/models"] = getCanonicalPath("#thisDirectory#models")
this.mappings["/app/wheels"] = getCanonicalPath("#thisDirectory#wheels")

And that sorted it. I wonder what the next glitch is gonna be. But I'm gonna stop looking. For now…

RightoAgain.

--
AdamAgain

Sunday, 2 May 2021

CFML: pseudo-constructor polymorphic inheritance expectations management

G'day:

Well there's a sequence of words I never expected to write down.

I just ran across something that I was reasonably surprised by when I first saw it. But having looked at it some more, I'm not sure. So I thought I'd ask the 3-4 people who actually read this blog to offer their insight.

Consider this traditional example of polymorphism in play:

// Base.cfc
component {

    function runMe(){
        doThings()
    }

    function doThings(){
        writeOutput("BaseApp doThings called")
    }
}

// Sub.cfc
component extends=Base {

    public function doThings(){
        super.doThings()
        writeOutput("SubApp doThings called")
    }
}

// from StandardInheritanceTest.cfc
it("A subclass will override a base class method", () => {
    o = new Sub()
    o.runMe()

    expect(o.stack).toBe([
        "Base doThings called",
        "Sub doThings called"
    ])
})

The test (which passes) confirms what we'd expect: when runMe calls doThings, Because it's being called on a Sub object, the reference to doThings is referring to Sub.doThings even though the call is in Base. Hopefully no surprises there.

But what about this example:

// Base.cfc
component {

    this.stack = []

    doThings()

    function doThings(){
        this.stack.append("Base doThings called")
    }
}

// Sub.cfc
component extends=Base {

    function doThings(){
        super.doThings()
        this.stack.append("Sub doThings called")
    }
}

// from PseudoConstructorInheritanceTest.cfc
it("A subclass will override a base class method", () => {
    o = new Sub()

    expect(o.stack).toBe([
        "Base doThings called",
        "Sub doThings called"
    ])
})

The difference here is the call to doThings is not done by the test, it's done within the pseudo-constructor of the Base component.

And in this case the test fails:

Expected [[Base doThings called, Sub doThings called]] but received [[Base doThings called]]

It would seem the pseudo-constructor code of a base-class is not aware it's being called from a sub-class. This doesn't seem right to me?

I'm running this code on Lucee, but I ran equivalent code on ColdFusion and the results were the same (so I guess that's something). And given the behaviour is the same on both I'm thinking this is more me not understanding something, rather than a bug. What do you think?

BTW whilst testing this I found out I can get the behaviour I actually want with a slight tweak to Sub.cfc. I changed from this:

component extends=Base {

    function doThings(){
        super.doThings()
        this.stack.append("Sub doThings called")
    }
}

To this:

component extends=Base {

    doThings()

    function doThings(){
        this.stack.append"Sub doThings called")
    }
}

IE: I replicate the way doThings is called in Base.cfc: from the pseudo-constructor. Both calls are made. For my purposes this will work fine. But I still do find it curious.

Or am I being daft?

Righto.

--
Adam

How TDD and automated testing helped me solve an Nginx config problem I had created for myself

G'day:

I have a "website" I'm building on as part of a series of articles I'm writing about Lucee / CFWheels / Docker. I have a Docker container running Nginx which proxies requests for CFML code to a Docker container running Lucee.

Due to the nature of web applications, I have my web-accessible assets - JS, CSS, image and "entry point" index.cfm and Application.cfc files - in a public directory off my app root; and adjacent to that I have a src directory for my code, and vendor directory for third-party code (like stuff I install from ForgeBox via CommandBox).

This /public abstraction needs to be hidden from the end user: they're going to want to be browsing to - for example - http://example.com/, not http://example.com/public. So this means the proxy from Nginx to Lucee also needs to deal with that. It is imperative that no CFML files are publicly exposed other than the aforementioned index.cfm and Application.cfc

I'm terrible at configuring Nginx, and mak a lot of mistakes. Some mistakes are obvious because Nginx flat-out refuses to start. Others are less obvious, and bleed out as "unexpected behaviour" later in the piece. Knowing this, I approached the exercise in a TDD fashion; identifying what cases need addressing and writing tests for them, and then doing the config work to make each pass. Note: my wording is ambiguous there: I did not write more than one test at a time. I wrote a test, then got the config to make that test pass. Then I wrote the next test and reconfigured to make that pass (whilst also keeping the earlier ones passing too). I've already detailed some of this in my earleir article "Adding TestBox, some tests and CFConfig into my Lucee container".

As a result of these efforts to get Nginx working how I expected it to, I ended up with these green tests:

(Ignore how I'm skipping some of these tests. This is down to a bug in CFWheels that doesn't report 404 situations with an actual 404 status code when in dev mode. This does demonstrate though how I identified a shortcoming in behaviour whilst TDDing the work, that said).

And that's great. At every step as I was configuring more and more of Nginx's handling of requests, I could see that any change I made had a) addressed the new requirement; b) not broken any previous requirement. And it was indeed the reality at times that something I tried fixed one issue, but broke something else.

Back to those tests I'm skipping. This showed to me that I was short some cases: CFWheels handles things differently from how I expect it to in some situations, so I figured I had better have two flavours of test: one set for proxying to non-CFWheel-handled URLs; another set for when the URLs are within CFWheels domain. And I'm glad I did make this call, because it showed-up a bug in my Nginx config:

And when I checked those URLs, I saw the problem:

expectedParamValue = "expectedValue"
// ... rest of test not relevant here
expect(response.fileContent).toInclude(
    "Expected query param value: [#expectedParamValue#]",
    "Query parameter value was incorrect (URL: #testUrl#)"
)

But what I was seeing at that URL was this:

Expected query param value: [expectedValue?testParam=expectedValue]

The query string part (including the ?) was being appended to the URL sent to Lucee twice (same problem for both those failing tests). I was pretty puzzled how my previous non-Wheels test was passing, and it still seemed legit. Bemusing. However I was actively appending the query string in my proxy_pass URL, so that was "clearly wrong":

proxy_pass  http://cfml-in-docker.lucee:8888/public$fastcgi_script_name$is_args$args;

I got rid of that, and figured "I had better go and check why those other tests are passing after I re-run the tests here, to check that change:


Dammit. Now the CFWheels-specific tests are passing, but the non-Wheels ones are failing. Time for me to RTFM, cos I'm clearly doing something wrong here.

The first thing I'm going wrong is here:

proxy_pass  http://cfml-in-docker.lucee:8888/public$fastcgi_script_name;

$fastcgi_script_name is a PHP thing, and whilst it coincidentally holds the URI I want, it's the wrong thing to have here. So I put $request_uri back in there.

Right and that broke all the path_info CFWheels tests, so wasn't right. I decided to read more closely. It turns out that request_uri is the whole original URI (including the path_info and query string), and thus is ignoring my rewrites, and it was the reason that the query params were getting doubled up. In my rewrite I had this:

location @rewrite {
    rewrite ^/(.*)? /index.cfm$request_uri last;
    rewrite ^ /index.cfm last;
    return 404;
}

I just wanted $uri, which is just the document URI part of the requested URI, and it also reflects any changes made to that URI during rewrites and what-have-you. So once I used that in my rewrite and for my proxy_pass URL, the tests now look better:

I've abbreviated how long it took me to work this out, and how many cycles of trial end error it too me. Having those automated tests in place were gold because after each iteration I knew how wrong I was - for all cases - in half a second. I didn't need to manually go "OK, did it work for this?" "did it break this other one?" etc, for 20-odd tests.

It was also a big help to me to take the TDD path here, and stop and think & reason about exactly what my expectations ought to be for each of the cases I had. It also lead me to add more cases, such as the combinations of "it has both path_info and query parameters", as well as realising the path through the Nginx config was different for URLs aimed at Wheels (which are completely rewritten), and the ones directly to files in the public directory. I could easily cover both cases by duplicating the tests and changing the URLs sligthly.

Things seem to be working now, but if I find something else wrong, I will first work out what my expectations of it to be right are, and write a quick test for it. Then I'll fix it (without breaking anything else).

For now though: I'm fed-up with Nginx & CFML & CFWheel and I'm gonna do something els for a while. But I'll be back to it later this afternoon: I'm wll behind wehre I want to be with this stuff, and using the bank-holiday weekend to catch up a bit.

The "final" state of my Nginx site config is (docker/nginx/sites/default.conf):

server {
    listen 80;
    listen [::]:80;

    #rewrite_log on;

    server_name cfml-in-docker.frontend;
    root /usr/share/nginx/html;
    index index.html index.cfm;

    resolver 127.0.0.11;

    location / {
        try_files $uri $uri/ @rewrite;
    }

    location @rewrite {
        rewrite ^/(.*)? /index.cfm$uri last;
        rewrite ^ /index.cfm last;
        return 404;
    }

    location ~ \.(?:cfm|cfc)\b {
        proxy_http_version  1.1;
        proxy_set_header    Connection "";
        proxy_set_header    Host                $host;
        proxy_set_header    X-Forwarded-Host    $host;
        proxy_set_header    X-Forwarded-Server  $host;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;     ## CGI.REMOTE_ADDR
        proxy_set_header    X-Forwarded-Proto   $scheme;                        ## CGI.SERVER_PORT_SECURE
        proxy_set_header    X-Real-IP           $remote_addr;
        expires             epoch;

        proxy_pass  http://cfml-in-docker.lucee:8888/public$uri$is_args$args;
    }

    location ~ /\.ht {
        deny all;
    }
}

Righto, where's me shooting game?

--
Adam