G'day:
On Fri/Sat (it's currently Sunday evening, but I'll likely not finish this until Monday now) I started looking at getting some CFML stuff running on Lucee in a Docker container. If you like you can read about that stuff: "Using Docker to strum up an Nginx website serving CFML via Lucee" and "Repro for Lucee weirdness". This article resumes from where I got to with the former one, so that one might be good for some context.
Full disclosure: I spent all today messing around in a spike: experimenting with stuff, and now am finally happy I have got something to report back on, so I have rolled-back the spike and am going to do the "production" version of it via TDD again. I just say this - and it's not the first time - if yer doing TDD it's OK to spike-out and do a bunch of stuff to work out how to do things without testing every step. Especially if yer like me and start from a position of having NFI what you need to do. However once yer done: revert everything and start again, testing-first as you go. What I've done here is save all my stuff in a branch, and now I'm looking at a diff of that and main, as a reference to what I actually need to do, and what is fluff that represents a dead end, or something I didn't need to do anyhow, or whatever.
It needs to only expose public stuff to the public
As per the article subject line, today I'm gonna install a bit more tooling and get me in a better position to do some dev work. The first thing I noticed is that as things stand, the Nginx wesbite is serving everything in the Lucee application root (/var/www), whereas that directory is going to be a home directory for the application code, test code and third-party apps, so I don't want that browsable. I'm going to shift things around a bit. A notional directory structure would be along these lines:
├── public
│ ├── css
│ ├── images
│ ├── js
│ ├── Application.cfc
│ ├── favicon.ico
│ └── index.cfm
├── src
│ └── Application.cfc
├── test
└── vendor
├── cfwheels
└── testbox
I've taken a fairly Composer / PHP approach there, but I could equally follow a more Java-centric approach to the directory structure:
├── com
│ └── ortussolutions
│ └── testBox
├── me
│ └── adamcameron
│ └── cfmlInDocker
│ ├── src
│ │ └── Application.cfc
│ └── test
├── org
│ └── cfwheels
│ └── cfwheels
└── public
├── css
├── images
├── js
├── Application.cfc
├── favicon.ico
└── index.cfm
The point being: the website directory and my code and other people's code should be kept well away from one another. That's just common-sense.
Anyway, back to the point. Whichever way I organise the rest of things, only stuff that is supposed to be browsed-to should be browsable. Everything else should not be. So I'm gonna move the website's docroot, as well as the files that need to be served. This is just a "refactoring" exercise, so no tests should change here. We just want to make sure they still all pass.
This just means some changes to docker-compose.yml:
lucee:
build:
context: ./lucee
volumes:
- ../public:/var/www
- ../public:/var/www/public
- ../root/src:/var/www/src
- ../test:/var/www/test
- ../var/log/tomcat:/usr/local/tomcat/log
- ../var/log/lucee:/opt/lucee/web/logs
- ./lucee/root_home:/root
And the website config (docker/nginx/sites/default.conf):
location ~ \.(?:cfm|cfc) {
# ...
proxy_pass http://cfml-in-docker.lucee:8888$fastcgi_script_name$is_args$args;
proxy_pass http://cfml-in-docker.lucee:8888/public$fastcgi_script_name$is_args$args;
}
Once a rebuild the containers, I get two failing tests. Oops: I did not expect that. What's going on? Checking the front-end, the public-facing website still behaves the same. So… erm …?
One thing that didn't occur to me when doing this change is that a couple of the tests are hitting the internal Lucee website (reminder: the public-facing website for me is http://cfml-in-docker.frontend/ and the internal Lucee web site is http://cfml-in-docker.lucee:8888/). And that internal website still points to /var/www/, so where previously I'd access http://cfml-in-docker.lucee:8888/remoteAddrTest.cfm, now the URL for the backend site is be http://cfml-in-docker.lucee:8888/public/remoteAddrTest.cfm. This is by design (kinda, just… for now), but I forgot about this when I made the change.
This means to me that my change is not simply a refactoring: therefore I need to start with a failing tests. I roll back my config changes, fix the tests so they hit http://cfml-in-docker.lucee:8888/public/gdayWorld.cfm and http://cfml-in-docker.lucee:8888//public/remoteAddrTest.cfm respectively, and watch them fail. Good. Now I roll forward my config changes again and see the tests pass: cool. Job done.
Later when I'm reconfiguring things I might remap it to /var/www/public, if I can work out how to do that without hacking Tomcat config files too much. But remember the test case here: It needs to only expose public stuff to the public. And we've achieved that. Let's not worry about a test case we don't need to address for now. Moving on…
It can run tests with TestBox
Currently I am running my tests via a separate container running PHP and PHPUnit. This has been curling the website to test Nginx and Lucee behaviour. Now that I have Lucee working, I can shift the tests to TestBox, which is - as far as I know - the current state of the art when it comes to testing CFML code. It provides both xUnit and Jasmine-style testing syntax.
The test for this is going to be a "physician heal thyself" kind of affair. I'm going to write a TestBox test. Once I can run it and it doesn't just go splat: job done. The test is simply this:
component extends=testbox.system.BaseSpec {
function run() {
describe("Baseline TestBox tests", () => {
it("can verify that true is, indeed, true", () => {
expect(true).toBe(true)
})
})
}
}
Testbox seems to be web-based. I'd much prefer just running my tests from the shell like I would any other testing framework I've used in the last 7-8 years, but CFML has much more "it's aimed at websites, so evertything is implemented as a website" mentality (hammers and nails spring to mind here). So be it I guess. I do see that TestBox does have the ability to integrate with Ant, but that seems to be via an HTTP request as well. Hrm. What I know is I can't simply do something like testbox /path/to/my/tests or similar. What I do need to do is write a small runner file (runTests.cfm), which I then browse to:
<cfscript>
testBox = new testbox.system.TestBox(directory="cfmlInDocker.test")
result = testBox.run(
reporter = "testbox.system.reports.SimpleReporter"
)
writeOutput(result)
</cfscript>
To use that testbox.system and cfmlInDocker.test paths, I need to define mappings for them at application level (ie: not in the same file that uses it, but a different unrelated file, Application.cfc):
component {
this.mappings = {
"/cfmlInDocker/test" = expandPath("/test"),
"/testbox" = expandPath("/vendor/testbox")
}
}
And when I browse to that, I get a predictable error:
Let's call that our "failing test".
OK so right, we install TestBox from ForgeBox (think packagist.org or npmjs.com). And to install stuff from ForgeBox I need CommandBox. And that is pretty straight forward; just a change to my Lucee Dockerfile:
FROM lucee/lucee:5.3
RUN apt-get update
RUN apt-get install vim --yes
COPY ./root_home/.bashrc /root/.bashrc
COPY ./root_home/.vimrc /root/.vimrc
WORKDIR /var/www
RUN curl -fsSl https://downloads.ortussolutions.com/debs/gpg | apt-key add -
RUN echo "deb https://downloads.ortussolutions.com/debs/noarch /" | tee -a /etc/apt/sources.list.d/commandbox.list
RUN apt-get update && apt-get install apt-transport-https commandbox --yes
RUN echo exit | box
EXPOSE 8888
That last step there is because CommandBox needs to configure itself before it works, so I might as well do that when it's first installed.
Once I rebuild the container with that change, we can get CommandBox to install TestBox for us:
√ | Installing package [forgebox:testbox]
| √ | Installing package [forgebox:cbstreams@^1.5.0]
| √ | Installing package [forgebox:mockdatacfc@^3.3.0+22]
root@b73f0836b708:/var/www#
root@b73f0836b708:/var/www#
root@b73f0836b708:/var/www# ll vendor/
total 12
drwxr-xr-x 3 root root 4096 Apr 19 09:23 ./
drwxr-xr-x 1 root root 4096 Apr 19 09:23 ../
drwxr-xr-x 9 root root 4096 Apr 19 09:23 testbox/
root@b73f0836b708:/var/www#
Note that commandbox is glacially slow to do anything, so be patient rather than be like me going "WTH is going on here?" Check this out:
**************************************************
* CommandBox Help
**************************************************
Here is a list of commands in this namespace:
// help stuff elided…
To get further help on any of the items above, type "help command name".
real 0m48.508s
user 0m10.298s
sys 0m2.448s
root@b73f0836b708:/var/www#
48 bloody seconds?!?!. Now… fine. I'm doing this inside a Docker container. But even still. Blimey fellas. This is the equivalent for composer:
Usage:
help [options] [--] [<command_name>]
// help stuff elided…
To display the list of available commands, please use the list command.
real 0m0.223s
user 0m0.053s
sys 0m0.035s
root@b21019120bca:/usr/share/cfml-in-docker#
That is more what I'd expect. I suspect they are strumming up a CFML server inside the box application to execute CFML code to do the processing. Again: hammer and nails eh? But anyway, it's not such a big deal. The important thing is: did it work?
Yes it bloody did! Cool! Worth the wait, I say.
I still need to find a way to run it from the shell, and I also need to work out how to integrate it into my IDE, but I have a minimum baseline of being able to run tests now, so that is cool.
The installation process also generated a box.json file, which is the equivalent of a composer.json / packages.json file:
{
"devDependencies":{
"testbox":"^4.2.1+400"
},
"installPaths":{
"testbox":"vendor/testbox/"
}
}
It doesn't seem to have the corresponding lock file though, so I'm wondering how deployment works. The .json dictates what could be installed (eg: for testbox it's stating it could be anything above 4.2.1+400 but less than 5.0), but there's nothing controlling what is installed. EG: specifically 4.2.1+400. If I run this process tomorrow, I might get 4.3 instead. It doesn't matter so much with dev dependencies, but for production dependencies, one wants to make sure that whatever version is being used on one box will also be what gets installed on another box. Which is why one needs some lock-file concept. The Composer docs explain this better than I have been (and NPM works the same way). Anyway, it's fine for now.
Now that I have the box.json file, I can simply run box install in my Dockerfile:
# …
WORKDIR /var/www
RUN curl -fsSl https://downloads.ortussolutions.com/debs/gpg | apt-key add -
RUN echo "deb https://downloads.ortussolutions.com/debs/noarch /" | tee -a /etc/apt/sources.list.d/commandbox.list
RUN apt-get update && apt-get install apt-transport-https commandbox --yes
RUN echo exit | box
COPY ./box.json /var/www/box.json
RUN mkdir -p /var/www/vendor
RUN box install
EXPOSE 8888
It runs all the same tests via TestBox as it does via PHPUnit
I'm not going to do some fancy test that actually tests that my tests match some other tests (I'm not that retentive about TDD!). I'm just going to implement the same tests I've already got on PHPUnit in TestBox. Just as some practise at TestBox really. I've used it in the past, but I've forgotten almost everything I used to know about it.
…
Actually that was pretty painless. I'm glad I took the time to properly document CFScript syntax a few years ago, as I'd forgotten how Railo/Lucee handled tags-in-script, and the actual Lucee docs weren't revealing this to me very quickly. That was the only hitch I had along the way.
All the tests are variations on the same theme, so I'll just repeat one of the CFCs here (NginxProxyToLuceeTest.cfc):
component extends=testbox.system.BaseSpec {
function run() {
describe("Tests Nginx proxies CFML requests to Lucee", () => {
it("proxies a CFM request to Lucee", () => {
http url="http://cfml-in-docker.frontend/gdayWorld.cfm" result="response";
expect(response.status_code).toBe( 200, "HTTP status code incorrect")
expect(response.fileContent.trim()).toBe( "G'day world!", "Response body incorrect")
})
it("passes query values to Lucee", () => {
http url="http://cfml-in-docker.frontend/queryTest.cfm?testParam=expectedValue" result="response";
expect(response.status_code).toBe( 200, "HTTP status code incorrect")
expect(response.fileContent.trim()).toBe( "expectedValue", "Query parameter value was incorrect")
})
it("passes the upstream remote address to Lucee", () => {
http url="http://cfml-in-docker.lucee:8888/public/remoteAddrTest.cfm" result="response";
expectedRemoteAddr = response.fileContent
http url="http://cfml-in-docker.lucee:8888/public/remoteAddrTest.cfm" result="testResponse";
actualRemoteAddr = testResponse.fileContent
expect(actualRemoteAddr).toBe(expectedRemoteAddr, "Remote address was incorrect")
})
})
}
}
The syntax for making an http request uses that weirdo syntax that is neither fish nor fowl (and accordingly confuses Lucee itself as to what's a statement and what isn't, hence needing the semi-colon), but other than that it's all quite tidy.
And evidence of them all running:
I can get rid of the PHP container now!
It uses CFConfig to make some Lucee config tweaks
An observant CFMLer will notice that I did not var my variables in the code above. To non-CFMLers: one generally needs to actively declare a variable as local to the function its in (var myLocalVariable = "something"), otherwise without that var keyword it's global to the object it's in. This was an historically poor design decision by Macromedia, but we're stuck with it now. Kinda. Lucee has a setting such that the var is optional. And I've switched this setting on for this code.
Traditionally settings like this need to be managed through the Lucee Administrator GUI, but I don't wanna have to horse around with that: it's a daft way of setting config. There's no easy out-of-the-box way of making config changes like this outside the GUI, but there's a tool CFConfig that let's me do it with "code". Aside: why is this not called ConfigBox?
Before I do the implementation, I can actually test for this:
component extends=testbox.system.BaseSpec {
function run() {
describe("Tests Lucee's config has been tweaked'", () => {
it("has 'Local scope mode' set to 'modern'", () => {
testVariable = "value"
expect(variables).notToHaveKey("testVariable", "testVariable should not be set in variables scope")
expect(local).toHaveKey("testVariable", "testVariable should be set in local scope")
})
})
}
}
Out of the box that first expectation will fail. Let's fix that.
Installing CFConfig is done via CommandBox/Forgebox, and I can do that within the Dockerfile:
RUN box install commandbox-cfconfig
Then I can make that setting change, thus:
RUN box cfconfig set localScopeMode=modern to=/opt/lucee/web
I'm going to do one more tweak whilst I'm here. The Admin UI requires a coupla passwords to be set, and by default one needs to do the initial setting via putting it in a file on the server and importing it. Dunno what that's all about, but I'm not having a bar of it. We can sort this out with the Dockerfile and CFConfig too:
FROM lucee/lucee:5.3
ARG LUCEE_PASSWORD
# a bunch of stuff elided for brevity…
RUN box install commandbox-cfconfig
RUN box cfconfig set localScopeMode=modern to=/opt/lucee/web
RUN box cfconfig set adminPassword=${LUCEE_PASSWORD} to=/opt/lucee/web # for web admin
RUN echo ${LUCEE_PASSWORD} > /opt/lucee/server/lucee-server/context/password.txt # for server admin (actually seems to deal with both, now that I check)
EXPOSE 8888
That argument is passed by docker-compose, via docker-compose.yml:
lucee:
build:
context: ./lucee
args:
- LUCEE_PASSWORD=${LUCEE_PASSWORD}
And that in turn is passed-in via the shell when the containers are built:
I'd rather use CFConfig for both the passwords, but I could not find a setting to set the server admin one. I'll ask the CFConfig bods. I did find a setting to just disable the login completely (adminLoginRequired), but I suspect that setting is only for ColdFusion, not Lucee. It didn't work for me on Lucee anyhow.
It has written enough for today
I was gonna try to tackle getting CFWheels installed and running in this exercise too, but this article is already long enough and this seems like a good place to pause. Plus I've just spotted someone being wrong on the internet, and I need to go interject there first.
Righto.
--
Adam
