G'day
This will be a bit of a scrappy article just summarising some changes to my project environment since the last article in this series on Lucee / CFWheels / Docker; "Adding TestBox, some tests and CFConfig into my Lucee container". By the end of that article I'd got Nginx proxying calls to Lucee, and some tests to verify its integrity and my expectations of how it ought to be working. I'm about to continue with an article about getting CFWheels to work (URL TBC), but before that - and for the ake of full disclosure - I'll detail these wee changes I've made.
It can connect Lucee to a MariaDB database and fetch records
The test summarises the aim here. /test/integration/TestDatabaseConnection.cfc:
component extends=testbox.system.BaseSpec {
function run() {
describe("Tests we can connect to the database", () => {
it("can retrieve test records", () => {
expectedRecords = queryNew("id,value", "int,varchar", [
[101, "Test row 1"],
[102, "Test row 2"]
])
actualRecords = queryExecute("SELECT id, value FROM test ORDER BY id")
expect(actualRecords).toBe(expectedRecords)
})
})
}
}
Note that this filed under test/integration because it's testing the integration between Lucee and the DB, rather than any business logic.
I've aded some config to the test suite's Application.cfc too:
component {
this.mappings = {
"/cfmlInDocker/test" = expandPath("/test"),
"/testbox" = expandPath("/vendor/testbox")
}
this.localmode = "modern"
this.datasources["cfmlInDocker"] = {
type = "mysql",
host = "database.backend",
port = 3306,
database = "cfmlindocker",
username = "cfmlindocker",
password = server.system.environment.MYSQL_PASSWORD,
custom = {
useUnicode = true,
characterEncoding = "UTF-8"
}
}
this.datasource = "cfmlInDocker"
}
One key thing to note here is that I am setting this.localmode in here. Previous I was setting this in Lucee's global config via CFConfig, but Zac Spitzer dropped me a line and pointed out it could be set at runtime in Application.cfc. This is a much more elegant approach, so I'm running with it.
Other than that I'm setting a data source. Note I'm picking up the password from the environment, not hard-coding it. This is passed by the docker-compose.yml file:
lucee:
build:
context: ./lucee
args:
- LUCEE_PASSWORD=${LUCEE_PASSWORD}
environment:
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
For the implementation of this requirement I've added a Docker container for MariaDB, added a test table into it and tested that Lucee can read data from it. This was all straight forward. Here are the file changes:
/docker/mariadb/Dockerfile:
FROM mariadb:latest
COPY ./docker-entrypoint-initdb.d/ /docker-entrypoint-initdb.d/
COPY ./conf/logging.cnf /etc/mysql/conf.d/logging.cnf
RUN chmod -R 644 /etc/mysql/conf.d/logging.cnf
CMD ["mysqld"]
EXPOSE 3306
Nothing mysterious there. I'm using the entrypoint to create the DB table and populate it (docker-entrypoint-initdb.d/1.createAndPopulateTestTable.sql):
USE cfmlindocker;
CREATE TABLE test (
id INT NOT NULL,
value VARCHAR(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB;
INSERT INTO test (id, value)
VALUES
(101, 'Test row 1'),
(102, 'Test row 2')
;
ALTER TABLE test MODIFY COLUMN id INT auto_increment;
I'm also moving logging to a different directory so I can see them on my host machine (via conf/logging.cnf):
[mysqld]
log_error = /var/log/mariadb/error.log
This is all wired-together in docker-compose.yml
mariadb:
build:
context: ./mariadb
environment:
- MYSQL_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
- MYSQL_DATABASE=${MYSQL_DATABASE}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
ports:
- "3306:3306"
volumes:
- mysqlData:/var/lib/mariadb
- ./mariadb/root_home:/root
- ../var/log:/var/log
stdin_open: true
tty: true
networks:
backend:
aliases:
- database.backend
volumes:
mysqlData:
Note that I am sticking the DB data into a Docker volume instead of in a volume from my host machine. This means I need to take some care if I ever get around to adding non-test data into it, but for the time being it saves cluttering up my host machine with DB files, plus it's easier during initial configuration to completely reset the DB. It's easy enough to change later on when I need to.
I'm setting some of those magic environment variable in .env:
COMPOSE_PROJECT_NAME=cfml-in-docker
MYSQL_DATABASE=cfmlindocker
MYSQL_USER=cfmlindocker
# the following are to be provided to `docker-compose up`
LUCEE_PASSWORD=
DATABASE_ROOT_PASSWORD=
MYSQL_PASSWORD=
And the passwords when I build the containers:
adam@DESKTOP-QV1A45U:/mnt/c/src/cfml-in-docker/docker$ DATABASE_ROOT_PASSWORD=123 MYSQL_PASSWORD=1234 LUCEE_PASSWORD=12345 docker-compose up --build --detach --force-recreate
It got rid of CFConfig
Both the Lucee settings I needed to change with CFConfig before hand can be done natively with Lucee, so I didn't need CFConfig any more. I might need it again later, in which case I will re-install it. But for now it's dead-weight.
RUN box install commandbox-cfconfig
RUN box cfconfig set localScopeMode=modern to=/opt/lucee/web
RUN box cfconfig set adminPassword=${LUCEE_PASSWORD} to=/opt/lucee/web
RUN echo ${LUCEE_PASSWORD} > /opt/lucee/server/lucee-server/context/password.txt # this handles the passwords for both server and web admins
It can run the tests from the shell
Running TestBox's tests in a browser is all very pretty, but not very practical. Fortunately I read the TestBox docs some more and found out how to run them from the shell. They show how to run it from within CommandBox's own special shell here in "TestBox integration › Test runner", but that's weird and no help to me. However I finally twigged that it seems that whatever one might do within the special shell, one can also call from the normal shell via the box command. All I needed to do to enable this was to tell CommandBox how to run the tests in docker/lucee/box.json, which is used by CommandBox in the docker/lucee/Dockerfile:
{
"devDependencies":{
"testbox":"^4.2.1+400"
},
"installPaths":{
"testbox":"vendor/testbox/"
},
"testbox":{
"runner":"http://localhost:8888/test/runTests.cfm"
}
}
COPY ./box.json /var/www/box.json
RUN mkdir -p /var/www/vendor
RUN box install
This has the benefit that the test run doesn't simply return a 200-OK all the time whether tests all passed or not; it exits with a 1 if there's any test failures. So it's usable in a CI/CD situation.
It resolves the slowness with CommandBox
In the previous article I observed that running stuff with CommandBox seemed to have about a 45sec overhead for any action. I tracked this down to the fact that I have my /root/home directory as a volume from my host machine so my various shell histories persist across container rebuilds. And I then realised that CommandBox dumps a whole lot of shite in that directory which it needs to load every time it runs. Because of the shenanigans Docker needs to do when bridging from its file system across to WSL across to the native Windows file systems, these operations are S-L-O-W. OK for a few files. Not OK for stacks of them.
Fortunately CommandBox can be configured to put its temp files elsewhere, so I have configured it to put them in /var/temp instead. As they regenerate if they are missing, this seems like the best place for them. It also prevents clutter leaking out of my container and onto my host machine. This is done via a commandbox.properties file:
commandbox_home=/var/tmp/commandbox
Which I copy into place in the Dockerfile. CommandBox picks it up automatically when I place it there:
COPY ./commandbox.properties /usr/local/bin/commandbox.properties
Good stuff. Now it only takes about 5sec for box to start doing anything, which is fine.
It no longer has the problem with path_info
I covered the shortfall in how Lucee handles path_info in "Repro for Lucee weirdness". I've managed to work around this. Kind of. In a way that solves the problem for this project anyhow.
Well I guess really it is just "learning to live with it". I've done some other experimentation with CFWheels, and all it uses path_info for is indeed to implement semi-user-friendly URLs tacked on to index.cfm. It has no need for any other .cfm file to use its path_info, so the default mappings are actually fine as they are.
However it occurred to me when I was configuring Nginx to do its part of the user-friendly URLs that all requests coming into Lucee from the web server will land in /public, so I could just put in a servlet mapping for the index.cfm in that directory (from web.xml):
<servlet-mapping>
<servlet-name>CFMLServlet</servlet-name>
<url-pattern>*.cfm</url-pattern>
<url-pattern>*.cfml</url-pattern>
<url-pattern>*.cfc</url-pattern>
<url-pattern>/index.cfm/*</url-pattern>
<url-pattern>/index.cfc/*</url-pattern>
<url-pattern>/index.cfml/*</url-pattern>
<url-pattern>/public/index.cfm/*</url-pattern>
</servlet-mapping>
One might think that instead of using <url-pattern>/public/index.cfm/*</url-pattern>, I might be able to just specify a match for the entire directory, like this: <url-pattern>/public/*</url-pattern>. From a POV of Tomcat's expectations this ought to be good enough, but from Lucee's perspective it doesn't see that as a "anything in that directory", it's expecting that pattern to be a file that matches a CFML file, so when I tried that I just got an error along the lines of "/public is a directory". Ah well. FWIW, ColdFusion said pretty much the same thing.
One downside to this is that I cannot work out how to add a servlet mapping just for this Lucee application, so I need to replace the entire Tomcat web.xml file, with another one with just one additional line (the original file is 4655 lines long). This is less than ideal, and I've followed it up on the Lucee Slack channel. I just copy the file over in the Dockerfile:
COPY ./root_home/.bashrc /root/.bashrc
COPY ./root_home/.vimrc /root/.vimrc
COPY ./web.xml /usr/local/tomcat/conf/web.xml
I had to rename my test file to index.cfm (so this means the test will need to just go once I install CFWheels which needs that file), but for now I was able to test the change:
it("passes URL path_info to Lucee correctly", () => {
testPathInfo = "/additional/path/info/"
http url="http://cfml-in-docker.frontend/index.cfm#testPathInfo#" result="response";
expect(response.status_code).toBe(200, "HTTP status code incorrect")
expect(response.fileContent.trim()).toBe(testPathInfo, "PATH_INFO value was incorrect")
})
<cfoutput>#CGI.path_info#</cfoutput>
Oh! And the Nginx changes! docker/nginx/sites/default.conf:
location / {
try_files $uri $uri/ =404;
try_files $uri $uri/ @rewrite;
}
location @rewrite {
rewrite ^/(.*)? /index.cfm$request_uri last;
rewrite ^ /index.cfm last;
}
(Thanks to the ColdBox docs for those)
It no longer needs PHP to test things
I'm happy that TestBox is working well enough now that I don't need to test things with PHPUnit, and that's all the PHP container was for, so I've removed all that stuff.
That's it. In the next article I shall continue from here, and get CFWheels set up in a waythat doesn't require the entire application being a) messed in with my own code; b) in a web browsable directory. Stay tuned…
Righto.
--
Adam
