Adam Cameron's Dev Blog: October 2020

Friday, 23 October 2020

Quickly running ColdFusion via Docker

G'day:
Firstly, don't take this as some sort of informed tutorial as to how things should be done. I am completely new to Docker, and don't yet know my arse from my elbow with it. Today I decided to look at an issue my mate was having with ColdFusion (see "ColdFusion: looking at an issue Mingo had with ehcache and cachePut and cacheGet"). These days I do not have CF installed anywhere, so to look into this issue, I needed to run it somehow. I was not gonna do an actual full install just for this. Also I've been trying - slowly - to get up to speed with Docker, and this seemed to be a sitter for doing something newish with it. Also note that this is not the sort of issue I could just use trycf.com to run some code, as it required looking at some server config.

All I'm gonna do here is document what I had to do today to solve the problem in that other article.

I am running on Windows 10 Pro, I have WSL installed, and I'm running an Ubuntu distribution, and using bash within that. I could use powershell, but I so seldom use it, I'm actually slightly more comfortable with bash. For general Windows command-line stuff I usually just use the default shell, and it didn't even occur to me until just now to check if the docker CLI actually works in that shell, given all the docs I have read use either Powershell or some *nix shell. I just checked and seems it does work in the default shell too. Ha. Ah well, I'm gonna stick with bash for this.

I've also already got Docker Desktop & CLI installed and are running. I've worked through the Getting Started material on the Docker website, and that's really about it, other than a coupla additional experiments.

Right. I'd been made aware that Adobe maintain some Docker images for ColdFusion, but no idea where they are or that sort of jazz, so just googled it ("docker coldfusion image"). The first match takes me to "Docker images for ColdFusion", and on there are the exact Docker commands to run CF / code in CF in various ways. What I wanted to do was to run the ColdFusion REPL, and mess around with its config files. The closest option to what I wanted was to run ColdFusions built-in web server to serve a website. I figured it'd use an image with a complete ColdFusion install, so whilst I didn't want or need the web part of it, I could SSH into the container and use the REPL that way. So the option for me was this one:

Run ColdFusion image in daemon mode (that link is just a deep link to the previous one above). That gives the command:

docker run -dt -p 8500:8500 -v c:/wwwroot/:/app \
-e acceptEULA=YES -e password=ColdFusion123 -e enableSecureProfile=true \
eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest

(I've just split that over multiple lines for readability).

Here we're gonna be running the container detached (-d), so once the container runs, control returns to my shell. I'm not at all clear why they're using the pseudo-TTY option here (-t / --tty). From all my reading (basically this Q&A on Stack Overflow: "Confused about Docker -t option to Allocate a pseudo-TTY"), it's more for when not in detached mode?

We're also exposing (publishing) the container's port 8500 to the host. IE: when I make HTTP requests to port 8500 in my PC's browser, it will be routed to the container (and CF's inbuilt web server in the container will be listening).

Lastly on that line we're mounting C:\wwwroot as a volume (-v) in the container as /app. IE: files in my PC's C:\wwwroot dir will be availed within the container in the /app dir.

The second line is just a bunch of environment variables (-e) that get set in the container... the ColdFusion installation process are expecting these I guess.

And the last line is the address of where the CF image is.

I'm not going to use exactly this command actually. I'm on Ubuntu so I don't have a C:\wwwroot, and I don't even care what's in the web root for this exercise (I'm just wanting the REPL, remember), so I'm just going to point it to a temp directory in my home dir (~/tmp). Also I see no reason to enable the secure profile. That's just a pain in the arse. Although it would probably not matter one way or the other.

Also I want to give my container a name (--name) for easy reference.

So here's what I've got:

docker run --name cf -dt -p 8500:8500 -v ~/tmp:/app \
-e acceptEULA=YES -e password=ColdFusion123 -e enableSecureProfile=false \
eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest

Let's run that:

adam@DESKTOP-QV1A45U:~$ docker run --name cf -dt -p 8500:8500 -v ~/tmp:/app \
> -e acceptEULA=YES -e password=ColdFusion123 -e enableSecureProfile=false \
> eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest
Unable to find image 'eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest' locally
latest: Pulling from cf/coldfusion
b234f539f7a1: Pull complete
55172d420b43: Pull complete
5ba5bbeb6b91: Pull complete
43ae2841ad7a: Pull complete
f6c9c6de4190: Pull complete
b3121a6492e2: Pull complete
cf911c0e9bab: Pull complete
ef2cca3cfd53: Pull complete
f7a96a442bce: Pull complete
5f2632eaddf0: Pull complete
f4563a5d4acb: Pull complete
ad1e188590ee: Pull complete
Digest: sha256:12e7dc19bb642a0f67dba9a6d0a2e93f805de0a968c6f92fdcb7c9c418e1f1e8
Status: Downloaded newer image for eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest
c7b703635a5358f0ea91c3f2549d8ec4a308812c44b2476bd6ad4c00b2e32c12
adam@DESKTOP-QV1A45U:~$

This takes a frew minutes to run cos the ColdFusion image is 570MB and it also downloads a bunch of other stuff. As it goes, you see progress like this:

...
ef2cca3cfd53: Pull complete
f7a96a442bce: Downloading [=================================================> ] 560.9MB/570.4MB
f7a96a442bce: Pull complete
...

In theory ColdFusion should now be installed and listening on http://localhost:8500, and...

Cool. I've had some issues running the REPL before if the CFAdmin installation step hasn't been run at least once, so I let that go (the password is the one from the environment variable, yeah? ColdFusion123).

I can see the image and the container in docker now too:

adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
eaps-docker-coldfusion.bintray.io/cf/coldfusion latest 31a6d984cf30 2 months ago 1.03GB
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c7b703635a53 eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest "sh /opt/startup/sta…" 10 minutes ago Up 10 minutes (healthy) 8016/tcp, 45564/tcp, 0.0.0.0:8500->8500/tcp cf
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$

Next we need to SSH into the container.

Tip from Pete Freitag

In his comment below, Pete points out that given I'm doing all this on the same physical machine, I don't need to use ssh, I can just do this:

docker exec -it cf /bin/bash

And this works fine:

adam@DESKTOP-QV1A45U:~$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker start cf
cf
adam@DESKTOP-QV1A45U:~$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c7b703635a53 eaps-docker-coldfusion.bintray.io/cf/coldfusion:latest "sh /opt/startup/sta…" 3 days ago Up 13 seconds (health: starting) 8016/tcp, 45564/tcp, 0.0.0.0:8500->8500/tcp cf
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ curl -s -o /dev/null -I -w "%{http_code}" http://localhost:8500/CFIDE/administrator/
200
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker exec -it cf /bin/bash
root@c7b703635a53:/opt#
root@c7b703635a53:/opt#
root@c7b703635a53:/opt# sed -n '484,485'p /opt/coldfusion/cfusion/lib/ehcache.xml
copyOnWrite="true"
/>
root@c7b703635a53:/opt#
root@c7b703635a53:/opt#
root@c7b703635a53:/opt# exit
exit
adam@DESKTOP-QV1A45U:~$

Here I show:

ColdFusion container not running
Starting the ColdFusion container
Showing the container started
Showing it is up and running and responding to requests with a 200
Using Pete's docker exec approach
Showing the change to the file I needed to make to get the caching to work as expected
Exiting the container's bash, back to my own PC's bash

It never occurred to me to try to do something like this, given traditionally I am never physically positioned on the boxes I am working on, so need to use ssh.

Cheers Pete!

For that I'm using some geezer called Jeroen Peters' docker-ssh container. I found this from a bunch of googling, and people saying how not do use SSH from within containers, and instead have the SSH server in a container of its own. Seems legit.

I've modified his docker command slightly to a) point to my cf container, also to detach once it's started:

adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker run --detach -e FILTERS={\"name\":[\"^/cf$\"]} -e AUTH_MECHANISM=noAuth \
--name sshd-web-server1 -p 2222:22 --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
jeroenpeeters/docker-ssh
5447950ce4d4d7ced8ef61696a64b8af9c4c01139c4fde58fa558ef25ba89b58
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$

I can now SSH into the container:

adam@DESKTOP-QV1A45U:~$ ssh localhost -p 2222

###############################################################
## Docker SSH ~ Because every container should be accessible ##
###############################################################
## container | /cf ##
###############################################################

/opt $

I'm in! Cool!

Just to prove to myself further that I am actually "inside" the container, I jumped over to another bash session on my PC, and stuck a new file in that ~/tmp directory:

adam@DESKTOP-QV1A45U:~$ cd tmp
adam@DESKTOP-QV1A45U:~/tmp$ cat > new_file
G'day world
^Z
[2]+ Stopped cat > new_file
adam@DESKTOP-QV1A45U:~/tmp$ ll
total 20
drwxr-xr-x 3 adam adam 4096 Oct 23 19:10 ./
drwxr-xr-x 6 adam adam 4096 Oct 23 19:09 ../
drwxr-xr-x 4 999 999 4096 Oct 23 18:25 WEB-INF/
-rwxr-xr-x 1 999 root 370 Oct 23 18:24 crossdomain.xml*
-rw-r--r-- 1 adam adam 12 Oct 23 19:10 new_file
adam@DESKTOP-QV1A45U:~/tmp$
adam@DESKTOP-QV1A45U:~/tmp$ cat new_file
G'day world
adam@DESKTOP-QV1A45U:~/tmp$

And checked it from within the container:

/opt $ cd /app
/app $ # before

/app $ ll
total 16
drwxr-xr-x 3 1000 1000 4096 Oct 23 18:09 ./
drwxr-xr-x 1 root root 4096 Oct 23 17:24 ../
drwxr-xr-x 4 cfuser cfuser 4096 Oct 23 17:25 WEB-INF/
-rwxr-xr-x 1 cfuser root 370 Oct 23 17:24 crossdomain.xml*
/app $
/app $ # after
/app $ ll
total 20
drwxr-xr-x 3 1000 1000 4096 Oct 23 18:10 ./
drwxr-xr-x 1 root root 4096 Oct 23 17:24 ../
drwxr-xr-x 4 cfuser cfuser 4096 Oct 23 17:25 WEB-INF/
-rwxr-xr-x 1 cfuser root 370 Oct 23 17:24 crossdomain.xml*
-rw-r--r-- 1 1000 1000 12 Oct 23 18:10 new_file
/app $
/app $ cat new_file

G'day world

/app $

OK so I think I'm happy with that. So... now to run the code...

/app $ cd /opt/coldfusion/cfusion/bin/
/opt/coldfusion/cfusion/bin $ ./cf.sh
ColdFusion started in interactive mode. Type 'q' to quit.
cf-cli>foo = { bar = 1 };
cachePut( 'foobar', foo );

foo.bar = 2;

writeDump( cacheGet( 'foobar' ) );
struct

BAR: 1

cf-cli>
cf-cli>cf-cli>2
cf-cli>cf-cli>struct

BAR: 2

cf-cli>^Z
[1]+ Stopped ./cf.sh
/opt/coldfusion/cfusion/bin $

All this is discussed in the previous article (ColdFusion: looking at an issue Mingo had with ehcache and cachePut and cacheGet). This is showing the problem though... the answer should be 1 not 2

I have to fix this cache setting...

/opt/coldfusion/cfusion/bin $ cd ../lib
/opt/coldfusion/cfusion/lib $ ll ehcache.xml
-rwxr-xr-x 1 cfuser bin 25056 Jun 25 2018 ehcache.xml*
/opt/coldfusion/cfusion/lib $ vi ehcache.xml
bash: vi: command not found
/opt/coldfusion/cfusion/lib $

Erm... wat??

Wow OK so this is a really slimmed down container (except the 100s of MB of ColdFusion stuff I mean). I can fix this...

/opt/coldfusion/cfusion/lib $ sudo apt-get update
bash: sudo: command not found
/opt/coldfusion/cfusion/lib $

WAAAAAH. OK this is getting annoying now. On a whim I just tried without sudo

/opt/coldfusion/cfusion/lib $ apt-get update
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
[etc]
Fetched 29.4 MB in 10s (2758 kB/s)
Reading package lists... Done
/opt/coldfusion/cfusion/lib $
/opt/coldfusion/cfusion/lib $ apt-get install vi
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package vi
/opt/coldfusion/cfusion/lib $

Fine. One step forward, one step sideways. Grumble. Another whim:

/opt/coldfusion/cfusion/lib $ apt-get install vim
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
[heaps of stuff]
/opt/coldfusion/cfusion/lib $

Ooh! Progress. But... does it work? (hard to get proof of this as vi takes over the shell, but... yes it worked).

I now just have to restart the CF container, and test the fix:

/opt/coldfusion/cfusion/lib $ exit
exit
There are stopped jobs.
/opt/coldfusion/cfusion/lib $ exit
exit
Connection to localhost closed.
adam@DESKTOP-QV1A45U:~$ docker stop cf
cf
adam@DESKTOP-QV1A45U:~$ docker start cf
cf
adam@DESKTOP-QV1A45U:~$ ssh localhost -p 2222
###############################################################
## Docker SSH ~ Because every container should be accessible ##
###############################################################
## container | /cf ##
###############################################################

/opt $ cd coldfusion/cfusion/bin/
/opt/coldfusion/cfusion/bin $ ./cf.sh
ColdFusion started in interactive mode. Type 'q' to quit.
cf-cli>foo = { bar = 1 };
cachePut( 'foobar', foo );

foo.bar = 2;

writeDump( cacheGet( 'foobar' ) );
struct

BAR: 1

cf-cli>
cf-cli>cf-cli>2
cf-cli>cf-cli>struct

BAR: 1

cf-cli>

WOOHOO! I managed to do something useful with Docker (and then reproduce it all again when writing this article). Am quite happy about that.

Final thing... if yer reading this and are going "frickin' hell Cameron, you shouldn't do [whatever] like [how I did it]", please let me know. This is very much a learning exercise for me.

Anyhow... this is the most work I've done in about four months now, and I need a... well... another beer.

Righto.

--
Adam

ColdFusion: looking at an issue Mingo had with ehcache and `cachePut` and `cacheGet`

G'day:

Bloody Coldfusion. OK so why am I writing about a ColdFusion issue? Well about 80% of it is "not having a great deal else to do today", about 10% of being interested in this issue Mingo found. And 10% it being an excuse to mess around with Docker a bit. I am currently - and slowly - teaching myself about Docker, so there's some practise for me getting a ColdFusion instance up and running on this PC (which I no-longer have any type of CFML engine installed on).

OK so what's the issue. Mingo posted this on Twitter:

Just in case you wanna run that code, here it is for copy and paste:

foo = { bar = 1 };
cachePut( 'foobar', foo );

foo.bar = 2;

writeDump( cacheGet( 'foobar' ) );

Obviously (?) what one would expect here is {bar:1}. What gets put into cache would be a copy of the struct right?

Well... um... here goes...

/opt/coldfusion/cfusion/bin $ ./cf.sh
ColdFusion started in interactive mode. Type 'q' to quit.
cf-cli>foo = { bar = 1 };
struct
BAR: 1

cf-cli>cachePut( 'foobar', foo );
cf-cli>foo.bar = 2;
2
cf-cli>writeDump( cacheGet( 'foobar' ) );
struct
BAR: 2

cf-cli>

... errr... what?

It looks like ColdFusion is like only putting a reference to the struct into cache. So any code changing the data in the struct is changing it in CFML as well as changing it in cache. This does not seem right.

I did a quick google and found a ticket in Adobe's system about this: CF-3989480 - cacheGet returns values by reference. The important bit to note is that it's closed with

Not a great explanation from Adobe there, fortunately Rob Bilson had commented further up with a link to a cached version of an old blog article of his, explaining what's going on: "Issue with Ehcache and ColdFusion Query Objects". It's a slightly different situation, but it's the same underlying "issue". Just to copy and paste the relevant bit from his article:

Update: It looks like this is actually expected behavior in Ehcache. Unfortunately, it's not documented in the ColdFusion documentation anywhere, but Ehcache actually has two configurable parameters (as of v. 2.10) called copyOnRead and copyOnWrite that determine whether values returned from the cache are by reference or copies of the original values. By default, items are returned by reference. Unfortunately we can't take advantage of these parameters right now as CF 9.0.1 implements Ehcache 2.0.

I decided to have a look what we could do about this on ColdFusion 2018, hoping that its embedded implementation of Ehcache has been updated since Rob wrote that in 2010.

Firstly I checked the Ehcache docs for these two settings: copyOnWrite and copyOnRead. This is straight forward (from "copyOnRead and copyOnWrite cache configuration"):

<cache name="copyCache"
    maxEntriesLocalHeap="10"
    eternal="false"
    timeToIdleSeconds="5"
    timeToLiveSeconds="10"
    copyOnRead="true"
    copyOnWrite="true">
  <persistence strategy="none"/>
  <copyStrategy class="com.company.ehcache.MyCopyStrategy"/>
</cache>

The docs also confirm these are off by default

Next where's the file?

/opt/coldfusion $ find . -name ehcache.xml
./cfusion/lib/ehcache.xml
/opt/coldfusion $

Cool. BTW I just guessed at that file name.

So in there we have this (way down at line 471):

<!--
Mandatory Default Cache configuration. These settings will be applied to caches
created programmtically using CacheManager.add(String cacheName).

The defaultCache has an implicit name "default" which is a reserved cache name.
-->
<defaultCache
    maxElementsInMemory="10000"
    eternal="false"
    timeToIdleSeconds="86400"
    timeToLiveSeconds="86400"
    overflowToDisk="false"
    diskSpoolBufferSizeMB="30"
    maxElementsOnDisk="10000000"
    diskPersistent="false"
    diskExpiryThreadIntervalSeconds="3600"
    memoryStoreEvictionPolicy="LRU"
    clearOnFlush="true"
    statistics="true"
/>

That looked promising, so I updated it to use copyOnWrite:


<defaultCache
    maxElementsInMemory="10000"
    eternal="false"
    timeToIdleSeconds="86400"
    timeToLiveSeconds="86400"
    overflowToDisk="false"
    diskSpoolBufferSizeMB="30"
    maxElementsOnDisk="10000000"
    diskPersistent="false"
    diskExpiryThreadIntervalSeconds="3600"
    memoryStoreEvictionPolicy="LRU"
    clearOnFlush="true"
    statistics="true"
    copyOnWrite="false"
/>

Whatever I put into cache, I want it to be decoupled from the code immediately, hence doing the copy-on-write.

I restarted CF and ran the code again:

/opt/coldfusion/cfusion/bin $ ./cf.sh
ColdFusion started in interactive mode. Type 'q' to quit.
cf-cli>foo = { bar = 1 };
struct

BAR: 1

cf-cli>cachePut( 'foobar', foo );
cf-cli>foo.bar = 2;
2

cf-cli>writeDump( cacheGet( 'foobar' ) );

struct
BAR: 1

cf-cli>

Yay! We are getting the "expected" result now: 1

Don't really have much else to say about this. I'm mulling over writing down what I did to get ColdFusion 2018 up and running via Docker instead of installing it. Let's see if I can be arsed...

Righto.

Adam

Friday, 16 October 2020

Ben Brumm writes an interesting article on Hierarchical Data in SQL

G'day:

This is solely a heads-up regarding some potential reading for you.

A bloke called Ben Brumm came across one of my old articles, "CFML: an exercise in converting a nested set tree into an adjacency list tree". He's hit me up and asked me to add a link to that to an article he's written "Hierarchical Data in SQL: The Ultimate Guide". I've done that, but that article is buried in the mists of time back in 2015. I figured he could benefit from a more contemporary nudge too. So here it is. Me nudging.

In other news people have been urging me to get back to this blog, but I'm afraid I still am not finding anything new to write about. So... erm... yeah. If something leaps out at me I do fully intend to come back to this, but... not for now.

Righto.

Adam

About

I've been a web developer since 2001: 13yrs as a CFML developer; 6yrs as a PHP dev; and now back adjacent to the CFML community but focusing on code quality, design and testing. As of 2023, I am migrating my dev focus back to PHP again.

The code I write and discuss here is pretty much just looking at random conundrums I encounter in my day job.

I tend to be a bit "forthright" in my opinions, I am indelicate, and I tend to swear too much. This will come out occasionally here: I make no apology for it.

Everything said here is my own opinion. Feel free to disagree with me :-)