Monday, 14 July 2025

On the other hand... Docker Swarm

G'day:

(That's a callback of sorts to today's earlier article Clustering a PHP app with Kubernetes (or "Why the f*** do I put myself through this shite?"), which clocks in a ~5500 words).

To do the equivalent in a dev environment with Docker Swarm, one does this:

$ docker swarm init --advertise-addr 127.0.0.1
Swarm initialized: current node (fn3o7kmnep81j28xjsth094z6) is now a manager.

And then one does this:

$ docker service create \
    --name php \
    --replicas 3 \
    --publish published=9000,target=9000 \
    --env-file docker/envVars.public \
    --env-file docker/php/envVars.public \
    --env-file docker/php/envVars.prod.public \
    --env-file docker/envVars.private \
    --env-file docker/php/envVars.private \
    --host host.docker.internal:host-gateway \
    adamcameron/php8-on-k8s:latest
    
fw3j4qtrpe2lgesr0x3tbeiaa
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service fw3j4qtrpe2lgesr0x3tbeiaa converged    
    

And then… erm… no actually that's it. That works. Done.


Obviously this (and the other article) and for dev-ony messing around with things, and I have no idea how to manage Kubernetes or Docker Swarm. But one took a week to work out WTF. And the other took about an hour. Inciuding the time it's gonna take to write this.

Source code has been updated in the repo I was using before, and tagged as 0.9. It's changes to the README.md, and making a label more generic ("Pod name" => "Instance ID").

Oh, and I was working from these docs: Manuals / Docker Engine / Swarm mode. Beyond a point I was able to guess what I needed to do.

Righto.

--
Adam

Clustering a PHP app with Kubernetes (or "Why the f*** do I put myself through this shite?")

G'day:

The subtitle is a serious question there: whilst I like doing programming stuff, I really could do without all the rest of the shite we need to do these days to get apps running. In the past "we've had people for that" (SysOps / SRE Team / etc), so I'd never needed to care about this sort of thing. However now I find myself in smaller outfits that don't have dedicated people for this, and I've had to pick up some knowledge of Docker and that sort of thing. It's fine, but… right at the border of my comfort zone. And it's a "needs must" thing, not something I think is really how I ought to be spending my time.

Over the last four years, I've been "leading" our efforts with dockerising our apps, streamlining them etc. At a point we needed to improve our uptime to be largely seamless for our users, and for reasons that were above my pay grade (literally), we weren't given budget to put Docker Swarm in; instead I heath-robinson-ed an implementation of blue/green deployment leveraging how quickly Nginx could reload config to point to different PHP-FPM upstreams. Way less than ideal, but it worked, and it's about all I could manage with my limited knowledge / time / interest in the subject matter. That said, I was actually quite pleased with myself that I managed to work out how to get it working. But still: it's not how things should be done.

I've left that role now, and during some time I have available to do some R&D to fill some gaps in my tech knowledge, I've decided I had better find out how to do that blue/green shiz in a more polished fashion.

Kubernetes has been something that's perplexed me for years. Mostly cos I heard ppl talk about it, and my eyes started to glaze over. All I could take on board was "[something something] multiple containers", "largely impenetrable to understand", "way more complicated than it ought to be". Great. Anyhow, I decided to see if I could perform this experiment:

  • Set up a small "G'day world" app with a Nginx, PHP and MariaDB containers, dockerised, orchestrated with docker compose.
  • Leave the Nginx and MariaDB containers as is.
  • For a production-ish environment, orchestrate the PHP container via Kubernetes; leaving dev to be orchestrated via Docker-only.
  • Obviously comms from Nginx->PHP and PHP->MariaDB still need to work (in both dev and prod).

Baseline / control

First things first, I need a stable Nginx-PHP8-MariaDB-Docker environment. This is pretty familiar to me, and I have some code lying around that just needs a bit of polish (adamcameron/php8). I've written about its initial config before, in PHP: returning to PHP and setting up a PHP8 dev environment. The Dockerfiles for this (Nginx, PHP, MariaDB) are routine, and nothing comment-worthy. The PHP part of the docker-compose.yml file is as follows:

php:
        build:
            context: php
            dockerfile: Dockerfile

        env_file:
            - envVars.public
            - php/envVars.public
            - envVars.private
            - php/envVars.private

        stdin_open: true
        tty: true

        volumes:
            - ..:/var/www

There's four environment variables files there, two public, two private:

# docker/envVars.public
MARIADB_HOST=mariadb
MARIADB_PORT=3306
MARIADB_DATABASE=db1
MARIADB_USER=user1


# docker/php/envVars.public
APP_CACHE_DIR=/var/cache/symfony
APP_LOG_DIR=/var/log/symfony


# docker/envVars.private
MARIADB_PASSWORD=123


docker/php/envVars.private
APP_SECRET=does_not_matter

That mostly speaks for itself. The different directories being used is just cos the MariaDB container also uses the files with the MariaDB stuff in them. The ones in the docker/php directory are specific to the PHP container. The private files are not in source control.

Note also there is that one volume used to mount the source code from the host machine into the container file system. Standard stuff.

I bring this lot up:

$ pwd
~/src/php8-on-k8s

$ docker compose -f docker/docker-compose.yml down --volumes
[output snipped]

$ docker compose -f docker/docker-compose.yml build --no-cache
[output snipped]

$ docker compose -f docker/docker-compose.yml up --detach
[output snipped]


# wait for a bit for composer install to finish in the PHP container;
# and the MariaDB container to run its /docker-entrypoint-initdb.d scripts


$ docker container ls --format "table {{.Names}}\t{{.Status}}"
NAMES                   STATUS
php8-on-k8s-nginx-1     Up 2 minutes (healthy)
php8-on-k8s-php-1       Up 2 minutes (healthy)
php8-on-k8s-mariadb-1   Up 2 minutes


$ docker exec php8-on-k8s-php-1 composer test-all

./composer.json is valid
PHPUnit 12.2.6 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.10 with Xdebug 3.4.4
Configuration: /var/www/phpunit.xml

...............                                                   15 / 15 (100%)

Time: 00:02.428, Memory: 28.00 MB

Tests Database objects
 ✔ It has a proc called sleep_and_return which idles for n seconds before returning its parameter value

Tests of Symfony installation
 ✔ It serves the home page
 ✔ It can run the console in a shell

Tests of Symfony testing
 ✔ It serves the default welcome page after installation

Tests of environment variables
 ✔ The expected environment variables exist with data set "APP_CACHE_DIR"
 ✔ The expected environment variables exist with data set "APP_LOG_DIR"
 ✔ The expected environment variables exist with data set "APP_SECRET"
 ✔ The expected environment variables exist with data set "MARIADB_DATABASE"
 ✔ The expected environment variables exist with data set "MARIADB_HOST"
 ✔ The expected environment variables exist with data set "MARIADB_PASSWORD"
 ✔ The expected environment variables exist with data set "MARIADB_PORT"
 ✔ The expected environment variables exist with data set "MARIADB_USER"

Tests of the Greeter class
 ✔ It greets formally by default
 ✔ It greets informally

Tests of the PHP installation
 ✔ It has the expected PHP version

OK (15 tests, 20 assertions)

Generating code coverage report in HTML format ... done [00:00.010]

And, of course: does it actually serve its homepage:

(Not very exciting, I know).

But it's up and healthy. I note I forgot to do a health-check on the MariaDB container. Also note in general I have automated tests on the DB connection, Symfony install, environment, and some stub code. This all just checks the moving parts are moving together and in the right direction. This is basically the "control" for my experiment: via Docker, everything is connected and working.

I realise this is pretty meaningless without looking at the code, but it's all on Github, linked-to above, and for completeness it's @ https://github.com/adamcameron/php8-on-k8s/tree/0.41. That it's the 0.41 tag is significant. The main branch has moved on, and is quite different. The 0.41 tag is what I used to get as far as we are currently: Docker containers up, running, tested.


WTF is Kubernetes?

I seriously didn't really know at this point. I knew it was something to do with orchestrating clusters of containers. But that was about it. I mean to frame my ignorance I didn't even know that it still uses Docker containers. I had ass-u-me`d that Kubernetes had its own container mechanics. But no: it's all and only about the orchestration. So that is something at least: I don't need to learn a new container-creation lingo.

I read the Wikipedia page (Kubernetes), and most of it just had me going "huh?". The most interesting thing was from the criticism section:

A common criticism of Kubernetes is that it is too complex. Google admitted this as well.

https://en.wikipedia.org/wiki/Kubernetes#Criticism

Great.

I thought I'd have a look at some stuff on YT to see if it was any help. I watched these two:

Both were informative, and had me begin to realise that I can probably ignore most of the complexity of Kubernetes, and really only had to focus on "deployments" and "services" (and config thereof). Everything else will "just work", and I guess only needs tinkering with if yer a Kubernetes admin, which I will never need to be. So that's heartening.

The first thing I needed to do was to get Kubernetes installed. My mate Brendan has said "you'll need Minikube: it's like the dev/test/tinkering version of Kubernetes". This was re-iterated on the first page of the Learn Kubernetes Basics. So let's do that (I worked through all the tutorials in that section of the docs. I'll just include the highlights here).

From minikube start (Linux » Debian approach for WSL running on Windows).

$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
[output snipped]

$ sudo dpkg -i minikube_latest_amd64.deb
[output snipped]

$ minikube start
😄  minikube v1.36.0 on Ubuntu 22.04 (amd64)
✨  Automatically selected the docker driver
📌  Using Docker driver with root privileges
❗  For an improved experience it's recommended to use Docker Engine instead of Docker Desktop.
Docker Engine installation instructions: https://docs.docker.com/engine/install/#server
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🚜  Pulling base image v0.0.47 ...
💾  Downloading Kubernetes v1.33.1 preload ...
    > preloaded-images-k8s-v18-v1...:  347.04 MiB / 347.04 MiB  100.00% 11.77 M
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
🐳  Preparing Kubernetes v1.33.1 on Docker 28.1.1 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

# checking that things work (as per the tutorial):
$  kubectl get po -A
NAMESPACE     NAME                               READY   STATUS    RESTARTS      AGE
kube-system   coredns-674b8bbfcf-5gm2v           1/1     Running   0             33s
kube-system   etcd-minikube                      1/1     Running   0             38s
kube-system   kube-apiserver-minikube            1/1     Running   0             38s
kube-system   kube-controller-manager-minikube   1/1     Running   0             38s
kube-system   kube-proxy-szq8k                   1/1     Running   0             33s
kube-system   kube-scheduler-minikube            1/1     Running   0             38s
kube-system   storage-provisioner                1/1     Running   1 (12s ago)   36s

Minikube also comes with a great web-based dashboard. This needs to be run from its own terminal window, as it needs to stay running:

$ minikube dashboard --port 40000

🔌  Enabling dashboard ...
    ▪ Using image docker.io/kubernetesui/metrics-scraper:v1.0.8
    ▪ Using image docker.io/kubernetesui/dashboard:v2.7.0
💡  Some dashboard features require the metrics-server addon. To enable all features please run:

        minikube addons enable metrics-server

🤔  Verifying dashboard health ...
🚀  Launching proxy ...
🤔  Verifying proxy health ...
🎉  Opening http://127.0.0.1:40000/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...
👉  http://127.0.0.1:40000/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/

I learned after letting it use random ports for a while that one can force it to stay on a port with that port setting. Browsing to that URL:

There's not much to see, but clearly something is running OK.


Back to my docker containers

I'd forgotten one step in all this. My PHP container is only dev-ready: I need to build a prod-ready version. Plus currently all the Docker containers are using the internal Docker network provided by docker compose, whereas I will need them to communicate-out via the host machine, as the PHP container won't be managed by docker compose any more. I'll just run through the changes here:

# docker/nginx/etc/nginx/conf.d/default.conf

upstream php-fpm {
    server php:9000;
    server host.docker.internal:9000;
    keepalive 32;
}

As per above: this needs to forward the requests to the host machine; there will be no PHP container on the Docker network, in "production mode". This will also work fine in "dev mode".

# .env

APP_ENV=dev
# stick stuff in here that all envs need


# docker/php/envVars.dev.public
APP_ENV=dev


docker/php/envVars.prod.public
APP_ENV=prod

This was just wrong. One cannot set the APP_ENV environment variable in Symfony's .env system, as Symfony needs to know the environment before the .env files are loaded (which seems very obvious now that I say it). So now docker compose needs to load that env file as well (see below).

# docker/docker-compose.yml

services:
  nginx:
    container_name: nginx
    build:
      context: nginx
      dockerfile: Dockerfile

    ports:
      - "8080:80"

    stdin_open: true
    tty: true

    volumes:
      - ../public:/usr/share/nginx/html/

    depends_on:
        - php

  php:
    container_name: php
    build:
      context: php
      dockerfile: Dockerfile
      dockerfile: Dockerfile.dev

    env_file:
      - envVars.public
      - php/envVars.public
      - php/envVars.dev.public
      - envVars.private
      - php/envVars.private

    ports:
      - "9000:9000"

    stdin_open: true
    tty: true

    volumes:
      - ..:/var/www

  mariadb:
    container_name: db
    build:
      context: mariadb
      dockerfile: Dockerfile

    env_file:
      - envVars.public
      - envVars.private
      - mariadb/envVars.private

    ports:
      - "3380:3306"

    stdin_open: true
    tty: true

    volumes:
      - mariadb-data:/var/lib/mariadb

volumes:
  mariadb-data:

Top to bottom here:

  • Unrelated to this exercise, but I never remember to give my containers names. Adding this in.
  • The Nginx container can't rely on the PHP container being up before it starts any more, as the latter won't be being orchestrated by docker when in prod mode. There's no harm here really.
  • We need a specific .dev Dockerfile now, as the requirements for dev and prod are slightly different. See below.
  • And we need to expose the PHP-FPM port, so Nginx can talk to it still, given Nginx is now forwarding via the host network, not the Docker network.

The main changes need to be made to the Dockerfile. Here's the original (docker/php/Dockerfile):

FROM php:8.4.10-fpm-bookworm

RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "zip", "unzip", "git", "vim"]

RUN echo "alias ll='ls -alF'" >> ~/.bashrc
RUN echo "alias cls='clear; printf \"\033[3J\"'" >> ~/.bashrc

RUN cp /usr/local/etc/php/php.ini-development /usr/local/etc/php/php.ini
COPY usr/local/etc/php/conf.d/php8-on-k8s.ini /usr/local/etc/php/conf.d/php8-on-k8s.ini

COPY --from=composer:latest /usr/bin/composer /usr/local/bin/composer

RUN pecl install xdebug && docker-php-ext-enable xdebug
COPY usr/local/etc/php/conf.d/xdebug.ini /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
COPY usr/local/etc/php/conf.d/error_reporting.ini /usr/local/etc/php/conf.d/error_reporting.ini

RUN apt-get install -y libicu-dev && docker-php-ext-configure intl && docker-php-ext-install intl

RUN [ \
    "apt-get", "install", "-y",  \
    "libz-dev", \
    "libzip-dev", \
    "libfcgi0ldbl" \
]
RUN docker-php-ext-configure zip && docker-php-ext-install zip
RUN docker-php-ext-configure pdo_mysql && docker-php-ext-install pdo_mysql
RUN docker-php-ext-configure opcache && docker-php-ext-install opcache

RUN curl -1sLf 'https://dl.cloudsmith.io/public/symfony/stable/setup.deb.sh' | bash
RUN ["apt-get", "install", "-y", "symfony-cli"]

RUN git config --global --add safe.directory /var/www


RUN mkdir -p /var/cache && chown www-data:www-data /var/cache && chmod 775 /var/cache
RUN mkdir -p /var/log && chown www-data:www-data /var/log && chmod 775 /var/log


HEALTHCHECK \
    --start-period=30s \
    --interval=30s \
    --timeout=3s \
    --retries=3 \
    CMD SCRIPT_FILENAME=/var/www/bin/healthCheck.php REQUEST_METHOD=GET cgi-fcgi -bind -connect localhost:9000 | grep 'pong' || exit 1

WORKDIR /var/www
ENV COMPOSER_ALLOW_SUPERUSER 1


COPY --chmod=755 usr/local/bin/entrypoint.sh /usr/local/bin/
ENTRYPOINT ["entrypoint.sh"]



EXPOSE 9000

I've highlighted all the dev-specific bits in this:

  • We need to use the php.ini-production when in production mode.
  • We do not need to run xdebug in production mode.
  • This is slightly wrong. I need to create and give perms to the dev/prod subdirectories of these paths.
  • The entry point runs composer install, but we'll not need to do this in production mode, as we'll do it at build-time in the image.

What I've ended up doing is to push all the common stuff up into a Dockerfile.base, and building that separately:

docker build -f docker/php/Dockerfile.base -t adamcameron/php8-on-k8s-base .

From there I have two files:

# docker/php/Dockerfile.dev

FROM adamcameron/php8-on-k8s-base

RUN cp /usr/local/etc/php/php.ini-development /usr/local/etc/php/php.ini
COPY usr/local/etc/php/conf.d/php8-on-k8s.ini /usr/local/etc/php/conf.d/php8-on-k8s.ini

RUN pecl install xdebug && docker-php-ext-enable xdebug
COPY usr/local/etc/php/conf.d/xdebug.ini /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini

# need to use 777 as both php-fpm and php-cli will write to these directories
RUN mkdir -p /var/cache/symfony/dev && chown www-data:www-data /var/cache/symfony/dev && chmod 777 /var/cache/symfony/dev

RUN git config --global --add safe.directory /var/www

COPY --chmod=755 usr/local/bin/entrypoint.sh /usr/local/bin/
ENTRYPOINT ["entrypoint.sh"]

EXPOSE 9000

This is the dev-specific stuff from the original file we had. Note how it's using FROM adamcameron/php8-on-k8s-base as its base image, which has all the common stuff abstracted away; this just focuses on what it is to be a dev container image.

This is the prod version:

# docker/php/Dockerfile.prod

FROM adamcameron/php8-on-k8s-base

WORKDIR /var/www

RUN cp /usr/local/etc/php/php.ini-production /usr/local/etc/php/php.ini
COPY docker/php/usr/local/etc/php/conf.d/php8-on-k8s.ini /usr/local/etc/php/conf.d/php8-on-k8s.ini

# need to use 777 as both php-fpm and php-cli will write to these directories
RUN mkdir -p /var/cache/symfony/prod && chown www-data:www-data /var/cache/symfony/prod && chmod 777 /var/cache/symfony/prod

COPY .env .env
COPY .env.prod .env.prod
COPY config config
COPY src src
COPY public public
COPY templates templates

COPY composer.json composer.lock .
RUN composer install --no-interaction --no-ansi --no-progress --prefer-dist --no-dev

EXPOSE 9000

Key points:

  • As per the dev version: most stuff is now abstracted away in the adamcameron/php8-on-k8s-base image, and this just focuses on what it is to be a production container image.
  • We use the php.ini-production file as the basis of our php.ini settings.
  • We creat the prod subdirectories of Symfony's /var/cache directory.
  • We aren't wanting to maintain the app files from the host machine with the production container, so we copy all the app's files into the image's file system. Note we're not copying the tests into the image's filesystem. We won't be needing these in the production environment.
  • And we don't install any of the dev dependencies when we do the composer install. Also note we are doing composer install at build time, not run time. This makes container start-up faster.
  • Unlike the dev version of this file, we have no special entrypoint directive. All the container needs to do to finish start-up is to run php-pfm, which is what the underlying PHP image already does.

We need to re-test both the updated dev set-up (via docker compose), and the new prod set-up… for now, still via Docker, but via docker run. I want to know two things:

  • The dev tweaks don't break the dev environment.
  • The prod version works, as a baseline before we start trying to make it work via Kubernetes
$ docker compose  -f docker/docker-compose.yml down --volumes
[success output snipped]


$ docker compose  -f docker/docker-compose.yml build --no-cache
[success output snipped]


$ docker compose  -f docker/docker-compose.yml up --detach
[success output snipped]

$ docker exec php composer test-all
[success output snipped]
OK (15 tests, 23 assertions)
[success output snipped]

$  curl http://php8-on-k8s.local:8080/
<!DOCTYPE html>
<html>
    <head>
        <meta charset="UTF-8">
        <title>Welcome!</title>
        <link rel="icon" type="image/x-icon" href="/favicon.ico">
    </head>
    <body>
    Hello world from Symfony<br>
    Mode: dev<br>
    Pod name: unknown<br>
    DB version: 10.11.13-MariaDB-ubu2204<br>
    </body>
</html>

(Full disclousre, I accidentally broke two tests with the changes to the homepage output. So much for me and TDD eh?)

OK, so the new dev config works. Now to build and run the container in prod mode:

# get rid of the existing dev container
$ docker compose  -f docker/docker-compose.yml down php
[success output snipped]

$ docker build -f docker/php/Dockerfile.prod -t adamcameron/php8-on-k8s .
[success output snipped]

$ docker run \
     --name php \
     --restart unless-stopped \
     -p 9000:9000 \
     --env-file docker/envVars.public \
     --env-file docker/php/envVars.public \
     --env-file docker/php/envVars.prod.public \
     --env-file docker/envVars.private \
     --env-file docker/php/envVars.private \
     --add-host=host.docker.internal:host-gateway \
     --detach \
     -it \
     adamcameron/php8-on-k8s:latest
[success output snipped]

Looks promising. We can't run the tests to verify the install / config this time as we did not install them in the prod container, so we'll just check the homepage:

$ curl http://php8-on-k8s.local:8080/
<!DOCTYPE html>
<html>
    <head>
        <meta charset="UTF-8">
        <title>Welcome!</title>
        <link rel="icon" type="image/x-icon" href="/favicon.ico">
    </head>
    <body>
    Hello world from Symfony<br>
    Mode: prod<br>
    Pod name: unknown<br>
    DB version: 10.11.13-MariaDB-ubu2204<br>
    </body>
</html>

The key point here is Mode: prod. We can also check some Symfony config:

$ docker exec php symfony console about | grep -b3 -a4 "Environment"
469- -------------------- -------------------------------------------
536-  Kernel
603- -------------------- -------------------------------------------
670-  Type                 App\Kernel
737:  Environment          prod
804-  Debug                false
871-  Charset              UTF-8
938-  Cache directory      /var/cache/symfony/prod (668 KiB)
1005-  Build directory      /var/cache/symfony/prod (668 KiB)

All good.


Converting all that to something Kubernetes can work with

OK. The distillation of my reading of the Kubernetes docs (and some discussion with Co-pilot, who has been very helpful in all this) is that I need to look at four things:

Deployment
A deployment is a set of pods, where "pod" is Kubernetes lingo for "some containers". It's kinda the Kubernetes equivalent of a bunch of containers defined in a docker-compose.yml file. In my case I have just the one container per pod. The pod is the "definition mechanism" for what is clustered. I'm gonna get Kubernetes to run three pods, each with my PHP container in it. It's the "cattle" version of my container's "pet", in a way I guess. Ref: Pets vs Cattle
ConfigMap
A config map is what Kubernetes uses where Docker would use an env-file. True to form for Kubernetes, they're more complicated. But I have been able to ignore most of the complication.
Secrets
And secrets are what one uses in Kubernetes instead of what I do with non-source-controlled env-files. Again, more complicated than they need to be, but most of that can swerved.
Service
A service is the bit of the Kubernetes system that exposes the pods in the deployment over the network.

k8s/deployment.yaml

Full disclosure, I asked Co-pilot to do the work here, based on the docker-compose.yaml file. It nailed it. There are also other tools out there to perform these conversions, so it's a legit approach I think. Rather than slavishly reading the docs and understanding the minutiae of what's going on in here.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: php
spec:
    replicas: 3
    selector:
        matchLabels:
            app: php
    template:
        metadata:
            labels:
                app: php
        spec:
            containers:
                - name: php
                  image: adamcameron/php8-on-k8s:latest
                  ports:
                      - containerPort: 9000
                  envFrom:
                      - configMapRef:
                            name: php-config
                      - secretRef:
                            name: php-secret
                  env:
                    - name: POD_NAME
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.name
                    - name: CONTAINER_NAME
                      value: php
            restartPolicy: Always

Notes:

  • replicas: 3 is telling Kubernetes I want three pods built from this template. So: three containers running my app.
  • image: adamcameron/php8-on-k8s:latest: This needs to come from Docker Hub; it won't pick the image up locally. It's fine, but it caught me out initially.
  • containerPort: 9000: this is telling the pod what port the container expects to be listening on. Exposing that to downstream services is handled by the service config (see below).
  • The envFrom bit is the equiv of the env-file bit of docker-compose.yaml.
  • And the env bit is more like the inline env part of a docker-compose file. Here I'm exposing an env var POD_NAME with value of the unique reference to the pod containing the container. I'm using this on the home page to demonstrate that each request could be serviced by any of the pods.
  • And CONTAINER_NAME is the equiv of the similar-named value in docker-compose.yaml.

k8s/php-config.yaml

apiVersion: v1
data:
  APP_CACHE_DIR: /var/cache/symfony
  APP_ENV: prod
  APP_LOG_DIR: /var/log/symfony
  MARIADB_DATABASE: db1
  MARIADB_PORT: "3380"
  MARIADB_USER: user1
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: php-config

One thing to observe here is that I am not setting MARIADB_HOST. This is because I need to use the host's network, and in Docker I'd use host.docker.internal, but Kubernetes does not have this, so I have to set it separately. See the section on k8s/bin/createDeployment.sh, below.

k8s/php.secret.yaml

NB: this file is not source-controlled.

apiVersion: v1
kind: Secret
metadata:
  name: php-secret
type: Opaque
data:
  MARIADB_PASSWORD: MTIz
  APP_SECRET: ZG9lc19ub3RfbWF0dGVy

The chief difference here is that the values are base-64 encoded. This is easy to do in bash:

$ echo -n '123' | base64
MTIz

Again the name is to what the deployment.yaml is referring.

k8s/service.yaml

apiVersion: v1
kind: Service
metadata:
    name: php
spec:
    type: LoadBalancer
    selector:
        app: php
    ports:
        - protocol: TCP
          port: 9000
          targetPort: 9000
          nodePort: 31000

Notes:

  • type: LoadBalancer. With Minikube, this is the easiest way to expose container (via their pods) to the outside world. Kubernetes itself expects other kit to sit in front of it to be the load balancer, but Minikube comes with one out of the box.
  • The port / targetPort bits are the equivalent of exposing ports in docker-compose.yaml. This only exposes the port within the Kubernetes network, it does not in itself expose the port - and therefore the container - to the "outside world": that's the nodePort bit, which is what Minikube's "load-balancer" exposes the service as. There is a twist to that statement in my Windows environment, but I'll get to that.

One thing different in Kubernetes config compared to how Docker works is these files are not what it actually uses to configure things, they're just the source data which we need to "apply" to the Kubernetes config:

$ kubectl apply -f k8s/deployment.yaml
deployment.apps/php created

And having done that, there some action in the Dashboard:

I mean red is not good, but at least it's seen the deployment config, and I can see the three pods I asked for, etc. What's the problem though?

This is entirely fair enough: I've said it needs some config, and I have not given it yet. OK so let's do that:

$ kubectl apply -f k8s/php-config.yaml
configmap/php-config created

I can then restart my deployment:

And the news is not quite what I expected:

The pod seems to have it right: it's missing its secret (cos I ain't applied it yet!); I'd expect the deployment error to be the same. Ah well. Let's crack on:

$ kubectl apply -f k8s/php.secret.yaml
secret/php-secret created

Woohoo!

Look at that green :-)

I can now also access the containers:

$ kubectl exec -it php-8dc55f46d-875lw -- bash
# printenv | grep -i mariadb
MARIADB_PORT=3380
MARIADB_USER=user1
MARIADB_DATABASE=db1
MARIADB_PASSWORD=123

Oh right: I've not fixed that missing MARIADB_HOST issue yet. Let's do that now.

k8s/bin/applyConfig.sh

#!/bin/bash

set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"

kubectl apply -f "$SCRIPT_DIR/php-config.yaml"
kubectl apply -f "$SCRIPT_DIR/php.secret.yaml"
kubectl apply -f "$SCRIPT_DIR/deployment.yaml"
kubectl apply -f "$SCRIPT_DIR/service.yaml"

HOST_IP=$(ip route | awk '/default/ { print $3 }')
if [ -z "$HOST_IP" ]; then
  echo "Could not determine host IP"
  exit 1
fi
kubectl set env deployment/php MARIADB_HOST=$HOST_IP

There's a coupla things going on here, but we only need to look at the highlighted bit. I have to work out what the MariaDB address is dynamically, because there's no guarantee what the host's IP address will be. Co-pilot wrote this for me, and it seems to do the trick.

This file also demonstrates that one can apply new config to a deployment after the fact. This is what's going in in that last line there: adding a new env-var for the MARIADB_HOST value that we skipped in the initial configMap

I'll run the bit to fix the env var directly now:

$ HOST_IP=$(ip route | awk '/default/ { print $3 }')
if [ -z "$HOST_IP" ]; then
  echo "Could not determine host IP"
  exit 1
fi

$ echo $HOST_IP
172.24.176.1

$ kubectl set env deployment/php MARIADB_HOST=$HOST_IP
deployment.apps/php env updated

Wait for a bit so Kubernetes can restart all the containers with the new config.

$ kubectl exec -it php-7b57f5f6b9-bklkb -- bash
# printenv | grep -i mariadb
MARIADB_PORT=3380
MARIADB_HOST=172.24.176.1
MARIADB_USER=user1
MARIADB_DATABASE=db1
MARIADB_PASSWORD=123

Cool: it's there now. And we can test connectivity to it:

# apt update
[success output snipped]

# apt install mariadb-client -y
[success output snipped]

# mysql --host=$MARIADB_HOST --port=$MARIADB_PORT --database=$MARIADB_DATABASE -u$MARIADB_USER -p$MARIADB_PASSWORD
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 453
Server version: 10.11.13-MariaDB-ubu2204 mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [db1]>

Nice.

The observant might notice th first time I exec-ed into the container, the container name was php-8dc55f46d-875lw. This time it was php-7b57f5f6b9-bklkb. When I updated the deployment config, Kubernetes shut down and removed the containers in my app, and rebuilt them from scratch with the new environment variable. The best thing is, it does this in a controlled fashion: rebuilding one pod at a time, and only continuing if the pod rebuild was successful. So the app was never down during that exercise.

OK the next step is to configure the service:

$ kubectl apply -f k8s/php.secret.yaml
secret/php-secret created

Having run that, I saw the service config appear in Kubernetes dashboard, but it was sitting on orange (I'll spare you the screencap: you get the idea - there was orange instead of green). This was because I'm running the service in loadBalancer mode, with Minikube providing the load-balancer. Forthis to work, I also need to run minikube tunnel:

$ minikube tunnel
✅  Tunnel successfully started

📌  NOTE: Please do not close this terminal as this process must stay alive for the tunnel to be accessible ...

🏃  Starting tunnel for service php.

Note that it specifically sees my php service. Having run that, the board is green (screepcap this time):

OK so look at what port it's opened to the outside world. 9000. Not the 31000 I had set. Now: this saves me some work getting Nginx to use 31000 in prod mode and 9000 in dev mode so it's cool; but I don't understand what is going on here. But I don't care enough to drill into it.

Having done that, everything is working. If I hit the site in a browser I get the expected results:

Hello world from Symfony
Mode: prod
Pod name: php-7b57f5f6b9-sbqxk
DB version: 10.11.13-MariaDB-ubu2204

And if I reload, I can see different pods being used:

Hello world from Symfony
Mode: prod
Pod name: php-7b57f5f6b9-t8lgj
DB version: 10.11.13-MariaDB-ubu2204

Cool!

This all took a while - I started reading docs a week ago, and also worked on this yesterday (Sunday). Four days getting it to work; a solid day to write up. Doing the write-up (which basically required me to do everything a second time) really helped cement my knowledge. Now, I am not claiming any real expertise with Kubernetes, but I do think I have the starting-point knowledge I need as a dev to get my Dockerised app into Kubernetes in a prod environment. Provided I have someone who knows what they are doing to do the management of that prod environment.

I have some tidy-up to do with the source code, so I don't have the most up-to-date tag for it as yet, but the current state of the app is on Github at php8-on-k8s (tag 0.8 ATM).

Righto.

--
Adam

Friday, 16 May 2025

I got an email

G'day:

Yeah, not much happening here, I know. But I got a spam email and I could not help reply to it. And it's relevant to [shit that I know about], so I'm porting it here, and will send them the link.

The spammer is fishing for CFML resources, and references CFCamp… asking me if I'm going. Well: no; but they also ask this:

It would also be nice to share thoughts on where ColdFusion is headed

I see.

Obviously I have an answer to that. Here goes.

I answer this as someone who still uses CFML every day.

Where is ColdFusion headed? Down the toilet. It's in the bowl already, but the flush button has been pressed.

If you are still stacking your basket with CFML eggs, you're committing business suicide.

CFML is an irrelevant language in 2025. If your company is still using it, it's time to think of shifting your skillset to PHP or JS (JS being slightly more glamorous, but less mature. Still.). There are better languages than those two, obviously, but migrating your devs will be harder; so I recommend sticking with easy stuff. On the whole, CFML devs are not very good at what they do: they know their five (or is it six?) tags, and never push the boat out any further than that. This will be enough to port one's skills to easy and forgiving languages like the afore-mentioned (and maybe Python), but not beyond that. So migrating to other languages is gonna be difficult for them. I have first hand professional experience of this.

Don't get me wrong: there is nothing (ish) wrong with CFML as a language. The issue is no-one uses it, and the idea of paying for a language environment went the way of the dodo in the early 2000s. Lucee is an attempt to remediate the $$$ side of that, but its relevance to anything is orders of magnitude smaller than ColdFusion's, and ColdFusion's relevance is… close to zero in the broader web industry to start with. So Lucee's relevance is something to do with 0 / 0 (don't @ me, it's an analogy, not maths). In turn, the number of human resources out there - CFML devs - is very small compared to other languages. That pool is also stunted by the fact that most people who might start with CFML and can do better do indeed move on to different things. Moving actively away from CFML. So the ones that are left are largely aspirationless, or trapped in their circumstances. There are some very bloody good programmers out there using CFML - don't get me wrong - but they are very few, and very far between. And are the exception.

So it's very hard to build and maintain a CFML team. One can hire newbies and train them up, sure. But I think this is almost professionally immoral to do so: you are thinking only of yer own requirements, not thinking of the newbies' overall career. It's a dick move. The point here is that it's commercially difficult to maintain a functional CFML team. It's a risk one ought not undertake, or accidentally get involved in.

It's telling that recently the main stalwart of the CFML community - Ortus Solutions (ColdBox etc) - have begun to distance themselves from CFML: creating their own "BoxLang" which is a drop-in replacement for CFML, but one doesn't need to mention CFML when talking to clients. This move, to me, was the community death nell for CFML. The *Box community is basically "taking their ball and going home", as it were. Good luck to them.

I also know smaller agencies that use CFML for their core services, but they go out of their way to hide that from their clients due to CFML's industry toxicity. "We use a JVM language, and that gives you the stability of the Java platform". It's a stretch to describe CFML as a JVM language, but one does what one has to do, I guess.

Adobe sticks with CFML because they know which side their bread is buttered on, and there's a lot of legacy ColdFusion code in the US Govt and a few other large enterprises in USA. That's a lot of licences for Adobe, but not many opportunities for CFMLers.

I (genuinely, and in places dearly ~) like my friends in the CFML world, but I wish they would move on.

So - no - I will not be attending CFCamp this year. For the same reason I would not be attending FortranCamp, or COBOLCamp. This is not to say that CFCamp is not a great conference / meet-up / catch-up / jolly for people still in the CFML community, but its relevance is minimal for me these days.

HTH.

I'm hoping to have more to write here soon. For reasons I can't go in to quite yet…

Righto.

--
Adam

Sunday, 1 September 2024

Design patterns: singleton

G'day:

(Long time no see, etc)

The other day on the CFML Slack channel, whilst being in a pedantic mood, I pointed out to my mate John Whish (OG CFMLer) that he was using the term "singleton" when all he really meant was "an object that is reused". A brief chat in the thread and in DM ensued, and was all good. We both revisited the docs on the Singleton Pattern, and refreshed & improved our underatanding of it, and were better for it. Cool. The end. Short article.

...

..

Then the "well ackshually" crowd showed up, and engaged in a decreasingly meritorious diatribe in how there's no difference between a class that implements the Singleton Pattern, and an object that one happens to decide to reuse: they're both just singletons.

OK so the reasoning wasn't quite that daft (well from one quarter, anyhow), but the positioning was missing a degree of nuance, and there was too much doubling down in the "I'm right" dept that I just gave up: fine fellas, you do you. In the meantime I was still talking to John in DM, and I mentioned I was now keen to see how an actual singleton might come together in CFML, so there was likely a blog article ensuing. I predicted CFML would throw up some barriers to doing this smoothly, which is interesting to me; and hopefully other readers - if I have any still? - can improve their understanding of the design pattern.

I'll put to bed the "debate" first.

The notion of a singleton comes from the Singleton Pattern, which is one of the perennial GoF Design Patterns book. It's a specific technique to achieve an end goal.

What's the end goal? Basically one of code organisation around the notion that some objects are intended to be re-used, and possibly even more strongly: must be re-used in the context of the application they are running in. One should not have more than one of these objects in play. An obvious, oft-cited, and as it turns out: unhelpful, example might be a Logger. Or a DatabaseConnection. One doesn't need to create and initialise a new DatabaseConnection object every time one wants to talk to the DB: one just wants to get on with it. If one needed to instantiate the DatabaseConnection every time it was used, the code gets unwieldy, breaks the Single Responsibility Principle, and is prone to error. Here's a naïve example:

numbers = new DatabaseConnection({
    host = "localhost",
    port = "3306",
    user = "root",
    password = "123", // eeeeek
    database = "myDb"
}.execute("SELECT * FROM numbers"))

One does not wanna be doing all that every time one wants to make a call to the DB. It means way too much code is gonna need to know minutiae of how to connect to the DB. One will quickly point out the deets don't need to be inline like that (esp the password!), and can be in variables. But then you still need to pass the DB credentials about the place.

There's plenty of ways to solve this, but the strategy behind the Singleton Pattern is to create a class that controls the usage of itself, such that when an instance is called for, the calling code always gets the same instance. The key bit here is a class that controls the usage of itself[…] always[…] the same instance. That is what defines a singleton.

My derision in the Slack conversation was that the other participants were like "yeah but one can do that with a normal object just by only creating one of them and reusing it (muttermutterDIcontainer)". Yes. Absolutely. One can def do that, and often it's exactly what is needed. And DI containers are dead useful! But that's "a normal object […] and reusing it". Not a singleton. Words have frickin meanings(*). I really dunno why this is hard to grasp.

It's like someone pointing to white vase with blue art work on it, and going "this is my Ming vase". And when one then points out it says "IKEA" on the bottom, they go "doesn't matter. White with blue detail, and can put flowers in it. That's what a Ming vase is, for all intensive purposes (sic)". "You know a Ming vase is a specific sort of vase, right?". "DOESN'T MATTER STILL HOLDS FLOWERS". OK mate.

Digression, around that (*) I put above. I'm a firm believer in words are defined by their usage, not by some definition written down somewhere. A dictionary, for example, catalogues usage, it doesn't dictate usage. This is fine, but it's also less than ideal that words like "irregardless" end up in the dictionary because people are too…erm… "lacking a sense of nuance"… to get easy things right. This is pretty much where I am coming from here: "Singleton" means something; it'd be grand if the usage of it didn't get diluted due to people "lacking a sense of nuance" to get easy things right. And then to double-down on it is just intellectually-stunted, and not something I think we should be revelling in. I do also feel rather like Cnut vs the surf in this regard though.

Ah well.


Anyhoo, can I come up with a singleton implementation in CFML?

The first step of this didn't go well:

// Highlander.cfc
component {

    private Highlander function init() {
        throw "should not be runnable"
    }
}
// test.cfm    
connor = new Highlander()

writeDump(connor)

I'd expect an exception here, but both ColdFusion and Lucee just ignore the init method, and I get an object. Sigh.

This is easily worked-around, but already my code is gonna need to be less idiomatic than I'd like it to be:

public Highlander function init() {
    throw "Cannot be instantiated directly. Use getInstance"
}

Now I get the exception.

Next I start working on the getInstance method:

public static Highlander function getInstance() {
    return createObject("Highlander")
}
// test.cfm
connor = Highlander::getInstance()

writeDump(connor)

This still returns a new instance of the object every time, but it's simply a first step. To easily show whether instances are the same or different, I'm gonna given them an ID:

// Highlander.cfc
component {

    variables.id = createUuid()

    public Highlander function init() {
        throw "Cannot be instantiated directly. Use getInstance"
    }

    public static Highlander function getInstance() {
        return createObject("Highlander")
    }

    public string function getId() {
        return variables.id
    }
}
// test.cfm
connor = Highlander::getInstance()

writeDump(connor)

goner = Highlander::getInstance()

writeDump([
    connor = connor.getId(),
    goner = goner.getId()
])

 

See how the IDs are different: they're different objects.

We solve this by making getInstance only create the object instance once for the life of the class (not the object: the class).

public static Highlander function getInstance() {
    static.instance = isNull(static.instance)
        ? createObject("Highlander")
        : static.instance

        return static.instance
}

It checks if there's already an instance of itself that it's created before. If so: return it. If not, create and store the instance, and then return it.

Now we get better results from the test code:

 

Now it's the same ID. Note that this is not isolated to that request: it sticks for every request for the life of the class (which is usually the life of the JVM, or until the class needs to be recompiled). I'm altering my writeDump call slightly:

writeDump(
    label = "Executed @ #now().timeFormat('HH:mm:ss')#",
    var = [
        connor = connor.getId(),
        goner = goner.getId()
    ]
)

 

 

The ID sticks across requests. It's not until I restarts my ColdFusion container that the static class object is recreated, and I get a new ID:

 

One flaw in this implementation is that there's nothing to stop the calling code using createObject rather than using new to try to create an instance of the object. EG: this "works":

// test2.cfm
connor = createObject("Highlander")
goner = createObject("Highlander")

writeDump(
    label = "Executed @ #now().timeFormat('HH:mm:ss')#",
    var = [
        connor = connor.getId(),
        goner = goner.getId()
    ]
)

 

When I say this "works" I am setting the bar very low, in that "it doesn't error": it's not how the Highlander class is intended to be used though.

Oh: in case it's not clear why there's no exception here: it's cos when one uses createObject, the init method is not automatically called.

Can I guard against this?

Sigh.

OK, on Lucee I can do this with minimal fuss:

// Highlander.cfc
component {

    if (static?.getInstanceUsed !== true) {
        throw "Cannot be instantiated directly. Use getInstance"
    }

    variables.id = createUuid()

    public Highlander function init() {
        throw "Cannot be instantiated directly. Use getInstance"
    }

    public static Highlander function getInstance() {
        static.getInstanceUsed = true

        static.instance = isNull(static.instance)
            ? createObject("Highlander")
            : static.instance

        static.getInstanceUsed = false

        return static.instance
    }

    public string function getId() {
        return variables.id
    }
}

What's going on here? The conceit is that the class's pseudo-constructor code is only executed during object creation, and when we are creating an object via getInstance we disable the "safety" in the pseudo-constructor, but then re-enable it once we're done creating the instance. if we don't use getInstance, then the safety has either never been set - exception - or it's been set to false by an erstwhile call to getInstance - also exception.

Looking at that code, I can see that there's a race-condition potential with getInstance's setting/unsetting of the safety, so in the real world that code should be locked.

As I alluded to above: this code does not work in ColdFusion, because ColdFusion has a bug in that the pseudo-constructor is run even when running static methods, so the call to getInstance incorrectly calls the pseudo-constructor code, and triggers the safety check. Sigh. I can't be arsed coming up with a different way to work around this just for ColdFusion. I will raise a bug with them though (TBC). ColdFusion also has another bug in that static?.getInstanceUsed !== true errors if static.getInstance is null, as the === doesn't like it. I guess I'll raise that too (also TBC).

So. There we go. A singleton implementation.


PHP's OOP is more mature than CFML's, so a PHP implementation of this is a bit more succinct/elegant:

class Highlander {
    
    private ?string $id;
    
    private static self $instance;
    
    private function __construct() {
        $this->id = uniqid();
    }
    
    public function getId() : string {
        return $this->id;
    }
    
    public static function getInstance() : static {
        self::$instance = self::$instance ?? new static();
        return static::$instance;
    }
}

$connor = Highlander::getInstance();
$goner = Highlander::getInstance();

var_dump([
    'connor' => $connor->getId(),
    'goner' => $goner->getId()
]);

$transient = new Highlander();
array(2) {
  ["connor"]=>
  string(13) "66d457f9d4cd1"
  ["goner"]=>
  string(13) "66d457f9d4cd1"
}

Fatal error: Uncaught Error: Call to private Highlander::__construct() from global scope in /home/user/scripts/code.php:30
Stack trace:
#0 {main}
  thrown in /home/user/scripts/code.php on line 30

And that's that.

Righto.

--
Adam

Friday, 29 March 2024

OT: Starfield

G'day:

I finally relented on the Steam nagging screen, and am writing a review of Starfield, which I have been playing pretty much daily since it was released. I just noticed I'm writing an essay rather than a review, so figured I'll "own" it and post it here as well as give it to Steam/Bethesda.


Would you recommend this game other players? Yes.

There's some good gameplay in this, but around the edges. I wish the rating system on this had at least a "maybe" option as well as "yes" and "no". this is a 6/10 game (I'd rank Skyrim and Fallout 4 as 10/10 games, to compare).

The main quest is sterile and repetitive, and the main NPCs are on the whole obnoxious and unengaging. Companion AI doesn't seem improved over FO4: they are always just in the way, rather than being helpful. Too much dialogue is too repetitive. Note to the Bethesda scripting team: each canned script line should only be used once (maybe once per humanspace week or something, just not every time I come within 1m of the person). Silent NPCs would be better than irritating (and vaccuous) repetition. it detracts from immersion. Can't wait for Bethesda to integrate chatgpt-ish AI into NPC comms; it would mix it up a lot, and - I'm afraid - probably better than the current script-writing which is mediocre.

The faction quests are too short and linear, but more interesting than the main quest.

Inventory imbalance needs a look at. One needs tonnes of resources to play the game effectively as an outpost builder, but it's too hard to ferry about the place without mods to adjust / negate weight. It's not fun to be a cargo carrier (especially as the cargo capacity of ownable spacecraft are so low - at leat to start with, but they never get great).

Intersystem cargo transport is unusable on my rig, because as soon as I have more than one of them, the game blocks every few seconds (I guess it's doing cargo placement calculations? This only really needs to be done when arriving at an outpost, and could be rolled in to the load-screen-time, surely?), and it gets worse and worse with the more links added. I have read a lot of ppl have had this. The rest of the gameplay in the game is fine on this PC, so it's not just down to a badly spec'ed machine, I think: it's badly implemented code in the game. If cargo transport ran transparently in the background (like it does in FO4), then this would solve a lot of the inventory management issues. I just have a mod that zeros-out the weight of resources, so I carry them all around with me in my rucksack ;-)

Combat is fun, I'm not getting tired of that. And I like stealing the baddies' ships. That said, the need to use the scanner in combat to trigger some of the "social manipulations" is clunky. I know a lot of people have already told you that.

I also like planetary surveying (if a bit mindnumbing ;-), and I like building outposts, but they're a bit buggy. Still. After six months. But it's the part of the game I enjoy the most: finding the right spot to build, and then building. Object snapping could do with some work though.

The space part of the game is disappointing. There's no real travel, and no space exploration. It's just "fast travel exploration". Other games - even decades old - do this better.

I think the radiant quests in Starfield are fun though. It could do with a few more bases when doing the "kill the baddy" quests, but I'm not tiring of them yet. They are very "no-brainer", but it's telling they're more engaging than than the scripted quests. The scripted quests are a bit brain-dead and no real variation on "push the blue button so you can get to the red button… then to the yellow button…" sorta thing we had in Doom, 30 years ago. I know all RPG quests are basically like that, but it seems more apparent in SF than in FO4 or Skyrim. I think the level designers phoned-it-in a bit in Starfield.

I've played Starfield a lot (really a lot; less than I work and sleep, but more than anything else since it was released), but most of this is just as something I do with the sound off whilst I watch telly or YT, or listen to a book or use my brain and 50% of my attention for something else. This is a poor comparison to when I replay Skyrim or FO4, which I still actually enjoy.

I sure hope this is down to some vagary of the Starfield team composition or leadership, and not something that is going to bleed back in to the other franchises Bethesda own.

Right. I have some Ecliptic mercs to spacejack.

--
Adam

Sunday, 17 March 2024

CFML: solving a CFML problem with Java. Kinda.

G'day:

The other day on the CMFL Slack channel, Nick Petrie asked:

Anyone know of a native or plugin-based solution to pretty-formatting XML for display on the page? I'd like to output the XML nested this:
I'll use a simplified example> we wanna render this:
<aaa><bbb ccc="ddd"><eee/></bbb></aaa>
Like this:
<aaa>
    <bbb ccc="ddd">
        <eee/>
    </bbb>
</aaa

There's no native CFML way of doing this, but I figured "this is a solved problem: there'll be an easy way to do it in Java". I googled java prettier xml, and the first match was on the Baeldung website (Pretty-Print XML in Java) which is a site I trust to have good answers. But I checked that and a few others, and they all seem to be solving the problem the same way. So I decided to run with that.

The task at hand is to convert this to CFML (still using the Java libs to do the work, I mean, just runnable in CFML):

public static String prettyPrintByTransformer(String xmlString, int indent, boolean ignoreDeclaration) {

    try {
        InputSource src = new InputSource(new StringReader(xmlString));
        Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(src);

        TransformerFactory transformerFactory = TransformerFactory.newInstance();
        transformerFactory.setAttribute("indent-number", indent);
        Transformer transformer = transformerFactory.newTransformer();
        transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
        transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
        transformer.setOutputProperty(OutputKeys.INDENT, "yes");

        Writer out = new StringWriter();
        transformer.transform(new DOMSource(document), new StreamResult(out));
        return out.toString();
    } catch (Exception e) {
        throw new RuntimeException("Error occurs when pretty-printing xml:\n" + xmlString, e);
    }
}

To be very clear, this is not my code, it's from Pretty-Print XML in Java.

And also to be clear: I'm not gonna be doing anything unique or difficult or insightful or anything like that in this article. All I'm gonna do is to show how easy it is to convert native Java code to native CFML code, and this is a topical use case. It's gonna be a function with some object creation and some variable assigments. I'm gonna go line-by-line through that function above, and CFMLerise it. I just made that word up.

I'm gonna be using trycf.com to write and run this code, and the aim is to have a version that runs in both CF and Lucee.

First, let's run with the same function signature:

unformattedXml = '<aaa><bbb ccc="ddd"><eee/></bbb></aaa>' 

public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {

}

prettyPrintByTransformer(unformattedXml)

In each step I'll be using that same unformattedXml input value, and just running the function to ensure it's got no compilation errors and each new statement I add "works" (in that it doesn't have runtime errors). The function won't do anything useful until it's done.

In this first step note that I've given the latter two parameters sensible defaults. This is a change from the Java version.

First things first: I don't like how they've put that largely pointless try/catch in the Java code. To me that sort of error-handling should be in the calling code if needed, not embedded in the implementation. If the implementation errors-out: let it. The actual exception will be more useful than swallowing the real exception and throwing a contrived one.

I'll include each statement I'm converting as a comment above the CFML version. This is only so I can draw your focus to it. I'd never include comments of this nature in my actual code.

public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
    // InputSource src = new InputSource(new StringReader(xmlString));
    var xmlAsStringReader = createObject("java", "java.io.StringReader").init(xmlString) // new StringReader(xmlString)
    var src = createObject("java", "org.xml.sax.InputSource").init(xmlAsStringReader)
}

I could have done this without the intermediary variable, but CFML is quite cumbersome making Java object proxies, so I think the code is clearer spread over two statements.

    // Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(src);
    var document = createObject("java", "javax.xml.parsers.DocumentBuilderFactory").newInstance().newDocumentBuilder().parse(src)
    // TransformerFactory transformerFactory = TransformerFactory.newInstance();
    var transformerFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance()

Now: this statement did not work for me:

transformerFactory.setAttribute("indent-number", indent);

The CFML equivalent is exactly the same as it happens, but I was just getting "Not supported: indent-number" (CF) or "TransformerFactory does not recognise attribute 'indent-number'." (Lucee). I presume it's a library version difference, but I was fairly limited in my troubleshooting as I was running this on trycf.com. Although I did verify I was getting the same thing on my local CF2023 container too. I googled a bit and found a work around, but I'll come back to this a bit further down.

    // Transformer transformer = transformerFactory.newTransformer();
    var transformer = transformerFactory.newTransformer()
    /*
        transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
        transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
        transformer.setOutputProperty(OutputKeys.INDENT, "yes");
    */
    var outputKeys = createObject("java", "javax.xml.transform.OutputKeys")
    transformer.setOutputProperty(outputKeys.ENCODING, "UTF-8")
    transformer.setOutputProperty(outputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no")
    transformer.setOutputProperty(outputKeys.INDENT, "yes")
    transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", indent)

Once again, I need an intermediary variable here for the outputKeys class proxy. Just to save repetition in this case.

Also note the last line there is extra: it's the solution for the bit that wasn't working earlier. Easy.

    // Writer out = new StringWriter();
    var out = createObject("java", "java.io.StringWriter").init()
    // transformer.transform(new DOMSource(document), new StreamResult(out));
    var documentAsDomSource = createObject("java", "javax.xml.transform.dom.DOMSource").init(document)
    var outAsStreamResult = createObject("java", "javax.xml.transform.stream.StreamResult").init(out)
    transformer.transform(documentAsDomSource, outAsStreamResult)

More intermediary variables here.

    // return out.toString();
    return out.toString()

And that's it. I've not got much commentary in all that lot, because it's all so straight forward [shrug].

The end result in one piece is thus:

public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
    var xmlAsStringReader = createObject("java", "java.io.StringReader").init(xmlString)
    var src = createObject("java", "org.xml.sax.InputSource").init(xmlAsStringReader)
    
    var document = createObject("java", "javax.xml.parsers.DocumentBuilderFactory").newInstance().newDocumentBuilder().parse(src)
    
    var transformerFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance()
    var transformer = transformerFactory.newTransformer()
    
    var outputKeys = createObject("java", "javax.xml.transform.OutputKeys")
    transformer.setOutputProperty(outputKeys.ENCODING, "UTF-8")
    transformer.setOutputProperty(outputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
    transformer.setOutputProperty(outputKeys.INDENT, "yes")
    transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", indent)
    
    var out = createObject("java", "java.io.StringWriter").init()
    
    var documentAsDomSource = createObject("java", "javax.xml.transform.dom.DOMSource").init(document)
    var outAsStreamResult = createObject("java", "javax.xml.transform.stream.StreamResult").init(out)
    transformer.transform(documentAsDomSource, outAsStreamResult)

    return out.toString()
}

And a coupla test runs:

formattedXml = prettyPrintByTransformer(unformattedXml)
writeOutput("<pre>#encodeForHtml(formattedXml)#</pre>")
<aaa>
    <bbb ccc="ddd">
        <eee/>
    </bbb>
</aaa>
formattedXml = prettyPrintByTransformer(unformattedXml, 8, false)
writeOutput("<pre>#encodeForHtml(formattedXml)#</pre>")
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<aaa>
        <bbb ccc="ddd">
                <eee/>
        </bbb>
</aaa>

The output above is from Lucee. On CF we get this:

<?xml version="1.0" encoding="UTF-8"?><aaa>
        <bbb ccc="ddd">
                <eee/>
        </bbb>
</aaa>

Note this is nothing to do with the CFML code, it'll be some Java library variation on the Lucee and CF set-ups on trycf.com.

Right, so all this lot shows is that there's no reason to shy away from implementing a CFML version of some Java code you might find that solves a problem. So in turn, if you have some "algorithmic" issue that you wanna solve in CFML, don't shy away from googling for Java solutions, and checking how easy/hard it is to convert.

A runnable version of this code is @ trycf.com.

Righto.

--
Adam

Friday, 27 October 2023

I'm a big meany again

G'day

Chortle:

I'm new to cf development and recently joined the cfml slack community. I noticed some curmudgeonly posts from this fella at first and didn't think anything of it... I'm now realizing that he is an a**h*** to people all the time. For example, there was a question posted today that currently has 14 replies. A few of them are from others trying to answer OP's question; 7 are this guy explaining how stupid of a question it is. What gives? I'm sure he's a knowledgeable and respected developer, but the pedantry is off the charts!

DJgrassyknoll on Reddit

Guilty as charged I suppose.

However the charges are pretty specious. I think the thread in question was the one with a question (paraphrase) "is there a function in CFML that can convert JSON to XML?". This is a largely non-sensical question, and without further elaboration, can only be furnished with non-sensical and unhelpful answers. I don't wanna do that. I wanna get the person's problem solved. I did point this out to the bod in question, and explained why, with both a real-world analogy, plus a code demonstration! Here's the code version:

json = serializeJson({key="value"})
xml = xmlParse("<nowItIsXml>#encodeForXml(json)#</nowItIsXml>")

writeDump([
    json = json,
    xml = xml
])

Completely answers the question as stated, but is a completely frickin useless answer. Other than to illustrate the shortcomings of the question.

I've been answering people's CFML questions for years, and the biggest blocker to actually solving their problem is coercing what their actual problem is out from their brain. Really a lot of people don't seem to get we can't see the code in front of them, or the Jira ticket (sic) they're trying to deal with, or any other thoughts on the matter they might have had beyond what they type in as their question. So… not to be put off… I will explain this, and try to get a decent question out of them. So we all can understand it and then get their issue sorted. I will also often post links to How To Ask Questions The Smart Way and/or The SSCCE. I'm not going to apologise for that. It's good advice and if more people took it on board, they would improve, and also their chances of getting good help when they need it would improve. How is that bad?

We're all busy people, so the quicker and easier it is for us to get the info we need in front of us to help: the more likely we are to help. As one of the participants in that thread said "[when] people ask shitty questions, I mostly will just ignore it". That's your call pal. But I won't ignore them. I'll still try to help.

Ah - fuck it - people can say what they like about me. I don't care (and hey it's given me an in to blow some of the dust off this thing). But try to have a well-formed train of thought before doing so. And also poss sink some time into helping people to understand what it's like before stirring the pot in yer little sewing circle.

But yeah cheers for the new strapline quote.

Righto.

--
Adam

PS: props for using "curmudgeonly" though. Good word.