Saturday, 21 January 2023

PHP: returning to PHP and setting up a PHP8 dev environment

G'day:

I need to do some PHP work, and for that I need to have a PHP dev environment. I'm very rusty when it comes to PHP - I've not touched it for 2-3 years or so and my old brain doesn't hold on to things very well - and since that time I have shifted to using Docker for my environments anyhow. I've never used PHP in Docker before. So there's a challenge. And what's this? PHP is now up to version 8.2, with 8.3 being tested. The last time I touched PHP 7.2 was the new thing (we were still mostly on 5.5 at that time, that said).

Therefore I have a mini project ahead of me:

  • Get PHP8.2 running in a Docker container.
  • Get Nginx running in a different container, proxying requests to the PHP one.
  • Have Composer up and running.
  • So I can install PHPUnit.
  • And run some basic tests of the installation.
  • With code-coverage reporting on the tests (code coverage requires a debug module to be installed and running too).
  • Also get PHPMD and PHPCS running too.
  • Bonus: be able to run the tests from my IDE, on my host machine.

Success here will be to be able to view the HTML code coverage report, served by Nginx, showing code being covered by testing.

Full disclosure: I did all this a few nights ago, and I am repeating the exercise now for the purposes of this article.

Application file structure

This shows the file system layout I'm aiming for:

/var/www# tree -L 1
.
|-- docker
|-- html
|-- src
|-- test
`-- vendor

5 directories, 0 files
/var/www#
  • docker.Docker stuff like docker-compose.yml and sub-directories for the various containers' Dockerfiles and other config / assets are in here.
  • html. Files that will be served by Nginx go in here.
  • src. Application code goes here.
  • test. Test code goes here.
  • vendor. The app's Composer dependencies go in here.

This is all standard PHP-app stuff, except my personal decision of how to organise the Docker files.


PHP in a container

docker-compose.yml

The docker-compose.yml service definition is pretty simple:

version: "3"

services:
  php:
    build:
      context: php
      dockerfile: Dockerfile

    stdin_open: true
    tty: true

    volumes:
      - ..:/var/www

/var/www is the directory the container expected to see PHP stuff in, so I ran with it. It doesn't really matter.

.env

Oh I have a wee .env file too:

COMPOSE_PROJECT_NAME=php8

Just so the container names are a bit more on-point when they get created.

Dockerfile

The Dockerfile, on the other hand, is a bit complicated, and took me ages to google all the crap I needed to get together to make PHP 8.2 work in a container with a real-world set of extensions loaded, etc. Deep breath…

FROM php:8.2.1-fpm

RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "zip", "unzip", "git", "vim"]

COPY php.ini /usr/local/etc/php/php.ini

COPY --from=composer:latest /usr/bin/composer /usr/local/bin/composer

RUN pecl install xdebug && docker-php-ext-enable xdebug
COPY conf.d/xdebug.ini /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
COPY conf.d/error_reporting.ini /usr/local/etc/php/conf.d/error_reporting.ini

RUN apt-get install -y libicu-dev && docker-php-ext-configure intl && docker-php-ext-install intl

RUN ["apt-get", "install", "-y", "libz-dev", "libzip-dev"]
RUN docker-php-ext-configure zip && docker-php-ext-install zip
RUN docker-php-ext-configure bcmath && docker-php-ext-install bcmath
RUN docker-php-ext-configure pdo_mysql && docker-php-ext-install pdo_mysql
RUN docker-php-ext-configure opcache && docker-php-ext-install opcache

RUN curl -1sLf 'https://dl.cloudsmith.io/public/symfony/stable/setup.deb.sh' | bash
RUN ["apt-get", "install", "-y", "symfony-cli"]

WORKDIR /var/www
ENV COMPOSER_ALLOW_SUPERUSER 1

I'll go line-by-line, where there is anything noteworthy, or to explain my decisions.

  • php-fpm. I readily concede I have no idea what all the tag variants of PHP images are on Docker Hub. But I have used php-fpm in the past and know it to work. So: running with it. I am specifically not using the Alpine variant as it doesn't come with BASH, and life is too short to negotiate ASH instead. And I am not trying to economise on disk space for this container anyhow.
  • Baseline APT stuff. Composer needs zip, unzip and git (I learned this by trial and error). I need vim.
  • PHP doesn't have a php.ini file by default although needs it. It ships with php.ini-development and php.ini-production, and it's down to the dev to pick which to use. This file is base on php.ini-development, with the following changes (mostly from recommendations from Symfony › Performance › Use the OPcache Byte Code Cache):
    • realpath_cache_size = 4096k
    • realpath_cache_ttl = 600
    • date.timezone = Europe/London
    • opcache.enable=1
    • opcache.memory_consumption=256
    • opcache.max_accelerated_files=20000

    These are all just a matter of "uncommenting the example setting and tweak its value": normal php.ini stuff. No doubt I will further tweak that as I go, but that's a start.
  • Install Composer. It seems odd that Composer doesn't have the ubiquity that there's an APT package for it.
  • Install Xdebug. PHPUnit needs this for code coverage analysis. Plus at some stage I might start coding like a grown-up and use line debugging. Maybe.
  • I was following along the instructions on "Setup Step Debugging in PHP with Xdebug 3 and Docker Compose" to install Xdebug, and it suggested these settings to use. Yeah cool. I do not know any better. See below this list for the file contents.
  • All of this lot are just libraries that the cited PHP extensions need to be able to run.
  • The ultimate object of the exercise (well: the next part of the exercise) is to get Symfony installed in this app. Whilst setting up the PHP extensions I knew I would be wanting, I thought to look-up what Symfony would need too, and its guidance was to install the Symfony CLI and it would tell me (via symfony check:requirements. Hence installing this now. It was Symfony that reminded me to install all the highlighted extensions. handy. It's also handy that the PHP Docker image comes with those docker-php-ext-configure and docker-php-ext-configure utils, as it makes it a lot easier. There's also a bit of dependency-heck (not quite bad enough to use the work "hell" here) going on installing them, cos sometimes - like with libz-dev and libzip-dev - there are upsteam dependencies needed too. I think I got off pretty lightly here, just needing those two.
  • /var/www is where I want to land when I start a shell on the container.
  • I need to set this otherwise Composer complains about installing stuff as root. This'd be an issue in a production container, maybe. But it's not an issue on dev IMO.

Here are those PHP config files I mentioned above in the Xdebug bit:

# docker/php/conf.d/xdebug.ini

zend_extension=xdebug

[xdebug]
xdebug.mode=develop,debug,coverage
xdebug.client_host=host.docker.internal
xdebug.start_with_request=no
  • I added coverage to this, for PHPUnit.
  • The suggested setting for start_with_request was yes for this, but this meant IntelliJ would interrupt PHPUnit every time I ran my tests from the shell, so I've switched it off.
# docker/php/conf.d/error_reporting.ini
error_reporting=E_ALL

Normally I'd set this directly in php.ini, but I actually did this part of the config before I remembered about php.ini needing to be configured, so stuck with it.


After doing that lot I could build the container and bring it up, and run composer install:

/mnt/c/src/containers/php8/docker$ docker-compose build
[+] Building 4.0s (25/25) FINISHED
[...]
 => exporting to image                                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                                     0.0s
 => => writing image sha256:3e9dfd6aca1527f8d0906a0d9f2b2ec2c74ddc0bd9ea9d2b8f0d8b1dce773951                                                                                                                0.0s
 => => naming to docker.io/library/php8-php                                                                                                                                                                 0.0s

/mnt/c/src/containers/php8/docker$ docker compose up --detach
[+] Running 2/2
 ⠿ Network php8_default  Created                                                                                                                                                                            0.0s
 ⠿ Container php8-php-1  Started                                                                                                                                                                            0.5s

/mnt/c/src/containers/php8/docker$ docker exec -it php8-php-1 /bin/bash

/var/www# composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 49 installs, 0 updates, 0 removals
  - Downloading 
    [...]
Generating autoload files
41 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
/var/www#
/var/www# composer validate
./composer.json is valid

composer.json

Speaking of Composer, here's the composer.json file thusfar (don't worry too much about it: I'm including it here for completeness):

{
    "name" : "adamcameron/php8",
    "description" : "PHP8 containers",
    "type" : "project",
    "license" : "proprietary",
    "require": {
        "php" : "^8.2",
        "ext-iconv": "*",
        "ext-pdo_mysql": "*",
        "ext-mbstring": "*",
        "ext-intl": "*",
        "ext-json": "*",
        "ext-curl": "*",
        "ext-simplexml": "*",
        "ext-zip": "*",
        "ext-pcre": "*",
        "ext-ctype": "*",
        "ext-session": "*",
        "ext-tokenizer": "*",
        "ext-bcmath": "*",
        "ext-zend-opcache": "*",
        "monolog/monolog": "^3.2.0",
        "doctrine/dbal": "^3.5.3"
    },
    "require-dev": {
        "phpunit/phpunit": "^9.5.28",
        "phpmd/phpmd": "^2.13.0",
        "squizlabs/php_codesniffer": "^3.7.1"
    },
    "autoload": {
        "psr-4": {
            "adamcameron\\php8\\": "src/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "adamcameron\\php8\\test\\": "test/"
        }
    },
    "scripts" : {
        "test": "phpunit --testdox test",
        "phpmd": "phpmd src,test text phpmd.xml",
        "phpcs": "phpcs src test",
        "test-all": [
            "@test",
            "@phpmd",
            "@phpcs"
        ]
    }
}

The require section there is not stuff I needed for the install, it's also a bunch of baseline stuff I know I will need for the app I'm heading towards. I don't think there's anything surprising there.

I also already have some PHPUnit, phpmd and phpcs scripts in there. I'll get to those next…

Testing the PHP container

It would not be me if I didn't test stuff. I will admit I did not TDD the bits above, cos that would just be mad. However I wanted to test things worked, so have put a few tests in. Plus part of this is testing the debug module and PHPUnit work together as well.

PHPUnit

Here's the phpunit.xml.dist file:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd"
        colors="true"
        cacheResult="false"
        testdox="true"
        stopOnFailure="true"
        stopOnError="true"
        stopOnWarning="true"
        failOnWarning="true"
>
    <coverage>
        <include>
            <directory suffix=".php">src</directory>
        </include>
        <report>
            <html outputDirectory="html/test-coverage-report/" />
        </report>
    </coverage>
    <testsuites>
        <testsuite name="Integration tests">
            <directory>test/integration/</directory>
        </testsuite>
        <testsuite name="Unit tests">
            <directory>test/unit/</directory>
        </testsuite>
    </testsuites>
</phpunit>

All standard stuff, and there's nothing I can say about it that the docs don't already say.

Tests

A a few minimal tests, which hopefully are self-explanatory:

// test/integration/PhpTest.php

namespace adamcameron\php8\test\integration;

use PHPUnit\Framework\TestCase;

/** @testdox Tests of the PHP installation */
class PhpTest extends TestCase
{
    /** @testdox It has the expected PHP version */
    public function testPhpVersion()
    {
        $expectedPhpVersion = "8.2";
        $actualPhpVersion = phpversion();
        $this->assertStringStartsWith(
            $expectedPhpVersion,
            $actualPhpVersion,
            "Expected PHP version to start with $expectedPhpVersion, but got $actualPhpVersion"
        );
    }
}
// test/integration/ComposerTest.php

namespace adamcameron\php8\test\integration;

use PHPUnit\Framework\TestCase;

/** @testdox Tests of the Composer installation */
class ComposerTest extends TestCase
{
    /** @testdox It passes composer validate */
    public function testComposerValidates()
    {
        exec("composer validate 2> /dev/null", $output, $returnCode);
        $this->assertEquals(
            0,
            $returnCode,
            "Composer validate failed: " . implode("\n", $output)
        );
    }
}

These two test three things: PHP is running the version I expect; Composer is happy it's configured properly; and PHPUnit itself is running otherwise all this would go splat.

For the code coverage testing I need some source code to test:

// src/Greeter.php

namespace adamcameron\php8;

class Greeter
{
    public const FORMAL = 1;
    public const INFORMAL = 2;

    public static function greet(string $name, int $style = self::FORMAL): string
    {
        if ($style === self::FORMAL) {
            return "Hello, $name";
        }
        return "Hi, $name";
    }
    
}

And a test:

// test/unit/GreeterTest.php

namespace adamcameron\php8\test\unit;

use adamcameron\php8\Greeter;

use PHPUnit\Framework\TestCase;

/** @testdox Tests of the Greeter class */
class GreeterTest extends TestCase
{
    /** @testdox It greets formally */
    public function testFormalGreeting()
    {
        $name = "Zachary";
        $expectedGreeting = "Hello, $name";
        $actualGreeting = Greeter::greet($name, Greeter::FORMAL);
        $this->assertEquals(
            $expectedGreeting,
            $actualGreeting,
            "Expected greeting to be $expectedGreeting, but got $actualGreeting"
        );
    }

    /** @testdox It greets informally */
    public function testInformalGreeting()
    {
        $this->markTestSkipped("skipping this so the coverage report is more interesting");
        $name = "Zachary";
        $expectedGreeting = "Hey, $name";
        $actualGreeting = Greeter::greet($name, Greeter::INFORMAL);
        $this->assertEquals(
            $expectedGreeting,
            $actualGreeting,
            "Expected greeting to be $expectedGreeting, but got $actualGreeting"
        );
    }
}

Note how I am skipping one of the tests. This is so code coverage is not 100%.

Running the tests

root@e8896f5d5bd6:/var/www# composer test
> phpunit --testdox test
PHPUnit 9.5.28 by Sebastian Bergmann and contributors.

Tests of the Composer installation
  It passes composer validate

Tests of the PHP installation
  It has the expected PHP version

Tests of the Greeter class
  It greets formally
  It greets informally

Time: 00:05.675, Memory: 10.00 MB

Summary of non-successful tests:

Tests of the Greeter class
  It greets informally
OK, but incomplete, skipped, or risky tests!
Tests: 4, Assertions: 3, Skipped: 1.

Generating code coverage report in HTML format ... done [00:01.492]
root@e8896f5d5bd6:/var/www#

Cool! It all worked. Let's have a look at the code coverage report. Because I don't have Nginx installed yet I can't browse to it, but I can just open the file in a browser:

I've drilled down the report slightly to show the file I was testing. It's correctly showing that I have only tested one path in the logic. Excellent. This proves that the Xdebug extension is running.

phpmd and phpcs

I've installed these too, and have used a fairly stock config file for each (see: phpmd.xml and phpcs.xml). Running them is dead boring as there's hardly any code, and IntelliJ makes sure it's formatted well:

/var/www# composer phpmd
> phpmd src,test text phpmd.xml
/var/www# composer phpcs
> phpcs src test

FILE: /var/www/src/Greeter.php
------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
------------------------------------------------------------------------------------------
 18 | ERROR | [x] The closing brace for the class must go on the next line after the body
------------------------------------------------------------------------------------------
PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
------------------------------------------------------------------------------------------

Time: 2.51 secs; Memory: 6MB

Script phpcs src test handling the phpcs event returned with error code 2
/var/www#

Ha! I didn't actually expect that. I clearly didn't run this before I did my final commit. If you scroll up to the Greeter.php file, it's complaining that there's an empty line between the last method closing brace and the class's closing brace:

That breaks one of PRS-12's rules. Fair cop. And hey: a good test that it's working!


Nginx in a container

docker-compose.yml

The relevant bit is this:

nginx:
  build:
    context: nginx
    dockerfile: Dockerfile

  ports:
    - "8008:80"

  stdin_open: true
  tty: true

  volumes:
    - ../html:/usr/share/nginx/html/

Nothing interesting there.

Dockerfile

FROM nginx:alpine
WORKDIR /usr/share/nginx/
COPY ./nginx.conf /etc/nginx/nginx.conf
COPY ./sites/ /etc/nginx/sites-available/
COPY ./conf.d/ /etc/nginx/conf.d/
CMD ["nginx"]
EXPOSE 80

Also nothing noteworthy here. Everything is in the config files.

Nginx config files

I freely admin to pretty much lifting these from other projects I already had. I don't really know what I'm doing with Nginx. I learn enough to achieve some goal, then I forget it all within about 5min.

// docker/nginx/nginx.conf
user  nginx;
worker_processes  4;
daemon off;

error_log  /var/log/nginx/error.log debug;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    access_log  /var/log/nginx/access.log;
    sendfile        on;
    keepalive_timeout  65;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-available/*.conf;
}
// docker/nginx/conf.d/default.conf
upstream php-upstream {
    server php:9000;
}
// docker/nginx/sites/default.conf
server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    server_name localhost;
    root /usr/share/nginx/html;
    index index.html index.php;

    location / {
        autoindex on;
        try_files $uri $uri/ /index.php$is_args$args;
    }

    location ~ \.php$ {
        try_files $uri /index.php =404;
        fastcgi_pass php-upstream;
        fastcgi_index index.php;
        fastcgi_buffers 16 16k;
        fastcgi_buffer_size 32k;
        fastcgi_param SCRIPT_FILENAME /var/www/html/$fastcgi_script_name;
        fastcgi_read_timeout 600;
        include fastcgi_params;
    }

    location ~ /\.ht {
        deny all;
    }
}

I hope it doesn't seem dismissive or that I'm tired of writing in that I add nothing here. I literally don't know what most of that stuff does, other than where it's obvious.

A test PHP file to browse to

I need to be able to test that Nginx is passing stuff to PHP:

<?php
// html/test.php
phpinfo();

Having done all that, I rebuild the containers (and note I now have an Nginx one as well), and bring them up.

If I browse to http://localhost:8008/test.php, I get this:

And if I run the PHPUnit tests again, I can now browse to the report via http://localhost:8008/test-coverage-report/. Cool.


Success

OK so I'm gonna consider that a victory. I did some tinkering around in IntelliJ and I can run the unit tests from in there as well, all via drilling into the Docker container and running it from in there, and presenting the results in IntelliJ:

However that took more dicking around than I can be arsed with re-doing right now. I needed a coupla extensions installed, and set the PHP interpreter to be locatable via the config in docker-compose.yml. It's handy anyhow. I need my team to set all this up, so I'll get the first one of them I dump all this on to work it out and write it down, and I'll report back.

I've linked to all the individual files as I reference them, but you could also clone the repo, checkout the 1.0 tag, and you should be able to set this up locally and have a look if you so choose. There are full instructions in the readme.md file. I have had one of my team test the instructions out on their own PC, and they seemed to have worked.

Righto.

--
Adam

Friday, 20 January 2023

PHP: PrimaryReadReplicaConnection - configuration / usage example

G'day:

I've been dusting off my out-of-date PHP skills (such as they are), and I had a right arse of a time finding any documentation for PrimaryReadReplicaConnections via Google, so I'm adding something here for the next time I need to find it. Because knowing me I won't remember.

Back when I was last doing PHP for a living, DBAL had a MasterSlaveConnection class. This was used to create a single composite connnection object that comprising one primary connection - likely to a read/write DB user - and an array of secondary connections - likely to read-replicas of the same. The way it works is that for read-only operations (SELECTs basically). The reasoning being that this would ease up traffic on the primary DB for queries that don't need to be done on it, using the read replicas instead. But for writes, it uses the primary. It's a good idea.

I'm returning to PHP and I need to mentor my team, so I've been writing some code examples to show them how things work in PHP. I googled "MasterSlaveConnection" to RTFM, and found this issue on Github: "RFC: Rename MasterSlaveConnection". OK so MasterSlaveConnection connection has been renamed to PrimaryReadReplicaConnection. Makes sense. So I googled that instead. Frickin nothing. Nothing useful anyhow. Some stuff on setting up a connection in a Doctrine YAML file, and some other references, but I can't find actual docs with an actual useful example of "do it like this".

Before I start, here's a vanilla DBAL connection example for context:

private function getDbalConnection() : Connection
{
    $parameters = $this->getConnectionParameters();
    return DriverManager::getConnection([
        'dbname' => $parameters->database,
        'user' => $parameters->username,
        'password' => $parameters->password,
        'host' => $parameters->host,
        'port' => $parameters->port,
        'driver' => 'pdo_mysql'
    ]);
}

And here's the equivalent for a PrimaryReadReplicaConnection:

private function getPrimaryReadReplicaConnection() : PrimaryReadReplicaConnection
{
    $parameters = $this->getConnectionParameters();
    return DriverManager::getConnection([
        'wrapperClass' => PrimaryReadReplicaConnection::class,
        'driver' => 'pdo_mysql',
        'primary' => [
            'host' => $parameters->host,
            'port' => $parameters->port,
            'user' => $parameters->username,
            'password' => $parameters->password,
            'dbname' => $parameters->database
        ],
        'replica' => [[
            'host' => $parameters->readOnlyHost,
            'port' => $parameters->port,
            'user' => $parameters->username,
            'password' => $parameters->password,
            'dbname' => $parameters->database
        ]]
    ]);
}

It's pretty basic really. As with a normal DBAL connection one still uses the DriverManager::getConnection factory method to create a connection.

The difference is with the PrimaryReadReplicaConnection case is one needs to pass the deets of the primary and all the replicas instead of just the one set of connection config:

  • wrapperClass: the class name of the PrimaryReadReplicaConnection class.
  • driver: which DB I'm using.
  • primary; an associative array of the usual connection params for the connection to the primary (read/write) DB connection.
  • replica; an indexed array of associative arrays of connection params to the read replicas. One can pass as many of these as one likes, and I believe the connection wrapper controls how to pick one to sue for a given query. I'm only using one here because I only have one read replica.

I think that replica key should be plural, right? Oh well: can't always get it right I guess.

Here are some tests describing its operations / behaviour:

/** @testdox it will use a read-only connection if it can */
public function testPrimaryReadReplicaConnectionReadOnlyConnection()
{
    $connection = $this->getPrimaryReadReplicaConnection();

    $result = $connection->executeQuery("SELECT @@VERSION");

    $this->assertFalse($connection->isConnectedToPrimary());
}

false because it's not connected to the primary: it's connected to a replica because it's only doing reading.

/** @testdox a connection will start on a replica then change to the primary and stay there after a write operation */
public function testPrimaryReadReplicaConnectionSwitchToPrimary()
{
    $testName = "TEST_NAME_" . uniqid();

    $connection = $this->getPrimaryReadReplicaConnection();

    $this->assertFalse($connection->isConnectedToPrimary(), "Should start on a replica");
    $connection->beginTransaction();
    try {
        $this->assertTrue($connection->isConnectedToPrimary(), "Should be on the primary in a transaction");
        $connection->executeStatement(
            "INSERT INTO features (name,value) VALUES (:name, :value)", [
            "name" => $testName,
            "value" => "TEST_VALUE",
        ]);

        $newRow = $connection->executeQuery("SELECT value FROM features WHERE name = ?", [$testName]);
        $this->assertEquals("TEST_VALUE", $newRow->fetchOne());
        $this->assertTrue($connection->isConnectedToPrimary(), "Should still be on the primary after a read following a write");
    } finally {
        $connection->rollBack();
    }
    $this->assertTrue($connection->isConnectedToPrimary(), "Should still be on the primary after a rollback");

    $connection = $this->getPrimaryReadReplicaConnection();
    $this->assertFalse($connection->isConnectedToPrimary(), "A new connection should start on a replica");
}

This second one shows something important. A given connection might start on a read replica, but once there's been a first write operation (in this case the transaction singles some writing about to happen, intrinsically) - so the connection switches to the primary writeable connection - it stays on that primary connection. This is for the sake of data reliability: if one writes some data to primary and then tries to read it back later (for whatever reason), there'd be no guarantee the read replica(s) will have received the updates themselves yet, so it's safer to stay on the primary. Good thinking.

And a new connection within the same code will start afresh using a replica if it can.

OK, that's it. I just wanted this noted down some place.

Righto.

--
Adam

Thursday, 8 December 2022

I need another two devs

G'day:

On hold

I am still looking for two devs, but the exact job spec is being revised. So… well… ignore this for now.

I've previously posted similar offerings:

We are growing our team some more, this time looking for two devs. Previously we'd only been hiring senior / very experienced CFML engineers; this time I'm happy to talk to more mid-level devs who might like to join a team of seniors and take an opportunity to improve their CFML skills, and their general engineering skills.

We have a B2B app running on CFWheels (but don't let that worry you; very little of the work really involves anything to do with Wheels itself), that requires ongoing support, maintenance, and new development.

We are starting to port away from CFML and replatforming on Kotlin, and there'll be opportunity to come along on that ride as well, although the primary focus in the medium term is supporting the CFML platform. This is the platform that makes all our money and pays all our wages, and is under active development still. I don't want to make this sound like this is just a maintenance role on a legacy platform because we have a shiny new platform too. That's not the case at all. This is a strong CFML job, and we all do CFML every day.

The details of the role - which I've lifted from our ad: "Application Developer (UK)" - are:

Competencies:

  • Experience with CFML.
  • Experience maintaining and building on existing legacy applications.
  • Experience developing new web applications/web services.

Highly desirable:

  • Some exposure to test automation (eg: unit testing).
  • Exposure to OOP development practices in CFML.
  • Some knowledge of design principles such as MVC, SOLID and other common design patterns.
  • Experience with React.js or similar client-side application technology.
  • Experience with TDD practices.
  • Some familiarity with Dockerised development environments.
  • Some familiarity with Agile principles, and experience delivering value in an Agile fashion.
  • Experience with CFWheels.

Logistics-wise this is a permanent (ie: we are not offering it to contractors) remote-first position within the UK. I will not consider any other "unofficial" attempts to work from an offshore location. You must be resident in the UK, and be able to work here. We have an office but I've never set foot in it. But if you want to work in Bournemouth, there's a desk for you there if that's your thing. The non-dev ppl in that office are all nice :-).

Secondly, we are only able to hire employees who are able to live and work in the UK without visa sponsorship.

We have two openings for this role at the moment.

I see this as a good role for a person who's reasonably competent in CFML but feel there's room to improve, and would like the opportunity to join an existing experienced team whose aim is to enhance everyone's game. Maybe you're comfortable with CFML but not getting a chance to stretch yourself in your current role? Drop me a line.

If you want to have a chat about this, you can ping me in the CFML Slack channel (if for some reason you're a CFML dev reading this and not already in here, you can join via cfml-slack.herokuapp.com). Or you can send yer CV through to the email address on the job spec page linked-to above. I'm not interested in talking to recruiters for now, just in case you are one (and reading this?). For now I'm only wanting to talk to people in the dev community directly.

Righto.

--
Adam

Saturday, 19 November 2022

Off-topic / no technical content: I have a dilemma

G'day:

This is off-topic and has no technical content to it at all. It is about a personal situation I have. This is the only place I have to post large wodges of text publicly, hence putting it here. And hey it's my blog: I'll post what I want ;-)

I have a dilemma.

Context / history

My neighbour and I share a front door to the street, and we had got onto nodding terms with each other. When her new baby arrived I fussed a bit over the new wee girl and got to talking more. My neighbour has three girls, approximately 15, 8, <1. They live in a flat that is at most 2 bedrooms. It will be like mine, which is to say it will be very very basic.

She is on a key meter for her electricty, and these display the balance on the meter, which is inside the front door, so I pass it daily. Often (once a month? Every coupla months?) it's in the red, and their power is off.

We each pay for one of the lights in the shared area (stairs, hallway to the front door), and I have been making a point of giving her £20 every few months to cover her share of it.

She has had bailiffs around chasing her ex, who owes the council money for parking fines.

She has had the police around to escort her current partner away from the flat, as he was being abusive. He is, btw, back.

Someone from a charity brought a bag of food around "for the lady who lives upstairs with the kids. She looks like she could do with some food".

I think it's reasonable to say she's struggling with life.

Recently she came to me in a flap saying she hd a debt she needed to pay now, and she couldn't and could she borrow £20? I offered £50, and she did not hesitate to take it. I told her she did not have to pay me back as I could afford it, and she needed it more than me. This is not terribly altruistic of me: I give £50 to charity each month, and that month I gave it to her instead. [shrug].

A few weeks later she texted me saying she needed another £20 in her account as she needed to pay a bill urgently. And she'd pay me back. I transferred £50, and reassured her she didn't need to pay me back, but that we can't keep operating like this.

She's knocked on the door to give me £20 back - I reiterated she didn't need to but she insisted - but within the day she meekly came back and asked if she could take it back again. Because reasons. Fine.

This is all fine. Thusfar.

However now she is texting me every week almost for more money, and I have drawn a line under this enterprise saying "look, 'borrowing' money from your neighbour is not a workable way of solving your issues. I don't know what is, but this has to stop. Please don't ask me for more money".

The calls/texts continue. I am ignoring them. She is still friendly when I meet her on the landing or on the street.

She has actually paid back the £120 she ended up "owing" me. I asked he to keep it: she refused.

Here is my dilemma

She's broke. The UK is not a good place to be broke. Social services are struggling, food banks are struggling. Really a lot of people are struggling. I don't want anyone reading this as anything as a fact that is contextually significant: she is from an eastern European country, her English is not great and I suspect her education is not great either. She does not appear to have any family over here.

On the other hand I am not struggling. I can afford to just give her the money (and I can also afford to also keep giving £££ to charity each month too if I wanted to). I'm not a fountain of cash, but £50 here or there doesn't matter.

Theoretically I stand by my position of "asking yer neighbour for money is not a sustainable solution to your problems". But she has kids who need to eat & have a roof & be warm. Should theoretical economics trump hungry kids?

I have found myself thinking "this is not my problem" or "this should not *be* my problem". But it's my neighbour. Someone in my community. Who needs help and has asked for help.

Whether or not the govt or local council should be dealing with this is neither here nor there. We have a shit, evil (actually evil) government, and we know they don't give a shit about poor people. I can't change the govt (other than one vote at a time), but I can change this situation maybe.

I feel I am being more "middle-class" than I am being a caring person. But all the points above nag at me. I'm currently holding the position of "no, no more money; I am not the solution to your problems". But I suspect I might be mistaken, or being overly "priveleged" about this because I can afford to prevaricate over principles vs hunger.

Thoughts? And if you don't wanna put them publicly here, my email address is on the blog's Communications policy page. Or a lot of my readers know other channels to ping me on and discuss.

Righto.

--
Adam

Thursday, 3 November 2022

Kotlin: more operator overloading

G'day:

The Kotlin koans are still focusing on operator overloading, so so am I. Previously (Kotlin: overriding operators), I had a mess around with overloading:

This evening I'm gonna look at a coupla others.

Plus operator

Like… just the + operator. As in a + b.

This is a daft example, but I'm gonna define a buildable Critter class. A Critter can have a number of arms, legs and heads. Why? Oh I don't know. It just seemed funny to me at the time.

To do this I need to overload the plus operator multiple times:

class Critter(){
    private var heads = mutableListOf<Head>()
    private var arms = mutableListOf<Arm>()
    private var legs = mutableListOf<Leg>()

    operator fun plus(head: Head) : Critter {
        heads.add(head)
        return this
    }
    operator fun plus(arm: Arm) : Critter {
        arms.add(arm)
        return this
    }
    operator fun plus(leg: Leg) : Critter  {
        legs.add(leg)
        return this
    }
}

I'm overloading the plus operator three times. One each for Critter + Head, Critter + Arm, Critter + Leg. Note also that I'm specifically returning the Critter itself from these operations, so that further operations can be chained.

For the purposes of this exercise, the Arm, Leg, Head classes are a bit light on implementation:

class Arm
class Leg
class Head

For the test, I've also added a helper method:

fun describe()= """
        Critter with ${heads.size} ${English.plural("head", heads.size)},
        and ${arms.size} ${English.plural("arm", arms.size)},
        and ${legs.size} ${English.plural("leg", legs.size)}
    """.trimIndent().replace("\n", " ")

(That pluralising thing is a third-party library I dug up. No need to worry about that just now).

And now we can have the rest of our test code (all the above was the "arrange" part of the test):

val critter = Critter()
critter + Head()
critter + Arm() + Arm()
critter + Leg() + Leg() + Leg() + Leg()

critter.describe() shouldBe "Critter with 1 head, and 2 arms, and 4 legs"

It's silly, but I quite like that.

For the sake of completeness, I've tested the chaining works with different types:

fun `it can mix-up the body parts` () {
    val critter = Critter() + Head() + Arm() + Leg() + Leg() + Arm() + Leg() + Head() + Leg()

    critter.describe() shouldBe "Critter with 2 heads, and 2 arms, and 4 legs"
}

Invoke operator

I've messed around with this sort of thing with PHP's __invoke before. The invoke operator function represents the () operator one calls functions with; IE: if f is the function, one uses the () operator to call it: f(). Implementing this operator on a class lets one call objects like methods. Should one want to do that. Which, admittedly, is pretty rarely. But still.

class Logger {
    private var _log = mutableListOf<String>()
    val log: List<String>
        get() = _log

    operator fun invoke(message: String) {
        _log += message
    }
}

Here I have a logger class whose objects can be called as methods:

val logMessage = Logger() // logMessage is an object: an instance of Logger
logMessage("This is a log message")
logMessage("This is another log message")

The bit that effects this is the invoke function.

And the assert part of the test:

logMessage.log shouldBe listOf("This is a log message", "This is another log message")

The next thing I thought to try is how I might have a variadic invoke function: one I can pass any number of arguments to. This example is a bit contrived, but it shows it. Here a Task is an object that is initialised with a lambda, and when the task is invoked as a function, it runs the lambda with whatever the invocation is passed:

class Task(var lambda : (args: Array<String>) -> List<String>) {
    operator fun invoke(vararg args: String) : List<String> {
        return lambda(arrayOf(*args))
    }
}

One thing that is annoying is that a lambda can't be defined as having a vararg parameter, so I need to use an array of things there. However the invoke function can use vararg, so that's cool.

The * there is the spread operator, which converts the varargs passed to the invoke call to an array that the lambda needs. I could not find any other docs for this other than a reference in the vararg docs linked in the previous paragraph.

So I create my task with its lambda:

val task = Task { args -> args.toList() }

And I check the results:

val taskResult = task("This", "is", "a", "variadic", "invoke", "handler")
taskResult shouldBe listOf("This", "is", "a", "variadic", "invoke", "handler")

Both those lines look similar, so it's important to note the first one is passing six separate arguments to task (which is an object remember, not a function!), and the second line is one list with six items. The lambda coverted the varargs from the invoke call to the list it returned.


And that is as far as I got tonight, so I'll leave off here.

Here's the code:

Righto.

--
Adam

Monday, 31 October 2022

Kotlin: ranges

G'day:

Yet another example of me starting a Kotlin koans exercise, seeing the first interesting word in the koans task description and getting lost in researching how the thing works.

Test class names

This is just an aside. You know how I've been showing all these test methods like this (spoiler for the next section):

@Test
fun `a range of integers should be an IntRange`() {
    val range = 1..10

    range.shouldBeInstanceOf<IntRange>()
}

Where the method name is in back-ticks and a human-readable sentence? On a whim I went "I wonder if…" and I tried this:

internal class `Tests of ranges` {

    @Nested
    inner class `Tests of ranges on Ints` {
    // etc

And it only bloody works on class names too! How cool is that?

In the test output that lot shows like this:


Ranges

From the docs:

A range defines a closed interval in the mathematical sense: it is defined by its two endpoint values which are both included in the range.

Let's look at some tests that show how they work:

fun `it should return true for a value in a range`() {
    val range = 1..10
    val value = 5

    (value in range) shouldBe true
}
fun `it should return false for a value not in a range`() {
    val range = 1..10
    val value = 11

    (value in range) shouldBe false
}
fun `it should return true for values at the boundaries of the range`() {
    val range = 1..10

    (1 in range) shouldBe true
    (10 in range) shouldBe true
}

Those are pretty straight forward. What type is it?

fun `a range of integers should be an IntRange`() {
    val range = 1..10

    range.shouldBeInstanceOf<IntRange>()
}

And how to use it?

fun `it can be iterated over`() {
    val range = 1..10
    var list = mutableListOf<Int>()

    list = range.fold(list) { acc, i -> acc.apply { add(i) } }

    list shouldBe listOf(1,2,3,4,5,6,7,8,9,10)
}

It can have a step:

fun `it should return true for values at the steps`() {
    val range = 1..10 step 2

    listOf(1,3,5,7,9).forEach {
        (it in range) shouldBe true
    }
}

Also be mindful of what elements are in the range:

@Test
fun `the upper bound might not be included`() {
    val range = 1 until 10 step 2

    (10 in range) shouldBe false
}
fun `it should return false for values not at the steps`() {
    val range = 1..10 step 2

    listOf(2,4,6,8,10).forEach {
        (it in range) shouldBe false
    }
}

Note the type on a stepped-range is different:

fun `a stepped range is an IntProgression`() {
    val range = 1..10 step 2

    range.shouldNotBeInstanceOf<IntRange>()
    range.shouldBeInstanceOf<IntProgression>()
}

They work in reverse:

fun `a step can be downwards`() {
    val range = 10 downTo 1 step 2

    listOf(10,8,6,4,2).forEach {
        (it in range) shouldBe true
    }
    (1 in range) shouldBe false
}

And one can access some properties about the range:

fun `it has some properties`() {
    val range = 1..10 step 2

    range.first shouldBe 1
    range.last shouldBe 9
    range.step shouldBe 2
}

Note how it's the max value here, not the upper bound of the range.

And they can be reversed (and by inference here, compared):

fun `it can be reversed`() {
    val range = 1..10 step 2
    val reversed = range.reversed()

    (reversed == (9 downTo 1 step 2)) shouldBe true
}

One can make a range of doubles, but they're not iterable:

fun `it does not implement iterator like a range of ints does`() {
    val intRange = 1..10
    intRange.shouldBeInstanceOf<Iterable<*>>()

    val doubleRange = 1.0..10.0
    doubleRange.shouldNotBeInstanceOf<Iterable<*>>()
}

However one can still check whether something is within the range:

fun `its elements can be compared to other values`() {
    val range = 1.0..10.0
    val value = 5.5

    (value in range) shouldBe true
}

Ranges of chars are more useful (I guess…):

fun `a range of chars can have a step`() {
    val range = 'z' downTo 'a' step 2

    listOf('z','x','v','t','r','p','n','l','j','h','f','d','b').forEach {
        (it in range) shouldBe true
    }
}

One can have ranges of strings, but they don't seem to support stepping or downto

fun `a range of strings can have a step`() {
    val range = "a".."f"

    listOf("a","b","c","d","e","f").forEach {
        (it in range) shouldBe true
    }
}

But they can be multi-char:

fun `the strings in the range can be multi-character`() {
    val range = "aa".."cc"

    listOf("aa","ab","ac","ba","bb","bc","ca","cb","cc").forEach {
        (it in range) shouldBe true
    }
}

I tried to be clever and create a range of strings and then reverse it, but Kotlin sez no:

I can make a range of enum elements (which I thought was at least a bit clever of me to try):

@Test
fun `a range of MaoriNumbersEnum should not allow other elements from the enum`() {
    val range = MI.TAHI..MI.WHA // tahi, rua, toru, wha

    (MI.RUA in range) shouldBe true
    (MI.RIMA in range) shouldBe false // rima is 5
}

@Test
fun `a range of enums is not iterable` () {
    val range = MI.TAHI..MI.WHA

    range.shouldNotBeInstanceOf<Iterable<*>>()
}

But as you can see, they too aren't iterable.

I think I need to set myself an assignment to write extension functions so I can iterate over my Maori numbers. But not this evening.

Lastly - and so I can answer the koans question - any class that implements Comparable can be used in a range.

Here's my MyDate class back from the previous article:

data class MyDate(val year: Int, val month: Int, val dayOfMonth: Int) : Comparable<MyDate> {
    override fun compareTo(other: MyDate): Int {
        return when {
            year != other.year -> year.compareTo(other.year)
            month != other.month -> month.compareTo(other.month)
            else -> dayOfMonth.compareTo(other.dayOfMonth)
        }
    }
}

And using it in a range:

fun `a range of objects implementing Comparable can be used as a range`() {
    val testDates = MyDate(1955, 3, 25)..MyDate(1955, 3, 28)

    testDates.start shouldBe MyDate(1955, 3, 25)
    testDates.endInclusive shouldBe MyDate(1955, 3, 28)

    val restDay = MyDate(1955, 3, 27)
    (restDay in testDates) shouldBe true
}

(26 bonus points for anyone who can work out what that date range is. There is a clue in there. And I doubt anyone who is not an NZ cricket fan has a hope in hell of getting it).

It looks like making an iterable range of MyDate objects isn't that hard (esp if I get Copilot to do a lot of it). First I need to implement a "next" method in MyDate:

data class MyDate(val year: Int, val month: Int, val dayOfMonth: Int) : Comparable<MyDate> {
    private val date = LocalDate.of(year, month, dayOfMonth)

    override fun compareTo(other: MyDate): Int {
        return when {
            year != other.year -> year.compareTo(other.year)
            month != other.month -> month.compareTo(other.month)
            else -> dayOfMonth.compareTo(other.dayOfMonth)
        }
    }

    fun nextDay(): MyDate {
        val nextDate = date.plusDays(1)
        return MyDate(nextDate.year, nextDate.monthValue, nextDate.dayOfMonth)
    }
}

Then I need to create the range class:

class MyDateRange(override val start: MyDate, override val endInclusive: MyDate) : ClosedRange<MyDate>, Iterable<MyDate> {
    override fun iterator(): Iterator<MyDate> {
        return object : Iterator<MyDate> {
            var current = start

            override fun hasNext(): Boolean {
                return current <= endInclusive
            }

            override fun next(): MyDate {
                val result = current
                current = current.nextDay()
                return result
            }
        }
    }
}

Copilot wrote all of that for me, once I gave it the class name, and then asked it to implement Iterable as well.

I have to admit I kinda spiked this code to see how hard it would be to implement before deciding whether to even include this in the article, so I back-filled the test on this one.

fun `an iterable range can be made by implementing Comparable and Iterable`() {
    val testDates = MyDateRange(MyDate(1955, 3, 25), MyDate(1955, 3, 28))

    (MyDate(1955, 3, 27) in testDates) shouldBe true
    (MyDate(1955, 3, 29) in testDates) shouldBe false

    var list = mutableListOf<MyDate>()

    list = testDates.fold(list) { acc, date -> acc.apply { add(date) } }

    list shouldBe listOf(
        MyDate(1955, 3, 25),
        MyDate(1955, 3, 26),
        MyDate(1955, 3, 27),
        MyDate(1955, 3, 28)
    )
}

This is a good start for me to work out how to make a fully-functioning enum range. Later.

And that is a chunk of code, and not much text. I think the tests largely speak for themselves; the only tricky bit was that last MyDateRange thing, and I'll come back to that. Lemme know if this strategy of using "tests as narrative" works? Cheers.

The code is here: RangesTest.kt.

Righto.

--
Adam

CFML: AND and OR operators not doing what one might expect

G'day:

A question came up on the CFML Slack forums today about some "unexpected behaviour" with ColdFusion's and operator. Here's an example (CF2021):

cf-cli>5 && 7
7

cf-cli>5 || 7
5

Compared to Lucee:

CFSCRIPT-REPL: 5  && 7
true
CFSCRIPT-REPL: 5 || 7
true

The questioner was puzzled by ColdFusion's behaviour, expecting to see something more like Lucee's result.

The docs for these (Expressions-Developing guide › Boolean operators) says this:

AND or &&
Return True if both arguments are True; return False otherwise. For example, True AND True is True, but True AND False is False.

On the face of it Lucee is getting it right, and ColdFusion is being weird, however this is a case of Lucee following the docs, and the docs being wrong.

CFML's behaviour with the AND and OR operators is not quite as the docs say, they behave the way described clearly in the JavaScript docs (JavaScript › Expressions and operators › Logical AND (&&)):

More generally, the operator returns the value of the first falsy operand encountered when evaluating from left to right, or the value of the last operand if they are all truthy.

I only cite the JS docs cos they're clear and well-worded, not because of any connection to CFML.

I guess it works like this in languages with truthy/falsy values as opposed to strict true/false because the operand-values can have meaning beyond just being boolean.

I checked some other languages I had to hand:

adam@DESKTOP-QV1A45U:~$ node -i
> 5 && 7
7
> 5 || 7
5

(And JS run in the browser is the same)

Ruby:

adam@DESKTOP-QV1A45U:~$ irb
irb(main):001:0> 5 && 7
=> 7
irb(main):002:0> 5 || 7
=> 5
irb(main):003:0>

Groovy:

groovy:000> 5 && 7
===> true
groovy:000> 5 || 7
===> true
groovy:000>

PHP:

php > echo 5 && 7;
1
php > echo 5 || 7;
1

Clojure:

user=> (and 5 7)
7
user=> (or 5 7)
5

There's a strong precedent there, but obviously it could have gone either way. CFML just went the way shown. So it's a bug in Lucee that it doesn't work the same as ColdFusion.

BTW: ColdFusion has worked that way since at least CF5. I just checked.

Why am I writing this? Well cos it's not correctly documented anywhere, so I figured I'd jot something down in the hope Google picks it up.

Righto.

--
Adam