Monday, 11 January 2021

Part 4: Tweaks I made to my Bash environment in my Docker containers

G'day:

Intro

Please note that this is a sub-article of a larger body of work that is an exercise in setting up a Vue.js-driven website backed by PHP8 and MariaDB running in Docker containers. All of this is completely new to me, so is a learning exercise, rather than some exposition of my wisdom (which I have none of). I initially intended the whole exercise to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers (this article)
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the fourth article in the series, and - chronologically - follows on from Part 3: PHPUnit. That said, this article is reasonably stand-alone, so not sure if it's really necessary to read everything else first. The only real "cross over" is that I'm experimenting within Docker containers that I created in the earlier articles. I guess you can refer back to the other articles if anything I say here seems to be making assumptions you can't make.

Also please note that there is very little that is earthshattering here. It's more an exercise of me - as a relative *nix noob - getting stuff working how I've become accustomed to it. I discuss nothing tricky or advanced here.

Today's exercise

Whilst doing all the crap to get Nginx, PHP and PHPUnit working, I was spending an awful lot of time in and out of Bash, running various bits of code and testing stuff and the like. I found there were a few annoying things about running Bash in these containers:

  • The Debian distro that the PHP container uses doesn't include ping and other network utils I was needing to use.
  • Bash history works for the life of the container, but does not - obviously - live between rebuilds of a container. This was a pain in the butt as I had a bunch of things to run every rebuild to check stuff.
  • I needed to add a coupla aliases: ll which everyone has; and one, cls, that I've got that is a bit nicer than just clear

In this article I'm gonna work through implementing those, and a coupla other things.

Adding the networking utils was a one-liner in the docker/php-fpm/Dockerfile:

FROM php:8.0-fpm
RUN apt-get update
RUN apt-get install git --yes
RUN apt-get install net-tools iputils-ping --yes
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
#COPY ./.bashrc /root/.bashrc
ENV XDEBUG_MODE=coverage
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

That just installs both net-tools and iputils-ping. The --yes just suppresses confirmation the installation asks, which breaks the Docker build if I don't answer it in a batch fashion: the Docker build process is not interactive.

Sorting out the Bash history was slightly trickier. My initial plan was to just mount a stubbed file in the container (this from docker/docker-compose.yml):

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
      - ./php-fpm/.bash_history:/root/.bash_history
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

This has some drawbacks. The build broke if ./php-fpm/.bash_history wasn't in the host file system. To guarantee it was there, I needed it in source control. But… by its very nature it's getting changed all the time, which gets annoying when I go to commit stuff. There's ways to work around this in git, but those were causing issues as well.

In the end I decided to take this approach:

    volumes:
      - ..:/usr/share/fullstackExercise
      - ./php-fpm/root_home:/root

That's the entire home directory for the root user. By default it has nothing in it, so I don't need to replicate much in the file system in source control: just make the directory exist, and .gitignore .bash_history:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ tree -aF --dirsfirst -L 2 php-fpm/root_home/
php-fpm/root_home/
├── .bash_history*
└── .gitignore*

0 directories, 2 files

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ cat php-fpm/root_home/.gitignore
*
!.gitignore

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

And this just works! Below I show rebuilding my containers, going into bash on the PHP container, showing the empty history, doing some stuff then exiting. Then I repeat the whole operation and you can see the history is sticking across containers now.

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down --remove-orphans
Stopping docker_nginx_1   ... done
Stopping docker_php-fpm_1 ... done
Removing docker_nginx_1   ... done
Removing docker_php-fpm_1 ... done
Removing network docker_backend
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building nginx
[...]
Building php-fpm
[...]
Creating docker_nginx_1   ... done
Creating docker_php-fpm_1 ... done
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@fd43b546b8c0:/usr/share/fullstackExercise# history
    1  history
root@fd43b546b8c0:/usr/share/fullstackExercise# cat ~/.bash_history
root@fd43b546b8c0:/usr/share/fullstackExercise# ls
LICENSE    _public        composer.lock  log        phpmd.xml    public  test
README.md  composer.json  docker         phpcs.xml  phpunit.xml  src     vendor
root@fd43b546b8c0:/usr/share/fullstackExercise# exit
exit
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down --remove-orphans
Stopping docker_php-fpm_1 ... done
Stopping docker_nginx_1   ... done
Removing docker_php-fpm_1 ... done
Removing docker_nginx_1   ... done
Removing network docker_backend
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building nginx
[...]
Building php-fpm
[...]
Creating docker_nginx_1   ... done
Creating docker_php-fpm_1 ... done
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@9b7b60d6208a:/usr/share/fullstackExercise# history
    1  history
    2  cat ~/.bash_history
    3  ls
    4  exit
    5  history
root@9b7b60d6208a:/usr/share/fullstackExercise# cat ~/.bash_history
history
cat ~/.bash_history
ls
exit
root@9b7b60d6208a:/usr/share/fullstackExercise# exit
exit
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Quite pleased with that, I am.

Given how I'm voluming-in the entire home directory, doing the aliases as easy enough, I just needed to drop a .bashrc into that docker/php-fpm/root_home directory. The image doesn't have a .bashrc by default, so this is fine.

alias ll='ls -alF'
alias cls='clear; printf "\033[3J"'

There's a coupla things to note here. Initially by accident I saved this file with CRLF line endings, and so the ll alias didn't work:

root@50fbd03160ef:/usr/share/fullstackExercise# ll
's: invalid option -- '
Try 'ls --help' for more information.
root@50fbd03160ef:/usr/share/fullstackExercise#

That's a trap for young players there (even old burnt-out ones like me, too). It was not immediately obvious what the issue is there, so it took a bit of googling and stack-overflow-ing to find the answer. Something to remember. But once one sorts that out, it works fine:

root@5962e5abd527:/usr/share/fullstackExercise# ll
total 173
drwxrwxrwx 1 1000 1000    512 Dec 14 20:14 ./
drwxr-xr-x 1 root root   4096 Dec 14 18:32 ../
drwxrwxrwx 1 1000 1000    512 Dec 14 19:03 .git/
-rwxrwxrwx 1 1000 1000     49 Dec 14 17:59 .gitignore*
drwxrwxrwx 1 1000 1000    512 Dec 14 20:14 .idea/
-rwxrwxrwx 1 1000 1000  35149 Dec 11 11:29 LICENSE*
-rwxrwxrwx 1 1000 1000    119 Dec 11 11:29 README.md*
-rwxrwxrwx 1 1000 1000    563 Dec 14 14:01 composer.json*
-rwxrwxrwx 1 1000 1000 123727 Dec 14 17:00 composer.lock*
drwxrwxrwx 1 1000 1000    512 Dec 14 19:10 docker/
drwxrwxrwx 1 1000 1000    512 Dec  5 13:19 log/
-rwxrwxrwx 1 1000 1000    479 Dec 14 17:00 phpcs.xml*
-rwxrwxrwx 1 1000 1000   1913 Dec 14 17:00 phpmd.xml*
-rwxrwxrwx 1 1000 1000    709 Dec 14 17:00 phpunit.xml*
drwxrwxrwx 1 1000 1000    512 Dec 14 17:00 public/
drwxrwxrwx 1 1000 1000    512 Dec 14 15:36 src/
drwxrwxrwx 1 1000 1000    512 Dec 14 17:00 test/
drwxrwxrwx 1 1000 1000    512 Dec 14 14:01 vendor/
root@5962e5abd527:/usr/share/fullstackExercise#

The cls alias I make just solves something that annoys me about clear is that it doesn't clear the scrollback history when SSHed into the box, so if one mouse-scrolls up, one can still move past where you cleared. This makes things hard to find sometimes, and defeats the purpose of clear IMO. The escape sequence in the alias alias cls='clear; printf "\033[3J"' just clears the scrollback as well as the screen. Or something that amounts to that anyhow. Shrug. I just copy & pasted it once upon a time, and still use it.

That's all I had for this one. Next stop: "Part 5: MariaDB".

Righto.

--
Adam

Friday, 8 January 2021

Part 3: PHPUnit

G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit (this article)
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the third article in the series, and follows on from Part 2: PHP. It's probably best to go have a breeze through the earlier articles first.

PHPUnit

I got slightly ahead of myself and added PHPUnit into composer.json when I was working through the PHP configuration part of this exercise, so it's already installed. Before I run it though, I need a phpunit.xml file, so I'll chuck one of those in:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd"
        colors="true"
        forceCoversAnnotation="true"
        cacheResult="false"
>
    <coverage>
        <include>
            <directory suffix=".php">src</directory>
        </include>
        <report>
            <html outputDirectory="public/test-coverage-report/" />
        </report>
    </coverage>
    <testsuites>
        <testsuite name="Functional tests">
            <directory>test/functional/</directory>
        </testsuite>
        <testsuite name="Unit tests">
            <directory>test/unit/</directory>
        </testsuite>
    </testsuites>
</phpunit>

There's no real surprises here. I've got two separate test suits: one for functional tests in which I'll test those test web-browseable files I created in the previous article; and one for unit tests. To test the code coverage config here I'll need some actual code to test and cover.

I had some drama getting PHPUnit working with code coverage, and that in itself is covered in a separate article, PHPUnit: get code coverage reporting working on PHP 8 / PHPUnit 9.5. The stuff I've written there is very focused on PHPUnit and not so much on the Docker side of things, or the testing in the context of this notional application I'm putting together, hence splitting it out into its own article, and also so I can focus on the Docker side of things here.

First things first, I need some tests! I decided to functional-test the two web browseable files: gdayWorld.html and gdayWorld.php. As a reminder their contents are (respectively):

<!doctype html>

<html lang="en">
<head>
    <meta charset="utf-8">

    <title>G'day world</title>
</head>

<body>
<h1>G'day world</h1>
<p>G'day world</p>
</body>
</html>

And:

<?php

$message = "G'day World";
echo $message;

So for the tests I'm just gonna make sure I can hit them, and the HTML one has the expected title, heading and content; and the PHP one just has the expected string. I could horse around with PHP's native curl implementation to make these work, but its programming interface written like something out of 1995, so I tend to avoid it where I can. I'm gonna use Guzzle instead. Also, and this is slightly OTT I know, but I like using Symfony's Response constants when checking for HTTP status codes to make the code more clear, so I'm adding in symfony/http-foundation. Lastly as I'll be using PHP's DOM API for testing the HTML, I'm placating a warning in PHPStorm that says "ooh but ext-dom might not be installed!" So I'm forcing that too. Oh and I like keeping my code tidy so I'm also slinging PHPMD and PHPCS in there too. My composer.json file becomes:

{
    "name": "adamcameron/full-stack-exercise",
    "description": "Full Stack Exercise",
    "license": "GPL-3.0-or-later",
    "require-dev": {
        "phpunit/phpunit": "^9.5",
        "guzzlehttp/guzzle": "^7",
        "symfony/http-foundation": "^5.2",
        "ext-dom": "*",
        "phpmd/phpmd": "^2.9",
        "squizlabs/php_codesniffer": "^3.5"
    },
    "autoload": {
        "psr-4": {
            "adamCameron\\fullStackExercise\\": "src/"
        }
    },
    "autoload-dev": {
        "adamCameron\\fullStackExercise\\test\\": "test/"
    }
}

Now I can write some tests. I did all this incrementally, but you know how to do that, so here is the "final" (see below for why I put that in quotes) version of WebServerTest:

namespace adamCameron\fullStackExercise\test\functional\_public; // "public" is reserved

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class WebServerTest extends TestCase
{
    /** @coversNothing */
    public function testGdayWorldHtmlReturnsExpectedContent()
    {
        $expectedContent = "G'day world";


        $client = new Client([
            'base_uri' => 'http://localhost/'
        ]);

        $response = $client->get('gdayWorld.html');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $html = $response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);

        $xpathDocument = new \DOMXPath($document);

        $hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasTitle);

        $hasHeading = $xpathDocument->query('/html/body/h1[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasHeading);

        $hasContent = $xpathDocument->query('/html/body/p[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasContent);
    }
}

And I triumphantly run this:

/usr/share/fullstackExercise # vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning:       No code coverage driver available

E                                                                   1 / 1 (100%)

Time: 00:00.438, Memory: 6.00 MB

There was 1 error:

1) adamCameron\fullStackExercise\test\functional\_public\WebServerTest::testGdayWorldHtmlReturnsExpectedContent
GuzzleHttp\Exception\ConnectException: cURL error 7: Failed to connect to localhost port 80: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://localhost/gdayWorld.html

Doh! But… but… but… I quickly went and hit http://localhost/gdayWorld.html in my browser and it was fine. Then the penny dropped. I'm not running the tests from my host machine. I'm running them from with the PHP container. I've told the host machine about the Nginx container's web server; but I've not told the PHP container about it. Reminder as to what the docker-compose.yml is like at the moment:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t

I read a whole bunch of stuff on networking in Docker. I didn't find the Docker docs very useful at the time, but now I re-read them knowing how I'm supposed to interpret them, they seem clear. Not sure if that's an indictment of me or the docs. Or both. I also looked at a whole bunch of Stack Overflow Q&A and the answers were conflicting and divergent. However after distilling what I could from all these sources, it's really pretty easy. Here's the updated version:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend
        
networks:
  backend:
    driver: "bridge"

I just added the networks section, and then told the Nginx and PHP containers to be on that backend network. NB: backend has no significance as a word here, it's just a label, and one used in the docs I was reading. After rebuild, I could now see the Nginx server from the PHP container:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/sh
/usr/share/fullstackExercise # curl http://nginx/gdayWorld.php
G'day World


/usr/share/fullstackExercise #

Note that I'm using the Nginx services container name there as the host name, ie:

services:
  nginx:

Not that it matters, but that seems a bit manky to me, so I wanted to specify a hostname here. That's just a matter of giving the Nginx container a hostname:

services:
  nginx:
    build:
      context: ./nginx
    hostname: webserver.backend
    volumes:
      # etc
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/sh
/usr/share/fullstackExercise # curl http://webserver.backend/gdayWorld.php
G'day World


/usr/share/fullstackExercise #

Now my functional tests should work, provided I use that new hostname:

/usr/share/fullstackExercise # vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning:       No code coverage driver available

..                                                                   1 / 1 (100%)

Time: 00:00.462, Memory: 6.00 MB

OK (2 test, 6 assertions)
/usr/share/fullstackExercise #

Cool!

Oh there was a separate test for gdayWorld.php too:

namespace adamCameron\fullStackExercise\test\functional\_public; // "public" is reserved

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class PhpTest extends TestCase
{
    /** @coversNothing */
    public function testGdayWorldPhpReturnsExpectedContent()
    {
        $client = new Client([
            'base_uri' => 'http://webserver.backend/'
        ]);

        $response = $client->get('gdayWorld.php');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $content = $response->getBody()->getContents();

        $this->assertSame("G'day world", $content);
    }
}

I'm glad I wrote these tests, because this one identified a small bug I had introduced into gdayWorld.php. I'd fixed it by the time I catpured that output above, but the first run was less positive:

/usr/share/fullstackExercise # vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning:       No code coverage driver available

F.                                                                  2 / 2 (100%)

Time: 00:00.509, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\_public\PhpTest::testGdayWorldPhpReturnsExpectedContent
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'G'day world'
+'G'day World'

/usr/share/fullstackExercise/test/functional/PhpTest.php:24

FAILURES!
Tests: 2, Assertions: 6, Failures: 1.
/usr/share/fullstackExercise #

Yay for testing! I did not contrive this situation as an example of "always test first!", but there it is. This is hugely trivial code, but I still messed it up, and simply eyeballing it did not spot the bug.

Speaking of being observant… you will no-doubt have noticed the warning about code coverage driver above. I still need to install XDebug to make this work. This is a matter of adding this into the Dockerfile:

FROM php:fpm-alpine
RUN apk --update --no-cache add git
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

Or at least that was the theory:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building php-fpm
Step 1/8 : FROM php:fpm-alpine
---> 6bd7d9173974
Step 2/8 : RUN apk --update --no-cache add git
---> Using cache
---> 098d91282e3e
Step 3/8 : RUN docker-php-ext-install pdo_mysql
---> Using cache
---> 6f74a2ec5bb1
Step 4/8 : RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
---> Running in 12444e0a094c
downloading xdebug-3.0.1.tgz ...
Starting to download xdebug-3.0.1.tgz (214,467 bytes)
.............................................done: 214,467 bytes
87 source files, building
running: phpize
Configuring for:
PHP Api Version: 20200930
Zend Module Api No: 20200930
Zend Extension Api No: 420200930
Cannot find autoconf. Please check your autoconf installation and the
$PHP_AUTOCONF environment variable. Then, rerun this script.

ERROR: `phpize' failed
ERROR: Service 'php-fpm' failed to build : The command '/bin/sh -c pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug' returned a non-zero code: 1
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Sigh. I have encountered this sort of thing before with other containers. Alpine is really really pared down, and doesn't include a bunch of packages that tools might need to run. This is fine for a lot of situations, but it's also a pain in the arse for others. In this case I added in autoconf, but then I needed to install a C compiler too. And then after that I think it wanted something else. Sod that. I just stopped using Alpine and wend back to the Debian version of the container, in the Dockerfile:

FROM FROM php:8.0-fpm
RUN apt-get update --yes && apt-get install git --yes
ENV XDEBUG_MODE=coverage
NB that changed from:
FROM :fpm-alpine
RUN apk --update --no-cache add git

Note: the apk / apt-get change is just the difference in package manager between Alpine and Debian. I won't show you the installation of this because it's 70kB of telemetry, all of which culminates in:

Creating docker_php-fpm_1 ... done Creating docker_nginx_1 ... done

And now we can run our functional tests, and the code coverage report should create (even if it hasn't got anything in it yet, cos I'm not code-covering those functional tests):

root@2e5f56af2f54:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning: Incorrect filter configuration, code coverage will not be processed
..                                                                  2 / 2 (100%)

Time: 00:00.493, Memory: 6.00 MB

OK (2 tests, 6 assertions)
root@2e5f56af2f54:/usr/share/fullstackExercise#

Grrrr… what now?. I reviewed the docs and my phpunit.xml was legit-looking, and validated fine, so I was flumoxed. But then I came across this issue with PHPUnit: Misleading error message when no files present in coverage path. That sums it up. I have this in my phpunit.xml file:

<coverage>
    <include>
        <directory suffix=".php">src</directory>
    </include>
    <report>
        <html outputDirectory="public/test-coverage-report/" />
    </report>
</coverage>

But I don't actually have a src/ directory yet. Once I added that, and also added the public/ directory into that <include> block, I get a report. At the same time, I will add a stub PHP class and test thereof into the src/ directory as well, to better test the reporting:

root@2e5f56af2f54:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

...                                                                 3 / 3 (100%)

Time: 00:02.050, Memory: 12.00 MB

OK (3 tests, 7 assertions)

Generating code coverage report in HTML format ... done [00:00.780]
root@2e5f56af2f54:/usr/share/fullstackExercise#

And the coverage report generates fine too:
above



below

The test for MyClass's needsTesting method is as simple as you might imagine:

namespace adamCameron\fullStackExercise\test\unit;

use adamCameron\fullStackExercise\MyClass;
use PHPUnit\Framework\TestCase;

/** @coversDefaultClass adamCameron\fullStackExercise\MyClass */
class MyClassTest extends TestCase
{
    private $myClass;

    protected function setUp(): void
    {
        $this->myClass = new MyClass();
    }

    /** @covers ::needsTesting */
    public function testNeedsTesting()
    {

        $needsTesting = $this->myClass->needsTesting();
        $this->assertTrue($needsTesting);
    }
}

OK, so I'm in a good place to be able to TDD some PHP code now. But before I do that, I want to get a MariaDB container added into the mix as well, and write an intergration test for its connectivity. Before I get to that though, as I've been doing the work on this project up until now - especially when troubleshooting the networking and the PHPUnit issue I had - I've tweaked some stuff with my Bash environment, and I wanted to rip that out into a separate wee article. So before we get onto MariaDB, I'll write that up: Tweaks I made to my Bash environment in my Docker containers.

Righto.

--
Adam

Thursday, 7 January 2021

Part 2: PHP

G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP (this article)
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the second article in the series, and follows on from Part 1: Intro & Nginx. I'ts probably best to go have a breeze through that one first.

PHP

The aim for this step is to stick an index.php file in the public directory, and have Nginx pass it over to PHP for processing. At the same time I'm going to get it to run composer install for good measure, so I'll add composer.json in there too. Oh, and PHP will end up needing to use PDO to talk to the DB, so I'll add that extension in now as well. Conveniently, Martin's article does all this stuff too, so all I need to do is copy & paste & tweak some files. Hopefully.

First up, docker/php-fpm/Dockerfile:

FROM php:fpm-alpine
RUN apk --update --no-cache add git
RUN docker-php-ext-install pdo_mysql
COPY --from=composer /usr/bin/composer /usr/bin/composer
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

This is largely the same as Martin's one; I've just got rid of some DB stuff he had in his, and pointed the WORKDIR to where the application code will be. note that /usr/share/fullstackExercise/ is the parent directory that contains the web root (/public), and composer.json, /src and /test etc.

One interesting thing is how composer is… erm… availed to the container: the files are just copied straight from Docker's image, then installed in the container.

Next is a simple config file to configure how Nginx passes PHP stuff onwards (docker/nginx/conf.d/default.conf). This is lock-stock from Martin's acticle, and I am just going "yeah… seems legit…?" (I'm such a pro):

upstream php-upstream {
    server php-fpm:9000;
}

And we need to stick the PHP stuff back into the Nginx site configuration file too now (docker/nginx/sites/default.conf):

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    server_name localhost;
    root /usr/share/fullstackExercise/public;
    index index.html;

    location / {
        autoindex on;
        try_files $uri $uri/ /index.php$is_args$args;
    }

    location ~ \.php$ {
        try_files $uri /index.php =404;
        fastcgi_pass php-upstream;
        fastcgi_index index.php;
        fastcgi_buffers 16 16k;
        fastcgi_buffer_size 32k;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_read_timeout 600;
        include fastcgi_params;
    }

    location ~ /\.ht {
        deny all;
    }
}

That's all boilerplate stuff really.

Finally we crank up the php-fpm container via docker/docker-compose.yml:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t

The PHP bit is pretty simple here. It just mounts the app directory. The only other new thing in here is that I've set both containers to be able to be shelled into with docker exec.

So. Does any of this stuff work? First I need to shut-down what I've got running:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down --remove-orphans
Stopping docker_nginx_1   ... done
Removing docker_nginx_1   ... done
Removing network docker_default
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

And then crank it all back up again. This outputs a lot of stuff, most of which I've elided here, and most of what I've kept I've only done so for the sake of completeness, and showing it ran OK. Note that I'm now running docker-compose with the --detach option. This just gives me my prompt back after the process runs:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_default" with the default driver
Building php-fpm
Step 1/7 : FROM php:fpm-alpine
fpm-alpine: Pulling from library/php
188c0c94c7c5: Already exists
45f8bf6cfdbe: Pull complete
[...]
Status: Downloaded newer image for php:fpm-alpine
---> 6bd7d9173974
Step 2/7 : RUN apk --update --no-cache add git
---> Running in 4beacfe86bf2
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/3) Installing expat (2.2.9-r1)
(2/3) Installing pcre2 (10.35-r0)
(3/3) Installing git (2.26.2-r0)
Executing busybox-1.31.1-r19.trigger
OK: 27 MiB in 34 packages
Removing intermediate container 4beacfe86bf2
---> 9838e800a674
Step 3/7 : RUN docker-php-ext-install pdo_mysql
---> Running in d38de20fec54
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/29) Installing m4 (1.4.18-r1)
[...]
----------------------------------------------------------------------
Libraries have been installed in:
/usr/src/php/ext/pdo_mysql/modules

[...]
----------------------------------------------------------------------

Build complete.
Don't forget to run 'make test'.

Installing shared extensions: /usr/local/lib/php/extensions/no-debug-non-zts-20200930/
[...]
OK: 27 MiB in 34 packages
Removing intermediate container d38de20fec54
---> 97967ee1a4b2
Step 4/7 : COPY --from=composer /usr/bin/composer /usr/bin/composer
latest: Pulling from library/composer
11c513a1b503: Pull complete
[...]
Status: Downloaded newer image for composer:latest
---> 3b2d08ac01d3
Step 5/7 : WORKDIR /usr/share/fullstackExercise/
---> Running in 26dc7beac904
Removing intermediate container 26dc7beac904
---> 0331a636bbe7
Step 6/7 : CMD composer install ; php-fpm
---> Running in d89f5a03182f
Removing intermediate container d89f5a03182f
---> 98b0bf2fc6aa
Step 7/7 : EXPOSE 9000
---> Running in b4eea18c4960
Removing intermediate container b4eea18c4960
---> c3a5321dd671

Successfully built c3a5321dd671
Successfully tagged docker_php-fpm:latest
Building nginx
[...]
Successfully built 78383fe534b7
Successfully tagged docker_nginx:latest
Creating docker_php-fpm_1 ... done
Creating docker_nginx_1 ... done
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

And, again, the last step is to test that PHP running stuff, just via /public/gdayWorld.php for now:

<?php

$message = "G'day World";
echo $message;

When I hit http://localhost/gdayWorld.php, I get:

Good enough for me. I mean I should perhaps include a file from /src to make sure that PHP can find it, but I will admit this did not occur to me at the time.

In the next article, Part 3: PHPUnit, I will do as indicated: get PHPUnit working, and backfilling some tests of the code I've written so far.

Righto.

--
Adam

Wednesday, 6 January 2021

Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers - Part 1: Intro & Nginx

G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx (this article)
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

I've already written everything down to and including "Using Symfony", and will release those over the coming days. Originally I was gonna finish the whole series before releasing anything, because I know what I'm like: there's a chance I'll get bored and not finish the series, so would be a bit rubbish to release the first n parts but not finish it. But then it occurred to me that that approach is not very agile, plus if I actually release what I've done, I'm probably more likely to see it through to completion. Each article is stand-alone anyhow, for the most part. I appreciate my mates Dara McGann and Brian Sadler also giving me a nudge on this.

Intro

This "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers" exercise should be hilarious (at my expense, I mean), cos I've never touched Vue.js, only know enough about Nginx to get myself into trouble (and sometimes - but not always - back out of trouble again), never touched Symfony really. or PHP8 yet. And still a noob with Docker, and all this requires me to work with docker-compose, which I have not ever touched until about 10min ago. So I'll warn you up front that these articles just logs my journey through all this stuff, and any knowledge I am applying is coming to me via googling and reading stuff on Stack Overflow (etc), about 5min before I put it into practice. Nothing here is an expert's view, and no promises as to whether I'm doing anything correctly. Not sure why yer reading it, actually.

The motivation behind this work is multi-fold:

  • my client-side exposure is very out of date. I have not written any front-end production code for… blimey five years probably.
  • Accordingly I have not touched any of the new fancy JS frameworks, so figured I should take a look at Vue.js.
  • It'll need some sort of back-end. I'd usually use Silex for this, but it's end-of-life now, and the recommendation is to use Symfony instead. I've done the smallest amount on Symfony (supervising other devs whilst they do the work, basically), so this is an excuse to do some actual work with it.
  • PHP8's just been released so I'll use that, although I can't see it'll make any difference from 7.x for the purposes of what I'll be doing.
  • I've messed around with individual containers a bit in Docker now, but it's all been very ad hoc. This is a good excuse to tie some stuff together with Dockerfiles and docker-compose.yml.

To be completely frank, I am unenthused about this concept of "full stack" development. In my exposure to full-stack devs, what it means is instead of being good at one thing; they're very bloody ordinary at a bunch of things ("jack of all trades; masters of none"). I understand it might be useful in smaller shops, I guess. I've always been in the position that wherever I've worked there's been enough bods to have dedicated people in each role. I'm not saying there're no people who are good at an entire stack, but the bulk of people seem not to be. It strikes me as being the enterprise of people without the discipline to stick to one thing. The flipside of this is that I've been overlooked for two roles recently because they want people who do both client-side and back-end dev, and I will always admit to being weak on the front end. However I'm also pragmatic… if I need to improve my exposure here... so be it.

Right. Off we go. I'm gonna work through this in a stepped fashion; getting each component working (to a "G'day World" degree) before integrating the next component.

Before I start actually, I need to point out I am basing my approach on the excellent article "Symfony 5 development with Docker", by Martin Pham. I just found it on Google, but it's covering much of the same territory I need to cover.

Directory structure

I'm following Martin's lead here for the most part. The key bit is putting all the docker-related files into their own directory in my app directory:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise$ tree -F --dirsfirst -L 2 .
.
├── docker/
│   ├── nginx/
│   └── docker-compose.yml*
├── log/
│   └── nginx/
├── public/
│   └── gdayWorld.html*
├── LICENSE*
└── README.md*

5 directories, 4 files
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise$

(This was taken after I did the Nginx config, below, but it better shows what goes where if I include the Nginx stuff now).

Oh Martin also has the log directory as logs on the host and log in the containers. That bit me on the bum the first time I tried this (see further down), so I'm sticking with a uniform log throughout.

Nginx

Firstly I'm just going to get Nginx serving gdayWorld.html on localhost:80. I'm not yet going to worry about PHP or Symfony or anything like that.

I've largely copied Martin's docker/nginx/Dockerfile:

FROM nginx:alpine
WORKDIR /usr/share/nginx/
CMD ["nginx"]
EXPOSE 80

The only difference is I've changed the WORKDIR value from /var/www to /usr/share/nginx/ which seems to be where Nginx would naturally put its files. Martin's using /var/www because "the same directory with PHP-FPM since we’re gonna share this with a mounted volume" (that's from his article). However I will specifically not be doing that. I will be keeping the website directory and the application source code separate, as one usually would with a modern PHP app. Only the "entry-point" index.php will be in the web root; the rest will be in the adjacent src directory. In the context of this file, it doesn't matter here, but the same directory is referenced later where it does matter, and I want to keep things consistent. Also this seems to be a reference to the Nginx app directory really, not anything related to the web root, per se?

And again, the docker/nginx/nginx.conf file is lifted from Martin's article:

user  nginx;
worker_processes  4;
daemon off;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    access_log  /var/log/nginx/access.log;
    sendfile        on;
    keepalive_timeout  65;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-available/*.conf;
}

And the separate site config file, docker/nginx/sites/default.conf:

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    server_name localhost;
    root /usr/share/fullstackExercise/public;
    index index.html;

    location / {
         try_files $uri $uri/;
    }

    location ~ /\.ht {
        deny all;
    }
}

One difference here is that I'm setting the web root to be /usr/share/fullstackExercise/public, rather than a subdirectory of the Nginx application directory as Martin had it. /public is where the app's web-browseable files will be home (Symfony's location for these is that directory, relative to the application root).

I don't need the PHP stuff in this file yet, so I've removed it for now. We're only interested in serving HTML at the moment.

Lastly the docker-compose.yml file is cut down from the one in the article to just handle Nginx for now:

version: '3'

services:
    nginx:
        build:
            context: ./nginx
        volumes:
            - ../public:/usr/share/fullstackExercise/public
            - ./nginx/nginx.conf:/etc/nginx/nginx.conf
            - ./nginx/sites/:/etc/nginx/sites-available
            - ./nginx/conf.d/:/etc/nginx/conf.d
            - ../log:/var/log
        ports:
            - "80:80"

As mentioned above, the only significant change here is that that first volume has been changed from being ../src:/var/www (PHP src directory) to be ../public:/usr/share/fullstackExercise/public again, as per above.

Oh! One last file! public/gdayWorld.html:

<!doctype html>

<html lang="en">
<head>
    <meta charset="utf-8">

    <title>G'day world</title>
</head>

<body>
<h1>G'day world</h1>
<p>G'day world</p>
</body>
</html>

We should now be… good to go. Let's try it…

adam@DESKTOP-QV1A45U:~$ cd /mnt/c/src/fullstackExercise/
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise$ cd docker/
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up
Building nginx
Step 1/4 : FROM nginx:alpine
---> 98ab35023fd6
Step 2/4 : WORKDIR /usr/share/nginx/
---> Running in 2223848549f7
Removing intermediate container 2223848549f7
---> 6f9bba05771d

Step 3/4 : CMD ["nginx"] ---> Running in 46c32d7862a7
Removing intermediate container 46c32d7862a7
---> 99cd0a9bb3fe
Step 4/4 : EXPOSE 80
---> Running in 62cb63572cab
Removing intermediate container 62cb63572cab
---> 7ae273be9176

Successfully built 7ae273be9176
Successfully tagged docker_nginx:latest
WARNING: Image for service nginx was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating docker_nginx_1 ... done
Attaching to docker_nginx_1
nginx_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx_1 | 10-listen-on-ipv6-by-default.sh: error: /etc/nginx/conf.d/default.conf is not a file or does not exist
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx_1 | /docker-entrypoint.sh: Configuration complete; ready for start up

This is promising. So over to a web browser on my host machine:

Cool!

Umm… OK… that didn't actually go as smooth for me as I make it sound there. The first time around - because I did not notice that discrepancy between log/logs in Martin's article - I ended up referencing a non-existent path in docker-compose.yml, and the docker compose call failed:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build Creating network "docker_default" with the default driver Building nginx Step 1/4 : FROM nginx:alpine [...etc...] Successfully built 7ae273be9176 [...etc...] nginx_1 | /docker-entrypoint.sh: Configuration complete; ready for start up nginx_1 | nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (2: No such file or directory) nginx_1 | 2020/12/05 19:10:50 [emerg] 1#1: open() "/var/log/nginx/error.log" failed (2: No such file or directory) docker_nginx_1 exited with code 1 adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

That was simply cos I had this in my docker-compose.yml:

- ../logs:/var/log

Instead of this:

- ../log:/var/log

(In the file system, the directory is log). I very quickly spotted this and fixed it, and tried to back-out what I had done:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down
Traceback (most recent call last):
File "bin/docker-compose", line 3, in <module>
File "compose/cli/main.py", line 67, in main
File "compose/cli/main.py", line 123, in perform_command
File "compose/cli/command.py", line 69, in project_from_options
File "compose/cli/command.py", line 125, in get_project
File "compose/cli/command.py", line 184, in get_project_name
File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory
[2052] Failed to execute script docker-compose
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Try as I might I could not work out what file was missing. Nice one, btw, docker-compose for not actually saying what file-read didn't work. Amateur hour there.

After way too long of reading bug reports on Docker and various Stack Overflow Q&A, I spotted a solution ("Failed to execute script docker-compose"). The person answering just suggested "make sure Docker Desktop is running". That sounded off to me (of course it's running), but a comment on the answer made me look again: "I read this, thought what an answer for dummies, and then realised a few minutes later it was the answer I needed! DOH". I I went "oh all right then", and restarted Docker Desktop, dropped out of my bash instance and into a new one, and now:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down
Removing docker_nginx_1 ... done
Removing network docker_default
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Then I could just go docker-compose up --build, and the thing worked properly.

It's perhaps important to note I can replicate this at will, so this is just a workaround for some issue with Docker, but at least it got me moving forward. As I've been working through the rest of the articles in this series, I am getting this issue a lot: at least once a day. So it's not just caused by paths being wrong as per above. It seems to kick off quite often if I have changed docker-compose.yml to add/remove sections of config, or sometimes if I change files via the host machine file system (ie: my Windows desktop environment), rather than via the Ubuntu/WSL file system. I've not been able to tie it down to one particular thing. Something to look out for though.

Now to integrate PHP into the mix, see the next part of this series: Part 2: PHP.

Righto.

--
Adam

Wednesday, 9 December 2020

PHPUnit: get code coverage reporting working on PHP 8 / PHPUnit 9.5

 G'day:

I'm still beavering away getting a PHP dev environment working via Docker. As I mentioned in the previous article, I'm writing an article about how I'm doing that as I go, but I've had a few issues today that have made me pause that.

My current issue has been getting PHPUnit working properly. This has nothing to do with the Docker environment (as far as I can tell), it's just... me and PHPUnit not playing nicely together. The solution to my issue was not immediately apparent to me... it's taken me about three hours to sort this out. So I figured it might be worthwhile writing down.

I have a barebones "G'day World" website running in a coupla Docker containers (one for Nginx, one for PHP). That's all working fine, I can get "G'day world" being delivered to the browser via both HTML and PHP requests. Before I start writing any really-really code, I decided I better get PHPUnit working. TDD 'n' all that. I will admit I felt dirty yesterday when I wrote the gdayWorld.html and gdayWorld.php and ran them without tests; I just browsed to them and went "seems legit". Not the way I like to operate.

I sat down to write some tests. My gdayWorld.php is... basic:

<?php

$message = "G'day World";
echo $message;

I don't have any framework or anything installed as yet, so that's as good as it gets. That is in the web root. To test that I'd be needing to test that by curling it and checking the response, then I remembered Symfony has a WebTestCase for this sort of thing, then I decided I could not be arsed looking into that right now. All I need is a test to run to check PHPUnit is running. So... erm... here we go:

    /** @coversNothing */
    public function testNothing()
    {
        $this->assertTrue(false);
    }

This is enough to test PHPUnit. I'll write a better test once I know everything is working.

root@5fdd4abf5d20:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                 1 / 1 (100%)

Time: 00:00.166, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\public\WebServerTest::testNothing
Failed asserting that false is true.

/usr/share/fullstackExercise/test/functional/public/WebServerTest.php:12

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
root@5fdd4abf5d20:/usr/share/fullstackExercise#

Perfect. Now I can make it pass ;-)

$this->assertTrue(true);
root@5fdd4abf5d20:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.                                 1 / 1 (100%)

Time: 00:00.127, Memory: 6.00 MB

OK (1 test, 1 assertion)
root@5fdd4abf5d20:/usr/share/fullstackExercise#

So far, so good. My next step whilst I'm configuring stuff on PHPUnit, I decided to get the code coverage working too. I just added the code-coverage reporting bit to phpunit.xml and re-ran the tests:

root@eb28587ca66f:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning: No code coverage driver available

.                                 1 / 1 (100%)

Time: 00:00.210, Memory: 6.00 MB

OK (1 test, 1 assertion)
root@eb28587ca66f:/usr/share/fullstackExercise#

Doh! Forgot about that. I need to install Xdebug.

This was way more horsing around than it could have been, but this is down to the vagaries of the Docker set-up I was using. Once I decided to change direction it was a one-liner in me Dockerfile, and all good.

root@37d06fcf9209:/usr/share/fullstackExercise# php -v
PHP 8.0.0 (cli) (built: Dec 1 2020 03:33:03) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.0-dev, Copyright (c) Zend Technologies
with Xdebug v3.0.1, Copyright (c) 2002-2020, by Derick Rethans
root@37d06fcf9209:/usr/share/fullstackExercise#

Run the tests again...

root@37d06fcf9209:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning: XDEBUG_MODE=coverage or xdebug.mode=coverage has to be set

.                                 1 / 1 (100%)

Time: 00:00.220, Memory: 6.00 MB

OK (1 test, 1 assertion)
root@37d06fcf9209:/usr/share/fullstackExercise#

OK. Fine. Makes sense I s'pose. I added in the ini setting into my phpunit.xml:

<php>
  <ini name="xdebug.mode" value="coverage" />
</php>

No difference. I figured the ini setting is probably changed too late in the piece, as the xdebug.mode will need to be set before PHP starts running any code. And perhaps the environment variable approach would work such that the actual environment variable was set before hand. As it happens it ain't - all that happens is the value is added to the $_ENV array, which is hardly the same thing. From the docs:



However this step is still a relevant one cos we see a behavioural change here:

root@37d06fcf9209:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.
Segmentation fault
root@37d06fcf9209:/usr/share/fullstackExercise#

Splat!

I'm gonna spare you a coupla hours of looking in to how to troubleshoot segmentation faults here. Nothing I read helped, and there's nothing useful relating to getting them with PHPUnit, other than a bunch of people getting these segmentation faults.

I started to wonder if it was an issue with Xdebug and PHP8, given PHP8 is so new and sometimes in the last I've noted that Xdebug has needed updating when there's a point release of PHP. But looking into this (at Xdebug: Supported Versions and Compatibility), the current version of Xdebug has been around for a while, and they say it's good to go with PHP8:

I was now thinking about versions of stuff, so I decided to rollback PHP to 7.4, and PHPUnit to 8.5. I realise I should have only done one at a time, but it did not occur to me when doing this that when troubleshooting things, only change one thing per test iteration.

root@41f51888adff:/usr/share/fullstackExercise# php -v
PHP 7.4.13 (cli) (built: Dec 1 2020 04:25:48) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Xdebug v3.0.1, Copyright (c) 2002-2020, by Derick Rethans
root@41f51888adff:/usr/share/fullstackExercise# vendor/bin/phpunit --version
PHPUnit 8.5.13 by Sebastian Bergmann and contributors.

root@41f51888adff:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 8.5.13 by Sebastian Bergmann and contributors.

.Code coverage needs to be enabled in php.ini by setting 'xdebug.mode' to 'coverage'
root@41f51888adff:/usr/share/fullstackExercise#

This verifies we are back to PHP 7.4 and PHPUnit 8.5. And we don't have the segmentation fault any more, but... now PHP is ignoring both of the settings in phpunit.xml. I have this in place:

<php>
  <env name="XDEBUG_MODE" value="coverage"/>
  <ini name="xdebug.mode" value="coverage"/>
</php>

At this point I read up on the <ini> and <env> settings in phpunit.xml, and confirmed both of them are set too late for Xdebug to read them anyhow. Quite odd the way PHPUNit 9.5 reacts to the XDEBUG_MODE being set in that case. Anyway, I need to set them before PHP is run, so I set a really-really environment variable in my Dockerfile:

ENV XDEBUG_MODE=coverage

After rebuilding my containers:

root@6053a26dc3eb:/usr/share/fullstackExercise# echo $XDEBUG_MODE
coverage
root@6053a26dc3eb:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 8.5.13 by Sebastian Bergmann and contributors.

..                                 2 / 2 (100%)

Time: 1.01 seconds, Memory: 6.00 MB

OK (2 tests, 2 assertions)

Generating code coverage report in HTML format ... done [596 ms]
root@6053a26dc3eb:/usr/share/fullstackExercise#

Hurrah! Let's have a look at the report:

This is correct: I'm not testing that code in /public yet.

I also quickly tried to use the xdebug.mode ini file setting. I dropped a /usr/local/etc/php/conf.d/phpunit-code-coverage-xdebug.ini with just xdebug.mode=coverage in it into my container's file system:

root@f3cd4a0715a5:/usr/share/fullstackExercise# cat /usr/local/etc/php/conf.d/phpunit-code-coverage-xdebug.ini
xdebug.mode=coverage
root@f3cd4a0715a5:/usr/share/fullstackExercise# php --ini
Configuration File (php.ini) Path: /usr/local/etc/php
Loaded Configuration File: (none)
Scan for additional .ini files in: /usr/local/etc/php/conf.d
Additional .ini files parsed: /usr/local/etc/php/conf.d/docker-php-ext-pdo_mysql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sodium.ini,
/usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini,
/usr/local/etc/php/conf.d/phpunit-code-coverage-xdebug.ini

root@f3cd4a0715a5:/usr/share/fullstackExercise# php -i | grep xdebug.mode
xdebug.mode => coverage => coverage
root@f3cd4a0715a5:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 8.5.13 by Sebastian Bergmann and contributors.

..                                 2 / 2 (100%)

Time: 1.07 seconds, Memory: 6.00 MB

OK (2 tests, 2 assertions)

Generating code coverage report in HTML format ... done [869 ms]
root@f3cd4a0715a5:/usr/share/fullstackExercise#

Perfect.

My supposition here is that the issue all along was that I needed to set one of either the ini file option, or the environment variable option before PHPUnit runs. PHPUnit 8 correctly sez "OI!" if I don't; whereas PHPUnit 9 doesn't check correctly, and instead messes things up. Let's roll forward to PHP8 and PHPUnit 9.5 again and see:

root@f7d29f40ad62:/usr/share/fullstackExercise# php -v
PHP 8.0.0 (cli) (built: Dec 1 2020 03:33:03) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.0-dev, Copyright (c) Zend Technologies
with Xdebug v3.0.1, Copyright (c) 2002-2020, by Derick Rethans
root@f7d29f40ad62:/usr/share/fullstackExercise# vendor/bin/phpunit --version
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

root@f7d29f40ad62:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.
..                                 2 / 2 (100%)

Time: 00:01.535, Memory: 10.00 MB

OK (2 tests, 2 assertions)

Generating code coverage report in HTML format ... done [00:00.605]
root@f7d29f40ad62:/usr/share/fullstackExercise#

Hoo-frickin-ray!!

So that supposition was correct then.

I guess I could have saved myself a bunch of time here by just setting the .ini setting or the environment variable in the correct environment right from the outset. Then again had PHPUnit 9 not changed its warnings, I would have known that straight away: PHPUnit 8 was pretty clear on this. The problem and the solution here were pretty mundane, but I learned a lot about how to do things with Docker (which will all go into the other article I'm writing at the same time as this one), some stuff about Xdebug, some stuff about PHP and some stuff about PHPUnit. So it was a worthwhile afternoon of scratching my head. I'm also pleased I've sat here and done 8hrs of "work" for the second day running. Hopefully this will stick and I'll work my way out of my unemployed malaise before long. We'll see.

PHPUnit 9.5 on PHP 7.4

As I said earlier, I should not have changed two things at once when testing this, as it makes it harder to draw accurate conclusions. I omitted to say I did then go back and test PHPUnit 9.5 on PHP 7.4, so as to check whether the issue was due to PHPUnit 9.5, or whether it was some combination of that and PHP version. Here are the test results when running PHPUnit 9.5 on PHP 7.4 with XDEBUG_MODE set in phpunit.xml and not in the environment. xdebug.mode is not set anywhere:

root@81c63ae50626:/usr/share/fullstackExercise# php -v
PHP 7.4.13 (cli) (built: Dec 1 2020 04:25:48) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Xdebug v3.0.1, Copyright (c) 2002-2020, by Derick Rethans
root@81c63ae50626:/usr/share/fullstackExercise# vendor/bin/phpunit --version
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.
root@81c63ae50626:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.
Segmentation fault
root@81c63ae50626:/usr/share/fullstackExercise#

To me this demonstrates the issue is with PHPUnit 9.5 and nothing else.

Anyway... now for a beer.

Righto.


--
Adam

Docker: resolving "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?" issue

G'day:

Firstly some framing. This morning I sat down to do some more Docker stuff, and started to get this issue again:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down
Traceback (most recent call last):
File "bin/docker-compose", line 3, in <module>
File "compose/cli/main.py", line 67, in main
File "compose/cli/main.py", line 123, in perform_command
File "compose/cli/command.py", line 69, in project_from_options
File "compose/cli/command.py", line 125, in get_project
File "compose/cli/command.py", line 184, in get_project_name
File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory
[2052] Failed to execute script docker-compose
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

(I say "again", cos I have been getting this a lot whilst working on another article that's taken me a coupla days to write so far, and won't be done until tomorrow. Note: this issue is not the one I am looking at in this article… I'm looking at that "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?" issue.)

OK, so enough context. Back to "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?".

From a lot of googling, most suggestions start with a de-install/re-install Docker Desktop, so decided to try that first. It's important to note that a lot of people had tried this, it had worked for a while, but then it started again. I think it's addressing a symptom not the problem, but I might as well be on the most recent version of Docker Desktop anyhow.

After doing the upgrade I happened to have PhpStorm open, and I absent-mindedly clicked the "refresh" button on the config for the connection to Docker I had. It errored out saying "got no PHP container mate", which was true because the de-install of Docker Desktop got rid of the containers I had. To verify this, I hit the shell:

adam@DESKTOP-QV1A45U:~$ docker container ls
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
adam@DESKTOP-QV1A45U:~$

Erm… OK, that's new. now I had been messing with the Docker daemon settings in Docker Desktop trying to get PhpStorm connecting, so went back in there and re-dis-enabled(!) Docker daemon with no TLS support. I mean this setting:

PhpStorm would stop connecting, but in the mean time I could get my containers up and running. I hit the shell again:

adam@DESKTOP-QV1A45U:~$ docker container ls
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
adam@DESKTOP-QV1A45U:~$

Bugger.

adam@DESKTOP-QV1A45U:~$ docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
adam@DESKTOP-QV1A45U:~$

[I said more than bugger this time]

Great. In troubleshooting one issue, I have caused another one. Excellent. Thanks Docker. Thanks me.

After a bunch of googling and restarting randomly and shaking my head at everyone's solution for this issue which seemed to largely be "switch it off and switch it back on again", etc, I landed on someone commenting "weird thing is though… it's all still fine in Powershell. It's just broke in WS". I checked:

PS C:\Users\camer> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
PS C:\Users\camer>

Interesting. Something to do with WSL then. Then I remembered a setting I thought seemed odd in Docker Desktop when I gave it the once-over after re-install:

I did not think much of this at the time, because Ubuntu is my default WSL distro, so I figured that was fine. After all… everything had been working fine like that. On a whim, I decided to switch those two toggles back on though:

And now back to bash:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Hurrah. So basically when I de-/re-installed Docker Desktop, it did not maintain its existing settings, and defaulted those two to be off? Weird, but I'm guessing so.

Anyway, no answer I saw via my googlings had this particular solution, so I figrued I'd write it up. This might be obvious to everyone who already knows, but there's enough people out there who clearly don't already know for it to possibly be useful. Perhaps. I'm also not suggesting it's the solution to everyone getting this "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?" issue, but it's something to check. Ah anyway it's a far easier and quicker article to write than the one I'd been working on. Quick win 'n' all. I'll have the longer article done tomorrow I hope.

Righto.

--
Adam