Thursday, 14 January 2021

Part 7: Using Symfony

G'day:

Familiar boilterplate about how this is one in a series follows. Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony (this article)
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the seventh article in the series, and follows on from Part 6: Installing Symfony. It's probably best to go have a breeze through the earlier articles first, in particular the immediately preceding one. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject. Also I was only originally "planning" one article on getting the Symfony stuff sorted out, but yesterday's exercise was more involved that I'd've liked, so I stopped once I had Symfony serving up its welcome page, and today focusing on the config / code of getting a route, controller, model, view (actually there's no model or view in this endpoint; it's all just done in the controller) up and running.

Two caveats before I start.

Firstly: this is the first time I've done anything with Symfony other than reviewing the work of other members of my team - the ones doing the actual work. I have some familiarity with Symfony, but it's very very superficial.

Secondly, I will also be up front that I don't like Symfony's approach to MVC frameworking. Symfony is kind of a "lifestyle choice": it's opinionated, it expects you to do things a certain way, and it puts its nose into everything. This is opposed to something like Silex which simply provides the bare bones wiring to handle the ubiquitous web application requirement of guiding processing through routing, controller, model and view; other than that it just gets out of the way and is pretty invisible. I loved Silex, but sadly it's EOL so I need to move on. And, yeah, I'm being curmudgeonly and pre-judgemental as I go, I know. I do know Symfony is immensely popular, and I also know from my exposure to using its various independent libraries that it's been developed in a thoughtful, thorough manner. I expect I'm largely just being change-averse here. However I'm forewarning you now, as this will no-doubt come across in my tone, and my patience when things don't go exactly how I'd like them too (like some sort of spoilt brat). But if you've read this blog before… you probably already expect this.

Now that I have that out of my system (maybe), what's my aim for today? When I started yesterday, the intent of my work was described in this functional test:

/** @coversNothing */
public function testGreetingsEndPointReturnsPersonalisedGreeting()
{
    $testName = 'Zachary';
     $expectedGreeting = (object) [
        'name' => $testName,
        'greeting' => "G'day $testName"
    ];

    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get(
        "greetings/$testName/",
        ['http_errors' => false]
    );
    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

    $contentTypes = $response->getHeader('Content-Type');
    $this->assertCount(1, $contentTypes);
    $this->assertSame('application/json', $contentTypes[0]);

    $responseBody = $response->getBody();
    $responseObject = json_decode($responseBody);
    $this->assertEquals($expectedGreeting, $responseObject);
}

In humanspeke, what I'm gonna do is:

  • create a route /greetings/[some name here]/;
  • return an object as JSON;
  • that confirms the name sent, plus has a greeting string for that name.

Very "Hello World". I did indicate I was not gonna be pushing the boat out too much here.

To start let's run that test, and watch it fail with a 404…

root@5565d6f15aca:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.F....                                                              6 / 6 (100%)

Time: 00:04.414, Memory: 14.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\tests\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
Failed asserting that 404 matches expected 200.

/usr/share/fullstackExercise/tests/functional/SymfonyTest.php:63

FAILURES!
Tests: 6, Assertions: 12, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.162]
root@5565d6f15aca:/usr/share/fullstackExercise#

That one failure is the one we want to see, so that's good: I have a test failing in exactly they way I expect it to be, so now I need to work out how to add a route and all that sort of jazz. Time to RTFM. Back soon.

Right so the docs for Symfony have been incredibly helpful so far. That's not sarcasm: they've been really bloody good! I'd been looking at the Installing & Setting up the Symfony Framework page, and one of its last links is to Create your First Page in Symfony. This steps one through setting up a route and the controller that services it. The first example did not have a "runtime" parameter in the URL slug like I need here, but that was covered in another linked-to page Routing (specifically the Route Parameters section of same). That was all the information I seemed to need for my code, so off I went.

Routing is done in config/routes.yaml, and there's an example in there already. So it was easy to stick my new route in:

#index:
#    path: /
#    controller: App\Controller\DefaultController::index

greetings:
  path: /greetings/{name}
  controller: adamCameron\fullStackExercise\Controller\GreetingsController::doGet

Curly braces around the {name} just mean that that part of the URL slug is dynamic, and its value is passed to the controller method. I could (and should!) put validation on this, but that's getting ahead of myself. The current requirement I have set only handles the happy path, so we'll stick to that.

The controller goes in the src/Controller directory. I'm more accustomed to using headlessCamelCase for my namespace parts, but I'll stick with Symfony's precedent here:

namespace adamCameron\fullStackExercise\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;

class GreetingsController extends AbstractController
{
    public function doGet(string $name) : Response
    {
        $greetingsResponse = [
            'name' => $name,
            'greeting' => "G'day $name"
        ];

        return new JsonResponse($greetingsResponse);
    }
}

This shows how the dynamic part of the slug from the route passes through into the controller method. Cool. From there I've gone my own way from the docs there cos Silex uses Symfony's request/response mechanism, so I know I can just return a JsonResponse like that, and it'll handle the 200-OK and the application/json part of the requirement. The docs integrate using Twig here to render some output, but there's no need for that complexity here. I suppose here the creation of $greetingsResponse is my "model", and the decision to just return a JsonResponse is my "view".

Being quite pleased with myself at how simple that was, I ran my test to verify how clever I was:

root@3fc72bf44d38:/usr/share/fullstackExercise# vendor/bin/phpunit tests/functional/SymfonyTest.php --filter=testGreetingsEndPointReturnsPersonalisedGreeting
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 00:02.607, Memory: 8.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\tests\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
Failed asserting that 500 matches expected 200.

/usr/share/fullstackExercise/tests/functional/SymfonyTest.php:63

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.790]
root@3fc72bf44d38:/usr/share/fullstackExercise#

rrarr! OK, not so clever after all it seems. Hitting the URL in the browser gives me more information:


What is this thing on about? See where it's looking for my controller? It's looking for a class App\Controller\GreetingsController, but that class is not in App/Controller, it's in adamCameron/fullStackExercise/Controller, and that's where the route code says it is. I have re-checked everything, and it's all legit.

Sigh. I could guess what it is. Symfony being a) "clever", and b) "opinionated". If you've just read the previous article, you might recall me raising my eyebrow at this bit in composer.json:

"autoload": {
    "psr-4": {
        "App\\": "src/",
        "adamCameron\\fullStackExercise\\": "src/"
    }
},

At the time I was just observing what a dumb namespace that was, but I'm now guessing that in Symfony's opinion that is the namespace we all should be using. I remembered something about some other autowiring config that Symfony has, and guessed there was something in there that might cater to classes in the App namespace, but not other namespaces. Even if the namespaces are explicit in the code (as they are here). I located the culprit in config/services.yaml:

services:
    # default configuration for services in *this* file
    _defaults:
        autowire: true      # Automatically injects dependencies in your services.
        autoconfigure: true # Automatically registers your services as commands, event subscribers, etc.

    # makes classes in src/ available to be used as services
    # this creates a service per class whose id is the fully-qualified class name
    App\:
        resource: '../src/'
        exclude:
            - '../src/DependencyInjection/'
            - '../src/Entity/'
            - '../src/Kernel.php'
            - '../src/Tests/'

    # controllers are imported separately to make sure services can be injected
    # as action arguments even if you don't extend any base controller class
    App\Controller\:
        resource: '../src/Controller/'
        tags: ['controller.service_arguments']

    # add more service definitions when explicit configuration is needed
    # please note that last definitions always *replace* previous ones

So there's magic taking place for App; I guess I need to make it magical for my actual namespace instead. I'm never gonna write code with App as its namespace, so I'm just gonna punt that I can change that to adamCameron\fullStackExercise\: and adamCameron\fullStackExercise\Controller\: and it'll all be fine. I think ATM I only need to monkey with the Controller one, but I might as well do both now I guess. So with that change in place, I re-run the test:

root@3fc72bf44d38:/usr/share/fullstackExercise# vendor/bin/phpunit tests/functional/SymfonyTest.php --filter=testGreetingsEndPointReturnsPersonalisedGreeting
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:06.440, Memory: 8.00 MB

OK (1 test, 4 assertions)

Generating code coverage report in HTML format ... done [00:01.825]
root@3fc72bf44d38:/usr/share/fullstackExercise#

All right, that was easily located / solved and is not the end of the world. However this from Symfony's error message irks me: "The file was found but the class was not in it, the class name or namespace probably has a typo". No mate. That's not the problem at all. The problem is that despite me giving you the exact fully-qualified namespaced class name in the route config, and the namespace in the class file was correct, and in the right place in the file system… and if you just left things be then you would have found the correct class. But: no. You had to try to be clever, and you failed miserably. I really frickin' hate it when technology goes "nono [patronising grimace] I've got this, I'll do it for you" (I know some devs like this too…), and it gets it wrong. I'm all for Symfony having the helpers there should one want them, but it shouldn't make these assumptions. I guess had it gone "hey aaah… that namespace yer using for yer controller? Ya need to configure that to work mate. Go have a look in config.yaml", then I'd just be going "yeah nice one: thanks for that". But then again most people would probably not have written this ranty paragraph at all, and just moved on with their lives, eh? Hey at least I'm self-aware.

All in all, even with the config/namespace hiccup, getting a simple route working was pretty bloody easy here. And this is a good thing about Symfony.

That was actually pretty short. It was about 500 words and some test-runs longer, as I had encountered a weird issue which I was moaning about. However when doing a final read-through of this before pressing "send", I looked at it some more, and it turns out it was just me being a div, and the problem was firmly of the PEBCAK variety. So… erm… I decided to delete that bit. Ahem.

Next I have to start doing some front-end-ish stuff so I've got an excuse to try out Vue.js. This should be interesting. I've never even looked @ Vue.js before. I bet I'll disagree with some of its opinions eh? Stay tuned…

Righto.

--
Adam

Wednesday, 13 January 2021

Part 6: Installing Symfony

G'day:

These initial boilerplate paragraphs must be getting annoying by now. Sorry. But anyhow, here we go again. Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony (this article)
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the sixth article in the series, and follows on from Part 5: MariaDB. It's probably best to go have a breeze through the earlier articles first. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject.

For the earlier articles I had mostly tried-out what I was intending to do before I started writing about it. This time I am writing this at the same time as doing the work. To contextualise, all I have done at the moment is open up that article of Martin Pham's I've been following along ("Symfony 5 development with Docker"), and also got the Symfony docs up: "Installing & Setting up the Symfony Framework / Creating Symfony Applications".

First things first I want a failing test for my minimum requirements here, which is that when I GET /greeting/[some name here]/, I receive back a JSON response, along these lines: {"name":"[some name here]","greeting":"G'day [some name here]"}. Here's a test that tests this (tests/functional/SymfonyTest.php):

namespace adamCameron\fullStackExercise\test\functional;

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class SymfonyTest extends TestCase
{
    /** @coversNothing */
    public function testGreetingsEndPointReturnsPersonalisedGreeting()
    {
        $testName = 'Zachary';
        $expectedGreeting = (object) [
            'name' => $testName,
            'greeting' => "G'day $testName"
        ];

        $client = new Client([
            'base_uri' => 'http://webserver.backend/'
        ]);

        $response = $client->get("greeting/$testName/");
        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $contentTypes = $response->getHeader('Content-Type');
        $this->assertCount(1, $contentTypes);
        $this->assertSame('application/json', $contentTypes[0]);

        $responseBody = $response->getBody();
        $responseObject = json_decode($responseBody);
        $this->assertEquals($expectedGreeting, $responseObject);
    }
}

The code is pretty clear, I just perform the GET using "Zachary" as the name, and in return I verify I get a 200 OK response, it's JSON and it has the expected data structure in the body. Simple.

I run the test to check I get a fail with a 404:

root@20ad01609407:/usr/share/fullstackExercise# vendor/bin/phpunit test/functional/SymfonyTest.php
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 00:00.670, Memory: 6.00 MB

There was 1 error:

1) adamCameron\fullStackExercise\test\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
GuzzleHttp\Exception\ClientException: Client error: 'GET http://webserver.backend/greetings/Zachary/' resulted in a '404 Not Found' response:
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1. (truncated...)

OK what? Why the hell is Guzzle deciding a 404 response - which is completely legitimate - is in some way "exceptional", and warrants an exception being thrown. That's daft. Lemme go RTFM and see if there's a way of me configuring Guzzle to not get ahead of itself and just behave. back in a bit…

… OK, easily solved. There's a config param for the get call that switches that behaviour off. Thanks to the Guzzle docs for making that easy to find ("Request options / http-errors").

$response = $client->get(
    "greetings/$testName/",
    ['http_errors' => false]
);

And now we get a test that fails as we'd expect it to:

root@20ad01609407:/usr/share/fullstackExercise# vendor/bin/phpunit test/functional/SymfonyTest.php
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 00:00.549, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
Failed asserting that 404 matches expected 200.

/usr/share/fullstackExercise/test/functional/SymfonyTest.php:28

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.166]
root@20ad01609407:/usr/share/fullstackExercise#

We're now OK to work out how to get Symfony installed and running, and make an end point that behaves as we need it to.

Or are we? I think I might have got a bit ahead of myself here. I'm back looking at Martin's article, and he says

Let's proceed to the Symfony installation:

$ symfony new src

[…] open http://localhost, you will see a Symfony 5 welcome screen.

OK so I think our first test should be that that Symfony 5 welcome screen works. I've got no idea what the screen says, so let me google for expectations. Right according to Creating your First Symfony App and Adding Authentication by Olususi Oluyemi (from a random Google Images search for "symfony 5 welcome screen") it should look like this:

I'm gonna write a quick test to check for that, and bypass that other test for now:

/** @coversNothing */
public function testSymfonyWelcomeScreenDisplays()
{

    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get(
        "/",
        ['http_errors' => false]
    );
    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

    $html = $response->getBody();
    $document = new \DOMDocument();
    $document->loadHTML($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
    $this->assertCount(1, $hasTitle);
}

I was scratching my head guessing what the mark-up around the "Welcome to Symfony x.y.z" text in the body is. It could be <h2> and <h3> tags, or it could just be different font-size stylings. Then I noticed the <title> content, and just decided to check for that. If that string is there: we're good. To start with, we want to be not good, so let's run that test:

root@20ad01609407:/usr/share/fullstackExercise# vendor/bin/phpunit test/functional/SymfonyTest.php
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 00:00.555, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\SymfonyTest::testSymfonyWelcomeScreenDisplays
Failed asserting that actual size 0 matches expected size 1.

/usr/share/fullstackExercise/test/functional/SymfonyTest.php:32

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.195]
root@20ad01609407:/usr/share/fullstackExercise#

Right so both Martin's article and the Symfony docs say I need to install Symfony (via Composer), and create an application with it, so I'll do this now.

The first hiccup is that the Symfony installer assumes that yer Symfony app is the centre of the world, and it needs to be the one that creates yer project directory for you. Great. Thanks for that. There's probably (?) a way of coercing it to understand that it's just a frickin' framework and I just need to add it to my existing app which is already underway. This is not so far--fetched I think, but after 15min of googling I drew a blank, so I decided to take a different approach. I'm gonna blow away my stuff to allow Symfony to play its silly little game, and then reintegrate my stuff back into the stuff Symfony creates. After all it's just some tests and composer.json config at present really.

/me thinks things through… OK that won't work. I can't shift all the files out of the fullstackExercise directory because all the Docker stuff is in there, and I can't bring my containers up if they're not there. Without the containers up, I don't have PHP or Composer so I can't do the Symfony install. To be blunt here: screw you, Symfony (well it's actually Composer that is bleating about the directory not being empty I guess. So screw Composer). With Silex? Just add an entry to composer.json, add a line to index.php and that's it. This is how a microframework should work.

Plan B is to do things the other way around. I need to shift my fullStackExercise project directory to have a different name (fullStackExercise.bak) temporarily so I can allow Symfony to create its own one. To do this I need to rejig my docker/docker-compose.yml file and the docker/php-fpm/Dockerfile to reference fullstackExercise.bak. Note that docker/nginx/sites/default.conf file also references root /usr/share/fullstackExercise/public, so at this point I decided that for now I'll just bring up the PHP container for this, so stripped the Nginx and MariaDB stuff out of docker-compose.yml completely. So these are the file changes:

docker-compose.yml:

version: '3'

services:

  php-fpm:
    build:
      context: ./php-fpm
    environment:
      - DATABASE_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
    volumes:
      - ../..:/usr/share/src # was ..:/usr/share/fullstackExercise
      - ./php-fpm/root_home:/root
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

networks:
  backend:
    driver: "bridge"

Note that I've shifted the main volume up one level in the host's directory hierarchy. This is because I need to let Symfony recreate that fullStackExercise at that parent level.

php-fpm/Dockerfile:

FROM php:8.0-fpm
RUN apt-get update
RUN apt-get install git --yes
RUN apt-get install net-tools iputils-ping --yes
RUN apt-get install zip unzip --yes
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
ENV XDEBUG_MODE=coverage
#WORKDIR  /usr/share/fullstackExercise.bak/
WORKDIR  /usr/share/src/
CMD composer install ; php-fpm
EXPOSE 9000

I've made the same change to the WORKDIR here too, to match docker-compose.yml. I've also added in a line to install zip and unzip into the container, because when I was testing this, Composer was bleating about them not being present.

Now in theory if I drop my containers and bring them back up again, I should be able to bash into the PHP container and do the Symfony install:

adam@DESKTOP-QV1A45U:~$ cd /mnt/c/src/fullstackExercise.bak/docker
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ docker-compose down --remove-orphans
Removing network docker_backend
WARNING: Network docker_backend not found.
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ # OK I forgot I had already brought them down

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building php-fpm
[...]
Successfully built 4b8851fefd22
Successfully tagged docker_php-fpm:latest
Creating docker_php-fpm_1 ... done

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@87420db3da90:/usr/share/src# ll
total 4
drwxrwxrwx 1 1000 1000  512 Jan  1 16:16 ./
drwxr-xr-x 1 root root 4096 Jan  1 16:40 ../
drwxrwxrwx 1 1000 1000  512 Dec 20 12:58 fullstackExercise.bak/
root@87420db3da90:/usr/share/src#
root@87420db3da90:/usr/share/src# # so far so good
root@87420db3da90:/usr/share/src#
root@87420db3da90:/usr/share/src# composer create-project symfony/skeleton fullstackExercise
Creating a "symfony/skeleton" project at "./fullstackExercise"
Installing symfony/skeleton (v5.2.99)
[...]

Some files may have been created or updated to configure your new packages.
Please review, edit and commit them: these files are yours.


What's next?


  * Run your application:
    1. Go to the project directory
    2. Create your code repository with the git init command
    3. Download the Symfony CLI at https://symfony.com/download to install a development web server

  * Read the documentation at https://symfony.com/doc

root@87420db3da90:/usr/share/src#

So far so good. now I just need to move a bunch of stuff back from my app into this one. I'll go ahead and do it, and report back when done, and list what I needed to do…

  • .git/, .idea/ (this is PHPStorm's project config), docker/, log/ directories could just be copied straight across.
  • As could LICENSE, phpcs.xml, phpmd.xml, phpunit.xml, README.md.
  • And src/MyClass.php, public/gdayWorld.html and public/gdayWorld.php.
  • The only overlap in .gitignore was the vendor/ directory, otherwise I could just move my bits back in.
  • composer.json was fairly straight forward, but I raised an eyebrow that Symfony seems to suggest using App as a top-level namespace:
    "autoload": {
        "psr-4": {
            "App\\": "src/",
    

    To me, using a namespace "App" is like calling a variable something like indexVariable, or totalVariable or something, or naming a class AnimalClass. Ugh. It gets even worse with the testing namespace: App\\Tests\\. For now I'm just adding my own namespacing in as well. Hopefully I don't have to use Symfony's approach here.
  • Symfony uses tests as its test directory; I've always used test, but I'm ambivalent about this so am sticking with Symfony's lead here: I'll change my directory to tests. This will require me changing the namespaces in the test source code files too.
  • Because I've changed composer.json by hand, I'll need get rid of composer.lock and vendor/ and do a composer install again.
  • Lastly I've reverted the Docker files I'd changed above back to how they were before: removing the references to fullstackExercise.bak, and bringing back the Ngxin and MariaDB containers.

OK having done all that, and now will rebuild my containers. Before starting this paragraph I had done all this and had started to fight with an issue I thought I was having with PHPUnit, so I didn't capture the screen showing it all working. It's all just Docker bumpf, then some Composer bumpf, none of which was very surprising, so I'll spare you the detail. Back to the unit test run:

root@51b5afdff6ea:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F....                                                               5 / 5 (100%)

Time: 00:02.896, Memory: 14.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\tests\functional\SymfonyTest::testSymfonyWelcomeScreenDisplays
Failed asserting that 404 matches expected 200.

/usr/share/fullstackExercise/tests/functional/SymfonyTest.php:23

FAILURES!
Tests: 5, Assertions: 10, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.152]
root@51b5afdff6ea:/usr/share/fullstackExercise#

Doh!

Things get weirder cos I hit the site on my host browser:

And that seems fine. First troubleshooting step: factor-out all the code I'm running, and manually curl it from PHP:

root@51b5afdff6ea:/usr/share/fullstackExercise# php -a
Interactive shell

php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_URL, "webserver.backend");
php > curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
php > $output = curl_exec($ch);
php > curl_close($ch);
php > echo $output;
<!DOCTYPE html>
<html dir="ltr" lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="robots" content="noindex,nofollow,noarchive,nosnippet,noodp,notranslate,noimageindex" />
    <title>Welcome to Symfony!</title>

So there's all the content. Fine.

Next I want to check just Guzzle by itself, to demonstrate to myself it is working fine. I hand-cranked some Guzzle calls from outside of my test code:

root@51b5afdff6ea:/usr/share/fullstackExercise# php -a
Interactive shell

php > require 'vendor/autoload.php';
php > $client = new \GuzzleHttp\Client(['base_uri' => 'http://webserver.backend']);
php > $response = $client->get("/");

Warning: Uncaught GuzzleHttp\Exception\ClientException: Client error: `GET http://webserver.backend/` resulted in a `404 Not Found` response:
<!DOCTYPE html>
<html dir="ltr" lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="robots" content="noindex, (truncated...)

It's still frickin' 404ing.

I scratched my head a lot and googled why guzzle might 404 on something that everything else 200s on. Then I saw it.

That mark-up that is spewing out in the Guzzle error is not generic 404 response. What it is is the content from the Welcome to Symfony! page. I checked something else in my browser…

That "welcome" page? It has a 404 status. Now I can see what they're doing here, and they even warn about it if one pays attn:

However to me a welcome page - that actually has welcome content on it, therefore is precisely what one would be expecting on that URL- is not a 404 situation. Ah well.

So my test is reporting correctly! (Yay for tests!) It's just that the expectations I gave the test were wrong. I need to expect a 404, not a 200. I'll make that change, and now the tests all pass:

root@51b5afdff6ea:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.....                                                               5 / 5 (100%)

Time: 00:07.614, Memory: 14.00 MB

OK (5 tests, 11 assertions)

Generating code coverage report in HTML format ... done [00:01.173]
root@51b5afdff6ea:/usr/share/fullstackExercise#

Phew. BTW I had to change my test a bit more than I expected:

/** @coversNothing */
public function testSymfonyWelcomeScreenDisplays()
{

    $client = new Client([
        'base_uri' => 'http://webserver.backend'
    ]);

    $response = $client->get(
        "/",
        ['http_errors' => false]
    );
    // unexpectedly perhaps: the welcome page returns a 404. This is "by design"
    $this->assertEquals(Response::HTTP_NOT_FOUND, $response->getStatusCode());

    $html = $response->getBody();
    $document = $this->loadHtmlWithoutHtml5ErrorReporting($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
    $this->assertCount(1, $hasTitle);
}

private function loadHtmlWithoutHtml5ErrorReporting($html) : \DOMDocument
{
    $document = new \DOMDocument();
    libxml_use_internal_errors(TRUE);
    $document->loadHTML($html);
    libxml_clear_errors();

    return $document;
}

There's a bug in PHP's HTML loading that makes it barf on HTML5 tags (see "PHP DOMDocument errors/warnings on html5-tags", so I needed to put some mitigation in for that. It's not a great solution, but it works.

Now to try to work out how to implement my /greetings/Zachary route…

But not for now. I've just checked the length of this, and not counting all the code and telemetry, this is already close to 1800 words (it'll be over by the time I finish this outro). I think this exercise of getting the Symfony framework code into my project and up and running is sufficient for a stand-alone article. I'll do a separate article for the "and now let's get it to do something except return a confusing (to me. Cringe) 404 response". I'll crack on with that tomorrow, over here: "Part 7: Using Symfony".

Righto.

-- 
Adam

Tuesday, 12 January 2021

Part 5: MariaDB

G'day:

Please note that I initially intended this to be a part of a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB (this article)
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the fifth article in the series, and follows on from Part 4: Tweaks I made to my Bash environment in my Docker containers. It's probably best to go have a breeze through the earlier articles first just to contextualise things. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject.

OK, let's get on with this MariaDB stuff.

Firstly; why am I using MariaDB instead of MySQL? Initially I started pottering around with MySQL on Docker for another piece of work I was doing, and I ran up against a show-stopper that doesn't seem to have a resolution. It's detailed on GitHub at "MySQL docker 5.7.6 and later fails to initialize database", and demonstrated here:

adam@DESKTOP-QV1A45U:~$ docker pull mysql
Using default tag: latest
latest: Pulling from library/mysql
6ec7b7d162b2: Pull complete
[...]
a369b92bfc99: Pull complete
Digest: sha256:365e891b22abd3336d65baefc475b4a9a1e29a01a7b6b5be04367fcc9f373bb7
Status: Downloaded newer image for mysql:latest
docker.io/library/mysql:latest
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker create --name mysql --expose 3306 -p 3306:3306 --interactive --volume /var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123 --tty mysql
dfabbf24a7b76831cdb95d20302cc46587cfeb2f7b9f63ac2d907fb8505b07b8
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker start --interactive mysql
2020-12-20 12:18:56+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.22-1debian10 started.
2020-12-20 12:18:56+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2020-12-20 12:18:56+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.22-1debian10 started.
2020-12-20 12:18:56+00:00 [Note] [Entrypoint]: Initializing database files
2020-12-20T12:18:56.174257Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.22) initializing of server in progress as process 46
2020-12-20T12:18:56.183150Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2020-12-20T12:18:56.186621Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2020-12-20T12:18:58.227909Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
mysqld: Cannot change permissions of the file 'ca.pem' (OS errno 1 - Operation not permitted)
2020-12-20T12:19:00.524436Z 0 [ERROR] [MY-010295] [Server] Could not set file permission for ca.pem
2020-12-20T12:19:00.524927Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
2020-12-20T12:19:00.525836Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-12-20T12:19:02.414805Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.22) MySQL Community Server - GPL.
adam@DESKTOP-QV1A45U:~$

I tried everything suggested in that thread, everything else I could find, and nothing improved the situation. However this entry on that issue page above looked like good advice:



So I just decided to run with MariaDB instead, and that worked perfectly:

adam@DESKTOP-QV1A45U:~$ docker create --name mariadb --expose 3306 -p 3306:3306 --interactive --volume /var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123 --tty mariadb
Unable to find image 'mariadb:latest' locally
latest: Pulling from library/mariadb
da7391352a9b: Pull complete
[...]
a33f860b4aa6: Pull complete
Digest: sha256:cdc553f0515a8d41264f0855120874e86761f7c69407b5cfbe49283dc195bea8
Status: Downloaded newer image for mariadb:latest
bb2a4128911e52f2b16a25c4f994fe12eeec3c36a7e9e188cba2758522785522
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker start mariadb
2020-12-20 12:27:15+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.8+maria~focal started.
2020-12-20 12:27:15+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2020-12-20 12:27:15+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.8+maria~focal started.
2020-12-20 12:27:15+00:00 [Note] [Entrypoint]: Initializing database files
[... bunch of stuff snipped ...]
mariadb
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$
adam@DESKTOP-QV1A45U:~$ docker exec -it mariadb mariadb --user=root --password=123
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 10.5.8-MariaDB-1:10.5.8+maria~focal mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@VERSION;
+-------------------------------------+
| @@VERSION |
+-------------------------------------+
| 10.5.8-MariaDB-1:10.5.8+maria~focal |
+-------------------------------------+
1 row in set (0.000 sec)

MariaDB [(none)]>

Now I don't doubt it's possible to get MySQL working in my environment, but… erm… shrug. I don't care. I just need a DB running. I'm not here to fart-arse around with DBs any more than absolutely necessary.

We should actually now back-up a bit. All that stuff above was done a while ago - although I replicated it just now for the sake of the notes above - and for my current exercise we're getting ahead of ourselves. I'm gonna start this exercise with a failing test (test/integration/DatabaseTest.php):

namespace adamCameron\fullStackExercise\test\integration;

use PHPUnit\Framework\TestCase;
use \PDO;

class DatabaseTest extends TestCase
{
    /** @coversNothing */
    public function testDatabaseVersion()
    {
        $connection = new PDO(
            'mysql:dbname=mysql;host=database.backend',
            'root',
            '123'
        );

        $statement = $connection->query("show variables where variable_name = 'innodb_version'");
        $statement->execute();

        $version = $statement->fetchAll();

        $this->assertCount(1, $version);
        $this->assertSame('10.5.8', $version[0]['Value']);
    }
}

Note: in a better world, I'd never have my user and password hard-coded there (see further down for where I address this), even in a test. And I'd not be using root (I don't address this one quite yet though). Also checking the version right down to the patch level is egregious, I know. Just checking for 10 would perhaps be better there.

This fails as one would expect:

root@5962e5abd527:/usr/share/fullstackExercise# vendor/bin/phpunit test/integration/
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 00:00.245, Memory: 6.00 MB

There was 1 error:

1) adamCameron\fullStackExercise\test\integration\DatabaseTest::testDatabaseVersion
PDOException: SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known

/usr/share/fullstackExercise/test/integration/DatabaseTest.php:15

Caused by
PDOException: PDO::__construct(): php_network_getaddresses: getaddrinfo failed: Name or service not known

/usr/share/fullstackExercise/test/integration/DatabaseTest.php:15

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Generating code coverage report in HTML format ... done [00:01.786]

Once we've installed the MariaDB container, got it up and running and networked it: this test should pass.

I'm back to taking Martin Pham's lead in the Docker config for all this (reminder, from his article "Symfony 5 development with Docker")

Here's the docker/mariadb/Dockerfile:

FROM mariadb:latest
CMD ["mysqld"]
EXPOSE 3306
Quick sidebar whilst I'm doing a final edit of this. If I'm testing for a specific version 10.5.8, should I perhaps be forcing that version here too? Hmmm… probably.

No surprises there. Next the stuff in docker/docker-compose.yml:

  mariadb:
    build:
      context: ./mariadb
    environment:
      - MYSQL_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
    ports:
      - "3306:3306"
    volumes:
      - ./mariadb/data:/var/lib/mysql
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      backend:
        aliases:
          - database.backend

One new thing for me here is the ${DATABASE_ROOT_PASSWORD}. Looking at Martin's set-up, he's also got a file docker/.env. Seems to me one can sling environment variables in there, and docker-compose picks them up automatically. So I've just got this (docker/.env):

DATABASE_ROOT_PASSWORD=123

I'll come back to this in a bit.

Also note that I'm configuring MariaDB to put its data in a volume back on the host machine. This is so the data persists when I shut the container down.

And now I should be able to start everything up, and it'll just work. Right?

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building nginx
[...]
Successfully built 7cb155649c3b
Successfully tagged docker_nginx:latest

Building php-fpm
[...]
Successfully built e483795cc006
Successfully tagged docker_php-fpm:latest

Building mariadb
[...]
Successfully built 1f05ad3e3ad3
Successfully tagged docker_mariadb:latest

Creating docker_mariadb_1 ... done
Creating docker_nginx_1 ... done
Creating docker_php-fpm_1 ... done

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

OK that's more promising than I expected. How about that integration test?

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@4861480bcbad:/usr/share/fullstackExercise# vendor/bin/phpunit test/integration/
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:00.200, Memory: 6.00 MB

OK (1 test, 2 assertions)

Generating code coverage report in HTML format ... done [00:01.215]
root@4861480bcbad:/usr/share/fullstackExercise#

Gasp (I actually did gasp a bit). Blimey. It only went and worked first time.

I'm intrigued by this .env stuff. I figured if I can set that root password in the .env file, then I can shift it out of my integration test, and just use the environment variable. So I've updated the php-fpm section of docker-compose.yml to also set than environment variable:

php-fpm:
  build:
    context: ./php-fpm
  environment:
    - DATABASE_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
  # etc

And update the integration test to use that:

$connection = new PDO(
    'mysql:dbname=mysql;host=database.backend',
    'root',
    $_ENV['DATABASE_ROOT_PASSWORD']
);

And test that's all OK:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down --remove-orphans
[...] adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
[...] adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@48dedafac625:/usr/share/fullstackExercise# env | grep DATABASE_ROOT_PASSWORD
DATABASE_ROOT_PASSWORD=123
root@48dedafac625:/usr/share/fullstackExercise# vendor/bin/phpunit test/integration/
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:00.196, Memory: 6.00 MB

OK (1 test, 2 assertions)

Generating code coverage report in HTML format ... done [00:01.195]
root@48dedafac625:/usr/share/fullstackExercise#

Cool!

Right so that's all that done: it was very easy (thanks largely to Martin's guidance). Note that once I get underway with the app I'll have a specific database to use, and specific credentials to use against it; at that point I'll stop using the root credentials in that test. But for where we are now - just checking that the DB is up and networked and PHP can see it: the test I'm doing is fine.

That was quite a brief article. In the next one I'll get on to installing Symfony.

Righto.

--
Adam

Monday, 11 January 2021

Part 4: Tweaks I made to my Bash environment in my Docker containers

G'day:

Intro

Please note that this is a sub-article of a larger body of work that is an exercise in setting up a Vue.js-driven website backed by PHP8 and MariaDB running in Docker containers. All of this is completely new to me, so is a learning exercise, rather than some exposition of my wisdom (which I have none of). I initially intended the whole exercise to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers (this article)
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the fourth article in the series, and - chronologically - follows on from Part 3: PHPUnit. That said, this article is reasonably stand-alone, so not sure if it's really necessary to read everything else first. The only real "cross over" is that I'm experimenting within Docker containers that I created in the earlier articles. I guess you can refer back to the other articles if anything I say here seems to be making assumptions you can't make.

Also please note that there is very little that is earthshattering here. It's more an exercise of me - as a relative *nix noob - getting stuff working how I've become accustomed to it. I discuss nothing tricky or advanced here.

Today's exercise

Whilst doing all the crap to get Nginx, PHP and PHPUnit working, I was spending an awful lot of time in and out of Bash, running various bits of code and testing stuff and the like. I found there were a few annoying things about running Bash in these containers:

  • The Debian distro that the PHP container uses doesn't include ping and other network utils I was needing to use.
  • Bash history works for the life of the container, but does not - obviously - live between rebuilds of a container. This was a pain in the butt as I had a bunch of things to run every rebuild to check stuff.
  • I needed to add a coupla aliases: ll which everyone has; and one, cls, that I've got that is a bit nicer than just clear

In this article I'm gonna work through implementing those, and a coupla other things.

Adding the networking utils was a one-liner in the docker/php-fpm/Dockerfile:

FROM php:8.0-fpm
RUN apt-get update
RUN apt-get install git --yes
RUN apt-get install net-tools iputils-ping --yes
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
#COPY ./.bashrc /root/.bashrc
ENV XDEBUG_MODE=coverage
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

That just installs both net-tools and iputils-ping. The --yes just suppresses confirmation the installation asks, which breaks the Docker build if I don't answer it in a batch fashion: the Docker build process is not interactive.

Sorting out the Bash history was slightly trickier. My initial plan was to just mount a stubbed file in the container (this from docker/docker-compose.yml):

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
      - ./php-fpm/.bash_history:/root/.bash_history
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

This has some drawbacks. The build broke if ./php-fpm/.bash_history wasn't in the host file system. To guarantee it was there, I needed it in source control. But… by its very nature it's getting changed all the time, which gets annoying when I go to commit stuff. There's ways to work around this in git, but those were causing issues as well.

In the end I decided to take this approach:

    volumes:
      - ..:/usr/share/fullstackExercise
      - ./php-fpm/root_home:/root

That's the entire home directory for the root user. By default it has nothing in it, so I don't need to replicate much in the file system in source control: just make the directory exist, and .gitignore .bash_history:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ tree -aF --dirsfirst -L 2 php-fpm/root_home/
php-fpm/root_home/
├── .bash_history*
└── .gitignore*

0 directories, 2 files

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ cat php-fpm/root_home/.gitignore
*
!.gitignore

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

And this just works! Below I show rebuilding my containers, going into bash on the PHP container, showing the empty history, doing some stuff then exiting. Then I repeat the whole operation and you can see the history is sticking across containers now.

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down --remove-orphans
Stopping docker_nginx_1   ... done
Stopping docker_php-fpm_1 ... done
Removing docker_nginx_1   ... done
Removing docker_php-fpm_1 ... done
Removing network docker_backend
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building nginx
[...]
Building php-fpm
[...]
Creating docker_nginx_1   ... done
Creating docker_php-fpm_1 ... done
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@fd43b546b8c0:/usr/share/fullstackExercise# history
    1  history
root@fd43b546b8c0:/usr/share/fullstackExercise# cat ~/.bash_history
root@fd43b546b8c0:/usr/share/fullstackExercise# ls
LICENSE    _public        composer.lock  log        phpmd.xml    public  test
README.md  composer.json  docker         phpcs.xml  phpunit.xml  src     vendor
root@fd43b546b8c0:/usr/share/fullstackExercise# exit
exit
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose down --remove-orphans
Stopping docker_php-fpm_1 ... done
Stopping docker_nginx_1   ... done
Removing docker_php-fpm_1 ... done
Removing docker_nginx_1   ... done
Removing network docker_backend
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building nginx
[...]
Building php-fpm
[...]
Creating docker_nginx_1   ... done
Creating docker_php-fpm_1 ... done
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@9b7b60d6208a:/usr/share/fullstackExercise# history
    1  history
    2  cat ~/.bash_history
    3  ls
    4  exit
    5  history
root@9b7b60d6208a:/usr/share/fullstackExercise# cat ~/.bash_history
history
cat ~/.bash_history
ls
exit
root@9b7b60d6208a:/usr/share/fullstackExercise# exit
exit
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Quite pleased with that, I am.

Given how I'm voluming-in the entire home directory, doing the aliases as easy enough, I just needed to drop a .bashrc into that docker/php-fpm/root_home directory. The image doesn't have a .bashrc by default, so this is fine.

alias ll='ls -alF'
alias cls='clear; printf "\033[3J"'

There's a coupla things to note here. Initially by accident I saved this file with CRLF line endings, and so the ll alias didn't work:

root@50fbd03160ef:/usr/share/fullstackExercise# ll
's: invalid option -- '
Try 'ls --help' for more information.
root@50fbd03160ef:/usr/share/fullstackExercise#

That's a trap for young players there (even old burnt-out ones like me, too). It was not immediately obvious what the issue is there, so it took a bit of googling and stack-overflow-ing to find the answer. Something to remember. But once one sorts that out, it works fine:

root@5962e5abd527:/usr/share/fullstackExercise# ll
total 173
drwxrwxrwx 1 1000 1000    512 Dec 14 20:14 ./
drwxr-xr-x 1 root root   4096 Dec 14 18:32 ../
drwxrwxrwx 1 1000 1000    512 Dec 14 19:03 .git/
-rwxrwxrwx 1 1000 1000     49 Dec 14 17:59 .gitignore*
drwxrwxrwx 1 1000 1000    512 Dec 14 20:14 .idea/
-rwxrwxrwx 1 1000 1000  35149 Dec 11 11:29 LICENSE*
-rwxrwxrwx 1 1000 1000    119 Dec 11 11:29 README.md*
-rwxrwxrwx 1 1000 1000    563 Dec 14 14:01 composer.json*
-rwxrwxrwx 1 1000 1000 123727 Dec 14 17:00 composer.lock*
drwxrwxrwx 1 1000 1000    512 Dec 14 19:10 docker/
drwxrwxrwx 1 1000 1000    512 Dec  5 13:19 log/
-rwxrwxrwx 1 1000 1000    479 Dec 14 17:00 phpcs.xml*
-rwxrwxrwx 1 1000 1000   1913 Dec 14 17:00 phpmd.xml*
-rwxrwxrwx 1 1000 1000    709 Dec 14 17:00 phpunit.xml*
drwxrwxrwx 1 1000 1000    512 Dec 14 17:00 public/
drwxrwxrwx 1 1000 1000    512 Dec 14 15:36 src/
drwxrwxrwx 1 1000 1000    512 Dec 14 17:00 test/
drwxrwxrwx 1 1000 1000    512 Dec 14 14:01 vendor/
root@5962e5abd527:/usr/share/fullstackExercise#

The cls alias I make just solves something that annoys me about clear is that it doesn't clear the scrollback history when SSHed into the box, so if one mouse-scrolls up, one can still move past where you cleared. This makes things hard to find sometimes, and defeats the purpose of clear IMO. The escape sequence in the alias alias cls='clear; printf "\033[3J"' just clears the scrollback as well as the screen. Or something that amounts to that anyhow. Shrug. I just copy & pasted it once upon a time, and still use it.

That's all I had for this one. Next stop: "Part 5: MariaDB".

Righto.

--
Adam

Friday, 8 January 2021

Part 3: PHPUnit

G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit (this article)
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the third article in the series, and follows on from Part 2: PHP. It's probably best to go have a breeze through the earlier articles first.

PHPUnit

I got slightly ahead of myself and added PHPUnit into composer.json when I was working through the PHP configuration part of this exercise, so it's already installed. Before I run it though, I need a phpunit.xml file, so I'll chuck one of those in:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd"
        colors="true"
        forceCoversAnnotation="true"
        cacheResult="false"
>
    <coverage>
        <include>
            <directory suffix=".php">src</directory>
        </include>
        <report>
            <html outputDirectory="public/test-coverage-report/" />
        </report>
    </coverage>
    <testsuites>
        <testsuite name="Functional tests">
            <directory>test/functional/</directory>
        </testsuite>
        <testsuite name="Unit tests">
            <directory>test/unit/</directory>
        </testsuite>
    </testsuites>
</phpunit>

There's no real surprises here. I've got two separate test suits: one for functional tests in which I'll test those test web-browseable files I created in the previous article; and one for unit tests. To test the code coverage config here I'll need some actual code to test and cover.

I had some drama getting PHPUnit working with code coverage, and that in itself is covered in a separate article, PHPUnit: get code coverage reporting working on PHP 8 / PHPUnit 9.5. The stuff I've written there is very focused on PHPUnit and not so much on the Docker side of things, or the testing in the context of this notional application I'm putting together, hence splitting it out into its own article, and also so I can focus on the Docker side of things here.

First things first, I need some tests! I decided to functional-test the two web browseable files: gdayWorld.html and gdayWorld.php. As a reminder their contents are (respectively):

<!doctype html>

<html lang="en">
<head>
    <meta charset="utf-8">

    <title>G'day world</title>
</head>

<body>
<h1>G'day world</h1>
<p>G'day world</p>
</body>
</html>

And:

<?php

$message = "G'day World";
echo $message;

So for the tests I'm just gonna make sure I can hit them, and the HTML one has the expected title, heading and content; and the PHP one just has the expected string. I could horse around with PHP's native curl implementation to make these work, but its programming interface written like something out of 1995, so I tend to avoid it where I can. I'm gonna use Guzzle instead. Also, and this is slightly OTT I know, but I like using Symfony's Response constants when checking for HTTP status codes to make the code more clear, so I'm adding in symfony/http-foundation. Lastly as I'll be using PHP's DOM API for testing the HTML, I'm placating a warning in PHPStorm that says "ooh but ext-dom might not be installed!" So I'm forcing that too. Oh and I like keeping my code tidy so I'm also slinging PHPMD and PHPCS in there too. My composer.json file becomes:

{
    "name": "adamcameron/full-stack-exercise",
    "description": "Full Stack Exercise",
    "license": "GPL-3.0-or-later",
    "require-dev": {
        "phpunit/phpunit": "^9.5",
        "guzzlehttp/guzzle": "^7",
        "symfony/http-foundation": "^5.2",
        "ext-dom": "*",
        "phpmd/phpmd": "^2.9",
        "squizlabs/php_codesniffer": "^3.5"
    },
    "autoload": {
        "psr-4": {
            "adamCameron\\fullStackExercise\\": "src/"
        }
    },
    "autoload-dev": {
        "adamCameron\\fullStackExercise\\test\\": "test/"
    }
}

Now I can write some tests. I did all this incrementally, but you know how to do that, so here is the "final" (see below for why I put that in quotes) version of WebServerTest:

namespace adamCameron\fullStackExercise\test\functional\_public; // "public" is reserved

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class WebServerTest extends TestCase
{
    /** @coversNothing */
    public function testGdayWorldHtmlReturnsExpectedContent()
    {
        $expectedContent = "G'day world";


        $client = new Client([
            'base_uri' => 'http://localhost/'
        ]);

        $response = $client->get('gdayWorld.html');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $html = $response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);

        $xpathDocument = new \DOMXPath($document);

        $hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasTitle);

        $hasHeading = $xpathDocument->query('/html/body/h1[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasHeading);

        $hasContent = $xpathDocument->query('/html/body/p[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasContent);
    }
}

And I triumphantly run this:

/usr/share/fullstackExercise # vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning:       No code coverage driver available

E                                                                   1 / 1 (100%)

Time: 00:00.438, Memory: 6.00 MB

There was 1 error:

1) adamCameron\fullStackExercise\test\functional\_public\WebServerTest::testGdayWorldHtmlReturnsExpectedContent
GuzzleHttp\Exception\ConnectException: cURL error 7: Failed to connect to localhost port 80: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://localhost/gdayWorld.html

Doh! But… but… but… I quickly went and hit http://localhost/gdayWorld.html in my browser and it was fine. Then the penny dropped. I'm not running the tests from my host machine. I'm running them from with the PHP container. I've told the host machine about the Nginx container's web server; but I've not told the PHP container about it. Reminder as to what the docker-compose.yml is like at the moment:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t

I read a whole bunch of stuff on networking in Docker. I didn't find the Docker docs very useful at the time, but now I re-read them knowing how I'm supposed to interpret them, they seem clear. Not sure if that's an indictment of me or the docs. Or both. I also looked at a whole bunch of Stack Overflow Q&A and the answers were conflicting and divergent. However after distilling what I could from all these sources, it's really pretty easy. Here's the updated version:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend
        
networks:
  backend:
    driver: "bridge"

I just added the networks section, and then told the Nginx and PHP containers to be on that backend network. NB: backend has no significance as a word here, it's just a label, and one used in the docs I was reading. After rebuild, I could now see the Nginx server from the PHP container:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/sh
/usr/share/fullstackExercise # curl http://nginx/gdayWorld.php
G'day World


/usr/share/fullstackExercise #

Note that I'm using the Nginx services container name there as the host name, ie:

services:
  nginx:

Not that it matters, but that seems a bit manky to me, so I wanted to specify a hostname here. That's just a matter of giving the Nginx container a hostname:

services:
  nginx:
    build:
      context: ./nginx
    hostname: webserver.backend
    volumes:
      # etc
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/sh
/usr/share/fullstackExercise # curl http://webserver.backend/gdayWorld.php
G'day World


/usr/share/fullstackExercise #

Now my functional tests should work, provided I use that new hostname:

/usr/share/fullstackExercise # vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning:       No code coverage driver available

..                                                                   1 / 1 (100%)

Time: 00:00.462, Memory: 6.00 MB

OK (2 test, 6 assertions)
/usr/share/fullstackExercise #

Cool!

Oh there was a separate test for gdayWorld.php too:

namespace adamCameron\fullStackExercise\test\functional\_public; // "public" is reserved

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class PhpTest extends TestCase
{
    /** @coversNothing */
    public function testGdayWorldPhpReturnsExpectedContent()
    {
        $client = new Client([
            'base_uri' => 'http://webserver.backend/'
        ]);

        $response = $client->get('gdayWorld.php');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $content = $response->getBody()->getContents();

        $this->assertSame("G'day world", $content);
    }
}

I'm glad I wrote these tests, because this one identified a small bug I had introduced into gdayWorld.php. I'd fixed it by the time I catpured that output above, but the first run was less positive:

/usr/share/fullstackExercise # vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning:       No code coverage driver available

F.                                                                  2 / 2 (100%)

Time: 00:00.509, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\_public\PhpTest::testGdayWorldPhpReturnsExpectedContent
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'G'day world'
+'G'day World'

/usr/share/fullstackExercise/test/functional/PhpTest.php:24

FAILURES!
Tests: 2, Assertions: 6, Failures: 1.
/usr/share/fullstackExercise #

Yay for testing! I did not contrive this situation as an example of "always test first!", but there it is. This is hugely trivial code, but I still messed it up, and simply eyeballing it did not spot the bug.

Speaking of being observant… you will no-doubt have noticed the warning about code coverage driver above. I still need to install XDebug to make this work. This is a matter of adding this into the Dockerfile:

FROM php:fpm-alpine
RUN apk --update --no-cache add git
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

Or at least that was the theory:

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building php-fpm
Step 1/8 : FROM php:fpm-alpine
---> 6bd7d9173974
Step 2/8 : RUN apk --update --no-cache add git
---> Using cache
---> 098d91282e3e
Step 3/8 : RUN docker-php-ext-install pdo_mysql
---> Using cache
---> 6f74a2ec5bb1
Step 4/8 : RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
---> Running in 12444e0a094c
downloading xdebug-3.0.1.tgz ...
Starting to download xdebug-3.0.1.tgz (214,467 bytes)
.............................................done: 214,467 bytes
87 source files, building
running: phpize
Configuring for:
PHP Api Version: 20200930
Zend Module Api No: 20200930
Zend Extension Api No: 420200930
Cannot find autoconf. Please check your autoconf installation and the
$PHP_AUTOCONF environment variable. Then, rerun this script.

ERROR: `phpize' failed
ERROR: Service 'php-fpm' failed to build : The command '/bin/sh -c pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug' returned a non-zero code: 1
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$

Sigh. I have encountered this sort of thing before with other containers. Alpine is really really pared down, and doesn't include a bunch of packages that tools might need to run. This is fine for a lot of situations, but it's also a pain in the arse for others. In this case I added in autoconf, but then I needed to install a C compiler too. And then after that I think it wanted something else. Sod that. I just stopped using Alpine and wend back to the Debian version of the container, in the Dockerfile:

FROM FROM php:8.0-fpm
RUN apt-get update --yes && apt-get install git --yes
ENV XDEBUG_MODE=coverage
NB that changed from:
FROM :fpm-alpine
RUN apk --update --no-cache add git

Note: the apk / apt-get change is just the difference in package manager between Alpine and Debian. I won't show you the installation of this because it's 70kB of telemetry, all of which culminates in:

Creating docker_php-fpm_1 ... done Creating docker_nginx_1 ... done

And now we can run our functional tests, and the code coverage report should create (even if it hasn't got anything in it yet, cos I'm not code-covering those functional tests):

root@2e5f56af2f54:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Warning: Incorrect filter configuration, code coverage will not be processed
..                                                                  2 / 2 (100%)

Time: 00:00.493, Memory: 6.00 MB

OK (2 tests, 6 assertions)
root@2e5f56af2f54:/usr/share/fullstackExercise#

Grrrr… what now?. I reviewed the docs and my phpunit.xml was legit-looking, and validated fine, so I was flumoxed. But then I came across this issue with PHPUnit: Misleading error message when no files present in coverage path. That sums it up. I have this in my phpunit.xml file:

<coverage>
    <include>
        <directory suffix=".php">src</directory>
    </include>
    <report>
        <html outputDirectory="public/test-coverage-report/" />
    </report>
</coverage>

But I don't actually have a src/ directory yet. Once I added that, and also added the public/ directory into that <include> block, I get a report. At the same time, I will add a stub PHP class and test thereof into the src/ directory as well, to better test the reporting:

root@2e5f56af2f54:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

...                                                                 3 / 3 (100%)

Time: 00:02.050, Memory: 12.00 MB

OK (3 tests, 7 assertions)

Generating code coverage report in HTML format ... done [00:00.780]
root@2e5f56af2f54:/usr/share/fullstackExercise#

And the coverage report generates fine too:
above



below

The test for MyClass's needsTesting method is as simple as you might imagine:

namespace adamCameron\fullStackExercise\test\unit;

use adamCameron\fullStackExercise\MyClass;
use PHPUnit\Framework\TestCase;

/** @coversDefaultClass adamCameron\fullStackExercise\MyClass */
class MyClassTest extends TestCase
{
    private $myClass;

    protected function setUp(): void
    {
        $this->myClass = new MyClass();
    }

    /** @covers ::needsTesting */
    public function testNeedsTesting()
    {

        $needsTesting = $this->myClass->needsTesting();
        $this->assertTrue($needsTesting);
    }
}

OK, so I'm in a good place to be able to TDD some PHP code now. But before I do that, I want to get a MariaDB container added into the mix as well, and write an intergration test for its connectivity. Before I get to that though, as I've been doing the work on this project up until now - especially when troubleshooting the networking and the PHPUnit issue I had - I've tweaked some stuff with my Bash environment, and I wanted to rip that out into a separate wee article. So before we get onto MariaDB, I'll write that up: Tweaks I made to my Bash environment in my Docker containers.

Righto.

--
Adam