Showing posts with label Symfony. Show all posts
Showing posts with label Symfony. Show all posts

Saturday, 6 March 2021

Symfony & TDD: adding endpoints to provide data for front-end workshop / registration requirements

G'day:

That's probably a fairly enigmatic title if you have not read the preceding article: "Vue.js: using TDD to develop a data-entry form". It's pretty long-winded, but the gist of it is summarised in this copy and pasted tract:

Right so the overall requirement here is (this is copy and pasted from the previous article) to construct an event registration form (personal details, a selection of workshops to register for), save the details to the DB and echo back a success page. Simple stuff. Less so for me given I'm using tooling I'm still only learning (Vue.js, Symfony, Docker, Kahlan, Mocha, MariaDB).

There's been two articles around this work so far:

There's also a much longer series of articles about me getting the Docker environment running with containers for Nginx, Node.js (and Vue.js), MariaDB and PHP 8 running Symfony 5. It starts with "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers - Part 1: Intro & Nginx" and runs for 12 articles.

In the previous article I did the UI for the workshop registration form…

… and the summary one gets after submitting one's registration:

Today we're creating the back-end endpoint to fetch the list of workshops in that multiple select, and also another endpoint to save the registration details that have been submitted (ran out of time for this bit). The database we'll be talking to is as follows:

(BTW, dbdiagram.io is a bloody handy website... I just did a mysqldump of my tables (no data), imported it their online charting tool, and... done. Cool).

Now… as for me and Symfony… I'm really only starting out with it. The entirety of my hands-on exposure to it is documented in "Part 6: Installing Symfony" and "Part 7: Using Symfony". And the "usage" was very superficial. I'm learning as I go here.

And as-always: I will be TDDing every step, using a tight cycle of identify a case (eg: "it needs to return a 200-OK status for GET requests on the endpoint /workshops"); create tests for that case; do the implementation code just for that case.


It needs to return a 200-OK status for GET requests on the /workshops endpoint

This is a pretty simple test:

namespace adamCameron\fullStackExercise\spec\functional\Controller;

use adamCameron\fullStackExercise\Kernel;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;

describe('Tests of WorkshopController', function () {

    beforeAll(function () {
        $this->request = Request::createFromGlobals();
        $this->kernel  = new Kernel('test', false);
    });

    describe('Tests of doGet', function () {
        it('needs to return a 200-OK status for GET requests', function () {

            $request = $this->request->create("/workshops/", 'GET');
            $response = $this->kernel->handle($request);

            expect($response->getStatusCode())->toBe(Response::HTTP_OK);
        });
    });
});

I make a request, I check the status code of the response. That's it. And obviously it fails:

root@58e3325d1a16:/usr/share/fullstackExercise# vendor/bin/kahlan --spec=spec/functional/Controller/workshopController.spec.php --lcov="var/tmp/lcov/coverage.info" --ff

[error] Uncaught PHP Exception Symfony\Component\HttpKernel\Exception\NotFoundHttpException: "No route found for "GET /workshops/"" at /usr/share/fullstackExercise/vendor/symfony/http-kernel/EventListener/RouterListener.php line 136

F                                                                   1 / 1 (100%)


Tests of WorkshopController
  Tests of doGet
    ✖ it needs to return a 200-OK status for GET requests
      expect->toBe() failed in `.spec/functional/Controller/workshopController.spec.php` line 22

      It expect actual to be identical to expected (===).

      actual:
        (integer) 404
      expected:
        (integer) 200

Perfect. Now let's add a route. And probably wire it up to a controller class and method I guess. Here's what I've got:

# backend/config/routes.yaml
workshops:
  path: /workshops/
  controller: adamCameron\fullStackExercise\Controller\WorkshopsController::doGet
// backend/src/Controller/WorkshopsController.php
namespace adamCameron\fullStackExercise\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;

class WorkshopsController extends AbstractController
{
    public function doGet() : JsonResponse
    {
        return new JsonResponse(null);
    }
}

Initially this continued to fail with a 404, but I worked out that Symfony caches a bunch of stuff when it's not in debug mode, so it wasn't seeing the new route and/or the controller until I switched the Kernel obect initialisation to switch debug on:

// from workshopsController.spec.php, as above:

$this->kernel  = new Kernel('test', true);

And then the actual code ran. Which is nice:

Passed 1 of 1 PASS in 0.108 seconds (using 8MB)

(from now on I'll just let you know if the tests pass, rather than spit out the output).

Bye Kahlan, hi again PHPUnit

I've been away from this article for an entire day, a lot of which was down to trying to get Kahlan to play nicely with Symfony, and also getting Kahlan's own functionality to not obstruct me from forward progress. However I've hit a wall with some of the bugs I've found with it, specifically "Documented way of disabling patching doesn't work #378" and "Bug(?): Double::instance doesn't seem to cater to stubbing methods with return-types #377". These render Kahlan unusable for me.

The good news is I sniffed around PHPUnit a bit more, and discovered its testdox functionality which allows me to write my test cases in good BDD fashion, and have those show up in the test results. It'll either "rehydrate" a human-readable string from the test name (testTheMethodDoesTheThing becomes "test the method does the thing"), or one can specify an actual case via the @testdox annotation on methods and the classes themselves (I'll show you below, outside this box). This means PHPUnit will achieve what I need, so I'm back to using that.

OK, backing up slightly and switching over to the PHPUnit test (backend/tests/functional/Controller/WorkshopsControllerTest.php):

/**
 * @testdox it needs to return a 200-OK status for GET requests
 * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
 */
public function testDoGetReturns200()
{
    $this->client->request('GET', '/workshops/');

    $this->assertEquals(Response::HTTP_OK, $this->client->getResponse()->getStatusCode());
}
> vendor/bin/phpunit --testdox 'tests/functional/Controller/WorkshopsControllerTest.php' '--filter=testDoGetReturns200'
PHPUnit 9.5.2 by Sebastian Bergmann and contributors.

Tests of WorkshopController
it needs to return a 200-OK status for GET requests

Time: 00:00.093, Memory: 12.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in HTML format ... done [00:00.682]
root@5f9133aa9de3:/usr/share/fullstackExercise#

Cool.


It returns a collection of workshop objects, as JSON

The next case is:

/**
 * @testdox it returns a collection of workshop objects, as JSON
 */
public function testDoGetReturnsJson()
{
}

This is a bit trickier, in that I actually need to write some application code now. And wire it into Symfony so it works. And also test it. Via Symfony's wiring. Eek.

Here's a sequence of thoughts:

  • We are getting a collection of Workshop objects from [somewhere] and returning them in JSON format.
  • IE: that would be a WorkshopCollection.
  • The values for the workshops are stored in the DB.
  • The WorkshopCollection will need a way of getting the data into itself. Calling some method like loadAll
  • That will need to be called by the controller, so the controller will need to receive a WorkshopCollection via Symfony's DI implementation.
  • A model class like WorkshopCollection should not be busying itself with the vagaries of storage. It should hand that off to a repository class (see "The Repository Pattern"), which will handle the fetching of DB data and translating it from a recorset to an array of Workshop objects.
  • As WorkshopsRepository will contain testable data-translation logic, it will need unit tests. However we don't want to have to hit the DB in our tests, so we will need to abstract the part of the code that gets the data into something we can mock away.
  • As we're using Doctrine/DBAL to connect to the database, and I'm a believer in "don't mock what you don't own", we will put a thin (-ish) wrapper around that as WorkshopsDAO. This is not completely "thin" because it will "know" the SQL statements to send to its connector to get the data, and will also "know" the DBAL statements to get the data out and pass back to WorkshopsRepository for modelling.

That seems like a chunk to digest, and I don't want you to think I have written any of this code, but this is the sequence of thoughts that leads me to arrive at the strategy for handling the next case. I think from the first bits of that bulleted list I can derive sort of how the test will need to work. The controller doesn't how this WorkshopCollection gets its data, but it needs to be able to tell it to do it. We'll mock that bit out for now, just so we can focus on the controller code. We will work our way back from the mock in another test. For now we have backend/tests/functional/Controller/WorkshopsControllerTest.php

/**
 * @testdox it returns a collection of workshop objects, as JSON
 * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
 */
public function testDoGetReturnsJson()
{
    $workshops = [
        new Workshop(1, 'Workshop 1'),
        new Workshop(2, 'Workshop 2')
    ];

    $this->client->request('GET', '/workshops/');

    $resultJson = $this->client->getResponse()->getContent();
    $result = json_decode($resultJson, false);

    $this->assertCount(count($workshops), $result);
    array_walk($result, function ($workshopValues, $i) use ($workshops) {
        $workshop = new Workshop($workshopValues->id, $workshopValues->name);
        $this->assertEquals($workshops[$i], $workshop);
    });
}

To make this pass we need just enough code for it to work:

class WorkshopsController extends AbstractController
{

    private WorkshopCollection $workshops;

    public function __construct(WorkshopCollection $workshops)
    {
        $this->workshops = $workshops;
    }

    public function doGet() : JsonResponse
    {
        $this->workshops->loadAll();

        return new JsonResponse($this->workshops);
    }
}
class WorkshopCollection implements \JsonSerializable
{
    /** @var Workshop[] */
    private $workshops;

    public function loadAll()
    {
        $this->workshops = [
            new Workshop(1, 'Workshop 1'),
            new Workshop(2, 'Workshop 2')
        ];
    }

    public function jsonSerialize()
    {
        return $this->workshops;
    }
}

And thanks to Symphony's dependency-injection service container's autowiring, all that just works, just like that. That's the test for that end point done.

Now there was all thant bumpf I mentioned about repositories and DAOs and connectors and stuff. As part of the refactoring part of this, we are going to push our implementation right back to the DAO. This allows us to complete the parts of the code in the WorkshopCollection, WorkshopsRepository and just mock-out the DAO for now.

class WorkshopCollection implements \JsonSerializable
{
    private WorkshopsRepository $repository;

    /** @var Workshop[] */
    private $workshops;

    public function setRepository(WorkshopsRepository $repository) : void
    {
        $this->repository = $repository;
    }

    public function loadAll()
    {
        $this->workshops = $this->repository->selectAll();
    }

    public function jsonSerialize()
    {
        return $this->workshops;
    }
}

My thinking here is:

  • It's going to need a WorkshopsRepository to get stuff from the DB.
  • It doesn't seem right to me to pass in a dependency to a model as a constructor arguments. The model should work without needing a DB connection; just the methods around storage interaction should require the repository. On the other hand the only thing the collection does now is to be able to load the stuff from the DB and serialise it, so I'm kinda coding for the future here, and I don't like that. But I'm sticking with it for reasons we'll come to below.
  • I also really hate model classes with getters and setters. This is usually a sign of bad OOP. But here I have a setter, to get the repository in there.

The reason (it's not a reason, it's an excuse) I'm not passing in the repo as a constructor argument and instead using a setter is because I wanted to check out how Symfony's service config dealt with the configuration of this. If yer classes all have type-checked constructor args, Symfony just does it all automatically with no code at all (just a config switch). However to handle using the setRepository method I needed a factory method to do so. The config for it is thus (in backend/config/services.yaml):

adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory: ~
adamCameron\fullStackExercise\Model\WorkshopCollection:
    factory: ['@adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory', 'getWorkshopCollection']

Simple! And the code for WorkshopCollectionFactory:

class WorkshopCollectionFactory
{
    private WorkshopsRepository $repository;

    public function __construct(WorkshopsRepository $repository)
    {
        $this->repository = $repository;
    }

    public function getWorkshopCollection() : WorkshopCollection
    {
        $collection = new WorkshopCollection();
        $collection->setRepository($this->repository);

        return $collection;
    }
}

Also very simple. But, yeah, it's an exercise in messing about, and there's no way I should have done this. I should have just used a constructor argument. Anyway, moving on.

The WorkshopsRepository is very simple too:

class WorkshopsRepository
{
    private WorkshopsDAO $dao;

    public function __construct(WorkshopsDAO $dao)
    {
        $this->dao = $dao;
    }

    /** @return Workshop[] */
    public function selectAll() : array
    {
        $records = $this->dao->selectAll();
        return array_map(
            function ($record) {
                return new Workshop($record['id'], $record['name']);
            },
            $records
        );
    }
}

I get some records from the DAO, and map them across to Workshop objects. Oh! Workshop:

class Workshop implements \JsonSerializable
{
    private int $id;
    private string $name;

    public function __construct(int $id, string $name)
    {
        $this->id = $id;
        $this->name = $name;
    }

    public function jsonSerialize()
    {
        return (object) [
            'id' => $this->id,
            'name' => $this->name
        ];
    }
}

And lastly I mock WorkshopsDAO. I can't implement any further down the stack of this process because the DAO is what uses the DBAL Connector object, and I don't own that, so if I actually started to use it, I'd be hitting the DB. Or hitting the ether and getting an error. Either way: no good for our test. So a mocked DAO:

class WorkshopsDAO
{
    public function selectAll() : array
    {
        return [
            ['id' => 1, 'name' => 'Workshop 1'],
            ['id' => 2, 'name' => 'Workshop 2']
        ];
    }
}

Having done all that refactoring, we check if our test is still good, and it is. I can verify this is not a trick of the light by changing some of that data in the DAO, and watch the test break (which it does). I can also now go back to the test and stick some more code-coverage annotations in:

/**
 * @testdox it returns a collection of workshop objects, as JSON
 * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
 * @covers \adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory
 * @covers \adamCameron\fullStackExercise\Repository\WorkshopsRepository
 * @covers \adamCameron\fullStackExercise\Model\WorkshopCollection
 * @covers \adamCameron\fullStackExercise\Model\Workshop
 */

And see that all the code is indeed covered:


It returns the expected workshops from the database

But now we need to implement the real DAO. once we do that, our test will break because the DAO will suddenly start hitting the DB, and we'll be getting back whatever is in the DB, not our expected canned response. Plus we don't want this test to hit the DB anyhow. So first we're gonna mock-out the DAO using PHPUnit's mocks instead of our code-mock. To do this turned out to be a bit tricky, initially, given Symfony's DI container is looking after all the dependencies for us, but fortunately when in test mode, Symfony allows us to hack into that container. I've updated my test, thus:

public function testDoGetReturnsJson()
{
    $workshopDbValues = [
        ['id' => 1, 'name' => 'Workshop 1'],
        ['id' => 2, 'name' => 'Workshop 2']
    ];

    $this->mockWorkshopDaoInServiceContainer($workshopDbValues);

    // ... unchanged ...

    array_walk($result, function ($workshopValues, $i) use ($workshopDbValues) {
        $this->assertEquals($workshopDbValues[$i], $workshopValues);
    });
}

private function mockWorkshopDaoInServiceContainer($returnValue = []): void
{
    $mockedDao = $this->createMock(WorkshopsDAO::class);
    $mockedDao->method('selectAll')->willReturn($returnValue);

    $container = $this->client->getContainer();
    $workshopRepository = $container->get('test.WorkshopsRepository');

    $reflection = new \ReflectionClass($workshopRepository);
    $property = $reflection->getProperty('dao');
    $property->setAccessible(true);
    $property->setValue($workshopRepository, $mockedDao);
}

We are popping a mocked WorkshopsDAO into the WorkshopsRepository object in the container So when the repo calls it, it'll be calling the mock.

Oh! to be able to access that 'test.WorkshopsRepository' container key, we need to expose it via the services_test.xml container config:

services:
  test.WorkshopsRepository:
    alias: adamCameron\fullStackExercise\Repository\WorkshopsRepository
    public: true

And running that, the test works, and is ignoring the reallyreally DAO.

To test the final DAO implementation, we're gonna do an end-to-end test:

class WorkshopControllerTest extends WebTestCase
{
    private KernelBrowser $client;

    public static function setUpBeforeClass(): void
    {
        $dotenv = new Dotenv();
        $dotenv->load(dirname(__DIR__, 3) . "/.env.test");
    }

    protected function setUp(): void
    {
        $this->client = static::createClient(['debug' => false]);
    }

    /**
     * @testdox it returns the expected workshops from the database
     * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
     * @covers \adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory
     * @covers \adamCameron\fullStackExercise\Model\WorkshopCollection
     * @covers \adamCameron\fullStackExercise\Repository\WorkshopsRepository
     * @covers \adamCameron\fullStackExercise\Model\Workshop
     */
    public function testDoGet()
    {
        $this->client->request('GET', '/workshops/');
        $response = $this->client->getResponse();
        $workshops = json_decode($response->getContent(), false);

        /** @var Connection */
        $connection = static::$container->get('database_connection');
        $expectedRecords = $connection->query("SELECT id, name FROM workshops ORDER BY id ASC")->fetchAll();

        $this->assertCount(count($expectedRecords), $workshops);
        array_walk($expectedRecords, function ($record, $i) use ($workshops) {
            $this->assertEquals($record['id'], $workshops[$i]->id);
            $this->assertSame($record['name'], $workshops[$i]->name);
        });
    }
}

The test is pretty familiar, except it's actually getting its expected data from the database, and making sure the whole process, end to end, is doing what we want. Currently when we run this it fails because we still have our mocked DAO in place (the real mock, not the… mocked mock. Um. You know what I mean: the actual DAO class that just returns hard-coded data). Now we put the proper DAO code in:

class WorkshopsDAO
{
    private Connection $connection;

    public function __construct(Connection $connection)
    {
        $this->connection = $connection;
    }

    public function selectAll() : array
    {
        $sql = "
            SELECT
                id, name
            FROM
                workshops
            ORDER BY
                id ASC
        ";
        $statement = $this->connection->executeQuery($sql);

        return $statement->fetchAllAssociative();
    }
}

And now if we run the tests:

> vendor/bin/phpunit --testdox
PHPUnit 9.5.2 by Sebastian Bergmann and contributors.

Tests of WorkshopController
it needs to return a 200-OK status for GET requests
it returns a collection of workshop objects, as JSON

Tests of baseline Symfony install
it displays the Symfony welcome screen
it returns a personalised greeting from the /greetings end point

PHP config tests
gdayWorld.php outputs G'day world!

Webserver config tests
It serves gdayWorld.html with expected content

End to end tests of WorkshopController
it returns the expected workshops from the database

Tests that code coverage analysis is operational
It reports code coverage of a simple method correctly

Time: 00:00.553, Memory: 22.00 MB

OK (8 tests, 26 assertions)

Generating code coverage report in HTML format ... done [00:00.551]
root@5f9133aa9de3:/usr/share/fullstackExercise#

Nice one!

And if we look at code coverage:

I was gonna try to cover the requirements for the process ot save the form fields in the article too, but it took ages to work out how some of the Symfony stuff worked, plus I sunk about a day into trying to get Kahlan to work, and this article is already super long anyhow. I now have the back-end processing sorted out to update the front-end form to actually use the values from the DB instead of test values. I might look at that tomorrow (see "TDDing the reading of data from a web service to populate elements of a Vue JS component" for that exercise). I need a rest from Symfony.


It needs to drink some beer

I'm rushing the outro of this article because I am supposed to be on a webcam with a beer in my hand in 33min, and I need to proofread this still…

Righto.

--
Adam

Thursday, 4 March 2021

Kahlan: getting it working with Symfony 5 and generating a code coverage report

G'day:

This is not the article I intended to write today. Today (well: I hoped to have it done by yesterday, actually) I had hoped to be writing about my happy times doing using TDD to implement a coupla end-points I need in my Symfony-driven web service. I got 404 (chuckle) words into that and then was blocked by trying to get Kahlan to play nice for about five hours (I did have dinner in that time too, but it was add my desk, and with a scowl on my face googling stuff). And that put me at 1am so I decided to go to bed. I resumed today an hour or so ago, and am just now in the position to get going again. But I've decided to document that lost six hours first.

I sat down to create a simple endpoint to fetch some data, and started by deciding on my first test case, which was "It needs to return a 200-OK status for GET requests on the /workshops endpoint". I knew Symfony did some odd shenanigans to be able to functionally test right from a route slug rather than having tests directly instantiating controller classes and such. I checked the docs and all this is done via an extension of PHPUnit, using WebTestCase. But I don't wanna use PHPUnit for this. Can you imagine my test case name? Something like: testItNeedsToReturnA200OKStatusForGetRequestsOnTheWorkshopsEndpoint. Or I could break PSR-12/PSR-1 and make it (worse) with test_it_needs_to_Return_a_200_OK_status_for_get_requests_on_the_workshops_endpoint (this is why I will never use phpspec). Screw that. I'm gonna work out how to do these Symfony WebTestCase tests in Kahlan.

Not so fast mocking PHPUNit there, Cameron

2021-03-06

Due to some - probably show-stopping - issues I'm seeing with Kahlan, I have been looking at PHPUnit some more. I just discovered the textdox reporting functionality it has, which makes giving test case names much clearer.

/** @testdox Tests the /workshops endpoint methods */
class workshopsEndPointTet {
    /** @testdox it needs to return a 200-OK status for GET requests */
    function testReturnStatus() {}
}

This will output in the test runs as:

Perfect. I still prefer the style of code Kahlan uses, but… this is really good to know.

First things first, I rely heavily on PHPUnit's code coverage analysis, so I wanted to check out Kahan's offering. The docs seem pretty clear ("Code Coverage"), and seems I just want to be using the lcov integration Kahlan offers, like this:

vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info"
genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info

I need to install lcov first via my Dockerfile:

RUN apt-get install lcov --yes

OK so I did all that, and had a look at the report:

Pretty terrible coverage, but it's working. Excellent. But drilling down into the report I see this:

>

This is legit reporting because I have no tests for the Kernel class, but equally that class is generated by Symfony and I don't want to cover that. How do I exclude it from code coverage analysis? I'm looking for Kahlan's equivalent of PHPUnit's @codeCoverageIgnore. There's nothing in the docs, and all I found was a passing comment against an issue in Github asking the same question I was: "Exclude a folder in code coverage #321". The answer is to do this sort of thing to my kahlan-config.php file:

use Kahlan\Filter\Filters;
use Kahlan\Reporter\Coverage;
use Kahlan\Reporter\Coverage\Driver\Xdebug;

$commandLine = $this->commandLine();
$commandLine->option('no-header', 'default', 1);

Filters::apply($this, 'coverage', function($next) {
    if (!extension_loaded('xdebug')) {
        return;
    }
    $reporters = $this->reporters();
    $coverage = new Coverage([
        'verbosity' => $this->commandLine()->get('coverage'),
        'driver'    => new Xdebug(),
        'path'      => $this->commandLine()->get('src'),
        'exclude'   => [
            'src/Kernel.php'
        ],
        'colors'    => !$this->commandLine()->get('no-colors')
    ]);
    $reporters->add('coverage', $coverage);
});

That seems a lot of messing around to do something that seems like it should be very simple to me. I will also note that Kahlan - currently - has no ability to suppress code coverage at a method or code-block level either (see "Skip individual functions in code coverage? #333"). This is not a deal breaker for me in this work, but it would be a show-stopper on any of the codebases I have worked on in the last decade, as they've all been of dubious quality, and all needed some stuff to be actively "overlooked" as they're not testable as they currently stand, and we (/) like my baseline code coverage report to have 100% coverage reported, and be entirely green. This is so if any omissions creep in, they're easy to spot (see "Yeah, you do want 100% test coverage"). Anyway, I'll make that change and omit Kernel from analysis:

root@13038aa90234:/usr/share/fullstackExercise# vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info"

.................                                                 17 / 17 (100%)



Expectations   : 51 Executed
Specifications : 0 Pending, 0 Excluded, 0 Skipped

Passed 17 of 17 PASS in 0.527 seconds (using 7MB)

Coverage Summary
----------------

Total: 33.33% (1/3)

Coverage collected in 0.001 seconds


root@13038aa90234:/usr/share/fullstackExercise# genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info
Reading data file var/tmp/lcov/coverage.info
Found 2 entries.
Found common filename prefix "/usr/share/fullstackExercise"
Writing .css and .png files.
Generating output.
Processing file src/MyClass.php
Processing file src/Controller/GreetingsController.php
Writing directory view page.
Overall coverage rate:
  lines......: 33.3% (1 of 3 lines)
  functions..: 50.0% (1 of 2 functions)
root@13038aa90234:/usr/share/fullstackExercise#

And the report now doesn't mention Kernel:

Cool.

Now to implement that test case. I need to work out how to run a Symfony request without using WebTestCase. Well I say "I need to…" I mean I need to google someone else who's already done it, and copy them. I have NFI how to do it, and I'm not prepared to dive into Symfony code to find out how. Fortunately someone has already cracked this one: "Functional Test Symfony 4 with Kahlan 4". It says "Symfony 4", but I'll check if it works on Symfony 5 too. I also happened back to the Kahlan docs, and they mention the same guy's solution ("Integration with popular frameworks › Symfony"). This one points to a library to encapsulate it (elephantly/kahlan-bundle), but that is actively version-constrained to only Symfony 4. Plus it's not seen any work since 2017, so I suspect it's abandoned.

Anyway, back to samsonasik's blog article. It looks like this is the key bit:

$this->request = Request::createFromGlobals();
$this->kernel  = new Kernel('test', false);
$request = $this->request->create('/lucky/number', 'GET');
$response = $this->kernel->handle($request);

That's how to create a Request and get Symfony's Kernel to run it. Easy. Hopefully. Let's try it.

namespace adamCameron\fullStackExercise\spec\functional\Controller;

use adamCameron\fullStackExercise\Kernel;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;

describe('Tests of GreetingsController', function () {

    beforeAll(function () {
        $this->request = Request::createFromGlobals();
        $this->kernel  = new Kernel('test', false);
    });

    describe('Tests of doGet', function () {
        it('returns a JSON greeting object', function () {
            $testName = 'Zachary';

            $request = $this->request->create("/greetings/$testName", 'GET');
            $response = $this->kernel->handle($request);

            expect($response->getStatusCode())->toBe(Response::HTTP_OK);
        });
    });
});

I'm not getting too ambitious here, and it's not addressing the entire test case yet. I'm just making the request and checking its response status code.

And this just goes splat:

root@13038aa90234:/usr/share/fullstackExercise# vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --ff

E                                                                 18 / 18 (100%)


Tests of GreetingsController
  Tests of doGet
    ✖ it returns a JSON greeting object
      an uncaught exception has been thrown in `vendor/symfony/framework-bundle/Kernel/MicroKernelTrait.php` line 91

      message:`Kahlan\PhpErrorException` Code(0) with message "`E_WARNING` require(/tmp/kahlan/usr/share/fullstackExercise/src/config/bundles.php): Failed to open stream: No such file or directory"

        [NA] - vendor/symfony/framework-bundle/Kernel/MicroKernelTrait.php, line  to 91

Eek. I had a look into this, and the code in question is try to do this:

$contents = require $this->getProjectDir().'/config/bundles.php';

And the code in getProjectDir is thus:

<pre class="source-code"><code>public function getProjectDir()
{
    if (null === $this-&gt;projectDir) {
        $r = new \ReflectionObject($this);

        if (!is_file($dir = $r-&gt;getFileName())) {
            throw new \LogicException(sprintf('Cannot auto-detect project dir for kernel of class &quot;%s&quot;.', $r-&gt;name));
        }

        $dir = $rootDir = \dirname($dir);
        while (!is_file($dir.'/composer.json')) {
            if ($dir === \dirname($dir)) {
                return $this-&gt;projectDir = $rootDir;
            }
            $dir = \dirname($dir);
        }
        $this-&gt;projectDir = $dir;
    }

    return $this-&gt;projectDir;
}
</code></pre>

The code starts in the directory of the current file, and traverses up the directory structure until it finds the directory with composer.json.If it doesn't find that, then - somewhat enigmatically, IMO - it just says "ah now, we'll just use the directory we're in now. It'll be grand". To me if it expects to find what it's looking for by looking up the ancestor directory path and that doesn't work: throw an exception. Still. In the normal scheme of things, this would work cos the app's Kernel class - by default - seems to live in the src directory, which is one down from where composer.json is.

So why didn't it work? Look at the directory that it's trying to load the bundles from: /tmp/kahlan/usr/share/fullstackExercise/src/config/bundles.php. Where? /tmp/kahlan/usr/share/fullstackExercise/. Ain't no app code in there, pal. It's in /usr/share/fullstackExercise/. Why's it looking in there? Because the Kernel object that is initiating all this is at /tmp/kahlan/usr/share/fullstackExercise/src/Kernel.php. It's not the app's own one (at /usr/share/fullstackExercise/src/Kernel.php), it's all down to how Kahlan monkey-patches everything that's going to be called by the test code, on the off chance you want to spy on anything. It achieves this by loading the source code of the classes, patching the hell out of it, and saving it in that /tmp/kahlan. The only problem with this is that when Symfony traverses up from where the patched Kernel class is… it never finds composer.json, so it just takes a guess at where the project directory is. And it's not a well-informed guess.

I'm not sure who I blame more here, to be honest. Kahlan for patching everything and running code from a different directory from where it's supposed to be; or Symfony for its "interesting" way to know where the project directory is. I have an idea here, Symfony: you could just ask me. Or even force me tell it. Ah well. Just trying to be "helpful" I s'pose.

Anyway, I can exclude files from being patched, according to the docs:

  --exclude=<string>                  Paths to exclude from patching. (default: `[]`).

I tried sticking the path to Kernel in there: src/Kernel.php, and that didn't work. I hacked about in the code and it doesn't actually want a path, it wants the fully-qualified class name, eg: adamCameron\fullStackExercise\Kernel. I've raised a ticket for this with Kahlan, just to clarify the wording there: Bug: bad nomenclature in help: "path" != "namespace".

This does work…

root@13038aa90234:/usr/share/fullstackExercise# vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --ff --exclude=adamCameron\\fullStackExercise\\Kernel

E                                                                 18 / 18 (100%)


Tests of GreetingsController
  Tests of doGet
    ✖ it returns a JSON greeting object
      an uncaught exception has been thrown in `vendor/symfony/deprecation-contracts/function.php` line 25

      message:`Kahlan\PhpErrorException` Code(0) with message "`E_USER_DEPRECATED` Please install the \"intl\" PHP extension for best performance."

        [NA] - vendor/symfony/deprecation-contracts/function.php, line  to 25
        trigger_deprecation - vendor/symfony/framework-bundle/DependencyInjection/FrameworkExtension.php, line  to 253

This is not exactly what I want, but it's a different error, so Symfony is finding itself this time, and then just faceplanting again. However when I look into the code, it's this:

if (!\extension_loaded('intl') && !\defined('PHPUNIT_COMPOSER_INSTALL')) {
    trigger_deprecation('', '', 'Please install the "intl" PHP extension for best performance.');
}

// which in turn...

function trigger_deprecation(string $package, string $version, string $message, ...$args): void
{
    @trigger_error(($package || $version ? "Since $package $version: " : '').($args ? vsprintf($message, $args) : $message), \E_USER_DEPRECATED);
}

So Symfony is very quietly raising a flag that it suggests I have that extension installed. But only as a deprecation notice, and even then it's @-ed out. Somehow Kahlan is getting hold of that and going "nonono, this is worth stopping for". No it ain't. Ticket raised: "Q: should trigger_error type E_USER_DEPRECATED cause testing to halt?".

Anyway, the point is a legit one, so I'll install the intl extension. I initially thought it was just a matter of slinging this in the Dockerfile:

RUN apt-get install --yes zlib1g-dev libicu-dev g++
RUN docker-php-ext-install intl

But that didn't work, I needed a bunch of Other Stuff too:

RUN apt-get install --yes zlib1g-dev libicu-dev g++
RUN docker-php-ext-install pdo_mysql
RUN docker-php-ext-configure intl
RUN docker-php-ext-install intl

(Thanks to the note in docker-php-ext-install intl fails #57 for solving that for me).

After rebuilding the container, let's see what goes wrong next:

root@58e3325d1a16:/usr/share/fullstackExercise# composer coverage
> vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --exclude=adamCameron\\fullStackExercise\\Kernel

..................                                                18 / 18 (100%)



Expectations   : 52 Executed
Specifications : 0 Pending, 0 Excluded, 0 Skipped

Passed 18 of 18 PASS in 0.605 seconds (using 12MB)

Coverage Summary
----------------

Total: 100.00% (3/3)

Coverage collected in 0.001 seconds


> genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info
Reading data file var/tmp/lcov/coverage.info
Found 2 entries.
Found common filename prefix "/usr/share/fullstackExercise"
Writing .css and .png files.
Generating output.
Processing file src/MyClass.php
Processing file src/Controller/GreetingsController.php
Writing directory view page.
Overall coverage rate:
  lines......: 100.0% (3 of 3 lines)
  functions..: 100.0% (2 of 2 functions)
root@58e3325d1a16:/usr/share/fullstackExercise#

(I've stuck a Composer script in for this, btw):

"coverage": [
    "vendor/bin/kahlan --lcov=\"var/tmp/lcov/coverage.info\" --exclude=adamCameron\\\\fullStackExercise\\\\Kernel",
    "genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info"
]

And most promising of all is this:

All green! I like that.

And now I'm back to where I wanted to be, yesterday, as I typed that 404th word of the article I was meant to be working on. 24h later now.

Righto.

--
Adam

Thursday, 14 January 2021

Part 7: Using Symfony

G'day:

Familiar boilterplate about how this is one in a series follows. Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony
  7. Using Symfony (this article)
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the seventh article in the series, and follows on from Part 6: Installing Symfony. It's probably best to go have a breeze through the earlier articles first, in particular the immediately preceding one. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject. Also I was only originally "planning" one article on getting the Symfony stuff sorted out, but yesterday's exercise was more involved that I'd've liked, so I stopped once I had Symfony serving up its welcome page, and today focusing on the config / code of getting a route, controller, model, view (actually there's no model or view in this endpoint; it's all just done in the controller) up and running.

Two caveats before I start.

Firstly: this is the first time I've done anything with Symfony other than reviewing the work of other members of my team - the ones doing the actual work. I have some familiarity with Symfony, but it's very very superficial.

Secondly, I will also be up front that I don't like Symfony's approach to MVC frameworking. Symfony is kind of a "lifestyle choice": it's opinionated, it expects you to do things a certain way, and it puts its nose into everything. This is opposed to something like Silex which simply provides the bare bones wiring to handle the ubiquitous web application requirement of guiding processing through routing, controller, model and view; other than that it just gets out of the way and is pretty invisible. I loved Silex, but sadly it's EOL so I need to move on. And, yeah, I'm being curmudgeonly and pre-judgemental as I go, I know. I do know Symfony is immensely popular, and I also know from my exposure to using its various independent libraries that it's been developed in a thoughtful, thorough manner. I expect I'm largely just being change-averse here. However I'm forewarning you now, as this will no-doubt come across in my tone, and my patience when things don't go exactly how I'd like them too (like some sort of spoilt brat). But if you've read this blog before… you probably already expect this.

Now that I have that out of my system (maybe), what's my aim for today? When I started yesterday, the intent of my work was described in this functional test:

/** @coversNothing */
public function testGreetingsEndPointReturnsPersonalisedGreeting()
{
    $testName = 'Zachary';
     $expectedGreeting = (object) [
        'name' => $testName,
        'greeting' => "G'day $testName"
    ];

    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get(
        "greetings/$testName/",
        ['http_errors' => false]
    );
    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

    $contentTypes = $response->getHeader('Content-Type');
    $this->assertCount(1, $contentTypes);
    $this->assertSame('application/json', $contentTypes[0]);

    $responseBody = $response->getBody();
    $responseObject = json_decode($responseBody);
    $this->assertEquals($expectedGreeting, $responseObject);
}

In humanspeke, what I'm gonna do is:

  • create a route /greetings/[some name here]/;
  • return an object as JSON;
  • that confirms the name sent, plus has a greeting string for that name.

Very "Hello World". I did indicate I was not gonna be pushing the boat out too much here.

To start let's run that test, and watch it fail with a 404…

root@5565d6f15aca:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.F....                                                              6 / 6 (100%)

Time: 00:04.414, Memory: 14.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\tests\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
Failed asserting that 404 matches expected 200.

/usr/share/fullstackExercise/tests/functional/SymfonyTest.php:63

FAILURES!
Tests: 6, Assertions: 12, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.162]
root@5565d6f15aca:/usr/share/fullstackExercise#

That one failure is the one we want to see, so that's good: I have a test failing in exactly they way I expect it to be, so now I need to work out how to add a route and all that sort of jazz. Time to RTFM. Back soon.

Right so the docs for Symfony have been incredibly helpful so far. That's not sarcasm: they've been really bloody good! I'd been looking at the Installing & Setting up the Symfony Framework page, and one of its last links is to Create your First Page in Symfony. This steps one through setting up a route and the controller that services it. The first example did not have a "runtime" parameter in the URL slug like I need here, but that was covered in another linked-to page Routing (specifically the Route Parameters section of same). That was all the information I seemed to need for my code, so off I went.

Routing is done in config/routes.yaml, and there's an example in there already. So it was easy to stick my new route in:

#index:
#    path: /
#    controller: App\Controller\DefaultController::index

greetings:
  path: /greetings/{name}
  controller: adamCameron\fullStackExercise\Controller\GreetingsController::doGet

Curly braces around the {name} just mean that that part of the URL slug is dynamic, and its value is passed to the controller method. I could (and should!) put validation on this, but that's getting ahead of myself. The current requirement I have set only handles the happy path, so we'll stick to that.

The controller goes in the src/Controller directory. I'm more accustomed to using headlessCamelCase for my namespace parts, but I'll stick with Symfony's precedent here:

namespace adamCameron\fullStackExercise\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;

class GreetingsController extends AbstractController
{
    public function doGet(string $name) : Response
    {
        $greetingsResponse = [
            'name' => $name,
            'greeting' => "G'day $name"
        ];

        return new JsonResponse($greetingsResponse);
    }
}

This shows how the dynamic part of the slug from the route passes through into the controller method. Cool. From there I've gone my own way from the docs there cos Silex uses Symfony's request/response mechanism, so I know I can just return a JsonResponse like that, and it'll handle the 200-OK and the application/json part of the requirement. The docs integrate using Twig here to render some output, but there's no need for that complexity here. I suppose here the creation of $greetingsResponse is my "model", and the decision to just return a JsonResponse is my "view".

Being quite pleased with myself at how simple that was, I ran my test to verify how clever I was:

root@3fc72bf44d38:/usr/share/fullstackExercise# vendor/bin/phpunit tests/functional/SymfonyTest.php --filter=testGreetingsEndPointReturnsPersonalisedGreeting
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 00:02.607, Memory: 8.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\tests\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
Failed asserting that 500 matches expected 200.

/usr/share/fullstackExercise/tests/functional/SymfonyTest.php:63

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.790]
root@3fc72bf44d38:/usr/share/fullstackExercise#

rrarr! OK, not so clever after all it seems. Hitting the URL in the browser gives me more information:


What is this thing on about? See where it's looking for my controller? It's looking for a class App\Controller\GreetingsController, but that class is not in App/Controller, it's in adamCameron/fullStackExercise/Controller, and that's where the route code says it is. I have re-checked everything, and it's all legit.

Sigh. I could guess what it is. Symfony being a) "clever", and b) "opinionated". If you've just read the previous article, you might recall me raising my eyebrow at this bit in composer.json:

"autoload": {
    "psr-4": {
        "App\\": "src/",
        "adamCameron\\fullStackExercise\\": "src/"
    }
},

At the time I was just observing what a dumb namespace that was, but I'm now guessing that in Symfony's opinion that is the namespace we all should be using. I remembered something about some other autowiring config that Symfony has, and guessed there was something in there that might cater to classes in the App namespace, but not other namespaces. Even if the namespaces are explicit in the code (as they are here). I located the culprit in config/services.yaml:

services:
    # default configuration for services in *this* file
    _defaults:
        autowire: true      # Automatically injects dependencies in your services.
        autoconfigure: true # Automatically registers your services as commands, event subscribers, etc.

    # makes classes in src/ available to be used as services
    # this creates a service per class whose id is the fully-qualified class name
    App\:
        resource: '../src/'
        exclude:
            - '../src/DependencyInjection/'
            - '../src/Entity/'
            - '../src/Kernel.php'
            - '../src/Tests/'

    # controllers are imported separately to make sure services can be injected
    # as action arguments even if you don't extend any base controller class
    App\Controller\:
        resource: '../src/Controller/'
        tags: ['controller.service_arguments']

    # add more service definitions when explicit configuration is needed
    # please note that last definitions always *replace* previous ones

So there's magic taking place for App; I guess I need to make it magical for my actual namespace instead. I'm never gonna write code with App as its namespace, so I'm just gonna punt that I can change that to adamCameron\fullStackExercise\: and adamCameron\fullStackExercise\Controller\: and it'll all be fine. I think ATM I only need to monkey with the Controller one, but I might as well do both now I guess. So with that change in place, I re-run the test:

root@3fc72bf44d38:/usr/share/fullstackExercise# vendor/bin/phpunit tests/functional/SymfonyTest.php --filter=testGreetingsEndPointReturnsPersonalisedGreeting
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:06.440, Memory: 8.00 MB

OK (1 test, 4 assertions)

Generating code coverage report in HTML format ... done [00:01.825]
root@3fc72bf44d38:/usr/share/fullstackExercise#

All right, that was easily located / solved and is not the end of the world. However this from Symfony's error message irks me: "The file was found but the class was not in it, the class name or namespace probably has a typo". No mate. That's not the problem at all. The problem is that despite me giving you the exact fully-qualified namespaced class name in the route config, and the namespace in the class file was correct, and in the right place in the file system… and if you just left things be then you would have found the correct class. But: no. You had to try to be clever, and you failed miserably. I really frickin' hate it when technology goes "nono [patronising grimace] I've got this, I'll do it for you" (I know some devs like this too…), and it gets it wrong. I'm all for Symfony having the helpers there should one want them, but it shouldn't make these assumptions. I guess had it gone "hey aaah… that namespace yer using for yer controller? Ya need to configure that to work mate. Go have a look in config.yaml", then I'd just be going "yeah nice one: thanks for that". But then again most people would probably not have written this ranty paragraph at all, and just moved on with their lives, eh? Hey at least I'm self-aware.

All in all, even with the config/namespace hiccup, getting a simple route working was pretty bloody easy here. And this is a good thing about Symfony.

That was actually pretty short. It was about 500 words and some test-runs longer, as I had encountered a weird issue which I was moaning about. However when doing a final read-through of this before pressing "send", I looked at it some more, and it turns out it was just me being a div, and the problem was firmly of the PEBCAK variety. So… erm… I decided to delete that bit. Ahem.

Next I have to start doing some front-end-ish stuff so I've got an excuse to try out Vue.js. This should be interesting. I've never even looked @ Vue.js before. I bet I'll disagree with some of its opinions eh? Stay tuned…

Righto.

--
Adam

Wednesday, 13 January 2021

Part 6: Installing Symfony

G'day:

These initial boilerplate paragraphs must be getting annoying by now. Sorry. But anyhow, here we go again. Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

  1. Intro / Nginx
  2. PHP
  3. PHPUnit
  4. Tweaks I made to my Bash environment in my Docker containers
  5. MariaDB
  6. Installing Symfony (this article)
  7. Using Symfony
  8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
  9. I mess up how I configure my Docker containers
  10. An article about moving files and changing configuration
  11. Setting up a Vue.js project and integrating some existing code into it
  12. Unit testing Vue.js components

As indicated: this is the sixth article in the series, and follows on from Part 5: MariaDB. It's probably best to go have a breeze through the earlier articles first. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject.

For the earlier articles I had mostly tried-out what I was intending to do before I started writing about it. This time I am writing this at the same time as doing the work. To contextualise, all I have done at the moment is open up that article of Martin Pham's I've been following along ("Symfony 5 development with Docker"), and also got the Symfony docs up: "Installing & Setting up the Symfony Framework / Creating Symfony Applications".

First things first I want a failing test for my minimum requirements here, which is that when I GET /greeting/[some name here]/, I receive back a JSON response, along these lines: {"name":"[some name here]","greeting":"G'day [some name here]"}. Here's a test that tests this (tests/functional/SymfonyTest.php):

namespace adamCameron\fullStackExercise\test\functional;

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class SymfonyTest extends TestCase
{
    /** @coversNothing */
    public function testGreetingsEndPointReturnsPersonalisedGreeting()
    {
        $testName = 'Zachary';
        $expectedGreeting = (object) [
            'name' => $testName,
            'greeting' => "G'day $testName"
        ];

        $client = new Client([
            'base_uri' => 'http://webserver.backend/'
        ]);

        $response = $client->get("greeting/$testName/");
        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $contentTypes = $response->getHeader('Content-Type');
        $this->assertCount(1, $contentTypes);
        $this->assertSame('application/json', $contentTypes[0]);

        $responseBody = $response->getBody();
        $responseObject = json_decode($responseBody);
        $this->assertEquals($expectedGreeting, $responseObject);
    }
}

The code is pretty clear, I just perform the GET using "Zachary" as the name, and in return I verify I get a 200 OK response, it's JSON and it has the expected data structure in the body. Simple.

I run the test to check I get a fail with a 404:

root@20ad01609407:/usr/share/fullstackExercise# vendor/bin/phpunit test/functional/SymfonyTest.php
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 00:00.670, Memory: 6.00 MB

There was 1 error:

1) adamCameron\fullStackExercise\test\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
GuzzleHttp\Exception\ClientException: Client error: 'GET http://webserver.backend/greetings/Zachary/' resulted in a '404 Not Found' response:
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1. (truncated...)

OK what? Why the hell is Guzzle deciding a 404 response - which is completely legitimate - is in some way "exceptional", and warrants an exception being thrown. That's daft. Lemme go RTFM and see if there's a way of me configuring Guzzle to not get ahead of itself and just behave. back in a bit…

… OK, easily solved. There's a config param for the get call that switches that behaviour off. Thanks to the Guzzle docs for making that easy to find ("Request options / http-errors").

$response = $client->get(
    "greetings/$testName/",
    ['http_errors' => false]
);

And now we get a test that fails as we'd expect it to:

root@20ad01609407:/usr/share/fullstackExercise# vendor/bin/phpunit test/functional/SymfonyTest.php
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 00:00.549, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\SymfonyTest::testGreetingsEndPointReturnsPersonalisedGreeting
Failed asserting that 404 matches expected 200.

/usr/share/fullstackExercise/test/functional/SymfonyTest.php:28

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.166]
root@20ad01609407:/usr/share/fullstackExercise#

We're now OK to work out how to get Symfony installed and running, and make an end point that behaves as we need it to.

Or are we? I think I might have got a bit ahead of myself here. I'm back looking at Martin's article, and he says

Let's proceed to the Symfony installation:

$ symfony new src

[…] open http://localhost, you will see a Symfony 5 welcome screen.

OK so I think our first test should be that that Symfony 5 welcome screen works. I've got no idea what the screen says, so let me google for expectations. Right according to Creating your First Symfony App and Adding Authentication by Olususi Oluyemi (from a random Google Images search for "symfony 5 welcome screen") it should look like this:

I'm gonna write a quick test to check for that, and bypass that other test for now:

/** @coversNothing */
public function testSymfonyWelcomeScreenDisplays()
{

    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get(
        "/",
        ['http_errors' => false]
    );
    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

    $html = $response->getBody();
    $document = new \DOMDocument();
    $document->loadHTML($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
    $this->assertCount(1, $hasTitle);
}

I was scratching my head guessing what the mark-up around the "Welcome to Symfony x.y.z" text in the body is. It could be <h2> and <h3> tags, or it could just be different font-size stylings. Then I noticed the <title> content, and just decided to check for that. If that string is there: we're good. To start with, we want to be not good, so let's run that test:

root@20ad01609407:/usr/share/fullstackExercise# vendor/bin/phpunit test/functional/SymfonyTest.php
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 00:00.555, Memory: 6.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\test\functional\SymfonyTest::testSymfonyWelcomeScreenDisplays
Failed asserting that actual size 0 matches expected size 1.

/usr/share/fullstackExercise/test/functional/SymfonyTest.php:32

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.195]
root@20ad01609407:/usr/share/fullstackExercise#

Right so both Martin's article and the Symfony docs say I need to install Symfony (via Composer), and create an application with it, so I'll do this now.

The first hiccup is that the Symfony installer assumes that yer Symfony app is the centre of the world, and it needs to be the one that creates yer project directory for you. Great. Thanks for that. There's probably (?) a way of coercing it to understand that it's just a frickin' framework and I just need to add it to my existing app which is already underway. This is not so far--fetched I think, but after 15min of googling I drew a blank, so I decided to take a different approach. I'm gonna blow away my stuff to allow Symfony to play its silly little game, and then reintegrate my stuff back into the stuff Symfony creates. After all it's just some tests and composer.json config at present really.

/me thinks things through… OK that won't work. I can't shift all the files out of the fullstackExercise directory because all the Docker stuff is in there, and I can't bring my containers up if they're not there. Without the containers up, I don't have PHP or Composer so I can't do the Symfony install. To be blunt here: screw you, Symfony (well it's actually Composer that is bleating about the directory not being empty I guess. So screw Composer). With Silex? Just add an entry to composer.json, add a line to index.php and that's it. This is how a microframework should work.

Plan B is to do things the other way around. I need to shift my fullStackExercise project directory to have a different name (fullStackExercise.bak) temporarily so I can allow Symfony to create its own one. To do this I need to rejig my docker/docker-compose.yml file and the docker/php-fpm/Dockerfile to reference fullstackExercise.bak. Note that docker/nginx/sites/default.conf file also references root /usr/share/fullstackExercise/public, so at this point I decided that for now I'll just bring up the PHP container for this, so stripped the Nginx and MariaDB stuff out of docker-compose.yml completely. So these are the file changes:

docker-compose.yml:

version: '3'

services:

  php-fpm:
    build:
      context: ./php-fpm
    environment:
      - DATABASE_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
    volumes:
      - ../..:/usr/share/src # was ..:/usr/share/fullstackExercise
      - ./php-fpm/root_home:/root
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

networks:
  backend:
    driver: "bridge"

Note that I've shifted the main volume up one level in the host's directory hierarchy. This is because I need to let Symfony recreate that fullStackExercise at that parent level.

php-fpm/Dockerfile:

FROM php:8.0-fpm
RUN apt-get update
RUN apt-get install git --yes
RUN apt-get install net-tools iputils-ping --yes
RUN apt-get install zip unzip --yes
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
ENV XDEBUG_MODE=coverage
#WORKDIR  /usr/share/fullstackExercise.bak/
WORKDIR  /usr/share/src/
CMD composer install ; php-fpm
EXPOSE 9000

I've made the same change to the WORKDIR here too, to match docker-compose.yml. I've also added in a line to install zip and unzip into the container, because when I was testing this, Composer was bleating about them not being present.

Now in theory if I drop my containers and bring them back up again, I should be able to bash into the PHP container and do the Symfony install:

adam@DESKTOP-QV1A45U:~$ cd /mnt/c/src/fullstackExercise.bak/docker
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ docker-compose down --remove-orphans
Removing network docker_backend
WARNING: Network docker_backend not found.
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ # OK I forgot I had already brought them down

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ docker-compose up --build --detach
Creating network "docker_backend" with driver "bridge"
Building php-fpm
[...]
Successfully built 4b8851fefd22
Successfully tagged docker_php-fpm:latest
Creating docker_php-fpm_1 ... done

adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise.bak/docker$ docker exec --interactive --tty docker_php-fpm_1 /bin/bash
root@87420db3da90:/usr/share/src# ll
total 4
drwxrwxrwx 1 1000 1000  512 Jan  1 16:16 ./
drwxr-xr-x 1 root root 4096 Jan  1 16:40 ../
drwxrwxrwx 1 1000 1000  512 Dec 20 12:58 fullstackExercise.bak/
root@87420db3da90:/usr/share/src#
root@87420db3da90:/usr/share/src# # so far so good
root@87420db3da90:/usr/share/src#
root@87420db3da90:/usr/share/src# composer create-project symfony/skeleton fullstackExercise
Creating a "symfony/skeleton" project at "./fullstackExercise"
Installing symfony/skeleton (v5.2.99)
[...]

Some files may have been created or updated to configure your new packages.
Please review, edit and commit them: these files are yours.


What's next?


  * Run your application:
    1. Go to the project directory
    2. Create your code repository with the git init command
    3. Download the Symfony CLI at https://symfony.com/download to install a development web server

  * Read the documentation at https://symfony.com/doc

root@87420db3da90:/usr/share/src#

So far so good. now I just need to move a bunch of stuff back from my app into this one. I'll go ahead and do it, and report back when done, and list what I needed to do…

  • .git/, .idea/ (this is PHPStorm's project config), docker/, log/ directories could just be copied straight across.
  • As could LICENSE, phpcs.xml, phpmd.xml, phpunit.xml, README.md.
  • And src/MyClass.php, public/gdayWorld.html and public/gdayWorld.php.
  • The only overlap in .gitignore was the vendor/ directory, otherwise I could just move my bits back in.
  • composer.json was fairly straight forward, but I raised an eyebrow that Symfony seems to suggest using App as a top-level namespace:
    "autoload": {
        "psr-4": {
            "App\\": "src/",
    

    To me, using a namespace "App" is like calling a variable something like indexVariable, or totalVariable or something, or naming a class AnimalClass. Ugh. It gets even worse with the testing namespace: App\\Tests\\. For now I'm just adding my own namespacing in as well. Hopefully I don't have to use Symfony's approach here.
  • Symfony uses tests as its test directory; I've always used test, but I'm ambivalent about this so am sticking with Symfony's lead here: I'll change my directory to tests. This will require me changing the namespaces in the test source code files too.
  • Because I've changed composer.json by hand, I'll need get rid of composer.lock and vendor/ and do a composer install again.
  • Lastly I've reverted the Docker files I'd changed above back to how they were before: removing the references to fullstackExercise.bak, and bringing back the Ngxin and MariaDB containers.

OK having done all that, and now will rebuild my containers. Before starting this paragraph I had done all this and had started to fight with an issue I thought I was having with PHPUnit, so I didn't capture the screen showing it all working. It's all just Docker bumpf, then some Composer bumpf, none of which was very surprising, so I'll spare you the detail. Back to the unit test run:

root@51b5afdff6ea:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

F....                                                               5 / 5 (100%)

Time: 00:02.896, Memory: 14.00 MB

There was 1 failure:

1) adamCameron\fullStackExercise\tests\functional\SymfonyTest::testSymfonyWelcomeScreenDisplays
Failed asserting that 404 matches expected 200.

/usr/share/fullstackExercise/tests/functional/SymfonyTest.php:23

FAILURES!
Tests: 5, Assertions: 10, Failures: 1.

Generating code coverage report in HTML format ... done [00:01.152]
root@51b5afdff6ea:/usr/share/fullstackExercise#

Doh!

Things get weirder cos I hit the site on my host browser:

And that seems fine. First troubleshooting step: factor-out all the code I'm running, and manually curl it from PHP:

root@51b5afdff6ea:/usr/share/fullstackExercise# php -a
Interactive shell

php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_URL, "webserver.backend");
php > curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
php > $output = curl_exec($ch);
php > curl_close($ch);
php > echo $output;
<!DOCTYPE html>
<html dir="ltr" lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="robots" content="noindex,nofollow,noarchive,nosnippet,noodp,notranslate,noimageindex" />
    <title>Welcome to Symfony!</title>

So there's all the content. Fine.

Next I want to check just Guzzle by itself, to demonstrate to myself it is working fine. I hand-cranked some Guzzle calls from outside of my test code:

root@51b5afdff6ea:/usr/share/fullstackExercise# php -a
Interactive shell

php > require 'vendor/autoload.php';
php > $client = new \GuzzleHttp\Client(['base_uri' => 'http://webserver.backend']);
php > $response = $client->get("/");

Warning: Uncaught GuzzleHttp\Exception\ClientException: Client error: `GET http://webserver.backend/` resulted in a `404 Not Found` response:
<!DOCTYPE html>
<html dir="ltr" lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="robots" content="noindex, (truncated...)

It's still frickin' 404ing.

I scratched my head a lot and googled why guzzle might 404 on something that everything else 200s on. Then I saw it.

That mark-up that is spewing out in the Guzzle error is not generic 404 response. What it is is the content from the Welcome to Symfony! page. I checked something else in my browser…

That "welcome" page? It has a 404 status. Now I can see what they're doing here, and they even warn about it if one pays attn:

However to me a welcome page - that actually has welcome content on it, therefore is precisely what one would be expecting on that URL- is not a 404 situation. Ah well.

So my test is reporting correctly! (Yay for tests!) It's just that the expectations I gave the test were wrong. I need to expect a 404, not a 200. I'll make that change, and now the tests all pass:

root@51b5afdff6ea:/usr/share/fullstackExercise# vendor/bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.....                                                               5 / 5 (100%)

Time: 00:07.614, Memory: 14.00 MB

OK (5 tests, 11 assertions)

Generating code coverage report in HTML format ... done [00:01.173]
root@51b5afdff6ea:/usr/share/fullstackExercise#

Phew. BTW I had to change my test a bit more than I expected:

/** @coversNothing */
public function testSymfonyWelcomeScreenDisplays()
{

    $client = new Client([
        'base_uri' => 'http://webserver.backend'
    ]);

    $response = $client->get(
        "/",
        ['http_errors' => false]
    );
    // unexpectedly perhaps: the welcome page returns a 404. This is "by design"
    $this->assertEquals(Response::HTTP_NOT_FOUND, $response->getStatusCode());

    $html = $response->getBody();
    $document = $this->loadHtmlWithoutHtml5ErrorReporting($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
    $this->assertCount(1, $hasTitle);
}

private function loadHtmlWithoutHtml5ErrorReporting($html) : \DOMDocument
{
    $document = new \DOMDocument();
    libxml_use_internal_errors(TRUE);
    $document->loadHTML($html);
    libxml_clear_errors();

    return $document;
}

There's a bug in PHP's HTML loading that makes it barf on HTML5 tags (see "PHP DOMDocument errors/warnings on html5-tags", so I needed to put some mitigation in for that. It's not a great solution, but it works.

Now to try to work out how to implement my /greetings/Zachary route…

But not for now. I've just checked the length of this, and not counting all the code and telemetry, this is already close to 1800 words (it'll be over by the time I finish this outro). I think this exercise of getting the Symfony framework code into my project and up and running is sufficient for a stand-alone article. I'll do a separate article for the "and now let's get it to do something except return a confusing (to me. Cringe) 404 response". I'll crack on with that tomorrow, over here: "Part 7: Using Symfony".

Righto.

-- 
Adam