Showing posts with label Kahlan. Show all posts
Showing posts with label Kahlan. Show all posts

Saturday 6 March 2021

Symfony & TDD: adding endpoints to provide data for front-end workshop / registration requirements

G'day:

That's probably a fairly enigmatic title if you have not read the preceding article: "Vue.js: using TDD to develop a data-entry form". It's pretty long-winded, but the gist of it is summarised in this copy and pasted tract:

Right so the overall requirement here is (this is copy and pasted from the previous article) to construct an event registration form (personal details, a selection of workshops to register for), save the details to the DB and echo back a success page. Simple stuff. Less so for me given I'm using tooling I'm still only learning (Vue.js, Symfony, Docker, Kahlan, Mocha, MariaDB).

There's been two articles around this work so far:

There's also a much longer series of articles about me getting the Docker environment running with containers for Nginx, Node.js (and Vue.js), MariaDB and PHP 8 running Symfony 5. It starts with "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers - Part 1: Intro & Nginx" and runs for 12 articles.

In the previous article I did the UI for the workshop registration form…

… and the summary one gets after submitting one's registration:

Today we're creating the back-end endpoint to fetch the list of workshops in that multiple select, and also another endpoint to save the registration details that have been submitted (ran out of time for this bit). The database we'll be talking to is as follows:

(BTW, dbdiagram.io is a bloody handy website... I just did a mysqldump of my tables (no data), imported it their online charting tool, and... done. Cool).

Now… as for me and Symfony… I'm really only starting out with it. The entirety of my hands-on exposure to it is documented in "Part 6: Installing Symfony" and "Part 7: Using Symfony". And the "usage" was very superficial. I'm learning as I go here.

And as-always: I will be TDDing every step, using a tight cycle of identify a case (eg: "it needs to return a 200-OK status for GET requests on the endpoint /workshops"); create tests for that case; do the implementation code just for that case.


It needs to return a 200-OK status for GET requests on the /workshops endpoint

This is a pretty simple test:

namespace adamCameron\fullStackExercise\spec\functional\Controller;

use adamCameron\fullStackExercise\Kernel;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;

describe('Tests of WorkshopController', function () {

    beforeAll(function () {
        $this->request = Request::createFromGlobals();
        $this->kernel  = new Kernel('test', false);
    });

    describe('Tests of doGet', function () {
        it('needs to return a 200-OK status for GET requests', function () {

            $request = $this->request->create("/workshops/", 'GET');
            $response = $this->kernel->handle($request);

            expect($response->getStatusCode())->toBe(Response::HTTP_OK);
        });
    });
});

I make a request, I check the status code of the response. That's it. And obviously it fails:

root@58e3325d1a16:/usr/share/fullstackExercise# vendor/bin/kahlan --spec=spec/functional/Controller/workshopController.spec.php --lcov="var/tmp/lcov/coverage.info" --ff

[error] Uncaught PHP Exception Symfony\Component\HttpKernel\Exception\NotFoundHttpException: "No route found for "GET /workshops/"" at /usr/share/fullstackExercise/vendor/symfony/http-kernel/EventListener/RouterListener.php line 136

F                                                                   1 / 1 (100%)


Tests of WorkshopController
  Tests of doGet
    ✖ it needs to return a 200-OK status for GET requests
      expect->toBe() failed in `.spec/functional/Controller/workshopController.spec.php` line 22

      It expect actual to be identical to expected (===).

      actual:
        (integer) 404
      expected:
        (integer) 200

Perfect. Now let's add a route. And probably wire it up to a controller class and method I guess. Here's what I've got:

# backend/config/routes.yaml
workshops:
  path: /workshops/
  controller: adamCameron\fullStackExercise\Controller\WorkshopsController::doGet
// backend/src/Controller/WorkshopsController.php
namespace adamCameron\fullStackExercise\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;

class WorkshopsController extends AbstractController
{
    public function doGet() : JsonResponse
    {
        return new JsonResponse(null);
    }
}

Initially this continued to fail with a 404, but I worked out that Symfony caches a bunch of stuff when it's not in debug mode, so it wasn't seeing the new route and/or the controller until I switched the Kernel obect initialisation to switch debug on:

// from workshopsController.spec.php, as above:

$this->kernel  = new Kernel('test', true);

And then the actual code ran. Which is nice:

Passed 1 of 1 PASS in 0.108 seconds (using 8MB)

(from now on I'll just let you know if the tests pass, rather than spit out the output).

Bye Kahlan, hi again PHPUnit

I've been away from this article for an entire day, a lot of which was down to trying to get Kahlan to play nicely with Symfony, and also getting Kahlan's own functionality to not obstruct me from forward progress. However I've hit a wall with some of the bugs I've found with it, specifically "Documented way of disabling patching doesn't work #378" and "Bug(?): Double::instance doesn't seem to cater to stubbing methods with return-types #377". These render Kahlan unusable for me.

The good news is I sniffed around PHPUnit a bit more, and discovered its testdox functionality which allows me to write my test cases in good BDD fashion, and have those show up in the test results. It'll either "rehydrate" a human-readable string from the test name (testTheMethodDoesTheThing becomes "test the method does the thing"), or one can specify an actual case via the @testdox annotation on methods and the classes themselves (I'll show you below, outside this box). This means PHPUnit will achieve what I need, so I'm back to using that.

OK, backing up slightly and switching over to the PHPUnit test (backend/tests/functional/Controller/WorkshopsControllerTest.php):

/**
 * @testdox it needs to return a 200-OK status for GET requests
 * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
 */
public function testDoGetReturns200()
{
    $this->client->request('GET', '/workshops/');

    $this->assertEquals(Response::HTTP_OK, $this->client->getResponse()->getStatusCode());
}
> vendor/bin/phpunit --testdox 'tests/functional/Controller/WorkshopsControllerTest.php' '--filter=testDoGetReturns200'
PHPUnit 9.5.2 by Sebastian Bergmann and contributors.

Tests of WorkshopController
it needs to return a 200-OK status for GET requests

Time: 00:00.093, Memory: 12.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in HTML format ... done [00:00.682]
root@5f9133aa9de3:/usr/share/fullstackExercise#

Cool.


It returns a collection of workshop objects, as JSON

The next case is:

/**
 * @testdox it returns a collection of workshop objects, as JSON
 */
public function testDoGetReturnsJson()
{
}

This is a bit trickier, in that I actually need to write some application code now. And wire it into Symfony so it works. And also test it. Via Symfony's wiring. Eek.

Here's a sequence of thoughts:

  • We are getting a collection of Workshop objects from [somewhere] and returning them in JSON format.
  • IE: that would be a WorkshopCollection.
  • The values for the workshops are stored in the DB.
  • The WorkshopCollection will need a way of getting the data into itself. Calling some method like loadAll
  • That will need to be called by the controller, so the controller will need to receive a WorkshopCollection via Symfony's DI implementation.
  • A model class like WorkshopCollection should not be busying itself with the vagaries of storage. It should hand that off to a repository class (see "The Repository Pattern"), which will handle the fetching of DB data and translating it from a recorset to an array of Workshop objects.
  • As WorkshopsRepository will contain testable data-translation logic, it will need unit tests. However we don't want to have to hit the DB in our tests, so we will need to abstract the part of the code that gets the data into something we can mock away.
  • As we're using Doctrine/DBAL to connect to the database, and I'm a believer in "don't mock what you don't own", we will put a thin (-ish) wrapper around that as WorkshopsDAO. This is not completely "thin" because it will "know" the SQL statements to send to its connector to get the data, and will also "know" the DBAL statements to get the data out and pass back to WorkshopsRepository for modelling.

That seems like a chunk to digest, and I don't want you to think I have written any of this code, but this is the sequence of thoughts that leads me to arrive at the strategy for handling the next case. I think from the first bits of that bulleted list I can derive sort of how the test will need to work. The controller doesn't how this WorkshopCollection gets its data, but it needs to be able to tell it to do it. We'll mock that bit out for now, just so we can focus on the controller code. We will work our way back from the mock in another test. For now we have backend/tests/functional/Controller/WorkshopsControllerTest.php

/**
 * @testdox it returns a collection of workshop objects, as JSON
 * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
 */
public function testDoGetReturnsJson()
{
    $workshops = [
        new Workshop(1, 'Workshop 1'),
        new Workshop(2, 'Workshop 2')
    ];

    $this->client->request('GET', '/workshops/');

    $resultJson = $this->client->getResponse()->getContent();
    $result = json_decode($resultJson, false);

    $this->assertCount(count($workshops), $result);
    array_walk($result, function ($workshopValues, $i) use ($workshops) {
        $workshop = new Workshop($workshopValues->id, $workshopValues->name);
        $this->assertEquals($workshops[$i], $workshop);
    });
}

To make this pass we need just enough code for it to work:

class WorkshopsController extends AbstractController
{

    private WorkshopCollection $workshops;

    public function __construct(WorkshopCollection $workshops)
    {
        $this->workshops = $workshops;
    }

    public function doGet() : JsonResponse
    {
        $this->workshops->loadAll();

        return new JsonResponse($this->workshops);
    }
}
class WorkshopCollection implements \JsonSerializable
{
    /** @var Workshop[] */
    private $workshops;

    public function loadAll()
    {
        $this->workshops = [
            new Workshop(1, 'Workshop 1'),
            new Workshop(2, 'Workshop 2')
        ];
    }

    public function jsonSerialize()
    {
        return $this->workshops;
    }
}

And thanks to Symphony's dependency-injection service container's autowiring, all that just works, just like that. That's the test for that end point done.

Now there was all thant bumpf I mentioned about repositories and DAOs and connectors and stuff. As part of the refactoring part of this, we are going to push our implementation right back to the DAO. This allows us to complete the parts of the code in the WorkshopCollection, WorkshopsRepository and just mock-out the DAO for now.

class WorkshopCollection implements \JsonSerializable
{
    private WorkshopsRepository $repository;

    /** @var Workshop[] */
    private $workshops;

    public function setRepository(WorkshopsRepository $repository) : void
    {
        $this->repository = $repository;
    }

    public function loadAll()
    {
        $this->workshops = $this->repository->selectAll();
    }

    public function jsonSerialize()
    {
        return $this->workshops;
    }
}

My thinking here is:

  • It's going to need a WorkshopsRepository to get stuff from the DB.
  • It doesn't seem right to me to pass in a dependency to a model as a constructor arguments. The model should work without needing a DB connection; just the methods around storage interaction should require the repository. On the other hand the only thing the collection does now is to be able to load the stuff from the DB and serialise it, so I'm kinda coding for the future here, and I don't like that. But I'm sticking with it for reasons we'll come to below.
  • I also really hate model classes with getters and setters. This is usually a sign of bad OOP. But here I have a setter, to get the repository in there.

The reason (it's not a reason, it's an excuse) I'm not passing in the repo as a constructor argument and instead using a setter is because I wanted to check out how Symfony's service config dealt with the configuration of this. If yer classes all have type-checked constructor args, Symfony just does it all automatically with no code at all (just a config switch). However to handle using the setRepository method I needed a factory method to do so. The config for it is thus (in backend/config/services.yaml):

adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory: ~
adamCameron\fullStackExercise\Model\WorkshopCollection:
    factory: ['@adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory', 'getWorkshopCollection']

Simple! And the code for WorkshopCollectionFactory:

class WorkshopCollectionFactory
{
    private WorkshopsRepository $repository;

    public function __construct(WorkshopsRepository $repository)
    {
        $this->repository = $repository;
    }

    public function getWorkshopCollection() : WorkshopCollection
    {
        $collection = new WorkshopCollection();
        $collection->setRepository($this->repository);

        return $collection;
    }
}

Also very simple. But, yeah, it's an exercise in messing about, and there's no way I should have done this. I should have just used a constructor argument. Anyway, moving on.

The WorkshopsRepository is very simple too:

class WorkshopsRepository
{
    private WorkshopsDAO $dao;

    public function __construct(WorkshopsDAO $dao)
    {
        $this->dao = $dao;
    }

    /** @return Workshop[] */
    public function selectAll() : array
    {
        $records = $this->dao->selectAll();
        return array_map(
            function ($record) {
                return new Workshop($record['id'], $record['name']);
            },
            $records
        );
    }
}

I get some records from the DAO, and map them across to Workshop objects. Oh! Workshop:

class Workshop implements \JsonSerializable
{
    private int $id;
    private string $name;

    public function __construct(int $id, string $name)
    {
        $this->id = $id;
        $this->name = $name;
    }

    public function jsonSerialize()
    {
        return (object) [
            'id' => $this->id,
            'name' => $this->name
        ];
    }
}

And lastly I mock WorkshopsDAO. I can't implement any further down the stack of this process because the DAO is what uses the DBAL Connector object, and I don't own that, so if I actually started to use it, I'd be hitting the DB. Or hitting the ether and getting an error. Either way: no good for our test. So a mocked DAO:

class WorkshopsDAO
{
    public function selectAll() : array
    {
        return [
            ['id' => 1, 'name' => 'Workshop 1'],
            ['id' => 2, 'name' => 'Workshop 2']
        ];
    }
}

Having done all that refactoring, we check if our test is still good, and it is. I can verify this is not a trick of the light by changing some of that data in the DAO, and watch the test break (which it does). I can also now go back to the test and stick some more code-coverage annotations in:

/**
 * @testdox it returns a collection of workshop objects, as JSON
 * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
 * @covers \adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory
 * @covers \adamCameron\fullStackExercise\Repository\WorkshopsRepository
 * @covers \adamCameron\fullStackExercise\Model\WorkshopCollection
 * @covers \adamCameron\fullStackExercise\Model\Workshop
 */

And see that all the code is indeed covered:


It returns the expected workshops from the database

But now we need to implement the real DAO. once we do that, our test will break because the DAO will suddenly start hitting the DB, and we'll be getting back whatever is in the DB, not our expected canned response. Plus we don't want this test to hit the DB anyhow. So first we're gonna mock-out the DAO using PHPUnit's mocks instead of our code-mock. To do this turned out to be a bit tricky, initially, given Symfony's DI container is looking after all the dependencies for us, but fortunately when in test mode, Symfony allows us to hack into that container. I've updated my test, thus:

public function testDoGetReturnsJson()
{
    $workshopDbValues = [
        ['id' => 1, 'name' => 'Workshop 1'],
        ['id' => 2, 'name' => 'Workshop 2']
    ];

    $this->mockWorkshopDaoInServiceContainer($workshopDbValues);

    // ... unchanged ...

    array_walk($result, function ($workshopValues, $i) use ($workshopDbValues) {
        $this->assertEquals($workshopDbValues[$i], $workshopValues);
    });
}

private function mockWorkshopDaoInServiceContainer($returnValue = []): void
{
    $mockedDao = $this->createMock(WorkshopsDAO::class);
    $mockedDao->method('selectAll')->willReturn($returnValue);

    $container = $this->client->getContainer();
    $workshopRepository = $container->get('test.WorkshopsRepository');

    $reflection = new \ReflectionClass($workshopRepository);
    $property = $reflection->getProperty('dao');
    $property->setAccessible(true);
    $property->setValue($workshopRepository, $mockedDao);
}

We are popping a mocked WorkshopsDAO into the WorkshopsRepository object in the container So when the repo calls it, it'll be calling the mock.

Oh! to be able to access that 'test.WorkshopsRepository' container key, we need to expose it via the services_test.xml container config:

services:
  test.WorkshopsRepository:
    alias: adamCameron\fullStackExercise\Repository\WorkshopsRepository
    public: true

And running that, the test works, and is ignoring the reallyreally DAO.

To test the final DAO implementation, we're gonna do an end-to-end test:

class WorkshopControllerTest extends WebTestCase
{
    private KernelBrowser $client;

    public static function setUpBeforeClass(): void
    {
        $dotenv = new Dotenv();
        $dotenv->load(dirname(__DIR__, 3) . "/.env.test");
    }

    protected function setUp(): void
    {
        $this->client = static::createClient(['debug' => false]);
    }

    /**
     * @testdox it returns the expected workshops from the database
     * @covers \adamCameron\fullStackExercise\Controller\WorkshopsController
     * @covers \adamCameron\fullStackExercise\Factory\WorkshopCollectionFactory
     * @covers \adamCameron\fullStackExercise\Model\WorkshopCollection
     * @covers \adamCameron\fullStackExercise\Repository\WorkshopsRepository
     * @covers \adamCameron\fullStackExercise\Model\Workshop
     */
    public function testDoGet()
    {
        $this->client->request('GET', '/workshops/');
        $response = $this->client->getResponse();
        $workshops = json_decode($response->getContent(), false);

        /** @var Connection */
        $connection = static::$container->get('database_connection');
        $expectedRecords = $connection->query("SELECT id, name FROM workshops ORDER BY id ASC")->fetchAll();

        $this->assertCount(count($expectedRecords), $workshops);
        array_walk($expectedRecords, function ($record, $i) use ($workshops) {
            $this->assertEquals($record['id'], $workshops[$i]->id);
            $this->assertSame($record['name'], $workshops[$i]->name);
        });
    }
}

The test is pretty familiar, except it's actually getting its expected data from the database, and making sure the whole process, end to end, is doing what we want. Currently when we run this it fails because we still have our mocked DAO in place (the real mock, not the… mocked mock. Um. You know what I mean: the actual DAO class that just returns hard-coded data). Now we put the proper DAO code in:

class WorkshopsDAO
{
    private Connection $connection;

    public function __construct(Connection $connection)
    {
        $this->connection = $connection;
    }

    public function selectAll() : array
    {
        $sql = "
            SELECT
                id, name
            FROM
                workshops
            ORDER BY
                id ASC
        ";
        $statement = $this->connection->executeQuery($sql);

        return $statement->fetchAllAssociative();
    }
}

And now if we run the tests:

> vendor/bin/phpunit --testdox
PHPUnit 9.5.2 by Sebastian Bergmann and contributors.

Tests of WorkshopController
it needs to return a 200-OK status for GET requests
it returns a collection of workshop objects, as JSON

Tests of baseline Symfony install
it displays the Symfony welcome screen
it returns a personalised greeting from the /greetings end point

PHP config tests
gdayWorld.php outputs G'day world!

Webserver config tests
It serves gdayWorld.html with expected content

End to end tests of WorkshopController
it returns the expected workshops from the database

Tests that code coverage analysis is operational
It reports code coverage of a simple method correctly

Time: 00:00.553, Memory: 22.00 MB

OK (8 tests, 26 assertions)

Generating code coverage report in HTML format ... done [00:00.551]
root@5f9133aa9de3:/usr/share/fullstackExercise#

Nice one!

And if we look at code coverage:

I was gonna try to cover the requirements for the process ot save the form fields in the article too, but it took ages to work out how some of the Symfony stuff worked, plus I sunk about a day into trying to get Kahlan to work, and this article is already super long anyhow. I now have the back-end processing sorted out to update the front-end form to actually use the values from the DB instead of test values. I might look at that tomorrow (see "TDDing the reading of data from a web service to populate elements of a Vue JS component" for that exercise). I need a rest from Symfony.


It needs to drink some beer

I'm rushing the outro of this article because I am supposed to be on a webcam with a beer in my hand in 33min, and I need to proofread this still…

Righto.

--
Adam

Thursday 4 March 2021

Kahlan: getting it working with Symfony 5 and generating a code coverage report

G'day:

This is not the article I intended to write today. Today (well: I hoped to have it done by yesterday, actually) I had hoped to be writing about my happy times doing using TDD to implement a coupla end-points I need in my Symfony-driven web service. I got 404 (chuckle) words into that and then was blocked by trying to get Kahlan to play nice for about five hours (I did have dinner in that time too, but it was add my desk, and with a scowl on my face googling stuff). And that put me at 1am so I decided to go to bed. I resumed today an hour or so ago, and am just now in the position to get going again. But I've decided to document that lost six hours first.

I sat down to create a simple endpoint to fetch some data, and started by deciding on my first test case, which was "It needs to return a 200-OK status for GET requests on the /workshops endpoint". I knew Symfony did some odd shenanigans to be able to functionally test right from a route slug rather than having tests directly instantiating controller classes and such. I checked the docs and all this is done via an extension of PHPUnit, using WebTestCase. But I don't wanna use PHPUnit for this. Can you imagine my test case name? Something like: testItNeedsToReturnA200OKStatusForGetRequestsOnTheWorkshopsEndpoint. Or I could break PSR-12/PSR-1 and make it (worse) with test_it_needs_to_Return_a_200_OK_status_for_get_requests_on_the_workshops_endpoint (this is why I will never use phpspec). Screw that. I'm gonna work out how to do these Symfony WebTestCase tests in Kahlan.

Not so fast mocking PHPUNit there, Cameron

2021-03-06

Due to some - probably show-stopping - issues I'm seeing with Kahlan, I have been looking at PHPUnit some more. I just discovered the textdox reporting functionality it has, which makes giving test case names much clearer.

/** @testdox Tests the /workshops endpoint methods */
class workshopsEndPointTet {
    /** @testdox it needs to return a 200-OK status for GET requests */
    function testReturnStatus() {}
}

This will output in the test runs as:

Perfect. I still prefer the style of code Kahlan uses, but… this is really good to know.

First things first, I rely heavily on PHPUnit's code coverage analysis, so I wanted to check out Kahan's offering. The docs seem pretty clear ("Code Coverage"), and seems I just want to be using the lcov integration Kahlan offers, like this:

vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info"
genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info

I need to install lcov first via my Dockerfile:

RUN apt-get install lcov --yes

OK so I did all that, and had a look at the report:

Pretty terrible coverage, but it's working. Excellent. But drilling down into the report I see this:

>

This is legit reporting because I have no tests for the Kernel class, but equally that class is generated by Symfony and I don't want to cover that. How do I exclude it from code coverage analysis? I'm looking for Kahlan's equivalent of PHPUnit's @codeCoverageIgnore. There's nothing in the docs, and all I found was a passing comment against an issue in Github asking the same question I was: "Exclude a folder in code coverage #321". The answer is to do this sort of thing to my kahlan-config.php file:

use Kahlan\Filter\Filters;
use Kahlan\Reporter\Coverage;
use Kahlan\Reporter\Coverage\Driver\Xdebug;

$commandLine = $this->commandLine();
$commandLine->option('no-header', 'default', 1);

Filters::apply($this, 'coverage', function($next) {
    if (!extension_loaded('xdebug')) {
        return;
    }
    $reporters = $this->reporters();
    $coverage = new Coverage([
        'verbosity' => $this->commandLine()->get('coverage'),
        'driver'    => new Xdebug(),
        'path'      => $this->commandLine()->get('src'),
        'exclude'   => [
            'src/Kernel.php'
        ],
        'colors'    => !$this->commandLine()->get('no-colors')
    ]);
    $reporters->add('coverage', $coverage);
});

That seems a lot of messing around to do something that seems like it should be very simple to me. I will also note that Kahlan - currently - has no ability to suppress code coverage at a method or code-block level either (see "Skip individual functions in code coverage? #333"). This is not a deal breaker for me in this work, but it would be a show-stopper on any of the codebases I have worked on in the last decade, as they've all been of dubious quality, and all needed some stuff to be actively "overlooked" as they're not testable as they currently stand, and we (/) like my baseline code coverage report to have 100% coverage reported, and be entirely green. This is so if any omissions creep in, they're easy to spot (see "Yeah, you do want 100% test coverage"). Anyway, I'll make that change and omit Kernel from analysis:

root@13038aa90234:/usr/share/fullstackExercise# vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info"

.................                                                 17 / 17 (100%)



Expectations   : 51 Executed
Specifications : 0 Pending, 0 Excluded, 0 Skipped

Passed 17 of 17 PASS in 0.527 seconds (using 7MB)

Coverage Summary
----------------

Total: 33.33% (1/3)

Coverage collected in 0.001 seconds


root@13038aa90234:/usr/share/fullstackExercise# genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info
Reading data file var/tmp/lcov/coverage.info
Found 2 entries.
Found common filename prefix "/usr/share/fullstackExercise"
Writing .css and .png files.
Generating output.
Processing file src/MyClass.php
Processing file src/Controller/GreetingsController.php
Writing directory view page.
Overall coverage rate:
  lines......: 33.3% (1 of 3 lines)
  functions..: 50.0% (1 of 2 functions)
root@13038aa90234:/usr/share/fullstackExercise#

And the report now doesn't mention Kernel:

Cool.

Now to implement that test case. I need to work out how to run a Symfony request without using WebTestCase. Well I say "I need to…" I mean I need to google someone else who's already done it, and copy them. I have NFI how to do it, and I'm not prepared to dive into Symfony code to find out how. Fortunately someone has already cracked this one: "Functional Test Symfony 4 with Kahlan 4". It says "Symfony 4", but I'll check if it works on Symfony 5 too. I also happened back to the Kahlan docs, and they mention the same guy's solution ("Integration with popular frameworks › Symfony"). This one points to a library to encapsulate it (elephantly/kahlan-bundle), but that is actively version-constrained to only Symfony 4. Plus it's not seen any work since 2017, so I suspect it's abandoned.

Anyway, back to samsonasik's blog article. It looks like this is the key bit:

$this->request = Request::createFromGlobals();
$this->kernel  = new Kernel('test', false);
$request = $this->request->create('/lucky/number', 'GET');
$response = $this->kernel->handle($request);

That's how to create a Request and get Symfony's Kernel to run it. Easy. Hopefully. Let's try it.

namespace adamCameron\fullStackExercise\spec\functional\Controller;

use adamCameron\fullStackExercise\Kernel;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;

describe('Tests of GreetingsController', function () {

    beforeAll(function () {
        $this->request = Request::createFromGlobals();
        $this->kernel  = new Kernel('test', false);
    });

    describe('Tests of doGet', function () {
        it('returns a JSON greeting object', function () {
            $testName = 'Zachary';

            $request = $this->request->create("/greetings/$testName", 'GET');
            $response = $this->kernel->handle($request);

            expect($response->getStatusCode())->toBe(Response::HTTP_OK);
        });
    });
});

I'm not getting too ambitious here, and it's not addressing the entire test case yet. I'm just making the request and checking its response status code.

And this just goes splat:

root@13038aa90234:/usr/share/fullstackExercise# vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --ff

E                                                                 18 / 18 (100%)


Tests of GreetingsController
  Tests of doGet
    ✖ it returns a JSON greeting object
      an uncaught exception has been thrown in `vendor/symfony/framework-bundle/Kernel/MicroKernelTrait.php` line 91

      message:`Kahlan\PhpErrorException` Code(0) with message "`E_WARNING` require(/tmp/kahlan/usr/share/fullstackExercise/src/config/bundles.php): Failed to open stream: No such file or directory"

        [NA] - vendor/symfony/framework-bundle/Kernel/MicroKernelTrait.php, line  to 91

Eek. I had a look into this, and the code in question is try to do this:

$contents = require $this->getProjectDir().'/config/bundles.php';

And the code in getProjectDir is thus:

<pre class="source-code"><code>public function getProjectDir()
{
    if (null === $this-&gt;projectDir) {
        $r = new \ReflectionObject($this);

        if (!is_file($dir = $r-&gt;getFileName())) {
            throw new \LogicException(sprintf('Cannot auto-detect project dir for kernel of class &quot;%s&quot;.', $r-&gt;name));
        }

        $dir = $rootDir = \dirname($dir);
        while (!is_file($dir.'/composer.json')) {
            if ($dir === \dirname($dir)) {
                return $this-&gt;projectDir = $rootDir;
            }
            $dir = \dirname($dir);
        }
        $this-&gt;projectDir = $dir;
    }

    return $this-&gt;projectDir;
}
</code></pre>

The code starts in the directory of the current file, and traverses up the directory structure until it finds the directory with composer.json.If it doesn't find that, then - somewhat enigmatically, IMO - it just says "ah now, we'll just use the directory we're in now. It'll be grand". To me if it expects to find what it's looking for by looking up the ancestor directory path and that doesn't work: throw an exception. Still. In the normal scheme of things, this would work cos the app's Kernel class - by default - seems to live in the src directory, which is one down from where composer.json is.

So why didn't it work? Look at the directory that it's trying to load the bundles from: /tmp/kahlan/usr/share/fullstackExercise/src/config/bundles.php. Where? /tmp/kahlan/usr/share/fullstackExercise/. Ain't no app code in there, pal. It's in /usr/share/fullstackExercise/. Why's it looking in there? Because the Kernel object that is initiating all this is at /tmp/kahlan/usr/share/fullstackExercise/src/Kernel.php. It's not the app's own one (at /usr/share/fullstackExercise/src/Kernel.php), it's all down to how Kahlan monkey-patches everything that's going to be called by the test code, on the off chance you want to spy on anything. It achieves this by loading the source code of the classes, patching the hell out of it, and saving it in that /tmp/kahlan. The only problem with this is that when Symfony traverses up from where the patched Kernel class is… it never finds composer.json, so it just takes a guess at where the project directory is. And it's not a well-informed guess.

I'm not sure who I blame more here, to be honest. Kahlan for patching everything and running code from a different directory from where it's supposed to be; or Symfony for its "interesting" way to know where the project directory is. I have an idea here, Symfony: you could just ask me. Or even force me tell it. Ah well. Just trying to be "helpful" I s'pose.

Anyway, I can exclude files from being patched, according to the docs:

  --exclude=<string>                  Paths to exclude from patching. (default: `[]`).

I tried sticking the path to Kernel in there: src/Kernel.php, and that didn't work. I hacked about in the code and it doesn't actually want a path, it wants the fully-qualified class name, eg: adamCameron\fullStackExercise\Kernel. I've raised a ticket for this with Kahlan, just to clarify the wording there: Bug: bad nomenclature in help: "path" != "namespace".

This does work…

root@13038aa90234:/usr/share/fullstackExercise# vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --ff --exclude=adamCameron\\fullStackExercise\\Kernel

E                                                                 18 / 18 (100%)


Tests of GreetingsController
  Tests of doGet
    ✖ it returns a JSON greeting object
      an uncaught exception has been thrown in `vendor/symfony/deprecation-contracts/function.php` line 25

      message:`Kahlan\PhpErrorException` Code(0) with message "`E_USER_DEPRECATED` Please install the \"intl\" PHP extension for best performance."

        [NA] - vendor/symfony/deprecation-contracts/function.php, line  to 25
        trigger_deprecation - vendor/symfony/framework-bundle/DependencyInjection/FrameworkExtension.php, line  to 253

This is not exactly what I want, but it's a different error, so Symfony is finding itself this time, and then just faceplanting again. However when I look into the code, it's this:

if (!\extension_loaded('intl') && !\defined('PHPUNIT_COMPOSER_INSTALL')) {
    trigger_deprecation('', '', 'Please install the "intl" PHP extension for best performance.');
}

// which in turn...

function trigger_deprecation(string $package, string $version, string $message, ...$args): void
{
    @trigger_error(($package || $version ? "Since $package $version: " : '').($args ? vsprintf($message, $args) : $message), \E_USER_DEPRECATED);
}

So Symfony is very quietly raising a flag that it suggests I have that extension installed. But only as a deprecation notice, and even then it's @-ed out. Somehow Kahlan is getting hold of that and going "nonono, this is worth stopping for". No it ain't. Ticket raised: "Q: should trigger_error type E_USER_DEPRECATED cause testing to halt?".

Anyway, the point is a legit one, so I'll install the intl extension. I initially thought it was just a matter of slinging this in the Dockerfile:

RUN apt-get install --yes zlib1g-dev libicu-dev g++
RUN docker-php-ext-install intl

But that didn't work, I needed a bunch of Other Stuff too:

RUN apt-get install --yes zlib1g-dev libicu-dev g++
RUN docker-php-ext-install pdo_mysql
RUN docker-php-ext-configure intl
RUN docker-php-ext-install intl

(Thanks to the note in docker-php-ext-install intl fails #57 for solving that for me).

After rebuilding the container, let's see what goes wrong next:

root@58e3325d1a16:/usr/share/fullstackExercise# composer coverage
> vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --exclude=adamCameron\\fullStackExercise\\Kernel

..................                                                18 / 18 (100%)



Expectations   : 52 Executed
Specifications : 0 Pending, 0 Excluded, 0 Skipped

Passed 18 of 18 PASS in 0.605 seconds (using 12MB)

Coverage Summary
----------------

Total: 100.00% (3/3)

Coverage collected in 0.001 seconds


> genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info
Reading data file var/tmp/lcov/coverage.info
Found 2 entries.
Found common filename prefix "/usr/share/fullstackExercise"
Writing .css and .png files.
Generating output.
Processing file src/MyClass.php
Processing file src/Controller/GreetingsController.php
Writing directory view page.
Overall coverage rate:
  lines......: 100.0% (3 of 3 lines)
  functions..: 100.0% (2 of 2 functions)
root@58e3325d1a16:/usr/share/fullstackExercise#

(I've stuck a Composer script in for this, btw):

"coverage": [
    "vendor/bin/kahlan --lcov=\"var/tmp/lcov/coverage.info\" --exclude=adamCameron\\\\fullStackExercise\\\\Kernel",
    "genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info"
]

And most promising of all is this:

All green! I like that.

And now I'm back to where I wanted to be, yesterday, as I typed that 404th word of the article I was meant to be working on. 24h later now.

Righto.

--
Adam

Monday 22 February 2021

Troubleshooting an issue with Kahlan

G'day:

I'm ramping-up to do some more development with this fullstackExercise codebase I've been working on (see "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers", parts 1-12), and as part of the yak-shaving phase of things, I decided to port my PHP tests from PHPUnit to Kahlan. The reason for this is twofold. Firstly, I much prefer the describe / it syntax to writing and running tests over xUnit; and secondly - other than perpetually finding new ways to be frustrated by how PHPUnit handles/documents things - I am not learning a great deal about PHPUnit any more. So Kahlan fits in well with this learning exercise.

Porting the tests is easy, largely cos my tests are currently minimal in number and in complexity. But this PHPUnit class (test/functional/ExampleComTest.php):

namespace adamCameron\kahlanIssue\test\functional;

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class ExampleComTest extends TestCase
{
    /** @coversNothing */
    public function testExampleDotComReturnsExpectedContent()
    {
        $expectedContent = "Example Domain";


        $client = new Client([
            'base_uri' => 'http://example.com/'
        ]);

        $response = $client->get('index.html');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $html = $response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);

        $xpathDocument = new \DOMXPath($document);

        $hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasTitle);

        $hasHeading = $xpathDocument->query('/html/body/div/h1[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasHeading);
    }
}

Becomes this Kahlan spec(spec/functional/example.com.spec.php):

namespace adamCameron\fullStackExercise\spec\functional;

use GuzzleHttp\Client;
use Symfony\Component\HttpFoundation\Response;

describe('Tests that example.com can be curled', function () {

    beforeAll(function () {
        $client = new Client([
            'base_uri' => 'http://example.com/'
        ]);

        $this->response = $client->get('index.html');

        $html = $this->response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);
        $this->xpathDocument = new \DOMXPath($document);
    });

    it("should have expected status code", function () {
        expect($this->response->getStatusCode())->toBe(Response::HTTP_OK);
    });

    it("should have expected title", function () {
        $hasTitle = $this->xpathDocument->query('/html/head/title[text() = "Example Domain"]');
        expect($hasTitle)->toHaveLength(1);
    });

    it("should have expected heading", function () {
        $hasHeading = $this->xpathDocument->query('/html/body/div/h1[text() = "Example Domain"]');
        expect($hasHeading)->toHaveLength(1);
    });
});

The problems for me started when I came to run that test in Kahlan. I got this:

            _     _
  /\ /\__ _| |__ | | __ _ _ __
 / //_/ _` | '_ \| |/ _` | '_ \
/ __ \ (_| | | | | | (_| | | | |
\/  \/\__,_|_| |_|_|\__,_|_| |_|

The PHP Test Framework for Freedom, Truth and Justice.

src directory  :
spec directory : /mnt/c/src/kahlanIssue/spec

                                                                    3 / 3 (100%)


Tests that example.com can be curled
  an uncaught exception has been thrown in `vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php` line 158

  message:`ParseError` Code(0) with message "Unclosed '{' on line 142 does not match ')'"

    [NA] - vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php, line  to 158
    Kahlan\Jit\ClassLoader::loadFile() - vendor/kahlan/kahlan/src/Jit/ClassLoader.php, line 759
    Kahlan\Jit\ClassLoader::loadClass() - vendor/guzzlehttp/guzzle/src/Utils.php, line 95
    [… etc …]


Expectations   : 0 Executed
Specifications : 0 Pending, 0 Excluded, 0 Skipped

Passed 0 of 1 FAIL (EXCEPTION: 1) in 0.015 seconds (using 7MB)

Eek, OK that's a compilation error, not a runtime exception, which is odd. I checked the code of CurlMultiHandler.php to see if anything was broken in there, and unsurprisingly it wasn't. I decided to chuck a try / catch around the code in question, and see if the actual issue was something other than being reported. I found it odd that Kahlan was reporting it as an Exception, when do me a code-parsing issue is an out-and-out Error. Two different things. So I wondered if Kahlan was doing something odd. I ran the code again and the error spewed out again. What the hell? Then I looked more closely. The error had changed slightly. Now it says this:

an uncaught exception has been thrown in `vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php` line 201

message:`ParseError` Code(0) with message "Unclosed '{' on line 192 does not match ')'"

Note before it was complaining about CurlMultiHandler.php. Now it's StreamHandler.php. OK. I'll play yer silly game. I stuck a try / catch around that code, and re-ran. Now it errors with this:

an uncaught exception has been thrown in `vendor/guzzlehttp/promises/src/Promise.php` line 151

message:`ParseError` Code(0) with message "Unclosed '{' on line 148 does not match ')'"

Grrr. OK, one last time with the try / catch:

            _     _
  /\ /\__ _| |__ | | __ _ _ __
 / //_/ _` | '_ \| |/ _` | '_ \
/ __ \ (_| | | | | | (_| | | | |
\/  \/\__,_|_| |_|_|\__,_|_| |_|

The PHP Test Framework for Freedom, Truth and Justice.

src directory  :
spec directory : /mnt/c/src/kahlanIssue/spec

...                                                                 3 / 3 (100%)



Expectations   : 3 Executed
Specifications : 0 Pending, 0 Excluded, 0 Skipped

Passed 3 of 3 PASS in 0.565 seconds (using 7MB)

Um. OK. Why's it working? Next I backed-out all my debug code and ran it again. And the tests worked still. Next I blew-away the entire vendor directory and re-did composer install, and the tests continued to work.

It's worth noting here that I know those are all Guzzle files, but If I run the equivalent code in PHPUnit, there is never any issue, hence me doubting it's anything to do with Guzzle specifically, it's how Kahlan is calling Guzzle. Probably with vendor/kahlan/kahlan/src/Jit/ClassLoader.php, which is the preceding file in the callstack, each time. I have to admit I did not look into that file to see what it is doing. I'll crack on with that shortly.

At this point I was thinking that it might be something to do with me running all this in a Docker container, and decided I should verify this in the native file system. At the same time I decided to create a simple repro case: my initial experiments at this stage had not been with the example spec above, they were with one of the fullstack-exercise test specs, and those required the rest of the Docker containers to be up and running, so too complicated to be portable (see "Short, Self Contained, Correct (Compilable), Example"). Accordingly I created the spec above, in a stand-alone application (as per the kahlan-issue repo on Github).

I sighed and installed PHP 8 and Composer on my PC - something I was hoping to avoid needing to do every again, now that I'm using Docker - and ran the test again. Again PHPUnit ran fine, but Kahlan errored-out. The act of putting debug code into the files erroring was enough to make that file now parse and compile, and the error moved on to the next file. Always the same three files. After touching the files once, the problem stayed away. Blowing away vendor and re-doing composer install did not make the erroring situation recur. However if I stuck the app in a different base directory - eg: I had it set up in C:\src\kahlanIssue initially, but moved it to C:\temp\kahlanIssue - then the error recurred again, until I tweakd the files. I noted that putting just whitespace into the file did not "fix" the issue; it needed to be something that PHP would need to parse. Even if I restarted the PC, once I had "touched" those three files once, even if I reverted the touches, the problem did not recur. That was weird.

I was also able to replicate the issue in all of:

  • on Ubuntu within a Docker container (host machine is a Windows PC);
  • on Ubuntu via WSL on that same Windows PC;
  • natively on that Windows PC;
  • natively on Windows on a different PC.

It was only when I downgraded from PHP 8.0.2 to 7.4.15 that I could no longer replicate the issue. So I guess something odd Kahlan is doing to load the PHP files worked fine in previous versions of PHP, but PHP 8 doesn't like it. Or something like that.

I think now I have enough to go on to raise an issue with the Kahlan bods, which I shall do now… done: 370.

All that kinda killed my Sunday afternoon (and now Monday morning writing this article). Hopefully I can get some actual work done this evening though…

Status update on this

The bod from the Kahlan project (specifically Simon Jaillet) jumped on this straight away. In the mean time I looked at what was going on in that vendor/kahlan/kahlan/src/Jit/ClassLoader.php file I mentioned earlier, which was the last touch point in Kahlan before the error. Part of that loading process was to take the original files, monkey-patch the hell out of them to make mocking easier, and then load the monkey-patched version of the file instead of the original one. It also cached the monkey-patched version so it only had to do that exercise once. When it was monkey-patching the erroring files on PHP8, it was ending up with this sort of thing. Original code:

// Step through the task queue which may add additional requests.
P\Utils::queue()->run();

And patching it to be this:

// Step through the task queue which may add additional requests.($__KMONKEY__396__?$__KMONKEY__396__:
$__KMONKEY__396::queue())->run();

Note how the first part of the patched code is stuck on the comment line, not the code line. It should be like this:

// Step through the task queue which may add additional requests.
($__KMONKEY__396__?$__KMONKEY__396__:$__KMONKEY__396::queue())->run();

And when I ran Kahlan on PHP 7.4, it was patching it correctly (as per the last block, above).

The bloke from Kahlan located the issue, which is down to a change in how PHP8 tokenises comments (see "Backward Incompatible Changes > Tokenizer":

T_COMMENT tokens will no longer include a trailing newline. The newline will instead be part of a following T_WHITESPACE token.

That explains that.

I raised the issue at 13:27, and it was fixed by 15:16. I've updated my Kahlan version, and it does indeed sort-out the issue. Brilliant work!

Righto.

--
Adam