Sunday 17 March 2024

CFML: solving a CFML problem with Java. Kinda.

G'day:

The other day on the CMFL Slack channel, Nick Petrie asked:

Anyone know of a native or plugin-based solution to pretty-formatting XML for display on the page? I'd like to output the XML nested this:
I'll use a simplified example> we wanna render this:
<aaa><bbb ccc="ddd"><eee/></bbb></aaa>
Like this:
<aaa>
    <bbb ccc="ddd">
        <eee/>
    </bbb>
</aaa

There's no native CFML way of doing this, but I figured "this is a solved problem: there'll be an easy way to do it in Java". I googled java prettier xml, and the first match was on the Baeldung website (Pretty-Print XML in Java) which is a site I trust to have good answers. But I checked that and a few others, and they all seem to be solving the problem the same way. So I decided to run with that.

The task at hand is to convert this to CFML (still using the Java libs to do the work, I mean, just runnable in CFML):

public static String prettyPrintByTransformer(String xmlString, int indent, boolean ignoreDeclaration) {

    try {
        InputSource src = new InputSource(new StringReader(xmlString));
        Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(src);

        TransformerFactory transformerFactory = TransformerFactory.newInstance();
        transformerFactory.setAttribute("indent-number", indent);
        Transformer transformer = transformerFactory.newTransformer();
        transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
        transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
        transformer.setOutputProperty(OutputKeys.INDENT, "yes");

        Writer out = new StringWriter();
        transformer.transform(new DOMSource(document), new StreamResult(out));
        return out.toString();
    } catch (Exception e) {
        throw new RuntimeException("Error occurs when pretty-printing xml:\n" + xmlString, e);
    }
}

To be very clear, this is not my code, it's from Pretty-Print XML in Java.

And also to be clear: I'm not gonna be doing anything unique or difficult or insightful or anything like that in this article. All I'm gonna do is to show how easy it is to convert native Java code to native CFML code, and this is a topical use case. It's gonna be a function with some object creation and some variable assigments. I'm gonna go line-by-line through that function above, and CFMLerise it. I just made that word up.

I'm gonna be using trycf.com to write and run this code, and the aim is to have a version that runs in both CF and Lucee.

First, let's run with the same function signature:

unformattedXml = '<aaa><bbb ccc="ddd"><eee/></bbb></aaa>' 

public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {

}

prettyPrintByTransformer(unformattedXml)

In each step I'll be using that same unformattedXml input value, and just running the function to ensure it's got no compilation errors and each new statement I add "works" (in that it doesn't have runtime errors). The function won't do anything useful until it's done.

In this first step note that I've given the latter two parameters sensible defaults. This is a change from the Java version.

First things first: I don't like how they've put that largely pointless try/catch in the Java code. To me that sort of error-handling should be in the calling code if needed, not embedded in the implementation. If the implementation errors-out: let it. The actual exception will be more useful than swallowing the real exception and throwing a contrived one.

I'll include each statement I'm converting as a comment above the CFML version. This is only so I can draw your focus to it. I'd never include comments of this nature in my actual code.

public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
    // InputSource src = new InputSource(new StringReader(xmlString));
    var xmlAsStringReader = createObject("java", "java.io.StringReader").init(xmlString) // new StringReader(xmlString)
    var src = createObject("java", "org.xml.sax.InputSource").init(xmlAsStringReader)
}

I could have done this without the intermediary variable, but CFML is quite cumbersome making Java object proxies, so I think the code is clearer spread over two statements.

    // Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(src);
    var document = createObject("java", "javax.xml.parsers.DocumentBuilderFactory").newInstance().newDocumentBuilder().parse(src)
    // TransformerFactory transformerFactory = TransformerFactory.newInstance();
    var transformerFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance()

Now: this statement did not work for me:

transformerFactory.setAttribute("indent-number", indent);

The CFML equivalent is exactly the same as it happens, but I was just getting "Not supported: indent-number" (CF) or "TransformerFactory does not recognise attribute 'indent-number'." (Lucee). I presume it's a library version difference, but I was fairly limited in my troubleshooting as I was running this on trycf.com. Although I did verify I was getting the same thing on my local CF2023 container too. I googled a bit and found a work around, but I'll come back to this a bit further down.

    // Transformer transformer = transformerFactory.newTransformer();
    var transformer = transformerFactory.newTransformer()
    /*
        transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
        transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
        transformer.setOutputProperty(OutputKeys.INDENT, "yes");
    */
    var outputKeys = createObject("java", "javax.xml.transform.OutputKeys")
    transformer.setOutputProperty(outputKeys.ENCODING, "UTF-8")
    transformer.setOutputProperty(outputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no")
    transformer.setOutputProperty(outputKeys.INDENT, "yes")
    transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", indent)

Once again, I need an intermediary variable here for the outputKeys class proxy. Just to save repetition in this case.

Also note the last line there is extra: it's the solution for the bit that wasn't working earlier. Easy.

    // Writer out = new StringWriter();
    var out = createObject("java", "java.io.StringWriter").init()
    // transformer.transform(new DOMSource(document), new StreamResult(out));
    var documentAsDomSource = createObject("java", "javax.xml.transform.dom.DOMSource").init(document)
    var outAsStreamResult = createObject("java", "javax.xml.transform.stream.StreamResult").init(out)
    transformer.transform(documentAsDomSource, outAsStreamResult)

More intermediary variables here.

    // return out.toString();
    return out.toString()

And that's it. I've not got much commentary in all that lot, because it's all so straight forward [shrug].

The end result in one piece is thus:

public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
    var xmlAsStringReader = createObject("java", "java.io.StringReader").init(xmlString)
    var src = createObject("java", "org.xml.sax.InputSource").init(xmlAsStringReader)
    
    var document = createObject("java", "javax.xml.parsers.DocumentBuilderFactory").newInstance().newDocumentBuilder().parse(src)
    
    var transformerFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance()
    var transformer = transformerFactory.newTransformer()
    
    var outputKeys = createObject("java", "javax.xml.transform.OutputKeys")
    transformer.setOutputProperty(outputKeys.ENCODING, "UTF-8")
    transformer.setOutputProperty(outputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
    transformer.setOutputProperty(outputKeys.INDENT, "yes")
    transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", indent)
    
    var out = createObject("java", "java.io.StringWriter").init()
    
    var documentAsDomSource = createObject("java", "javax.xml.transform.dom.DOMSource").init(document)
    var outAsStreamResult = createObject("java", "javax.xml.transform.stream.StreamResult").init(out)
    transformer.transform(documentAsDomSource, outAsStreamResult)

    return out.toString()
}

And a coupla test runs:

formattedXml = prettyPrintByTransformer(unformattedXml)
writeOutput("<pre>#encodeForHtml(formattedXml)#</pre>")
<aaa>
    <bbb ccc="ddd">
        <eee/>
    </bbb>
</aaa>
formattedXml = prettyPrintByTransformer(unformattedXml, 8, false)
writeOutput("<pre>#encodeForHtml(formattedXml)#</pre>")
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<aaa>
        <bbb ccc="ddd">
                <eee/>
        </bbb>
</aaa>

The output above is from Lucee. On CF we get this:

<?xml version="1.0" encoding="UTF-8"?><aaa>
        <bbb ccc="ddd">
                <eee/>
        </bbb>
</aaa>

Note this is nothing to do with the CFML code, it'll be some Java library variation on the Lucee and CF set-ups on trycf.com.

Right, so all this lot shows is that there's no reason to shy away from implementing a CFML version of some Java code you might find that solves a problem. So in turn, if you have some "algorithmic" issue that you wanna solve in CFML, don't shy away from googling for Java solutions, and checking how easy/hard it is to convert.

A runnable version of this code is @ trycf.com.

Righto.

--
Adam

Friday 27 October 2023

I'm a big meany again

G'day

Chortle:

I'm new to cf development and recently joined the cfml slack community. I noticed some curmudgeonly posts from this fella at first and didn't think anything of it... I'm now realizing that he is an a**h*** to people all the time. For example, there was a question posted today that currently has 14 replies. A few of them are from others trying to answer OP's question; 7 are this guy explaining how stupid of a question it is. What gives? I'm sure he's a knowledgeable and respected developer, but the pedantry is off the charts!

DJgrassyknoll on Reddit

Guilty as charged I suppose.

However the charges are pretty specious. I think the thread in question was the one with a question (paraphrase) "is there a function in CFML that can convert JSON to XML?". This is a largely non-sensical question, and without further elaboration, can only be furnished with non-sensical and unhelpful answers. I don't wanna do that. I wanna get the person's problem solved. I did point this out to the bod in question, and explained why, with both a real-world analogy, plus a code demonstration! Here's the code version:

json = serializeJson({key="value"})
xml = xmlParse("<nowItIsXml>#encodeForXml(json)#</nowItIsXml>")

writeDump([
    json = json,
    xml = xml
])

Completely answers the question as stated, but is a completely frickin useless answer. Other than to illustrate the shortcomings of the question.

I've been answering people's CFML questions for years, and the biggest blocker to actually solving their problem is coercing what their actual problem is out from their brain. Really a lot of people don't seem to get we can't see the code in front of them, or the Jira ticket (sic) they're trying to deal with, or any other thoughts on the matter they might have had beyond what they type in as their question. So… not to be put off… I will explain this, and try to get a decent question out of them. So we all can understand it and then get their issue sorted. I will also often post links to How To Ask Questions The Smart Way and/or The SSCCE. I'm not going to apologise for that. It's good advice and if more people took it on board, they would improve, and also their chances of getting good help when they need it would improve. How is that bad?

We're all busy people, so the quicker and easier it is for us to get the info we need in front of us to help: the more likely we are to help. As one of the participants in that thread said "[when] people ask shitty questions, I mostly will just ignore it". That's your call pal. But I won't ignore them. I'll still try to help.

Ah - fuck it - people can say what they like about me. I don't care (and hey it's given me an in to blow some of the dust off this thing). But try to have a well-formed train of thought before doing so. And also poss sink some time into helping people to understand what it's like before stirring the pot in yer little sewing circle.

But yeah cheers for the new strapline quote.

Righto.

--
Adam

PS: props for using "curmudgeonly" though. Good word.

Monday 1 May 2023

PHP / Symfony: working through "Symfony: The Fast Track", part 5: Twig stuff, and irritation

G'day:

I'm back on the case working through "Symfony: The Fast Track". This will be part five of this series, after the first four:

The page I'm starting on is Building the User Interface, which seems more interesting than the previous section. I also see mention of my old friend Twig in the first subsection of this, so it will be good to look at that some more and see how it's moved along since I last did PHP UI stuff about 7-8 years ago (god is it that long?).


Using Twig for the Templates

The first thing I need to do is to restore a twig file I deleted during the last exercise, thinking it was redundant. So: back comes templates/base.html.twig. I'll be referring back to this, so will dump it out here, even though it's boilerplate, and not my code:

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>{% block title %}Welcome!{% endblock %}</title>
    <link
    	rel="icon"
        href="
            data:image/svg+xml,
            <svg
            	xmlns=%22http://www.w3.org/2000/svg%22
                viewBox=%220 0 128 128%22
            >
                <text y=%221.2em%22 font-size=%2296%22>⚫️</text>
            </svg>
        "
    >
    {# Run `composer require symfony/webpack-encore-bundle` to start using Symfony UX #}
    {% block stylesheets %}
        {{ encore_entry_link_tags('app') }}
    {% endblock %}

    {% block javascripts %}
        {{ encore_entry_script_tags('app') }}
    {% endblock %}
</head>
<body>
{% block body %}{% endblock %}
</body>
</html>

And one for templates/conference/index.html.twig:

{% extends 'base.html.twig' %}

{% block title %}Conference Guestbook{% endblock %}

{% block body %}
    <h2>Give your feedback!</h2>

    {% for conference in conferences %}
        <h4>{{ conference }}</h4>
    {% endfor %}
{% endblock %}

Note that it inherits from the base one, and overrides its title and body blocks. I'll not discuss all the syntax vagaries as they're well documented: Twig Documentation. It's easy enough to read and infer what's going on, anyhow.

The next section is entitled "Using Twig in a Controller" and I panicked as I thought they meant they were gonna put Twig code into a controller method (remember they've been a bit woolly about what goes in a controller already, so it would not have surprised me), but fortunately no. It's just describing how to use the twig file:

public function index(Environment $twig, ConferenceRepository $conferenceRepository): Response
{
    // return of inline HTML (!!) removed

    return new Response($twig->render(
        "conference/index.html.twig",
        ["conferences" => $conferenceRepository->findAll()]
    ));
}

The conferences there are then used in the loop in the twig.


Creating the Page for a Conference

OK so here's a cool thing Symfony does. I am creating the page for the route /conference/{id}, and if my controller method takes a Conference object, Symfony is clever enough to go "I see what yer doing there" and goes and gets the conference for that ID. EG, I've added this routing to get to my controller method:

config/routes/conference.yaml

# …

show:
    path: /{id}
    methods: [ GET ]
    controller: adamcameron\symfonythefasttrack\Controller\ConferenceController::show

And the controller method the exercise has me write is this (nothing noteworthy really):

public function show(Environment $twig, Conference $conference, CommentRepository $commentRepository): Response
{
    return new Response($twig->render(
        "conference/show.html.twig",
        [
            "conference" => $conference,
            "comments" => $commentRepository->findBy(
                ["conference" => $conference],
                ["createdAt" => "DESC"]
            )
        ]
    ));
}

The routing takes an ID; the controller receives the equivalent Conference object.

Note that whilst it's neat that Symfony does this, that controller they've had me write is borderline not great MVC in my books. I don't really think controllers should be aware of things like "repositories". Repositories are a storage consideration, and should be hidden away with the model. What should be happening here I think is that the controller receives its Conference, but the Conference should implement whatever is necessary to get its Comment collection from wherever it put it (and via whatever mechanism it did it with). This is not the controller's job. This is a good example of how using generic ORM crap leads to subpar code. I realise this is a tutorial about using Twig, but I think on the whole this book is encouraging some pretty poor design decisions because that better shows-off Symfony's bells and whistles.

Ahem. Moving along…

For reference here's the twig template for the view:

{% extends 'base.html.twig' %}

{% block title %}Conference Guestbook - {{ conference }}{% endblock %}

{% block body %}
    <h2>{{ conference }} Conference</h2>

    {% if comments|length > 0 %}
        {% for comment in comments %}
            {% if comment.photofilename %}
                <img src="{{ asset('uploads/photos/' ~ comment.photofilename) }}"/>
            {% endif %}

            <h4>{{ comment.author }}</h4>
            <small>
                {{ comment.createdAt|format_datetime('medium', 'short') }}
            </small>

            <p>{{ comment.text }}</p>
        {% endfor %}
    {% else %}
        <div>No comments have been posted yet for this conference.</div>
    {% endif %}
{% endblock %}

And the test shows the initial part of the template implementation presents the detail from the correct conference correctly:

/** @testdox /conference/{id} end point returns a 200 OK for a valid conference */
public function testShow()
{
    $kernel = new Kernel("test", true);
    $kernel->boot();

    $container = $kernel->getContainer();

    $connection = $container->get("database_connection");
    $testConference = $connection->executeQuery("
        SELECT id, city, year
        FROM conference LIMIT 1
    ")->fetchAssociative();
    if (!$testConference) {
        $this->markTestSkipped("No conferences in database, test aborted");
    }

    $client = static::createClient();
    $client->request('GET', '/conference/' . $testConference['id']);

    $this->assertResponseIsSuccessful();

    $this->assertSelectorTextContains(
        'title',
        sprintf(
            "Conference Guestbook - %s %s",
            $testConference['city'],
            $testConference['year']
        )
    );
}

I presumed it would 404 if the {id} requested wasn't in the DB:

/** @testdocs /conference/{id} end point returns a 404 if the conference is not found */
public function testShowNotFound()
{
    $client = static::createClient();
    $client->request('GET', '/conference/999999');

    $this->assertResponseStatusCodeSame(Response::HTTP_NOT_FOUND);
}

And this passes. NB: the 404 page is a general Symfony one, but that's fine. It's handling it correctly.

It's important to note that I have not written any code relating to Conferences or the storage thereof in this chapter of the exercise. I'm simply using the stuff Symfony generated for me during the previous exercise.

Oh one additional thing I did need to do was to install the twig/intl-extra dependency, because I'm using that format_datetime Twig filter. How did I know I needed to do this? Symfony told me (well, OK, I have also just scrolled ahead in the lesson, and it also told me there):

That's quite cool. I note it's claiming it's a syntax error though, which it is not. If it was a syntax error, the code wouldn't compile, let alone run to the point to which it works out I have a Twig filter missing, and which one it is and where to get it from. It's a MissingTwigExtensionError or something. But a runtime error of some sort, not a compile-time one. Still: semantics aside, it's helpful. It does seem a bit odd to me though that something in either Twig or Symfony clearly knows what all the extensions are and what external libs they're in. This seems a bit of an inappropriate overreach to me, and a bit of a break of the Open-Closed Principle. Still: productivity over dogma, eh?

Oh! So far I have no comments in the DB, so all that Twig bit about rendering them doesn't do anything. All I'm getting is this:

Time to test the comments bit, before I move on.

[… more time passes than I would like to admit…]

I went round the houses with this a bit. I had to install a bunch of Node.js stuff in my Node container because one of the Twig function's internal workings checked for [something] and if it didn't exist: threw a PHP exception. I didn't preserve the error, but this code (in show.html.twig) was the culprit:

<img src="{{ asset('uploads/photos/' ~ comment.photofilename) }}"/>

The asset function was complaining that public/build/entrypoints.json was missing. I googled about and found out I needed to have symfony/webpack-encore-bundle installed, the instructions for which (that previous link) seem to not be as complete as they could be for codebases not using Symfony Flex. Still: the residual issues I had were all fairly googleable, and easily fixed. Sorry to be a bit vague there, but I did this lot on Fri/Sat - it's now Sun - and I can't be arsed digging through my browser history to recreate the - fairly unrewarding - sequence of hurdles I bumbled over to get my PHP app to stop erroring because I didn't have some webpack shit configured properly.

BTW: this is another thing I don't think a server-side MVC framework should be investing its time in. There's plenty of client-side frameworks for handling client-side stuff. Server-side frameworks should just busy themselves with their own job. Seems to me like another case of "if your only tool is a hammer, every problem starts looking like a nail".

Anyway, my test stopped erroring, and after some small effort I got it to go green. I actually ended up with two tests here:

/** @testdox /conference/{id} also displays any comment associated with the conference */
public function testShowComments()
{
    $testConference = $this->getConferenceToTestWith();

    $client = static::createClient();
    $client->request("GET", "/conference/" . $testConference->getId());

    $this->assertResponseIsSuccessful();
    $this->assertCommentersAreCorrect($client, $testConference);
}

private function getConferenceToTestWith(): Conference
{
    $container = Container::getContainer();

    $repo = $container->get("testing.ConferenceRepository");
    $qb = $repo->createQueryBuilder("cf");

    $qb->select("cf")
        ->setMaxResults(1)
        ->leftJoin("cf.comments", "cm")
        ->groupBy("cf.id")
        ->having("count(cm.id) > 1");
    $testConference = $qb->getQuery()->getResult();

    if (!$testConference) {
        $this->markTestSkipped("No suitable conferences in database, test aborted");
    }

    return $testConference[0];
}

private function getExpectedCommenters(Conference $testConference): array
{
    $expectedCommenters = $testConference->getComments()->toArray();
    usort($expectedCommenters, function ($e1, $e2) {
        return $e2->getCreatedAt() <=> $e1->getCreatedAt();
    });
    $expectedCommenters = array_map(function ($comment) {
        return $comment->getAuthor();
    }, $expectedCommenters);
    return $expectedCommenters;
}

private function assertCommentersAreCorrect(KernelBrowser $client, Conference $testConference): void
{
    $commenters = $client
        ->getCrawler()
        ->filter("body > h4")
        ->each(function ($node) {
            return $node->text();
        });

    $expectedCommenters = $this->getExpectedCommenters($testConference);
    $this->assertEquals($expectedCommenters, $commenters);
}

This one goes right through from the controller to the DB and back: comparing the data from the DB (expected) matching the data returned in the mark-up (actual).

I don't wanna be hitting the DB in my tests as a rule, especially if it's only a mechanism to get data to then see what the logic does with it. If I'm testing the logic, I want the expediency of using canned data that best exercises the logic. I tried to write a unit test of this, but it proved to be more difficult than I had patience for. The problem being that all the "logic" is in the controller method:

public function show(
    Environment $twig,
    Conference $conference,
    CommentRepository $commentRepository
): Response {
    return new Response($twig->render(
        "conference/show.html.twig",
        [
            "conference" => $conference,
            "comments" => $commentRepository->findBy(
                ["conference" => $conference],
                ["createdAt" => "DESC"]
            )
        ]
    ));
}

So to unit test that, I need to be able to mock the collaborators. Mocking my ones - the Conference and the CommentsRepository - was easy. Dealing with the Twig Environment was beyond my patience. And I wasn't even trying to do anything fancy with it: all I wanted was to spy on it so I could test the values exiting my test unit. But Symfony is very very convoluted in how it configures things, and I could not work out a minimum Environment that would actually initialise without erroring. Please note that I was very aware of "don't mock what you don't own" during all of this, and beyond a point I had decided that this was the wrong approach as well as being a fool's errand.

In the end I wrote a more functional test. I stuck with the Environment that Symfony had configured in its DI container, and the checked the results of its rendering. This is not so bad: it's all just done in code and no hits to external services (the DB), but it resulted in more code than I'm really happy with. There's no need to pore over this: I'm not gonna discuss it further. I only included to show the amount of effort involved.

/** @testdox /conference/{id} displays correct conference and comments */
public function testShow()
{
    $conferenceTitle = "MOCKED_CONFERENCE";
    list($twigEnvironment, $conference, $commentRepository) = $this->getTestDependencies($conferenceTitle);
    $controller = new ConferenceController();

    $result = $controller->show($twigEnvironment, $conference, $commentRepository);

    $this->assertResponseIsCorrect($result, $conferenceTitle, $commentRepository);
}

private function getTestDependencies(string $conferenceTitle): array
{
    $twigEnvironment = self::getContainer()->get(TwigEnvironment::class);

    $conference = $this->getMockedConference($conferenceTitle);

    $commentRepository = $this->getMockedCommentRepository();
    return array($twigEnvironment, $conference, $commentRepository);
}

private function getMockedConference(string $conferenceTitle): Conference
{
    $conference = $this->getMockBuilder(Conference::class)
        ->disableOriginalConstructor()
        ->getMock();
    $conference
        ->expects($this->any())
        ->method("__toString")
        ->willReturn($conferenceTitle);
    return $conference;
}

private function getMockedCommentRepository(): CommentRepository
{
    $commentCount = 3;
    $commentsToTestFor = range(1, $commentCount);

    $testComments = array_map(function ($i) {
        return (new Comment())
            ->setAuthor("COMMENTER" . $i)
            ->setText("COMMENT" . $i)
            ->setCreatedAt(new \DateTimeImmutable("2020-01-0{$i}"));
    }, $commentsToTestFor);

    $commentRepository = $this->getMockBuilder(CommentRepository::class)
        ->disableOriginalConstructor()
        ->getMock();
    $commentRepository
        ->method("findBy")
        ->willReturn($testComments);
    return $commentRepository;
}

private function assertResponseIsCorrect(Response $result, string $conferenceTitle, mixed $commentRepository): void
{
    $this->assertEquals(Response::HTTP_OK, $result->getStatusCode());

    $xpathDocument = $this->getContextAsXpathDocument($result);
    $this->assertTitleIsCorrect($xpathDocument);
    $this->assertSubheadingIsCorrect($xpathDocument, $conferenceTitle);
    $this->assertCommentsAreCorrect($xpathDocument, $commentRepository);
}

private function getContextAsXpathDocument(Response $result): DOMXPath
{
    $content = $result->getContent();
    $document = new DOMDocument();
    $document->loadHTML($content, LIBXML_NOWARNING | LIBXML_NOERROR);
    $xpathDocument = new DOMXPath($document);
    return $xpathDocument;
}

private function assertTitleIsCorrect(DOMXPath $xpathDocument): void
{
    $title = $xpathDocument->query('/html/head/title[text()]');
    $this->assertCount(1, $title);
    $this->assertEquals("Conference Guestbook - MOCKED_CONFERENCE", $title->item(0)->textContent);
}

private function assertSubheadingIsCorrect(DOMXPath $xpathDocument, string $conferenceTitle): void
{
    $h2 = $xpathDocument->query('/html/body/h2[text()]');
    $this->assertCount(1, $h2);
    $this->assertEquals("$conferenceTitle Conference", $h2->item(0)->textContent);
}

private function assertCommentsAreCorrect(DOMXPath $xpathDocument, CommentRepository $commentRepository): void
{
    $testComments = $commentRepository->findBy([], []);
    $commentCount = count($testComments);

    $commentAuthors = $xpathDocument->query('/html/body/h4[text()]');
    $this->assertCount($commentCount, $commentAuthors);
    $commentTexts = $xpathDocument->query('/html/body/p[text()]');
    $this->assertCount($commentCount, $commentTexts);
    $commentDates = $xpathDocument->query('/html/body/small[text()]');
    $this->assertCount($commentCount, $commentDates);

    array_walk($testComments, function ($comment, $i) use ($commentAuthors, $commentTexts, $commentDates) {
        $this->assertEquals($comment->getAuthor(), $commentAuthors->item($i)->textContent);
        $this->assertEquals($comment->getText(), $commentTexts->item($i)->textContent);

        $commentCreatedAtFormatted = $comment->getCreatedAt()->format("M j, Y, h:i A");
        $this->assertEquals($commentCreatedAtFormatted, trim($commentDates->item($i)->textContent));
    });
}

There's a moral to all this. "Don't mock what you don't own" issues aside, testing code that is tightly-coupled to the framework - like a controller method pretty much is - is gonna have overhead. As such: keep as little code in yer controllers as possible: have yer business logic within yer model, because you "own" this, and there will be less over-engineers framework code getting in the way.


Linking Pages Together & Pagination & some refactoring

Next they've got some more Twig advice about using its methods to build predictable URLs insted of hard-coding them, eg:

<a href="/conference/{{ conference.id }}">View</a>
<a href="{{ path('conference', { id: conference.id }) }}">View</a>

Good advice. I'm not gonna dwell on it further than that, other than sticking this code into my Twig template (after I test for it, obvs). I'll spare you the code, but it'll be in source control if you decide you do want to look at it. I got slightly tripped-up there as I had changed my route's name to be conference_show, and that reference to conference is the route's name. Easily fixed though.

Next it shows how to paginate record sets in the UI, which is pleasingly straight forward. A method in the repository:


public const PAGINATOR_PER_PAGE = 2;

// …

public function getPaginator(Conference $conference, int $offset): Paginator
{
    $query = $this->createQueryBuilder('c')
        ->andWhere('c.conference = :conference')
        ->setParameter('conference', $conference)
        ->orderBy('c.createdAt', 'DESC')
        ->setMaxResults(self::PAGINATOR_PER_PAGE)
        ->setFirstResult($offset)
        ->getQuery();

    return new Paginator($query);
}

Pass a paginator instead of the articles from the controller:

public function show(
    Request $request,
    Environment $twig,
    Conference $conference,
    CommentRepository $commentRepository
): Response {
    $offset = max(0, $request->query->getInt('offset', 0));
    $paginator = $commentRepository->getPaginator($conference, $offset);

    return new Response($twig->render(
        "conference/show.html.twig",
        [
            "conference" => $conference,
            "comments" => $commentRepository->findBy(["conference" => $conference], ["createdAt" => 'DESC']),
            "comments" => $paginator,
            'previous' => $offset - CommentRepository::PAGINATOR_PER_PAGE,
            'next' => min(count($paginator), $offset + CommentRepository::PAGINATOR_PER_PAGE)
        ]
    ));
}

And then add the pagination stuff to the view:

<div>There are {{ comments|length }} comments.</div>
{% for comment in comments %}
    {% if comment.photofilename %}
        <img src="{{ asset('uploads/photos/' ~ comment.photofilename) }}"/>
    {% endif %}

    <h4>{{ comment.author }}</h4>
    <small>
        {{ comment.createdAt|format_datetime('medium', 'short') }}
    </small>

    <p>{{ comment.text }}</p>
{% endfor %}

{% if previous >= 0 %}
    <a href="{{ path('conference_show', { id: conference.id, offset: previous }) }}">Previous</a>
{% endif %}
{% if next < comments|length %}
    <a href="{{ path('conference_show', { id: conference.id, offset: next }) }}">Next</a>
{% endif %}

The only change I made to this was that the suggested name for the pagination method was getCommentPaginator, but given it was in the CommentRepository, I thought repeating "comment" there was tautological, so changed it to getPaginator.

Oh, the page now looks like this:

Cool.

One thing to note here is that in refactoring my tests of the ConferenceController show method to accommodate the pagination, I pushed the coupling of the test class over the top of what PHPMD was happy with (the threshold is the default 13 classes), so I refactored that into three classes: Test, Assertions and Dependencies.

tests
`-- php
    `-- Functional
        `-- Controller
            `-- ConferenceController
                |-- Index
                |   `-- Test.php
                `-- Show
                    |-- Assertions.php
                    |-- Dependencies.php
                    `-- Test.php

I think this is all caused by testing code in a controller; having to mess around with Requests and Environments and having to test for rendered mark-up instead of just values-in and value-out adds "unnecessary" overhead. I am realy irked (in a laughing way) by what I just read in the next section which is the refactoring bit…

You might have noticed that both methods in ConferenceController take a Twig environment as an argument. Instead of injecting it into each method, let's leverage the render() helper method provided by the parent class

Symfony the Fast Track › Building the User Interface › Refactoring the Controller

Oh FFS. Ha. OK. Fine. Tell me that now.

So I can simplify things a bit:

public function show(
    Request $request,
    Environment $twig,
    Conference $conference,
    CommentRepository $commentRepository
): Response {
    $offset = max(0, $request->query->getInt('offset', 0));
    $paginator = $commentRepository->getPaginator($conference, $offset);

    return new Response($twig->render('conference/index.html.twig', [
    return $this->render(
        "conference/show.html.twig",
        [
            "conference" => $conference,
            "comments" => $paginator,
            'previous' => $offset - CommentRepository::PAGINATOR_PER_PAGE,
            'next' => min(count($paginator), $offset + CommentRepository::PAGINATOR_PER_PAGE)
        ]
    );
    ]));
}

That's good, but I really don't know why they didn't lead with that. There was nothing that relied on the Twig Environment in those methods, so why pass it in? Just to give an extra paragraph at the end?

Oh I also had to tweak a couple of the tests that broke because of this: for that render method to be available, I needed to initialise the controller class with the DI container. This was easy though:

$controller = new ConferenceController();
$container = self::getContainer();
$controller->setContainer($container);

End of section / article

Man. I only got through one page of the docs with this article, and it also took me four bites to write, even though it didn't really cover much ground. I think the issue is that a) I lack self-discipline; b) I'm not finding this stuff particularly compelling. I also got deeply frustrated with the amount of dicking around I had to do to try to convince Symfony to let me at its dependencies when I was trying to test stuff, and also the waste of time that was getting Encore working, because - as I said earlier - client-side libs are nothing to do with a server-side MVC framework, and should not get in the way. The more I use Symfony, the more I miss Silex.

I'm not looking forward to the re-drafting & read-throughs ofthis before I press "send", as I suspect it's a bit of a half-hearted mess. But I've come this far, so I'm buggered if I'm not publishing it. Well done if you read this far.

Righto.

--
Adam

Monday 17 April 2023

CFML: Into the Box 2023

Howdy partner:

Well here's something interesting (well: to me, anyhow). I will be attending the Into the Box conference in Houston, TX in May (17th - 19th). This has come about due to a very kind gesture from Luis / Ortus Solutions. Thank-you so much for this!

I know nothing about Houston - I have been through the airport on the way to somewhere else a coupla times, and that's it - so am getting there a coupla days early to have a look around and make sure the local craft beer bars are patronised.

The schedule of Into the Box itself looks pretty good too, and cover a lot of bases other than just the CFML / Boxiverse side of things. I see "testing" and "TDD" mentioned a coupla times in the schedule, so will be in my element.

The best thing about conferences though is to get to put faces to names, and meet people properly. I might be a surly anti-social type, but I really value the friendships I have built from meeting people at industry conferences, and having an agendaless chat with them over a beer or a watercooler, and got to know them better. I do hope to catch up with a bunch of people from the community who I know only as words appearing in a chat app / forum / etc. And hopefully re-meet some familiar faces too.

And the proof is in the pudding on this "conference socialising" thing: after saying farewell to Houston on the Sunday after the conference, I'm heading over to North Carolina for a few days to hang out with Dan & Jennette Skaggs (and family). I first met D & J @ CF.Obective() (or maybe Dev.Objective()?) many years ago, and we've become really good friends since then. And having had them come visit me on my side of the pond a coupla times; it's my turn to come see their part of the world. I can't wait.

So come and say "G'day" if you see me around the conference, and let's talk about... whatevs. And maybe let's go for a beer :-D

Y'all come back now, hear? (*)

--
Adam


(*) yeah, all right, enough of that.

Tuesday 11 April 2023

Getting Windows Terminal to open my Ubuntu Bash session in the right directory

G'day:

This is a follow on from the previous two articles:

All that is working well, but I had a wee problem with switching my Windows Terminal config from starting Git Bash shells to do the equivalent with directly running a Bash shell on my WSL Ubuntu filesystem.

For Git Bash, I have profiles set up like this:

{
    "commandline": "C:\\apps\\Git\\git-bash.exe",
    "guid": "{be9a184b-1c89-4ac5-88da-3ef93cd5ec98}",
    "hidden": false,
    "icon": "C:\\apps\\Git\\mingw64\\share\\git\\git-for-windows.ico",
    "name": "Git Bash (SymfonyFastTrack)",
    "startingDirectory": "\\\\wsl$\\Ubuntu\\home\\adam\\src\SymfonyFastTrack",
    "tabTitle": "Git Bash (SymfonyFastTrack)"
}

And when I open a terminal with that profile I get:

adam@DESKTOP MINGW64 //wsl$/ubuntu/home/adam/src/SymfonyFastTrack (main)
$

I figured it would be easy with an Ubuntu one, I'd copy the default one:

{
    "guid": "{2c4de342-38b7-51cf-b940-2309a097f518}",
    "hidden": false,
    "name": "Ubuntu",
    "source": "Windows.Terminal.Wsl"
}

Change its GUID, give it a different name and a startingDirectory value and done. But no: one cannot have more than one profile for Windows.Terminal.Wsl it would seem: It didn't show up (it also didn't error, which Windows Terminal is pretty good at when it doesn't like something).

So now I have two issues: getting a second Ubuntu Bash shell profile working at all, and then: point it to the correct starting directory.

The first thing I found out is that in the current version of Windows Terminal, the startingDirectory is not honoured for Ubuntu anyhow. So that's a non-starter. I did a lot of googling, and that turned up nothing. Then I turned to a more powerful search engine: I Mingo-ed it. He didn't quite nail the solution, but he got me onto the right track: asking ChatGPT. I dunno why I didn't start there. After a bit of back and forth, ChatGPT and I came up with this:

{
    "commandline": "wsl.exe -d Ubuntu /bin/bash --rcfile <(echo \"source ~/.bashrc; cd ~/src/SymfonyFastTrack\")",
    "guid": "{3933fa46-657f-4db9-ad6a-2bee51554bc5}",
    "icon": "C:\\Users\\camer\\AppData\\Local\\wt\\bash.png",
    "name": "Bash (SymfonyFastTrack)",
    "tabTitle": "Bash (SymfonyFastTrack)"
}

Explanation:

  • wsl.exe -d Ubuntu is the long form of what I tried before with source:Windows.Terminal.Wsl; name: Ubuntu.
  • /bin/bash says to run Bash.
  • --rcfile says "using this RC file (eg: instead of .bashrc.
  • <(echo [etc]): instead of using an actual file, take it from stdout.
  • source ~/.bashrc: first my actual .bashrc.
  • cd ~/src/SymfonyFastTrack\: but then switch to this directory.

And when I use this profile, I get what I want:

adam@DESKTOP //wsl$/ubuntu/home/adam/src/SymfonyFastTrack (main) $

Cool. All done.

This was another article which is largely me fumbling around being a n00b, but we all start that way with things I guess, so maybe this will short-circuit all the goolging, mingoing and ChatGPTing I had to do to arrive at the solution.

And - seriously Mingo - cheers for helping with this.

Righto.

--
Adam

Sunday 9 April 2023

Changing my WSL Bash prompt to include my current Git branch

G'day:

Note: there's no real original research / thinking in this. it's just the result of me googling stuff, and arriving at a result. I'm writing it mostly so I have a record of it.

Yesterday I shifted my source code directories from my Windows file system (eg: C:\src\myapp) to using the WSL file system (eg: ~/src/myapp). See Changing where I home my source code dramatically speeds up my Windows / WSL2 / Docker environment.

Up until now I'd been doing my dev from C:, my Docker stuff in WSL, and my Git carry-on from Git Bash. Given my files are in the WSL file system now, and I have Git installed in there too, I figured I might as well ditch Git Bash and use Git directly via Bash.

The only thing I needed to do to shift to this work pattern is to update my shell prompt to reflect what branch I'm currently in, if it's a version-controlled directory.

EG, in Bash I see this:

adam@DESKTOP MINGW64 //wsl$/ubuntu/home/adam/src/SymfonyFastTrack (main)
$

But in Bash I just see this:

adam@DESKTOP //wsl$/ubuntu/home/adam/src/SymfonyFastTrack $

I don't need the MINGW64 in there (I don't even know what it means, nor have I ever had to care. Also: no need to pipe up and tell me), but I do need the branch to be shown the current branch I'm on if I'm in a source-controlled directory, like how Git Bash does it.

I googled about how to change the Bash prompt, and found a few helpful notes. Links to those @ the bottom of this article. The end result is setting my prompt thus:

PS1='\[\e[32m\]\u@\h\[\e[39m\]:\[\e[94m\]\w \[\e[36m\]$(__git_ps1 "(%s)")\[\e[39m\]\$ '

This renders as:

adam@DESKTOP //wsl$/ubuntu/home/adam/src/SymfonyFastTrack (main) $

Explanation:

  • I got almost all my guidance for this from How to show current git branch with colors in Bash prompt.
  • This goes in my ~/.bashrc file. I slung it in at the bottom.
  • PS1 is the variable containing Bash's primary prompt formatting. There's also PS0 -> PS4, but we don't need to worry about those.
  • \[\e[32m\] is an escape sequence that sets the colour for all characters thenceforth. The colour codes I'm using are:
    32
    Green - for the user / computer name
    39
    Default
    94
    Light blue - for the path
    36
    Cyan - for the branch
  • \u@\h is the Bash escape sequence for the current user and the short host name, separated by a literal @.
  • \w is the escape sequence for the current path.
  • $(__git_ps1 "(%s)") is the Git bit. __git_ps1 is a function that returns the branch, basically. How it does this is explained in those docs.

And that's it. As I said, none of this info is my own work, it's all from the references listed below. Cheers to the bods behind that lot.

Righto.

--
Adam


References:

Saturday 8 April 2023

Changing where I home my source code dramatically speeds up my Windows / WSL2 / Docker environment

G'day:

This is more an admission of "not initially thinking things through" on my part, but the outcome has helped me a lot, so in case there are others out there who don't think things through, maybe this will be helpful to them as well.

Or people can just point and laugh at me for being so thick.

Either way, perhaps someone will get something out of this.

My dev environment is Windows (nono, that's not the "not thinking things through" bit, just behave please). All my applications run in Docker containers, and the way I get the code into the container during dev is via a volume from my file system. For example this snippet from one of my docker-compose.yml files:

version: "3"
services:

    # ...

    php:
        build:
            context: php
            dockerfile: Dockerfile

        env_file:
            - envVars.public
            - envVars.private

        stdin_open: true
        tty: true

        volumes:
            - ..:/var/www

I'm just using a volume there to mount my app directory as /var/www in the container.

So the source code for the app is in - say - C:\src\myApp.

When I'm building and starting my containers, I drop into a shell in WSL, navigate to /mnt/c/src/myApp/docker, and do the docker compose up from there.

On Windows 10 and with older versions of WSL2 and Docker, this worked reasonably well. The app was a bit slow, but only as much as a shrug seemed to be a reasonable reaction to it. It's only dev.

When I migrated to Windows 11 things slowed down a chunk more, and it's been getting progressively worse. I've been working on a Symfony app recently, and clearing its cache is taking about 3-4min. Clearly this is ballocks cos it's PHP and nothing is measured in minutes with PHP.

Also my rig was comparatively slower than the other bods in my team. For me the unit tests in our CFML project have gone from taking - about a year ago - 5min to run (already not great) to about 10min now. Obviously a lot of this is that the tests we inherited were not great (almost all hit the DB), and we've also been adding a lot more tests in that intervening year. Recently though I found out that for other teams members it was slow, but they were only meaning like 3min was slow. Oh I wish they only took me 3min to run.

Clearly something is wrong on this machine. It's 4yrs old, but it was reasonably high spec when I bought it, and its drive is an SSD. So: no excuses there. And it's not like I'm Bitcoin mining; I'm just doing file system operations.

Whatever it is: I need to fix it.

I concluded it was something to do with misconfiguration of Docker or WSL making file operations from my host machine being dog slow when run from the container. I googled around a bit and it seems a lot of other people have had similar problems; but various settings, registry hacks, and even disabling Windows Defender (not a viable solution long-term, but something to try) were not helping.

Then someone mentioned "when the files are in the native part of the WSL file system, not the /mnt/c partition, then the overhead of the WSL->Windows file system processing doesn't occur". Their solution was to develop the code locally, then automatically deploy it via SSH into the container.

At the same time, I read that whilst there is the /mnt/c mount inside WSL, there is also the reverse: \\wsl.localhost points to the WSL file system, specifically for me \\wsl.localhost\Ubuntu is the filesystem for the Ubuntu distro I am using.

Putting two and two together to see how close to four I could get it, I did this:

  • Got rid of my code from C: drive.
  • Instead: I checked-out my code within WSL into ~/src/myApp.
  • Ran all my docker stuff from there, in ~/src/myApp/docker.
  • In VSCode and IntelliJ, homed my projects in \\wsl.localhost\Ubuntu\home\adam\src\myApp.

When I run those tests that before took >10min to run, now they take around 50sec. That is more than an order of magnitude faster.

In my Symfony project the cache-clear now takes a few seconds. And the tests there run in a second or so too.

I realise I am perhaps inheriting some slowness in reverse by accessing \\wsl.localhost\Ubuntu from Windows, but I am only dealing with occasional file edits and such like. Speed there is not a problem. Not one I could perceive anyhow.

I wish I had sat down to sort this out a few months back now. I had aimlessly googled in the past for 10min or so trying to find an easy silver bullet, but never found it and each time I looked I saw the same stuff. Today I rolled up my sleeves and said "right, I'm fixing this", and after about an extra 45min of googling and trying stuff (and then backing-out each thing that didn't work again), I landed on the solution.

Righto.

--
Adam