tag:blogger.com,1999:blog-81415745615304329092024-03-17T20:29:24.703+00:00Adam Cameron's Dev Blog"Adam Cameron - what is this guy's deal?" - DJgrassyknollAdam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comBlogger1483125tag:blogger.com,1999:blog-8141574561530432909.post-37898567652390545012024-03-17T20:20:00.005+00:002024-03-17T20:28:53.358+00:00CFML: solving a CFML problem with Java. Kinda.<p>G'day:</p>
<p>The other day on the CMFL Slack channel, Nick Petrie asked:</p>
<blockquote>
Anyone know of a native or plugin-based solution to pretty-formatting XML for display on the page? I'd like to output the XML nested this:
</blockquote>
I'll use a simplified example> we wanna render this:
<pre class="source-code"><code><aaa><bbb ccc="ddd"><eee/></bbb></aaa>
</code></pre>
Like this:
<pre class="source-code"><code><aaa>
<bbb ccc="ddd">
<eee/>
</bbb>
</aaa
</code></pre>
<p>There's no native CFML way of doing this, but I figured "this is a solved problem: there'll be an easy way to do it in Java". I googled <a href="https://www.google.com/search?q=java+prettier+xml" target="_blank">java prettier xml</a>, and the first match was on the Baeldung website (<a href="https://www.baeldung.com/java-pretty-print-xml" target="_blank">Pretty-Print XML in Java</a>) which is a site I trust to have good answers. But I checked that and a few others, and they all seem to be solving the problem the same way. So I decided to run with that.</p>
<p>The task at hand is to convert this to CFML (still using the Java libs to do the work, I mean, just runnable in CFML):</p>
<blockquote cite="https://www.baeldung.com/java-pretty-print-xml">
<pre class="source-code"><code>public static String prettyPrintByTransformer(String xmlString, int indent, boolean ignoreDeclaration) {
try {
InputSource src = new InputSource(new StringReader(xmlString));
Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(src);
TransformerFactory transformerFactory = TransformerFactory.newInstance();
transformerFactory.setAttribute("indent-number", indent);
Transformer transformer = transformerFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
Writer out = new StringWriter();
transformer.transform(new DOMSource(document), new StreamResult(out));
return out.toString();
} catch (Exception e) {
throw new RuntimeException("Error occurs when pretty-printing xml:\n" + xmlString, e);
}
}
</code></pre>
</blockquote>
<p>
To be very clear, this is not my code, it's from <cite><a href="https://www.baeldung.com/java-pretty-print-xml" target="_blank">Pretty-Print XML in Java</a></cite>.
</p>
<p>And also to be clear: I'm not gonna be doing anything unique or difficult or insightful or anything like that in this article. All I'm gonna do is to show how easy it is to convert native Java code to native CFML code, and this is a topical use case. It's gonna be a function with some object creation and some variable assigments. I'm gonna go line-by-line through that function above, and CFMLerise it. I just made that word up.</p>
<p>I'm gonna be using trycf.com to write and run this code, and the aim is to have a version that runs in both CF and Lucee.</p>
<p>First, let's run with the same function signature:</p>
<pre class="source-code"><code>unformattedXml = '<aaa><bbb ccc="ddd"><eee/></bbb></aaa>'
public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
}
prettyPrintByTransformer(unformattedXml)
</code></pre>
<p>In each step I'll be using that same <samp>unformattedXml</samp> input value, and just running the function to ensure it's got no compilation errors and each new statement I add "works" (in that it doesn't have runtime errors). The function won't do anything useful until it's done.</p>
<p>In this first step note that I've given the latter two parameters sensible defaults. This is a change from the Java version.</p>
<p>First things first: I don't like how they've put that largely pointless try/catch in the Java code. To me that sort of error-handling should be in the calling code if needed, not embedded in the implementation. If the implementation errors-out: let it. The actual exception will be more useful than swallowing the real exception and throwing a contrived one.</p>
<p>I'll include each statement I'm converting as a comment above the CFML version. This is only so I can draw your focus to it. I'd never include comments of this nature in my actual code.</p>
<pre class="source-code"><code>public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
// InputSource src = new InputSource(new StringReader(xmlString));
var xmlAsStringReader = createObject("java", "java.io.StringReader").init(xmlString) // new StringReader(xmlString)
var src = createObject("java", "org.xml.sax.InputSource").init(xmlAsStringReader)
}
</code></pre>
<p>I could have done this without the intermediary variable, but CFML is quite cumbersome making Java object proxies, so I think the code is clearer spread over two statements.</p>
<pre class="source-code"><code> // Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(src);
var document = createObject("java", "javax.xml.parsers.DocumentBuilderFactory").newInstance().newDocumentBuilder().parse(src)
</code></pre>
<pre class="source-code"><code> // TransformerFactory transformerFactory = TransformerFactory.newInstance();
var transformerFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance()
</code></pre>
<p>Now: this statement did not work for me:</p>
<pre class="source-code"><code>transformerFactory.setAttribute("indent-number", indent);
</code></pre>
<p>The CFML equivalent is exactly the same as it happens, but I was just getting "<samp>Not supported: indent-number</samp>" (CF) or "<samp>TransformerFactory does not recognise attribute 'indent-number'.</samp>" (Lucee). I presume it's a library version difference, but I was fairly limited in my troubleshooting as I was running this on trycf.com. Although I did verify I was getting the same thing on my local CF2023 container too. I googled a bit and found a work around, but I'll come back to this a bit further down.</p>
<pre class="source-code"><code> // Transformer transformer = transformerFactory.newTransformer();
var transformer = transformerFactory.newTransformer()
</code></pre>
<pre class="source-code"><code> /*
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
*/
var outputKeys = createObject("java", "javax.xml.transform.OutputKeys")
transformer.setOutputProperty(outputKeys.ENCODING, "UTF-8")
transformer.setOutputProperty(outputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no")
transformer.setOutputProperty(outputKeys.INDENT, "yes")
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", indent)
</code></pre>
<p>Once again, I need an intermediary variable here for the <samp>outputKeys</samp> class proxy. Just to save repetition in this case.</p>
<p>Also note the last line there is extra: it's the solution for the bit that wasn't working earlier. Easy.</p>
<pre class="source-code"><code> // Writer out = new StringWriter();
var out = createObject("java", "java.io.StringWriter").init()
</code></pre>
<pre class="source-code"><code> // transformer.transform(new DOMSource(document), new StreamResult(out));
var documentAsDomSource = createObject("java", "javax.xml.transform.dom.DOMSource").init(document)
var outAsStreamResult = createObject("java", "javax.xml.transform.stream.StreamResult").init(out)
transformer.transform(documentAsDomSource, outAsStreamResult)
</code></pre>
<p>More intermediary variables here.</p>
<pre class="source-code"><code> // return out.toString();
return out.toString()
</code></pre>
<p>And that's it. I've not got much commentary in all that lot, because it's all so straight forward [shrug].</p>
<p>The end result in one piece is thus:</p>
<pre class="source-code"><code>public string function prettyPrintByTransformer(required string xmlString, numeric indent=4, boolean ignoreDeclaration=true) {
var xmlAsStringReader = createObject("java", "java.io.StringReader").init(xmlString)
var src = createObject("java", "org.xml.sax.InputSource").init(xmlAsStringReader)
var document = createObject("java", "javax.xml.parsers.DocumentBuilderFactory").newInstance().newDocumentBuilder().parse(src)
var transformerFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance()
var transformer = transformerFactory.newTransformer()
var outputKeys = createObject("java", "javax.xml.transform.OutputKeys")
transformer.setOutputProperty(outputKeys.ENCODING, "UTF-8")
transformer.setOutputProperty(outputKeys.OMIT_XML_DECLARATION, ignoreDeclaration ? "yes" : "no");
transformer.setOutputProperty(outputKeys.INDENT, "yes")
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", indent)
var out = createObject("java", "java.io.StringWriter").init()
var documentAsDomSource = createObject("java", "javax.xml.transform.dom.DOMSource").init(document)
var outAsStreamResult = createObject("java", "javax.xml.transform.stream.StreamResult").init(out)
transformer.transform(documentAsDomSource, outAsStreamResult)
return out.toString()
}
</code></pre>
<p>And a coupla test runs:</p>
<pre class="source-code"><code>formattedXml = prettyPrintByTransformer(unformattedXml)
writeOutput("<pre>#encodeForHtml(formattedXml)#</pre>")
</code></pre>
<pre class="source-code"><code><aaa>
<bbb ccc="ddd">
<eee/>
</bbb>
</aaa>
</code></pre>
<pre class="source-code"><code>formattedXml = prettyPrintByTransformer(unformattedXml, 8, false)
writeOutput("<pre>#encodeForHtml(formattedXml)#</pre>")
</code></pre>
<pre class="source-code"><code><?xml version="1.0" encoding="UTF-8" standalone="no"?>
<aaa>
<bbb ccc="ddd">
<eee/>
</bbb>
</aaa>
</code></pre>
<p>The output above is from Lucee. On CF we get this:</p>
<pre class="source-code"><code><?xml version="1.0" encoding="UTF-8"?><aaa>
<bbb ccc="ddd">
<eee/>
</bbb>
</aaa>
</code></pre>
<p>Note this is nothing to do with the CFML code, it'll be some Java library variation on the Lucee and CF set-ups on trycf.com.</p>
<p>Right, so all this lot shows is that there's no reason to shy away from implementing a CFML version of some Java code you might find that solves a problem. So in turn, if you have some "algorithmic" issue that you wanna solve in CFML, don't shy away from googling for Java solutions, and checking how easy/hard it is to convert.</p>
<p>A runnable version of this code is @ <a href="https://trycf.com/gist/5e82e888385faa10af67298f88d3cab4/acf2023?theme=monokai" target="_blank">trycf.com</a>.
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-18447384756868090002023-10-27T16:07:00.003+00:002023-11-03T15:30:20.354+00:00I'm a big meany again<p>G'day</p>
<p>Chortle:</p>
<blockquote cite="https://www.reddit.com/r/coldfusion/comments/17g4nqs/adam_cameron_what_is_this_guys_deal/">
<p>I'm new to cf development and recently joined the cfml slack community. I noticed some curmudgeonly posts from this fella at first and didn't think anything of it... I'm now realizing that he is an a**h*** to people all the time. For example, there was a question posted today that currently has 14 replies. A few of them are from others trying to answer OP's question; 7 are this guy explaining how stupid of a question it is. What gives? I'm sure he's a knowledgeable and respected developer, but the pedantry is off the charts!</p>
</blockquote>
<p>
<cite><a href="https://www.reddit.com/r/coldfusion/comments/17g4nqs/adam_cameron_what_is_this_guys_deal/" target="_blank">DJgrassyknoll on Reddit</a></cite>
</p>
<p>Guilty as charged I suppose.</p>
<p>However the charges are pretty specious. I think the thread in question was the one with a question (paraphrase) "is there a function in CFML that can convert JSON to XML?". This is a largely non-sensical question, and without further elaboration, can only be furnished with non-sensical and unhelpful answers. I don't wanna do that. I wanna get the person's problem solved. I did point this out to the bod in question, and explained why, with both a real-world analogy, plus a code demonstration! Here's <a href="https://trycf.com/gist/e3927e1c05a601187138121dafa04349/lucee5?theme=monokai" target="_blank">the code version</a>:</p>
<pre class="source-code"><code>json = serializeJson({key="value"})
xml = xmlParse("<nowItIsXml>#encodeForXml(json)#</nowItIsXml>")
writeDump([
json = json,
xml = xml
])
</code></pre>
<p>Completely answers the question as stated, but is a completely frickin useless answer. Other than to illustrate the shortcomings of the question.</p>
<p>I've been answering people's CFML questions for years, and the biggest blocker to actually solving their problem is coercing <strong>what their actual problem is</strong> out from their brain. Really a lot of people don't seem to get we can't see the code in front of them, or the Jira ticket (sic) they're trying to deal with, or any other thoughts on the matter they might have had beyond what they type in as their question. So… not to be put off… I will explain this, and try to get a decent question out of them. So we all can understand it and then get their issue sorted. I will also often post links to <a href="https://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">How To Ask Questions The Smart Way</a> and/or <a href="http://sscce.org/" target="_blank">The SSCCE</a>. I'm not going to apologise for that. It's good advice and if more people took it on board, they would improve, and also their chances of getting good help when they need it would improve. How is that bad?</p>
<p>We're all busy people, so the quicker and easier it is for us to get the info we need in front of us to help: the more likely we are to help. As one of the participants in that thread said "[when] people ask shitty questions, I mostly will just ignore it". That's your call pal. But I <em>won't</em> ignore them. I'll still try to help.</p>
<p>Ah - fuck it - people can say what they like about me. I don't care (and hey it's given me an in to blow some of the dust off this thing). But try to have a well-formed train of thought before doing so. And also poss sink some time into helping people to understand what it's like before stirring the pot in yer little sewing circle.</p>
<p>But yeah cheers for the new strapline quote.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
<p>PS: props for using "curmudgeonly" though. Good word.</p>Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-61688933203298089602023-05-01T17:45:00.002+00:002023-05-01T17:50:13.572+00:00PHP / Symfony: working through "Symfony: The Fast Track", part 5: Twig stuff, and irritation<p>G'day:</p>
<p>I'm back on the case working through "<a href="https://symfony.com/book" target="_blank">Symfony: The Fast Track</a>". This will be part five of <a href="https://blog.adamcameron.me/search/label/Symfony%20the%20Fast%20Track" target="_blank">this series</a>, after the first four:</p>
<ul>
<li><a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 1: preparation and pre-requisites (and not actually any Symfony!)</a></li>
<li><a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony_28.html">PHP / Symfony: working through "Symfony: The Fast Track", part 2: creating a controller (eventually)</a></li>
<li><a href="https://blog.adamcameron.me/2023/04/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 3: doing some ORM / DB config</a></li>
<li><a href="https://blog.adamcameron.me/2023/04/php-symfony-working-through-symfony_7.html">PHP / Symfony: working through "Symfony: The Fast Track", part 4: not really about Symfony, this one</a></li>
</ul>
<p>The page I'm starting on is <a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html" target="_blank">Building the User Interface</a>, which seems more interesting than the previous section. I also see mention of my old friend Twig in the first subsection of this, so it will be good to look at that some more and see how it's moved along since I last did PHP UI stuff about 7-8 years ago (god is it that long?).</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#using-twig-for-the-templates" target="_blank">Using Twig for the Templates</a></h3>
<p>The first thing I need to do is to restore a twig file I deleted during the last exercise, thinking it was redundant. So: back comes <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/templates/base.html.twig" target="_blank"><samp>templates/base.html.twig</samp></a>. I'll be referring back to this, so will dump it out here, even though it's boilerplate, and not my code:</p>
<pre class="source-code"><code><!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>{% block title %}Welcome!{% endblock %}</title>
<link
rel="icon"
href="
data:image/svg+xml,
<svg
xmlns=%22http://www.w3.org/2000/svg%22
viewBox=%220 0 128 128%22
>
<text y=%221.2em%22 font-size=%2296%22>⚫️</text>
</svg>
"
>
{# Run `composer require symfony/webpack-encore-bundle` to start using Symfony UX #}
{% block stylesheets %}
{{ encore_entry_link_tags('app') }}
{% endblock %}
{% block javascripts %}
{{ encore_entry_script_tags('app') }}
{% endblock %}
</head>
<body>
{% block body %}{% endblock %}
</body>
</html>
</code></pre>
<p>And one for <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/templates/conference/index.html.twig" target="_blank"><samp>templates/conference/index.html.twig</samp></a>:</p>
<pre class="source-code"><code>{% extends 'base.html.twig' %}
{% block title %}Conference Guestbook{% endblock %}
{% block body %}
<h2>Give your feedback!</h2>
{% for conference in <span class="xr xrt" data-index="conferences">conferences</span> %}
<h4>{{ conference }}</h4>
{% endfor %}
{% endblock %}
</code></pre>
<p>Note that it inherits from the base one, and overrides its title and body blocks. I'll not discuss all the syntax vagaries as they're well documented: <a href="https://twig.symfony.com/doc/3.x/" target="_blank">Twig Documentation</a>. It's easy enough to read and infer what's going on, anyhow.</p>
<p>The next section is entitled "<a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#using-twig-in-a-controller" target="_blank">Using Twig in a Controller</a>" and I panicked as I thought they meant they were gonna put Twig code into a controller method (remember <a href="https://blog.adamcameron.me/2023/04/php-symfony-working-through-symfony_7.html#menus_in_controller_method">they've been a bit woolly about what goes in a controller already</a>, so it would not have surprised me), but fortunately no. It's just describing how to <em>use</em> the twig file:</p>
<pre class="source-code"><code style="color:gray"><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/src/Controller/ConferenceController.php#L14-L20" target="_blank">public function index</a>(<span style="color:black">Environment $twig, ConferenceRepository $conferenceRepository</span>): Response
{
// return of inline HTML (!!) removed
<span style="color:black">return new Response($twig->render(
"conference/index.html.twig",
["<span class="xr xrt" data-index="conferences">conferences</span>" => $conferenceRepository->findAll()]
));</span>
}
</code></pre>
<p>The <samp class="xr xrd u" data-index="conferences">conferences</samp> there are then used in the loop in the twig.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#creating-the-page-for-a-conference" target="_blank">Creating the Page for a Conference</a></h3>
<p>OK so here's a cool thing Symfony does. I am creating the page for the route <samp>/conference/{id}</samp>, and if my controller method takes a Conference object, Symfony is clever enough to go "I see what yer doing there" and goes and gets the conference for that ID. EG, I've added this routing to get to my controller method:</p>
<pre class="source-code"><code><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/config/routes/conference.yaml" target="_blank">config/routes/conference.yaml</a>
# …
show:
path: <span class="xr xrt" data-index="routing-id">/{id}</span>
methods: [ GET ]
controller: adamcameron\symfonythefasttrack\Controller\ConferenceController::show
</code></pre>
<p>And the controller method the exercise has me write is this (nothing noteworthy really):</p>
<pre class="source-code"><code><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/src/Controller/ConferenceController.php#L22-L39" target="_blank">public function show</a>(Environment $twig, <span class="xr xrt" data-index="controller-object">Conference $conference</span>, CommentRepository $commentRepository): Response
{
return new Response($twig->render(
"conference/show.html.twig",
[
"conference" => $conference,
"comments" => $commentRepository->findBy(
["conference" => $conference],
["createdAt" => "DESC"]
)
]
));
}
</code></pre>
<p><span class="xr xrd u" data-index="routing-id">The routing takes an ID</span>; <span class="xr xrd u" data-index="controller-object">the controller receives the equivalent Conference object</span>.</p>
<p>Note that whilst it's neat that Symfony does this, that controller they've had me write is borderline not great MVC in my books. I don't really think controllers should be aware of things like "repositories". Repositories are a storage consideration, and should be hidden away with the model. What <em>should</em> be happening here I think is that the controller receives its Conference, but the Conference should implement whatever is necessary to get its Comment collection from wherever it put it (and via whatever mechanism it did it with). This is not the controller's job. This is a good example of how using generic ORM crap leads to subpar code. I realise this is a tutorial about using Twig, but I think on the whole this book is encouraging some pretty poor design decisions because that better shows-off Symfony's bells and whistles.</p>
<p>Ahem. Moving along…</p>
<p>For reference here's <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/templates/conference/show.html.twig" target="_blank">the twig template for the view</a>:</p>
<pre class="source-code"><code>{% extends 'base.html.twig' %}
{% block title %}Conference Guestbook - {{ conference }}{% endblock %}
{% block body %}
<h2>{{ conference }} Conference</h2>
{% if comments|length > 0 %}
{% for comment in comments %}
{% if comment.photofilename %}
<img src="{{ asset('uploads/photos/' ~ comment.photofilename) }}"/>
{% endif %}
<h4>{{ comment.author }}</h4>
<small>
{{ comment.createdAt|format_datetime('medium', 'short') }}
</small>
<p>{{ comment.text }}</p>
{% endfor %}
{% else %}
<div>No comments have been posted yet for this conference.</div>
{% endif %}
{% endblock %}
</code></pre>
<p>And the test shows the initial part of the template implementation presents the detail from <span class="xr xrd u" data-index="test-crrect-conference">the correct conference correctly</span>:</p>
<pre class="source-code"><code>/** @testdox /conference/{id} end point returns a 200 OK for a valid conference */
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Acceptance/Controller/ConferenceControllerTest.php#L24-L33" target="_blank">public function testShow</a>()
{
$kernel = new Kernel("test", true);
$kernel->boot();
$container = $kernel->getContainer();
$connection = $container->get("database_connection");
$testConference = $connection->executeQuery("
SELECT id, city, year
FROM conference LIMIT 1
")->fetchAssociative();
if (!$testConference) {
$this->markTestSkipped("No conferences in database, test aborted");
}
$client = static::createClient();
<span class="xr xrt" data-index="test-crrect-conference">$client->request('GET', '/conference/' . $testConference['id']);</span>
$this->assertResponseIsSuccessful();
$this->assertSelectorTextContains(
'title',
sprintf(
"Conference Guestbook - %s %s",
<span class="xr xrt" data-index="test-crrect-conference">$testConference['city']</span>,
<span class="xr xrt" data-index="test-crrect-conference">$testConference['year']</span>
)
);
}
</code></pre>
<p>I presumed it would 404 if the <samp>{id}</samp> requested wasn't in the DB:</p>
<pre class="source-code"><code>/** @testdocs /conference/{id} end point returns a 404 if the conference is not found */
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Acceptance/Controller/ConferenceControllerTest.php#L49-L56" target="_blank">public function testShowNotFound</a>()
{
$client = static::createClient();
$client->request('GET', '/conference/999999');
$this->assertResponseStatusCodeSame(Response::HTTP_NOT_FOUND);
}
</code></pre>
<p>And this passes. NB: the 404 page is a general Symfony one, but that's fine. It's handling it correctly.</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCRn-ylIfT6blWh2kELv3teyWnb0r77dwPBI0rI94GawbwQGPhTNgQoq_3-tQDfXxiDos3uH-uqT0Wt7lEd9abQLqmbi6k4Mf3SVJVwQplVzfQh_wuKLGGfth-AyznROT4Hkfo4OmDhhUo0aSkFNRsVvMsDFIzxWXz4mPj4lo56FsjLBF5-28Q3jRtrg/s844/404.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="291" data-original-width="844" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCRn-ylIfT6blWh2kELv3teyWnb0r77dwPBI0rI94GawbwQGPhTNgQoq_3-tQDfXxiDos3uH-uqT0Wt7lEd9abQLqmbi6k4Mf3SVJVwQplVzfQh_wuKLGGfth-AyznROT4Hkfo4OmDhhUo0aSkFNRsVvMsDFIzxWXz4mPj4lo56FsjLBF5-28Q3jRtrg/s320/404.png"/></a></div>
<p>It's important to note that I have not written any code relating to Conferences or the storage thereof in this chapter of the exercise. I'm simply using the stuff Symfony generated for me during the previous exercise.</p>
<p>Oh one additional thing I did need to do was to install the <samp>twig/intl-extra</samp> dependency, because I'm using that <samp>format_datetime</samp> <a href="https://twig.symfony.com/doc/3.x/templates.html#filters" target="_blank">Twig filter</a>. How did I know I needed to do this? Symfony told me (well, OK, I have also just scrolled ahead in the lesson, and it also told me there):</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOSeEJQ_MU-fLk2KOGG4sbdI1I1dKWyqWHsv_2JfeWkMS7YEtgUqb_j3NvBi29y7jYTD3LLZKhv_73t9IZWW8GBVGG5BbXgP5s3yZdRGNe1ez7N3eDSnoBFLvRIZDDn_Q691Q5pWPi8lfj0mdVmAAV5lfIZlTc1-wxn0lQmZo7m9SEzFQZE69J8UbWxw/s843/intl.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="153" data-original-width="843" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOSeEJQ_MU-fLk2KOGG4sbdI1I1dKWyqWHsv_2JfeWkMS7YEtgUqb_j3NvBi29y7jYTD3LLZKhv_73t9IZWW8GBVGG5BbXgP5s3yZdRGNe1ez7N3eDSnoBFLvRIZDDn_Q691Q5pWPi8lfj0mdVmAAV5lfIZlTc1-wxn0lQmZo7m9SEzFQZE69J8UbWxw/s320/intl.png"/></a></div>
<p>That's quite cool. I note it's claiming it's a syntax error though, which it is not. If it was a syntax error, the code wouldn't compile, let alone run to the point to which it works out I have a Twig filter missing, and which one it is and where to get it from. It's a MissingTwigExtensionError or something. But a runtime error of some sort, not a compile-time one. Still: semantics aside, it's helpful. It does seem a bit odd to me though that <em>something</em> in either Twig or Symfony clearly knows what all the extensions are and what external libs they're in. This seems a bit of an inappropriate overreach to me, and a bit of a break of the <a href="https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle" target="_blank">Open-Closed Principle</a>. Still: productivity over dogma, eh?</p>
<p>Oh! So far I have no comments in the DB, so all that Twig bit about rendering them doesn't do anything. All I'm getting is this:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGxkfXTln2bL-v6MLMNocsV5P27hBZ7jgwja223wbx5uyua6xwpJyVHYzbKF4Fi4lQxFsp72MzCXP23cSXqpX1PjBqkG8MdYQL5H6MtReQa0Jdw4rBnMjr47l2WIXHRl0FXfsnQt6fDR_OxkbficNv2CWGzYvUR4WtXdvv3l-2E_LJWyQSibW0XNYKKg/s374/no_comment.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="84" data-original-width="374" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGxkfXTln2bL-v6MLMNocsV5P27hBZ7jgwja223wbx5uyua6xwpJyVHYzbKF4Fi4lQxFsp72MzCXP23cSXqpX1PjBqkG8MdYQL5H6MtReQa0Jdw4rBnMjr47l2WIXHRl0FXfsnQt6fDR_OxkbficNv2CWGzYvUR4WtXdvv3l-2E_LJWyQSibW0XNYKKg/s320/no_comment.png"/></a></div>
<p>Time to test the comments bit, before I move on.</p>
<p>[… more time passes than I would like to admit…]</p>
<p>I went round the houses with this a bit. I had to install a bunch of Node.js stuff in my Node container because one of the Twig function's internal workings checked for [something] and if it didn't exist: threw a PHP exception. I didn't preserve the error, but this code (in show.html.twig) was the culprit:</p>
<pre class="source-code"><code><img src="{{ <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/templates/conference/show.html.twig#L12" target="_blank">asset</a>('uploads/photos/' ~ comment.photofilename) }}"/>
</code></pre>
<p>The <samp>asset</samp> function was complaining that <samp>public/build/entrypoints.json</samp> was missing. I googled about and found out <a href="https://symfony.com/doc/current/frontend/encore/installation.html" target="_blank">I needed to have <samp>symfony/webpack-encore-bundle</samp> installed</a>, the instructions for which (that previous link) seem to not be as complete as they could be for codebases not using Symfony Flex. Still: the residual issues I had were all fairly googleable, and easily fixed. Sorry to be a bit vague there, but I did this lot on Fri/Sat - it's now Sun - and I can't be arsed digging through my browser history to recreate the - fairly unrewarding - sequence of hurdles I bumbled over to get my PHP app to stop erroring because I didn't have some webpack shit configured properly.</p>
<p>BTW: this is another thing I don't think a server-side MVC framework should be investing its time in. There's plenty of client-side frameworks for handling client-side stuff. Server-side frameworks should just busy themselves with their own job. Seems to me like another case of "if your only tool is a hammer, every problem starts looking like a nail".</p>
<p><em>Anyway</em>, my test stopped erroring, and after some small effort I got it to go green. I actually ended up with two tests here:</p>
<pre class="source-code"><code>/** @testdox /conference/{id} also displays any comment associated with the conference */
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Acceptance/Controller/ConferenceControllerTest.php#L58-L115" target="_blank">public function testShowComments</a>()
{
$testConference = $this->getConferenceToTestWith();
$client = static::createClient();
$client->request("GET", "/conference/" . $testConference->getId());
$this->assertResponseIsSuccessful();
$this->assertCommentersAreCorrect($client, $testConference);
}
private function getConferenceToTestWith(): Conference
{
$container = Container::getContainer();
$repo = $container->get("testing.ConferenceRepository");
$qb = $repo->createQueryBuilder("cf");
$qb->select("cf")
->setMaxResults(1)
->leftJoin("cf.comments", "cm")
->groupBy("cf.id")
->having("count(cm.id) > 1");
$testConference = $qb->getQuery()->getResult();
if (!$testConference) {
$this->markTestSkipped("No suitable conferences in database, test aborted");
}
return $testConference[0];
}
private function getExpectedCommenters(Conference $testConference): array
{
$expectedCommenters = $testConference->getComments()->toArray();
usort($expectedCommenters, function ($e1, $e2) {
return $e2->getCreatedAt() <=> $e1->getCreatedAt();
});
$expectedCommenters = array_map(function ($comment) {
return $comment->getAuthor();
}, $expectedCommenters);
return $expectedCommenters;
}
private function assertCommentersAreCorrect(KernelBrowser $client, Conference $testConference): void
{
$commenters = $client
->getCrawler()
->filter("body > h4")
->each(function ($node) {
return $node->text();
});
$expectedCommenters = $this->getExpectedCommenters($testConference);
$this->assertEquals($expectedCommenters, $commenters);
}
</code></pre>
<p>This one goes right through from the controller to the DB and back: comparing the data from the DB (expected) matching the data returned in the mark-up (actual).</p>
<p>I don't wanna be hitting the DB in my tests as a rule, especially if it's only a mechanism to get data to then see what the logic does with it. If I'm testing the logic, I want the expediency of using canned data that best exercises the logic. I tried to write a unit test of this, but it proved to be more difficult than I had patience for. The problem being that all the "logic" is in the controller method:</p>
<pre class="source-code"><code><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/src/Controller/ConferenceController.php#L22-L39" target="_blank">public function show</a>(
Environment $twig,
Conference $conference,
CommentRepository $commentRepository
): Response {
return new Response($twig->render(
"conference/show.html.twig",
[
"conference" => $conference,
"comments" => $commentRepository->findBy(
["conference" => $conference],
["createdAt" => "DESC"]
)
]
));
}
</code></pre>
<p>So to unit test that, I need to be able to mock the collaborators. Mocking my ones - the Conference and the CommentsRepository - was easy. Dealing with the Twig Environment was beyond my patience. And I wasn't even trying to do anything fancy with it: all I wanted was to spy on it so I could test the values exiting my test unit. But Symfony is very very convoluted in how it configures things, and I could not work out a minimum Environment that would actually initialise without erroring. Please note that I was very aware of "don't mock what you don't own" during all of this, and beyond a point I had decided that this was the wrong approach as well as being a fool's errand.</p>
<p>In the end I wrote a more functional test. I stuck with the Environment that Symfony had configured in its DI container, and the checked the results of its rendering. This is not so bad: it's all just done in code and no hits to external services (the DB), but it resulted in more code than I'm really happy with. There's no need to pore over this: I'm not gonna discuss it further. I only included to show the amount of effort involved.</p>
<pre class="source-code"><code>/** @testdox /conference/{id} displays correct conference and comments */
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Functional/Controller/ConferenceController/Show/Test.php" target="_blank">public function testShow</a>()
{
$conferenceTitle = "MOCKED_CONFERENCE";
list($twigEnvironment, $conference, $commentRepository) = $this->getTestDependencies($conferenceTitle);
$controller = new ConferenceController();
$result = $controller->show($twigEnvironment, $conference, $commentRepository);
$this->assertResponseIsCorrect($result, $conferenceTitle, $commentRepository);
}
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Functional/Controller/ConferenceController/Show/Dependencies.php" target="_blank">private function getTestDependencies</a>(string $conferenceTitle): array
{
$twigEnvironment = self::getContainer()->get(TwigEnvironment::class);
$conference = $this->getMockedConference($conferenceTitle);
$commentRepository = $this->getMockedCommentRepository();
return array($twigEnvironment, $conference, $commentRepository);
}
private function getMockedConference(string $conferenceTitle): Conference
{
$conference = $this->getMockBuilder(Conference::class)
->disableOriginalConstructor()
->getMock();
$conference
->expects($this->any())
->method("__toString")
->willReturn($conferenceTitle);
return $conference;
}
private function getMockedCommentRepository(): CommentRepository
{
$commentCount = 3;
$commentsToTestFor = range(1, $commentCount);
$testComments = array_map(function ($i) {
return (new Comment())
->setAuthor("COMMENTER" . $i)
->setText("COMMENT" . $i)
->setCreatedAt(new \DateTimeImmutable("2020-01-0{$i}"));
}, $commentsToTestFor);
$commentRepository = $this->getMockBuilder(CommentRepository::class)
->disableOriginalConstructor()
->getMock();
$commentRepository
->method("findBy")
->willReturn($testComments);
return $commentRepository;
}
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Functional/Controller/ConferenceController/Show/Assertions.php" target="_blank">private function assertResponseIsCorrect</a>(Response $result, string $conferenceTitle, mixed $commentRepository): void
{
$this->assertEquals(Response::HTTP_OK, $result->getStatusCode());
$xpathDocument = $this->getContextAsXpathDocument($result);
$this->assertTitleIsCorrect($xpathDocument);
$this->assertSubheadingIsCorrect($xpathDocument, $conferenceTitle);
$this->assertCommentsAreCorrect($xpathDocument, $commentRepository);
}
private function getContextAsXpathDocument(Response $result): DOMXPath
{
$content = $result->getContent();
$document = new DOMDocument();
$document->loadHTML($content, LIBXML_NOWARNING | LIBXML_NOERROR);
$xpathDocument = new DOMXPath($document);
return $xpathDocument;
}
private function assertTitleIsCorrect(DOMXPath $xpathDocument): void
{
$title = $xpathDocument->query('/html/head/title[text()]');
$this->assertCount(1, $title);
$this->assertEquals("Conference Guestbook - MOCKED_CONFERENCE", $title->item(0)->textContent);
}
private function assertSubheadingIsCorrect(DOMXPath $xpathDocument, string $conferenceTitle): void
{
$h2 = $xpathDocument->query('/html/body/h2[text()]');
$this->assertCount(1, $h2);
$this->assertEquals("$conferenceTitle Conference", $h2->item(0)->textContent);
}
private function assertCommentsAreCorrect(DOMXPath $xpathDocument, CommentRepository $commentRepository): void
{
$testComments = $commentRepository->findBy([], []);
$commentCount = count($testComments);
$commentAuthors = $xpathDocument->query('/html/body/h4[text()]');
$this->assertCount($commentCount, $commentAuthors);
$commentTexts = $xpathDocument->query('/html/body/p[text()]');
$this->assertCount($commentCount, $commentTexts);
$commentDates = $xpathDocument->query('/html/body/small[text()]');
$this->assertCount($commentCount, $commentDates);
array_walk($testComments, function ($comment, $i) use ($commentAuthors, $commentTexts, $commentDates) {
$this->assertEquals($comment->getAuthor(), $commentAuthors->item($i)->textContent);
$this->assertEquals($comment->getText(), $commentTexts->item($i)->textContent);
$commentCreatedAtFormatted = $comment->getCreatedAt()->format("M j, Y, h:i A");
$this->assertEquals($commentCreatedAtFormatted, trim($commentDates->item($i)->textContent));
});
}
</code></pre>
<p>There's a moral to all this. "Don't mock what you don't own" issues aside, testing code that is tightly-coupled to the framework - like a controller method pretty much is - is gonna have overhead. As such: keep as little code in yer controllers as possible: have yer business logic within yer model, because you "own" this, and there will be less over-engineers framework code getting in the way.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#linking-pages-together" target="_blank">Linking Pages Together</a> & <a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#paginating-the-comments" target="_blank">Pagination</a> & <a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#refactoring-the-controller" target="_blank">some refactoring</a></h3>
<p>Next they've got some more Twig advice about using its methods to build predictable URLs insted of hard-coding them, eg:</p>
<pre class="source-code"><code><span style="text-decoration: line-through"><a href="/conference/{{ conference.id }}">View</a></span>
<a href="{{ <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/templates/conference/index.html.twig#L10" target="_blank">path</a>('conference', { id: conference.id }) }}">View</a>
</code></pre>
<p>Good advice. I'm not gonna dwell on it further than that, other than sticking this code into my Twig template (after I test for it, obvs). I'll spare you the code, but it'll be in source control if you decide you <em>do</em> want to look at it. I got slightly tripped-up there as I had changed my route's name to be <samp>conference_show</samp>, and that reference to <samp>conference</samp> is the route's name. Easily fixed though.</p>
<p>Next it shows how to paginate record sets in the UI, which is pleasingly straight forward. A method in the repository:</p>
<pre class="source-code"><code>
public const PAGINATOR_PER_PAGE = 2;
// …
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/src/Repository/CommentRepository.php#L20-L31" target="_blank">public function getPaginator</a>(Conference $conference, int $offset): Paginator
{
$query = $this->createQueryBuilder('c')
->andWhere('c.conference = :conference')
->setParameter('conference', $conference)
->orderBy('c.createdAt', 'DESC')
->setMaxResults(self::PAGINATOR_PER_PAGE)
->setFirstResult($offset)
->getQuery();
return new Paginator($query);
}
</code></pre>
<p><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/src/Controller/ConferenceController.php#L27-L37" target="_blank">Pass a paginator instead of the articles from the controller</a>:</p>
<pre class="source-code"><code style="color:gray">public function show(
<span style="color:black">Request $request,</span>
Environment $twig,
Conference $conference,
CommentRepository $commentRepository
): Response {
<span style="color:black">$offset = max(0, $request->query->getInt('offset', 0));
$paginator = $commentRepository->getPaginator($conference, $offset);</span>
return new Response($twig->render(
"conference/show.html.twig",
[
"conference" => $conference,
<span style="text-decoration: line-through">"comments" => $commentRepository->findBy(["conference" => $conference], ["createdAt" => 'DESC']),</span>
<span style="color:black">"comments" => $paginator,
'previous' => $offset - CommentRepository::PAGINATOR_PER_PAGE,
'next' => min(count($paginator), $offset + CommentRepository::PAGINATOR_PER_PAGE)</span>
]
));
}
</code></pre>
<p>And then add <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/templates/conference/show.html.twig#L23-L28" target="_blank">the pagination stuff to the view</a>:</p>
<pre class="source-code"><code style="color:gray"><span style="color:black"><div>There are {{ comments|length }} comments.</div></span>
{% for comment in comments %}
{% if comment.photofilename %}
<img src="{{ asset('uploads/photos/' ~ comment.photofilename) }}"/>
{% endif %}
<h4>{{ comment.author }}</h4>
<small>
{{ comment.createdAt|format_datetime('medium', 'short') }}
</small>
<p>{{ comment.text }}</p>
{% endfor %}
<span style="color:black">{% if previous >= 0 %}
<a href="{{ path('conference_show', { id: conference.id, offset: previous }) }}">Previous</a>
{% endif %}
{% if next < comments|length %}
<a href="{{ path('conference_show', { id: conference.id, offset: next }) }}">Next</a>
{% endif %}</span>
</code></pre>
<p>The only change I made to this was that the suggested name for the pagination method was <samp>getCommentPaginator</samp>, but given it was in the CommentRepository, I thought repeating "comment" there was tautological, so changed it to <samp>getPaginator</samp>.</p>
<p>Oh, the page now looks like this:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjB1aHCEH0VugN8udLDEHJfp01egpo_WhCxPRs5OSDQ_V7XX1bXWVeU01BdnkJvfhrj-AXyzHfm7doL9oAIfr9PnQ9EmqRrxLT1jpvNf7u1wSJCgRosUKNvKFK1SwgwdpJnRymq-NunMR0_iLYpkVX9Pk67exgVA76REW4EItKgkpsaqHHzEY3lQrs75g/s1600/comments_paginated.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="371" data-original-width="387" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjB1aHCEH0VugN8udLDEHJfp01egpo_WhCxPRs5OSDQ_V7XX1bXWVeU01BdnkJvfhrj-AXyzHfm7doL9oAIfr9PnQ9EmqRrxLT1jpvNf7u1wSJCgRosUKNvKFK1SwgwdpJnRymq-NunMR0_iLYpkVX9Pk67exgVA76REW4EItKgkpsaqHHzEY3lQrs75g/s1600/comments_paginated.png"/></a></div>
<p>Cool.</p>
<p>One thing to note here is that in refactoring my tests of the ConferenceController <samp>show</samp> method to accommodate the pagination, I pushed the coupling of the test class over the top of what PHPMD was happy with (the threshold is the default 13 classes), so I refactored that into three classes: Test, Assertions and Dependencies.</p>
<div class="cliBox"><pre>
tests
`-- php
`-- Functional
`-- Controller
`-- ConferenceController
|-- Index
| `-- Test.php
`-- Show
|-- Assertions.php
|-- Dependencies.php
`-- Test.php
</pre>
</div>
<p>I think this is all caused by testing code in a controller; having to mess around with Requests and Environments and having to test for rendered mark-up instead of just values-in and value-out adds "unnecessary" overhead. I am realy irked (in a laughing way) by what I just read in the next section which is the refactoring bit…</p>
<blockquote cite="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#refactoring-the-controller">
<p>You might have noticed that both methods in ConferenceController take a Twig environment as an argument. Instead of injecting it into each method, let's leverage the render() helper method provided by the parent class</p>
</blockquote>
<p><cite><a href="https://symfony.com/doc/6.2/the-fast-track/en/10-twig.html#refactoring-the-controller" target="_blank">Symfony the Fast Track › Building the User Interface › Refactoring the Controller</a></cite></p>
<p>Oh FFS. Ha. OK. Fine. Tell me that <em>now</em>.</p>
<p>So I can simplify things a bit:</p>
<pre class="source-code"><code style="color:gray">public function show(
Request $request,
<span style="text-decoration: line-through;color:black">Environment $twig,</span>
Conference $conference,
CommentRepository $commentRepository
): Response {
$offset = max(0, $request->query->getInt('offset', 0));
$paginator = $commentRepository->getPaginator($conference, $offset);
<span style="text-decoration: line-through;color:black">return new Response($twig->render('conference/index.html.twig', [</span>
<span style="color:black">return $this->render(</span>
"conference/show.html.twig",
[
"conference" => $conference,
"comments" => $paginator,
'previous' => $offset - CommentRepository::PAGINATOR_PER_PAGE,
'next' => min(count($paginator), $offset + CommentRepository::PAGINATOR_PER_PAGE)
]
<span style="color:black">);</span>
<span style="text-decoration: line-through;color:black">]));</span>
}
</code></pre>
<p>That's good, but I really don't know why they didn't lead with that. There was nothing that relied on the Twig Environment in those methods, so why pass it in? Just to give an extra paragraph at the end?</p>
<p>Oh I also had to tweak a couple of the tests that broke because of this: for that <samp>render</samp> method to be available, I needed to <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.9/tests/php/Functional/Controller/ConferenceController/Show/Test.php#L22-L24" target="_blank">initialise the controller class with the DI container</a>. This was easy though:</p>
<pre class="source-code"><code style="color:gray">$controller = new ConferenceController();
<span style="color:black">$container = self::getContainer();
$controller->setContainer($container);</span>
</code></pre>
<hr class="narrow">
<h3>End of section / article</h3>
<p>Man. I only got through one page of the docs with this article, and it also took me four bites to write, even though it didn't really cover much ground. I think the issue is that a) I lack self-discipline; b) I'm not finding this stuff particularly compelling. I also got deeply frustrated with the amount of dicking around I had to do to try to convince Symfony to let me at its dependencies when I was trying to test stuff, and also the waste of time that was getting Encore working, because - as I said earlier - client-side libs are nothing to do with a server-side MVC framework, and should not get in the way. The more I use Symfony, the more I miss <a href="https://github.com/silexphp/Silex" target="_blank">Silex</a>.</p>
<p>I'm not looking forward to the re-drafting & read-throughs ofthis before I press "send", as I suspect it's a bit of a half-hearted mess. But I've come this far, so I'm buggered if I'm <em>not</em> publishing it. Well done if you read this far.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-9829179102676378712023-04-17T12:10:00.001+00:002023-04-17T12:12:26.591+00:00CFML: Into the Box 2023<p>Howdy partner:</p>
<p>Well here's something interesting (well: to me, anyhow). I will be attending the <a href="https://www.intothebox.org/" target="_blank">Into the Box</a> conference in Houston, TX in May (17th - 19th). This has come about due to a very kind gesture from Luis / Ortus Solutions. Thank-you so much for this!</p>
<p>I know nothing about Houston - I have been through the airport on the way to somewhere else a coupla times, and that's it - so am getting there a coupla days early to have a look around and make sure the local craft beer bars are patronised.</p>
<p>The schedule of Into the Box itself looks pretty good too, and cover a lot of bases other than just the CFML / Boxiverse side of things. I see "testing" and "TDD" mentioned a coupla times in the schedule, so will be in my element.</p>
<p>The best thing about conferences though is to get to put faces to names, and <em>meet people</em> properly. I might be a surly anti-social type, but I really value the friendships I have built from meeting people at industry conferences, and having an agendaless chat with them over a beer or a watercooler, and got to know them better. I do hope to catch up with a bunch of people from the community who I know only as words appearing in a chat app / forum / etc. And hopefully re-meet some familiar faces too.</p>
<p>And the proof is in the pudding on this "conference socialising" thing: after saying farewell to Houston on the Sunday after the conference, I'm heading over to North Carolina for a few days to hang out with Dan & Jennette Skaggs (and family). I first met D & J @ CF.Obective() (or maybe Dev.Objective()?) many years ago, and we've become really good friends since then. And having had them come visit me on my side of the pond a coupla times; it's my turn to come see their part of the world. I can't wait.</p>
<p>So come and say "G'day" if you see me around the conference, and let's talk about... whatevs. And maybe let's go for a beer :-D</p>
<p>Y'all come back now, hear? (*)</p>
<p>-- <br>Adam</p>
<hr class="narrow">
<p>(*) yeah, all right, enough of that.</p>Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-70919465705116841842023-04-11T13:14:00.002+00:002023-04-11T13:16:34.515+00:00Getting Windows Terminal to open my Ubuntu Bash session in the right directory<p>G'day:</p>
<p>This is a follow on from the previous two articles:</p>
<ul>
<li><a href="https://blog.adamcameron.me/2023/04/changing-where-i-home-my-source-code.html">Changing where I home my source code dramatically speeds up my Windows / WSL2 / Docker environment</a></li>
<li><a href="https://blog.adamcameron.me/2023/04/changing-my-wsl-bash-prompt-to-include.html">Changing my WSL Bash prompt to include my current Git branch</a></li>
</ul>
<p>All that is working well, but I had a wee problem with switching my Windows Terminal config from starting Git Bash shells to do the equivalent with directly running a Bash shell on my WSL Ubuntu filesystem.</p>
<p>For Git Bash, I have profiles set up like this:</p>
<pre class="source-code"><code>{
"commandline": "C:\\apps\\Git\\git-bash.exe",
"guid": "{be9a184b-1c89-4ac5-88da-3ef93cd5ec98}",
"hidden": false,
"icon": "C:\\apps\\Git\\mingw64\\share\\git\\git-for-windows.ico",
"name": "Git Bash (SymfonyFastTrack)",
"startingDirectory": "\\\\wsl$\\Ubuntu\\home\\adam\\src\SymfonyFastTrack",
"tabTitle": "Git Bash (SymfonyFastTrack)"
}
</code></pre>
<p>And when I open a terminal with that profile I get:</p>
<div class="cliBox"><pre>
<span style="color:limeGreen">adam@DESKTOP</span> <span style="color:magenta">MINGW64</span> <span style="color:goldenRod">//wsl$/ubuntu/home/adam/src/SymfonyFastTrack</span> <span style="color:cyan">(main)</span>
$
</pre></div>
<p>I figured it would be easy with an Ubuntu one, I'd copy the default one:</p>
<pre class="source-code"><code>{
"guid": "{2c4de342-38b7-51cf-b940-2309a097f518}",
"hidden": false,
"name": "Ubuntu",
"source": "Windows.Terminal.Wsl"
}
</code></pre>
<p>Change its GUID, give it a different name and a <samp>startingDirectory</samp> value and done. But no: one cannot have more than one profile for <samp>Windows.Terminal.Wsl</samp> it would seem: It didn't show up (it also didn't error, which Windows Terminal is pretty good at when it doesn't like something).</p>
<p>So now I have two issues: getting a second Ubuntu Bash shell profile working <em>at all</em>, and then: point it to the correct starting directory.</p>
<p>The first thing I found out is that in the current version of Windows Terminal, the <samp>startingDirectory</samp> is not honoured for Ubuntu anyhow. So that's a non-starter. I did a lot of googling, and that turned up nothing. Then I turned to a more powerful search engine: I <a href="https://mingo.nl/" target="_blank">Mingo</a>-ed it. He didn't quite nail the solution, but he got me onto the right track: asking ChatGPT. I dunno why I didn't start there. After a bit of back and forth, ChatGPT and I came up with this:</p>
<pre class="source-code"><code>{
"commandline": "<span class="xr xrt" data-index="wsl">wsl.exe -d Ubuntu</span> <span class="xr xrt" data-index="bash">/bin/bash</span> <span class="xr xrt" data-index="rcfile">--rcfile</span> <span class="xr xrt" data-index="stdout"><(echo \"</span><span class="xr xrt" data-index="original-bashrc">source ~/.bashrc</span>; <span class="xr xrt" data-index="this-dir">cd ~/src/SymfonyFastTrack</span><span class="xr xrt" data-index="stdout">\")</span>",
"guid": "{3933fa46-657f-4db9-ad6a-2bee51554bc5}",
"icon": "C:\\Users\\camer\\AppData\\Local\\wt\\bash.png",
"name": "Bash (SymfonyFastTrack)",
"tabTitle": "Bash (SymfonyFastTrack)"
}
</code></pre>
<p>Explanation:</p>
<ul>
<li><samp class="xr xrd u" data-index="wsl">wsl.exe -d Ubuntu</samp> is the long form of what I tried before with <samp>source:Windows.Terminal.Wsl; name: Ubuntu</samp>.</li>
<li><samp class="xr xrd u" data-index="bash">/bin/bash</samp> says to run Bash.</li>
<li><samp class="xr xrd u" data-index="rcfile">--rcfile</samp> says "using this RC file (eg: instead of <samp>.bashrc</samp>.</li>
<li><samp class="xr xrd u" data-index="stdout"><(echo [etc])</samp>: instead of using an actual file, take it from <samp>stdout</samp>.</li>
<li><samp class="xr xrd u" data-index="original-bashrc">source ~/.bashrc</samp>: first my actual <samp>.bashrc</samp>.</li>
<li><samp class="xr xrd u" data-index="this-dir">cd ~/src/SymfonyFastTrack\</samp>: but then switch to this directory.</li>
</ul>
<p>And when I use this profile, I get what I want:</p>
<div class="cliBox"><pre>
<span style="color:limeGreen">adam@DESKTOP</span> <span style="color:cornflowerBlue">//wsl$/ubuntu/home/adam/src/SymfonyFastTrack</span> <span style="color:cyan">(main)</span> $
</pre></div>
<p>Cool. All done.</p>
<p>This was another article which is largely me fumbling around being a n00b, but we all start that way with things I guess, so maybe this will short-circuit all the goolging, mingoing and ChatGPTing I had to do to arrive at the solution.</p>
<p>And - seriously Mingo - cheers for helping with this.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-84448686937322639262023-04-09T12:57:00.001+00:002023-04-09T12:59:23.110+00:00Changing my WSL Bash prompt to include my current Git branch<p>G'day:</p>
<p>Note: there's no real original research / thinking in this. it's just the result of me googling stuff, and arriving at a result. I'm writing it mostly so I have a record of it.</p>
<p>Yesterday I shifted my source code directories from my Windows file system (eg: <samp>C:\src\myapp</samp>) to using the WSL file system (eg: <samp>~/src/myapp</samp>). See <a href="https://blog.adamcameron.me/2023/04/changing-where-i-home-my-source-code.html">Changing where I home my source code dramatically speeds up my Windows / WSL2 / Docker environment</a>.</p>
<p>Up until now I'd been doing my dev from <samp>C:</samp>, my Docker stuff in WSL, and my Git carry-on from Git Bash. Given my files are in the WSL file system now, <em>and</em> I have Git installed in there too, I figured I might as well ditch Git Bash and use Git directly via Bash.</p>
<p>The only thing I needed to do to shift to this work pattern is to update my shell prompt to reflect what branch I'm currently in, if it's a version-controlled directory.</p>
<p>EG, in Bash I see this:</p>
<div class="cliBox"><pre>
<span style="color:limeGreen">adam@DESKTOP</span> <span style="color:magenta">MINGW64</span> <span style="color:goldenRod">//wsl$/ubuntu/home/adam/src/SymfonyFastTrack</span> <span style="color:cyan">(main)</span>
$
</pre></div>
<p>But in Bash I just see this:</p>
<div class="cliBox"><pre>
<span style="color:limeGreen">adam@DESKTOP</span> <span style="color:cornflowerBlue">//wsl$/ubuntu/home/adam/src/SymfonyFastTrack</span> $
</pre></div>
<p>I don't need the <samp>MINGW64</samp> in there (I don't even know what it means, nor have I ever had to care. Also: no need to pipe up and tell me), but I do need the branch to be shown the current branch I'm on if I'm in a source-controlled directory, like how Git Bash does it.</p>
<p>I googled about how to change the Bash prompt, and found a few helpful notes. Links to those @ the bottom of this article. The end result is setting my prompt thus:</p>
<div class="cliBox">
<span class="xr xrt" data-index="PS1">PS1</span>='<span class="xr xrt" data-index="colour-green"><span class="xr xrt" data-index="colours">\[\e[32m\]</span><span class="xr xrt" data-index="user-host">\u@\h</span></span><span class="xr xrt" data-index="colour-default"><span class="xr xrt" data-index="colours">\[\e[39m\]</span>:</span><span class="xr xrt" data-index="colour-light-blue"><span class="xr xrt" data-index="colours">\[\e[94m\]</span><span class="xr xrt" data-index="path">\w</span></span> <span class="xr xrt" data-index="colour-cyan"><span class="xr xrt" data-index="colours">\[\e[36m\]</span><span class="xr xrt" data-index="git-ps1">$(__git_ps1 "(%s)")</span></span><span class="xr xrt" data-index="colour-default"><span class="xr xrt" data-index="colours">\[\e[39m\]</span>\$ </span>'
</div>
<p>This renders as:</p>
<div class="cliBox"><pre>
<span style="color:limeGreen">adam@DESKTOP</span> <span style="color:cornflowerBlue">//wsl$/ubuntu/home/adam/src/SymfonyFastTrack</span> <span style="color:cyan">(main)</span> $
</pre></div>
<p>Explanation:</p>
<ul>
<li>I got almost all my guidance for this from <a href="https://thucnc.medium.com/how-to-show-current-git-branch-with-colors-in-bash-prompt-380d05a24745" target="_blank">How to show current git branch with colors in Bash prompt</a>.</li>
<li>This goes in my <a href="#bashrc_target" name="bashrc_src"><samp>~/.bashrc</samp> file</a>. I slung it in at the bottom.</li>
<li><span class="xr xrd u" data-index="PS1"><a href="#PS1_target" name="PS1_src"><samp>PS1</samp></a></span> is the variable containing Bash's primary prompt formatting. There's also <samp>PS0</samp> -> <samp>PS4</samp>, but we don't need to worry about those.</li>
<li>
<span class="xr xrd u" data-index="colours"><samp>\[\e[32m\]</samp> is an escape sequence that sets the colour</span> for all characters thenceforth. The <a href="#colours_target" name="colours_src">colour codes</a> I'm using are:
<dl class="inline">
<dt><samp>32</samp></dt>
<dd><span class="xr xrd u" data-index="colour-green">Green - for the user / computer name</span></dd>
<dt><samp>39</samp></dt>
<dd><span class="xr xrd u" data-index="colour-default">Default</span></dd>
<dt><samp>94</samp></dt>
<dd><span class="xr xrd u" data-index="colour-light-blue">Light blue - for the path</span></dd>
<dt><samp>36</samp></dt>
<dd><span class="xr xrd u" data-index="colour-cyan">Cyan - for the branch</span></dd>
</dl>
</li>
<li><span class="xr xrd u" data-index="user-host"><samp>\u@\h</samp></span> is the <a href="#esc_target" name="esc_src">Bash escape sequence</a> for the current user and the short host name, separated by a literal <samp>@</samp>.</li>
<li><span class="xr xrd u" data-index="path"><samp>\w</samp></span> is the escape sequence for the current path.</li>
<li><span class="xr xrd u" data-index="git-ps1"><samp>$(__git_ps1 "(%s)")</samp></span> is the Git bit. <a href="#git-ps1_target" name="git-ps1_src"><samp>__git_ps1</samp></a> is a function that returns the branch, basically. How it does this is explained in those docs.</li>
</ul>
<p>And that's it. As I said, none of this info is my own work, it's all from the references listed below. Cheers to the bods behind that lot.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
<hr class="narrow">
<p>References:</p>
<ul>
<li><a href="https://thucnc.medium.com/how-to-show-current-git-branch-with-colors-in-bash-prompt-380d05a24745" target="_blank" name="howto_target">How to show current git branch with colors in Bash prompt</a> <a href="#howto_src">^</a></li>
<li><a href="https://wiki.archlinux.org/title/Bash/Prompt_customization" target="_blank" name="PS1_target">archlinux.org › Bash/Prompt customization</a> <a href="#PS1_src">^</a></li>
<li><a href="https://wiki.archlinux.org/title/Bash#Configuration_files" target="_blank" name="bashrc_target">archlinux.org › Configuration files</a> <a href="#bashrc_src">^</a></li>
<li><a href="https://misc.flogisoft.com/bash/tip_colors_and_formatting" target="_blank" name="colours_target">FLOZz' MISC » bash:tip_colors_and_formatting</a> <a href="#colours_src">^</a></li>
<li><a href="https://www.gnu.org/software/bash/manual/html_node/Controlling-the-Prompt.html" target="_blank" name="esc_target">Bash Reference Manual › Features › 6.9 Controlling the Prompt</a> <a href="#esc_src">^</a></li>
<li><a href="https://git-scm.com/book/uz/v2/Appendix-A%3A-Git-in-Other-Environments-Git-in-Bash" target="_blank" name="git-ps1_target">Git › A1.4 Appendix A: Git in Other Environments - Git in Bash</a> <a href="#git-ps1_src">^</a></li>
</ul>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-76558997597875657722023-04-08T17:42:00.001+00:002023-04-08T17:42:46.184+00:00Changing where I home my source code dramatically speeds up my Windows / WSL2 / Docker environment<p>G'day:</p>
<p>This is more an admission of "not initially thinking things through" on my part, but the outcome has helped me a lot, so in case there are others out there who don't think things through, maybe this will be helpful to them as well.</p>
<p>Or people can just point and laugh at me for being so thick.</p>
<p>Either way, perhaps someone will get something out of this.</p>
<p>My dev environment is Windows (nono, that's not the "not thinking things through" bit, just behave please). All my applications run in Docker containers, and the way I get the code into the container during dev is via a volume from my file system. For example this snippet from one of my <a href="https://github.com/adamcameron/php8/blob/1.18.1/docker/docker-compose.yml" target="_blank"><samp>docker-compose.yml</samp></a> files:</p>
<pre class="source-code"><code style="color:gray">version: "3"
services:
# ...
<span style="color:black">php:</span>
build:
context: php
dockerfile: Dockerfile
env_file:
- envVars.public
- envVars.private
stdin_open: true
tty: true
<span style="color:black">volumes:</span>
<span style="color:black">- <span class="xr xrt" data-index="app-dir">..</span>:<span class="xr xrt" data-index="mount">/var/www</span></span>
</code></pre>
<p>I'm just using a volume there to mount <span class="xr xrd u" data-index="app-dir">my app directory</span> <span class="xr xrd u" data-index="mount">as /var/www in the container</span>.</p>
<p>So the source code for the app is in - say - <samp>C:\src\myApp</samp>.</p>
<p>When I'm building and starting my containers, I drop into a shell in WSL, navigate to <samp>/mnt/c/src/myApp/docker</samp>, and do the <samp>docker compose up</samp> from there.</p>
<p>On Windows 10 and with older versions of WSL2 and Docker, this worked reasonably well. The app was a bit slow, but only as much as a shrug seemed to be a reasonable reaction to it. It's only dev.</p>
<p>When I migrated to Windows 11 things slowed down a chunk more, and it's been getting progressively worse. I've been working on a Symfony app recently, and clearing its cache is taking about 3-4min. Clearly this is ballocks cos it's PHP and nothing is measured in minutes with PHP.</p>
<p>Also my rig was comparatively slower than the other bods in my team. For me the unit tests in our CFML project have gone from taking - about a year ago - 5min to run (already not great) to about 10min now. Obviously a lot of this is that the tests we inherited were not great (almost all hit the DB), and we've also been adding a lot more tests in that intervening year. Recently though I found out that for other teams members it was slow, but they were only meaning like <strong>3min</strong> was slow. Oh I wish they only took me 3min to run.</p>
<p>Clearly something is wrong on this machine. It's 4yrs old, but it was reasonably high spec when I bought it, and its drive is an SSD. So: no excuses there. And it's not like I'm Bitcoin mining; I'm just doing file system operations.</p>
<p>Whatever it is: I need to fix it.</p>
<p>I concluded it was something to do with misconfiguration of Docker or WSL making file operations from my host machine being dog slow when run from the container. I googled around a bit and it seems a lot of other people have had similar problems; but various settings, registry hacks, and even disabling Windows Defender (not a viable solution long-term, but something to try) were not helping.</p>
<p>Then someone mentioned "when the files are in the native part of the WSL file system, not the <samp>/mnt/c</samp> partition, then the overhead of the WSL->Windows file system processing doesn't occur". Their solution was to develop the code locally, then automatically deploy it via SSH into the container.</p>
<p>At the same time, I read that whilst there is the <samp>/mnt/c</samp> mount inside WSL, there is also the reverse: <samp>\\wsl.localhost</samp> points to the WSL file system, specifically for me <samp>\\wsl.localhost\Ubuntu</samp> is the filesystem for the Ubuntu distro I am using.</p>
<p>Putting two and two together to see how close to four I could get it, I did this:</p>
<ul>
<li>Got rid of my code from <samp>C:</samp> drive.</li>
<li>Instead: I checked-out my code within WSL into <samp>~/src/myApp</samp>.</li>
<li>Ran all my docker stuff from there, in <samp>~/src/myApp/docker</samp>.</li>
<li>In VSCode and IntelliJ, homed my projects in <samp>\\wsl.localhost\Ubuntu\home\adam\src\myApp</samp>.</li>
</ul>
<p>When I run those tests that before took >10min to run, now they take around 50sec. That is more than an order of magnitude faster.</p>
<p>In my Symfony project the cache-clear now takes a few seconds. And the tests there run in a second or so too.</p>
<p>I realise I am perhaps inheriting some slowness in reverse by accessing <samp>\\wsl.localhost\Ubuntu</samp> from Windows, but <strong>I</strong> am only dealing with occasional file edits and such like. Speed there is not a problem. Not one I could perceive anyhow.</p>
<p>I wish I had sat down to sort this out a few months back now. I had aimlessly googled in the past for 10min or so trying to find an easy silver bullet, but never found it and each time I looked I saw the same stuff. Today I rolled up my sleeves and said "right, I'm fixing this", and after about an extra 45min of googling and trying stuff (and then backing-out each thing that didn't work again), I landed on the solution.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-38027394722280556122023-04-07T15:02:00.008+00:002023-05-12T15:53:10.171+00:00PHP / Symfony: working through "Symfony: The Fast Track", part 4: not really about Symfony, this one<p>G'day:</p>
<p>Once again I'm gonna continue working through "<a href="https://symfony.com/book" target="_blank">Symfony: The Fast Track</a>". This will be part four of <a href="https://blog.adamcameron.me/search/label/Symfony%20the%20Fast%20Track" target="_blank">this series</a>, after the first three:</p>
<ul>
<li><a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 1: preparation and pre-requisites (and not actually any Symfony!)</a></li>
<li><a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony_28.html">PHP / Symfony: working through "Symfony: The Fast Track", part 2: creating a controller (eventually)</a></li>
<li><a href="https://blog.adamcameron.me/2023/04/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 3: doing some ORM / DB config</a></li>
</ul>
<p>The page I'm starting on is <a href="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html" target="_blank">Setting up an Admin Backend</a>, which sounds very situation-specific to me (so: not very portable knowledge to acquire), but I guess I'll learn "Symfony's way of ~". And there might be some useful stuff in there. Let's see.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html" target="_blank">Setting up an Admin Backend</a></h3>
<h4><a href="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html#installing-more-dependencies" target="_blank">Installing more dependencies</a></h4>
<p>First they describe some Symfony concepts: Symfony Components: low-level stuff like routing, HTTP etc; and Symfony Bundles: higher-level stuff like wrappers for third-party libs. Fair enough.</p>
<p>Hrm. More Symfony opinions now. There's a "feature" of Symfony "aliases". Their wording:</p>
<blockquote cite="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html">
Aliases are shortcuts for popular Composer packages. Want an ORM for your application? Require <samp>orm</samp>. Want to develop an API? Require <samp>api</samp>. These aliases are automatically resolved to one or more regular Composer packages. They are opinionated choices made by the Symfony core team.
</blockquote>
<cite><a href="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html" target="_blank">Symfony: The Fast Track › Setting up an Admin Backend › Installing more Dependencies</a></cite>
<p>Righto then.</p>
<p>This is a bit pathetic:</p>
<blockquote>
Another neat feature is that you can always omit the <samp>symfony</samp> vendor. Require <samp>cache</samp> instead of <samp>symfony/cache</samp>.
</blockquote>
<cite>ibid</cite>
<p>Mate. Yer bragging about omitting seven keystrokes. Well: other than the fact I've got to type in <samp>symfony composer</samp> rather than just <samp>composer</samp>. How was this even a good use of anyone's dev time, let alone brag about it being not only a feature, but a "neat" one? Sigh.</p>
<p>They're getting me to do this: <samp>symfony composer req "admin:^4"</samp>, which installs <samp>easycorp/easyadmin-bundle</samp>, and adds a reference to it in <samp>config/bundles.php</samp>,and adds <samp>config/packages/uid.yaml</samp>. The latter looks intriguing, I guess I'll find about about it later.</p>
<hr class="narrow">
<h4><a href="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html#configuring-easyadmin" target="_blank">Configuring EasyAdmin</a></h4>
<p>Next up I'm configuring this EasyAdmin thing:</p>
<div class="cliBox"><pre>
root:/var/www# symfony console make:admin:dashboard
<span style="color:limeGreen">Which class name do you prefer for your Dashboard controller?</span> [<span style="color:goldenRod">DashboardController</span>]:
>
<span style="color:limeGreen">In which directory of your project do you want to generate "DashboardController"?</span> [<span style="color:goldenRod">src/Controller/Admin/</span>]:
>
<div style="background-color:limeGreen;color:black;width:35em">
[OK] Your dashboard class has been successfully generated.
</div>
Next steps:
* Configure your Dashboard at "src/Controller/Admin/DashboardController.php"
* Run "make:admin:crud" to generate CRUD controllers and link them from the Dashboard.
root:/var/www#
</pre>
</div>
<p>This has added a controller with some boilerplate (I'll spare you, but I'll link through to it in source control once it's there). There's a slight bug in the generation, in that it has not picked up my app's PSR-4 namespace, so it's written out this:</p>
<pre class="source-code"><code>namespace App\Controller\Admin;
</code></pre>
Which I will change to be:
<pre class="source-code"><code>namespace adamcameron\symfonythefasttrack\Controller\Admin;
</code></pre>
<p>I kinda think this should have looked at the <samp>maker.yaml</samp> config file that Symfony itself suggested I create to set the <samp>root_namespace</samp> (see <a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony_28.html#maker-namespace">It looks like your app may be using a namespace other than "App"</a> in the second article of this series). Ah well: never mind: it's an easy fix.</p>
<p>It's also put an annotation in to handle the routing which I am not gonna run with, I'll stick it in the routing config where it belongs.</p>
<p>And because I'm changing something here, I'm gonna make a test for it. I should really have done this before running the wizard I guess. I didn't think about it. I also didn't know what the wizard was gonna do at the time, however I could have read ahead. Anyways, this has configured a <samp>/admin/</samp> end point, so I will make sure it returns a 200:</p>
<pre class="source-code"><code>namespace adamcameron\symfonythefasttrack\tests\Acceptance\Controller;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Component\HttpFoundation\Response;
/** @testdox Tests the endpoints in the DashboardController */
class <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.8/tests/php/Acceptance/Controller/DashboardControllerTest.php" target="_blank">DashboardControllerTest</a> extends WebTestCase
{
/** @testdox The index endpoint returns a 200 response */
public function testIndex()
{
$client = static::createClient();
$client->request('GET', '/admin/');
$this->assertEquals(Response::HTTP_OK, $client->getResponse()->getStatusCode());
}
}
</code></pre>
<p>(This changes in the next step to be a 302, but I adjusted the test accordingly behind the scenes).</p>
<p>I've added a new test suite too, for acceptance tests (hitting the "front" of pages, and verifying they do whatever they are supposed to. For now, 200ing will be fine).</p>
<p>This test still passes after I reconfigure the routing. And I also get a page rendering in the browser:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMj0si3482EnYg3iCXmjUbec9CtzdbykNczhw-y_hoRtjr76maEitBQlxgkSEFI9h1zJSZ-Y7P_IgdfxVxJ6RkBGeH1xKp0yuzbsbovKy7kwUXFGBPcpuEb_XDMcNK_dbjG83xSu5x8tdM2Jspn2MIaZPVNN5hkOIIaceRURcrKtvimih1czLWbSi4nA/s896/easyadmin_default.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="776" data-original-width="896" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMj0si3482EnYg3iCXmjUbec9CtzdbykNczhw-y_hoRtjr76maEitBQlxgkSEFI9h1zJSZ-Y7P_IgdfxVxJ6RkBGeH1xKp0yuzbsbovKy7kwUXFGBPcpuEb_XDMcNK_dbjG83xSu5x8tdM2Jspn2MIaZPVNN5hkOIIaceRURcrKtvimih1czLWbSi4nA/s320/easyadmin_default.png"/></a></div>
<p>Next I'm building a CRUD UI for the entities I've created. This is done via <samp>symfony console make:admin:crud</samp>.I'll spare you the detail unless there's something interesting.</p>
<p>[…]</p>
<p>Nope, nothing interesting except it asked me what namespace to use this time, so I was able to give it the correct one. It <em>defaulted</em> to Symfony's default <samp>App</samp> one though.</p>
<p>The process has created two skeleton controller classes. They don't have any routing in them, which is "interesting". The next step is wiring them into that initial dashbaord page I created, so let's see - just in the browser - what happens when I've done that.</p>
<p id="menus_in_controller_method">I don't like how the menu items are added to this subsystem. They're done through a method in the controller:</p>
<pre class="source-code"><code>public function <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.8/src/Controller/Admin/DashboardController.php#L29-L35" target="_blakn">configureMenuItems</a>(): iterable
{
yield MenuItem::linkToDashboard('Dashboard', 'fa fa-home');
yield MenuItem::linktoRoute('Back to the website', 'fas fa-home', 'homepage');
yield MenuItem::linkToCrud('Conferences', 'fas fa-map-marker-alt', Conference::class);
yield MenuItem::linkToCrud('Comments', 'fas fa-comments', Comment::class);
}
</code></pre>
<p>A controller should only be busying itself with marrying-up which model should provide the data for the response, and which view to use to render the data. <em>Here</em> it is defining the data too. I mean this is a third-party package it's demonstrating here, so this approach is not strictly Symfony's doing; but Symfony is choosing to use this package, so I don't think it's great that official Symfony guidance should be encouraging poor practice like this.</p>
<p>It's also got me to reconfigure the index route handler method to redirect to the CRUD UI for conferences:</p>
<pre class="source-code"><code>public function <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.8/src/Controller/Admin/DashboardController.php#L15-L21" target="_blank">index</a>(): Response
{
$routeBuilder = $this->container->get(AdminUrlGenerator::class);
$url = $routeBuilder->setController(ConferenceCrudController::class)->generateUrl();
return $this->redirect($url);
}
</code></pre>
<p>When I reloaded the index page, I got a 500 error: the <samp>conference</samp> table was missing. Ah. I have rebuilt the containers since the last article, <em>and</em> I blew away the volume the DB data was in. I need to rerun the migration to get those tables back.</p>
<hr class="narrow">
<h4>"Doctrine: know your limits!"(<a href="#enfield" name="enfield_back">*</a>)</h4>
<p>Oh for goodness sake.</p>
<p>I just re-ran the migration to recreate the Conference and Comment tables, and I noticed... Symfony blew away the rest of the tables in the database (OK, granted, there was only one other table). WTH, Symfony? Then I looked at the migration file:</p>
<pre class="source-code"><code style="color:gray">public function up(Schema $schema): void
{
// this up() migration is auto-generated, please modify it to your needs
// [snip for brevity]
$this->addSql('DROP TRIGGER IF EXISTS notify_trigger ON messenger_messages;');
$this->addSql('CREATE TRIGGER notify_trigger AFTER INSERT OR UPDATE [etc]');
$this->addSql('ALTER TABLE comment ADD CONSTRAINT FK_9474526C604B8382 FOREIGN KEY [etc]');
<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.7.1/migrations/Version20230402161944.php#L42" target="_blank"><span style="color:black" class="xr xrt" data-index="drop-test">$this->addSql('DROP TABLE test');</span></a>
}
</code></pre>
<p><span class="xr xrd u" data-index="drop-test">You <em>what</em>, son?</span>. Who the hell told you to do that? (/me hastily checks the <samp>make:migration</samp> process to confirm I didn't just go "yeah yeah yeah, delete the other tables. Cos like of course I want you to do that". No, I had not).</p>
<p>I googled about the place, and found this: <a href="https://www.doctrine-project.org/projects/doctrine-migrations/en/3.6/reference/generating-migrations.html#ignoring-custom-tables" target="_blank">Doctrine › Migrations › Generating Migrations › Ignoring Custom Tables</a>:</p>
<blockquote cite="https://www.doctrine-project.org/projects/doctrine-migrations/en/3.6/reference/generating-migrations.html#ignoring-custom-tables">
<p>If you have custom tables which are not managed by Doctrine you will need to tell Doctrine to ignore these tables. Otherwise, everytime you run the <samp>diff</samp> command, Doctrine will try to drop those tables. You can configure Doctrine with a schema filter.</p>
</blockquote>
<p><strong>What?</strong> Just: no. Doctrine, I have said "map these entities". Your job is to - let's see if you follow my thinking here - map these entities. Nothing else. Leave the rest of the frickin DB alone. It's not your business. Jesus.</p>
<p>At least they go on to say:</p>
<blockquote>
If you use the DoctrineBundle with Symfony you can set the <samp>schema_filter</samp> option in your configuration.
</blockquote>
<p>And over on the Symfony side of things: <a href="https://symfony.com/bundles/DoctrineMigrationsBundle/current/index.html#manual-tables" target="_blank">Symfony › Bundles › DoctrineMigrationsBundle › Manual Tables</a>:</p>
<blockquote cite="https://symfony.com/bundles/DoctrineMigrationsBundle/current/index.html#manual-tables">
<p>If you follow a specific scheme you can configure doctrine/dbal to ignore those tables. Let's say all custom tables will be prefixed by <samp>t_</samp>. In this case you just have have to add the following configuration option to your doctrine configuration:</p>
<pre class="source-code"><code>doctrine:
dbal:
schema_filter: ~^(?!t_)~
</code></pre>
<p>Note that if you have multiple connections configured then the <samp>schema_filter</samp> configuration will need to be placed per-connection.</p>
</blockquote>
<p>OK, two things. </p>
<ul>
<li>What if you're <em>not</em> a lunatic from the 1990s and don't put hungarian notation on the beginning of things, so there isn't a prefix to match <strong>all the rest of the tables in your database</strong>?</li>
<li>Why the hell are you tightly coupling this to connection config? If I don't want Doctrine to delete my shit, then that's not gonna be - by default - connection specific. It's gonna be a blanket "don't delete my shit, you weirdo!?" across the board. I can see how maybe it could be overridden on a connection-by-connection basis (maybe?), but this should be a top-level Doctrine config thing (and "don't do it" being the <em>default</em> behaviour).
</ul>
<p>Ugh. But OK, I've added this to my connection:</p>
<pre class="source-code"><code style="color:gray"><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.8/config/packages/doctrine.yaml#L15" target="_blank">doctrine</a>:
dbal:
default_connection: default
connections:
default:
wrapper_class: Doctrine\DBAL\Connections\PrimaryReadReplicaConnection
dbname: '%env(resolve:POSTGRES_PRIMARY_DB)%'
host: '%env(resolve:POSTGRES_PRIMARY_HOST)%'
port: '%env(resolve:POSTGRES_PRIMARY_PORT)%'
user: '%env(resolve:POSTGRES_PRIMARY_USER)%'
password: '%env(resolve:POSTGRES_PRIMARY_PASSWORD)%'
driver: pdo_pgsql
server_version: 15
charset: utf8
<span style="color:black">schema_filter: ~^(?!<span class="xr xrt" data-index="one-table">test</span>)~</span>
replicas:
replica1:
dbname: '%env(resolve:POSTGRES_REPLICA_DB)%'
host: '%env(resolve:POSTGRES_REPLICA_HOST)%'
port: '%env(resolve:POSTGRES_REPLICA_PORT)%'
user: '%env(resolve:POSTGRES_REPLICA_USER)%'
password: '%env(resolve:POSTGRES_REPLICA_PASSWORD)%'
charset: utf8
</code></pre>
<p>I guess I'm lucky <span class="xr xrd u" data-index="one-table">I only have the one table to exclude, or that "pattern" could get quite weighty</span>, quite quickly.</p>
<div class="updateBox">
As an aside: I found out that when running <samp>symfony console make:migration</samp>, it'll try to run <em>all</em> the files in the <samp>migrations</samp> directory. As each has its own time-stamp-unique file name, I probably only want to keep the most recent one in source control? Or at least only one in that directory at any given time, anyhow. There might be a way of telling it to only run one migration. I should look into that.
</div>
<p>Anyway: I have all my tables back in the DB now, so the page renders an empty Conference CRUD page:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeHTkZoQXVhAMyaBpmjq7gJkfqA8s7tnuZ_TZ6HEvF7Gn4KD0x8xgykyS7Jg1FdCGiiCjVxO97sYxy8ATEraloWm3onYekyMg0slbjTxTCpfZYL6nrwSb0YXg_YQG21JQKjekZcsDLSTbYAvKK5KaIu6oYyzSwgAMx-MnmwmSsKgt9KtbseI-IJDlcHA/s970/conference.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="375" data-original-width="970" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeHTkZoQXVhAMyaBpmjq7gJkfqA8s7tnuZ_TZ6HEvF7Gn4KD0x8xgykyS7Jg1FdCGiiCjVxO97sYxy8ATEraloWm3onYekyMg0slbjTxTCpfZYL6nrwSb0YXg_YQG21JQKjekZcsDLSTbYAvKK5KaIu6oYyzSwgAMx-MnmwmSsKgt9KtbseI-IJDlcHA/s320/conference.png"/></a></div>
<p>And it all works fine once I allow the <samp>/admin/</samp> route to process POST requests instead of just GETs (this is on me: I didn't realise I'd be needing to allow POSTs when I set up the routing).</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUxOg6VBQqryDGnGRB8bWxdVDblvtHK7oYPEZpCzSCCUo9K587MDRZacTQfVeYXDFgrIVo_lnWsb08J-K8dzUpy9ghw-S9_C6dD91n3a-18hglbIK46Cze9kr0hev_FDG-hrZssofzDLkLRJG6COf0T_lsH5qMx4MIufAeVM_I3vma70QdlWgfFK3YNA/s981/conferences.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="315" data-original-width="981" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUxOg6VBQqryDGnGRB8bWxdVDblvtHK7oYPEZpCzSCCUo9K587MDRZacTQfVeYXDFgrIVo_lnWsb08J-K8dzUpy9ghw-S9_C6dD91n3a-18hglbIK46Cze9kr0hev_FDG-hrZssofzDLkLRJG6COf0T_lsH5qMx4MIufAeVM_I3vma70QdlWgfFK3YNA/s320/conferences.png"/></a></div>
<p>I had less luck initially adding in comments, as the steps are slightly out of order in the book. I got onto this fix via googling "symfony the fast track comment UI broken", and landed on the GitHub issue to get it fixed: <a href="https://github.com/symfony/symfony-docs/issues/15252#issuecomment-830874810" target="_blank">[Book] Step 9 Issue: Cannot create new Comment</a>. So: in case you are following along here: after the step of testing/trying out the "Add conference" functionality, don't continue to add a comment; skip ahead and do the "<a href="https://symfony.com/doc/6.2/the-fast-track/en/9-backend.html#customizing-easyadmin" target="_blank">Customizing EasyAdmin</a>" bit first. It's just the next step.</p>
<p>This step adds code to configure how the form fields should work:</p>
<pre class="source-code"><code>class <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.8/src/Controller/Admin/CommentCrudController.php" target="_blank">CommentCrudController</a> extends AbstractCrudController
{
public static function getEntityFqcn(): string
{
return Comment::class;
}
public function configureCrud(Crud $crud): Crud
{
return $crud
->setEntityLabelInSingular('Conference Comment')
->setEntityLabelInPlural('Conference Comments')
->setSearchFields(['author', 'text', 'email'])
->setDefaultSort(['createdAt' => 'DESC'])
;
}
public function configureFilters(Filters $filters): Filters
{
return $filters
->add(EntityFilter::new('conference'))
;
}
public function configureFields(string $pageName): iterable
{
yield AssociationField::new('conference');
yield TextField::new('author');
yield EmailField::new('email');
yield TextareaField::new('text')
->hideOnIndex()
;
yield TextField::new('photoFilename')
->onlyOnIndex()
;
$createdAt = DateTimeField::new('createdAt')->setFormTypeOptions([
'html5' => true,
'years' => range(date('Y'), ((int)date('Y')) + 5),
'widget' => 'single_text',
]);
if (Crud::PAGE_EDIT === $pageName) {
yield $createdAt->setFormTypeOption('disabled', true);
} else {
yield $createdAt;
}
}
}
</code></pre>
<p>(Again: directly in the controller. Bleah).</p>
<hr class="narrow">
<p>That's it for that page. It didn't really have much to do with Symfony though, did it? It was all about this third-party admin app that I have no interest in whatsoever. This is no slight on EasyAdmin, it looks slick. But this is a Symfony book, not an EasyAdmin book.</p>
<p>I'm a bit annoyed at my experiences working through that page, so I'm leaving off for now.</p>
<p>There not much code that I would be willing to put my name next to in this effort, but I'll link to it anyhow: <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.8" target="_blank"><samp>1.8</samp></a>.</p>
<p>The next part is here: <a href="https://blog.adamcameron.me/2023/05/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 5: Twig stuff, and irritation</a>.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
<p>(<a name="enfield" href="#enfield_back">*</a>) [<a href="https://www.youtube.com/watch?v=LS37SNYjg8w" target="_blank">cough</a>]</p>Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-64282397144413345662023-04-02T17:16:00.004+00:002023-04-08T18:01:04.656+00:00PHP / Symfony: working through "Symfony: The Fast Track", part 3: doing some ORM / DB config<p>G'day:</p>
<p>Today I'm gonna continue working through "<a href="https://symfony.com/book" target="_blank">Symfony: The Fast Track</a>". This will be part three of this series, after the first two:</p>
<ul>
<li><a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 1: preparation and pre-requisites (and not actually any Symfony!)</a></li>
<li><a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony_28.html">PHP / Symfony: working through "Symfony: The Fast Track", part 2: creating a controller (eventually)</a></li>
</ul>
<p>I also had a brief interlude yesterday whilst I messed around with the DB connnection driver the app was using.: <a href="https://blog.adamcameron.me/2023/04/symfony-doctrine-dbal.html">Symfony / Doctrine / DBAL: convincing/configuring it to use a PrimaryReadReplicaConnection connection</a>. This was not part of the Symfony book, just something I wanted to do.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/7-database.html" target="_blank">Setting up a Database</a></h3>
<p>Nothing much to see here. It's about setting up a PostgreSQL Docker container. Done already.</p>
<p>It also has some stuff that demonstrates another irk for me when it comes to frameworks that are getting a bit self important, and overreaching their job (which is <em>to be a bloody web framework. <strong>Just</strong> that</em>). Clock this:</p>
<blockquote cite="https://symfony.com/doc/6.2/the-fast-track/en/7-database.html#accessing-the-local-database">
<p>Using the psql command-line utility might prove useful from time to time. But you need to remember the credentials and the database name. […]</p>
<p>[…] Thanks to these conventions, accessing the database via symfony run is much easier:</p>
<div class="cliBox"><pre>symfony run psql</pre></div>
</blockquote>
<cite><a href="https://symfony.com/doc/6.2/the-fast-track/en/7-database.html#accessing-the-local-database" target="_blank">Symfony: The Fast Track › Setting up a Database › Accessing the Local Database
</a></cite>
<p>Lads. You've "done a thing" that saves the person passing a <samp>-U</samp> and <samp>-d</samp> param to <samp>psql</samp>. But coupling it to the framework, and requiring the PostgreSQL client being installed in yer PHP environment. Just… <em>why?</em></p>
<p>In contrast, here's me logging into the client <em>in the PostgreSQL container</em>:</p>
<div class="cliBox"><pre>psql -U user1primary -d db1primary
</pre></div>
<p>(Because I'm on the "server" I don't need a password, as auth is handed off to whatever mechanism I used to start the shell on the server, which is handy). So it was a good use of Symfony's time implementing the work to save <em>those</em> coupla dozen keystrokes. Ugh.</p>
<p>It also goes in to how to do some stuff with the production environment they introduced a few chapters back, but I'm not using that so I ignored it.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/8-doctrine.html" target="_blank">Describing the Data Structure</a></h3>
<h4>Bumpf</h4>
<p>This starts by discussing <samp>config/packages/doctrine.yaml</samp> and how it works, and about <samp>DATABASE_URL</samp>. You can read about my opinion of <samp>DATABASE_URL</samp> in yesterday's article (<a href="https://blog.adamcameron.me/2023/04/symfony-doctrine-dbal.html">Symfony / Doctrine / DBAL: convincing/configuring it to use a PrimaryReadReplicaConnection connection</a>). Ah to be fair in a simple situation it would work well, but it does seem like a weird way to manage the configuration of the various params the DB needs to connect. Even if they <em>are</em> ultimately used via a URL. Bad coupling of disparate concepts, IMO.</p>
<hr class="narrow">
<h4>Creating entities / repositories & property relationships</h4>
<p>Next: running a wizard to create some boilerplate.The input values I am providing for this lot are:</p>
<pre>
city, string, 255, no;
year, string, 4, no;
isInternational, boolean, no.
</pre>
<div class="cliBox"><pre>
root :/var/www# symfony console make:entity Conference
<span style="color:blue">created</span>: src/Entity/Conference.php
<span style="color:blue">created</span>: src/Repository/ConferenceRepository.php
Entity generated! Now let's add some fields!
You can always add more fields later manually or by re-running this command.
<span style="color:limegreen">New property name (press <return> to stop adding fields)</span>:
> city
<span style="color:limegreen">Field type (enter <span style="color:GoldenRod">?</span> to see all types)</span> [<span style="color:GoldenRod">string</span>]:
>
<span style="color:limegreen">Field length</span> [<span style="color:GoldenRod">255</span>]:
>
<span style="color:limegreen">Can this field be null in the database (nullable) (yes/no)</span> [<span style="color:GoldenRod">no</span>]:
>
<span style="color:GoldenRod">updated</span>: src/Entity/Conference.php
<span style="color:limegreen">Add another property? Enter the property name (or press <return> to stop adding fields)</span>:
> year
<span style="color:limegreen">Field type (enter <span style="color:GoldenRod">?</span> to see all types)</span> [<span style="color:GoldenRod">string</span>]:
>
<span style="color:limegreen">Field length</span> [<span style="color:GoldenRod">255</span>]:
> 4
<span style="color:limegreen">Can this field be null in the database (nullable) (yes/no)</span> [<span style="color:GoldenRod">no</span>]:
>
<span style="color:GoldenRod">updated</span>: src/Entity/Conference.php
<span style="color:limegreen">Add another property? Enter the property name (or press <return> to stop adding fields)</span>:
> isInternational
<span style="color:limegreen">Field type (enter <span style="color:GoldenRod">?</span> to see all types)</span> [<span style="color:GoldenRod">boolean</span>]:
>
<span style="color:limegreen">Can this field be null in the database (nullable) (yes/no)</span> [<span style="color:GoldenRod">no</span>]:
>
<span style="color:GoldenRod">updated</span>: src/Entity/Conference.php
<span style="color:limegreen">Add another property? Enter the property name (or press <return> to stop adding fields)</span>:
>
<div style="padding:1em; color:white;background-color:limegreen;width:5em">Success!</div>
Next: When you're ready, create a migration with <span style="color:limegreen">php bin/console make:migration</span>
root :/var/www#
</pre></div>
<p>As this has stated, it's created two files: <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.7/src/Entity/Conference.php" target="_blank"><samp>src/Entity/Conference.php</samp></a> and <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.7/src/Repository/ConferenceRepository.php" target="_blank"><samp>src/Repository/ConferenceRepository.php</samp></a>. Let's have a look.</p>
<p>The <samp>Conference</samp> has a lot of repetition, so I'll elide a bunch of it, and only show the <samp>city</samp> property's bumpf:</p>
<pre class="source-code"><code>#[ORM\Entity(repositoryClass: ConferenceRepository::class)]
class Conference
{
#[ORM\Id]
#[ORM\GeneratedValue]
#[ORM\Column]
private ?int $id = null;
#[ORM\Column(length: 255)]
private ?string $city = null;
// ...
public function getId(): ?int
{
return $this->id;
}
public function getCity(): ?string
{
return $this->city;
}
public function setCity(string $city): self
{
$this->city = $city;
return $this;
}
// ...
}
</code></pre>
<p>Firstly: ewwww… annotations (well: <a href="https://www.php.net/manual/en/language.attributes.overview.php" target="_blank">PHP attributes</a> in this case, but it amounts to the same thing). I hate having storage-specific shite in my code. I'd much rather a discrete mapping file (YAML or something), and keep the storage considerations as the second-class citizen that it should be. However I suppose this is a necessary evil with ORM shite these days (can you tell I'm not completely sold on ORM as a concept? ;-)). There is currently a way of doing the mapping with YAML - <a href="https://www.doctrine-project.org/projects/doctrine-orm/en/2.14/reference/yaml-mapping.html" target="_blank">Doctrine › ORM › YAML Mapping</a> - but at the top of that page they say it's deprecated in favour of "one of the other mappings", which seems to mean <a href="https://www.doctrine-project.org/projects/doctrine-orm/en/2.14/reference/xml-mapping.html#xml-mapping" target="_blank">XML</a> or <a href="https://www.doctrine-project.org/projects/doctrine-orm/en/2.14/reference/php-mapping.html#php-mapping" target="_blank">with actual PHP</a>. In 2023 someone is advocating moving from YAML to XML. Cute. Anyway: for now I'll stick with the attributes. It's not the worst thing about this code.</p>
<p>It's funny that the docs page I'm following says "Note that the class itself is a plain PHP class with no signs of Doctrine". I mean… <em>except all the Doctrine-specific attributes</em>, you mean. Which are 24% of the statements in the class.</p>
<p>The <em>worst</em> thing is the getter and setter methods all properties have created by default. I'm not that happy with encouraging anti-patterns like this. See <a href="https://martinfowler.com/bliki/TellDontAsk.html" target="_blank">Tell-Don't-Ask</a> and I also think it's a gateway drug for <a href="https://en.wikipedia.org/wiki/Law_of_Demeter" target="_blank">Law of Demeter</a> violations. It's an enabler for bad OOP. I guess if their wizard thing just created the class and the properties (and the [muttermutter] ORM annotations), then it wouldn't seem like there was much of a point in having the wizard: it's not saving much effort.</p>
<p>I can see that I'm gonna need to bundle these "entities" away somewhere away from my actual application model, and just call on them to handle the storage side of things.</p>
<p>Oh yeah the <samp>ConferenceRepository</samp> class:</p>
<pre class="source-code"><code>/**
* @extends ServiceEntityRepository<Conference>
*
* @method Conference|null find($id, $lockMode = null, $lockVersion = null)
* @method Conference|null findOneBy(array $criteria, array $orderBy = null)
* @method Conference[] findAll()
* @method Conference[] findBy(array $criteria, array $orderBy = null, $limit = null, $offset = null)
*/
class ConferenceRepository extends ServiceEntityRepository
{
public function __construct(ManagerRegistry $registry)
{
parent::__construct($registry, Conference::class);
}
public function save(Conference $entity, bool $flush = false): void
{
$this->getEntityManager()->persist($entity);
if ($flush) {
$this->getEntityManager()->flush();
}
}
public function remove(Conference $entity, bool $flush = false): void
{
$this->getEntityManager()->remove($entity);
if ($flush) {
$this->getEntityManager()->flush();
}
}
<span style="color:gray">// /**
// * @return Conference[] Returns an array of Conference objects
// */
// public function findByExampleField($value): array
// {
// return $this->createQueryBuilder('c')
// ->andWhere('c.exampleField = :val')
// ->setParameter('val', $value)
// ->orderBy('c.id', 'ASC')
// ->setMaxResults(10)
// ->getQuery()
// ->getResult()
// ;
// }
// public function findOneBySomeField($value): ?Conference
// {
// return $this->createQueryBuilder('c')
// ->andWhere('c.exampleField = :val')
// ->setParameter('val', $value)
// ->getQuery()
// ->getOneOrNullResult()
// ;
// }</span>
}
</code></pre>
<p>I <em>love</em> how they are pretending PHP is Java and it has generics with stuff like <samp>ServiceEntityRepository<Conference></samp>. Why do that? But then again I'm asking the same question about that entire comment block. What's the point? You have the code already defining all that lot.</p>
<p>It's not as bad (or is it?) as all the generated commented-out code at the bottom. WTaF?</p>
<p>Other than that: yeah cool… it's separated the definition of the entities from the storage thereof. I'll have to see how the code to save an object works. Currently all I can see tying these two together is the <samp>#[ORM\Entity(repositoryClass: ConferenceRepository::class)]</samp> attribute on the <samp>Conference</samp> class.</p>
<p>Next it gets me to create a <samp>Comment</samp> entity. I'll spare you the bulk of the detail, but this is the config:</p>
<pre>
author, string, 255, no;
text, text, no;
email, string, 255, no;
createdAt, datetime_immutable, no.
</pre>
<p>Actually there was one interesting thing here:</p>
<div class="cliBox"><pre>
<span style="color:limegreen">Add another property? Enter the property name (or press <return> to stop adding fields)</span>:
> createdAt
<span style="color:limegreen">Field type (enter <span style="color:GoldenRod">?</span> to see all types)</span> [<span style="color:GoldenRod">datetime_immutable</span>]:
>
</pre></div>
<p>It seems to have clocked from the name <samp>createdAt</samp> that it should be a date time. That's quite cool.</p>
<p>Next I need to establish the relationship between the two entities, which is done by running <samp>symfony console make:entity Conference</samp> again:</p>
<div class="cliBox"><pre>
root:/var/www# symfony console make:entity Conference
Your entity already exists! So let's add some new fields!
<span style="color:limegreen">New property name (press <return> to stop adding fields)</span>:
> comments
<span style="color:limegreen">Field type (enter <span style="color:GoldenRod">?</span> to see all types)</span> [<span style="color:GoldenRod">string</span>]:
> OneToMany
<span style="color:limegreen">What class should this entity be related to?:</span>
> Comment
A new property will also be added to the <span style="color:GoldenRod">Comment</span> class
so that you can access and set the related <span style="color:GoldenRod">Conference</span> object from it.
<span style="color:limegreen">New field name inside Comment [<span style="color:GoldenRod">conference</span>]:</span>
>
<span style="color:limegreen">Is the <span style="color:GoldenRod">Comment</span>.<span style="color:GoldenRod">conference</span> property allowed to be null (nullable)? (yes/no)</span> [<span style="color:GoldenRod">yes</span>]:
> no
Do you want to activate <span style="color:GoldenRod">orphanRemoval</span> on your relationship?
A <span style="color:GoldenRod">Comment</span> is "orphaned" when it is removed from its related <span style="color:GoldenRod">Conference</span>.
e.g. <span style="color:GoldenRod">$conference->removeComment($comment)</span>
NOTE: If a <span style="color:GoldenRod">Comment</span> may *change* from one <span style="color:GoldenRod">Conference</span> to another, answer "no".
<span style="color:limegreen">Do you want to automatically delete orphaned
<span style="color:GoldenRod">adamcameron\symfonythefasttrack\Entity\Comment</span> objects (orphanRemoval)? (yes/no)</span> [<span style="color:GoldenRod">no</span>]:
> yes
<span style="color:GoldenRod">updated</span>: src/Entity/Conference.php
<span style="color:GoldenRod">updated</span>: src/Entity/Comment.php
<span style="color:limegreen">Add another property? Enter the property name (or press <return> to stop adding fields)</span>:
>
<div style="padding:1em; color:white;background-color:limegreen;width:5em">Success!</div>
Next: When you're ready, create a migration with <span style="color:limegreen">php bin/console make:migration</span>
root:/var/www#
</pre></div>
<p>OK that's quite cool. It also does some auto-complete for me too:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh8yzkZGceU0LmIGTjq3fxkKSywllYZ4tCCRmMqQqdrjYZMppkT_4gNwI_yfcfPGFJ3e93VgsTz_10iBTVMJZ81ZNjPTerOzImlUK_2fVXlEUx5mT5aBV9Zigb2y7OYEK2mBM0l7HXaQf4c6Svi-pfDd6dZf6Zd5agcahWLJXPhRH9FVPDU4C8cAMuHw/s1600/oneToMany_autocomplete.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="189" data-original-width="620" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh8yzkZGceU0LmIGTjq3fxkKSywllYZ4tCCRmMqQqdrjYZMppkT_4gNwI_yfcfPGFJ3e93VgsTz_10iBTVMJZ81ZNjPTerOzImlUK_2fVXlEUx5mT5aBV9Zigb2y7OYEK2mBM0l7HXaQf4c6Svi-pfDd6dZf6Zd5agcahWLJXPhRH9FVPDU4C8cAMuHw/s1600/oneToMany_autocomplete.png"/></a></div>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQBBq0mxmEeGnNKHbdpNNY-jbR_SKH1oaJUVMqtSNzM_1IV0ATyymchZA4Ucosj7xFr7Ow0nLPtlEIBK7z9bZUaXwlIXs6t6hsrImJyqba8j_UHo-msmw_oE-ltE371uaAaX740lKFZFr3ZfbunRvNv3EuSkxpqTeRismiQ2u4M2GUsccrMrqqalVp1g/s1600/comment_autocomplete.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="74" data-original-width="453" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQBBq0mxmEeGnNKHbdpNNY-jbR_SKH1oaJUVMqtSNzM_1IV0ATyymchZA4Ucosj7xFr7Ow0nLPtlEIBK7z9bZUaXwlIXs6t6hsrImJyqba8j_UHo-msmw_oE-ltE371uaAaX740lKFZFr3ZfbunRvNv3EuSkxpqTeRismiQ2u4M2GUsccrMrqqalVp1g/s1600/comment_autocomplete.png"/></a></div>
<p>That endeavour has added this to the <samp>Conference</samp> class:</p>
<pre class="source-code"><code>#[ORM\OneToMany(mappedBy: 'conference', targetEntity: Comment::class, orphanRemoval: true)]
private Collection $comments;
</code></pre>
<p>And this stuff too:</p>
<pre class="source-code"><code>/**
* @return Collection<int, Comment>
*/
public function getComments(): Collection
{
return $this->comments;
}
public function addComment(Comment $comment): self
{
if (!$this->comments->contains($comment)) {
$this->comments->add($comment);
$comment->setConference($this);
}
return $this;
}
public function removeComment(Comment $comment): self
{
if ($this->comments->removeElement($comment)) {
// set the owning side to null (unless already changed)
if ($comment->getConference() === $this) {
$comment->setConference(null);
}
}
return $this;
}
</code></pre>
<p>And to the <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.7/src/Entity/Comment.php" target="_blank"><samp>Comment</samp></a> class:</p>
<pre class="source-code"><code>#[ORM\ManyToOne(inversedBy: 'comments')]
#[ORM\JoinColumn(nullable: false)]
private ?Conference $conference = null;
</code></pre>
<hr class="narrow">
<h4>Migrating the Database</h4>
<p>OK I had wondered what this term I'd seen mentioned "migrations" was all about. It's how to apply the entity schema to the DB schema, by the sounds of it. Migrating the entity schema? Not sure that's the term I'd use: to me during a "migration" one moves from one place to another; but one <em>ends up</em> in the new place. It's applying the mapping, innit? Ah well: naming stuff is hard.</p>
<p>It'll be interesting to see if this works given I have not set up the DB exactly the way they wanted me to. Plus - peril - there is already data in it. Nothing ventured, nothing gained though right? Here goes:</p>
<div class="cliBox"><pre>
root:/var/www# symfony console make:migration
<div style="padding:1em; color:white;background-color:limegreen;width:5em">Success!</div>
Next: Review the new migration <span style="color:limegreen">"migrations/Version20230402161944.php"</span>
Then: Run the migration with <span style="color:limegreen">php bin/console doctrine:migrations:migrate</span>
See <span style="color:GoldenRod">https://symfony.com/doc/current/bundles/DoctrineMigrationsBundle/index.html</span>
root:/var/www#
</pre></div>
<p>Oh right, I'm getting ahead of myself. It's just prepped the script, not run anything yet.</p>
<p>This is what it generated (as <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.7/migrations/Version20230402161944.php" target="_blank"><samp>migrations/Version20230402161944.php</samp></a>):</p>
<pre class="source-code"><code>/**
* Auto-generated Migration: Please modify to your needs!
*/
final class Version20230402161944 extends AbstractMigration
{
public function getDescription(): string
{
return '';
}
public function up(Schema $schema): void
{
// this up() migration is auto-generated, please modify it to your needs
$this->addSql('CREATE SEQUENCE comment_id_seq INCREMENT BY 1 MINVALUE 1 START 1');
$this->addSql('CREATE SEQUENCE conference_id_seq INCREMENT BY 1 MINVALUE 1 START 1');
$this->addSql('
CREATE TABLE comment (
id INT NOT NULL,
conference_id INT NOT NULL,
author VARCHAR(255) NOT NULL,
text TEXT NOT NULL,
email VARCHAR(255) NOT NULL,
created_at TIMESTAMP(0) WITHOUT TIME ZONE NOT NULL,
photo_filename VARCHAR(255) DEFAULT NULL,
PRIMARY KEY(id)
)'
);
$this->addSql('CREATE INDEX IDX_9474526C604B8382 ON comment (conference_id)');
$this->addSql('COMMENT ON COLUMN comment.created_at IS \'(DC2Type:datetime_immutable)\'');
$this->addSql('
CREATE TABLE conference (
id INT NOT NULL,
city VARCHAR(255) NOT NULL,
year VARCHAR(4) NOT NULL,
is_international BOOLEAN NOT NULL,
PRIMARY KEY(id)
)
');
$this->addSql('
CREATE TABLE messenger_messages (
id BIGSERIAL NOT NULL,
body TEXT NOT NULL,
headers TEXT NOT NULL,
queue_name VARCHAR(190) NOT NULL,
created_at TIMESTAMP(0) WITHOUT TIME ZONE NOT NULL,
available_at TIMESTAMP(0) WITHOUT TIME ZONE NOT NULL,
delivered_at TIMESTAMP(0) WITHOUT TIME ZONE DEFAULT NULL,
PRIMARY KEY(id)
)
');
$this->addSql('CREATE INDEX IDX_75EA56E0FB7336F0 ON messenger_messages (queue_name)');
$this->addSql('CREATE INDEX IDX_75EA56E0E3BD61CE ON messenger_messages (available_at)');
$this->addSql('CREATE INDEX IDX_75EA56E016BA31DB ON messenger_messages (delivered_at)');
$this->addSql('CREATE OR REPLACE FUNCTION notify_messenger_messages() RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify(\'messenger_messages\', NEW.queue_name::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;');
$this->addSql('DROP TRIGGER IF EXISTS notify_trigger ON messenger_messages;');
$this->addSql('
CREATE TRIGGER notify_trigger
AFTER INSERT OR UPDATE
ON messenger_messages
FOR EACH ROW
EXECUTE PROCEDURE notify_messenger_messages();
');
$this->addSql('
ALTER TABLE comment
ADD CONSTRAINT FK_9474526C604B8382
FOREIGN KEY (conference_id)
REFERENCES conference (id)
NOT DEFERRABLE
INITIALLY IMMEDIATE
');
$this->addSql('DROP TABLE test');
}
public function down(Schema $schema): void
{
// this down() migration is auto-generated, please modify it to your needs
$this->addSql('CREATE SCHEMA public');
$this->addSql('DROP SEQUENCE comment_id_seq CASCADE');
$this->addSql('DROP SEQUENCE conference_id_seq CASCADE');
$this->addSql('CREATE TABLE test (id INT NOT NULL, value VARCHAR(50) NOT NULL)');
$this->addSql('ALTER TABLE comment DROP CONSTRAINT FK_9474526C604B8382');
$this->addSql('DROP TABLE comment');
$this->addSql('DROP TABLE conference');
$this->addSql('DROP TABLE messenger_messages');
}
}
</code></pre>
<p>Seems legit.</p>
<p>It's "interesting" that it's creating a few triggers in there. That <a href="https://www.postgresql.org/docs/current/sql-notify.html" target="_blank"><samp>pg_notify</samp> thing</a> looks interesting. I wonder what <samp>messenger_messages</samp> is?</p>
<p>OK <em>now</em> I'm sending all that to the DB:</p>
<div class="cliBox"><pre>
root:/var/www# symfony console doctrine:migrations:migrate
<span style="color:limegreen">WARNING!
You are about to execute a migration in database "db1primary"
that could result in schema changes and data loss.
Are you sure you wish to continue? (yes/no)</span> [<span style="color:goldenrod">yes</span>]:
>
<span style="color:limegreen">[notice] Migrating <span style="color:goldenrod">up</span> to <span style="color:goldenrod">DoctrineMigrations\Version20230402161944</span>
[notice] finished in <span style="color:goldenrod">462.3</span>ms, used <span style="color:goldenrod">20M</span> memory, <span style="color:goldenrod">1</span> migrations executed, <span style="color:goldenrod">15</span> sql queries</span>
<div style="padding:1em;color:black;background-color:limegreen;width:45em">[OK] Successfully migrated to version : DoctrineMigrations\Version20230402161944</div>
root:/var/www#
</pre></div>
<p>Blimey. So far: so good. Let's see what the DB has to say:</p>
<div class="cliBox"><pre>
root:/# psql -U user1primary -d db1primary
db1primary=# \dt
List of relations
Schema | Name | Type | Owner
--------+-----------------------------+-------+--------------
public | comment | table | user1primary
public | conference | table | user1primary
public | doctrine_migration_versions | table | user1primary
public | messenger_messages | table | user1primary
(4 rows)
</pre></div>
<div class="cliBox"><pre>
db1primary=# \d comment
Table "public.comment"
Column | Type | Collation | Nullable | Default
----------------+--------------------------------+-----------+----------+-------------------------
id | integer | | not null |
conference_id | integer | | not null |
author | character varying(255) | | not null |
text | text | | not null |
email | character varying(255) | | not null |
created_at | timestamp(0) without time zone | | not null |
photo_filename | character varying(255) | | | NULL::character varying
Indexes:
"comment_pkey" PRIMARY KEY, btree (id)
"idx_9474526c604b8382" btree (conference_id)
Foreign-key constraints:
"fk_9474526c604b8382" FOREIGN KEY (conference_id) REFERENCES conference(id)
</pre></div>
<div class="cliBox"><pre>
db1primary=# \d conference
Table "public.conference"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
id | integer | | not null |
city | character varying(255) | | not null |
year | character varying(4) | | not null |
is_international | boolean | | not null |
Indexes:
"conference_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "comment" CONSTRAINT "fk_9474526c604b8382" FOREIGN KEY (conference_id) REFERENCES conference(id)
</pre></div>
<p>It all seems fine! Good work.</p>
<hr class="narrow">
<p>The last bit is about doing stuff on the production DB which doesn't apply to what I'm doing, so I'm ignoring that. </p>
<p>And I think I will leave that here. With one thing or another that took me a while and it's Sunday afternoon (OK: evening now) and I wanna do something else.</p>
<p>For all my whining about annotations and ORM and nomenclature, I'm finding this stuff pretty polished. There's no boats being pushed out regarding complexity here, but it's nailing the simple stuff.</p>
<p>There's none of me own code in here, but I've committed and tagged it as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.7" target="_blank"><samp>1.7</samp></a> anyhow.</p>
<p>Oh, and Part 4 is done: <a href="https://blog.adamcameron.me/2023/04/php-symfony-working-through-symfony_7.html">PHP / Symfony: working through "Symfony: The Fast Track", part 4: not really about Symfony, this one</a>. Thought it's largely a waste of time. Maybe skip it.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-75043467460603459522023-04-01T18:19:00.003+00:002023-04-08T18:01:11.016+00:00Symfony / Doctrine / DBAL: convincing/configuring it to use a PrimaryReadReplicaConnection connection<p>G'day:</p>
<p>A while back I documented how to create/configure a <a href="https://github.com/doctrine/dbal/blob/3.6.1/src/Connections/PrimaryReadReplicaConnection.php" target="_blank">PrimaryReadReplicaConnection</a> connection in PHP. PrimaryReadReplicaConnection is the replacement for MasterSlaveConnection, which has been retired due to socially-insensitive nomenclature. This is all in "<a href="https://blog.adamcameron.me/2023/01/php-primaryreadreplicaconnection.html">PHP: PrimaryReadReplicaConnection - configuration / usage example</a>".</p>
<p>Today's exercise is to get one to work in my Symfony project (<a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.5" target="_blank">adamcameron/SymfonyFastTrack</a>).</p>
<p>Symfony's default DB connectivity is done via the <samp>DATABASE_URL</samp> environment variable which <strong>must</strong> be set (docs: <a href="https://symfony.com/doc/current/the-fast-track/en/8-doctrine.html#configuring-doctrine-orm" target="_blank">Configuring Doctrine ORM</a>). This is fine for simple situations, even though I personally think it's a daft way of handling connnection parameters: it's a bit "type couply" to using a URL to connect to a DB, which is not the only way of doing it. But falls flat pretty quickly once not in a simple situation. The problem is that the <samp>DATABASE_URL</samp> allows only for a single connection. It's gonna be pretty common to be using primary / replicas I think. I guess perhaps the Symfony thinking(?) is that this is better handled on a DB load balancer than in the app. However: I don't have one of those, and i have also seen enterprise-scale PHP-driven operations that also simply use a PrimaryReadReplicaConnection for this. I have a PrimaryReadReplicaConnection driver. I need to make it work. This is not well / clearly / at all documented.</p>
<p>Firstly: there is no escaping it. One needs to have <samp>DATABASE_URL</samp> set. And it needs to be valid. This is even if you remove references to it in config: the connection won't configure (and the application won't work) unless it exists. The Symfony docs say to use <samp>override_url</samp> to prevent this behaviour:</p>
<blockquote cite="https://symfony.com/bundles/DoctrineBundle/current/configuration.html#doctrine-dbal-configuration">
When specifying a url parameter, any information extracted from that URL will override explicitly set parameters unless override_url is set to true. An example database URL would be <samp>mysql://snoopy:redbaron@localhost/baseball</samp>, and any explicitly set driver, user, password and dbname parameter would be overridden by this URL. See the Doctrine <a href="https://www.doctrine-project.org/projects/doctrine-dbal/en/2.10/index.html" target="_blank">DBAL documentation</a> for more information.
</blockquote>
<cite><a href="https://symfony.com/bundles/DoctrineBundle/current/configuration.html#doctrine-dbal-configuration" target="_blank">Doctrine DBAL Configuration</a></cite>
<p>However that link to the Doctrine docs is obsolete, and that setting has been deprecated since 2.4 (we're on 2.9 now):</p>
<blockquote cite="https://github.com/doctrine/DoctrineBundle/blob/2.9.0/UPGRADE-2.4.md?plain=1#L9">
<pre>
UPGRADE FROM 2.3 to 2.4
=======================
Configuration
--------
* Setting the `host`, `port`, `user`, `password`, `path`, `dbname`, `unix_socket`
or `memory` configuration options while the `url` one is set has been deprecated.
* The `override_url` configuration option has been deprecated.
</pre>
</blockquote>
<cite><a href="https://github.com/doctrine/DoctrineBundle/blob/2.9.0/UPGRADE-2.4.md?plain=1#L9" target="_blank"><samp>DoctrineBundle/UPGRADE-2.4.md</samp></a></cite>
<p>OK. Great. Thanks for that. I <em>must</em> provide a <samp>DATABASE_URL</samp>. But any explicitly-set overrides I set in the actual connection config are ignored. And the functionality that used to permit me to actively say "FFS will you just do what yer told" (ie: <samp>override_url</samp>) has been deprecated. <em>And</em> the URL approach doesn't actually support the options I want to use, so it's completely useless to me. This seems pretty mickey mouse to me, from where I'm standing. But all right then; I'll play yer silly game.</p>
<p>After some trial end error (because there's no documentation that I can find), I came up with this (in <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/config/packages/doctrine.yaml#L4-L22" target="_blank"><samp>config/packages/doctrine.yaml</samp></a>):</p>
<pre class="source-code"><code>doctrine:
dbal:
default_connection: default
connections:
default:
wrapper_class: Doctrine\DBAL\Connections\PrimaryReadReplicaConnection
dbname: '%env(resolve:POSTGRES_PRIMARY_DB)%'
host: '%env(resolve:POSTGRES_PRIMARY_HOST)%'
port: '%env(resolve:POSTGRES_PRIMARY_PORT)%'
user: '%env(resolve:POSTGRES_PRIMARY_USER)%'
password: '%env(resolve:POSTGRES_PRIMARY_PASSWORD)%'
driver: pdo_pgsql
server_version: 15
charset: utf8
replicas:
replica1:
dbname: '%env(resolve:POSTGRES_REPLICA_DB)%'
host: '%env(resolve:POSTGRES_REPLICA_HOST)%'
port: '%env(resolve:POSTGRES_REPLICA_PORT)%'
user: '%env(resolve:POSTGRES_REPLICA_USER)%'
password: '%env(resolve:POSTGRES_REPLICA_PASSWORD)%'
charset: utf8
</code></pre>
<p>I have used the <samp>primary</samp> values in <samp>DATABASE_URL</samp> (in <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/docker/php/envVars.public#L11" target="_blank"><samp>docker/php/envVars.public</samp></a>):</p>
<pre class="source-code"><code># …
DATABASE_URL="postgresql://${POSTGRES_PRIMARY_USER}:${POSTGRES_PRIMARY_PASSWORD}@${POSTGRES_PRIMARY_HOST}:${POSTGRES_PRIMARY_PORT}/${POSTGRES_PRIMARY_DB}?serverVersion=15&charset=utf8"
</code></pre>
<p>To test that this actually works, and it doesn't just send everything to the <samp>primary</samp>, I have set up two completely separate DBs, with different credentials (and host, port, and even database name). These are separate PostgreSQL containers. I'll push the config to Github, but I won't repeat it here as it's really just a duplication of what I already had in this codebase. But here's links to the relevant bits of various files:</p>
<ul>
<li><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/docker/docker-compose.yml#L52-L98" target="_blank"><samp>docker/docker-compose.yml</samp></a></li>
<li><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/docker/postgres-primary/Dockerfile" target="_blank"><samp>docker/postgres-primary/Dockerfile</samp></a></li>
<li><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/docker/postgres-replica/Dockerfile" target="_blank"><samp>docker/postgres-replica/Dockerfile</samp></a></li>
</ul>
<p>It's important to note that in this experimentation the two DBs are completely separate, and there's no replication going on. I'm just testing the connection config is correctly switching between primary and replica, and it's easier to do when the databases have different data in them.</p>
<pre class="source-code"><code># primary.test
"id" "value"
101 "Test row 1"
102 "Test row 2"
104 "PRIMARY"
105 "TEST_VALUE"
106 "TEST_VALUE"
</code></pre>
<p>(You can see I've already been running some tests there)</p>
<pre class="source-code"><code># replica.test
101 "Test row 1"
102 "Test row 2"
103 "REPLICA"
</code></pre>
<pre class="source-code"><code>/** @testdox Writing to primary definitely does not impact the replica */
public function <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/tests/php/Functional/Doctrine/DefaultConnectionTest.php#L43-L76" target="_blank">testWritingToPrimaryDoesNotImpactReplica</a>()
{
$sqlForCount = "SELECT COUNT(1) AS count FROM test";
<span class="xr xrt" data-index="three-connections">$primaryConnection = $this->getPrimaryConnection();</span>
<span class="xr xrt" data-index="three-connections">$replicaConnection = $this->getReplicaConnection();</span>
<span class="xr xrt" data-index="need-mismatch">$initialPrimaryCount = $primaryConnection->executeQuery($sqlForCount)->fetchOne();</span>
<span class="xr xrt" data-index="need-mismatch">$initialReplicaCount = $replicaConnection->executeQuery($sqlForCount)->fetchOne();</span>
<span class="xr xrt" data-index="need-mismatch">$this->assertNotEquals(</span>
<span class="xr xrt" data-index="need-mismatch">$initialPrimaryCount,</span>
<span class="xr xrt" data-index="need-mismatch">$initialReplicaCount,</span>
<span class="xr xrt" data-index="need-mismatch">"Test aborted as the test requires the DBs to NOT be in sync (and they are)"</span>
<span class="xr xrt" data-index="need-mismatch">);</span>
<span class="xr xrt" data-index="three-connections">$defaultConnection = $this->getDefaultConnection();</span>
<span class="xr xrt" data-index="start-on-replica">$this->assertFalse($defaultConnection->isConnectedToPrimary(), "Connection did not start on replica");</span>
<span class="xr xrt" data-index="default-match-replica">$initialDefaultCount = $defaultConnection->executeQuery($sqlForCount)->fetchOne();</span>
<span class="xr xrt" data-index="default-match-replica">$this->assertEquals($initialDefaultCount, $initialReplicaCount, "Row count from default should match replica");</span>
<span class="xr xrt" data-index="insert-row">$defaultConnection->executeStatement("INSERT INTO test (value) VALUES (?)", ["TEST_VALUE"]);</span>
<span class="xr xrt" data-index="default-to-primary">$this->assertTrue(</span>
<span class="xr xrt" data-index="default-to-primary">$defaultConnection->isConnectedToPrimary(),</span>
<span class="xr xrt" data-index="default-to-primary">"Connection did not switch to primary after INSERT"</span>
<span class="xr xrt" data-index="default-to-primary">);</span>
$countFromDefault = $defaultConnection->executeQuery($sqlForCount)->fetchOne();
$countFromPrimary = $primaryConnection->executeQuery($sqlForCount)->fetchOne();
$countFromReplica = $replicaConnection->executeQuery($sqlForCount)->fetchOne();
<span class="xr xrt" data-index="default-match-primary">$this->assertEquals($countFromDefault, $countFromPrimary, "Row count from default should match primary");</span>
<span class="xr xrt" data-index="replica-no-change">$this->assertEquals($initialReplicaCount, $countFromReplica, "Row count from replica should not have changed");</span>
}</code></pre>
<p>
Here:</p>
<ul>
<li>I'm creating <span class="xr xrd u" data-index="three-connections">three connections, one each that directly queries the primary and replica DBs respectively, and one that is the app's default connection</span> (which is the <samp>PrimaryReadReplicaConnection</samp> one).</li>
<li>I get <span class="xr xrd u" data-index="need-mismatch">row counts from each, making sure they start off differently</span> (otherwise it'll be harder to see the difference, further down).</li>
<li>I then <span class="xr xrd u" data-index="default-match-replica">check the row count from the default connection matches the replica</span> (<span class="xr xrd u" data-index="start-on-replica">it should <em>start</em> using the replica connection</span>.</li>
<li>Then I <span class="xr xrd u" data-index="insert-row">insert a row using the default connection</span>. This has a dual effect:</li>
<li><span class="xr xrd u" data-index="default-to-primary">The default connection should now be pointing to the primary database</span>.</li>
<li><span class="xr xrd u" data-index="default-match-primary">So a row count from that connection should match the <em>primary</em> now</span>.</li>
<li><span class="xr xrd u" data-index="replica-no-change">And the replica count should be unchanged</span>.</li>
</ul>
<p>And <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.6/tests/php/Functional/Doctrine/DefaultConnectionTest.php#L78-L110" target="_blank">the helper functions</a>:</p>
<pre class="source-code"><code>private function getPrimaryConnection(): Connection
{
return DriverManager::getConnection([
"dbname" => getenv("POSTGRES_PRIMARY_DB"),
"user" => getenv("POSTGRES_PRIMARY_USER"),
"password" => getenv("POSTGRES_PRIMARY_PASSWORD"),
"host" => getenv("POSTGRES_PRIMARY_HOST"),
"port" => getenv("POSTGRES_PRIMARY_PORT"),
"driver" => "pdo_pgsql"
]);
}
private function getReplicaConnection(): Connection
{
return DriverManager::getConnection([
"dbname" => getenv("POSTGRES_REPLICA_DB"),
"user" => getenv("POSTGRES_REPLICA_USER"),
"password" => getenv("POSTGRES_REPLICA_PASSWORD"),
"host" => getenv("POSTGRES_REPLICA_HOST"),
"port" => getenv("POSTGRES_REPLICA_PORT"),
"driver" => "pdo_pgsql"
]);
}
public function getDefaultConnection(): PrimaryReadReplicaConnection
{
$kernel = new Kernel("test", true);
$kernel->boot();
$container = $kernel->getContainer();
return $container->get("doctrine.dbal.default_connection");
}
</code></pre>
<p>And that all worked, so that's good.</p>
<p>I'm gonna tag all this in GitHub as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.6" target="_blank"><samp>1.6</samp></a>, but I'm gonna roll-back the two different DBs and just point the replica to the same DB as the primary from now on. I want the <samp>PrimaryReadReplicaConnection</samp> connection in place in this project, but I don't wanna horse around keeping throw-away DBs actually in sync. That version is tagged as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.6.1" target="_blank"><samp>1.6.1</samp></a>.</p>
<p>And now <em>tomorrow</em> I can get on with what I <em>actually</em> wanted to be writing about today :-|</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-25228481611428089262023-03-28T23:58:00.007+00:002023-04-08T18:01:18.314+00:00PHP / Symfony: working through "Symfony: The Fast Track", part 2: creating a controller (eventually)<p>G'day:</p>
<h3>Sit. rep.</h3>
<p>OK so last time (<a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 1: preparation and pre-requisites (and not actually any Symfony!)</a>) I didn't make much Symfony progress, but I got my ducks (and my Docker containers) in a row, ready to start. I had worked through "<a href="https://symfony.com/doc/6.2/the-fast-track/en/1-tools.html" target="_blank">Symfony: The Fast Track › Checking your Work Environment</a>", and prepped as much as I could. Running <samp>symfony book:check-requirements</samp> yields this:</p>
<div class="cliBox"><pre>
root:/var/www# symfony book:check-requirements
<span style="color:green">[OK]</span> Git installed
<span style="color:green">[OK]</span> PHP installed version 8.2.4 (/usr/local/bin/php)
<span style="color:green">[OK]</span> PHP extension "xsl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "tokenizer" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "xml" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "redis" installed - <span style="color:Goldenrod">optional - needed only for chapter 31</span>
<span style="color:green">[OK]</span> PHP extension "amqp" installed - <span style="color:Goldenrod">optional - needed only for chapter 32</span>
<span style="color:green">[OK]</span> PHP extension "json" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "session" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "curl" installed - <span style="color:GoldenRod">optional - needed only for chapter 17 (Panther)</span>
<span style="color:green">[OK]</span> PHP extension "pdo_pgsql" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "mbstring" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "openssl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "sodium" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "zip" installed - <span style="color:GoldenRod">optional - needed only for chapter 17 (Panther)</span>
<span style="color:green">[OK]</span> PHP extension "gd" installed - <span style="color:GoldenRod">optional - needed only for chapter 23 (Imagine)</span>
<span style="color:green">[OK]</span> PHP extension "ctype" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "intl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> Composer installed
<span style="color:white;background-color:red">[KO]</span> Cannot find Docker, please install it https://www.docker.com/get-started
<span style="color:white;background-color:red">[KO]</span> Cannot find Docker Compose, please install it https://docs.docker.com/compose/install/
<span style="color:white;background-color:red">[KO]</span> Cannot find the npm package manager, please install it https://www.npmjs.com/
<div style="color:white;background-color:red;padding:1em;"> You should fix the reported issues before starting reading the book.</div>
root:/var/www#
</pre></div>
<p>I'm not bothered about the last three as I'm already in a Docker container, and Node.js is running in a separate container, pointing at the same application directory. Thinking about it, it's quite odd they require using Docker, but it doesn't occur to them to run the app itself in a container. I hope my approach doesn't some how unstick me somewhere along the way.</p>
<p>Right. Let's crack on with it. Next page…</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/2-project.html" target="_blank">Introducing the project</a></h3>
<p>Here they discuss "learning is doing", whilst not actually doing anything. But there's a diagram of where we get to in this project:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDOrhIl-2GsIkHwKHHJOWi7BlwHMcn7l38FmKi9yiakhCNCmeVs7oT9u7hIlnZzzDaQGWzDSdKXMaTXaFPU_ORZe5GckoeIHWzMcY2tsh4GcBGfiKxeoQ-q90CXmtVzNr025ykef8VsRUntdjmisJzFUC4TbBg_QAbCQ3HwSE-BxDZWGKeT0B5fjSC-Q/s7670/infrastructure.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="5049" data-original-width="7670" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDOrhIl-2GsIkHwKHHJOWi7BlwHMcn7l38FmKi9yiakhCNCmeVs7oT9u7hIlnZzzDaQGWzDSdKXMaTXaFPU_ORZe5GckoeIHWzMcY2tsh4GcBGfiKxeoQ-q90CXmtVzNr025ykef8VsRUntdjmisJzFUC4TbBg_QAbCQ3HwSE-BxDZWGKeT0B5fjSC-Q/s320/infrastructure.png"/></a></div>
<p>(this is no doubt © Symfony. I am using it in the context of critically evaluating their book, so I think this is "fair use". If someone from Symfony wishes to disagree: hit me up).</p>
<p>That's ambitious! Cool.</p>
<p>The also list how little code is needed to achieve this:</p>
<blockquote cite="https://symfony.com/doc/6.2/the-fast-track/en/2-project.html#looking-at-the-final-infrastructure-diagram">
<ul>
<li>20 PHP classes under <code translate="no" class="notranslate">src/</code> for the website;</li>
<li>550 PHP Logical Lines of Code (LLOC) as reported by <a href="https://github.com/sebastianbergmann/phploc" class="reference external" rel="external noopener noreferrer" target="_blank">PHPLOC</a>;</li>
<li>40 lines of configuration tweaks in 3 files (via attributes and YAML), mainly to configure the backend design;</li>
<li>20 lines of development infrastructure configuration (Docker);</li>
<li>100 lines of production infrastructure configuration (Platform.sh);</li>
<li>5 explicit environment variables.</li>
</ul>
</blockquote>
<p>
<cite><a href="https://symfony.com/doc/6.2/the-fast-track/en/2-project.html#looking-at-the-final-infrastructure-diagram">ibid.</a></cite>
</p>
<p>What's missing from that list? No tests. That's disappointing in 2023. I realise this book is about Symfony, but no code exercise in 2023 should normalise "no tests" as an approach. I will try to take a TDD approach with everything, if it seems viable (which it should be?).</p>
<p>There's instructions how to download / install / start the final working app. This seems a bit pre to me. I want to write the thing, not just look at it. "Learning is doing", I read somewhere, recently. I hope this step is not integral to the rest of the process cos I ain't doing it.</p>
<p>(Ah OK, a few paras on the author alludes to DIYing the code, so I guess this is just for reference. Good-o).</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/3-zero.html" target="_blank">Going from Zero to Production</a></h3>
<p>OK, so I now need to create a new Symfony app. This is done via a Symfony helper:</p>
<div class="cliBox"><pre>
symfony new guestbook --version=6.2 --php=8.1 --webapp --docker --cloud
</pre></div>
<p>I know from past experience ("<a href="https://blog.adamcameron.me/2021/01/creating-web-site-with-vuejs-nginx_13.html#needs-empty-directory">Part 6: Installing Symfony</a>" and "<a href="https://blog.adamcameron.me/2023/01/symfony-installing-in-my-php8-container.html#needs-empty-directory">Symfony: installing in my PHP8 container (for a second time, as it turns out)</a>") that Symfony whines if I try to create a new project in a directory that already has stuff in it, so I'll do what I did last time: install it in a temp directory then migrate stuff back into my project directory. I will also lift the test I used last time to verify it's working OK. <a href="https://blog.adamcameron.me/2023/01/symfony-installing-in-my-php8-container.html#symfony-test">The code is in that second article I linked to above</a>, I'll not repeat it here, but will <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/tests/php/Integration/InstallationTest.php#L43" target="_blank">link to it in the repo once I've tagged it</a>.</p>
<p>I also know that Symfony uses a bad namespace for its apps (<a href="https://blog.adamcameron.me/2023/01/symfony-getting-rid-of-app-namespace.html">Symfony: getting rid of App namespace and using a well-formed one</a>), so I'm going to sort that out straight away too. I'll not repeat any of the steps to make these changes and get that test passing, unless there's anything different I need to do this time. It's well-documented in that earlier article.</p>
<p>Bear with me for a bit. Back soon …</p>
<p>I ran this in a shell within my PHP container:</p>
<div class="cliBox"><pre>root:/var/www# cd /var/tmp
root:/var/tmp# symfony new SymfonyFastTrack --version=6.2 --php=8.2 --webapp --docker --cloud
</pre></div>
<p>I then copied the ensuing <samp>/var/tmp/SymfonyFastTrack</samp> (source) directory down to my host machine and did a file compare on that directory and my existing <samp>SymfonyFastTrack</samp> (destination) directory, doing the following:</p>
<ul>
<li>Where the source directory had files not in destination, I simply copied them across.</li>
<li>Where there were clear additions to a file for the new Symfony app, I just merged it (sometimes by hand) into my own version of the file. EG this diff of the two versions of <samp>composer.json</samp>:<br><br><div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6zqrIn1M3SpOVvWiS4wmY0o3IKU7OvCUA3OqvnWCnkDX7DMOEQkhj_4tfQmFV4zVmTQKmdv39UVv2KZ0oi01wiAUKfiP3AfL8iVDCyV-CFBKayAoRKwB8-mknb_gFPOrv5wGlls7TeuizzjJDRjOsKWjT0yo-fafZRwZP1q8fdl8vf8pmDX70YvplZA/s1211/composer-json-diff.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="320" data-original-height="1211" data-original-width="979" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6zqrIn1M3SpOVvWiS4wmY0o3IKU7OvCUA3OqvnWCnkDX7DMOEQkhj_4tfQmFV4zVmTQKmdv39UVv2KZ0oi01wiAUKfiP3AfL8iVDCyV-CFBKayAoRKwB8-mknb_gFPOrv5wGlls7TeuizzjJDRjOsKWjT0yo-fafZRwZP1q8fdl8vf8pmDX70YvplZA/s320/composer-json-diff.png"/></a></div></li>
<li>There was a <samp>docker-compose.yml</samp> and <samp>docker-compose-override.yml</samp> plonked into the root directory. These added containers for a stub/fake STMP server and a "BlackFire" server (whatever that is). I lifted the relevant bits of that out and put it in my own <samp>docker/docker-compose.yml</samp> file and deleted the ones in the root.</li>
<li>I followed the instructions about setting up a Blackfire account, got the various keys I needed and stuck them into a <samp>docker/blackfire/envVars.private</samp> (not source controlled) file which I got the <samp>docker/docker-compose.yml</samp> to load into the Blackfire container.</li>
<li>Symfony hard-codes its <samp>APP_SECRET</samp> value into the codebase. If it's supposed to be a secret then treat it that way: I lifted it out and set an env var for it (via <samp>docker/php/envVars.private</samp> (again, not source controlled)).</li>
<li>I set Symfony's <samp>DATABASE_URL</samp> as an environment variable instead of Symfony's config files. No reason to really, except every component of it (host, user, password, DB etc) came from adjacent environment variables, so I figured I'd keep it all in one place.</li>
<li>The <samp>/config/packages/doctrine.yaml</samp> file was setting a <samp>db_suffix</samp> value which had the effect of the Connection object trying to connect to a DB called <samp>db1test</samp> (it's just <samp>db1</samp>. I have no reason to add suffixes to my DB name because of the environment). I'm not sure why this would be out-of-the-box behaviour but… I suspect this is just one more example of me going "please stop trying to be helpful, Symfony", and I should get used to it. Oh yeah, this surfaced because the tests put the <samp>APP_ENV</samp> into <samp>test</samp> mode, and I have integration tests (see below) hitting the DB. Yay for tests.</li>
</ul>
<p>I also wrote a few tests; or, as I said above: lifted them from another project ([during proofreading]: I also said I wasn't going to repeat them here… hrm…):</p>
<ul>
<li>To test Symfony came up OK:<br><br>
<pre class="source-code"><code>/** @testdox It serves the Symfony welcome page after installation */
public function <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/tests/php/Integration/InstallationTest.php#L43" target="_blank">testSymfonyWelcomeScreenDisplays</a>()
{
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => "http://host.docker.internal:8062/",
CURLOPT_RETURNTRANSFER => 1
]);
$response = curl_exec($ch);
if (curl_error($ch)) {
$this->fail(sprintf("curl failed with [%s]", curl_error($ch)));
}
$this->assertEquals(Response::HTTP_NOT_FOUND, curl_getinfo($ch, CURLINFO_HTTP_CODE));
$document = new \DOMDocument();
$document->loadHTML($response, LIBXML_NOWARNING | LIBXML_NOERROR);
$xpathDocument = new \DOMXPath($document);
$hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
$this->assertCount(1, $hasTitle, "Could not find title 'Welcome to Symfony!'");
}
</code></pre>
</li>
<li>To test its console also runs OK:<br><br>
<pre class="source-code"><code>/** @testdox It can run the Symfony console in a shell */
public function <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/tests/php/Integration/InstallationTest.php#L67" target="_blank">testSymfonyConsoleRuns</a>()
{
$appRootDir = dirname(__DIR__, 3);
exec("{$appRootDir}/bin/console --help", $output, $returnCode);
$this->assertEquals(0, $returnCode);
$this->assertNotEmpty($output);
}
</code></pre>
</li>
<li>And to test that <samp>DATABASE_URL</samp> got loaded into the Connection object in Symfony's DI container:
<pre class="source-code"><code>/** @testdox It has configured the Connection in the container with the correct DATABASE_URL */
public function <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/tests/php/Integration/DbTest.php#L26" target="_blank">testContainerConnection</a>()
{
$container = $this->getContainer();
$connection = $container->get("database_connection");
$result = $connection->executeQuery("SELECT version() AS version");
$this->assertStringStartsWith("PostgreSQL 15", $result->fetchOne());
}
</code></pre>
</li>
</ul>
<p>I had the first two tests in there before I started the app installation process - a nod to TDD - but the third one I wrote after the fact to make sure my messing around had worked.</p>
<p>All my tests still/now pass, so I'm ready for the next step…</p>
<p>The rest of the page discussing pushing the "nothing" I so far have to production, under the guise of "deploy early and often". I have no aspirations to ship this to production, it's just a learning exercise. I guess there could be vagaries of deploying in "Production mode", but there's not even enough here yet to worry about that I think. I'll skip this for now.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/4-methodology.html" target="_blank">Adopting a Methodology</a></h3>
<p>A page dedicated to reminding one to commit one's work. I'm beginning to think the author was being paid by the word ;-)</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/5-debug.html" target="_blank">Troubleshooting Problems</a></h3>
<p>This page has some stuff about the debugging bumpf Symfony has at the bottom of the page (in dev mode wth debug on):</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXgkz_IRY_FgnNoQ-1em7nRFeuZRoyypUENRTLISOVblhYH8hHtLRylbaeX2NW5dIoxv9brYxnkIaetV5bKul7E2KsiDb_thVlbIinBAKFGqYtlSdPvyR0CsTh14vQ131gS5j2SpIVHD1rDxDJDu0m4FNBKjX-bV6fBVp-LtpM3bbvk2RTr8b004hIQg/s1600/debug_bar.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="76" data-original-width="771" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXgkz_IRY_FgnNoQ-1em7nRFeuZRoyypUENRTLISOVblhYH8hHtLRylbaeX2NW5dIoxv9brYxnkIaetV5bKul7E2KsiDb_thVlbIinBAKFGqYtlSdPvyR0CsTh14vQ131gS5j2SpIVHD1rDxDJDu0m4FNBKjX-bV6fBVp-LtpM3bbvk2RTr8b004hIQg/s1600/debug_bar.png"/></a></div>
<p>Handy.</p>
<p>It also discusses different environment modes. EG in dev mode the default first-install index page 404s with:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCuqpL61i0V8_m_liNBgGNPUDk6AT8pYEqDKhU0hVcTo_EZ4MQwrNx7cL1f9lYj8w92zsTe54SpzismCkabktKe1sWpQWsjmYiAJINs58lHKl4cy78FE2Q0b94UVkAZi2V1qQZ5Nu0WYAanXauciSEscOwNoy_aKgf2CYaw1y497MnwShsaqTtulf9jg/s1034/dev_mode.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="573" data-original-width="1034" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCuqpL61i0V8_m_liNBgGNPUDk6AT8pYEqDKhU0hVcTo_EZ4MQwrNx7cL1f9lYj8w92zsTe54SpzismCkabktKe1sWpQWsjmYiAJINs58lHKl4cy78FE2Q0b94UVkAZi2V1qQZ5Nu0WYAanXauciSEscOwNoy_aKgf2CYaw1y497MnwShsaqTtulf9jg/s320/dev_mode.png"/></a></div>
<p>Switch to <samp>prod</samp> mode, and one gets this instead:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWZabC9mHQ0FH2UfZYhVFXgni93Z5sc8DgP2dgHdjQYtjvioouyRvDVq3yV_oGI0DRnetLMn8Wi1a15geqvp6I6Cg3aGuv6kg2YNZ1fHy_B3oCLYyNd7XnlQ7bmLLaC11GDRapBbkXkV7rETXkJZz5LvjJmR5ZNNdJjSdKrxLMmGv-DaHqz8p16VYLDA/s682/prod_mode.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="221" data-original-width="682" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWZabC9mHQ0FH2UfZYhVFXgni93Z5sc8DgP2dgHdjQYtjvioouyRvDVq3yV_oGI0DRnetLMn8Wi1a15geqvp6I6Cg3aGuv6kg2YNZ1fHy_B3oCLYyNd7XnlQ7bmLLaC11GDRapBbkXkV7rETXkJZz5LvjJmR5ZNNdJjSdKrxLMmGv-DaHqz8p16VYLDA/s320/prod_mode.png"/></a></div>
<p>This is most easily done in <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.4/.env#L18" target="_blank"><samp>.env</samp></a>:</p>
<pre class="source-code"><code style="color:gray"># In all environments, the following files are loaded if they exist,
# the latter taking precedence over the former:
#
# * .env contains default values for the environment variables needed by the app
# * .env.local uncommitted file with local overrides
# * .env.$APP_ENV committed environment-specific defaults
# * .env.$APP_ENV.local uncommitted environment-specific overrides
#
# Real environment variables win over .env files.
#
# …
APP_ENV=<span style="text-decoration: line-through">dev</span><span style="color:black">prod</span>
</code></pre>
<p>I left the guidance about the order of env file loading there, and I like how real env vars trump them all.</p>
<p>I tried to write functional tests loading the app in different modes and checking their output, but <span class="xr xrd u" data-index="kernel-env-no">this won't work</span>:</p>
<pre class="source-code"><code style="color:gray">/** @testdox Testing variations of APP_ENV */
class AppEnvTest extends WebTestCase
{
/** @testdox it shows the Welcome page for the / 404 if in dev mode */
public function testDevMode()
{
$client = static::createClient([
<span class="xr xrt" data-index="kernel-env-no"><span style="color:black">'environment' => 'dev',</span></span>
'debug' => false
]);
$client->request('GET', '/');
$this->assertResponseStatusCodeSame(Response::HTTP_NOT_FOUND);
<span style="color:black">$this->assertSelectorTextContains('h1', 'Welcome to Symfony');</span>
}
/** @testdox it shows a plain error for / 404 if in prod mode */
public function testProdMode()
{
$client = static::createClient([
<span class="xr xrt" data-index="kernel-env-no"><span style="color:black">'environment' => 'prod',</span></span>
'debug' => false
]);
$client->request('GET', '/');
$this->assertResponseStatusCodeSame(Response::HTTP_NOT_FOUND);
<span style="color:black">$this->assertSelectorTextContains('h1', 'Oops! An Error Occurred');</span>
}
}
</code></pre>
<p>I got an initially enigmatic error:</p>
<div class="cliBox"><pre>LogicException: You cannot create the client used in functional tests if the "framework.test" config is not set to true.</pre></div>
<p>But I tracked it back through the code and found this:</p>
<pre class="source-code"><code>protected static function <a href="https://github.com/symfony/framework-bundle/blob/v6.2.7/Test/KernelTestCase.php#L90-L101" target="_blank">getContainer</a>(): ContainerInterface
{
if (!static::$booted) {
static::bootKernel();
}
try {
return self::$kernel->getContainer()->get('test.service_container');
} catch (ServiceNotFoundException $e) {
throw new \LogicException('Could not find service "test.service_container". Try updating the "framework.test" config to "true".', 0, $e);
}
}
</code></pre>
<p>This is called as part of my <samp>createClient</samp> call in the test. Bottom line: the only mode one can do functional tests in is in test mode. I'm not entirely on board with this, if I'm honest. This is not Symfony application code; this is code they have specifically written for testing. If I wanna create a test client in other modes, that ought to be up to me, not Symfony. A problem I have with these large frameworks that tout themselves as being "opinionated" (as if this is a good thing) is that they start second-guessing stuff, and every second-guess they make closes a door on a situation that didn't occur to them. This is why I much preferred simpler frameworks like Silex (RIP), which just did routing, DI and exposed a coupla other things, but otherwise got out of my way. Never mind though, it's not a biggie.</p>
<hr class="narrow">
<h3><a href="https://symfony.com/doc/6.2/the-fast-track/en/6-controller.html" target="_blank">Creating a Controller</a></h3>
<p>Cool. Maybe I'll be able to write some code now.</p>
<p>Not yet.</p>
<p>They show some of the <samp>symfony console</samp> stuff which is very… thorough. Here's the list function showing all the options under the <samp>make</samp> functioanlity:</p>
<div class="cliBox"><pre>
root:/var/www# symfony console list make
Symfony 6.2.7 (env: <span style="color:goldenrod">dev</span>, debug: <span style="color:goldenrod">true</span>) <span style="color:#ffd700;background-color:#0057b7">#StandWith</span><span style="background-color:#ffd700;color:#0057b7">Ukraine</span> <a href="https://sf.to/ukraine" target="_blank">https://sf.to/ukraine</a>
Usage:
command [options] [arguments]
<span style="color:goldenrod">Options:</span>
<span style="color:green"> -h, --help</span> Display help for the given command. When no command is given display help for the list command
<span style="color:green"> -q, --quiet</span> Do not output any message
<span style="color:green"> -V, --version</span> Display this application version
<span style="color:green"> --ansi|--no-ansi</span> Force (or disable --no-ansi) ANSI output
<span style="color:green"> -n, --no-interaction</span> Do not ask any interactive question
<span style="color:green"> -e, --env=ENV</span> The Environment name. <span style="color:goldenrod">[default: "dev"]</span>
<span style="color:green"> --no-debug</span> Switch off debug mode.
<span style="color:green"> -v|vv|vvv, --verbose</span> Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
<span style="color:goldenrod">Available commands for the "make" namespace:</span>
<span style="color:green"> make:auth</span> Creates a Guard authenticator of different flavors
<span style="color:green"> make:command</span> Creates a new console command class
<span style="color:green"> make:controller</span> Creates a new controller class
<span style="color:green"> make:crud </span> Creates CRUD for Doctrine entity class
<span style="color:green"> make:docker:database</span> Adds a database container to your docker-compose.yaml file
<span style="color:green"> make:entity</span> Creates or updates a Doctrine entity class, and optionally an API Platform resource
<span style="color:green"> make:fixtures</span> Creates a new class to load Doctrine fixtures
<span style="color:green"> make:form</span> Creates a new form class
<span style="color:green"> make:functional-test</span> Creates a new test class
<span style="color:green"> make:message</span> Creates a new message and handler
<span style="color:green"> make:messenger-middleware</span> Creates a new messenger middleware
<span style="color:green"> make:migration</span> Creates a new migration based on database changes
<span style="color:green"> make:registration-form</span> Creates a new registration form system
<span style="color:green"> make:reset-password</span> Create controller, entity, and repositories for use with symfonycasts/reset-password-bundle
<span style="color:green"> make:serializer:encoder</span> Creates a new serializer encoder class
<span style="color:green"> make:serializer:normalizer</span> Creates a new serializer normalizer class
<span style="color:green"> make:stimulus-controller</span> Creates a new Stimulus controller
<span style="color:green"> make:subscriber</span> Creates a new event subscriber class
<span style="color:green"> make:test</span> [make:unit-test|make:functional-test] Creates a new test class
<span style="color:green"> make:twig-component</span> Creates a twig (or live) component
<span style="color:green"> make:twig-extension</span> Creates a new Twig extension with its runtime class
<span style="color:green"> make:unit-test</span> Creates a new test class
<span style="color:green"> make:user</span> Creates a new security user class
<span style="color:green"> make:validator</span> Creates a new validator and constraint class
<span style="color:green"> make:voter</span> Creates a new security voter class
root:/var/www#
</pre></div>
<p>I'm buggered if I know what any of that means, but… yeah cool. All that stuff.</p>
<p>It looks like I'm going to be creating my controller with a wziard instead of just… creating/editing some files. I guess it saves me copying and pasting some boilerplate. Let's see what it does:</p>
<div class="cliBox" id="maker-namespace"><pre>
root:/var/www# symfony console make:controller ConferenceController
<span style="color:goldenRod"> ! [NOTE] It looks like your app may be using a namespace other than "App".
!
! To configure this and make your life easier,
see: <a href="https://symfony.com/bundles/SymfonyMakerBundle/current/index.html#configuration" target="_blank">https://symfony.com/doc/current/bundles/SymfonyMakerBundle/index.html#configuration</a>
In Generator.php line 62:</span>
<div style="color:white;background-color:red;padding:1em;">Could not determine where to locate the new class "App\Controller\ConferenceController",
maybe try with a full namespace like "\My\Full\Namespace\ConferenceController"
</div>
<span style="color:green">make:controller [--no-template] [--] [<controller-class>]</span>
root:/var/www#
</pre></div>
<p>That's actually quite helpful, cool.</p>
<p>Looking at those docs, I need to create <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/config/packages/dev/maker.yaml" target="_blank"><samp>config/packages/dev/maker.yaml</samp></a>:</p>
<pre class="source-code"><code># config/packages/dev/maker.yaml
# create this file if you need to configure anything
maker:
# tell MakerBundle that all of your classes live in an
# Acme namespace, instead of the default App
# (e.g. Acme\Entity\Article, Acme\Command\MyCommand, etc)
root_namespace: 'adamcameron\symfonythefasttrack'
</code></pre>
<p>Let's see if that helps.</p>
<p>After I saw a post on Stack Overflow advising that after adding the file I also needed to clear the cache, it all worked fine:</p>
<div class="cliBox"><pre>
root@4b91bad4b422:/var/www# symfony console make:controller ConferenceController
<span style="color:blue">created</span>: src/Controller/ConferenceController.php
<span style="color:blue">created</span>: templates/conference/index.html.twig
<div style="color:white;background-color:limegreen;padding:1em;">Success!
</div>
Next: Open your new controller class and add some pages!
root@4b91bad4b422:/var/www#
</pre></div>
<p>It just occurs to me that I've let down Team TDD a bit here. I shoulda had a test ready for this lot! I should have read ahead to see what I was gonna end up with, when running this code. I've written a test before I <em>ran</em> the code though:</p>
<pre class="source-code"><code>/** @testdox Tests the functionality of ConferenceController */
class <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/tests/php/Functional/Controller/ConferenceControllerTest.php" target="_blank">ConferenceControllerTest</a> extends WebTestCase
{
/** @testdox The index action returns a successful response */
public function testIndex()
{
$client = static::createClient();
$client->request("GET", "/conference");
$this->assertResponseIsSuccessful();
$this->assertSelectorTextContains("h1", "Hello ConferenceController!");
}
}
</code></pre>
<p>(I checked <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/templates/conference/index.html.twig" target="_blank">the twig file</a> for the file content).</p>
<p>This doesn't help with the "red" part of red-green-refactor, but it's something at least.</p>
<p>Looking at the generated controller:</p>
<pre class="source-code"><code>class <a hreef="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/src/Controller/ConferenceController.php" target="_blank">ConferenceController</a> extends AbstractController
{
<span class="xr xrt" data-index="munged-routing">#[Route('/conference', name: 'app_conference')]</span>
public function index(): Response
{
return $this->render('conference/index.html.twig', [
'controller_name' => 'ConferenceController',
]);
}
}
</code></pre>
<p>I'm not a fan of <span class="xr xrd u" data-index="munged-routing">munging routing into controllers</span>, so I'm gonna move that back into the routing config (<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/config/routes.yaml#L7-L9" target="_blank"><samp>routes.yaml</samp></a>):</p>
<pre class="source-code"><code>controllers:
resource:
path: ../src/Controller/
namespace: adamcameron\symfonythefasttrack\Controller
type: attribute
conference:
resource: <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.5/config/routes/conference.yaml" target="_blank">routes/conference.yaml</a>
prefix: /conference/
</code></pre>
<pre class="source-code"><code>conference:
path: /
methods: [GET]
controller: adamcameron\symfonythefasttrack\Controller\ConferenceController::index
</code></pre>
<p>And here the test helps: I can refactor that routing config, and my test stays green!</p>
<p>Everything else seems fine.</p>
<p>The rest of the page messes with that controller to show using inline HTML in the controller (just <em>why?</em> would you even show that??), and output some debugging stuff. The debug stuff was interesting. One can put a <samp>dump</samp> in one's code and it will send the dump down to the browser, but different from <samp>var_dump</samp> it's not part of the main response, it's part of the debug bar (which I have only just clocked is a separate request, done via AJAX from the main response). Example (no link to this code as I'm not keeping it):</p>
<pre class="source-code"><code>public function index(Request $request): Response
{
dump($request);
return new Response("<html><body></body></html>");
}
</code></pre>
<p>(note that for the debug bar to show up, it needs to be an "HTML" response with a <samp>body</samp> element)</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWytnbsy6BnNpA0NaYE3YAufrCLQvTzVldEGlYqEnUgy2chg3Q4a3SEVLmVyVRSuy9yH3CTCNf1mW4G3tp8eGhyUcx8gJKAkAJ-9yZBlygJCsx3dx--SbXHEW8w_sfrE1IhEZrd40B3RKjHCywMmFF1lgd-6f64FaR-feUBMDBKfs-AFC6JV2nj_BbEQ/s579/dump.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="529" data-original-width="579" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWytnbsy6BnNpA0NaYE3YAufrCLQvTzVldEGlYqEnUgy2chg3Q4a3SEVLmVyVRSuy9yH3CTCNf1mW4G3tp8eGhyUcx8gJKAkAJ-9yZBlygJCsx3dx--SbXHEW8w_sfrE1IhEZrd40B3RKjHCywMmFF1lgd-6f64FaR-feUBMDBKfs-AFC6JV2nj_BbEQ/s320/dump.png"/></a></div>
<p>Cool.</p>
<hr class="narrow">
<p>Right, that was a lot of effort to make not much progress. There was quite a bit of googling in places when things didn't quite go how I expected, which I just summarised in the text. Anyway, I'm nackered so am gonna draw a line under this one.</p>
<p>All the code for this lot is tagged as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.5" target="_blank">1.5</a>.</p>
<p>Oh and I've done the next article in this series: <a href="https://blog.adamcameron.me/2023/04/php-symfony-working-through-symfony.html">PHP / Symfony: working through "Symfony: The Fast Track", part 3: doing some ORM / DB config</a>.<p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-91770222873909213002023-03-25T22:04:00.010+00:002023-04-08T18:01:25.197+00:00PHP / Symfony: working through "Symfony: The Fast Track", part 1: preparation and pre-requisites (and not actually any Symfony!)<p>G'day:</p>
<p>This new PHP app we're shifting in at work is gonna be running Symfony. The other bods on the dev team have been working through "<a href="https://symfony.com/doc/6.2/the-fast-track/en/index.html" target="_blank">Symfony: The Fast Track</a>", and I figured I pretty much ought to as well. So… here goes. It's a big "book", so am not sure how much of it I'll be able to get through in one go, so this will be a multi-part series. Or just one part if I get bored and move on to something else, as if often my wont on this blog. Hopefully it'll keep me motivated. The stuff I've seen from the other peeps on the team is interesting though, so who knows.</p>
<hr class="narrow">
<h3>Pre-requisites</h3>
<p>I need PHP and Nginx running so I'll install the config for those first. I've actually already done this, tested it, and tagged it up on GitHub as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.0" target="_blank">adamcameron/SymfonyFastTrack</a>. The baseline for this was <a href="https://github.com/adamcameron/php8/tree/1.0" target="_blank">version 1.0 of my PHP 8 repo</a>. This repo has moved on a bit since it was a bare baseline, but that v1.0 tag was round about right and with a minimum of unnecessary crap, so I started with that: cloned it, and slapped the files in my <samp>SymfonyFastTrack</samp> repo. I fiddled around with some stuff, but nothing of consequence. Anyhoo, there's an installation set of two Docker containers there: Nginx and PHP. They expose the website on <samp>http://localhost:8062/</samp> on yer host machine. All the instructions are in the <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.0/README.md" target="_blank">README.md</a> file for the repo.</p>
<p>The only other thing I've done thusfar is set up a project in IntelliJ to talk to my PHP container so it can run tests and code quality tooling, etc.</p>
<p>There is nothing to do with Symfony in this codebase yet. This is next. When I start reading that book. Which'll be now.</p>
<h3>Symfony: The Fast Track</h3>
<p>I'm gonna start from the front and work my way to the back.</p>
<h4>First coupla sections</h4>
<p>There are "<a href="https://symfony.com/doc/6.2/the-fast-track/en/preface.html" target="_blank">Acknowledgements</a>" and "<a href="https://symfony.com/doc/6.2/the-fast-track/en/0-intro.html" target="_blank">What is it about?</a>" up front. Necessary I suppose, but no real value beyond the author's obligation to write them.</p>
<h4><a href="https://symfony.com/doc/6.2/the-fast-track/en/1-tools.html" target="+_blank">Checking your Work Environment</a></h4>
<p>The interesting things in here is that they say up front I'm gonna need PostgreSQL and Node.js installed. Oh and the Symfony CLI (which I actually have installed in the PHP container already, as it turns out. I forgot about that).</p>
<p>Other than that the requirements for this - to start with - are:</p>
<ul>
<li>An IDE - IntelliJ in my case.</li>
<li>A terminal. Well, um, yeah obviously (isn't it?).</li>
<li>Git. Yup, done.</li>
<li>PHP. The book is written for 8,1, and I have 8.2 installed.</li>
<li>Some PHP extensions, some of which I <em>do not</em> have installed yet.</li>
<li>Composer. Yup, obviously.</li>
<li>Docker.</li>
</ul>
<p>One good thing the Symfony CLI does is have a check to see if I meet the book's prerequisites:</p>
<div class="cliBox"><pre>
root:/var/www# symfony book:check-requirements
<span style="color:green">[OK]</span> Git installed
<span style="color:green">[OK]</span> PHP installed version 8.2.4 (/usr/local/bin/php)
<span style="color:white;background-color:red">[KO]</span> PHP extension "xsl" <span style="color:white;background-color:red">not found</span>, please install it - <span style="color:GoldenRod">not found</span>
<span style="color:green">[OK]</span> PHP extension "tokenizer" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "xml" installed - <span style="color:GoldenRod">required</span>
<span style="color:black;background-color:Goldenrod">[KO]</span> PHP extension "redis" <span style="color:black;background-color:Goldenrod">not found</span> - <span style="color:Goldenrod">optional - needed only for chapter 31</span>
<span style="color:black;background-color:Goldenrod">[KO]</span> PHP extension "amqp" <span style="color:black;background-color:Goldenrod">not found</span> - <span style="color:Goldenrod">optional - needed only for chapter 32</span>
<span style="color:green">[OK]</span> PHP extension "json" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "session" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "curl" installed - <span style="color:GoldenRod">optional - needed only for chapter 17 (Panther)</span>
<span style="color:white;background-color:red">[KO]</span> PHP extension "pdo_pgsql" <span style="color:white;background-color:red">not found</span>, please install it - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "mbstring" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "openssl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "sodium" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "zip" installed - <span style="color:GoldenRod">optional - needed only for chapter 17 (Panther)</span>
<span style="color:black;background-color:Goldenrod">[KO]</span> PHP extension "gd" <span style="color:black;background-color:Goldenrod">not found</span> - <span style="color:GoldenRod">optional - needed only for chapter 23 (Imagine)</span>
<span style="color:green">[OK]</span> PHP extension "ctype" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "intl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> Composer installed
<span style="color:white;background-color:red">[KO]</span> Cannot find Docker, please install it https://www.docker.com/get-started
<span style="color:white;background-color:red">[KO]</span> Cannot find Docker Compose, please install it https://docs.docker.com/compose/install/
<span style="color:white;background-color:red">[KO]</span> Cannot find the npm package manager, please install it https://www.npmjs.com/
<div style="color:white;background-color:red;padding:1em;"> You should fix the reported issues before starting reading the book.</div>
root:/var/www#
</pre></div>
<p>It's not seeing Docker as I'm in a shell of a container already. But I'll get all that other stuff installed now.</p>
<hr class="narrow">
<h4>PHP extensions</h4>
<p>That was mostly googling how to install the various extensions that were missing, doing it manually in the running container to test they were working, then adding the steps to <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.1/docker/php/Dockerfile" target="_blank"><samp>docker/php/Dockerfile</samp></a> so they're there when I rebuild the containers. These were the additions to the Dockerfile:</p>
<pre class="source-code"><code style="color:gray">FROM php:8.2.4-fpm
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "zip", "unzip", "git", "vim"]
COPY php.ini /usr/local/etc/php/php.ini
COPY home/.bash_history /root/.bash_history
COPY home/.bashrc /root/.bashrc
COPY --from=composer:latest /usr/bin/composer /usr/local/bin/composer
RUN pecl install xdebug && docker-php-ext-enable xdebug
COPY conf.d/xdebug.ini /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
COPY conf.d/error_reporting.ini /usr/local/etc/php/conf.d/error_reporting.ini
RUN apt-get install -y libicu-dev && docker-php-ext-configure intl && docker-php-ext-install intl
RUN ["apt-get", "install", "-y", <span style="color:black">"libpng-dev", "libpq-dev", "librabbitmq-dev", </span>"libxslt1-dev", "libz-dev", "libzip-dev"]
RUN docker-php-ext-configure bcmath && docker-php-ext-install bcmath
<span style="color:black">RUN docker-php-ext-configure gd && docker-php-ext-install gd</span>
RUN docker-php-ext-configure opcache && docker-php-ext-install opcache
RUN docker-php-ext-configure pdo_mysql && docker-php-ext-install pdo_mysql
<span style="color:black">RUN docker-php-ext-configure pdo_pgsql && docker-php-ext-install pdo_pgsql
RUN docker-php-ext-configure xsl && docker-php-ext-install xsl</span>
RUN docker-php-ext-configure zip && docker-php-ext-install zip
<span style="color:black">RUN pear config-set php_ini /usr/local/etc/php/php.ini && pecl install amqp && docker-php-ext-enable amqp
RUN pear config-set php_ini /usr/local/etc/php/php.ini && pecl install redis && docker-php-ext-enable redis</span>
RUN curl -1sLf 'https://dl.cloudsmith.io/public/symfony/stable/setup.deb.sh' | bash
RUN ["apt-get", "install", "-y", "symfony-cli"]
WORKDIR /var/www
ENV COMPOSER_ALLOW_SUPERUSER 1
</code></pre>
<p>I also enforced as many of them as I could in <samp>composer.json</samp>, eg:</p>
<pre class="source-code"><code style="color:gray">{
"name" : "adamcameron/symfonythefasttrack",
"description" : "Symfony: The Fast Track",
"type" : "project",
"license" : "LGPL-3.0-only",
"require": {
"php" : "^8.2",
"ext-bcmath": "*",
"ext-ctype": "*",
"ext-curl": "*",
"ext-dom": "*",
<span style="color:black">"ext-gd": "*",</span>
"ext-iconv": "*",
…
</code></pre>
<p>Which worked fine for all of them except the <samp>amqp</samp>: <samp>composer validate</samp> just went "what the hell is that?" I could not find out why so I omitted that one. It <em>is</em> installed though:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDSXACdQMu_cQ6jtMKz9OIPoIvTOgSgMsP2ipIVlbQoIo4Ecd4SNGDfdQ_1LcxLF71TIx9-BOwqG1BUHGSlVXWn7jSTW0sKAfLoKBzx8E1hKRqSfzgWY_Lsv33rT-LrMSPNebqfzecwWr7u-HFv_Hh2BsiidF39aw0fbI6YTRnSkUZ2_0a-EwzMlxlKw/s964/ampq.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="370" data-original-width="964" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDSXACdQMu_cQ6jtMKz9OIPoIvTOgSgMsP2ipIVlbQoIo4Ecd4SNGDfdQ_1LcxLF71TIx9-BOwqG1BUHGSlVXWn7jSTW0sKAfLoKBzx8E1hKRqSfzgWY_Lsv33rT-LrMSPNebqfzecwWr7u-HFv_Hh2BsiidF39aw0fbI6YTRnSkUZ2_0a-EwzMlxlKw/s320/ampq.png"/></a></div>
<p>I googled around for what's different about the AMPQ extension compared to the others, and how I might handle it in composer.json, but drew a blank. I thought it might be because it's a PECL extension, not a "normal" one, but then I'd also expect to get the same grief with the <samp>redis</samp> one, which I didn't. It's no big thing, so I've just moved on.</p>
<p>Symfony is much happier now:</p>
<div class="cliBox"><pre>
root:/var/www# symfony book:check-requirements
<span style="color:green">[OK]</span> Git installed
<span style="color:green">[OK]</span> PHP installed version 8.2.4 (/usr/local/bin/php)
<span style="color:green">[OK]</span> PHP extension "xsl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "tokenizer" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "xml" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "redis" installed - <span style="color:Goldenrod">optional - needed only for chapter 31</span>
<span style="color:green">[OK]</span> PHP extension "amqp" installed - <span style="color:Goldenrod">optional - needed only for chapter 32</span>
<span style="color:green">[OK]</span> PHP extension "json" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "session" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "curl" installed - <span style="color:GoldenRod">optional - needed only for chapter 17 (Panther)</span>
<span style="color:green">[OK]</span> PHP extension "pdo_pgsql" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "mbstring" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "openssl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "sodium" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "zip" installed - <span style="color:GoldenRod">optional - needed only for chapter 17 (Panther)</span>
<span style="color:green">[OK]</span> PHP extension "gd" installed - <span style="color:GoldenRod">optional - needed only for chapter 23 (Imagine)</span>
<span style="color:green">[OK]</span> PHP extension "ctype" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> PHP extension "intl" installed - <span style="color:GoldenRod">required</span>
<span style="color:green">[OK]</span> Composer installed
<span style="color:white;background-color:red">[KO]</span> Cannot find Docker, please install it https://www.docker.com/get-started
<span style="color:white;background-color:red">[KO]</span> Cannot find Docker Compose, please install it https://docs.docker.com/compose/install/
<span style="color:white;background-color:red">[KO]</span> Cannot find the npm package manager, please install it https://www.npmjs.com/
<div style="color:white;background-color:red;padding:1em;"> You should fix the reported issues before starting reading the book.</div>
root:/var/www#
</pre></div>
<p>The changes to get this far are tagged as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.1" target="_blank">1.1</a>. I'll crack on with adding a Node.js and a PostgreSQL container to the mix now, before I turn the next page. I don't need the PostgreSQL one yet, but I might as well do them both now, whilst I'm dicking around with Docker.</p>
<hr class="narrow">
<h4>Node.js</h4>
<p>There was not much do doing this. I think. I had <a href="https://github.com/adamcameron/nodejs/tree/1.0.6" target="_blank">an old scratch Node.js repo I created a while back</a>, and I know that works, so I co-opted the bits I wanted from that and that was it really.</p>
<p><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.2/docker/nodejs/Dockerfile" target="_blank"><samp>Dockerfile</samp></a>:</p>
<pre class="source-code"><code>FROM node:19-bullseye
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "vim"]
COPY home/.bashrc /root/.bashrc
COPY home/.bash_history /root/.bash_history
COPY home/.vimrc /root/.vimrc
WORKDIR /usr/share/nodejs/
</code></pre>
<p>Most of that is just puff to make my life easier when I'm ferretting around in the container shell. The only important bit is the <samp>FROM</samp> bit. Which seems to be the latest Debian-based image.</p>
<p><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.2/docker/docker-compose.yml#L28-L37" target="_blank"><samp>docker-compose.yml</samp></a> (just the relevant bit):</p>
<pre class="source-code"><code>nodejs:
build:
context: nodejs
dockerfile: Dockerfile
stdin_open: true
tty: true
volumes:
- ..:/usr/share/nodejs/
</code></pre>
<p>Nowt interesting there either.</p>
<p><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.2/package.json" target="_blank"><samp>package.json</samp></a>:</p>
<pre class="source-code"><code>{
"name": "symfony-the-fast-track",
"description": "Symfony: The Fast Track requires Node.js for some reason. So here it is.",
"version": "1.0.0",
"author": "Adam Cameron",
"license": "GPL-3.0-or-later",
"devDependencies": {
"chai": "^4.3.7",
"mocha": "^10.2.0"
},
"scripts": {
"test": "mocha tests/nodejs/**/*.js"
}
}
</code></pre>
<p>And it wouldn't be me if I didn't have a test (<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.2/tests/nodejs/integration/NodeTest.js" target="_blank"><samp>NodeTest.js</samp></a>):</p>
<pre class="source-code"><code>let chai = require("chai");
let expect = chai.expect;
describe("Test Node.js", () => {
it("has the expected version", () => {
expect(process.version).to.be.match(/v19\.\d+\.\d+/);
})
});
</code></pre>
<p>And having rebuild the containers and shelled into the <samp>nodejs</samp> one, I just did <samp>npm install</samp>, and my test was runnable:</p>
<div class="cliBox"><pre>
root:/usr/share/nodejs# npm test
> symfony-the-fast-track@1.0.0 test
> mocha tests/nodejs/**/*.js
Test Node.js
<span style="color:gray">✔ has the expected version
<span style="color:limegreen">1 passing</span> (4ms)</span>
root:/usr/share/nodejs#
</pre></div>
<p>I'm content that's doing its job.</p>
<p>One other thing I needed to do was to rearrgange the <samp>tests</samp> subdirectory hierarchy a bit:</p>
<div class="cliBox"><pre>
/usr/share/nodejs# tree tests
tests
|-- nodejs
| `-- integration
| `-- NodeTest.js
`-- php
|-- Integration
| |-- InstallationTest.php
| `-- PhpTest.php
|-- Unit
`-- bootstrap.php
5 directories, 4 files
root:/usr/share/nodejs#
</pre></div>
<p>I did not want the PHP tests munged in with the JS tests. Obvs I re-ran the PHP tests after I rehomed them. They were fine.</p>
<p>I've tagged all that as <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.2" target="_blank">1.2</a></p>
<hr class="narrow">
<h4>PostgreSQL</h4>
<p>Oddly, <a href="https://github.com/adamcameron/lucee_and_postgres/tree/1.0" target="_blank">I have another project with a PostgreSQL container in it</a>. So I'm copying that too.</p>
<p><a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/postgres/Dockerfile"><samp>Dockerfile</samp></a>:</p>
<pre class="source-code"><code>FROM postgres:15-bullseye
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "vim"]
COPY postgres/home/.bash_history /root/.bash_history
COPY shared/home/.bashrc /root/.bashrc
COPY shared/home/.vimrc /root/.vimrc
COPY postgres/docker-entrypoint-initdb.d/ /docker-entrypoint-initdb.d/
EXPOSE 5432
</code></pre>
<p>As with the Node.js one: a bunch of housekeeping there. I'm running a script (<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/postgres/docker-entrypoint-initdb.d/1.createAndPopulateTestTable.sql" target="_blank"><samp>1.createAndPopulateTestTable.sql</samp></a>) to install some data at least:</p>
<pre class="source-code"><code>CREATE TABLE test (
id int GENERATED ALWAYS AS IDENTITY (START WITH 101),
value VARCHAR(50) NOT NULL
);
INSERT INTO test (value)
VALUES
('Test row 1'),
('Test row 2')
;
</code></pre>
<p>The <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/docker-compose.yml#L52-L79" target="_blank"><samp>docker-compose.yml</samp></a> bit for this one is more interesting:</p>
<pre class="source-code"><code> postgres:
<span class="xr xrt" data-index="build-context">build:</span>
<span class="xr xrt" data-index="build-context">context: .</span>
<span class="xr xrt" data-index="build-context">dockerfile: postgres/Dockerfile</span>
env_file:
<span class="xr xrt" data-index="envVars-public">- shared/envVars.public</span>
<span class="xr xrt" data-index="envVars-private">- shared/envVars.private</span>
ports:
- "5432:5432"
volumes:
- postgresData:/var/lib/postgresql/data
stdin_open: true
tty: true
<span class="xr xrt" data-index="network">networks:</span>
<span class="xr xrt" data-index="network">backend:</span>
<span class="xr xrt" data-index="network">aliases:</span>
<span class="xr xrt" data-index="network">- database.backend</span>
volumes:
postgresData:
<span class="xr xrt" data-index="network">networks:</span>
<span class="xr xrt" data-index="network">backend:</span>
<span class="xr xrt" data-index="network">driver: "bridge"</span>
</code></pre>
<p>I need to pass some env vars into the containers now, including a password. I've put that in <samp class="xr xrd u" data-index="envVars-private">shared/envVars.private</samp>, and have excluded it from source control (<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/shared/.gitignore" target="_blank"><samp>docker/shared/.gitignore</samp></a>):</p>
<pre class="source-code"><code>envVars.private</code></pre>
<p>The other env vars don't need securing (<a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/shared/envVars.public" target="_blank"><samp class="xr xrd u" data-index="envVars-public">docker/shared/envVars.public</samp></a>)</p>
<pre class="source-code"><code>POSTGRES_DB=db1
POSTGRES_USER=user1
</code></pre>
<p>I've also added an explicit <span class="xr xrd u" data-index="network">network</span> now (not for any specific reason).</p>
<p>You might note I've also changed the <span class="xr xrd u" data-index="build-context">build context</span> here. I've done this across the board. This is because all th containers have the same <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/shared/home/.bashrc" target="_blank"><samp>.bashrc</samp></a> and <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/docker/shared/home/.vimrc" target="_blank"><samp>.vimrc</samp></a> requirements, so instead of having image-specific ones (eg: <samp>php/home/.bashrc</samp> and <samp>nginx/home/.bashrc</samp> having the same content), I've put them in that <samp>shared</samp> directory adjacent to the service-specific directories (I'm just showing the PHP and Postgres ones here; the Nginx and Node.js ones are equivalent):</p>
<div class="cliBox"><pre>
root:/var/www/docker# tree -a
.
// …
|-- php
| |-- Dockerfile
| |-- conf.d
| | |-- error_reporting.ini
| | `-- xdebug.ini
| |-- home
| | `-- .bash_history
| `-- php.ini
|
|-- postgres
| |-- Dockerfile
| |-- docker-entrypoint-initdb.d
| | `-- 1.createAndPopulateTestTable.sql
| `-- home
| `-- .bash_history
`-- shared
|-- .gitignore
|-- envVars.private
|-- envVars.public
`-- home
|-- .bashrc
`-- .vimrc
</pre></div>
<p>I've also added a test in the PHP container to make sure it can reach the DB:</p>
<pre class="source-code"><code>namespace adamcameron\symfonythefasttrack\tests\Integration;
use Doctrine\DBAL\Connection;
use Doctrine\DBAL\DriverManager;
use PHPUnit\Framework\TestCase;
use stdClass;
/** @testdox DB tests */
class <a href="https://github.com/adamcameron/SymfonyFastTrack/blob/1.3/tests/php/Integration/DbTest.php" target="_blank">DbTest</a> extends TestCase
{
/** @testdox It can connect to the DB using DBAL */
public function testDbalConnection()
{
<span class="xr xrt" data-index="version-check">$connection = $this->getDbalConnection();</span>
<span class="xr xrt" data-index="version-check">$result = $connection->executeQuery("SELECT version() AS version");</span>
<span class="xr xrt" data-index="version-check">$this->assertStringStartsWith("PostgreSQL 15", $result->fetchOne());</span>
}
private function getConnectionParameters() : stdClass
{
return (object) [
"host" => "database.backend",
"port" => "5432",
"database" => getenv("POSTGRES_DB"),
"username" => getenv("POSTGRES_USER"),
"password" => getenv("POSTGRES_PASSWORD")
];
}
private function getDbalConnection() : Connection
{
$parameters = $this->getConnectionParameters();
return DriverManager::getConnection([
"dbname" => $parameters->database,
"user" => $parameters->username,
"password" => $parameters->password,
"host" => $parameters->host,
"port" => $parameters->port,
"driver" => "pdo_pgsql"
]);
}
}
</code></pre>
<p>It simply <span class="xr xrd u" data-index="version-check">checks the major version of the DB</span>. If it can get that: it's connected enough.</p>
<p>I did not do a similar test in the Node.js container as I don't know that it will need to talk to the DB. I am guessing it's only going to be used to do client-side code building and crap like that. If I need this later: I'll do it later.</p>
<p>Once I rebuild the containers, that new test runs:</p>
<div class="cliBox"><pre>
root:/var/www# composer test
> phpunit tests --display-deprecations
PHPUnit 10.0.18 by Sebastian Bergmann and contributors.
Runtime: PHP 8.2.4 with Xdebug 3.2.1
Configuration: /var/www/phpunit.xml.dist
... 3 / 3 (100%)
Time: 00:05.572, Memory: 10.00 MB
DB tests
<span style="color:gray">✔</span> It can connect to the DB using DBAL
Tests of the whole installation
<span style="color:gray">✔</span> Nginx is serving PHP content on the host system
Tests of the PHP installation
<span style="color:gray">✔</span> It has the expected PHP version
<span style="background-color:limegreen;color:black">OK (3 tests, 5 assertions)</span>
Generating code coverage report in HTML format ... done [00:01.721]
</pre></div>
<p>And now… I'm sick of doing this for the day! I'm got a busy day tomorrow during the day, but a quiet Sunday evening might be a good time to start doing some actual Symfony stuff.</p>
<p>I've linked to all the files as I mention them, and there's four different tags in play in this article, but the current state of everything is here: <a href="https://github.com/adamcameron/SymfonyFastTrack/tree/1.3" target="_blank">1.3</a>.</p>
<p>I continue in the next part: <a href="https://blog.adamcameron.me/2023/03/php-symfony-working-through-symfony_28.html">PHP / Symfony: working through "Symfony: The Fast Track", part 2: creating a controller (eventually)</a>.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-40582956196424421692023-03-24T21:50:00.000+00:002023-03-24T21:50:28.799+00:00CFML / TestBox: spying on a method without mocking it<p>G'day:</p>
<p>Whilst looking for something else, I came across a ticket in TestBox's Jira system the other day that I had voted for a while back: <a href="https://ortussolutions.atlassian.net/jira/software/c/projects/TESTBOX/issues/TESTBOX-137" target="_blank">Ability to spy on existing methods: $spy()</a>. About a year or so ago I really had a need for this - hence finding the issue in there in the first place - sadly it was not around that the time, but I'm pretty pleased to see it's coming to TestBox 5.x soon (it still seems to be <a href="https://www.forgebox.io/view/testbox/version/5.0.0-snapshot" target="_blank">shipping to ForgeBox as a snapshot</a> for now, but it's installable).</p>
<p>I can't actually remember what I needed to be testing back when this first became relevant for me, but I work on a legacy application which was never written with testing in mind, and we need to spend a lot of time horsing about with mocking stuff out so as to be able to test code adjacent to stuff that absolutely won't run in a test environment. That's fine if one just wants to mock something away: TestBox has that nailed. But sometimes - <em>sometimes</em> - with legacy code one really needs to leave a piece of code running - its side-effects might be essential to part of the test for example - but also see what arguments it gets passed, probably because we're dealing with an method that hundreds of lines long, and the change we're making hits logic in multiple parts of it. In the perfect world we'd refactor code like that before we test it, but a) we don't reside in the perfect world; b) if you don't already have tests (we don't), then it's not "refactoring", it's "just changing shit". Sometimes we have to accept further technical debt and not "just change shit" to make testing easier. I really hate this, but it's a reality we have.</p>
<p>Anyway. The way one does this in a test framework is via <em>spying</em>. There's plenty of writing out there that differentiates between stubs, doubles, mocks and spies, but in my opinion that thinking is itself largely legacy. These days doing any of that is likely to be effected via a dedicated framework (like MockBox embedded in TestBox), and it's more a difference in usage rather than being anything really that different.</p>
<p>Let's have a look at some code.</p>
<p>For this exercise, I have this simple function to monkey about with:</p>
<pre class="source-code"><code>public string function reverseThisString(required string stringToReverse) {
return stringToReverse.reverse()
}
</code></pre>
<p>It reverses a string. Yeah I know CFML already has a function to do that (<strong>I'm using it here!</strong>), it's just something easy to use for some tests.</p>
<p>A baseline test here is simply to test it works:</p>
<pre class="source-code"><code>it("shows the reverseThisString working as a baseline", () => {
<span class="xr xrt" data-index="sut">sut = new SpyTest()</span>
result = sut.reverseThisString("G'day world")
expect(result).toBe("dlrow yad'G")
})
</code></pre>
<p>Yep. It works.</p>
<p>Oh, one thing to note in all these tests is that <samp>reverseThisString</samp> is <span class="xr xrd u" data-index="sut">actually within my test suite - <samp>SpyTest</samp></span> - so I'm actually instantiating an instance of the very class the test is running from (and then later mocking it and stuff). It's important to remember the instance of the class being used in the test run is not the same as the one I'm <em>testing</em> in the test run. if that makes sense.</p>
<p>Next let's demonstrate that <span class="xr xrd u" data-index="preparemock">preparing an object for mocking doesn't</span> actually disable an object's methods at all:</p>
<pre class="source-code"><code>component extends=BaseSpec {
function beforeAll() {
variables.mockbox = getMockBox()
}
function run() {
describe("Tests $spy function in TestBox", () => {
// …
it("shows how mocking an object does not impact its methods", () => {
sut = new SpyTest()
<span class="xr xrt" data-index="preparemock">mockbox.prepareMock(sut)</span>
result = sut.reverseThisString("G'day world")
<span class="xr xrt" data-index="still-works">expect(result).toBe("dlrow yad'G")</span>
})
// …
})
}
// …
}
</code></pre>
<p><samp>reverseThisString</samp> is still <span class="xr xrd u" data-index="still-works">doing it's thing</span>. It's not mocked.</p>
<p>And a quick look that a mocked object has a call log, but the call log <span class="xr xrd u" data-index="calllog-no-unmocked">doesn't include unmocked methods</span> (only <span class="xr xrd u" data-index="calllog-mocked">the mocked ones</span>):</p>
<pre class="source-code"><code>it("shows how an object's callLog does not include non-mocked methods", () => {
sut = new SpyTest()
mockbox.prepareMock(sut)
sut.$("mockMe")
sut.reverseThisString("G'day world")
sut.mockMe()
callLog = sut.$callLog()
<span class="xr xrt" data-index="calllog-mocked">expect(callLog).toHaveKey("mockMe")</span>
<span class="xr xrt" data-index="calllog-no-unmocked">expect(callLog).notToHaveKey("reverseThisString")</span>
})
</code></pre>
<p>Oh yeah, there's a <samp>mockMe</samp> method in there too:</p>
<pre class="source-code"><code>public void function mockMe() {
throw "Test is invalid: this should be mocked-out"
}
</code></pre>
<p>So this is the crux of it. Mocked methods have call logs, so we are in effect spying on them all the time. But unmocked methods: no.</p>
<p>No - to state the obvious, I hope - if one mocks a method, <span class="xr xrd u" data-index="mock-do-nowt">it does not do anything</span>:</p>
<pre class="source-code"><code>it("shows how mocking a method prevents it from executing", () => {
sut = new SpyTest()
mockbox.prepareMock(sut)
<span class="xr xrt" data-index="mock-do-nowt">sut.$("reverseThisString")</span>
result = sut.reverseThisString("G'day world")
<span class="xr xrt" data-index="mock-do-nowt">expect(isNull(result)).toBeTrue()</span>
})
</code></pre>
<p>Nuff said.</p>
<p>OK, so here's the solution: this new <samp>$spy</samp> functionality:</p>
<pre class="source-code"><code>it("shows how spying a method leaves it operational, and has a call log", () => {
sut = new SpyTest()
mockbox.prepareMock(sut)
<span class="xr xrt" data-index="spying">sut.$spy("reverseThisString")</span>
<span class="xr xrt" data-index="still-works">result = sut.reverseThisString("G'day world")</span>
<span class="xr xrt" data-index="still-works">expect(result).toBe("dlrow yad'G")</span>
callLog = sut.$callLog()
<span class="xr xrt" data-index="still-calllog">expect(callLog.reverseThisString[1][1]).toBe("G'day world")</span>
})
</code></pre>
<p><span class="xr xrd u" data-index="spying">Here I am just spying on my method</span>, not mocking it; so <span class="xr xrd u" data-index="still-works">when I call it: it still works</span>. But <span class="xr xrd u" data-index="still-calllog">I also have the call log</span>. Job done.</p>
<p>That's all there is to that. I hasten to add that this sort of feature is only useful to have <em>occasionally</em>, but sometimes with untestable legacy code it's a life-saver.</p>
<p>I would say that if you need to use this when testing <em>new</em> code that you're developing, yer likely doing something wrong. That said, if you have a real-world example where this is useful when testing well-written new code, please share.</p>
<p>And the code for this effort is here: <a href="https://github.com/adamcameron/cfml/blob/testbox_spy_testing/vendor/ortus/testbox/test/unit/spy/SpyTest.cfc" target="_blank"><samp>SpyTest.cfc</samp></a>.
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-6938706238401784132023-03-17T13:09:00.000+00:002023-03-17T13:09:05.830+00:00FAO ColdFusion users: you need to address a critical vuln in your system<p>G'day:</p>
<p>Just in case you happen to read this blog, but not Charlie Arehart's one, pls go over to his blog and read his article "<a href="https://www.carehart.org/blog/2023/3/17/coldfusion_march_2023_emergency_update" target="_blank">ColdFusion March 2023 emergency update, and what to do about it</a>", and follow-up.</p>
<p>This is serious, don't ignore it. I don't have time or inclination to look into it myself - I don't use ColdFusion any more - but figured I should do my bit to get the info out there.</p>
<p>It does not impact Lucee.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-90881401075213686382023-03-11T14:58:00.002+00:002023-04-08T18:04:38.441+00:00PHP / PHPUnit / TDD: unit testing abstract classes. Or not. <p>G'day:</p>
<p>One of my colleagues at work asked me about this, but it's a good topic to think about, so am gonna write about it here.</p>
<p>The question was pretty much as stated in the subject line there "if we have an abstract class… how do we go about unit testing that?". There's a coupla things to unpack here, one practical and one theoretical. I'll do the practical bit first.</p>
<h3>Practical</h3>
<p>I have got an abstract class (thanks to ChatGPT for writing the example code for me today, btw ;-))</p>
<pre class="source-code"><code>abstract class Shape
{
public function __construct(protected string $colour)
{
}
public function getColour(): string
{
return $this->colour;
}
abstract public function getArea();
}
</code></pre>
<p>And this will eventually have implementing classes like circles and squares and what not. The abstraction conceit being that all shapes have an area, but the algorithms for defining said area vary from shape to shape. That's easy to understand and pretty ubiquitous in "here's an example of abstract classes in action" situations. For now we're just testing the abstract class, so I'm not worrying about the implementations yet.</p>
<p>We want to test that <samp>getColour</samp> returns the shape's colour.</p>
<p>For a non-abstract class, we could do this:</p>
<pre class="source-code"><code>/** @testdox getColour returns the colour set by the constructor */
public function testGetColour()
{
$testColour = "red";
$shape = new Shape($testColour);
$actualColour = $shape->getColour();
$this->assertEquals($testColour, $actualColour);
}
</code></pre>
<p>Job done. Except <samp>Shape</samp> <em>is</em> an abstract class, so we get this instead:</p>
<div class="cliBox"><pre>
Error: Cannot instantiate abstract class adamcameron\php8\Shapes\Shape
/var/www/tests/Unit/Shapes/ShapeTest.php:15
</pre></div>
<p>All is not lost. Obviously this is well-trod ground and PHPUnit already deals with this sort of thing: We can use a partial mock to create an implemetation class at runtime:</p>
<pre class="source-code"><code>public function testGetColour()
{
$green = "karariki";
$shape = $this->getMockForAbstractClass(Shape::class, [$green]);
$actualColour = $shape->getColour();
$this->assertEquals($green, $actualColour);
}
</code></pre>
<blockquote cite="https://docs.phpunit.de/en/10.0/test-doubles.html?highlight=getMockForAbstractClass#getmockforabstractclass">
<h4><a href="https://docs.phpunit.de/en/10.0/test-doubles.html?highlight=getMockForAbstractClass#getmockforabstractclass" target="_blank">getMockForAbstractClass()</a></h4>
<p>The <samp>getMockForAbstractClass()</samp> method returns a mock object for an abstract class. All abstract methods of the given abstract class are mocked. This allows for testing the concrete methods of an abstract class.</p>
</blockquote>
<p>Easy. There's also a mock-builder variant of this too:</p>
<pre class="source-code"><code>public function testGetColourUsingMockBuilder()
{
$blue = "kikorangi";
$shape = $this
->getMockBuilder(Shape::class)
->setConstructorArgs([$blue])
->getMockForAbstractClass();
$actualColour = $shape->getColour();
$this->assertEquals($blue, $actualColour);
}
</code></pre>
<p>That's it, really. Original question answered.</p>
<hr class="narrow">
<h3>Theory</h3>
<p>The problem is that "how to test methods of an abstract class" begs the question. If we're doing TDD: how do we get to a point where we have an abstract class to test anyhow? It sounds to me like we're getting ahead of ourselves. I hasten to add the question from my team mate was a theoretical one: something that just popped into his head, and it was a good thing to know the answer for. But it's also good to reason the situation through from a TDD perspective.</p>
<p>Let's say the <em>end</em> of the story is "an abstract Shape class, and concrete classes for Square and Circle". But at the beginning of the story we had no classes at all, and no code. We had a requirement, which was probably along the lines of "we need to be able to get the colour of our Circle". Why is it a Circle and not a Shape? Because it's unlikely we're going to have a <em>real world</em> requirement that deals in abstract terms. It's more likely there'll be a concrete requirement to start with. But it's a good question: I'll think about that. We write our first test, for the Circle:</p>
<pre class="source-code"><code>/** @testdox getColour returns the Circle's colour */
public function testGetColour()
{
$orange = "karaka";
$circle = new Circle($orange, 1);
$actualColour = $circle->getColour();
$this->assertEquals($orange, $actualColour);
}
</code></pre>
<p>And once we see this failing, we implement what we need to get it to pass, which would be along these lines:</p>
<pre class="source-code"><code>class Circle
{
public function __construct(protected string $colour)
{
}
public function getColour(): string
{
return $this->colour;
}
}
</code></pre>
<p>Cool.</p>
<p>The next requirement comes along, which is that we need to get the circle's area. Fine, more of the same sort of thing:</p>
<pre class="source-code"><code>/** @testdocs getArea returns the Circle's area */
public function testGetArea()
{
$circle = new Circle("NOT_TESTED", 2);
$actualArea = $circle->getArea();
$this->assertEquals(pi() * 4, $actualArea);
}
</code></pre>
<p>And implementation:</p>
<pre class="source-code"><code style="color:grey">class Circle
{
public function __construct(private readonly string $colour<span style="color:black">, private readonly float $radius</span>)
{
}
// …
<span style="color:black">public function getArea(): float
{
return pi() * $this->radius ** 2;
}</span>
}
</code></pre>
<p>Now the twist comes in. The next requirement is "OK, sometimes the shapes will be squares instead of circles, but otherwise behave the same". Remember red-green-refactor here. After TDDing the Square's behaviour, we would end up with this implementation:</p>
<pre class="source-code"><code>class Square
{
public function __construct(private readonly string $colour, private readonly float $side)
{
}
public function getColour(): string
{
return $this->colour;
}
public function getArea(): float
{
return $this->side ** 2;
}
}
</code></pre>
<p>That's "red" & "green" done. Now for "refactor". Clearly we come to the conclusion that Circles and Squares are both Shapes; the colour behaviour is identical in both, but whilst both have the concept of "area", how it's derived is different, and the naming of the property that we use to derive the area - radius or side, respectively - also differs. <strong>This</strong> is when we decide we need our Shape abstract class. During refactoring. But the thing is during refactoring, the tests don't change. We extract the colour-handling into a base class, but leave the area handling in the implementation classes. We just make sure the base class says "I don't care how you do it, but you need to be able to answer the question 'what is your area'".</p>
<pre class="source-code"><code>abstract class Shape
{
public function __construct(protected string $colour)
{
}
public function getColour(): string
{
return $this->colour;
}
abstract public function getArea();
}
</code></pre>
<pre class="source-code"><code>class Circle extends Shape
{
public function __construct(string $colour, protected float $radius)
{
parent::__construct($colour);
}
public function getArea(): float
{
return pi() * $this->radius ** 2;
}
}
</code></pre>
<pre class="source-code"><code>class Square extends Shape
{
public function __construct(string $colour, protected float $side)
{
parent::__construct($colour);
}
public function getArea(): float
{
return $this->side ** 2;
}
}
</code></pre>
<p>We're not done refactoring yet. And this comes back to the requirement to test methods of an abstarct class. Currently in CircleTest and SquareTest we have these:</p>
<pre class="source-code"><code>public function testGetColour()
{
$orange = "karaka";
$circle = new Circle($orange, 1);
$actualColour = $circle->getColour();
$this->assertEquals($orange, $actualColour);
}
</code></pre>
<pre class="source-code"><code>public function testGetColour()
{
$red = "whero";
$square = new Square($red, 1);
$actualColour = $square->getColour();
$this->assertEquals($red, $actualColour);
}
</code></pre>
<p>Other than the colours I've chosen to use and one creates a Circle and one creates a Square, these are identical. As they should be as they're testing behaviour of their base class. So it <em>does</em> make a kind of sense to de-dupe this stuff, and push the one test up into <samp>ShapeTest</samp>. The test is the one from the first section of this article.</p>
<p>However I'm split either way on this. That we decided to use an abstract Shape class here is - IMO - "implementation detail", and we don't generally directly test implementation detail. So if we refactor those two tests, we might be catering to implementation detail too directly. I'm unsure. Also these tests happen to be very simple, so I think in a way this refactoring would be a case of "bad DRY" (see my earlier article "<a href="https://blog.adamcameron.me/2022/10/dry-dont-repeat-yourself.html" target="_blank">DRY: don't repeat yourself</a>"). Maybe if the tests were more complicated then there'd be a better case for de-duping the complexity ("good DRY"). I think in this case, either way would be A-OK. Personally I'm a pedant, and a bit "a place for everything, and everything in its place", so I'm gonna go ahead and do the refactoring, and have a unit test testing an abstract class.</p>
<p>The code for the tests is here: <a href="https://github.com/adamcameron/php8/tree/1.18/tests/Unit/Shapes" target="_blank"><samp>/tests/Unit/Shapes</samp></a>, and for the source code: <a href="https://github.com/adamcameron/php8/tree/1.18/src/Shapes" target="_blank"><samp>/src/Shapes</samp></a>.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-54869907499798276522023-03-10T20:32:00.003+00:002023-03-10T20:32:49.308+00:00CFML: ColdFusion 2023 is in public beta now<p>G'day:</p>
<p>As posted in the CFML Slack channel by Adobe ColdFusion star, Mark Takata, CF2023 is in public beta now. One can sign up for it here: <a href="https://www.adobeprerelease.com/beta/48C2D737-9CB4-445E-E39C-90CB6381919A" target="_blank">https://www.adobeprerelease.com/beta/48C2D737-9CB4-445E-E39C-90CB6381919A</a>.</p>
<p>I've not done much on it yet: I've tested some bugfixes I was interested in, and that's about it.</p>
<p>The first thing I did was to set up a Docker container running ColdFusion 2023 with a directory mapped into it from my host PC so I can run scratch code to test stuff. I prefer to do this than install stuff onto the host machine itself.</p>
<p>The project is on Github @ <a href="https://github.com/adamcameron/cf2023/tree/1.0" target="_blank">https://github.com/adamcameron/cf2023/tree/1.0</a>. The readme.md for it is:</p>
<blockquote>
<h3>CF2023 container</h3>
<p>This is the config for a minimal(-ish) Docker container running CF2023.</p>
<p>I've installed vim in it as well as I never don't need it, so why dick around not having it?</p>
<p>I add bash aliases for <samp>ll</samp> and <samp>cls</samp> as they're handy (sorry the latter is cos I'm a perennial Windows user. Shrug).</p>
<p>I've exposed the container's port <samp>8500</samp> as <samp>8523</samp> on the host.</p>
<p>I map the <samp>src</samp> directory of this project to the <samp>/app/src</samp> directory in the container. The webroot is <samp>/app</samp>. So you ought to be able to browse to <samp>http://localhost:8523/src/dumpServer.cfm</samp> to verify the install. And then, like, chuck other code in there.</p>
<p>I expose the log dir <samp>/opt/coldfusion/cfusion/logs</samp> as <samp>var/log/coldfusion/</samp>: I always find it helpful to have the logs right there to look at when I'm testing new stuff.</p>
<p>I copy the MySQL community driver into the container so one can make MySQL DSNs without fussing.</p>
<p>I've installed the CF debugger module as otherwise one can't switch on Robust Exception Handling, which is a hard requirement for dev, I think.</p>
<p>There's a shell script <samp>docker/rebuildContainer.sh</samp>. Use this to (re)build the container:</p>
<div class="cliBox"><pre>
~/cf2023/docker$ ./rebuildContainer.sh 123
</pre></div>
<p>All it does is remove any existing container, rebuild it, and bring it up.</p>
</blockquote>
<p>That's it. If this is useful to anyone getting up and running and being able to pitch-in testing ColdFusion 2023, then: cool. If not: shrug. Good on you for reading down to here though.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-57985601400547611212023-03-03T21:03:00.004+00:002023-03-06T10:51:34.447+00:00TIL: something new about regex processing that made me feel dumb<p>G'day:</p>
<p>I like to think I'm reasonably confident with my regex usage, indeed have in the past written at length on regex implementation and usage in CFML (summarised here: <a href="https://blog.adamcameron.me/2015/01/regular-expressions-in-cfml-link-summary.html">"Regular expressions in CFML" link summary</a>). </p>
<p>Today one of the denizens of the <a href="https://workingcode.dev/discord/" target="_blank">Working Code Podcast Discord channel</a> - Sean Callahan - popped a question into the "Code Help" subchannel, and discussion ensued. The question was innocuous:</p>
<blockquote>
<p>Why does this:</p>
<pre class="source-code"><code><cfscript>
str = REReplaceNoCase("AZGRRBCZCIQITYD", ".*", "X", "ALL");
WriteOutput(str);
</cfscript>
</code></pre>
<p>Return a single X? Testing on regexr.org gives me the matches that I would expect, which is any character except line breaks and matches all of them.</p>
</blockquote>
<p>I came to the discussion a bit later as I was busy having lunch, drinking beer and reading "<a href="https://pragprog.com/titles/tpp20/the-pragmatic-programmer-20th-anniversary-edition/" target="_blank">The Pragmatic Programmer</a>" at the pub; but clarified a bit: the expectation was that it should return "XXXXXXXXXXXXXXX", not just "X". This is fine, he just needed to tweak his pattern a bit to be <samp>"."</samp> rather than <samp>".*"</samp>: one char at a time, not all the chars at once. No mystery there.</p>
<p>However before he clarified I saw he'd mentioned testing the pattern behaviour on <a href="https://regexr.com/" target="_blank">https://regexr.com/</a>, and that it behaved differently from CFML with the same pattern. I figured "yeah JS vs CFML, but still, should be the same…", so ran some code in my browser console to verify what he was seeing:</p>
<div class="cliBox"><pre>
> "AZGRRBCZCIQITYD".replace(/.*/g,"X")
>- 'XX'
</pre></div>
<p>"Yeah see a single <samp>X</samp>… hang on WTF? <em>Two</em> <samp>X</samp>s???"</p>
<p>I ran the equivalent code in CFML:</p>
<div class="cliBox"><pre>
cf-cli>reReplace("AZGRRBCZCIQITYD", ".*", "X", "ALL")
X
</pre></div>
<p>Yeah that's what I expect. Now: my natural disposition is to assume CFML is doing something wrong when it differs from other systems, but I figured I should check elsewhere too.</p>
<div class="cliBox"><pre>
php > echo preg_replace("/.*/", "X", "AZGRRBCZCIQITYD");
XX
</pre></div>
<div class="cliBox"><pre>Welcome to Node.js v18.14.0.
> "AZGRRBCZCIQITYD".replace(/.*/g,"X")
'XX'
</pre></div>
<p> </p>
<div class="cliBox"><pre>irb(main):001:0> "AZGRRBCZCIQITYD".gsub(/.*/, "X")
=> "XX"
</pre></div>
<p>(That's Ruby)</p>
<p>And back to the ColdFusion REPL to call Java's <samp>replaceAll</samp> method on that string:</p>
<div class="cliBox"><pre>
cf-cli>s = "AZGRRBCZCIQITYD";
AZGRRBCZCIQITYD
cf-cli>s.replaceAll(".*", "X");
XX
</pre></div>
<p>Finally, thanks to Gavin's suggestion in the comments below, Perl:</p>
<div class="cliBox"><pre>
Perl> my $s ="AZGRRBCZCIQITYD"
AZGRRBCZCIQITYD
Perl> $s =~ s/.*/X/g
2
Perl> print "$s\n"
XX
1
</pre></div>
<p>Perl is the same as the others.</p>
<p>OK so <samp>XX</samp> is clearly the correct answer, and ColdFusion (and Lucee, I hasten to add) are getting it wrong. But <em>my</em> expectations matches CFML's, so why am <em>I</em> wrong?</p>
<p>Note that if one took the global flag off, then JS worked as I'd expect:</p>
<div class="cliBox"><pre>
> "AZGRRBCZCIQITYD".replace(/.*/,"X")
'X'
</pre></div>
<p>So it's clearly doing a second iteration, and that's turning up another replacement. But: the whole string has already been replaced. So… erm?</p>
<p>The original regex matches zero-or-more characters. If I change the regex to match one-or-more (which is probably what Sean should have been using in the first place, had he wanted to replace everything with one <samp>X</samp>), then I get the result I'd expect:</p>
<div class="cliBox"><pre>
> "AZGRRBCZCIQITYD".replace(/.+/g,"X")
'X'
</pre></div>
<p>So it's not doing two iterations there.</p>
<p>Then I clocked what was going on. After the first iteration matches and replaces all of <samp>"AZGRRBCZCIQITYD"</samp> with <samp>"X"</samp>, the second iteration in the initial example is… matching the residual empty string! This is why <samp>/.*/</samp> matches a second time and, <samp>/.+/</samp> doesn't.</p>
<p>This leaves me wondering how it's not still finding that empty string after the second and subsequent iterations though. I mean after matching the empty string the first time, there's still an empty string ready for the next time. And the time after that…</p>
<p>So I thought some more, and the way I've kinda explained it to myself is along these lines. A pseudocode algorithm:</p>
<ul>
<li>For the original "AZGRRBCZCIQITYD":</li>
<li>Starts at 0;</li>
<li>Matches from 0-15;</li>
<li>Replaces with "X".</li>
<li>Next iteration:</li>
<li>We're resuming at 15, which is different from 0, so do it again;</li>
<li>matches from 15-15;</li>
<li>replaces;</li>
<li>15 is the same as 15 so we're done here.</li>
<li>Exit.</li>
</ul>
<p>I doubt it's that, but that's a reasonable layperson's read of the situation I think. And I'm kinda happy that I worked through this exercise. All whilst having had three pints, btw ;-)</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-10099203509459208252023-02-19T22:50:00.004+00:002023-04-08T18:04:42.136+00:00PHP: looking at spatie/async some more<p>G'day:</p>
<p>I'm getting back to this <samp>spatie/async</samp> library today (see "<a href="https://blog.adamcameron.me/2023/02/php-looking-at-spatieasync.html">PHP: looking at <samp>spatie/async</samp></a>" for the first part of this).</p>
<p>Previously I have just been <a href="https://github.com/adamcameron/php8/blob/1.16/tests/Functional/SpatieAsync/AsyncTest.php#L24-L32" target="_blank">passing a callback to the <samp>add</samp> function</a> when adding a task to the pool: </p>
<pre class="source-code"><code>$pool->add(function () use ($connection, $i, $startTime) {
$result = $connection->executeQuery("CALL sleep_and_return(?)", [2]);
return sprintf(
"%d:%d:%d",
$i,
$result->fetchOne(),
microtime(true) - $startTime
);
});
</code></pre>
<p>This requires the task to be tightly coupled to the pool-handling, which is probably not what one wants to do. Instead, one can put the task logic into a Task class:</p>
<pre class="source-code"><code>namespace adamcameron\php8\Task;
use adamcameron\php8\tests\Integration\Fixtures\Database as DB;
use Doctrine\DBAL\Connection;
use Spatie\Async\Task;
class <a href="https://github.com/adamcameron/php8/blob/1.17/src/Task/SlowDbCallTask.php" target="_blank">SlowDbCallTask</a> extends Task
{
readonly private Connection $connection;
public function __construct(
readonly private int $i,
readonly private float $startTime
) {
}
public function configure()
{
$this->connection = DB::getDbalConnection();
}
public function run()
{
$result = $this->connection->executeQuery("CALL sleep_and_return(?)", [2]);
return sprintf(
"%d:%d:%d",
$this->i,
$result->fetchOne(),
microtime(true) - $this->startTime
);
}
}
</code></pre>
<p>So that's a bit nicer, especially when you see how the calling code in the test looks now:</p>
<pre class="source-code"><code>$pool->add(new SlowDbCallTask($i, $startTime));
</code></pre>
<p>One thing that took me a while to straighten out in my head was the way the <samp>configure</samp> method works / needs to be used. Initially, my implementation of the SlowDbCallTask class was a bit literal, and my constructor was thus:</p>
<pre class="source-code"><code>public function __construct(
readonly private Connection $connection,
readonly private int $i,
readonly private float $startTime
) {
}
</code></pre>
<p>This errored-out when I ran the test:</p>
<pre class="source-code"><code>Exception: Serialization of 'Closure' is not allowed
/var/www/vendor/spatie/async/src/Runtime/ParentRuntime.php:87
/var/www/vendor/spatie/async/src/Runtime/ParentRuntime.php:69
/var/www/vendor/spatie/async/src/Pool.php:140
/var/www/tests/Functional/SpatieAsync/TaskTest.php:20
</code></pre>
<p>I've run into this before: the object being used as the task handler needs to be serialised to get it into the PHP process that's running it, and not everything can be serialised. Less than ideal, but so be it. Then it occurred to me that I was being a div anyhow: I don't want to pass the Task's DB connection into it - there's no reason to - I can just initialise it in the Task's constructor, eg:</p>
<pre class="source-code"><code>public function __construct(
readonly private int $i,
readonly private float $startTime
) {
$this->connection = DB::getDbalConnection();
}
</code></pre>
<p>But same error, and same problem. When I handle this in the constructor, it's all being done in the calling code, so the same serialisation issue exists. I have to concede the penny did not drop without some googling, and I found <a href="https://github.com/spatie/async/issues/70#issuecomment-493377690" target="_blank">this helpful comment on an issue on GitHub</a>, which led me to <a href="https://github.com/spatie/async/issues/23#issuecomment-358938108" target="_blank">this other comment</a>:</p>
<blockquote cite="https://github.com/spatie/async/issues/23#issuecomment-358938108">
<p>It's important to remember that parallel processes are really separate processes. By using closures and a smart bootstrap, we're simulating the parent process as best as possible. There are however cases which won't work with that simple approach: not everything is serialisable.</p>
<p>That's where a <samp>Task</samp> comes in. Tasks allow more control over the bootstrap in the child process.</p>
<p>Start by taking a look at what the README says about tasks: <a href="https://github.com/spatie/async#working-with-tasks" target="_blank">https://github.com/spatie/async#working-with-tasks</a></p>
<p>The less objects and application things are passed to the child process, the better. […]</p>
</blockquote>
<p>Whilst it doesn't directly say it, this made me click that that <samp>config</samp> method is for (I was previously going "not sure why I need that"); the docs - linked to above - are of the sort that state <em>what</em> thing are, but not <em>why</em> they are they way they are, so don't directly explain this either. Basically if there's anything the Task will need that won't be serialisable: sling it in the <samp>config</samp> method. Makes complete sense to me now.</p>
<p>If one looks at the base Task class, we can see how it works:</p>
<pre class="source-code"><code>namespace Spatie\Async;
abstract class <a href="https://github.com/spatie/async/blob/1.5.5/src/Task.php" target="_blank">Task</a>
{
abstract public function configure();
abstract public function run();
public function __invoke()
{
$this->configure();
return $this->run();
}
}
</code></pre>
<hr class="narrow">
<p>There is a simplified way of implementing tasks too (not sure why this is needed: once one knows what's going on, it's not like the other approach is "complicated".</p>
<p>To prove I'm actually TDDing all this still, here's the test for this one:</p>
<pre class="source-code"><code>/** @TestDox It supports a simplified version of a task */
public function <a href="https://github.com/adamcameron/php8/blob/1.17/tests/Functional/SpatieAsync/TaskTest.php#L49-L59" target="_blank">testSimpleTask</a>()
{
$pool = Pool::create();
$pool->add(new SimpleTask());
$results = $pool->wait();
$this->assertCount(1, $results);
$this->assertEquals("G'day world from an async call", $results[0]);
}
</code></pre>
<p>And the implementation simply needs to be this:</p>
<pre class="source-code"><code>class <a href="https://github.com/adamcameron/php8/blob/1.17/src/Task/SimpleTask.php" target="_blank">SimpleTask</a>
{
public function __invoke()
{
return "G'day world from an async call";
}
}
</code></pre>
<p>(It's a bit mindless, fine. It's late).</p>
<p><a href="https://github.com/spatie/async#simple-tasks" target="_blank">As per the docs</a>: provided one has an <samp>__invoke</samp> method: that's it.</p>
<hr class="narrow">
<p>Speaking of TDD: in the first part of this - despite not starting with the test code listing - <a href="https://github.com/adamcameron/php8/blob/1.17/tests/Functional/SpatieAsync/TaskTest.php#L14-L46" target="_blank">I <em>did</em> TDD it</a>. Although I took it as kind of a refactoring, so I duplicated the test for the inline-task-via-callback version, and changed the code to instantiate my <samp>SlowDbCallTask</samp>. I did not directly test the <samp>configure</samp> and <samp>run</samp> implementations (with their own tests I mean), because that literally <em>is</em> implementation detail. The <em>feature</em> here is "the slow DB call [blah blah, whatever the feature actually is]", and whether it's done by inline callback or a Task class is neither here nor there. And that test does actually exercise all the code in the task class anyhow, so: job done.</p>
<p>Interestingly, the test coverage report disagrees with me:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOO-EaCLaCiCowzr5xFEesXlziUMbnGOGftgvUMOi4vqfYQwgGKV2FyOyZ1fW0LOQaWUGahPuupmfYJuwDXLoambxoie7oclrsUlOtbYGp2I1li8T7UXKQ9gwchUlaAaeophWIEHySWAObiLnHSEsmkWoVFYKZ4NQS7JbMk6Vz8wxzAIOoVFu4JY5ZHg/s1600/coverage.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="642" data-original-width="826" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOO-EaCLaCiCowzr5xFEesXlziUMbnGOGftgvUMOi4vqfYQwgGKV2FyOyZ1fW0LOQaWUGahPuupmfYJuwDXLoambxoie7oclrsUlOtbYGp2I1li8T7UXKQ9gwchUlaAaeophWIEHySWAObiLnHSEsmkWoVFYKZ4NQS7JbMk6Vz8wxzAIOoVFu4JY5ZHg/s1600/coverage.png"/></a></div>
<p>This is because the PHP process running that code is not the same one running the tests, I guess. Hrm. Maybe I <em>should</em> have separate tests. <em>But</em>: I am not of the disposition that I need to chase 100% LOC coverage … <em>but</em> (again) I discuss why ensuring 100% is something to aim for anyway in an article I wrote ages ago: "<a href="https://blog.adamcameron.me/2017/07/yeah-you-do-want-100-test-coverage.html">Yeah, you <em>do</em> want 100% test coverage</a>" (which should then be measured against this other article on the topic: "<a href="https://blog.adamcameron.me/2021/08/test-coverage-its-not-about-lines-of.html">Test coverage: it's not about lines of code</a>"). I think in this case - as the class is so simple - I'd put a <a href="https://docs.phpunit.de/en/10.0/annotations.html#codecoverageignore" target="_blank"><samp>@codeCoverageIgnore</samp></a> annotation on it. If the task class ended up having a bunch of moving parts in it that became trickier to test via its calling context, then I might consider testing those discretely. "It depends" [etc].</p>
<p>OK. That's the end of that section of the docs (I have written more about it than there was actual documentation that said. Ha. So I'm gonna finish up here. I think I have another article or so to write on this stuff yet. Let's seem.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-61992451608862743112023-02-16T22:39:00.000+00:002023-02-16T22:39:06.344+00:00ColdFusion: bug in runAsync<p>G'day:</p>
<p>In my article yesterday ("<a href="https://blog.adamcameron.me/2023/02/cfml-runasync-doesnt-seem-very.html">CFML: runAsync doesn't seem very asynchronous on Lucee</a>") I mentioned in passing that I'd found a bug in ColdFusion's <samp>runAsync</samp> implementation. I wasn't focusing on ColdFusion at the time, so I didn't try to repro it; however I have now.</p>
<p>Don't worry about the wall of code, it's the same thing three times to increase the odds of the bug situation to surface:</p>
<pre class="source-code"><code>// <a href="https://github.com/adamcameron/cfml/blob/CfRunAsyncBug/vendor/ColdFusion/2021/runAsyncBug/repro.cfm" traget="_blank">/vendor/ColdFusion/2021/runAsyncBug/repro.cfm</a>
telemetry = []
recordTelemetry = (label) => {
telemetry.append(label)
}
recordTelemetry("Begin")
recordTelemetry("f1 before define")
f1 = runAsync(() => {
recordTelemetry("f1 main")
}).then(() => {
recordTelemetry("f1 then1")
}).then(() => {
recordTelemetry("f1 then2")
})
recordTelemetry("f1 after define")
recordTelemetry("f2 before define")
f2 = runAsync(() => {
recordTelemetry("f2 main")
}).then(() => {
recordTelemetry("f2 then1")
}).then(() => {
recordTelemetry("f2 then2")
})
recordTelemetry("f2 after define")
recordTelemetry("f3 before define")
f3 = runAsync(() => {
recordTelemetry("f3 main")
}).then(() => {
recordTelemetry("f3 then1")
}).then(() => {
recordTelemetry("f3 then2")
})
recordTelemetry("f3 after define")
recordTelemetry("f1 before get")
f1.get()
recordTelemetry("f1 after get")
recordTelemetry("f2 before get")
f2.get()
recordTelemetry("f2 after get")
recordTelemetry("f3 before get")
f3.get()
recordTelemetry("f3 after get")
recordTelemetry("End")
writeDump(telemetry)
tallies = [
"Begin" = 0,
"f1 before define" = 0,
"f1 main" = 0,
"f1 then1" = 0,
"f1 then2" = 0,
"f1 after define" = 0,
"f2 before define" = 0,
"f2 main" = 0,
"f2 then1" = 0,
"f2 then2" = 0,
"f2 after define" = 0,
"f3 before define" = 0,
"f3 main" = 0,
"f3 then1" = 0,
"f3 then2" = 0,
"f3 after define" = 0,
"f1 before get" = 0,
"f1 after get" = 0,
"f2 before get" = 0,
"f2 after get" = 0,
"f3 before get" = 0,
"f3 after get" = 0,
"End" = 0
]
errors = tallies
.map((k) => telemetry.findAll(k).len())
.filter((_, v) => v != 1)
writeDump(errors)
</code></pre>
<p>This code is very similar to the cod ein the earlier article I linked to. I create three futures: <samp>f1</samp>, <samp>f2</samp>, <samp>f3</samp>; and each of those has a main task and two <samp>then</samp> handlers. All the task and <samp>then</samp> handlers do is append to a <samp>telemetry</samp> array. This is pared down from some code that actually did more stuff that I wanted to be async.</p>
<p>All going well (and 75% of the time, this is the case), we should end up with an array of 23 telemetry entries, one each for the entries in the code. We can't guarantee the order cos most of it is running asynchronously.</p>
<p>What I was seeing though, is sometimes entries were <em>missing</em>, but some others were doubled up. Here's an example of that:</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8sqnVpsvQxyGa4UtZw0BOBkGPtoHwJQY0MVu3TZAR2tQLZVqqWrJYqnuzgzuPekXpT1JRfjjDjdKZ-nTUvT4UPVCdtsSJ2V4LtCCRjH9CfT5FVyBRgO4F6m0_jfRlOqvY-FpddV6ImAmgfHCBRyQ7uzwvnKqu-9WcJ8QZNhM5EO-6JJh19vDRzqChDw/s1600/bug.png" style="display: block; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="573" data-original-width="133" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8sqnVpsvQxyGa4UtZw0BOBkGPtoHwJQY0MVu3TZAR2tQLZVqqWrJYqnuzgzuPekXpT1JRfjjDjdKZ-nTUvT4UPVCdtsSJ2V4LtCCRjH9CfT5FVyBRgO4F6m0_jfRlOqvY-FpddV6ImAmgfHCBRyQ7uzwvnKqu-9WcJ8QZNhM5EO-6JJh19vDRzqChDw/s1600/bug.png" /></a></div>
<p>See how the <samp>telemetry</samp> array has no entries for <samp>"f2 then1"</samp> and <samp>"f3 before get"</samp>; but instead has two each of <samp>"f3 then1"</samp> and <samp>"f2 before get"</samp>. Odd.</p>
<p>Also about 5% of the time I get this:</p>
<table border="0" cellpadding="0" cellspacing="0" style="background-color: #e7e7e7; font-family: verdana; width: 500px;"><tbody><tr><td align="left" id="tableProps2" valign="middle" width="500"><h1 id="textSection1" style="font-size: 13pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: normal; line-height: 15pt;">Variable LABEL is undefined.</h1></td></tr><tr><td colspan="2" id="tablePropsWidth" width="400"><span style="font-size: 8pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 11pt;"></span></td></tr><tr><td height=""> </td></tr><tr><td colspan="2" width="400"><span style="font-size: 8pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 11pt;">The error occurred in <b>/app/cfml/vendor/ColdFusion/2021/runAsyncBug/repro.cfm: line 4</b><br /></span></td></tr><tr><td colspan="2"><pre>2 : telemetry = []
3 : recordTelemetry = (label) => {
<b>4 : telemetry.append(label)</b>
5 : }
6 : </pre></td></tr></tbody></table><p><br /></p>
<p>There are always 23 entries; just some are wrong. There seems to be any number of wrong ones, but usually only a couple. The example above is a fairly typical run.</p>
<p>I tried a variant of the test with a lock around the write to the <samp>telemetry</samp> array:</p>
<pre class="source-code"><code>recordTelemetry = (label) => {
lock type="exclusive" name="telemetry" timeout=1 throwontimeout=true {
telemetry.append(label)
}
}
</code></pre>
<p>But this did not make a difference.</p>
<p>Also the fact I am using a function expression using closure over the <samp>telemetry</samp> array is significant. If I just have a function statement:</p>
<pre class="source-code"><code>function recordTelemetry(label) {
telemetry.append(label)
}
</code></pre>
<p>There is no problem.</p>
<p>I dunno what to make of this, but it'll give the Adobe ColdFusion Team something to work with anyhow.</p>
<p>Oh: this is not an issue with Lucee.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-81852729895749015632023-02-16T21:22:00.003+00:002023-02-16T23:15:55.834+00:00Lucee bug surfaced by CFWheels's "architecture"<p>G'day:</p>
<p>Well here's a fine use of my frickin time.</p>
<p>Lucee's released a new "stable" version and we're being encouraged to use it. Today I upgraded my dev environment to Lucee 5.3.10.120 (from 5.3.9.166 which seems quite stable and gave us no issues), ran our tests to see if anything obvious went wrong, and a whole lot of them went splat. All of them were variations of this (this is from my repro case, not our codebase):</p>
<table cellpadding="4" cellspacing="1" id="-lucee-err" style="background-color: #993300; border: 0px; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 11px;"><tbody><tr><td class="label" colspan="2" style="background-color: #ff9900; border: 0px solid rgb(53, 6, 6); color: #993300; font-weight: bold; line-height: 1.35; vertical-align: top; white-space: nowrap;">Lucee 5.3.10.120 Error (expression)</td></tr><tr><td class="label" style="background-color: #ff9900; border: 0px solid rgb(53, 6, 6); color: #993300; font-weight: bold; line-height: 1.35; vertical-align: top; white-space: nowrap;">Message</td><td style="background-color: #ffcc00; border: 0px solid rgb(53, 6, 6); color: #993300; line-height: 1.35;">Cannot update key [MAORI_NUMBERS] in static scope from component [cfml.vendor.lucee.staticFinal.C], that member is set to final</td></tr><tr><td class="label" style="background-color: #ff9900; border: 0px solid rgb(53, 6, 6); color: #993300; font-weight: bold; line-height: 1.35; vertical-align: top; white-space: nowrap;">Stacktrace</td><td style="background-color: #ffcc00; border: 0px solid rgb(53, 6, 6); color: #993300; line-height: 1.35;">The Error Occurred in<br /><a class="-lucee-icon-minus" id="__btn$1" style="background: url("data:image/gif;base64,R0lGODlhCQAJAIABAAAAAP///yH5BAEAAAEALAAAAAAJAAkAAAIQhI+hG8brXgPzTHllfKiDAgA7") left center no-repeat; cursor: pointer; padding: 4px 0px 4px 16px;"><b>/var/www/cfml/vendor/lucee/staticFinal/C.cfc: line 5</b></a><br /><blockquote class="expanded" id="__cst$1">3: static {<br />4: final static.MAORI_NUMBERS = ["tahi", "rua", "toru", "wha"]<br /><b>5: }</b><br />6: }<br /><br /></blockquote><a class="-lucee-icon-plus" id="__btn$2" style="background: url("data:image/gif;base64,R0lGODlhCQAJAIABAAAAAP///yH5BAEAAAEALAAAAAAJAAkAAAIRhI+hG7bwoJINIktzjizeUwAAOw==") left center no-repeat; cursor: pointer; padding: 4px 0px 4px 16px;"><b>called from</b> /var/www/cfml/vendor/lucee/staticFinal/C.cfc: line 1</a><br /><a class="-lucee-icon-plus" id="__btn$3" style="background: url("data:image/gif;base64,R0lGODlhCQAJAIABAAAAAP///yH5BAEAAAEALAAAAAAJAAkAAAIRhI+hG7bwoJINIktzjizeUwAAOw==") left center no-repeat; cursor: pointer; padding: 4px 0px 4px 16px;"><b>called from</b> /var/www/cfml/vendor/lucee/staticFinal/test.cfm: line 2</a></td></tr></tbody></table>
<p>The code in question is this:</p>
<pre class="source-code"><code>component extends=Base {
static {
final static.MAORI_NUMBERS = ["tahi", "rua", "toru", "wha"]
}
}
</code></pre>
<p>I am <em>not</em> trying to "update key [MAORI_NUMBERS] [etc]", I am simply trying to create the object.</p>
<p>Roll back to 5.3.9.166: code works.</p>
<p>My initial attempt to reproduce the error was just this:</p>
<pre class="source-code"><code>component {
static {
final static.MAORI_NUMBERS = ["tahi", "rua", "toru", "wha"]
}
}
</code></pre>
<p>But that worked fine, no errors.</p>
<p>Notice how I am not extending anything there? This is significant.</p>
<p>What's in Base.cfc? I'm kind of embarrassed to show you this.</p>
<pre class="source-code"><code>component {
include "include.cfm";
}
</code></pre>
<p>"WTF?" you might legitimately ask. Well: quite. The thing is I found this issue in a CFWheels application, and it's down to CFWheels's "architecture" that this bug in Lucee surfaces.</p>
<p>Look at this… <em>stuff</em>:</p>
<pre class="source-code"><code>component output="false" displayName="Model" {
include "model/functions.cfm";
include "global/functions.cfm";
include "plugins/standalone/injection.cfm";
}
</code></pre>
<p>That's how CFWheels implements <a href="https://github.com/cfwheels/cfwheels/blob/v2.4.0/wheels/Model.cfc" target="blank">its base model class</a>. They are pretending CFML has mixins (it doesn't) by using includes. All their classes seem to be defined as a series of include files. I just… just… I just… … <em>aaaaaaaah!!!</em> Just do me a favour and don't use CFWheels. Trust me.</p>
<p>Anyway, so this is why I'm writing shit code to reproduce this Lucee bug.</p>
<p>In <samp>include.cfm</samp> I have this:</p>
<pre class="source-code"><code>public function build() {
return this
}
</code></pre>
<p>(This is also how CFWheels initialises its model class objects: using a sort of factory method. So I'm replicating a pared-down version of that).</p>
<p>And <a href="https://github.com/cfwheels/cfwheels/blob/7960e9217071398559a773361bcc99285b2fc956/root.cfm" target="_blank">CFWheels creates its objects like this</a> (in <samp>test.cfm</samp>):</p>
<pre class="source-code"><code><cfinvoke component="C" method="build" returnVariable="o">
</code></pre>
<p>Yeah. <samp><cfinvoke></samp>. Ain't seen one of those in about a decade (and it was old code then…).</p>
<p>And if I run that code, I get the error concerned.</p>
<p>I've tried to pare it back further, but it seems I <em>need</em> the sub class, base class, include combo. Lucee is - it seems - doing something dodgy when it instantiates an object like this. No surprise they didn't catch it during their own tests, because this is a very - um - "edge case" approach to designing code.</p>
<p>Another weird thing is that if I restart my Docker container: the problem goes away. However if I then do something like change the name of <samp>include.cfm</samp> to <samp>includex.cfm</samp> or something, the problem comes back. Adding code to either CFC does <em>not</em> bring the issue back (if you see what I mean). It's def down to something about how a base class that has an include in it is first "created" (I hesitate to say "compiled" here, cos then I don't think the issue would magically go away between container restarts: the compiled code seems fine. Just the in-memory code after the compilation the first time round is crocked. Anyhow, I won't try to guess what's wrong, I'll leave that to the Lucee bods. They can have my repro case to help them (here on GitHub: <a href="https://github.com/adamcameron/cfml/tree/main/vendor/lucee/staticFinal" target="_blank"><samp>/vendor/lucee/staticFinal</samp></a>).</p>
<p>I encountered a ColdFusion bug during assessing this: time to write that one up. (update: this one has already been reported, as it turns out: <a href="https://tracker.adobe.com/#/view/CF-4213214" target="_blank">CF-4213214</a>).</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-16274409843367034532023-02-15T21:33:00.011+00:002023-02-17T08:56:26.052+00:00CFML: runAsync doesn't seem very asynchronous on Lucee (5.3.10.120)<p>G'day:</p>
<p>I was wanting to compare how other languages dealt with the promise-style async processing, as I was not completely convinced by what I was seeing when investigating spatie/async in the previous article: "<a href="https://blog.adamcameron.me/2023/02/php-looking-at-spatieasync.html">PHP: looking at spatie/async</a>". It seems solid, just doesn't behave how I'd expect in some situations. More on that in another article. I checked what JS does, and it was fine, and then I recalled in the back of my mind that CFML added something new to do with async processing a while ago. CF2018 as it turns out. <a href="https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-functions/functions-m-r/runasync.html" target="_blank"><samp>runAsync</samp></a>. And this was also added to Lucee at some point too (their docs don't say when something was added).</p>
<p>I knocked together a quick repro of what I was looking at in spatie/async and JS as a baseline; and then wtote an equivalent in CFML. But when I ran it on Lucee it didn't seem to be behaving how I'd expect, but in a different way from what I was trying to compare to in the other languages. When I ran the same code on ColdFusion though, it seemed to behave the way I expected. It seemed I have found a Lucee rabbit hole to go down.</p>
<p>I ended up with this repro that demonstrates what I'm seeing. There's a chunk of code, but it's a bunch of largely the same operations repeated a few times so as to provide usable telemetry. I'll go through it piecemeal though.</p>
<p>This is all being run on Lucee 5.3.10.120.</p>
<pre class="source-code"><code>start = getTickCount()
writeLog(file="runAsync", text="Begin @ #getTickCount() - start#")
</code></pre>
<p>I'll time things as I go.</p>
<pre class="source-code"><code><span class="xr xrt" data-index="log-whole">writeLog(file="runAsync", text="f1 before define @ #getTickCount() - start#")</span>
<span class="xr xrt" data-index="a-future"><span class="xr xrt" data-index="f1">f1</span> = runAsync(() => {</span>
<span class="xr xrt" data-index="log-before"><span class="xr xrt" data-index="log-current"><span class="xr xrt" data-index="exactly"><span class="xr xrt" data-index="main-task">writeLog(file="runAsync", text="f1 main before query @ #getTickCount() - start#")</span></span></span></span>
<span class="xr xrt" data-index="exactly"><span class="xr xrt" data-index="main-task"><span class="xr xrt" data-index="call-proc">queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});</span></span></span>
<span class="xr xrt" data-index="log-after"><span class="xr xrt" data-index="log-current"><span class="xr xrt" data-index="exactly"><span class="xr xrt" data-index="main-task">writeLog(file="runAsync", text="f1 main after query @ #getTickCount() - start#")</span></span></span></span>
})<span class="xr xrt" data-index="two-then">.then(() => {</span>
<span class="xr xrt" data-index="log-before"><span class="xr xrt" data-index="log-current"><span class="xr xrt" data-index="exactly">writeLog(file="runAsync", text="f1 then1 before query @ #getTickCount() - start#")</span></span></span>
<span class="xr xrt" data-index="exactly"><span class="xr xrt" data-index="call-proc">queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});</span></span>
<span class="xr xrt" data-index="log-after"><span class="xr xrt" data-index="log-current"><span class="xr xrt" data-index="exactly">writeLog(file="runAsync", text="f1 then1 after query @ #getTickCount() - start#")</span></span></span>
<span class="xr xrt" data-index="two-then">}).then(() => {</span>
<span class="xr xrt" data-index="log-before"><span class="xr xrt" data-index="log-current"><span class="xr xrt" data-index="exactly">writeLog(file="runAsync", text="f1 then2 before query @ #getTickCount() - start#")</span></span></span>
<span class="xr xrt" data-index="exactly"><span class="xr xrt" data-index="call-proc">queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});</span></span>
<span class="xr xrt" data-index="log-after"><span class="xr xrt" data-index="log-current"><span class="xr xrt" data-index="exactly">writeLog(file="runAsync", text="f1 then2 after query @ #getTickCount() - start#")</span></span></span>
<span class="xr xrt" data-index="two-then"><span class="xr xrt" data-index="a-future">})</span></span>
<span class="xr xrt" data-index="log-whole">writeLog(file="runAsync", text="f1 after define @ #getTickCount() - start#")</span>
</code></pre>
<ul>
<li><span class="xr xrd u" data-index="a-future">I create a future</span></li>
<li>And it has <span class="xr xrd u" data-index="main-task">a main task</span>, and <span class="xr xrd u" data-index="two-then">two <samp>then</samp> handlers</span>.</li>
<li>
Each of these do <span class="xr xrd u" data-index="exactly">exactly the same thing</span>:
<ul>
<li><span class="xr xrd u" data-index="call-proc">Call a proc in the DB</span>. See below for the implementation code, but you can probably tell it's a proc specifically designed to take time to execute. It's basically a DB-borne <samp>sleep</samp> call.</li>
<li>And <span class="xr xrd u" data-index="log-current">log the current run time</span> <span class="xr xrd u" data-index="log-before">before</span> and <span class="xr xrd u" data-index="log-after">after</span> the proc call.</li>
</ul>
</li>
<li>Oh and I <span class="xr xrd u" data-index="log-whole">log the timing either side of the whole oepration too</span>.</li>
</ul>
<p>Here's the proc:</p>
<pre class="source-code"><code>DROP PROCEDURE IF EXISTS sleep_and_return;
DELIMITER //
CREATE PROCEDURE sleep_and_return(IN seconds INT)
BEGIN
DO SLEEP(seconds);
SELECT seconds;
END //
DELIMITER ;
</code></pre>
<p>It just sleeps for the specified time and then returns the value it was passed.</p>
<p>Then I do exactly the same thing I did with <samp class="xr xrd u" data-index="f1">f1</samp> for <samp>f2</samp> and <samp>f3</samp> as well. Three identical sets of <samp>runAsync</samp> / <samp>then</samp> / <samp>then</samp> with the logging and the DB call.</p>
<p>After that, I sit back and <samp>get</samp> each of them, timing around each <samp>get</samp>:</p>
<pre class="source-code"><code>writeLog(file="runAsync", text="f1 before get @ #getTickCount() - start#")
f1.get()
writeLog(file="runAsync", text="f1 after get @ #getTickCount() - start#")
writeLog(file="runAsync", text="f2 before get @ #getTickCount() - start#")
f2.get()
writeLog(file="runAsync", text="f2 after get @ #getTickCount() - start#")
writeLog(file="runAsync", text="f3 before get @ #getTickCount() - start#")
f3.get()
writeLog(file="runAsync", text="f3 after get @ #getTickCount() - start#")
</code></pre>
<p>And one final entry in the log at the end:</p>
<pre class="source-code"><code>writeLog(file="runAsync", text="End @ #getTickCount() - start#")
</code></pre>
<p><a href ="https://github.com/adamcameron/cfml/blob/luceeAsyncTHing/cfmlLanguage/async/runAsync/usingSlowDb.cfm" target="_blank">All the code together</a> looks like this:</p>
<pre class="source-code"><code style="color:gray">start = getTickCount()
writeLog(file="runAsync", text="Begin @ #getTickCount() - start#")
writeLog(file="runAsync", text="f1 before define @ #getTickCount() - start#")
f1 = runAsync(() => {
writeLog(file="runAsync", text="f1 main before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f1 main after query @ #getTickCount() - start#")
}).then(() => {
writeLog(file="runAsync", text="f1 then1 before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f1 then1 after query @ #getTickCount() - start#")
}).then(() => {
writeLog(file="runAsync", text="f1 then2 before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f1 then2 after query @ #getTickCount() - start#")
})
writeLog(file="runAsync", text="f1 after define @ #getTickCount() - start#")
writeLog(file="runAsync", text="f2 before define @ #getTickCount() - start#")
f2 = runAsync(() => {
writeLog(file="runAsync", text="f2 main before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f2 main after query @ #getTickCount() - start#")
}).then(() => {
writeLog(file="runAsync", text="f2 then1 before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f2 then1 after query @ #getTickCount() - start#")
}).then(() => {
writeLog(file="runAsync", text="f2 then2 before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f2 then2 after query @ #getTickCount() - start#")
})
writeLog(file="runAsync", text="f2 after define @ #getTickCount() - start#")
writeLog(file="runAsync", text="f3 before define @ #getTickCount() - start#")
f3 = runAsync(() => {
writeLog(file="runAsync", text="f3 main before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f3 main after query @ #getTickCount() - start#")
}).then(() => {
writeLog(file="runAsync", text="f3 then1 before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f3 then1 after query @ #getTickCount() - start#")
}).then(() => {
writeLog(file="runAsync", text="f3 then2 before query @ #getTickCount() - start#")
queryExecute("CALL sleep_and_return(1)", [], {datasource="test"});
writeLog(file="runAsync", text="f3 then2 after query @ #getTickCount() - start#")
})
writeLog(file="runAsync", text="f3 after define @ #getTickCount() - start#")
writeLog(file="runAsync", text="f1 before get @ #getTickCount() - start#")
f1.get()
writeLog(file="runAsync", text="f1 after get @ #getTickCount() - start#")
writeLog(file="runAsync", text="f2 before get @ #getTickCount() - start#")
f2.get()
writeLog(file="runAsync", text="f2 after get @ #getTickCount() - start#")
writeLog(file="runAsync", text="f3 before get @ #getTickCount() - start#")
f3.get()
writeLog(file="runAsync", text="f3 after get @ #getTickCount() - start#")
writeLog(file="runAsync", text="End @ #getTickCount() - start#")
</code></pre>
<p>When I run this on CF2021, I get exactly what I'd expect:</p>
<pre class="source-code"><code>Begin @ 0
f1 before define @ 1
f1 main before query @ 2
f1 after define @ 3
f2 before define @ 4
f2 main before query @ 6
f2 after define @ 8
f3 before define @ 10
f3 after define @ 12
f3 main before query @ 12
f1 before get @ 13
f1 main after query @ 1006
f1 then1 before query @ 1008
f2 main after query @ 1011
f2 then1 before query @ 1012
f3 main after query @ 1015
f3 then1 before query @ 1016
f1 then1 after query @ 2010
f1 then2 before query @ 2012
f2 then1 after query @ 2014
f2 then2 before query @ 2016
f3 then1 after query @ 2023
f3 then2 before query @ 2024
f1 then2 after query @ 3014
f1 after get @ 3016
f2 before get @ 3017
f2 then2 after query @ 3018
f2 after get @ 3020
f3 before get @ 3021
f3 then2 after query @ 3026
f3 after get @ 3028
End @ 3029
</code></pre>
<p>Each of the <samp>f1</samp>, <samp>f3</samp>, <samp>f3</samp> operations are being executed in a "first come, first served", and independently of each other (ie: asynchronously). And it's all over and done with at about the time it takes the slowest one to finish (allowing for intrinsic overhead).</p>
<p>It's a different story when I run this on Lucee:</p>
<pre class="source-code"><code>Begin @ 1
f1 before define @ 19
f1 main before query @ 24
f1 main after query @ 1244
f1 then1 before query @ 1247
f1 then1 after query @ 2252
f1 after define @ 2255
<span class="xr xrt" data-index="lucee-f1-block">f1 then2 before query @ 2255</span>
<span class="xr xrt" data-index="lucee-f2-start">f2 before define @ 2255</span>
f2 main before query @ 2258
f1 then2 after query @ 3258
<span class="xr xrt" data-index="lucee-f2-block">f2 main after query @ 3263</span>
<span class="xr xrt" data-index="lucee-f2-block">f2 then1 before query @ 3266</span>
<span class="xr xrt" data-index="lucee-f2-block">f2 then1 after query @ 4270</span>
<span class="xr xrt" data-index="lucee-f2-block">f2 after define @ 4272</span>
<span class="xr xrt" data-index="lucee-f3-start">f3 before define @ 4273</span>
f2 then2 before query @ 4274
f3 main before query @ 4276
f2 then2 after query @ 5277
f3 main after query @ 5278
f3 then1 before query @ 5281
f3 then1 after query @ 6284
f3 after define @ 6286
f3 then2 before query @ 6286
f1 before get @ 6286
f1 after get @ 6287
f2 before get @ 6288
f2 after get @ 6289
f3 before get @ 6290
f3 then2 after query @ 7288
f3 after get @ 7289
End @ 7290
</code></pre>
<p>Here <samp>f1</samp> gets underway, but it really seems to mostly block until <samp class="xr xrd u" data-index="lucee-f1-block">[f1 then2 before query]</samp> until <samp class="xr xrt" data-index="lucee-f2-start">f2</samp> even gets a look-in. This is two seconds of synchronous execution before Lucee even finds out there's something else asynchronous to be cracking on with. Then <samp class="xr xrd u" data-index="lucee-f2-block">f2</samp> blocks for second before <samp class="xr xrd u" data-index="lucee-f3-start">f3</samp> gets a chance to be looked at. Then there's a mess of stopping and starting, sucking up even more time, for an end result of ~7sec for an "asynchronous" operation that would take ~9sec if it was synchronous. Not great.</p>
<p>I also hasten to add that this was not a case of bad luck how the "async-ness" happened to run this time. I've run this code dozens of times, and it's always around the same. I also have run it with just CFML's own <samp>sleep</samp> instead of the DB call, and it's still the same. It's not the work the tasks are doing, it's how Lucee is implementing the asynchronous thread management, I think.</p>
<p>Also if ColdFusion's handling of the same code was similar to Lucee's, I'd just chalk it up to "sometimes asynchronous code doesn't even up seeming that asynchronous". But given ColdFusion nails it, I really think there's something up with Lucee. Pity: I had a good use case for this stuff in our app. But now I'm thinking I need to go back to our simple <samp><cfthread></samp> implementations and check exactly how asynchronous <em>that</em> code is…</p>
<p>Full disclosure: I also found what seems to be a bug with how ColdFusion handles enclosed variable references asynchronously, but I can't work out what's going on with that yet, and I was half way through <em>this</em> bit of code, so I wanted to finish this first. I might come back to the ColdFusion thing "tomorrow".</p>
<p>The code here is reasonably portable (one just needs to create that proc, and a DSN), so I'd be dead keen to hear from other people if they can run it in their own Lucee environments, if you can be arsed, that is.</p>
<p>Or am I missing something? Entirely possible. Let me know.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-43044102840978869972023-02-15T15:47:00.000+00:002023-02-15T15:47:09.322+00:00PHP: I need another two devs<p>G'day:</p>
<p>I'm revisiting an article I posted in December, fishing for a coupla devs: <a href="https://blog.adamcameron.me/2022/12/i-need-another-two-devs.html">I need another two devs</a>. I parked that because we re-evaluated the direction we were going to take our replatforming exercise. In short we were going to be shift the CFML to Kotlin, but for various reasons that I won't go into here we've revised that plan to shift to PHP instead.</p>
<p>So. Now I'm looking for a couple of PHP devs to join my team.</p>
<p>We are growing our team some more, this time looking for two devs. Previously we'd only been hiring senior / very experienced engineers; this time I'm happy to talk to more mid-level devs who might like to join a team of seniors and take an opportunity to improve their dev skills, and their general engineering skills.</p>
<p>We have a B2B app running on a CFML-based framework. We are retiring that over the next couple of years, re-designing and re-implementing it in Symfony, on PHP8.2. The devs I am looking for will be working on the PHP side of the migration process. It's a green-fields project: currently the PHP app is at "G'day world" stage, and is not even deployed to the production environment yet, although it will be in the next couple of weeks.</p>
<p>There will be no requirement to do any dev work on the legacy CFML platform, however realistically one is gonna need to be reading / running the code whilst identifying / understanding the business logic, before re-implementing it in the PHP application. So CFML is not a necessary skill at all, but preparedness to understand the legacy code will be something one will need to do here.</p>
<p>You'd be joining a team of four devs (a mix of CFML and PHP), and a QA; this is from a total head count at Easy Direct Debits of 20-ish.</p>
<p>The details of the role - which I've lifted from our ad: "<a href="https://www.easydirectdebits.co.uk/job/uk-application-developer-job/" target="_blank">Application Developer (UK)</a>" - are: </p>
<p>Competencies:</p>
<ul>
<li>Experience with PHP.</li>
<li>Experience developing new web applications/web services.</li>
<li>Experience with Symfony.</li>
</ul>
<p>Highly desirable:</p>
<ul>
<li>Knowledge of design principles such as MVC, SOLID and other common design patterns.</li>
<li>Some familiarity with Agile principles, and experience delivering value in an Agile fashion.</li>
<li>Some exposure to test automation (eg: unit testing), and TDD practices.</li>
<li>Experience with client-side application technology (eg React.js or similar).</li>
<li>Some familiarity with Dockerised development environments.</li>
</ul>
<p>The full job spec has details of the company and all the usual HR malarky.</p>
<p>Logistics-wise this is a permanent (ie: we are not offering it to contractors) remote-first position within the UK. I will not consider any other "unofficial" attempts to work from an offshore location. You must be resident in the UK, and be able to work here, fulltime. We cannot offer visa sponsorship. We <em>have</em> a physical office but I've never set foot in it, and everyone else in the dev team are 100% remote as well; spread throughout England. But if you want to work in Bournemouth, there's a desk for you there if that's your thing. The people in that office are all nice :-).</p>
<p>As I said, we have two openings for this role at the moment.</p>
<p>If you want to have a chat about this, you can send yer CV through to the email address on the job spec page linked-to above. I'm not interested in talking to recruiters for now, just in case you are one (and reading this?): I'm only wanting to talk to people in the dev community directly.</p>
<p>Righto.</p>
<p>-- <br>
Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-77617518399027685772023-02-09T22:52:00.001+00:002023-02-09T22:55:13.693+00:00PHP: looking at spatie/async<p>G'day:</p>
<p>For no good reason at all, other than it piquing my interest, I've decided to mess around with <a href="https://github.com/spatie/async" target="_blank">spatie/async</a>, which is - in its own words:</p>
<blockquote cite="https://github.com/spatie/async">
[A] small and easy wrapper around PHP's PCNTL extension. It allows running of different processes in parallel, with an easy-to-use API.
</blockquote>
<p>I got onto it because after messing around with Guzzle the other day ("<a href="https://blog.adamcameron.me/2023/01/php-looking-at-ways-of-making-http.html">PHP: looking at ways of making HTTP requests</a>"), I got to wondering how PHP is doing async stuff. It occurs to me now I didn't think to just check which lib(s) Guzzle was using; I just googled, and found a few options, and <samp>spatie/async</samp> seemed to be the simplest / most-no-nonsense of the lot, so thought would start here.</p>
<p>I'm gonna "experiment via test", as is my wont.</p>
<hr class="narrow"><h3>Set-up</h3>
<p>First of all(*) though, I need to install PCNTL, which is not installed by default in PHP's stock Docker image. It's easy though, just add this to my <a href="https://github.com/adamcameron/php8/blob/1.15/docker/php/Dockerfile" target="_blank">Dockerfile</a> and rebuild my container:</p>
<pre class="source-code"><code>RUN docker-php-ext-configure pcntl && docker-php-ext-install pcntl
</code></pre>
<p>(*) I say "first of all", but - full disclosure - I missed the note in <strong>the first sentence of the docs</strong> (ie: that sentence I quoted above) that it was required, and wrote a whole bunch of code, and was left wondering "why is this not running async?". But only for like an hour or two. Ahem. Cough. Anyway. Moving right along.</p>
<p>First I have set up a proc that I can run that just takes ages to run. I'm gonna use this to tie-up PHP processing. Pardon my pathetic MySQL skillz:</p>
<pre class="source-code"><code>USE db1;
DROP PROCEDURE IF EXISTS <a href="https://github.com/adamcameron/php8/blob/1.15/docker/mariadb/docker-entrypoint-initdb.d/3.createProcs.sql" target="_blank">sleep_and_return</a>;
DELIMITER //
CREATE PROCEDURE sleep_and_return(IN seconds INT)
BEGIN
DO SLEEP(seconds);
SELECT seconds;
END //
DELIMITER ;
</code></pre>
<hr class="narrow"><h3>Baseline: it runs stuff asynchronously</h3>
<p>And now my first test:</p>
<pre class="source-code"><code>/** @testdox It can call a slow proc multiple times async */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L14" target="_blank">testSlowProcAsync</a>()
{
$connection = DB::getDbalConnection();
<span class="xr xrt" data-index="pool">$pool = Pool::create()</span>;
<span class="xr xrt" data-index="time-1">$startTime = microtime(true);</span>
for ($i = 1; $i <= <span class="xr xrt" data-index="three-tasks">3</span>; $i++) {
<span class="xr xrt" data-index="add">$pool->add</span>(function () use ($connection, $i, $startTime) {
$result = <span class="xr xrt" data-index="sleep-2">$connection->executeQuery("CALL sleep_and_return(?)", [<span class="xr xrd u" data-index="around-2"><span class="xr xrt" data-index="even-one">2</span></span>]);</span>
<span class="xr xrt" data-index="return-metrics-from-process">return sprintf(</span>
<span class="xr xrt" data-index="return-metrics-from-process">"%d:%d:%d",</span>
<span class="xr xrt" data-index="return-metrics-from-process">$i,</span>
<span class="xr xrt" data-index="return-metrics-from-process">$result->fetchOne(),</span>
<span class="xr xrt" data-index="return-metrics-from-process"><span class="xr xrt" data-index="take-note"><span class="xr xrt" data-index="last-number">microtime(true) - $startTime</span></span></span>
<span class="xr xrt" data-index="return-metrics-from-process">);</span>
});
}
<span class="xr xrt" data-index="pool-population-time">$poolPopulationTime = microtime(true) - $startTime;</span>
<span class="xr xrt" data-index="pretty-quick">$this->assertLessThanOrEqual(1, $poolPopulationTime);</span>
<span class="xr xrt" data-index="pool-time">$startTime = microtime(true);</span>
<span class="xr xrt" data-index="returned-from-wait">$results = $pool->wait();</span>
<span class="xr xrt" data-index="pool-time">$endTime = microtime(true);</span>
<span class="xr xrt" data-index="pool-time">$executionTime = $endTime - $startTime;</span>
<span class="xr xrt" data-index="pool-time-duration">$this->assertLessThan(3, $executionTime);</span>
<span class="xr xrt" data-index="verify-results">$this->assertCount(3, $results);</span>
<span class="xr xrt" data-index="verify-results"><span class="xr xrt" data-index="each-element">$this->assertContains</span>("1:2:2"<span class="xr xrt" data-index="failure-message">, $results, "1:2:2 not found in " . implode(",", $results)</span>);</span>
<span class="xr xrt" data-index="verify-results">$this->assertContains("2:2:2", $results, "<span class="xr xrt" data-index="not-2-2-2">2:2:<span class="xr xrt" data-index="last-number">2</span></span> not found in " . implode(",", $results));</span>
<span class="xr xrt" data-index="verify-results">$this->assertContains("3:2:2", $results, "3:2:2 not found in " . implode(",", $results));</span>
}
</code></pre>
<p>There's a bit going on here (it's not my first iteration of this test, I have to admit).</p>
<ul>
<li>The premise is that <span class="xr xrd u" data-index="sleep-2">each task I need to process async is a call to that proc which I am making sleep for two seconds</span>. <span class="xr xrd u" data-index="three-tasks">Three tasks</span> doing that is six seconds of synchronous work; or only around two seconds or so if each is running asynchronously.</li>
<li>This thing seems to work by having <span class="xr xrd u" data-index="pool">a pool</span>, <span class="xr xrd u" data-index="add">then adding processes</span> to run asynchronously.</li>
<li><span class="xr xrd u" data-index="time-1">I take note of the time before I start</span> adding processes to the pool, so <span class="xr xrd u" data-index="take-note">I can take note of how long each takes to run (should be ~2sec)</span>.</li>
<li><span class="xr xrd u" data-index="return-metrics-from-process">I return those metrics</span>, and they end up being what's <span class="xr xrd u" data-index="returned-from-wait">returned from the <samp>wait</samp></span> call at the end. Easy.</li>
<li><span class="xr xrd u" data-index="pool-population-time">I also note how long it took to populate the pool</span>. This should be <span class="xr xrd u" data-index="pretty-quick">pretty quick</span> (<span class="xr xrd u" data-index="even-one">less than even one of the processes running</span>).</li>
<li><span class="xr xrd u" data-index="pool-time">I then time how long the pool takes to run</span>. Each process is ~2sec, and if they're running asynchronously the whole thing should finish in <span class="xr xrd u" data-index="pool-time-duration">less time than it would take two of them to run</span>.</li>
<li><span class="xr xrd u" data-index="verify-results">Lastly I verify the results are what I'd expect</span>. Note that I'm not testing an exact array, but <span class="xr xrd u" data-index="each-element">just looking for each element</span>: as each process is async, I can't really assume that they'll complete (and populate that array) in an specific sequence.</li>
</ul>
<p>The metrics I'm grabbing are not as clever as they could be. I'm assuming each process will finish <span class="xr xrd u" data-index="around-2">around 2sec</span> after I <span class="xr xrd u" data-index="time-1">start the timer before adding them to the pool</span>. There's overhead in the adding, and calling the code as well as the 2sec pause at the DB, and sometimes the latter 1-2 assertions fail. I've added <span class="xr xrd u" data-index="failure-message">the failure message to see what actually gets returned</span>, but sadly I have not been able to see it fail since I added those. I imagine it's a case that ~3sec has passed between one of the latter process callbacks being added, and them actually finishing running. So like the metrics returned for that process might be <samp>2:2:3</samp>, not <samp class="xr xrd u" data-index="not-2-2-2">2:2:<span class="xr xrt" data-index="last-number">2</span></samp>: <span class="xr xrd u" data-index="last-number">the last number is the duration</span> since we started adding the processes to the pool.</p>
<div class="updateBox">
<h3>Update</h3>
<p>I'm writing this the day after I wrote the paragraph above.</p>
<p>I got the test failure, and this bears out my theory from before, pretty miuch:</p>
<div class="cliBox"><pre>
tests of spatie/async (https://github.com/spatie/async)
<span style="color:red">✘</span> It can call a slow proc multiple times async
<span style="color:red">┐</span>
<span style="color:red">├</span> <span style="background-color:red;color:white">1:2:2 not found in 3:2:3,1:2:3,2:2:3 </span>
<span style="color:red">├</span> <span style="background-color:red;color:white">Failed asserting that an array contains '1:2:2'.</span>
<span style="color:red">│</span>
<span style="color:red">╵</span> /var/www/tests/Functional/Async/SpatieAsyncTest.php:<span style="color:blue">41</span>
<span style="color:red">┴</span>
</pre></div>
<p>Note the array returned in the results: <samp>3:2:<span class="xr xrt" data-index="slow">3</span>,1:2:<span class="xr xrt" data-index="slow">3</span>,2:2:<span class="xr xrt" data-index="slow">3</span></samp>. In each of these <span class="xr xrd u" data-index="slow">the process took three seconds to complete</span>. Probably because I had only just started my containers, and the PHP code was not compiled and cached, and the DB had only just started. So, yeah, not a great test, but it helped show this issue, which is something.</p>
</div>
<hr class="narrow"><h3>Adding a <samp>then</samp> handler</h3>
<p>The next test is very similar, except I'm chaining a <samp>then</samp> handler onto the process:</p>
<pre class="source-code"><code style="color:gray">/** @testdox It uses a then handler which acts on the result */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L48" target="_blank">testSlowProcAsyncThen</a>()
{
$connection = DB::getDbalConnection();
$pool = Pool::create();
$startTime = microtime(true);
for ($i = 1; $i <= 3; $i++) {
$pool
->add(function () use ($connection) {
<span class="xr xrt" data-index="return-statement">return $connection->executeQuery("CALL sleep_and_return(?)", [2]);</span>
})
<span style="color:black"><span class="xr xrt" data-index="then-receives">->then(function ($result)</span> use ($i, $startTime) {
return sprintf(
"%d:%d:%d",
$i,
$result->fetchOne(),
microtime(true) - $startTime
);
});</span>
}
$startTime = microtime(true);
$results = $pool->wait();
$endTime = microtime(true);
$executionTime = $endTime - $startTime;
$this->assertLessThan(3, $executionTime);
$this->assertCount(3, $executionTimes);
$this->assertContains("1:2:2", $results, "1:2:2 not found in " . implode(",", $results));
$this->assertContains("2:2:2", $results, "2:2:2 not found in " . implode(",", $results));
$this->assertContains("3:2:2", $results, "3:2:2 not found in " . implode(",", $results));
}
</code></pre>
<ul>
<li>The <span class="xr xrd u" data-index="return-statement">task returns the DB statement this time</span>.</li>
<li>And <span class="xr xrd u" data-index="then-receives">the <samp>then</samp> handler receives that</span>, and now <em>it</em> returns the metrics.</li>
</ul>
<p>Everything else is the same.</p>
<p>And this test goes <strong>splat</strong>:</p>
<div class="cliBox"><pre>Exception : Serialization of 'PDOStatement' is not allowed</pre></div>
<p>OK, fair enough. Not everything is serialisable, and I'm guessing this is how objects are being passed to different PHP processes. Good to know. I will update my test <span class="xr xrd u" data-index="pass-result">to pass the result</span>, not the whole statement object:</p>
<pre class="source-code"><code>$pool
->add(function () use ($connection) {
<span style="text-decoration: line-through">return</span><span class="xr xrt" data-index="pass-result">$result = </span>$connection->executeQuery("CALL sleep_and_return(?)", [2]);
<span class="xr xrt" data-index="pass-result">return $result->fetchOne();</span>
})
->then(function ($result) use ($i, $startTime) {
return sprintf(
"%d:%d:%d",
$i,
$result,
microtime(true) - $startTime
);
});
</code></pre>
<p>But this is wrong too:</p>
<div class="cliBox"><pre>
1:2:2 not found in 2,2,2
Failed asserting that an array contains '1:2:2'.
</pre></div>
<p>What's going on here? Ah. OK, so the final result that is returned from <samp>$pool->wait()</samp> is the return values <em>from the initial process callbacks</em>, not the end result of the <samp>then</samp> handler. My process callbacks are returning the result of the proc call, which is just the number originally passed into it (<samp>2</samp>), hence the result array being <samp>[2,2,2]</samp>. If I want to access the result of the processing done in the <samp>then</samp> I'm gonna need to grab it separately:</p>
<pre class="source-code"><code style="color:gray">public function testSlowProcAsyncThen()
{
$connection = DB::getDbalConnection();
$pool = Pool::create();
<span style="color:black">$metrics = [];</span>
$startTime = microtime(true);
for ($i = 1; $i <= 3; $i++) {
$pool
->add(function () use ($connection) {
$result = $connection->executeQuery("CALL sleep_and_return(?)", [2]);
return $result->fetchOne();
})
->then(function ($result) use (<span style="color:black">&$metrics</span>, $i, $startTime) {
<span style="color:black">$metrics[]</span> = sprintf(
"%d:%d:%d",
$i,
$result,
microtime(true) - $startTime
);
});
}
$startTime = microtime(true);
$pool->wait();
$endTime = microtime(true);
$executionTime = $endTime - $startTime;
$this->assertLessThan(3, $executionTime);
$this-->assertCount(3, $results);
$this->assertCount(3, <span style="color:black">$metrics</span>);
$this->assertContains("1:2:2", <span style="color:black">$metrics</span>, "1:2:2 not found in " . implode(",", <span style="color:black">$metrics</span>));
$this->assertContains("2:2:2", <span style="color:black">$metrics</span>, "2:2:2 not found in " . implode(",", <span style="color:black">$metrics</span>));
$this->assertContains("3:2:2", <span style="color:black">$metrics</span>, "3:2:2 not found in " . implode(",", <span style="color:black">$metrics</span>));
}
</code></pre>
<p>And now this gives expected results.</p>
<hr class="narrow"><h3>Handling timeouts</h3>
<p>Next up, I'm having a look at the timeout one can put on a pool. Similar code again: </p>
<pre class="source-code"><code style="color:gray">/** @testdox It supports a timeout */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L86" target="_blank">testSlowProcAsyncTimeout</a>()
{
$connection = DB::getDbalConnection();
$pool = Pool::create();
<span style="color:black"><span class="xr xrt" data-index="timeout-1">$pool->timeout(1);</span>
<span class="xr xrt" data-index="timeout-handler">$timeOuts = [];</span></span>
$startTime = microtime(true);
for ($i = 1; $i <= 3; $i++) {
$pool
->add(function () use ($connection) {
$result = $connection->executeQuery("CALL sleep_and_return(?)", [2]);
return $result->fetchOne();
})
<span style="color:black"><span class="xr xrt" data-index="timeout-handler">->timeout(function () use (&$timeOuts, $i, $startTime) {</span>
<span class="xr xrt" data-index="timeout-handler">$timeOuts[] = sprintf(</span>
<span class="xr xrt" data-index="timeout-handler">"TIMED OUT ON ITERATION %d after %d seconds",</span>
<span class="xr xrt" data-index="timeout-handler">$i,</span>
<span class="xr xrt" data-index="timeout-handler">microtime(true) - $startTime</span>
<span class="xr xrt" data-index="timeout-handler">);</span>
<span class="xr xrt" data-index="timeout-handler">return false;</span>
<span class="xr xrt" data-index="timeout-handler">});</span></span>
}
$results = $pool->wait();
$endTime = microtime(true);
$executionTime = $endTime - $startTime;
<span style="color:black"><span class="xr xrt" data-index="timeout-handler">$this->assertEquals(</span>
<span class="xr xrt" data-index="timeout-handler">[</span>
<span class="xr xrt" data-index="timeout-handler">"TIMED OUT ON ITERATION 1 after 1 seconds",</span>
<span class="xr xrt" data-index="timeout-handler">"TIMED OUT ON ITERATION 2 after 1 seconds",</span>
<span class="xr xrt" data-index="timeout-handler">"TIMED OUT ON ITERATION 3 after 1 seconds"</span>
<span class="xr xrt" data-index="timeout-handler">],</span>
<span class="xr xrt" data-index="timeout-handler">$timeOuts</span>
<span class="xr xrt" data-index="timeout-handler">);</span></span>
$this->assertLessThan(2, $executionTime);
<span style="color:black"><span class="xr xrt" data-index="results-empty">$this->assertEquals([], $results);</span></span>
}
</code></pre>
<ul>
<li><span class="xr xrd u" data-index="timeout-1">I set a timeout of 1sec.</span></li>
<li><span class="xr xrd u" data-index="timeout-handler">I have a timeout handler that returns some metrics.</span></li>
<li><span class="xr xrd u" data-index="results-empty">Note that the value returned from wait is an empty array this time.</span></li>
</ul>
<hr class="narrow"><h3>Handling exceptions</h3>
<p>Next a much more simplified test, looking at exception handling:</p>
<pre class="source-code"><code>/** @testdox it supports exception handling */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L131" target="_blank">testAsyncException</a>()
{
$pool = Pool::create();
$pool
->add(function () {
throw new \Exception("This is an exception");
})
->catch(function (\Exception $exception) {
$this->assertEquals("This is an exception", $exception->getMessage());
});
$pool->wait();
}
</code></pre>
<p>There's no surprises there, except the assertion fails, because <samp>$exception->getMessage()</samp> is not <samp>"This is an exception"</samp>, it's this mess:</p>
<div class="cliBox"><pre>
'This is an exception\n
\n
#0 [internal function]: adamcameron\php8\tests\Functional\Async\SpatieAsyncTest::{closure}()\n
#1 /var/www/vendor/laravel/serializable-closure/src/Serializers/Native.php(91): call_user_func_array(Object(Closure), Array)\n
#2 [internal function]: Laravel\SerializableClosure\Serializers\Native->__invoke()\n
#3 /var/www/vendor/laravel/serializable-closure/src/SerializableClosure.php(48): call_user_func_array(Object(Laravel\SerializableClosure\Serializers\Native), Array)\n
#4 /var/www/vendor/spatie/async/src/Runtime/ChildRuntime.php(26): Laravel\SerializableClosure\SerializableClosure->__invoke()\n
#5 {main}'
</pre></div>
<p>Exactly that. The exception <em>message</em> has somehow been polluted with a stack trace.</p>
<p>Initially I thought it might be because the library was wrapping my exception in another one (kinda fair enough), but it's not: it's just an exception. If I wasn't a complete n00b with this (and how PHP handles async stuff), and also wasn't incredibly rusty with PHP, I'd probably raise an issue with the library maintainers. But: I'll mull it over a bit first, to see if I can work out what dumbarsery I might be engaging in.</p>
<hr class="narrow">
<p>I decided to push the exception down a bit, and <span class="xr xrd u" data-index="then-exception">throw it in a <samp>then</samp> handler</span>. <span class="xr xrd u" data-index="cannot-catch-then">Disappointing results here</span>:</p>
<pre class="source-code"><code>/** @testdox it does not support exception handling from a then handler */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L146" target="_blank">testAsyncExceptionFromThen</a>()
{
<span class="xr xrt" data-index="cannot-catch-then">$this->expectException(\Exception::class);</span>
<span class="xr xrt" data-index="cannot-catch-then">$this->expectExceptionMessage("This is an exception");</span>
$pool = Pool::create();
$pool
->add(function () {
// do nothing
})
<span class="xr xrt" data-index="then-exception">->then(function () {</span>
<span class="xr xrt" data-index="then-exception">throw new \Exception("This is an exception");</span>
<span class="xr xrt" data-index="then-exception">})</span>
<span class="xr xrt" data-index="cannot-catch-then">->catch(function (\Exception $exception) {</span>
<span class="xr xrt" data-index="cannot-catch-then">$this->assertStringStartsWith("This is an exception", $exception->getMessage());</span>
<span class="xr xrt" data-index="cannot-catch-then">});</span>
$pool->wait();
}
</code></pre>
<p><span class="xr xrd u" data-index="cannot-catch-then">The exception wasn't handled by the <samp>catch</samp> handler</span>. It seems it'll only catch exceptions from the main process.</p>
<hr class="narrow"><h3>Stopping the pool in its tracks</h3>
<p>I'm changing the test approach now, indeed I'm lifting their sample code from the docs and turning it into a test:</p>
<pre class="source-code"><code>/** @testdox a pool can be stopped */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L167" target="_blank">testPoolStop</a>()
{
$pool = Pool::create();
for ($i = 0; $i < 10000; $i++) {
$pool->add(function () {
return rand(0, 100);
})->then(function ($output) use ($pool) {
// If one of them randomly picks 100, end the pool early.
if ($output === 100) {
$pool->stop();
}
});
}
$results = $pool->wait();
$this->assertLessThan(10000, count($results));
<span class="xr xrt" data-index="the-100">$this->assertContains(100, $results);</span>
}
</code></pre>
<p>This is obviously really contrived, but it's a way to see that the <samp>stop</samp> method does the trick.</p>
<p>I decided to see if it would also work in a <samp>catch</samp> handler:</p>
<pre class="source-code"><code style="color:gray">/** @testdox a pool can be stopped in a catch */
public function <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L192" target="_blank">testPoolStopInCatch</a>()
{
$pool = Pool::create();
for ($i = 0; $i < 10000; $i++) {
$pool->add(function () {
<span style="color:black">$result = rand(0, 100);
if ($result === 100) {
throw new \Exception("Something went wrong");
}
return $result;
})->catch(function (\Exception $_) use ($pool) {
$pool->stop();
});</span>
}
$results = $pool->wait();
$this->assertLessThan(10000, count($results));
<span class="xr xrt" data-index="the-100"><span style="color:black">$this->assertNotContains(100, $results);</span></span>
}
</code></pre>
<p>Note that <span class="xr xrd u" data-index="the-100">in this case, unlike the previous one, the results <em>won't</em> include the <samp>100</samp></span>, because it never gets returned.</p>
<hr class="narrow">
<h3>Stopping the Cameron in its tracks</h3>
<p>There's more to look at, but this article is already quite long, and this is a reasonable stopping point because the next bit deals with alterntive syntaxes. I'll do that tomorrow (-ish).</p>
<p>This seems like pretty handy stuff so far, although there's that exception weirdness to mull over, and also I need to reason why I can't catch exceptions thrown in the <samp>then</samp> handlers.</p>
<p>All the methods are linked-to inline, but for the record they're all here: <a href="https://github.com/adamcameron/php8/blob/1.15/tests/Functional/Async/SpatieAsyncTest.php#L14" target="_blank"><samp>/tests/Functional/Async/SpatieAsyncTest.php</samp></a>.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-58092610237590276052023-02-05T22:46:00.003+00:002023-04-08T17:58:24.752+00:00Ugh. PHPUnit and dealing with deprecation notices<p>G'day:</p>
<h3>Today's plan</h3>
<p>I sat down to write an article and some code using the <a href="https://github.com/spatie/async" target="_blank">spatie/async</a> library. Not for any particular requirement I have, just that it sounds interesting. But I've just shelved that idea for today.</p>
<h3>Today's revised plan</h3>
<p>I added the package as a dependency and ran <samp>composer update</samp>. My first exercise after doing a <samp>composer update</samp> is always to run my automated tests to see what library updates have broken things. I'm glad I did because today it's PHPUnit itself that's breaking shit. Great.</p>
<p>So I backed-out <samp>spatie/async</samp> as a dependency, re-updated, and ran my tests again: yeah I still have problems. I checked and PHPUnit has updated from <a href="https://github.com/adamcameron/php8/blob/1.13.1/composer.lock#L4639" target="_blank"><samp>9.5.28</samp></a> to <a href="https://github.com/adamcameron/php8/blob/1.14/composer.lock#L4639" target="_blank"><samp>9.6.2</samp></a>. I hasten to add that the breakages don't seem to be bugs or problems, just that they've added a bunch of deprecation notices in for upcoming changes to PHPUnit 10. This is a good thing. But I feel that now that I know about them, I should look into what I need to do to fix them. And I also suspect there's a way to run PHP with deprecation warnings switched off, and I should know how to do this so I am going to find out.</p>
<p>The symptoms I am seeing are this sort of thing:</p>
<div class="cliBox"><pre style="color:gold">
Warning: Abstract test case classes with "Test" suffix are deprecated
<span style="color:white;">…</span>
┐
├ assertObjectHasAttribute() is deprecated and will be removed in PHPUnit 10.
┴
<span style="color:white;">…</span>
┐
├ Expecting E_ERROR and E_USER_ERROR is deprecated and will no longer be possible in PHPUnit 10.
┴
<span style="color:white;">…</span>
</pre></div>
<p>Those seem like the only three actually. Let's find out how to work around them.</p>
<hr class="narrow">
<h3>Not PHPUnit's finest hour</h3>
<p>First: how to suppress the warnings. This is not helped by the <samp>phpunit.xml</samp> page in the docs currently 404ing (this is a temporary thing I think: it was fine last week, I have it in my history). But I found a cached version. Having done some googling, there is no way of turning off the deprecation reporting it seems (it was requested, and actively rejected). I also found this article written by Sebastian Bergmann, PHP's lead: <a href="https://thephp.cc/articles/help-my-tests-stopped-working" target="_blank">Help! My tests stopped working</a> ("Deprecations" section). Relevant extract:</p>
<blockquote cite="https://thephp.cc/articles/help-my-tests-stopped-working">
<p>PHPUnit behaves differently compared to PHP and how it reports the usage of deprecated functionality. When a deprecated feature of PHP is used then a message of type E_DEPRECATED is created. This message, however, is only shown when the developer has configured PHP's error_reporting setting accordingly. In my experience, developers either do not have error_reporting configured correctly (it should be set to -1 for development so that you see all error messages) or never bother to look at error messages that get buried in some log file. This is why quite a few developers are surprised that their software stops working when a major version of PHP is released that removes previously deprecated functionality.</p>
<p>PHPUnit reports a test that uses deprecated functionality with a warning because I know how developers use, or rather not use, PHP's E_DEPRECATED messages. You cannot opt out of getting this information from PHPUnit.</p>
</blockquote>
<p>IMO this is a bit of a dick move (even if the intent is sound). PHPUnit: your job? Being a unit test framework. Stick to that. Leave it to me and PHP how deprecations ought to be dealt with. That's not part of your remit. If there was no prescribed way of handling (or not ~) in PHP then fair enough for PHPUnit to make its own call. And if PHPUnit was unaware of the prescribed PHP approach: also fair enough. But to be aware of it and pronounce "<em>well ackchyually…</em> I know better" is letting the side down a bit. IMO, obvs.</p>
<p>Still: it's given me a blog article topic I guess. So… erm…"cheers"…?</p>
<p>Firstly: the article also says this:</p>
<blockquote><p>By default, PHPUnit's command-line test runner exits with shell exit code 0 when the use of a deprecated feature is reported.</p></blockquote>
<p>I'm staring at an exit code of 1, currently, but I am guessing this is because I have <a href="https://github.com/adamcameron/php8/blob/1.14/phpunit.xml.dist#L8" target="_blank"><samp>failOnWarning="true"</samp> set in <samp>phpunit.xml.dist</samp></a>.</p>
<div class="cliBox"><pre>
<span style="background-color:gold;color:black;">WARNINGS!</span>
<span style="background-color:gold;color:black;">Tests: 79, Assertions: 208, Warnings: 8, Skipped: 1.</span>
Generating code coverage report in HTML format ... done [00:06.012]
root@19f76723847c:/var/www# echo $?
1
</pre></div>
<p>Yeah, this returns zero if I switch that setting off. It still sux though, cos I don't <em>want</em> to turn <strong>warnings</strong> off, I only want to not be pestered by PHPUnit's own <strong>deprecations</strong> interfering with the reporting of the tests of <em>my own code</em>, which - actually - are all fine. So this is a case of misreporting on PHPUnit's part. To me that's a bug (one implemented by design, but subpar-decision-making is as much a bug as shonky implementation is).</p>
<p>NB: the quickest fix for me here would be to roll back to the previous version of PHPUnit whilst I take time to scheduling in the fixing of the tests. I hasten to add I am completely fine making the code changes, that's not the issue here. It's the crap way it's being handled by PHPUnit is the only issue.</p>
<p>Anyway, we get it: I'm annoyed. Let's move on and do something useful.</p>
<hr class="narrow">
<h3>Abstract test case classes with "Test" suffix are deprecated</h3>
<p>My initial reaction to this is about as positive as my reaction to PHPUnit's deprecation handling. It's completely legit for test suites to have common notionality in an abstract sense. And the best description of this could very legitimately end in the word <samp>Test</samp>. An obvious example might be a polypmorhic sitution with the classes under test, which could have different imlpementations each with the same custom assertion needs. Or even the implementation of dependencies to the object under test might be the same. On the other hand, from a position of not having looked into why PHPUnit has had this change make, there seems no good reason to prevent this: this seems like meddling for the sake of it to me, but let's see if I can find out why they've done it. It would be great, btw, if these deprecation notices also linked to whatever issue caused the deprecation to be made. But I'll hunt through the code.</p>
<p>The first thing I have found is <a href="https://github.com/sebastianbergmann/phpunit/issues/5132" target="_blank">the Github issue that represents the work</a>: no explanation. No text <em>at all</em>, which is less than ideal: no work should be undertaken with a description of what/why, and also like some acceptance criteria so we know what the "definition of done" is for the given case. The only other thing I found is <a href="https://github.com/sebastianbergmann/phpunit/issues/4979" target="_blank">another Github issue of someone being bitten on the bum by it</a>, but there was only confirmation it's "expected" behaviour, but no explanation why. Oh well.</p>
<div class="updateBox">
Ha, OK, there's <a href="https://github.com/sebastianbergmann/phpunit/blob/9.6.3/ChangeLog-9.6.md?plain=1#L31" target="_blank">a 9.6 change log file</a> which also links through to #5132, which I found whilst fossicking around in my <samp>vendor/phpunit</samp> dir. Still no explanation of "why" though.
</div>
<p>This is at least easy to fix. I can simply rename the class in question.</p>
<p>In the case in question it's just this:</p>
<pre class="source-code"><code>abstract class <a href="https://github.com/adamcameron/php8/blob/1.14/tests/Integration/Http/HttpTestBase.php" target="_blank">HttpTest</a> extends TestCase
{
protected function getUserAgentForCurl(): string
{
return sprintf("curl/%s", curl_version()['version']);
}
protected function assertGitInfoIsCorrect(string $response): void
{
$myGitMetadata = json_decode($response);
$this->assertEquals('adamcameron', $myGitMetadata->login);
$this->assertEquals('Adam Cameron', $myGitMetadata->name);
}
}
</code></pre>
<p>TBH hoisting the <samp>getUserAgentForCurl</samp> helper method does not really contribute to a sense that there's an is-a relationship between this class and the subclasses (<samp>CurTest</samp>, <samp>GuzzleTest</samp>, <samp>PhPStreamTest</samp>, <samp>SymfonyHttpClientTest</samp>): this would better be in a dependency class if that's all that was in play here. But I def think the common need for the same custom assertion implementation fits the bill, so the abstractness of this class and its subclass hierarchy is legit. I also think <samp>HttpTest</samp> is the most fitting name for it, but… oh well. I'm just gonna rename it to <samp>HttpTestBase</samp> (runner up: <samp>HttpTests</samp> (plural)) I think. Not great - and not as fitting as <samp>HttpTest</samp> as it is now - but so be it.</p>
<hr class="narrow">
<h3><samp>assertObjectHasAttribute()</samp> is deprecated and will be removed in PHPUnit 10.</h3>
<p>I found the issue for this one straight away: <a href="https://github.com/sebastianbergmann/phpunit/issues/4601" target="_blank">Remove assertions that operate on class/object properties #4601</a>. This situation is legit. Quoting the ticket:</p>
<blockquote cite="https://github.com/sebastianbergmann/phpunit/issues/4601">
<p>PHPUnit currently refers to "fields" (see above) as "attributes". This is (or will become) confusing considering the introduction of attributes in PHP 8 and their support in PHPUnit.</p>
<p>PHPUnit will be changed to use the term "property" instead of "attribute" where "field" is meant.</p>
</blockquote>
<p>I'm onboard with this: it makes sense.</p>
<p>But. The replacement methods have not been implemented yet. So this is a good example where this thing oh PHPUnit's of raising deprecations as warnings is shit. I can't switch this one off "for now", knowing I have some work to do in future. I also don't want to turn off the PHPUnit setting that promotes warnings to exceptions because <em>I want to know</em> about legitimate warnings (from other sources). This is why PHP handles deprecations the way that it does. Sigh.</p>
<p>I want to get rid of the warnings - well I <em>have to</em>! - So I'm gonna implement my own <samp>assertObjectHasProperty</samp> assertion, using the same implementation as the current <samp>assertObjectHasAttribute</samp>. It's only used in one test class, so it's easy to integrate it into my codebase as well.</p>
<p>[beat]</p>
<p>OK so not exactly the same implementation as the current one uses private methods from its current home <a href="https://github.com/sebastianbergmann/phpunit/blob/9.6.2/src/Framework/Assert.php#L1335" target="_blank"><samp>/src/Framework/Assert.php</samp></a>. But I'll knock together as close an approximation as possible I guess.</p>
<p>[beat]</p>
<p>OK screw that: it's using other deprecated methods on other objects, and it's not a good use of my time reimplementing all this crap. I'm just gonna use a <samp>property_exists</samp> check inline where I am currently using <samp>assertObjectHasAttribute</samp>. There's only two occurrences.</p>
<hr class="narrow">
<h3><samp>Expecting E_ERROR and E_USER_ERROR is deprecated and will no longer be possible in PHPUnit 10.</samp></h3>
<p>This is another case of the deprecation being put in with no real explanation. It's issue <a href="https://github.com/sebastianbergmann/phpunit/issues/5062" target="_blank">#5062</a>, and the "explanation" is:</p>
<blockquote>
There are no replacements. PHPUnit 10 no longer converts E_* to exceptions, therefore E_* can no longer be expected.
</blockquote>
<p>Fine, but throw us a bone, pal: <em>why</em> does "PHPUnit 10 no longer converts <samp>E_*</samp> to exceptions, therefore <samp>E_*</samp> can no longer be expected". There must be some rationale? Let's go see if I can find anything about <em>that</em>… … … Nope. No issue I can find relating to that. All right (shakes head, and mutters "fuck sake, PHPUnit").</p>
<p>Right so I've addressed this one thus:</p>
<p>Before:</p>
<pre class="source-code"><code>/** @testdox It cannot be type-coerced */
public function testTypeCoercion()
{
$this->expectError(); // NB: not an exception; an error
$this->expectErrorMessageMatches("/.*MaoriNumbers could not be converted to int.*/");
$this->assertEquals(sprintf("ono: %d", MI::ONO), "ono: 6");
}
</code></pre>
<p>After:</p>
<pre class="source-code"><code>/** @testdox It cannot be type-coerced */
public function <a href="https://github.com/adamcameron/php8/blob/1.14/tests/Unit/System/Enum/WithUserCodeTest.php#L33" target="_blank">testTypeCoercion</a>()
{
$this->assertError(
function () {
$this->assertEquals(sprintf("ono: %d", MI::ONO), "ono: 6");
},
"MaoriNumbers could not be converted to int"
);
}
public function <a href="https://github.com/adamcameron/php8/blob/1.14/tests/Unit/System/Enum/CustomAssertionsTrait.php" tyarget="_blank">assertError</a>(callable $callback, string $message)
{
try {
$callback();
$this->fail("Expected an error, didn't get one");
} catch (AssertionFailedError $e) {
throw $e;
} catch (\Throwable $e) {
$this->assertStringContainsString($message, $e->getMessage());
}
}
</code></pre>
<p>This is not perfect, but it'll do for what I'm needing it for. I have a coupla instances of this I need to refactor, and both in the same package. So I'll chuck this custom assertion in a trait I guess. One of the few times I don't feel icky using traits.</p>
<hr class="narrow">
<h3>Conclusion</h3>
<p>This was not a very edifying exercise, and I don't really think anything's been improved by two out of three of these PHPUnit changes I've needed to work around. Still: it's not the end of the world, and I hasten to add for my gripes here, I still think PHPUnit is brilliant. Mostly ;-) (kidding, it's great).</p>
<p>I guess I'll finally get to write the article I wanted to write tomorrow evening.</p>
<p>The current state of this project is tagged as <a href="https://github.com/adamcameron/php8/tree/1.14" target="_blank">1.14</a>.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.comtag:blogger.com,1999:blog-8141574561530432909.post-6815172446546475872023-02-02T23:42:00.007+00:002023-02-02T23:44:33.439+00:00PHP / Doctrine/DBAL: adding a logger to an connection<p>G'day:</p>
<p>There's plenty of stuff online about how to do this the old deprecated way, but I couldn't find any example of how to do it the "current" way. So. Here's an example.</p>
<p>NB: the old way is some variant of this sort of thing:</p>
<pre class="source-code"><code>$logger = new \Doctrine\DBAL\Logging\DebugStack();
$config = new \Doctrine\DBAL\Configuration();
$config->setSQLLogger($logger);
$connection = DriverManager::getConnection(
$connectionParams,
$config
);
</code></pre>
<p>However both <samp>DebugStack</samp> and <samp>setSQLLogger</samp> are deprecated. The <samp>DebugStack</samp> thing is just a logger instance and I didn't want to use that anyhow, but being able to set the logger I <em>did</em> want to use would be nice.</p>
<p>I dug into <a href="https://github.com/doctrine/dbal/blob/3.5.3/src/Configuration.php#L68" target="_blank"><samp>setSQLLogger</samp></a>, and it says this:</p>
<pre class="source-code"><code>@deprecated Use {@see setMiddlewares()} and {@see \Doctrine\DBAL\Logging\Middleware} instead.
</code></pre>
<p>OK we'll do that then.</p>
<p>I'll spare you the code, but initially I was trying to set the middlewares (I'm not sure(*) that's the plural of "middleware", guys) on an existing connection object, and that was not working at all. Nothing was being logged. Then I discovered someone else with the same woes as me @ <a href="https://github.com/doctrine/dbal/issues/5366" target="_blank">https://github.com/doctrine/dbal/issues/5366</a>, and apparently one cannot actually do this, as the middlewares are only paid-attention-to when the connection is first created, so I needed to pass the config in during connection-creation. This is not well-documented either, but looking at the source code answered that.</p>
<p>Here's what I ended up with in my test:</p>
<pre class="source-code"><code>namespace adamcameron\php8\tests\Functional\Database;
use adamcameron\php8\tests\Integration\Fixtures\Database as DB;
use Doctrine\DBAL\Configuration;
use Doctrine\DBAL\Logging\Middleware;
use Monolog\Handler\TestHandler;
use Monolog\Logger;
use PHPUnit\Framework\TestCase;
/** @testdox Tests Database objects */
class <a href="https://github.com/adamcameron/php8/blob/1.13.1/tests/Functional/Database/DbLoggingTest.php" target="_blank">DbLoggingTest</a> extends TestCase
{
/** @testdox It can log SQL traffic */
public function testLogging()
{
$testLogger = new Logger("test");
$testHandler = new TestHandler();
$testLogger->setHandlers([$testHandler]);
$middleware = new Middleware($testLogger);
$config = new Configuration();
$config->setMiddlewares([$middleware]);
$connection = DB::getDbalConnection($config);
$result = $connection->executeQuery("SELECT * FROM numbers WHERE id = ?", [5]);
$this->assertEquals([5, 'five', 'rima'], $result->fetchNumeric());
<span class="xr xrt" data-index="filter">$withSql = array_filter($testHandler->getRecords(), function ($record) {</span>
<span class="xr xrt" data-index="filter">return in_array('SELECT * FROM numbers WHERE id = ?', $record->context);</span>
<span class="xr xrt" data-index="filter">});</span>
$this->assertCount(1, $withSql);
}
}
</code></pre>
<p>Given <a href="https://github.com/doctrine/dbal/blob/3.5.3/src/Logging/Middleware.php#L15" target="_blank"><samp>Middleware</samp></a> only supports <em>Loggers</em>, I'm not sure this is a great architectural win for DBAL: moving from a clear and on-point description of the functionality to a nebulous, vague, and actually inaccurate term like "Middleware". Oh well.</p>
<p>For the sake of completeness (and an explanation of why I needed to <span class="xr xrd u" data-index="filter">dick around filtering the log to get the one I was looking for</span>, this is what is logged for that call:</p>
<pre class="source-code"><code>array(2) {
[0] =>
class Monolog\LogRecord#363 (7) {
public readonly DateTimeImmutable $datetime =>
class Monolog\DateTimeImmutable#357 (4) {
private $useMicroseconds =>
bool(true)
public $date =>
string(26) "2023-02-02 23:27:14.519171"
public $timezone_type =>
int(3)
public $timezone =>
string(13) "Europe/London"
}
public readonly string $channel =>
string(4) "test"
public readonly Monolog\Level $level =>
enum Monolog\Level::Info : int(200);
public readonly string $message =>
string(35) "Connecting with parameters {params}"
public readonly array $context =>
array(1) {
'params' =>
array(6) {
...
}
}
public array $extra =>
array(0) {
}
public mixed $formatted =>
string(204) "
[2023-02-02T23:27:14.519171+00:00]
test.INFO:
Connecting with parameters {params}
{
"params":{
"dbname":"db1",
"user":"user1",
"password":<span class="xr xrt" data-index="redacted">"<redacted>"</span>,
"host":"mariadb",
"port":"3306",
"driver":"pdo_mysql"
}
}
[]
"
}
[1] =>
class Monolog\LogRecord#349 (7) {
public readonly DateTimeImmutable $datetime =>
class Monolog\DateTimeImmutable#348 (4) {
private $useMicroseconds =>
bool(true)
public $date =>
string(26) "2023-02-02 23:27:14.780537"
public $timezone_type =>
int(3)
public $timezone =>
string(13) "Europe/London"
}
public readonly string $channel =>
string(4) "test"
public readonly Monolog\Level $level =>
enum Monolog\Level::Debug : int(100);
public readonly string $message =>
string(65) "Executing statement: {sql} (parameters: {params}, types: {types})"
public readonly array $context =>
array(3) {
'sql' =>
string(34) "SELECT * FROM numbers WHERE id = ?"
'params' =>
array(1) {
...
}
'types' =>
array(1) {
...
}
}
public array $extra =>
array(0) {
}
public mixed $formatted =>
string(194) "
[2023-02-02T23:27:14.780537+00:00]
test.DEBUG:
Executing statement:
{sql}
(parameters: {params}, types: {types})
{
"sql":"SELECT * FROM numbers WHERE id = ?",
"params":{
"1":5
},
"types":{
"1":2
}
}
[]
"
}
}
</code></pre>
<p>(Well <em>that</em> pads the word count…)</p>
<p>I like how it's had the presence of mind to <span class="xr xrd u" data-index="redacted">redact the password</span> from the connection params.</p>
<p>Anyway, that's it. I just wanted that code written down somewhere were it can be found.</p>
<p>Righto.</p>
<p>-- <br>Adam</p>
<hr class="narrow">
<p>(*) In fact I've very bloody sure it oughtn't be (and, yesyesyes, I know it's been adopted as such, but this is LCD thinking at work. "-ware" already makes it a mass noun, so… <em>plural</em>).</p>Adam Cameronhttp://www.blogger.com/profile/04830762402027484810noreply@blogger.com