G'day:
Familiar boilerplate about how this is one in a series follows. Please note that I initially intended this entire exercise to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:
- Intro / Nginx
- PHP
- PHPUnit
- Tweaks I made to my Bash environment in my Docker containers
- MariaDB
- Installing Symfony
- Using Symfony
- Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer (this article)
- I mess up how I configure my Docker containers
- An article about moving files and changing configuration
- Setting up a Vue.js project and integrating some existing code into it
- Unit testing Vue.js components
As indicated: this is the eighth article in the series, and follows on from Part 7: Using Symfony. Unlike the previous articles in this series, it's reasonably stand-alone: it doesn't rely on any of the Nginx / PHP / MariaDB / Symfony stuff I've written about so far. I do continue building on my Docker setup, but in an isolated fashion. So I dunno if you'd specifically benefit from going back and reading the earlier articles in the series to contextualise this article. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject. I've said that previous sentence at the beginning of most of the articles, but it is particularly true of my exposure to Vue.js and everything I touch on in this article. One can hardly even call it "exposure": I've heard of Vue.js, I know it exists, and I have the Vue.js homepage open in the adjacent tab. That is the entirety of my exposure. Prior to this exercise I had not even heard of Puppeteer, and have only messed with Mocha and Chai very very superficially (JavaScript TDD: getting Mocha tests running on Bamboo). But such a distinct lack of knowledge has never stopped me blathering about stuff before, and I'm too old to change my ways now, so here goes.
Firstly: some full disclosure. I am going to write this article in good TDD sequence: tests first, then the web page I'm testing. In reality I did the Vue-powered web page first and then went "erm… ain't no way I'm gonna write this up without also having tests for it". Also the Vue side of things was so simple to get a "dynamic" "G'day World" page working that I hardly had anything to write about. So I'm gonna start where I ought to have, with the question of "how the hell am I going to test this?"
That question might seem really daft. My G'day World webpage looks like this:
Basically there's three testable mark-up elements: a title, heading and a paragraph all of which have "G'day World" in them. My intended Vue.js version of this will differ only in that it has "G'day world via Vue" as its intended content. When testing the flat mark-up version of this, all I need to do is curl the page, and then use a DOM parser to locate the <title>, <h1> and first <p> elements, and check their innerText values, eg:
/** @coversNothing */
public function testGdayWorldHtmlReturnsExpectedContent()
{
$expectedContent = "G'day world";
$client = new Client([
'base_uri' => 'http://webserver.backend/'
]);
$response = $client->get('gdayWorld.html');
$this->assertEquals(Response::HTTP_OK, $response->getStatusCode());
$html = $response->getBody();
$document = new \DOMDocument();
$document->loadHTML($html);
$xpathDocument = new \DOMXPath($document);
$hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
$this->assertCount(1, $hasTitle);
$hasHeading = $xpathDocument->query('/html/body/h1[text() = "' . $expectedContent . '"]');
$this->assertCount(1, $hasHeading);
$hasContent = $xpathDocument->query('/html/body/p[text() = "' . $expectedContent . '"]');
$this->assertCount(1, $hasContent);
}
This approach is no good for the Vue.js-driven version of this, because the $expectedContent won't be present in the mark-up: Vue.js will be swapping it in at runtime, dynamically. All the mark-up will have in it is some placeholder like {{message}}. I can't just test for that, because it does not test that the Vue code is actually running correctly.
My first task is to find out how people are testing dynamically-created mark-up documents these days. I just googled that. Most of the material specific to Vue-managed stuff revolves around testing Vue components. This is where I hope to get to (in the next and maybe final article in this series), but for now my first "G'day world via Vue" exercise will be based on the "Hello world" example in the Vue docs, and all be inline within one HTML document, eg:
<!DOCTYPE html>
<html>
<head>
<title>My first Vue app</title>
<script src="https://unpkg.com/vue"></script>
</head>
<body>
<div id="app">
{{ message }}
</div>
<script>
var app = new Vue({
el: '#app',
data: {
message: 'Hello Vue!'
}
})
</script>
</body>
</html>
No Vue components yet. After a bit more googling I landed on Puppeteer seeming like the way to go to test this sort of thing. From their home page:
Sounds cool.
The first thing I need to do is to add a Node.js container into my ever-growing family of containers. My initial approach was with this sorta thing in the docker/node/Dockerfile:
FROM node
WORKDIR /usr/share/fullstackExercise/
RUN npm install puppeteer
RUN npm install mocha
RUN npm install chai
RUN npm install chai-as-promised
This would work, but I recalled something about Node.js having a package.json file, similar to Composer's composer.json (I say similar... Composer is based on NPM after all). Anyhow this seemed like a better way to go so I created one of those (with help from NPM itself, which has a wizard to generate the baseline file, via npm init). I ended up with this:
{
"name": "fullstackexercise",
"description": "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers",
"version": "2.6.0",
"main": "index.js",
"directories": {
"test": "tests"
},
"devDependencies": {
"chai": "^4.2.0",
"mocha": "^8.2.1",
"puppeteer": "^5.5.0",
"chai-as-promised": "^7.1.1"
},
"scripts": {
"test": "mocha tests/**/*.js"
},
"repository": {
"type": "git",
"url": "git+https://github.com/adamcameron/fullstackExercise.git"
},
"author": "",
"license": "GPL-3.0-or-later",
"bugs": {
"url": "https://github.com/adamcameron/fullstackExercise/issues"
},
"homepage": "https://github.com/adamcameron/fullstackExercise#readme"
}
That's a mix of the stuff npm init created, and the equivalent values from composer.json. This means the Dockerfile just becomes:
FROM node
WORKDIR /usr/share/fullstackExercise/
RUN npm installAnd the relevant section from docker-compose.yml:
node:
build:
context: ./node
volumes:
- ..:/usr/share/fullstackExercise
- ./node/root_home:/root
stdin_open: true
tty: true
networks:
- backend
There's nothing unfamilar here. This all installs "fine":
Creating network "docker_backend" with driver "bridge"
Building nginx
[...]
Successfully built 7cb155649c3b
Successfully tagged docker_nginx:latest
Building php-fpm
[...]
Successfully built 2d3e6ba6d177
Successfully tagged docker_php-fpm:latest
Building mariadb
[...]
Successfully built 1f05ad3e3ad3
Successfully tagged docker_mariadb:latest
Building node
[...]
Step 5/5 : RUN npm install
[...]
Successfully built c9e1f709c33e
Successfully tagged docker_node:latest
Creating docker_mariadb_1 ... done
Creating docker_node_1 ... done
Creating docker_php-fpm_1 ... done
Creating docker_nginx_1 ... done
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$
adam@DESKTOP-QV1A45U:/mnt/c/src/fullstackExercise/docker$ docker exec --interactive --tty docker_node_1 /bin/bash
root@33992838c2fe:/usr/share/fullstackExercise# npm list
fullstackexercise@2.6.0 /usr/share/fullstackExercise
+-- chai-as-promised@7.1.1
+-- chai@4.2.0
+-- mocha@8.2.1
`-- puppeteer@5.5.0
root@33992838c2fe:/usr/share/fullstackExercise#
root@cd18a4f3f4d4:/usr/share/fullstackExercise# node -i
Welcome to Node.js v15.5.1.
Type ".help" for more information.
>
So far so good. I've grabbed some code from the Puppeteer getting started docs, and will give that a whirl:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
await browser.close();
})();
I don't wanna make screenshots of web pages, but it'll do for a start:
undefined
>
> (async () => {
... const browser = await puppeteer.launch();
... const page = await browser.newPage();
... await page.goto('https://example.com');
... await page.screenshot({path: 'example.png'});
...
... await browser.close();
... })();
Promise { <pending> }
> Uncaught Error: Failed to launch the browser process!
/usr/share/fullstackExercise/node_modules/puppeteer/.local-chromium/linux-818858/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
at onClose (/usr/share/fullstackExercise/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193:20)
>
Oh. That's not what I wanted to see. I googled / stack-overflow-ed this, and yeah it seems that Puppeteer makes some assumptions about what libs are installed, and I'll be short some. I saw various different lists of stuff it was missing, and different approaches to remedying it, but for starters I figured I'd just install that lib in my container, and see what happens next. I update docker/node/Dockerfile thus:
FROM node
RUN apt-get update
RUN apt-get install libnss3-dev --yes
WORKDIR /usr/share/fullstackExercise/
RUN npm install
After that I dropped my containers, rebuilt them, and ran that code again. I'm not expecting it to work yet, I just want to see that Puppeteer stops complaining about that library being missing:
Promise { >pending< }
> Uncaught Error: Failed to launch the browser process!
/usr/share/fullstackExercise/node_modules/puppeteer/.local-chromium/linux-818858/chrome-linux/chrome: error while loading shared libraries: libatk-bridge-2.0.so.0: cannot open shared object file: No such file or directory
It's a different library this time, so I'm calling this an "interim success". I'll rinse and repeat until I stop getting missing lib errors, and report back in a minute. Or so…
OK, so after some trial and error, I nailed it down to these missing libraries:
FROM node
RUN apt-get update
RUN apt-get install libnss3-dev libatk-bridge2.0-dev libx11-xcb1 libdrm-dev libxkbcommon-dev libgtk-3-dev libasound2-dev --yes
WORKDIR /usr/share/fullstackExercise/
RUN npm install
All I did to track them down is to copy and paste the reference to the lib in the error and search for them on Ubuntu Packages Search. Sometimes the library name was not an exact match with the reference mentioned in the error. Anyway, this lot works for me. Given all the stuff I had read on this listed different combinations of missing packages, I'm guessing it'll all be platform- and situation-specific, so don't treat that list as canonical. Righto, this is what happens now when I run the code:
> Uncaught Error: Failed to launch the browser process!
[0113/143914.607623:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
OK, what? I googled about and found a work around for this on Stack Overflow: Running headless Chrome / Puppeteer with --no-sandbox. It just showed me that one can pass that arg to Puppeteer, adn that makes the problem go away.
Warning
This is just a workaround and is not secure. A proper fix would be to not be running my code as root. However as this is experimental code and will never see production or be exposed externally, I'm happy to just bodge it. This is not an exercise in showing people how to secure Docker containers, and I would never want to go down that path, as I'm not qualified to discuss such things.
The code in question is now this:
const browser = await puppeteer.launch({args: ['--no-sandbox']});
And when I run this, I sat there looking at this for a while:
... })();
Promise { <pending> }
>
Nothing. Then I realised I'm a div cos this code doesn't output anything to stdout, it writes a file. So if I look in the correct place for the output, things are better:
-rwxrwxrwx 1 node node 19373 Jan 13 15:12 example.png*
root@00b5711ba57f:/usr/share/fullstackExercise#
And here it is:
Perfect!
What I want to get out of this though is inspecting the actual document, so I have further updated the code to be this:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({args: ['--no-sandbox']});
const page = await browser.newPage();
await page.goto('https://example.com');
const title = await page.title();
console.info(`The title is: ${title}`);
await browser.close();
})();
Here I'm just grabbing the title and outputing it. This will be part of my first test, so it's a reasonable next step here. And it works:
Promise { <pending> }
> The title is: Example Domain
>
I'm now content that Puppeteer works, and I have some code that executes, but I need to get this stuff into some tests now.
I've got Mocha (test framework), Chai (assertion library) and chai-as-promised (promise-oriented assertion library for Chai) installed. I'm not going to do a tutorial on running Mocha tests here, I'm just gonna look at my code. Here's my first test:
let puppeteer = require('puppeteer');
let chai = require("chai");
let chaiAsPromised = require("chai-as-promised");
chai.use(chaiAsPromised);
let should = chai.should();
describe("Baseline test of vue.js working", function () {
this.timeout(5000);
it("should return the correct page title", async function () {
let browser = await puppeteer.launch( {args: ["--no-sandbox"]});
let page = await browser.newPage();
await page.goto("http://webserver.backend/gdayWorld.html");
await page.title().should.eventually.equal("G'day world");
await page.close();
await browser.close();
});
});
Notes:
- This top bit is all just pulling in all the libs I need to use in the test.
- I had to put this timeout in, as I was getting a timeout error sometimes (see error below).
- We've already discussed this lot, it's from the Puppeteer example code.
- For now, I am running my tests against the flat-HTML version of the file. I'm doing this because I'm kinda testing my usage of the test framework for now, and I don't have the Vue.js-driven version of the file yet.
- And here's the assertion version of the key line in the sample code. I like this fluent syntax.
- The eventually bit is the key bit from chai-as-promised: it handles all the async / promise carry one, and once it's done, runs the assertion. Nice.
- And some tear down stuff. Note that I will refactor some of this stuff out from being inline in the test in the next iteration. For now I'm largely just copying and pasting the example code into a test, and tweaking it so it'll run.
And run it indeed does!
> fullstackexercise@2.6.0 test
> mocha tests/**/*.js
Baseline test of vue.js working
✓ should return the correct page title (4026ms)
1 passing (4s)
root@00b5711ba57f:/usr/share/fullstackExercise#
I like this thing that npm did for me when I created package.json. It asks me how to run my tests, and then writes a test script for me:
"scripts": {
"test": "mocha tests/**/*.js"
},
This is how I can just go npm test to run my tests. Very mundane to seasoned Node.js users obviously, but I am just a noob.
Oh, that timeout error I was getting:
> fullstackexercise@2.6.0 test
> mocha tests/**/*.js
Baseline test of vue.js working
1) should return the correct page title
0 passing (2s)
1 failing
1) Baseline test of vue.js working
should return the correct page title:
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/usr/share/fullstackExercise/tests/functional/public/GdayWorldViaVueTest.js)
at listOnTimeout (node:internal/timers:556:17)
at processTimers (node:internal/timers:499:7)
npm ERR! code 1
npm ERR! path /usr/share/fullstackExercise
npm ERR! command failed
npm ERR! command sh -c mocha tests/**/*.js
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-01-13T16_16_36_577Z-debug.log
root@00b5711ba57f:/usr/share/fullstackExercise#
So that's why I added that timeout. Initially I thought I was doing the async stuff wrong given the hinting it was giving me, but it turned out literally to be the case that the test was taking too long to run. I guess the HTTP request and document render takes time.
Brimming with confidence, I now set out do write my other tests, and at the same time refactor the test to be a bit more sensibly organised:
let puppeteer = require('puppeteer');
let chai = require("chai");
let chaiAsPromised = require("chai-as-promised");
chai.use(chaiAsPromised);
let should = chai.should();
describe("Baseline test of vue.js working", function () {
let browser;
let page;
this.timeout(5000);
const expectedText = "G'day world";
before (async function () {
browser = await puppeteer.launch( {args: ["--no-sandbox"]});
page = await browser.newPage();
await page.goto("http://webserver.backend/gdayWorld.html");
});
after (async function () {
await page.close();
await browser.close();
});
it("should return the correct page title", async function () {
await page.title().should.eventually.equal(expectedText);
});
it("should return the correct page heading", async function () {
let headingText = await page.$eval("h1", headingElement => headingElement.innerText);
headingText.should.equal(expectedText);
});
it("should return the correct page content", async function () {
let paragraphContent = await page.$eval("p", paragraphElement => paragraphElement.innerText);
paragraphContent.should.equal(expectedText);
});
});
- I've shifted all the Puppeteer setup/teardown into the appropriate handler functions, so they're only in there once for all three tests.
- This really makes the tests very simple and focused.
- For the tests needing the content of an element, I'm using this $eval method which passes the element to the callback once it's finally fetched…
- … this way we don't need the async-handling "should.eventually code, and just should is fine.
And when I run this:
> fullstackexercise@2.6.0 test
> mocha tests/**/*.js
Baseline test of vue.js working
✓ should return the correct page title
✓ should return the correct page heading
✓ should return the correct page content
✓ should return the correct page heading
3 passing (3s)
root@00b5711ba57f:/usr/share/fullstackExercise#
Wonderful. Now I'm happy my tests are doing the right thing (on the wrong file), I'm ready to implement my Vue.js version of the page.
The mark-up I have come up with has a slight change from the flat HTML version (public/gdayWorld.htm). The new file is public/gdayWorldViaVue.html:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title id="title">{{ message }}</title>
</head>
<body>
<div id="app">
<h1>{{ message }}</h1>
<p>{{ message }}</p>
</div>
</body>
</html>
The Vue objects need to be hung off an element via an id, hence that <div id="app"> being added to handle the heading and content. But I'm gonna need a second Vue object to deal with the <title>. I presume there's a way around this, but I've not got that far yet.
Before I add the Vue part of the code, I'm going to update my tests to hit this new file, and also expect the updated text values:
const expectedText = "G'day world via Vue";
before (async function () {
browser = await puppeteer.launch( {args: ["--no-sandbox"]});
page = await browser.newPage();
await page.goto("http://webserver.backend/gdayWorldViaVue.html");
});
As Vue.js is not wired in to the new file yet, the tests will fail, because the mark-up just contains the placeholder text ({{ message }}), and Vue has not rendered the dynamic values in their place yet.
> fullstackexercise@2.6.0 test
> mocha tests/**/*.js
Baseline test of vue.js working
1) should return the correct page title
2) should return the correct page heading
3) should return the correct page content
0 passing (3s)
3 failing
1) Baseline test of vue.js working
should return the correct page title:
AssertionError: expected '{{ message }}' to equal 'G\'day world via Vue'
+ expected - actual
-{{ message }}
+G'day world via Vue
at /usr/share/fullstackExercise/node_modules/chai-as-promised/lib/chai-as-promised.js:302:22
at processTicksAndRejections (node:internal/process/task_queues:93:5)
at async Context.<anonymous> (tests/functional/public/GdayWorldViaVueTest.js:29:9)
2) Baseline test of vue.js working
should return the correct page heading:
AssertionError: expected '{{ message }}' to equal 'G\'day world via Vue'
+ expected - actual
-{{ message }}
+G'day world via Vue
at Context.<anonymous> (tests/functional/public/GdayWorldViaVueTest.js:35:28)
at processTicksAndRejections (node:internal/process/task_queues:93:5)
3) Baseline test of vue.js working
should return the correct page content:
AssertionError: expected '{{ message }}' to equal 'G\'day world via Vue'
+ expected - actual
-{{ message }}
+G'day world via Vue
at Context.<anonymous> (tests/functional/public/GdayWorldViaVueTest.js:41:33)
at processTicksAndRejections (node:internal/process/task_queues:93:5)
npm ERR! code 3
npm ERR! path /usr/share/fullstackExercise
npm ERR! command failed
npm ERR! command sh -c mocha tests/**/*.js
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-01-13T16_57_07_203Z-debug.log
root@00b5711ba57f:/usr/share/fullstackExercise#
This is good news: we have a failing test! Now we can do the work. The Vue.js code to do all the work is very very very simple (public/assets/scripts/gdayWorldViaVue.js):
let appData = {message: "G'day world via Vue"};
new Vue({el: '#title', data: appData});
new Vue({el: '#app', data: appData});
As per above, this hangs a Vue object off each of the #title and #app <div> elements, and binds the appData object with the message in it to them. That's it.
Back in the HTML file I need to load in Vue.js and my own script file:
</div>
<script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js"></script>
<script src="assets/scripts/gdayWorldViaVue.js"></script>
</body>
</html>
Now I can run my tests, and test that Vue.js is indeed loading the message into the title, heading and content elements:
> fullstackexercise@2.6.0 test
> mocha tests/**/*.js
Baseline test of vue.js working
✓ should return the correct page title
✓ should return the correct page heading
✓ should return the correct page content
3 passing (3s)
root@00b5711ba57f:/usr/share/fullstackExercise#
Boom! It all works. That took a while, eh? (albeit longer to write it up and then to read about it, than to do the actual work).
For the sake of completeness, let's have a look at the page rendering in a browser. This is, after all, the "end user" requirement. Not simply that some tests pass ;-)
Perfect.
Now I admit that was an awful lot of work to test a full Vue-driven HTML document, and I won't be writing further tests like this (probably) because I'll be testing at Vue component level for the rest of the work. But I needed to get Node, Mocha and Chai all working, and - even though my initial Vue-driven test page was completely contrived - I still needed to test it.
In the next article I'll break this lot down into components and test those. All the time I will still have this test in place testing the end result expectations, and I think at least in the short term this is handy. But first I am gonna look at my green tests for a while, and have a beer. I'll be back on deck with this lot tomorrow.
Righto.
--
Adam
