Friday 27 November 2015

Unit testing (PHP): keeping sight of the intent of the tests

An article with some actual code in it! That'd become a rarity. This is actuallya real-world challenge I'm faced with in backfilling some unit tests. Yeah, I know I'm an advocate of TDD so that should mean there's no "backfilling" of tests: they all get done up front. But for reasons I won't go into... I'm backfilling testing for this function.

Here's the function:

public static function loadValidatorMetadata(ClassMetadata $metadata)
    $metadata->addPropertyConstraint('languageId', new Assert\Range(
        ['min' => 1, 'max' => self::$maxLegacyLanguageId]

    $metadata->addPropertyConstraint('destinationId', new Assert\Range(['min' => 1]));

    $metadata->addConstraint(new Assert\Callback('validateArrivalDate'));
    $metadata->addConstraint(new Assert\Callback('validateNights'));

    $metadata->addPropertyConstraint('firstName', new Assert\NotBlank());
    $metadata->addPropertyConstraint('firstName', new Assert\Length(
        ['min' => 1, 'max' => GroupEnquiryRepository::SHORT_STRING_MAX_LENGTH]
    $metadata->addPropertyConstraint('surName', new Assert\NotBlank());
    $metadata->addPropertyConstraint('surName', new Assert\Length(
        ['min' => 1, 'max' => GroupEnquiryRepository::SHORT_STRING_MAX_LENGTH]

    $metadata->addPropertyConstraint('email', new Assert\NotBlank());
    $metadata->addPropertyConstraint('email', new Assert\Email());

    $metadata->addPropertyConstraint('groupTypeId', new Assert\Range(['min' => 1]));
    $metadata->addPropertyConstraint('groupSize', new Assert\NotNull());
    $metadata->addPropertyConstraint('groupSize', new Assert\Range(
        ['min' => GroupEnquiryService::MIN_GROUPS_THRESHOLD, 'max' => BookingService::MAXIMUM_PEOPLE]
    $metadata->addPropertyConstraint('youngestAge', new Assert\Range(
        ['min' => 1, 'max' => GroupEnquiryService::MAX_AGE]

    $metadata->addPropertyConstraint('telephoneCountryCode', new Assert\NotBlank());
    $metadata->addPropertyConstraint('telephoneCountryCode', new Assert\Length(
        ['min' => 1, 'max' => GroupEnquiryRepository::SHORT_STRING_MAX_LENGTH]
    $metadata->addPropertyConstraint('phone', new Assert\NotBlank());
    $metadata->addPropertyConstraint('phone', new Assert\Length(
        ['min' => GroupEnquiryService::MIN_TELEPHONE_NUMBER_LENGTH]
    $metadata->addConstraint(new Assert\Callback('validateMobileCountryCode'));
    $metadata->addConstraint(new Assert\Callback('validateMobile'));

    $metadata->addConstraint(new Assert\Callback('validateAdditionalInfo'));

First things first: yeah, normally I'd not write a function that's 40-odd lines long, but in this case there's very little actual logic, plus it's only setting some validation config. BTW this is using the Symfony 2 validation framework (Symfony: validation), as implemented by the Silex microframework (ValidatorServiceProvider: validating objects).

To provide context, we have a class - basically a validation bean - thus:

class GroupEnquiry
    public $languageId;
    public $destinationId;
    public $arrivalDate;
    public $nights;
    public $firstName;
    public $surName;
    public $email;
    public $telephoneCountryCode;
    public $phone;
    public $mobileCountryCode;
    public $mobile;
    public $groupTypeId;
    public $groupSize;
    public $youngestAge;
    public $additionalInfo;

And that loadValidatorMetadata() function provides the constraints for Symfony to enforce. As you can see: there's range validation, existence validation, length validation, date validation, and a coupla compound constraints implemented via callbacks.

So, anyway, I need to write the testing for loadValidatorMetadata(). Initially I was staring at the code going and going "how the hell?".

Then I thought: I'll assert the addConstraint() and addPropertyConstraint() methods are called for the correct properties, and the correct number of times. But that seemed unhelpful. What's the point of checking that some constraint is applied to a property, but not checking what sort of constraint. IT all seemed too contrived.

Then I thought that the constraints themselves don't need testing here because I'm either testing our callbacks elsewhere, or Symfony have done their own testing. In theory this is a fine notion, but in reality it does not actually help me test our validation code works.

I was stumped as to how to test this stuff, without basically replicating the function in parallel for test. And that clearly sucks as far as effort and common sense goes. It was the wrong approach.

Thursday 26 November 2015

Guest author: a response to "Why good developers write bad code"

I could get used to this. Another one of my readers has provided my copy for today, in response to my earlier guest-authored article "Why good developers write bad code". This originally was a comment on that article (well: it still is) but it stands on its own merit, and Brian's cleared me to reproduce it as an article. Brian's a mate, one of my colleagues at The quote below is unabridged. I have corrected one typo, and the links are my own.

Brian says:

Really thought provoking post, Adam, and my immediate impression is that your mate seems to be describing an awful lot of dysfunctionality here. I'll get my declaration of interest out in the open straightaway and own up to being a massive Agile/XP fanboy, and consequently I think much of what your mate describes could be cured by the adoption of Agile/XP practices.

The "...but then he gets a chunky piece of work and, more often than you'd expect, it goes astray" quote is very revealing, because it would appear that there's no effort made to break large requirements down into smaller ones and we're on a slippery slope from here on in.

Developers often receive very coarse grained requirements and irrespective of our natural coding ability, we're not naturally adept at managing large pieces of work. That's not to say we can't manage large requirements, but without training, mentoring and support developers who might be first rate coders, can often struggle. In this case, it sounds like the developer in question is trying to eat an elephant in one sitting, and then being left alone to go dark and "fail". Reading between the lines it seems that this is accompanied by a seat of the pants estimate, which becomes a commitment as soon as he's uttered it, and before long we're into death march territory.

Also I wouldn't want to blame the developer in question, because as your mate points out, the same thing keeps happening time and again but there's no intervention or help from the guy's colleagues or his manager. Clearly they're at a loss as to how to fix the problem but unless someone can show him a better way, how is he going to improve?...

Anyway... if I was working with him I'd strongly encourage him to forget about eating the whole elephant, and just start by trying to nibble its trunk. Use TDD religiously from the outset, stick to the red-green-refactor cycle and You Ain't Gonna Need It (even more religiously), avoid Big Design Up Front, and let the application architecture evolve as he uncovers smells in the code that he's writing.

See how far he gets nibbling the elephant's trunk in one day, and if all goes then we'll reconvene and decide on the next bit of the elephant to consume, and carry on in that fashion. If he takes longer than a day - and we'll know that because we have daily stand-ups right? - then we'll pair up and see what the problem is. Invariably if the guy's a first rate coder, it will only be because he's bitten off more than he can chew, so get him to stop, re-plan the work, and decrease the size of the immediate task in front of him so he can actually finish it. In short, KISS, keep tasks small and try to conquer complexity. Writing code is intrinsically hard, we should actively avoid anything that makes it harder.

I think it's also worth bearing in mind that many, if not most developers, are naturally quite reluctant to provide honest progress reports if things aren't going well. Combine that with our tendency to give hopelessly optimistic estimates and we're already in a very bad place. I've found that it's almost endemic in our industry that developers are scared of giving bad news to anyone, let alone their line manager or project manager. Developers are usually clever people who spent much of their academic careers being praised for their intellectual prowess, and to be put in a position where they are suddenly the bad guys because they didn't hit an arbitrary deadline can be very distressing and something to avoid at all costs.

If there was just one thing I would wish for in the industry it would be to reform the attitude that project managers, team leads and developers have to estimation. Estimates should be treated by all stakeholders as just that, they are not cast iron commitments written in blood. They should be based on evidence (dude we go through six stages of peer review on every task, why did you give an estimate of 10 minutes?) and when the evidence is demonstrating that our estimates are wrong, then remaining estimates should be re-calibrated. All this is only possible if managers and developers are prepared to create a culture where realistic and responsible estimation, combined with predictable delivery is championed over masochistic estimation and constant failures to hit deadlines.

One final point, when your mate writes "it's a good idea to keep new features *out* of QA for as long as possible", I totally disagree! :-) I'd want feedback from the QA team as soon as possible. The shorter the feedback cycle the better and in an ideal world the QA team would be writing automated acceptance tests in parallel with my development efforts. I'm a firm believer in the agile principle that "Working software is the primary measure of progress" and feedback from QA I'm just guessing that the software I'm writing is actually working.

Sunday 22 November 2015

ColdFusion: request for transparency and inclusion from Adobe

A few days back I indicated some ire that a (IMO) wayward ER had been implemented for ColdFusion 2016: "ColdFusion: a piece of functionality should do one thing, and do it well". Poor old David Epler who raised it seems to feel a bit put upon cos it was something he raised three years ago, and with no further consultation Adobe have now implemented a solution. Without indication it was going to happen, or any communication with David about it, who now think it's perhaps not as good an idea as he originally might have thought.

This also got me thinking about another new feature in ColdFusion 2016: this whole ordered / sorted structs carry on (see "ColdFusion 2016: ordered/sorted structs"). Where's the ER for this one? I can't find one. So where did the impetus to do this work come from? Did Adobe just decide to do it off their own bat? If so why? Why the heck did they decide to do that work? Instead of any of a number of other features the community have actually asked for. Obviously there are probably some back channels via which people can ask for ERs - although there shouldn't be - but even if this came from some PHB or one of his minions at one of Adobe's special enterprise clients... why does this mean this is what gets implemented? All Adobe ought to say to the PHB et al in this case is "raise the ticket, engender some interest from the community, and let's see what people think".

Conversely the ColdFusion Team might have come up with this themselves. They really shouldn't do that, as - as far as I can tell - they are completely ignorant of CFML usage (I judge this based on everything they do and say about CFML, basically), so their decisions are not informed ones. But by accident they might have a decent idea... in which case they should... raise the ticket, engender some interest from the community, and let's see what people think. They definitely are not qualified by themselves to make decisions as to the direct on CFML.

I'm on the ColdFusion Pre-release Programme, and am under NDA to not divulge or discuss anything that gets mentioned on said programme (some people claim the PR has some sort of Fight-Club-esque rules about even mentioning it, or participation therein, but this is not true. They just want to make themselves come across as being "special". They do... but not in the way they think). So I won't. However I can speak freely on topics that have not had any discussion at all. And two of these are:

  1. Dave's issue about adding encoding functionality to <cfoutput>. This has been apparently implemented without any mention whatsoever on the pre-release.
  2. This sorted / ordered struct stuff. All I can say about that is that it exists in ColdFusion 2016. And there has been no discussion about its suitability or necessity for ColdFusion 2016 at all. Make of that what you will.
This is absolutely the wrong way of going about things. The ColdFusion Team might be fine Java developers (hey, they might be), but they seem woefully uninformed about CFML, and - as I have said already - don't use it, and don't really seem to have much of a handle on how we use it. They should not be making opaque decisions as to what goes into it. They have a CAB and a Pre-Realease Programme for this, and these decisions should be made there. And any work that is planned to be undertaken should be discussed out in the open, on the public bug tracker. An ER should be raised, public consultation should be allowed, and then the plan of how (or "if") the functionality will be undertaken should be proposed in public for further discussion. Not all opinions should necessarily be given equal weight (there's some bloody stupid opinions out there; some of them are my own), but they should be at least heard.

Here's an example of how it should be done, Here's the proposed schedule and feature-set for Java 9. There's no reason Adobe could not do that, at least at this high level. They could caveat the timeline with "all things being equal, and subject to change due to unforeseen circumstances", and they could caveat the feature list with "this is not a promise, but it's a guideline, and is also subject to change". We're all grown-ups and we understand things might prove tricky or not worth it, or something else might come along that's more important and needs to bump one of the "nice to have" features. And timeframes can change, etc. There's not even any commercial sensitivity to this, as ColdFusion isn't competing with anything. Well, other than "itself and its community" at times, I think.

PHP has similar timelines and have pretty organised RFC documentation these days for new features. I presume other language projects - well successful language projects - are similar.

Even LAS (those behind the Lucee CFML project, and the .lucee language) are heading in that direction, albeit in a more casual sort of way... but they're still finding their feet so it will probably round out as time passes.

I think the chief reason the ColdFusion Team don't take an inclusive communicative approach is that "inclusion" and "communication" are just devoid from their collective psyche, for whatever reason. They seem to treat their community with a degree of contempt that they seem to be "handing down manna from heaven" when they give us a ColdFusion release.

I think it's fine that Adobe marshal suggestions from their own team and from the "dark" part of their user base. But then they should open that up to the rest of the community so we can put our oar in. We have a great wealth of expertise in CFML usage to gauge how a feature might fit, and we've also got a great resource in the form of Sean Corfield who actually has designed languages in the past, so kinda knows how these things work (others in the CFML community might have experience here too, but I don't know about it). There's a few really old hands from the CAB and PR programme still around too, and they/we are kinda used to discussing how CFML should come together too.

Adobe have done a chunk of fine work in CFML, this is for sure. But they've also done some woeful shite, which I can't help but think never would have happened if they actually engaged their user base. As far as I can see there's no published plan for ColdFusion 2016, so I suspect there isn't really one. So it's not too late for them to kinda organise themselves a bit more, and include the rest of us in the language planning process. This would be bloody easy. For any public ER out there, simply tell us on Twitter or on Slack or on their own blog that they're looking at implementing it. And then encouraging people to have input. If there's not a public ER... create one, then do the same. Adobe have a vast pool of free planning resource. They should be using it.

Start using it.


Friday 20 November 2015

ColdFusion: a piece of functionality should do one thing, and do it well

One of my pet hates is having a function called doX(), and doX() does X, but it also does Y.

An example of this is CFML's isValid() function which not only checks validity - as one might expect - but also trims any string values its validating first. The function is not called isValidAfterTrim(), so it should not do the trimming. If one wants to trim a value before validating it... call trim() then isValid(). Simple. Another example is <cfdump> which picked up and abort attribute along the way. This was because some mungbean said "well usually I want to dump and abort, so why can't dump just abort too". And for some stupid reason Adobe went "oh yeah... let's do that". No. If you want to abort after you dump... do an abort after your dump. Don't change <cfdump> to be <cfdumpandabortifyoulike>. That's dumb. At least the Lucee ER to have <cfinclude> also do an abort got rejected ("Add optional abort Attribute to cfinclude"). FFS.

The reason to this is to keep units of functionality small, for starters. It makes them easier and more coherent to test. Also the more functionality one piles into a function (and for this purpose, a tag is just a function that looks funny ;-), the more potential for it being broken as it's maintained. Plus it makes code less clear and less clean.

A function should do one thing, and do it well.

Anyway, today I got wind of this ER: "encodeFor attribute for cfoutput, writeOutput". The intent is honourable, but it suffers from trying to do too much. The gist of it is:

While ColdFusion 10 added the various ESAPI encodeFor* functions, it is dependent upon the developer to properly wrap location where used with the appropriate function (e.g. <cfoutput>#EncodeForHTML(</cfoutput> ). Adding an attribute encodeFor negates the need for wrapping individual variables and would process the entire block contained within <cfoutput>cfoutput>cfoutput> for anything within #'s with the appropriate ESAPI EncodeFor* function specified.

On the face of it this sounds reasonable. It still smacks of "doing too much", so I echoed my position as much.

I sometimes doubt I'm being too dogmatic about these things, so I asked on the #CFML Slack channel, and John Whish came up with a great counter example:

<cfoutput encodeFor="HTML">
    <a href="whatever.cfm?" onclick="get(">Go to whatever.cfm?</a>

What's this gonna do? Encode all the expressions within it for HTML. Which is only correct for should be encoded for URL, JavaScript and URL for each of its three respective uses. However a less than discerning dev might not "get" that, and now their output is encoded incorrectly.

So what we'd need to then do is this:

<cfoutput encodeFor="HTML"></cfoutput>
    <a href="whatever.cfm?id=<cfoutput encodeFor="URL"></cfoutput>" onclick="get(<cfoutput encodeFor="JavaScript"></cfoutput>)">Go to whatever.cfm?id=<cfoutput encodeFor="URL"></cfoutput></a>

What a mess. What we could do is this:

    <a href="whatever.cfm?id=#encodeForUrl(" onclick="get(#encodeForJavaScript(">Go to whatever.cfm?id=#encodeForUrl(</a>

Look familiar? Yeah... we already have the correct (and safe) implementation in place.

We don't need to mess with something that's supposed to just output, and make it do something different as well. It's a bad approach to designing the language.

I reckon this ER should be pulled. It's a pity the work has already been done...



ColdFusion: help one of our community members with a PDF problem

One of our community members, Prabha (I dunno the rest of their name), is having some problems with PDFs. They came to me for help, but I don't know the first thing about PDFs nor how to manipulate them with ColdFusion. And to be completely honest, I am completely content maintaining that ignorance. But I know a bunch of people out there do know how to do this stuff, so they might be able to help.

The question is this one, on StackOverflow: "When exporting PDF with CFChart images in ColdFusion It shows embedded font error". I can't actually make head or tail of it, but there seems to be two things:

They're getting this error:

"cannot extract the embedded font 'PCBOHZ + TimesNewRomanPS-BoldMT. some characters may not display or print correctly ( OR ) cannot extract TimesNewRomanPS-BoldMT"

And clearly they'd rather not.

Secondly there's something about images missing:

In the generated PDF the images are showing as red cross marks, while creating PDF, CFDOCUMENT makes HTTP URL calls to coldfusion server to get the images from the CF virtual folder CFIDE/CFSERVLET (because these images are saved in this folder by cfchart tags based on the charting settings in CF administrator) [...]
It's all greek to me.

Now these should be two different questions on StackOverflow, but... err... shrug.

If there are any ColdFusion/PDF experts out there, can you have a look-see and try to help? Cheers.



Thursday 19 November 2015

... and... Adobe hot fixes it

Just a quick follow-up to this morning's article: "CAUTION: Latest ColdFusion 11 patch breaks (at least some) code using "/>" in CFML tags"

Well knock me over with a feather... they're released a hot-fix for it! How good is that?

It's attached to the ticket: "CFPOP doesn't create the query given by name="" with updater 7 installed".

I still think they need to withdraw update 7, bake that hot fix into it, and release it again (as update 8), but at least this should get Tom going.

I haven't tested it as I'm behind a firewall here and cannot get to a POP server, but I'll report back when I have confirmation it actually fixes the problem.

Good snappy work there Adobe. And good to see it's possible for you to release individual hot fixes.



Guest author: Why good developers write bad code

A while back one of my mates said there was a topic they had some thoughts on, and figured they should write a blog article about it. Their problem being they didn't have a blog. This seemed like a good way for me to get someone else to do my work for me, so I raised my hand to host it for them here. I've been waiting for... god... oh about three months for this. But 'ere' 'tis:

So there's a guy on your team, and he's a good developer. He knows the language inside out, he's up-to-date with all the latest technologies, he spots design patterns and code smells, and when you send him your code to review he cuts you a new asshole (and it turns out he's bloody right about everything)

... but then he gets a chunky piece of work and, more often than you'd expect, it goes astray. He's swearing at his screen, he's "almost done" for ages. Because it's late the code reviews are lighter than they should be, and QA raise a ton of bugs. He says they're all small but each one takes longer than the next to fix. When you try and spread out the bugfixing among the team it takes everyone else even longer so he ends up slogging through everything on his own. Eventually it's done and it works, but it's brittle and everyone is afraid to touch it. He blames it on over-complicated requirements and over-ambitious timelines and we all move on.

Maybe it was The Business's fault for coming up with stupid requirements, or just the fact that smart people can make mistakes in ingenious ways. Everyone gets unlucky now and again, but if a competent developer gets bogged down regularly then there's something else going on. What is it?

Good devs are often tinkerers, and are very good at figuring out ways to work around obstacles. The dark side of this is that obstacles can be seen as things-to-be-worked-around rather than problems to be solved.

At the code level this manifests as a reluctance to refactor. Tests make refactoring safer and easier, but even with good tests developers often see existing code as fixed, something to be built upon rather than restructured. The upshot of this is you get layers and layers of code, each layer adding a missing piece or working around a bad design decision in the one below, and the complexity quickly gets out of hand. This gets especially bad when time is tight, and guys don't want to go back to the start, and so treat their own almost-but-not-quite working code from a few days ago as immutable (especially after their feature has been through QA) - essentially they get into trial-and-error mode, and nothing gets re-thought, so underlying issues with their code design don't get fixed.

You also see it one level up from the code - when confronted with an over-complicated set of requirements, a capable developer will often just implement them as they are, logical contradictions and all. He'll bitch and moan, but will very seldom chase down a bad requirement and get it (or his interpretation of it) fixed so that it makes sense. We all bang on about elegance, but the requirements are the soil the code grows in, and without taking some responsibility for them your code cannot ever be consistently good.

There's no easy way to fix this. It's hard enough to find programmers capable of writing good code, harder still to find ones that consistently do so. If this is you, write tests you can trust then refactor refactor refactor. If for some reason you're not doing TDD then refactor anyway. If you've coded yourself into a corner create a new branch and work out a new approach in that. Be bold. It's just code, it's not set in stone, and neither are your requirements, so stop your moaning and take responsibility for them. And stay out of trial-and-error mode as much as you can.

If this is someone on a team you lead, TDD and code reviews (especially if they're done early enough to prevent a dev going too far down the wrong path) will help, also because bugs are often fixed by trial and error it's a good idea to keep new features *out* of QA for as long as possible. Really though there's no magic bullet - you'll just have to know who's susceptible to this kind of thing, and keep an eye out for it.

Yep, very true. I encounter this challenge all the time, and indeed sometimes I self-identify as having this problem too. I suppose we all do/did at some point.

CAUTION: Latest ColdFusion 11 patch breaks (at least some) code using "/>" in CFML tags

I twittered about this yesterday, but didn't have time to follow it up here for one reason or other.

I think one should hold off on installing ColdFusion 11 update 7 for the time being, as it definitely has a minor code-breaking glitch, but it could be more far-reaching. This still needs to be clarified by the Adobe ColdFusion Team.

Yesterday Tom Chiverton drew my attention to this new issue: CFPOP doesn't create the query given by name="" with updater 7 installed, wherein:

Pre-updater 7, and as documented, CFPOP's name argument can be used to name the query returned.

Post-updater 7, this errors with " Variable MAIL is undefined. <br>The error occurred on line 10. "
Line 10 being the CFDUMP. Trying to use it in a QoQ also fails.

Test Configuration

<cfpop server="xxx"
<cfdump var="#mail#" />

Seems inconvenient, and I think <cfpop> is a common enough tag that it needs an immediate fix. Bear inind that Update 7 is a security update, so all things being equal, ColdFusion 11 users really ought to have it in production as soon as they can.

Adobe then fed-back on the issue (props for feeding back quickly, btw), but the news ain't cool:

The work-around for this issue is to remove the "/" end tag "/" in cfpop tag, it would work.

Modified the code :

<cfpop server="xxx"

<cfdump var="#mail#" />

There is already a bug logged for this bug #3969304(though the description of the bug and the behavior is different. These are the side effects of having end tag), which is fixed and would be available in the next update.

My first reaction here is that "the next update" - unless otherwise stated - could be a coupla months off: the gap between update 6 and update 7 was 2.5 months or so. This is unacceptable. This bug was clearly introduced in update 7 - Adobe are not even contesting that - so they need to fix update 7. We can't be waiting around for 2-3 months until update 8 comes out.

Adobe, you need to own situations wherein you cause your clients grief.

Secondly... and this didn't occur to me initially... if closing the <cfpop /> tag causes it to not work... how many other tags are impacted by this? I don't see how the closed / not closed behaviour change can be individual-tag-specific. This needs clarification from Adobe.

[Actually "secondly" was a sense of epicaricary that people who unnecessarily close their CFML tags, further cluttering up already cluttered code have had their chickens come home to roost. But that is counterproductive in this situation].

Thirdly. This just demonstrates that Adobe need to rethink their security patching, and their approach to patching in general. A security patch should only impact the security issue. It should not be a roll-up of all previous patches, because it's too big a regression burden for the urgency of these security holes they keep finding in the sieve that is ColdFusion. Adobe need to change their work focus away from what's convenient for them, to be what's convenient for their clients. They also need to bear in mind that all of this, irrespective of the patch size and complexity is a) their fault; b) causing their clients work. This is another thing the Adobe ColdFusion Team need to start owning. They need to start acting like they're the stewards of enterprise grade software, basically.

So I think there's some question marks over ColdFusion 11 update 7 at the moment, and the ColdFusion Team need to get a hustle on to release update 8 as soon as they can.

I will be hitting the #CFML Slack channel to try to get this followed up ASAP, so we can get some visibility on all this.



Monday 16 November 2015

CFSummit (Tuesday): better frame of mind

Well I was in a better frame of mind, until BlogSpot just wiped my entire article when I pressed CTRL-Z. And then it saved the blankness. FFS.

Anyhow, I'll be quick here as this is old news now, but I'll go over what I went to see on Tuesday last week anyhow.

Firstly - and I've said this before and I'll say it again - one of the best things about conferences is the between-presentation hallway chatter: there's a lot to be learned there too. Breakfast is no different, and is a good opportunity to be parked next to new people and chat for a bit about what they do and what they know. What I learned on Tuesday breakfast is that Gavin can talk non-stop for about 15min without drawing a breath or pausing to let anyone else participate in the conversation ;-). Still: it gave me a good chance to eat my breakfast whilst only nodding and going "and... [hmmm, OK he's not done yet]... but... [nope, still not my turn apparently]... aah... [hmmmm]...". Bloody hell Gav, you like to talk, don't you? ;-) That said, he was expounding on some good ideas that came out of the panel discussion and Adobe's and the community's general attitude to ColdFusion, so it was worth listening to. I pointed everyone to this before on Twitter, but here it is again: "Building a Bigger Better CFML Community". Read it and act on it.

The first presentation of the day was Roy Fielding talking on "REST and ColdFusion". He pretty much wrote the book on REST, so was worth listening to. Although perhaps also read Adam Tuttle's more accessible version: "REST Assured: A Pragmatic Approach to API Design". The bottom line I took from Roy's presentation is re-affirmation that REST isn't anything tricky, it's just using HTTP to deliver data. Like how web pages already does this. He also made the bold observation that whilst JSON might be the serialisation mechanism du jour, this doesn't mean it's the only way to ship data around the place. Nor will it necessarily be the way things are done in a few years time. Remember how we all used to like XML, but now it's just crap? Well yeah. I await for the other shoe to drop with JSON, TBH.

Nolan was on next with "MVC With and Without a Framework". I decided to go to this one over Kev's on on continuous delivery (etc) because I was intrigued by some of the chatter I was part of - or overheard -  on Monday. There are really a lot of CFMLers who seem blissfully unaware of things like "design patterns" and "frameworks" and stuff like that, and I thought this presentation might be reaching out to them. It's a demographic I've largely ignored on this blog, but perhaps I should be doing some more grass-roots stuff. Nolan always presents well, and I found this interesting even though there was nothing new in it for me. One interesting thing he observed is that in the process of taking a MVCless / frameworkless site and putting it into FW/1... really a lot of the code stays mostly as-is, with just hope it's called changing, and a bit of ceremonial stuff around the edges. I was surprised at the site-by-side diffs he was doing.

I had to skip the next slot as I needed to go Skype my wee son. Although his mum didn't put him on, so that was a waste of an hour. Then it was lunchtime, and more random chatter with random people.

After lunch was Anit taking the throng through what's involved in upgrading previous versions of ColdFusion to CF11. I basically wanted to see if there was anything I didn't know about (no), and also to check out what Anit was like as a presenter. He's definitely the best of the Bangalore-based bods (the BBBs!): speaking very clearly and engagingly, and - if I can be a bit bold - with a less thick accent than most of the other bods from the ColdFusion Team. I know the USAn audience struggles with this at times. And indeed even I could only understand about 10% of what Tridib was saying on the panel discussion the previous day, and I was sitting 2m away from him!

Mr Blinky Hat was up next to go commando (he refused to confirm this one way or the other): "Go Commando, with CommandBox! CLI + REPL + Package Manager for CFML". CommandBox continues to impress, I gotta say.

Lastly I went to Charlie's presentation on "10 Common CF Server Challenges and How to Find/Solve Them". This was typically thorough: Charlie knows his stuff. But I do have to say that by this time I was conferenced-out, and wanted to get away from the concept of ColdFusion.

Elishia wrapped things up in the main room, and we were done.

I never understood this. Organising developers to go have a drink is like herding cats using nothing but a bucket of water, so it seemed to take about an hour to get from the conference to the bar 200m, but that's what I did (once again it was the Todd English PUB). Having learned my lesson about excesses the previous day, I kept things "under control" this evening. I was in bed by 3am (after diversions to one of the bars in the casino itself, and Tim Cunningham's suite for even more beer, after the point at which I should have known better). Dear god. And on Wednesday I had to fly back to the UK...

Anyway, it was a good gig, and I'm glad I made it. Mostly for the people I hung out with, I have to say, more than the presentations. This is not an indictment of the presentation quality at all, but as time goes on, ColdFusion stuff is less and less relevant to me. But I had a good time. Hey I can't remember all the people I caught up with... I'm both shocking with names/faces, plus when it happens at the bar, my brain doesn't record events anyhow. It was excellent hanging out with y'all anyhow.

And that's all the lunchtime I have available for writing, so that's that.



Wednesday 11 November 2015

CFSummit: Yeah yeah yeah, all right Rakshith...

So yesterday I said this:

To which Rakshith said this:

Them be fighting words, so we had a bet, and... having checked his presentation... I lost the bet.

So here's my penalty: you're right, I'm wrong. Sorry mate.

Now shouldn't you be here in the PUB having a pint???



Tuesday 10 November 2015

CFSummit (Monday): I never learn

Oh for the love of god: what did I do to myself? As I touched on yesterday - "CFSummit (Sunday): conference tips from a veteran" -  I had a bit of a late "night" on Sunday, so Monday morning was a bit "emotionally fraught" (read: I had a hellish hangover).

So I missed the keynote completely, as I was still trying to remember how to dress myself, and locating coffee (fortunately I managed to work out the first on that list before heading out for the second). I have subsequently learned that I was "announced" (whatever that means) at this keynote... so I'm in a bit of trouble for not being here. Yikes.

Once I actually arrived, Rakshith was giving us the once over ColdFusion 12, which was much the same as the presentation given at CFCamp. I've already written up what I am able to about that, in other articles:

I was hoping for some new code-centric stuff, but there wasn't really.

I had a chuckle when I thought I saw a reference to "Cold Fusion" on one of his slides, but he assured me I was seeing things (or "I was drunk", as he put it ;-).

Next up was Kevin Schmidt going through doing REST with CFML ("Building APIs with ColdFusion") - this was the code-centric side of things: how to configure one's CFCs etc. It was a good presentation and Kevin is an engaging speaker, but I was really let down to see all the code being written in tags. I know I keep harping on about this, but the community (and Adobe!) need to lead the way here in downplaying CFML being about tags. Tags have their place, but not in heavy lifting code. We need to start directing eyes away from tags for everything.

I was feeling dire so I went back to my room to try to smother myself with a pillow. And kipped for a coupla hours. I'm annoyed I missed Elishia's ColdFusion 12 API Manager presentation. And lunch. But so be it.

At 2pm I decided to check out Dan Fredericks' "CFML Features for More Modern Coding" presentation. I loved this... it was a true, sleeves-rolled-up code-centric presentation, and I think Dan's picked a topic that represents the cutting edge of complexity in CFML: closures and member functions (well: member functions aren't complex, but it's a mind shift fro CFMLers maybe). Hopefully his slide deck goes public at some point, and I'll try to link to it. I hope people enjoyed it, and got something out of it.

I still felt pretty sick.

3:30pm rolled around, and I had to get up on stage. And smile and stuff. It was the "Adobe Panel Discussion Q&A Session". This was interesting, and - again - I hope the audience took something away from it - but I think I was out of place being on the panel. The questions were mostly directed at Adobe, and I could not really participate too much in answering them. I think the panel should have just been Adobe people. I was there to field questions on code and dev practices and that sort of thing, but this is not the direction the Q&A took. It was more one of these marketing-speak things that I have indicated disillusionment with in the past: I don't want to be marketed at at a dev conference.

There were a number of things I wanted to talk about, but wasn't given the opportunity to. Oh well.

I think next time there should be maybe a dual (or do I mean "duel"?) panel thing: Adobe vs Community, or something. Or just get Adobe bods up there representing business, marketing, and tech, and let us community members ask them questions. Dunno. And it would have been better if I was in the audience, not on the panel, I think. Still: it was fun, and I really appreciate Elishia & the rest of the Adobe people giving me this opportunity. It must have seemed like a bit of a risk getting someone like me up on stage. I think I acquitted myself adequately, and I only accidentally swore once. That I can recall.

After that I surrendered, eschewed the chance for free beer and talking to people, and went back to bed. At 5pm. I woke up briefly at 2:30am and considered going downstairs, but my next thought occurred at 5am when I awoke properly.

So it was a bit of a waste of a day for me. There were a bunch of people I really wanted to catch up with and chat with, especially to see if we could get a conversation going around Dan's talk, and possibly try to corner Rakshith, hand him a beer, and have a chat.

The best thing was that I did get to meet Anit! I hardly spoke to him, but at least I've pressed the flesh with Adobe's resident CFML super hero. And I got a photo with him (it's been posted on the Slack channel, but I don't have it myself)! Cool. I also met Aaron Neff, who is one of ColdFusion's other superheros... he's by far the best CF tester Adobe have on tap. But his name is not one most of us will be too familiar with.

I reckon from what I saw and heard from other people, it was a really successful first day, and I hope today is just as good. And hopefully I manage to last beyond late afternoon, today. :-S



Monday 9 November 2015

CFSummit (Sunday): conference tips from a veteran

Yes. Oh yes. The day before one is supposed to be actually participating in a conference... make sure to be out drinking until 05:18. This will ensure clean sailing the "next" day, and everything will go swimmingly.

Or so I'm telling myself.

Judge for yourself tomorrow afternoon. On the bright side: everyone's expecting me to be angry...


Saturday 7 November 2015

CFSummit (Saturday): if I ever get there

This is going to be content-free, for the most part. Be warned. Indeed currently I don't have the content for this planned, but I'm stuck at Houston airport with beer in hand and... time to kill.

OK, so I started my trek to Las Vegas for CFSummit this morning, with the first hitch being the coffee stand at the Tube station was shut, so I have now endured an entire day without an espresso. This seems... wrong. It was probably a portent, and I did not notice. Dammit.

When choosing my flights I decided to pick one flying a 787, to see if they are any good (I'd not flown on one before). I can't say I was that impressed: it didn't even seem new (either in age or in design), so it was a bit disappointing. I did like the new window blind concept they have: it's not a blind, it's an... electrically-sensitive-gel-based thing (apparently: "How The Magical Windows in Boeing's 787 Dreamliner Work"), which has a button which makes the window get gradually darker. This is cool except for the fact they can be controller centrally too, and the cabin crew decided we were going to be in night-time mode for most of the flight (which was entirely actually daytime). So I missed my favourite part of flying over unfamiliar countries: looking down at what it looks like. All I could see was a dull blue colour. To put in perspective of how much light the gel will block when on "full strength", one can look directly at the sun through it... although it's still very bloody bright and not something to do for a sustained period of time. That aside... I remain very impressed with A380s as far as "modern aircraft" go, but don't see the fuss about 787s. And, yes, I know I am "comparing" different classes of aircraft there.

En flight I had a Goose Island IPA (from a can, no less), and that was bloody good. Actually I had three of those. Then they ran out of those and they gave me a Miller Lite instead. That was awful.

I watched Terminator: Genisys, which I thought was pretty good (going against general opinion, I think), indeed I'd put it second best in the series after the original. I rank those films: Terminator, T:G, T3, T:S / T2. I know I'm an outlier when it comes to ranking T2 so low, but I don't really think it brought anything new to the story: it was a rehash of the original with a bigger budget.

I also watched something called Air, which was "interesting", but... didn't quite get there. it was all right.

Oh and I watched the first three episodes of "Humans", which was really good.

One disappointing thing about this 787 was that whilst it had power outlets on economy seats (two outlets per three seats), it wasn't switched on. But I didn't need my laptop anyhow, so no problem really.

Getting into George Bush International Airport (after 10h in the air), I girded my loins for "The Immigration Experience". True to form, whilst having a cavernous immigration hall, with 40 desks to process people, the US Border Force decided the 1000-odd people in the queue would be best processed via never any more than a dozen desks being opened. It took around an hour to get to the front of the queue-feed-queue to get into an actual queue in front of a desk. And another 20min after that to get through. This was the at Philadelphia for CF.Objective() a coupla years ago. Entering USA is by far the worst border experience I have encountered (and I've been to Russia...).

At this point I had 10min to find my bag, re-check it, and board my LAS flight. Which was pretty much stymied by the queue for customs after picking up my bag. I'd missed my connection by then (my brain was actually going "hooray! I have enough time for a beer now!"), so rebooked on the next flight - no charge, and no batting of eyelids... this is just "normal" for USA domestic air travel apparently.

After that I had another 20min in the security queue, operated by drittereichian TSA staff, who would - I imagine - be gleeful if their jobs involved processing people straight from rail cars, if they had the chance.

Oh... just to wind back a queue or two: here's a question for my USAn readers... why does a desk clerk at an immigration desk need a gun? What's wrong with you lot?  The only way I can see a gun being necessary in that environment is in a "They Shoot Horses, Don't They?" kind of way, if the people in the queue give up the will to live before being processed.


Anyway, I missed my connection as I said, but I'm on the next flight to LV, which doesn't leave for a coupla hours (well: an hour or so now...), so I've parked-up at "Rocket and Rye" (they seem to have no website), who have good beer, not bad food, and a coupla people to talk to.

I recommend the Weekend Warrior Pale Ale, Ryed Hard, and the chicken quesadilla, btw.

By the time I get to LV this evening (touch down @ 11pm, apparently), I doubt I'll be in any sort of state or mood to say "g'day" to people, but let's see...


Finally at the hotel, and at 11:58pm... it's bed for me. I'm too old to be starting to go out at midnight.

Up in three hours to Skype with my boy, back in Ireland.


Friday 6 November 2015

PHP: getting my dev environment running on Nginx instead of Apache

We're shifting from Apache to Nginx at work, and despite having other bods to look after that side of things - I'm just a developer, I avoid devops if I possibly can - I still need to write rewrites and the like, and so I need to be passably au fait with Nginx and how its rewrite system works.

So I decided to get a local site up and running via Nginx this morning. Well: I say "I decided to"... I needed to ;-)

As with anything I write about PHP and that sort of thing... I profess no expertise in how I throw things together, so I would not take these as model instructions or anything like that. I'm just documenting what I did.

Note: I'm running Windows.

Download and run

First up I downloaded Nginx from their site. It requires no installation: just unzip and run. Obviously however I need to configure the thing if I'm to do anything more than stare at gdayWorld.html on the screen.


Googling a bit, Nginx uses PHP's FastCGI module to run PHP, and one needs to configure NGinx to pass requests to PHP that way. This is all documented here: "PHP-FastCGI on Windows".

Basically I need to run php-cgi.exe, in the background, telling it to listen on a port:

C:\apps\php\5\5\php-cgi.exe -b

(I'm running PHP 5.5 cos that's what we use on our live site)

Note - and this is one thing that caught me out cos the docs weren't 100% explicit (and I didn't engage my brain) - that port is not the port that Nginx listens to for incoming HTTP requests, it's just a port for Nginx and PHP to communicate on. And - obviously for those who do engage their brains - it does actually need to be a different port. Sigh... that stupidity held me up for about 15min of head-scratching until I started to pay attention to what I was doing.

php-cgi.exe just sits there running.

Nginx config

Here's my Nginx config file. Basically I took out all the commented-out clutter, changed some stuff to suit my environment, and put some extra bits in (as suggested by that page I link to above):

Wednesday 4 November 2015

CFSummit: they've got who on their discussion panel?

Ha. So this article just went up on the official ColdFusion blog: "Adobe ColdFusion Summit 2015 - Panel Participants Announced". The participant list is:

From Adobe:

Tridib Roy Chowdhury, Sr. Director of Products

Elishia Dvorak, ColdFusion Solution Consultant & Evangelist (moderating)

From the community:

Ray Uyemura – Director of Systems, City of Los Angeles

Leon O’Daniel – Sr. Web Application Developer and Leader of the Boeing ColdFusion Community of Practice, Boeing

Shirak Avakian – Sr. Web Architect, Century National Insurance Company

Adam Cameron – Sr. Software Engineer, Web Reservations International Group

Who's that frickin' turkey at the bottom there?

So... erm... that should be interesting.



ColdFusion 2016: arrays now passed by reference

There's not much to say about this one. ColdFusion is finally playing catch-up, and standardising its approach to passing complex objects.

Traditionally, ColdFusion has passed "simple values" (strings, numerics, dates, etc) by value, and complex objects (structs, queries, XML) by reference (provided one understands "pass by reference" with these caveats: "Complex data-types in CF, and how they're not copied by reference"). The one exception was arrays. Arrays were passed by value. Why? I don't bloody know.

Right from the outset Railo decided that was bloody daft and has never done this, and this has been inherited by Lucee.

Now ColdFusion has caught up, even if I think the specific implementation is lacking.

There is a new setting for Application.cfc, this.passArrayByReference, which one can set to true (the default is false). If one sets that, then the behavour of passed arrays is changed.

Let's have a look at some code running on ColdFusion 11:

function arrayToUpperCase(array){
    for (var i=1; i <= array.len(); i++){
        array[i] = array[i].ucase();
    return array;

rainbow    = ["Whero","Karaka","Kowhai","Kakariki","Kikorangi","Poropango","Papura"];

rainbowInUpperCase = arrayToUpperCase(rainbow);

writeDump(var=rainbow, label="rainbow", format="text");
writeDump(var=rainbowInUpperCase, label="rainbowInUpperCase", format="text");

This simple code has a function which takes an array, upper-cases each element and returns it. Afterwards, we dump both the original and returned arrays:

rainbow - array

1) Whero 
2) Karaka 
3) Kowhai 
4) Kakariki 
5) Kikorangi 
6) Poropango 
7) Papura 
rainbowInUpperCase - array


As you can see, when we modify the passed-in array, it does not impact the original array. Constrast this with the same operation with a struct:

function structToUpperCase(struct){
    for (var key in struct){
        struct[key] = struct[key].ucase();
    return struct;

rainbow    = {red="Whero", orange="Karaka", yellow="Kowhai", green="Kakariki", blue="Kikorangi", indigo="Poropango", purple="Papura"};

rainbowInUpperCase = structToUpperCase(rainbow);

writeDump(var=rainbow, label="rainbow", format="text");
writeDump(var=rainbowInUpperCase, label="rainbowInUpperCase", format="text");

rainbow - struct

rainbowInUpperCase - struct


As you can see, because structs are passed by references, the argument in the function references the same struct as in the calling code, so changes to the argument are reflected in the original.

Tuesday 3 November 2015

ColdFusion 2016: the long-awaited CLI

Another thing Adobe showed at CFCamp - and I have had clearance to write about - is that ColdFusion finally has a CLI. About bloody time. It's about... well... 15yrs overdue. Well to be fair to Allaire, ColdFusion 5 could run files from the command-line, but this functionality was omitted from ColdFusion between then and now.

Having a command-line is dead useful, because it means one doesn't need to have a web server running to run CFML code. Indeed one doesn't even have to have a ColdFusion server running to run CFML code! This is handy for scripting, or - for me - testing quick code snippets.

Here it is in action:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

D:\src\cfml\scopes\server>CF server_dump.cfm

        InstallKit: Native Windows
        appserver: JRun4
        expiration: {ts '2015-11-15 08:18:28'}
        productlevel: Evaluation
        productname: ColdFusion Server
        productversion: 11,0,01,296113
        rootdir: C:\apps\adobe\ColdFusion\2016\express\cfusion
        supportedlocales: [snip]
        updatelevel: 01
        additionalinformation: [empty string]
        arch: amd64
        buildnumber: [empty string]
        name: Windows 7
        version: 6.1


Note one good thing here right off the bat: <cfdump> detects it's being run via the CLI and doesn't piss about with all it's mark-up, it just does a text-only dump.

Note also one other interesting thing: JRun. Huh? I thought we were done with that? Anyway, let's not worry about that too much. Although I will raise it with Adobe.

Command line arguments

The ColdFusion CLI has two types of command line arguments. Firstly there are the familiar space-separated ones:

D:\src\CF2016\cli\args>cf ordered.cfm tahi rua toru wha
All arguments - array

1) tahi
2) rua
3) toru
4) wha

First argument: tahi

And also named ones:

D:\src\CF2016\cli\args>cf named.cfm one=tahi two=rua three=toru four=wha
All arguments - struct

four: wha
one: tahi
three: toru
two: rua

'two' argument: rua

The code for each of these examples demonstrates the new CLI functionality regaridng input and output:

// ordered.cfm
writeDump(var=CLI.getArgs(), label="All arguments");
CLI.writeLn("First argument: " & CLI.getArg(1));

// named.cfm
writeDump(var=CLI.getNamedArgs(), label="All arguments");
CLI.writeLn("'two' argument: " & CLI.getNamedArg("two"));

So ColdFusion has a new CLI object exposed when running on the CLI. Note that this is not a scope, it is an object. It has methods (as demonstrated above):

  • getArgs() - returns all space-separated arguments
  • getArg(n) - returns the nth argument
  • getNamedArgs() - returns all the name/value pairs (separated by =)
  • getNamedArg(name) - returns the value of the named argument
  • writeLn() - writes a string to standard out, followed by a new line
  • write() - as above, but without the new line

There's a few other methods I've not looked at yet, as demonstrated in this dump:

D:\src\CF2016\cli\cliObjectTests>cf dumpCli.cfm
object of coldfusion.runtime.CLIBridge

Class Name: coldfusion.runtime.CLIBridge
        addArg(java.lang.String) returns void
        clone() returns java.lang.Object
        getArg(int) returns java.lang.Object
        getArgs() returns coldfusion.runtime.Array
        getNamedArg(java.lang.String) returns java.lang.Object
        getNamedArgs() returns coldfusion.runtime.Struct
        putNamedArg(java.lang.String, java.lang.Object) returns void
        read() returns java.lang.Object
        write(java.lang.Object) returns void
        writeError(java.lang.String) returns void
        writeln(java.lang.Object) returns void

Monday 2 November 2015

ColdFusion 2016: ordered/sorted structs

One interesting new feature of ColdFusion 2016 is the ability to affect key ordering in structs, in two different ways. (This, btw, is something that has been exposed in the public domain, and I have made sure I am not breaking any NDA in writing this up).

Traditional structs do not have a sense of ordering in their keys. This is by design: a struct is an unordered collection. When iterating over a struct by key, obviously the keys will be iterated in a specific order, but there is no guarantee what that ordering will be as the specification does not... well... specify one, so it's down to the implementation. ColdFusion might expose the keys in alphabetical order; Lucee might do it in order of creation; some other CFML implementation might do it some other way. If one wants to control the order of how keys are accessed, one needs to extract the keys and sort them, then iterate over that. EG:

categoryTallies = {
    "gold" = 2,
    "premium" = 5309,
    "1st" = 37,
    "second" = 68113,
    "general" = 491

    writeOutput("Category: #category#; tally: #tally#<br>");


sorted = categoryTallies.sort(function(category1, category2){
    return sgn(category1 - category2); 
    writeOutput("Category: #category#; tally: #categoryTallies[category]#<br>");

Looking at the first part of this output in ColdFusion gives this:

Category: gold; tally: 2
Category: 1st; tally: 37
Category: premium; tally: 5309
Category: second; tally: 68113
Category: general; tally: 491

And pleasingly Lucee returns a different order, demonstrating my point:

Category: second; tally: 68113
Category: premium; tally: 5309
Category: general; tally: 491
Category: gold; tally: 2
Category: 1st; tally: 37

So ColdFusion's struct implementation returns the keys in the order they were added; Lucee... dunno. I dunno what order that it. But that's fine, I don't need to know because it's not relevant.

Looking further into the code, I get myself an array of keys sorted by tally, and then use those as the basis for an iteration outputing the same info, but now sorted:

Category: gold; tally: 2
Category: 1st; tally: 37
Category: general; tally: 491
Category: premium; tally: 5309
Category: second; tally: 68113

BTW, I was initially confused as to why that code only worked on ColdFusion, and not Lucee. I had not realised that structSort() (and therefore Struct.sort()) doesn't work the way its array equivalent does, and doesn't take a callback for the comparator. That's a bit rubbish that the implementation wasn't uniform across all the collection iteration methods. However it seems that in ColdFusion 2016, it's now been implemented. I guess this is part of this new struct implementation.

Sunday 1 November 2015


Blimey... the CFML Slack channel now has 1000 members! That's not bad going for an initiative only started a few months ago.

I doubt anyone from the CFML community reading this blog hasn't already signed up, but if not: join in:

Or use the form here.