I could get used to this. Another one of my readers has provided my copy for today, in response to my earlier guest-authored article "Why good developers write bad code". This originally was a comment on that article (well: it still is) but it stands on its own merit, and Brian's cleared me to reproduce it as an article. Brian's a mate, one of my colleagues at Hostelworld.com. The quote below is unabridged. I have corrected one typo, and the links are my own.
Brian says:
Really thought provoking post, Adam, and my immediate impression is that your mate seems to be describing an awful lot of dysfunctionality here. I'll get my declaration of interest out in the open straightaway and own up to being a massive Agile/XP fanboy, and consequently I think much of what your mate describes could be cured by the adoption of Agile/XP practices.
The "...but then he gets a chunky piece of work and, more often than you'd expect, it goes astray" quote is very revealing, because it would appear that there's no effort made to break large requirements down into smaller ones and we're on a slippery slope from here on in.
Developers often receive very coarse grained requirements and irrespective of our natural coding ability, we're not naturally adept at managing large pieces of work. That's not to say we can't manage large requirements, but without training, mentoring and support developers who might be first rate coders, can often struggle. In this case, it sounds like the developer in question is trying to eat an elephant in one sitting, and then being left alone to go dark and "fail". Reading between the lines it seems that this is accompanied by a seat of the pants estimate, which becomes a commitment as soon as he's uttered it, and before long we're into death march territory.
Also I wouldn't want to blame the developer in question, because as your mate points out, the same thing keeps happening time and again but there's no intervention or help from the guy's colleagues or his manager. Clearly they're at a loss as to how to fix the problem but unless someone can show him a better way, how is he going to improve?...
Anyway... if I was working with him I'd strongly encourage him to forget about eating the whole elephant, and just start by trying to nibble its trunk. Use TDD religiously from the outset, stick to the red-green-refactor cycle and You Ain't Gonna Need It (even more religiously), avoid Big Design Up Front, and let the application architecture evolve as he uncovers smells in the code that he's writing.
See how far he gets nibbling the elephant's trunk in one day, and if all goes then we'll reconvene and decide on the next bit of the elephant to consume, and carry on in that fashion. If he takes longer than a day - and we'll know that because we have daily stand-ups right? - then we'll pair up and see what the problem is. Invariably if the guy's a first rate coder, it will only be because he's bitten off more than he can chew, so get him to stop, re-plan the work, and decrease the size of the immediate task in front of him so he can actually finish it. In short, KISS, keep tasks small and try to conquer complexity. Writing code is intrinsically hard, we should actively avoid anything that makes it harder.
I think it's also worth bearing in mind that many, if not most developers, are naturally quite reluctant to provide honest progress reports if things aren't going well. Combine that with our tendency to give hopelessly optimistic estimates and we're already in a very bad place. I've found that it's almost endemic in our industry that developers are scared of giving bad news to anyone, let alone their line manager or project manager. Developers are usually clever people who spent much of their academic careers being praised for their intellectual prowess, and to be put in a position where they are suddenly the bad guys because they didn't hit an arbitrary deadline can be very distressing and something to avoid at all costs.
If there was just one thing I would wish for in the industry it would be to reform the attitude that project managers, team leads and developers have to estimation. Estimates should be treated by all stakeholders as just that, they are not cast iron commitments written in blood. They should be based on evidence (dude we go through six stages of peer review on every task, why did you give an estimate of 10 minutes?) and when the evidence is demonstrating that our estimates are wrong, then remaining estimates should be re-calibrated. All this is only possible if managers and developers are prepared to create a culture where realistic and responsible estimation, combined with predictable delivery is championed over masochistic estimation and constant failures to hit deadlines.
One final point, when your mate writes "it's a good idea to keep new features *out* of QA for as long as possible", I totally disagree! :-) I'd want feedback from the QA team as soon as possible. The shorter the feedback cycle the better and in an ideal world the QA team would be writing automated acceptance tests in parallel with my development efforts. I'm a firm believer in the agile principle that "Working software is the primary measure of progress" and feedback from QA I'm just guessing that the software I'm writing is actually working.
One thing I will chime in with late here - I did not notice it in the original article - Brian is dead right in his last paragraph. QA should be involved in the sprint delivery team, and we even include their work in our estimates. We have parent Jira ticket describing the work, sub tasks for the components of the work, including QA. Acceptance criteria go on the parent ticket - which represents something of business value we will deliver - and estimates go on the subtasks. I think keeping work away from QA possibly contributes to causing the situation discussed in the initial article, because it allows the dev(s) to engage in their own little "waterfall of scope creep".
I've seen this in practice. Indeed I've engaged in this in practice. In the past I've been permitted to work outside our Agile / sprint team process for a "special project" (so special we wrong-headedly decided to abandon our best practices!), and - not having well defined acceptance criteria, deadline, or oversight, I did a lot of good work... but a lot of good work that was well off in the wrong direction. This is like a coupla weeks' work. We kept a lot of it once I re-tasked it slightly (another coupla days), but a bunch of it was not really what we actually needed, or - because there had been no code review - unnecessarily convoluted and whilst it made sense to me, the eventual code reviewers were like "what the hell, Cameron?". Not my finest hour. However I equally blame my manager and the lead dev for deciding to work that way (I did initially push back on this, but caved fairly quickly, because I liked the idea of being left to my own devices. I'm wiser than that now).
Similarly I've seen other colleagues take on large tickets which should have been broken down, got mired in a development for a number of days, only to find out they had got the wrong end of the stick (not reading the requirements, and thinking they knew better: they did not). This was only partly (the junior ~) developer's fault. It was also the fault - if we're playing the blame game - of the team's planning, estimation and, to be blunt, which dev was given the work to do.
However in both cases (and another case I won't go into involving my own development work: similar story), we resolved the issue by being tighter with our process, and being more collaborative in the work. This goes back to one of Brian's earlier paragraphs about mentoring. Adherence to practices needs to be championed by someone on the team (Estragon, is it you on your team? Could it be?), that person needs to get buy-in from the stakeholders, their team leadership, and the people fulfilling the business analysis / project management / scrum master role (we roll all three of these into one at HW, for better or for worse: mostly better I think), and then the champion needs to mentor the other participants in the team. The early adopters or supporters from there can help mentor those slower to turn around. It's a collaborative, human-interaction exercise. Something sometimes we need to work on as a team.
I can speak from my experience at Hostelworld.com (yeah, we're owned by Hostelworld.com now, not HostelBookers.com), the successes we have had are from when we stuck to our agile practices; the failures (or "not wins", sometimes) have all come when we have sidelined or co-opted them.
I eagerly anticipate Estragon's response / thoughts on this.
Now... who's gonna write my next article for me?
Righto.
--
Adam