Wednesday 25 July 2012

The holy grail

No, this ain't about Grails, sorry. I dunno anything about Grails, so anything I write about it would be rubbish, anyhow.

What it is is a follow-up to a conversation (if that's the right word) on Twitter last night regarding ColdFusion and backwards compat between releases.

Adobe treats the notion of backwards compatibility in ColdFusion as a "Holy Grail": it's often cited as one of the most important considerations when undertaking changes/enhancements to the language.

Whilst I respect "maintaining backwards compat" as a goal, I do not respect the lengths Adobe go to sometimes to preserve backwards compat at the expense of... well...  doing work. I think this has lead to some inertia in the language, and has contributed to ColdFusion lagging behind both in comparison to its competitors (Grails etc), and even now its imitators (Railo & OpenBD). ColdFusion might be a RAD language from a development perspective, but it's a creaky old behemoth when looking at its release cycle and recent language innovations. Take note I am specifically referring to language innovations here, not the various plug-ins and wrappers for third-party efforts such as Hibernate and Ehcache integration.

When I say "language innovations", I mean stuff at a level I cannot integrate myself. Closures in CF10 would be an example here. Or Application.cfc from CFMX7, or CFCs in general from CFMX6.0. Continuing to improve CFScript coverage would be an example if this, too.

The reason for this is twofold.
  1. I can integrate third-party stuff myself.  It might not be as seamless as how Adobe would do it for me, but I can still do it.  On the other hand I can't really implement a fix to isValid(), or augment <cfloop> to support a GROUP attribute (thankfully - at last - added to the language in CF10).
  2. I use the base language constructs all day every day.  So improvements to the language are a more significant win to me than some new feature which I got by without having for all the previous versions of CF, so arguably I can still get by with it now (yes, I know, that logic is a bit facile because the same could be said of general language enhancements).
I've upgraded reasonable-sized applications from CF4.5 through to CF9 (not in one hit!), as well as cross-graded apps from CF8 to Railo and OpenBD.  I generally expect there to be backwards compat issues with my code, and assign time to reworking and retesting everything.  When I go to do an upgrade, I read the release notes for any deprecations (which would be a note to self to have any usage of deprecated code to be refactored before the next release), and obsolescences, as well as scanning what new functionality has been added which might conflict with my own code.  An example of this is when imageInfo() was added to CFML in CF8, we had to rework our code to use that, and remove our UDF that did the same thing.  More recently we had to rewrite a chunk of our application to work around a bug Adobe introduced (introduced by design, too, I hasten to add) into <cfabort>.  The holy grail of backwards compat wasn't a consideration there, it seems. And there's always stuff like this to deal with.  It's not a problem.

As well as the documented changes, there's always outlying cases wherein behaviour changes in an unexpected way because our code is doing unexpected things.  Adobe removed getBuiltInScopes() from getPageContext() which broke some of my code.  This is fair enough: it was specifically documented as not being supported, so it was on my head that I chose to use it.  That said, it was doing no harm, so they coulda just left it in!  Similarly with some of the behaviour of CF's Java classes for representing things like queries... I use some of those internal methods to work around shortfalls in CF's recordset support.  But I know they're unsupported, so I know I will need to retest all that stuff when I do an upgrade.

Doing rework for backwards compat issues is just part of the upgrade process.

Also let's think about why we're even upgrading in the first place.  When CFMX6.0 was coming out, I was in the fortunate position of being invited onto the prerelease, so had early access to the code.  We had our entire application converted from procedural (or more like just "spaghetti") CF5 code to semi-OO CFMX6.0 code even before CFMX6.0 was released (we got dispensation from Macromedia to do this).  Why?  Because CFCs rocked.  We were eagerly looking forward to upgrading our code, and doing a bunch of rework.  And this is the thing... people don't upgrade just to line Adobe's pockets, they upgrade to get the new features.  And not just to get them... to use them.  And this means... writing code.

So pay attention here, Adobe: we all expect there to be rework when a new version of ColdFusion comes out. We invite it.  We look forward to it.

Obviously playing with new features is more exciting that working around backwards compat issues, but equally if you happened to fix a bug that means I can remove my work-around code, I welcome this.  If you fix a bug that I was accidentally leveraging a side-effect of: I welcome it!  I want to know when my code is sub-optimal.

I am also pleased when you - Adobe - take responsibility for your cock-ups and fix them.  Everyone makes cock-ups: no-one has a problem with this.  But to not fix one due to some specious explanation that it would break backwards compat puts you in bad light, in my view. To be frank, I read reactions like that as "we can explain it away like this, and we can avoid that work".  I might be wrong, but it's the impression you give me.  And it's the impression that matters, not the validity of it.

As per the discussion doing the rounds at the moment, I agree CFML needs a bit of an overhaul to return it to being competitive.  And doing some of this might incur "backwards compatibility concerns".  Things like:
  • add member functions to all CF data types to perform what are currently function-based operations.  EG:
    • myArray.append(element) instead of arrayAppend(myArray, element)
    • myStruct.exists("someKey") instead of structKeyExists(myStruct, "someKey)
    • myList.append("new list element") instead of listAppend(myList, ("new list element") (and, yes, I know lists are just strings, before anyone reminds me ;-)
    • deprecate the current functions.
  • image~, spreadsheet~ [etc] functions, eg: imageNew(), spreadsheetWrite():
    • deprecate all of them
    • reimplement as object.method()
Please note I say deprecate here, not remove.  So, in theory, there are no backwards compat considerations here.

So, anyway, I want to get a feel for what other people think about the importance of backwards compatibility, and what kind of barrier to innovation it should be.

I'm going to chuck together a quick survey, and it would be great if you could fill it out, and also if you could "get it out there" to other people in the CF industry who aren't aware of this blog (which is most people, I know!).

The survey is here.

Speaking of survey's, I'm still running this one, gathering people's reactions to some bugs in CF, and how Adobe are dealing with them.  I've had some good responses, but not as many as I hoped for (and not enough to be a useful sample, to be honest).  If you have time, and you've not done so already, please consider giving me some feedback on that one too.  Cheers.

Time for work...