Tuesday, 26 January 2016

ColdFusion: a list of tickets I'm awaiting an update on

G'day:
This is just gonna be a list of ColdFusion tickets I've raised, which are currently marked "to fix" but have no indication of in which release they're gonna be fixed. As the ColdFusion 2016 programme has been under way for about six months now, it's a bit odd that Adobe haven't been more firm about what's going on with them. I'm gonna compile a list here, and put it on the ColdFusion Team's radar.

It's gonna be pretty dry reading. It's gonna be literally a list of links. You should probably just close the browser window now. Unless you're from the Adobe ColdFusion Team, I mean. IN which case you should open every single one of them and follow it up, pls.

Hey... I did tell you it was just gonna be a list of links.

Righto.

--
Adam

Sunday, 24 January 2016

Lucee 5: Platform modularity in CFML

G'day:
I had a long (and often protracted) conversation with Denny - who seems to do some work on the Lucee project, but I really don't quite get where he fits into things - and later in the piece Geoff, who we all know as Daemon's representative on the LAS "board", currently holding the role of President. The topic was "WTF is OSGi, what does it mean to Lucee, why the hell is causing such delays in Lucee 5's release and why do I - as a CFML developer - care?". It wasn't a formal topic, but it just happened to be what we were yapping about.

It was a very frustrating conversation at times because Denny - nice bloke that he is - doesn't seem to be able to empathise with other people: everything he says is always positioned from his own perspective, and he struggles to "get" that other people have different perspectives which are not the same as his. So the conversation always crept back to how it would benefit him, or someone identical to him. Irrelevant.

Geoff came along later in the piece and made a lot more sense.

Sean came in at the very end and had his own commentary to make, most of which was over night for my perspective so I've just caught up with it now.

The challenge here is it's easy to "get" what OSGi is all about... and as one researches I think even more questions about it - especially in the context of Lucee - arise, but... yeah... bottom line it compartmentalises packages so they can have different dependencies, which has been a bit of a blight in the Java world in the past. OSGi is one of the ways to work around that. I am taking a very high level position on this cos I ain't a Java dev, don't want to be one, and dependency management is all a bit yawn-inducing. It's an important consideration, sure, but not terribly interesting. But we all get it, yeah? Say Lucee's using one module that requires SomeDependency v1.0.0; for my code, I need to use SomeDependency v1.2.0 because it introduces a new feature I want to leverage. This is currently a bit of a challenge given the way the JVM loads its libraries: one can have version v1.0.0 of SomeDependency, or v1.2.0, but not both. And what if something in 1.2.0 happens to break something Lucee needs? Splat. There are ways around all this, but they're not streamlined. Oh... and the way OSGi manages modules means they can be (un)loaded on the fly. Yep... lovely... impacts almost no-one in the course of their day-to-day work.

Lucee 5 is being rewritten to follow / implement the OSGi specification, which will solve all this. Lovely. It's not - on the face of it - hugely exciting for most CFML developers. Most CFML developers don't use Java modules, and even ones like me who do occasionally... this version clash situation has simply never been much of a problem. Obviously it'll be more of a problem for people who do this stuff more often, but that is not most Lucee users. So I've always been asking "why do I care about this in the context of Lucee? Who's the target audience? How big is the target audience? Is it really worth all these bloody delays with Lucee 5 to accommodate a very niche audience?" Up until last night I never got a satisfactory explanation, and it really just seemed like something Micha decided to do because Micha felt like doing it. Or maybe an self-indulgence so that it can be claimed Lucee is OSGi compliant, even if it's mostly an unhelpful exercise to almost all users of Lucee. None of the messaging from LAS justified the effort to me. They don't need to justify themselves to me, sure, but I'm part of the community, and I'm sure I'm not the only person wondering.

And then Geoff explained it: modularisation of the platform.

You know how for years we've been asking Adobe to modularise ColdFusion? Obviously the same applies to Lucee, except in a smaller way. ColdFusion's got a huge footprint, most of which one doesn't need most of the time, so why should one need to install it? Lucee's footprint is much smaller, but the same still applies.

That's interesting-ish in itself, but it has a very compelling knock-on effect. If everything is a module, then enhancements and bugfixes become a lot easier to deal with. One can mess around within the scope of the module's codebase, and provided one doesn't do anything daft, it won't impact the outside world (it needs to maintain its touchpoints to the outside world, obviously: its API). This means the testing and footprint for work on a given module is much smaller. And it makes the job just... easier. And whre things are easier, things progress faster, and more smoothly.

It will also mean that organisations that like to stay up to date and can implement changes expediently can do so. And it means that organisations which need to move more slowly with such things can probably round out their checks and balances a lot faster. It's less bureaucratic overhead to say "we're updating Lucee's PDF module" than it is to say "we're updating our Lucee installation". This in turn means that there doesn't need to be as much hesitation on the part of LAS if they do want to make breaking changes to modules... organisations who aren't ready for the breaking changes don't need to implement them. This means there can be a more dynamic attitude to moving things forward, as the "no person left behind" idea isn't so entrenched in whatever work is done. This goes back to Adobe's glacial approach to doing anything: the whole system is implemented and released as one huge blob, and everything has backwards compatibility issues, and even the smallest backwards compat challenge in one distant corner of the ColdFusion system will prevent the rest of it moving forward at a pace appropriate to 2016.

Update:

Another good example where this sort of thing pays off is detailed in Mary Jo's comment below: she needed to run an up-to-date version of Solr cos CF's one is so woefully out of date, but it wasn't really doable on CF due to its monolithic approach to packaging, and dependency spaghetti. This sort of thing will be much easier with Lucee 5 (not that they'd be so slack at keeping their libs up to date anyhow, but you know what I mean).


This is bloody great news for Lucee. I'm quite sure why they're not messaging it this way. Here's what they say about the OSGI work on their website (this is the page promoting it):

OSGi

Lucee 5 is completely OSGi based, OSGi is the defacto standard in most Java enterprise environments, to manage bundles (jar libraries) used by the environment.

This means all libraries used are managed by Lucee itself as OSGi bundles, but this does not end with 3rd party libraries. Lucee handles it's own core as an OSGi bundle, Lucee archives (.lar files) and even our Java based extensions are all OSGi based.

So Lucee 5 is an OSGi engine that manages all libraries used.

What is the benefit?

OSGi is a framework used in most major enterprise environments, it is well tested and stable.

OSGi allows for the running of different versions of the same library at the same time. Lucee for example is not using internally the latest version of the HSQLDB data source, but this does not stop you from using whichever version of this data source you like. OSGi allows you to use whichever version of a library you like, independent of what version of the same library Lucee itself is using or any other applications in the same environment.

We can update every library Lucee is using at any time without issue and without the need to restart the JVM. This means that Lucee patches can include updated libraries, making Lucee a lot more flexible than it was in the past.

Can I benefit from OSGi somehow?

Yes you can, Lucee 5.0 also comes with some new build in functions to use OSGi features directly.

Today you use createObject('java',"my.class.Path") to create Java Objects, but with the power of OSGi under the hood we can do more!

Lucee 5 introduces the function javaProxy() to load Java objects, this functions interface is optimized to OSGi, so you can do something like this:

dtf=javaProxy("org.joda.time.format.DateTimeFormat", "org-joda-time", "2.1.0");

The function syntax is:
javaProxy(string className [, string bundleName, string bundleVersion]);

Only the class name is required all other arguments are optional. Lucee will also download the bundles from the update provider automatically if they are not available locally.

The Lucee update provider already provides a wide range of bundles in various versions and this will be added to continuously overtime.

You can therefore load a certain class in a specific version, even, for example, if the Lucee core is using the same class with a different version.

OSGi is the biggest change in the Lucee 5.0 release that brings Lucee to a completely new level. Whilst it maybe not a dazzling new feature, it is a necessary one for an engine to be used in the enterprise segment.


Zzzzzzzzz. Oh sorry, you got to the end of that did you? Well done. I mean one can perhaps infer the bigger picture there if one takes the time, but - even as a reasonable adept technical person - I read that, my eyes glaze over, and I'm going "so the f*** what?". Sorry, but I am.

But if one frames it in the context of real world CFML devs/admins, dispense with the buzzwords and start it with "we've listened to the community and Lucee 5's platform is now completely modular [etc]", then you would have kept my interest all along.

I think the issue is that these docs are written by people so far up the Jefferies Tubes of Lucee they have lost sight of the fact they're not the core audience of Lucee. This was where I was struggling with Denny last night, and where I struggle in general with a bunch of the messaging from Lucee. In short, I guess, the technical people shouldn't be writing it or presenting it.

Now Geoff didsay he pointed out all this at the keynote of Dev.Objective(), and - yeah - he's the sort of person to understand how to message these things, but a presentation at Dev.Objective() is no use to most people, because most people weren't there. And to message didn't seem to trickle back out anyhow, so I wonder if it hit home anyway?

I really think this is a potentially excellent feature of Lucee, but not in the way they are currently positioning it. The message should not be about OSGi, it should be about modularity of the Lucee platform, and what immediate benefit it brings to Lucee's CFML community. Not to the few engineers behind the scenes who care about the implementation (we shouldn't need to care about the implementation for one thing), but how it's going to impact my life, as a Lucee CFML developer. It's not that OSGi has anything to do with it. It's that it modularises the platform.

I think there's a better story for LAS to be telling here.

Also in closing: thanks Denny, Geoff and Sean for the interesting discussion last night.

Righto. Time for breakfast and - more importantly - coffee.

--
Adam

Saturday, 23 January 2016

Am I being bullied? Or am I a bully? I'm definitely being a Big Meany again

G'day:
Just a quick one.

Michael Smith has been doing the hard work with the annual "State of the CF Union Survey 2016". Go fill it in if you're a CFML dev and haven't done so yet. Michael has posted some intermediary results: "State of the CF Union survey 2016 - partial results".

I had a quick scan over the results - I'll leave it for someone else to comment on them in more depth - but I just had to share this observation respondent #23 made:

Glad Adam Cameron is stopping CF, maybe the community as a whole can start to heal from his years of negative post after negative post. He did more damage to CF than any other single factor IMO.
Aaw bless. I'm pleased you think I have that much impact in the community. Sadly I think you put rather too much stock in my community "penetration" and my importance in the bigger scheme of things. Almost all people who use ColdFusion have no frickin' idea who I am. And - of the ones that do know who I am - I get the impression really rather a lot agree with my position, even if a few (a lot?) of them don't necessarily like the way I articulate it. Hey... even I don't like the way I articulate it some times.

But yes, sure, there's a tiny wee minority such as yerself who seem to take it all a bit too seriously. Fortunately no-one else does. Well: I say "fortunately"... I sometimes wish Adobe took it a bit more seriously, but hey ho: they don't. It won't stop me trying.

I'm gobsmacked you place me as the top of the "problem list" for what ails ColdFusion. That is... um... really ignorant. And perhaps a wee bit detached from reality.

I think it's safe to say that - other than a few features I pushed to get into CFML and were successful with, and a few bugs I leaned on Adobe (or Railo, or Lucee) to get fixed and was successful with that - my impact on ColdFusion and its direction has been minimal. Same as with pretty much everyone in the community. Adobe don't really listen to us, after all.

I s'pose I should be annoyed that you put more stock in my whiney blog posts (or comments elsewhere) than you do in all the positive work I've done for the community, of which there's been a fair amount I think? Most of my articles here are discussing coding concepts or documenting the solutions to problems and the like, and a lot have pretty positive feedback. And I do spend a fair chunk of my spare time on Stack Overflow (where I am the top - read "most helpful" - users for ColdFusion. And specifically for ColdFusion 11, 10. And 9 and 8. And Lucee. And second to Peter Boughton for CFML and Railo), and before that on the Adobe ColdFusion forums where I'm one of the top users despite having not really touched the thing for a few years. That's not an exercise in hubris, it's more just a demonstration of exactly how bloody awful I must be if all that lot is trumped by the odd "being a big meany" commentary I make. Bloody hell, I'm a hell of a big meany in your eyes.

It's a shame - for you - that you are a bit of a "glass half empty, and I think what's in it seems to be piss" sort of person.

But good on you for taking your time to make a comment like that. I hope it brought you warmth. It did bring me a chuckle.

Righto, off to do something more useful with my time (which would be... you know... anything really). I'll let you get back to pushing pins into the eyes of your Adam Cameron dolly, or whatever it is you get up to on the weekend.

--
Adam

Dependency injection strategy discussion

G'day:
Our dev department is spread across two campuses: at our head office in Dublin, and the one I'm based at in London.

Yesterday (Friday) the London bods had our weekly Dev Team catch-up, to discuss various issues that cropped up during the week which need more concerted discussion than an "ad hoc make a decision and get on with it" situation might resolve.

One of the chief discussion points is our dependency injection strategy, and the question was raised if we're doing it right, as it seems there were a few code smells creeping in in places.

As coincidence would have it... when we got out of the meeting a few of us had an email in our inbox from one of our mates in Dublin who was amidst much the same dilemma in some code he was working on.

This resulted in a further discussion between the London people, rounding out the afternoon. I think we clarified a few things really well, re-formalised our DI strategy in our heads, and identified some code we need to look at to work out what the actual cause of the smell is. What was really good is at the end of it we all agreed with where we got to, and the direction to take. As we have a few people who like holding fair strong opinions in the team ([raises his hand gingerly]), this is encouraging.

The gist of the conversation is worth repeating I reckon, and I hope the relevant parties don't mind. And I reckon it shows our team in pretty good light so I reckon the bosses won't mind either. I'm sure I'll find out if not!

OK, so - as you probably know - we're a PHP shop. For the main project I work on (www.hostelbookers.com) - we have just finished retiring the old ColdFusion-based site ("The end of ColdFusion"), replacing it with a PHP one running on the Silex micro-framework. Silex is built around Pimple, Sensio's DI Container. I've written a bit about Silex and Pimple if you want to get the gist of it.

Very quickly, Pimple implements DI via the notion of service providers, one of which might look like this (this is sample code from an earlier blog article: it's not code from our own app, nor is any code in this article):

namespace \dac\silexdemo\providers;

class ServiceProvider implements ServiceProviderInterface {

    use Silex\ServiceProviderInterface;
    use Silex\Application;
    use \dac\silexdemo\beans;

    public function register(Application $app){
        $app["services.user"] = $app->share(function($app) {
            return new services\User($app["factories.user"], $app["services.guzzle.client"]);
        });        

        $app["services.guzzle.client"] = function() {
            return new Client();
        };
    }
    
    public function boot(Application $app){
        // no booting requirements
    }
}


There's a register() method within which one defines all the objects and their dependencies, and a boot() method for calling code on a a defined object to be run before it's used. As all of these are defined using closures, no objects are actually created until they need to be used. Cool.

In this example I'm defining some services, but we define everything via service providers: services, controllers, repositories, factories, helpers etc.

We're defining the user service object there, and you can see from that that its constructor expects a coupla dependencies:

function __construct($userFactory, $guzzleClient){
    $this->userFactory = $userFactory;
    $this->guzzleClient = $guzzleClient;
}

From there, all the code in UserService can use the dependency to help out with its logic. Fairly standard stuff.

The chief conceit here is that we definitely use configuration over convention, and we configure our objects explicitly with the dependencies they need. UserService needs a UserFactory and it needs a GuzzleClient, so we specifically pass those in.

Aside: a word on configuration over convention

I really dislike the "convention" approach to pretty much anything like this... DI config... routing config etc. The reason being that the framework should simply busy itself with frameworking stuff, it should not be dictating to me how I write my code. That is not the job of the framework (be it a DI one or an MVC one). Also conventions are very opinionated, and we all know what is said about opinions. Frameworks that implement convention over configuration position the code as if the framework is the central element to the application, wherein it really ought to simply be some code that sits off to one side, and just gets on with it. My application is what is central to the application.

People will claim that it's easier to use convention of configuration, but I don't believe "easier" should be a primary consideration when designing one's app. Especially when it's "easier to use the framework", rather than "intrinsically good design".

But I digress.

This is fine, and there's no alarm bells ringing there.

Well one slight alarm might that say the object we're defining might have some actual OO type properties. This is a really contrived example (not to mention wrong) example, but this illustrates it:

Friday, 22 January 2016

Groovy: making and using a jar file

G'day:
First up: there's nothing clever in any of this, and I'm approaching it as a newbie. I just had to work out how to do this - which I have - and want to document it. It's nothing you couldn't find on Google with 15min effort.

We need to start writing some Groovy - cool! - for some tests we need to write. We're using SoapUI to test some of our web service end points (JSON schema validation and the like) and its built-in scripting language is Groovy. Our app itself is PHP, but that's irrelevant as it's hidden behind HTTP requests anyhow.

We have some shared utils that we want to us in our tests, and we're thinking about sticking these in a jar file so they can be a) factored-out into a separate source control project; b) plus just making our tests more focused. So I have to find out how to actually compile Groovy code, and stick it in a jar. I know if we're using jars then the source language doesn't need to be Groovy, but we've already got a bunch of the code written, so this is a refactoring exercise.

For my part, all my Groovyage so far has just been individual test scripts, run by the command-line interpreter. I've never needed to compile anything.

I'm not in the situation to actually work with our real test utils yet, so this is just a contrived example.

Here's my class:

package me.adamcameron.greetingapp

class Greeter {

    String greet(String name){
        return "G'day ${name}";
    }

}

Note I've got the package statement in there too. This is homed in me/adamcameron/greetingapp.

I googled about how to compile Groovy, and strangely enough it's much the same as Java:

>groovyc me\adamcameron\greetingapp\Greeter.groovy

>

This creates me/adamcameron/greetingapp/Greeter.class. I then jar it up in the usual way:

>jar cvf greetingapp.jar -C me\adamcameron\greetingapp .
added manifest
adding: Greeter.class(in = 4996) (out= 2092)(deflated 58%)
adding: Greeter.groovy(in = 120) (out= 101)(deflated 15%)

>


From there I just need to test it:

// testGreeter.groovy
import me.adamcameron.greetingapp.*

greeter = new Greeter()

println greeter.greet("Zachary")

And for Groovy to know where the jar file is, I need it to be on the class path:

>groovy -cp greetingapp.jar testGreeter.groovy

G'day Zachary

>

So there we go.

I didn't expect it to be tricky, but pleased how easy it really is.

Told you there'd be nothing insightful in this one!

Righto.

--
Adam

Tuesday, 19 January 2016

Floating point arithmetic with decimals

G'day:
As a human... what is the value of z, after you process this pseudocode with your wetware:

x = 17.76
y = 100
z = x * y

Hopefully you'd say "1776". It was not a trick question.

And that's an integer, right? Correct.

CFML

Now... try this CFML code:

x = 17.76;
y  = 100;
z = x*y;

writeOutput(z);

1776 So far so good.

But what about this:

writeOutput(isValid("integer", z));

You might think "YES" (or true if yer on Lucee), however it's "NO".

And this is where young players fall into the trap. They get all annoyed with isValid() getting it wrong, etc. Which, to be fair, is a reasonable assumption with isValid(), but it's not correct in this instance. It's the young player who is mistaken.

If we now do this:

writeOutput(z.getClass().getName());

We get: java.lang.Double

OK, but 1776 can be a Double, sure. But CFML should still consider a Double 1776 as a valid integer, as it should be able to be treated like one. So why doesn't it? What if we circumvent CFML, and go straight to Java:

writeOutput(z.toString());

1776.0000000000002

Boom. Floating point arithmetic inaccuracy.

Never ever ever forget, everyone... when you multiply floating point numbers with decimals... you will get "unexpected" (but you should pretty much expect it!) floating point accuracy issues. This is for the perennial reason that what's easy for us to express in decimal is actually quite hard for a computer to translate into binary accurately.

Aside: we were chatting about all this on the CFML Slack channel this morning, and one person asked "OK, so how come 17.75 x 100 works and 17.76 x 100 does not?". This is because a computer can represent 0.75 in binary exactly (2-1 + 2-2), whereas 0.76 can only be approximated, hence causing the "issue".

The problem really is that CFML should simply output 1776.0000000000002 when we ask it, and it should not try to be clever and hide this stuff. Because it's significant information. Then when the young player output the value, they'd go "oh yeah, better round that" or whatever they need to do before proceeding. CFML is not helping here.

This is pretty ubiquitous in programming. Let's have a trawl through the various languages I can write the simplest of code in:

JavaScript


x = 17.76;
y = 100;
z = x * y

console.log(z);


>node jsVersion.js
1776.0000000000002

>

JS just does what it's told. Unsurprisingly.

Groovy


x = 17.76
y = 100
z = x * y
println "x * y: " + z

println "x: " + x.getClass().getName()
println "y: " + y.getClass().getName()
println "z: " + z.getClass().getName()
println "z: " + z.toString()


>groovy32 groovyVersion.groovy

x * y: 1776.00
x: java.math.BigDecimal
y: java.lang.Integer
z: java.math.BigDecimal
z: 1776.00
>


This is interesting. Whilst Groovy keeps the result as a float (specifically a BigDecimal) - which is correct - it truncates it to the total number of decimal places expressed in its factors. That's how I was taught to do it in Physics at school, so I like this. This second example makes it more clear:

x = 3.30
y = 7.70
z = x * y
println "x * y: " + z

println "x: " + x.getClass().getName()
println "y: " + y.getClass().getName()
println "z: " + z.getClass().getName()
println "z: " + z.toString()


>groovy32 more.groovy
x * y: 25.4100
x: java.math.BigDecimal
y: java.math.BigDecimal
z: java.math.BigDecimal
z: 25.4100
>

In 3.30 and 7.70 there are four decimal places expressed (ie: two for each factor), so Groovy maintains that accuracy. Nice!


Java


import java.math.BigDecimal;

class JavaVersion {

    public static void main(String[] args){
        double x = 17.76;
        int y = 100;
        System.out.println(x*y);
        
        BigDecimal x2 = new BigDecimal(17.76);
        BigDecimal y2 = new BigDecimal(100);
        System.out.println(x2.multiply(y2));
        
    }
}

Here I added a different variation because I was trying to see why the Groovy code behaved the way it did, but it didn't answer my question. I suspected that perhaps it was a BigDecimal thing how it decided on the accuracy of the result, but it wasn't:


>java JavaVersion
1776.0000000000002
1776.000000000000156319401867222040891647338867187500

>

This is a good demonstration of how a simply base-10 decimal fraction is actually an irrational number in binary.

Monday, 18 January 2016

hackread.com: putting the "hack" back into journalism

G'day:
I can't post this in response to the article in question as the website requires a Facebook login, one of which I do not have. Anyway, there's this exercise in journalistic cheapshottery (yes, I meant that to be an O not an I) "Program Languages That Generate Most Software Security Bugs". Specifically:

Similar findings from OWASP test results show that ColdFusion, PHP, and Classic ASP, in that order, are the worst languages when it comes to software security.

I'd usually leave it to Brad "CFDoberman" Wood to refute, but I'm undergoing a "slow start Monday" today, so I'll rise to it.

My response was gonna be as follows:

I'm not gonna dispute the raw stats, but I will dispute the analysis and the technical competence of the author, in the context of at least "ColdFusion" (the language is CFML, btw, not ColdFusion. That's like describing .Net as a language. Oh... you do that too. They're not languages).

But anyway, the chief failing on your part here is that CFML is implemented basically as a tag lib atop of Java (that's understating things somewhat, but for these purposes it's appropriate). It's a JVM language. Therefore it's incorrect to say that Java's security features are not available to a CFML application. They intrinsically *are* available. It's all baked in.

As for SQL injection prevention... the tools are there as well to prevent against them... one just needs to use them. Same with XSS (it's a single checkbox config option on the ColdFusion management server).

The problem is not the language or the platform: it's the developers. Most CFML developers are bloody useless. This stems from the fact CFML is very easy to program in, meaning the small niche of developers it attracts often times are just not very good at their jobs, because they don't need to be to be productive. I guess this is the language's "fault": it's too easy to use.

I'm far less au fait with PHP, but I know that SQL injection here is also the fault of the developer, not the language. The tools are all there... ppl are just bad at their jobs.

I can't speak for ASP.

On another tangent... taking cheap shots at ColdFusion as a source of security dismay seems a bit odd to me anyhow. It's such a niche product these days... what percentage of WWW volume does it account for? Even if 100% of sites running CF were vulnerable (which, obviously, they're not)... that would be probably equivalent risk as 1% of PHP sites, given the increased market slice of PHP over CFML. So where does the real-world risk lie then?

All in all, lazy uneducated journalism here. But I imagine your remit is more to get eyes on the page than to do well-considered analysis, or to understand what you're talking about.

Cheers

I really shouldn't pay attention to this crap, as I don't want to encourage people going to the page, but... so be it.

If someone has a Facebook login (someone must), you could do me a favour and post this response in the comments (please attribute it to me, and cross ref back to this article)?

Righto. Back to things that are actually a good use of my time.

--
Adam