Thursday, 18 February 2016

ColdFusion 2016: query iteration functions

G'day:
In ColdFusion 11, one of the best new features was the addition of collection iteration methods, ie stuff like Array.map(), Struct.each(), List.listReduce() etc. One glaring omission was the lack of these implemented for queries. I have no idea why they were omitted, but there you go.

Anyway, one of the few decent language features in ColdFusion 2016 is that these have finally been implemented. They mostly work as one would expect, but I'll run through examples of each anyhow.

each()

each() simply iterates over the query, calling the callback for each row. The function singature for the callback is:

void function(struct row, numeric index, query query)

An example of this in action is:

colours = queryNew("id,en,mi", "integer,varchar,varchar", [
    [1,"red","whero"],
    [2,"orange","kakariki"]
]);

colours.each(function(colour, index, colours){
    writeDump(arguments);
});

And the results:



Predictable and no surprises.

map()

map() is a bit more complicated. I'll start with an example to frame things:

colours = queryNew("id,en,mi", "integer,varchar,varchar", [
    [1,"red","whero"],
    [2,"orange","karaka"],
    [3,"yellow","kowhai"],
    [4,"green","kakariki"],
    [5,"blue","kikorangi"],
    [6,"indigo","poropango"],
    [10,"violet","papura"]
]);

maoriColours = colours.map(function(colour, index, colours){
    return {mi=colour.mi};
}, queryNew("mi","varchar"));    

writeDump(var=maoriColours);



The function signature for the callback is the same as before except it returns a struct:

struct function(struct row, numeric index, query query)

Note map() takes a second argument, which is a second query. This query acts as a template for the remapped query returned by map(). The map callback returns a struct with keys that are a subset of the column names in that query. Note here my template query has a column mi, and so does the struct I'm returning from the callback. If I try to return a different column name, I get an error:



It's important to note that that query is only used for the "schema" of the returned query, any rows in it are ignored. That whole second argument is optional, and if omitted the original query's schema is used instead.

reduce()

reduce() holds no surprises. Its callback signature is:

any function(any carry, struct row, numeric index, query query)

And an example is:

week = queryNew("id,en,mi", "integer,varchar,varchar", [
    [1,"Monday","Rāhina"],
    [2,"Tuesday","Rātū"],
    [3,"Wednesday","Rāapa"],
    [4,"Thursday","Rāpare"],
    [5,"Friday","Rāmere"],
    [6,"Saturday","Rāhoroi"],
    [7,"Sunday","Rātapu"]
]);

shortestMaoriDayName = week.reduce(function(shortest,number){
    if (shortest.len() == 0) return number.mi;
    return number.mi.len() < shortest.len() ? number.mi : shortest;
}, "");

writeOutput(shortestMaoriDayName);

This simply returns:

Rātū

sort()


The callback for this takes two elements to compare:

comparison function(any row1, any row2)

Where I say comparison as the return type here, I mean a numeric value that is <0, 0 or >0 indicating whether row1 is less then, equal to or greater than row2, by whatever yardstick is reflected by the callback. Same as any other sort handler. Example:

colours = queryNew("id,en,mi", "integer,varchar,varchar", [
    [1,"red","whero"],
    [2,"orange","karaka"],
    [3,"yellow","kowhai"],
    [4,"green","kakariki"],
    [5,"blue","kikorangi"],
    [6,"indigo","poropango"],
    [10,"violet","papura"]
]);

colours.sort(function(c1,c2){
    return compare(c1.mi, c2.mi);
});

writeDump(colours);

Note that sort() desn't return the sorted query - it'd be a lot better if it did, TBH, I think I might raise a bug for this (4119099) - it sorts the query inline. This prevents this method being chained, which is disappointing. The output for this is predictable:



(it's the Maori column that's sorted there... not so obvious given the ubiquity of Ks in Maori words).

filter()


Yay, here's the first definite bug. The filter() callback has this function signature:

boolean function(struct row, numeric index, query query)

And here's an example:
colours = queryNew("id,en,mi", "integer,varchar,varchar", [
    [1,"red","whero"],
    [2,"orange","karaka"],
    [3,"yellow","kowhai"],
    [4,"green","kakariki"],
    [5,"blue","kikorangi"],
    [6,"indigo","poropango"],
    [10,"violet","papura"]
]);

coloursWithoutK = colours.filter(function(colour, index, query){
writeDump(arguments);abort;
    return !colour.mi.find('k');
});

writeDump(coloursWithoutK);

This obviously (obviously!) returns a query with only the three rows which don't have the letter K in the Maori colours, right?



No. It returns an array. How did this get through QA (NB: there's a good reason why this was not fixed in the pre-release, but I cannot disclose why. It's not a good reason though).

So I'll raise a bug for that (4119103).

That's about it. There's a fail in sort() in that it does not return the sorted query, leaving the original alone, and an outright bug in the implementation of filter().

All in all: a good alpha-quality implementation of these functions though. And a good thing about ColdFusion 2016.

Righto.

--
Adam

Tuesday, 16 February 2016

ColdFusion 2016: trying to write about one thing; end up writing about another

G'day:
I was gonna do an article about ColdFusion 2016's new query iteration functions, but my fondness for using Maori as my sample data language has pointed me in the direction of a possible bug in ColdFusion 2016's CLI.

I've distilled it down to this:

CLI.writeln("kuputuhi tauira ki pūāhua nako");

If I run that via the ColdFusion 2016 CLI, I get this:


D:\src\CF12\cli\utf8>cf
CLI.writeln("kuputuhi tauira ki pūāhua nako");
^Z
kuputuhi tauira ki puahua nako
D:\src\CF12\cli\utf8>

Hmmm... where gone the diacritic marks?

Initially I put this down to a tweak I've made to the CLI batch file which allows me to put code straight into STDIN instead of a file and run it, so I ran the code old-school:

D:\src\CF12\cli\utf8>cf cfmlExample.cfm
kuputuhi tauira ki p┼½─?hua nako
D:\src\CF12\cli\utf8>

Yikes! Worse!

I figured there was a chance that Windows wasn't handling encoding in its own CLI box that well, so decided to see what PHP made of this:

<?php
echo "kuputuhi tauira ki pūāhua nako";

And this yields:

D:\src\CF12\cli\utf8>php phpExample.php
kuputuhi tauira ki pūāhua nako
D:\src\CF12\cli\utf8>

OK, so ColdFusion is doing no different from PHP here when running a file. On a whim I decided to try Ruby too, and this fared better:

puts "kuputuhi tauira ki pūāhua nako"


D:\src\CF12\cli\utf8>ruby rubyExample.rb
kuputuhi tauira ki pūāhua nako

D:\src\CF12\cli\utf8>


So it wasn't like it was impossible to display the right characters in a Windows CLI box, but I decided to google anyhow.

I found a PHP answer which said I needed to change the code page in the CLI box to support UTF-8, by doing this:


D:\src\CF12\cli\utf8>chcp 65001
Active code page: 65001

D:\src\CF12\cli\utf8>


This is a new one on me, but it seemed to sort PHP's issues out:


D:\src\CF12\cli\utf8>php phpExample.php
kuputuhi tauira ki pūāhua nako
D:\src\CF12\cli\utf8>


But it didn't really help ColdFusion:


D:\src\CF12\cli\utf8>cf cfmlExample.cfm
kuputuhi tauira ki pū�?hua nako
D:\src\CF12\cli\utf8>

On a further whim, I decided to extend my test bed to Python:

print("kuputuhi tauira ki pūāhua nako")


Without the chcp call, I got this:

D:\src\CF12\cli\utf8>py pythonExample.py
kuputuhi tauira ki pūāhua nako

D:\src\CF12\cli\utf8>


Which is exactly the same as ColdFusion's output. But once I make the chcp call:


D:\src\CF12\cli\utf8>chcp 65001
Active code page: 65001

D:\src\CF12\cli\utf8>py pythonExample.py
kuputuhi tauira ki pūāhua nako

D:\src\CF12\cli\utf8>

As you can see: it's fine.

So my conclusion here is that it's completely legit than ColdFusion might not be able to render UTF-8-encoded characters without the code page being actively changed, but even after that something is not right with the CLI. I supsect that when they load the file in, they don't give any thought to encoding at all, so the source code actually gets "corrupted" before it's run. I have to admit that my understanding of encoding is OK, but not fantastic, but when one throws Windows CLI code pages into the mix too... I dunno what my expectations ought to be. But it ain't working, that's fer sure.

Could someone please try this on *nix and see what they get when using a more robust shell?

Now it's too late for me to look at query iterations functions, so that might be a job for before work tomorrow.

Righto.

--
Adam

ColdFusion 2016: floor()

G'day:
OK, so there's a coupla other uninteresting features in ColdFusion 2016 which I'll look at. The first one is in response to a ticket John Whish raised a while back: Deprecate `int` in favour of `floor`.

SSIA, really, but not the grounds for doing so. The int() function doesn't really do what it's supposed to, as I wittered on about here: CFML: how should int() behave? int() should just return the integer part of a number (hint: it's in the name), but instead it performs a rounding exercise instead. This means with negative numbers, it gets the answer wrong.

Still: the horse has bolted and it's not like it can be fixed or dropped now. Speaking of fixing it, there is a function fix() which does what int() was supposed to do.

So what's this floor() function? It does exactly the same as int() currently does, but it's got a more sensible name when taken in conjunction with another already-existing ceiling(). The idea is that we abandon int() as being faulty, and promote floor() and ceiling() for the rounding down/up intergerising functions, and use fix() to just get the integer part. Basically: just forget about int() being a thing, and move on.

So this ain't much of an enhancement, it just does some language tidy-up.

Here, btw, is some code demonstrating what I'm on about:

<cfset numbers = [-3.6,-3.5,-3.4,-2.6,-2.5,-2.4,-0.5,0,0.4,0.5,0.6,1.5,1.6,2]>
<cfset handlers = [int,floor,ceiling,fix,round]>
<cfoutput>
<table border="1">
    <thead>
        <tr>
            <th>x</th>
            <cfloop array="#handlers#" item="handler">
                <th>#handler.getName()#</th>
            </cfloop>
        </tr>
    </thead>
    <tbody>
        <cfloop array="#numbers#" item="x">
            <tr>
                <td>#x#</td>
                <cfloop array="#handlers#" item="handler">
                    <td>#handler(x)#</td>
                </cfloop>
            </tr>
        </cfloop>
    </tbody>
</table>
</cfoutput>

Here's a rare instance of me using tags, eh? Well it's mostly output, and I needed a table to present it, so fair cop.

This outputs:

x int floor ceiling fix round
-3.6 -4 -4 -3 -3 -4
-3.5 -4 -4 -3 -3 -3
-3.4 -4 -4 -3 -3 -3
-2.6 -3 -3 -2 -2 -3
-2.5 -3 -3 -2 -2 -2
-2.4 -3 -3 -2 -2 -2
-0.5 -1 -1 0 0 0
0 0 0 0 0 0
0.4 0 0 1 0 0
0.5 0 0 1 0 1
0.6 0 0 1 0 1
1.5 1 1 2 1 2
1.6 1 1 2 1 2
2 2 2 2 2 2


All we see here is that int() (and now floor()) round to the next lowest integer, which might be a different integer from the integer part of the original number if the number is negative. ceiling() does the opposite: rounds to the next highest integer. fix() just returns the integer part of the number, and for good measure I've chucked round() in their too to show rounding to the nearest integer, which is yet another variation.

One thing Adobe have not done to finish this work is to mark int() as deprecated. That's kinda essential to this, as otherwise floor() is just adding clutter to the language, rather than being a step to tidy it up. I'll remind Adobe about this bit. Again.

So not very interesting, but a quick one to write up whilst waiting on a call.

Righto.

--
Adam

ColdFusion 201610.5.1: first attempt at installing...

G'day:
So I decided to bite the bullet, install that stupid Akamai download manager (in 2016, seriously?) and grabbed the ColdFusion 201610.5.1 trial.

Most of the way through the install process, I get this:



Just for the sake of people googling, I'll put that here again in text:

The Program can't start because MSVCR110.dll is missing from your computer. Try reinstalling the program to fix this problem.

I guess this is something to do with a C++ runtime I don't have installed, or something. I mean I do have the 2005, 2008, 2010, 2012, 2013 and 2015 ones installed. But not this one.

For the record, I'm doing this install on a Windows 10 Home 64-bit laptop.

One would think if an installer needed a dependency... the installer would know how to get hold of the dependency?

Oh well. I'll work out WTF, and report back.

Update:

Yeah, apparently I didn't have the 2011 redistributable, which is solved by installing the 2012 one. Which I already had installed. But, hey... I'll play their silly games and reinstalled it, and now ColdFusion 2016's installer has completed no worries.

I remain underwhelmed by ColdFusion 201610.5.1, needless to say.

Righto.

--
Adam

A more professional CFML blog: Ryan Guill

G'day:
(more professional than this one, I mean ;-)

Good news, CFML community: Ryan Guill has started blogging. It's @ http://ryanguill.com/.

He's off to a flying start with eight articles already:
If you don't know Ryan already, I can tell you he really knows his stuff, and is a great bloke, so what he writes will be worth reading.

Righto.

--
Adam

ColdFusion 2016: Adobe finally abandons CFML

G'day:
I should have suffixed that with "for all intents and purposes".

Today ColdFusion 2016 was released. My executive summary of this release is that it's by far the worst ColdFusion release since CFMX6.0. 6.0 was basically released as alpha quality, and not usable for I think three additional beta patches, finally being usable at 6.1.

ColdFusion 2016 continues Adobe's increasingly prevalent tendency to aim to appeal more to PHB types, but this time finally decides to completely marginalise the actual CFML developer. If ColdFusion 11's only addition to CFML had been <cfclient>... it'd still be a better ColdFusion release - in terms of CFML - than CF2016 is. That's how crap CF2016 is.

In this light, I think it's clear that Adobe have pretty much given up on the CFML community, instead thinking their "community" is a bunch of purchase-order-signing IT managers.

ColdFusion 2016 includes a bunch of non-CFML features, which are varying degrees of relevance and interest to the CFML developer, but there are only... 2 (f***ing two!!)... enhancements to the CFML language. We've been waiting two years for more stuff to go into the CFML language, and that's what we get. This is appallingly cynical from Adobe.

So what are the CFML features? Let me quote from the release docs:

Language enhancements

For safe navigation, “?.” operator has been introduced. Collections support for “ordered” and “sorted” has been introduced. Refer this section to know more about all the language enhancements.

That's it. Two things.

I've already written about both of the features they list above:


Both of these have changed since I wrote about them:
  • the scope of sorted/ordered structs has been pegged back a bit, and there's now only ordered structs, as Adobe realised "ordered" and "sorted" are synonymous terms, and they only really needed the one concept. The implementation they have shipped is a bare minimum to not take the piss, but it's not as good as the alpha version I wrote up. I'll leave it to you to examine the final implementation.
  • the implementation of the safe-navigation operator has been tweaked to also do a null-safety-check on the last right-hand-side operand (read the linked-to article for explanation of this), but it has been released in an incomplete state as it only supports dot notation, not bracket notation.
They've actually forgotten a coupla small things: there are now iteration methods for queries (.map().reduce() etc), and I heard someone mention there's also a valueArray() method now... I've not tested that.

I've had a quick look at the query iteration methods, but do't have anything to show for it yet. I'll get onto that next.

And actually there is another feature: arrays are now "passed by reference" rather than "passed by value", which is long overdue, and more a design bugfix than a feature IMO. And is that really a CFML feature? I guess it's code-centric.

To be fair there's also some tweaks to PDF functionality, but as the whole concept of PDFs makes me...

...zzzzzzzzzzzzzzzzzzz...

... sorry, dunno what happened: dropped off there for a second. Um... right... PDFs... yeah, one can do stuff with redaction and comments and metadata, but I have no idea what and intend to keep it that way.

The other actually good developer-centric feature of ColdFusion 2016 is the CLI, which is a welcome addition, and has been implemented well. But it's not a CFML enhancement. I wrote up the alpha implementation a while back: "ColdFusion 2016: the long-awaited CLI". It's not moved much since then, so I won't go into it more.

But what about all those tickets Adobe marked "to fix" shortly before and during the pre-release? Nothing, as far as I can tell. I know goalposts change during development of new software, but I really do think Adobe changed the status on those tickets purely to get people off their back, and they never really had any intention of fixing them in ColdFusion 2016. I'm sure someone will come back saying "they just said 'to fix', they never said when". That's just disingenuous. But I'm sure that was the particular combination of smoke and mirrors they were going to fall back on if anyone called them on it.

What other stuff is there?

NTLM authorisation has finally been added to I think all features which should have had it from the outset. This is not a new feature IMO, but basically bringing beta-quality features up to release quality. It makes them usable beyond general proof of concept.

There's a half-baked, feature which allows one to hive session storage off to a redis data store which isn't bad conceptually, but I don't think the realisation is that good. Still: this is an administrative feature, not a language feature.

The marquee features of ColdFusion 2016 are ColdFusion-administrative sort of stuff, indeed the main feature, the API Manager is a completely separate app, and I'm not sure why it's been shipped as part of ColdFusion. Note it's an API Manager, not a ColdFusion API Manager. SO it's like a separate Adobe product they bunged in with ColdFusion 2016 simply cos they got the same team developing it. Well: developing it at the expense of actually working on ColdFusion itself, it seems.

ColdFusion Builder has had a code security scanner built into it, but that's CFB not ColdFusion, so who cares about that? It requires using CFB to use it, so that pretty much invalidates it as far as being relevant goes, I think.

One caution I have here is that the pre-release cycle seemed very short this time around. It's usually about a year, I think; this time it was more like six months. I cannot go into detail about the inner workings of the PR due to an enduring NDA, but I will let the reduced duration speak for itself. As I would always advise: do not upgrade your environments yet. Perhaps put a ColdFusion 2016 server into your test lab and run your tests against it. But do not make plans to go live with it yet. I would usually recommend waiting until the .0.1 or .0.2 release of any new ColdFusion version before running with it, but if one takes the duration of the prerelease, I think one ought to consider this a public beta rather than the release version, so perhaps hold off longer even than usual.

What an abject disappoint.

As a footnote to this, Ray's written an article about his thoughts on ColdFusion 2016, as well as pushed the boat out a bit and made some (supposedly) NDA-friendly observations about the ColdFusion 2016 Pre-release programme. You've probably already read his stuff before landing here, but if not: Adobe ColdFusion 2016 Released.

I'll give those query iteration methods a whirl and report back...

--
Adam

Tuesday, 9 February 2016

"Adam Cameron is kind of a twat"

G'day:
I know it's puerile of me to revel in such things, but this made me laugh: "Adam Cameron is kind of a twat."

It is, of course, completely true.

Well done, "TimDuncanTheAlmighty". Well done.

Righto. All this twattery does not get done by me loafing around here. I better crack on with it.

--
Twat