ColdFusion 2016: arrays now passed by reference

G'day:
There's not much to say about this one. ColdFusion is finally playing catch-up, and standardising its approach to passing complex objects.

Traditionally, ColdFusion has passed "simple values" (strings, numerics, dates, etc) by value, and complex objects (structs, queries, XML) by reference (provided one understands "pass by reference" with these caveats: "Complex data-types in CF, and how they're not copied by reference"). The one exception was arrays. Arrays were passed by value. Why? I don't bloody know.

Right from the outset Railo decided that was bloody daft and has never done this, and this has been inherited by Lucee.

Now ColdFusion has caught up, even if I think the specific implementation is lacking.

There is a new setting for Application.cfc, this.passArrayByReference, which one can set to true (the default is false). If one sets that, then the behavour of passed arrays is changed.

Let's have a look at some code running on ColdFusion 11:

function arrayToUpperCase(array){
    for (var i=1; i <= array.len(); i++){
        array[i] = array[i].ucase();
    }
    return array;
}

rainbow    = ["Whero","Karaka","Kowhai","Kakariki","Kikorangi","Poropango","Papura"];

rainbowInUpperCase = arrayToUpperCase(rainbow);

writeDump(var=rainbow, label="rainbow", format="text");
writeDump(var=rainbowInUpperCase, label="rainbowInUpperCase", format="text");

This simple code has a function which takes an array, upper-cases each element and returns it. Afterwards, we dump both the original and returned arrays:

rainbow - array

1) Whero 
2) Karaka 
3) Kowhai 
4) Kakariki 
5) Kikorangi 
6) Poropango 
7) Papura 

rainbowInUpperCase - array

1) WHERO 
2) KARAKA 
3) KOWHAI 
4) KAKARIKI 
5) KIKORANGI 
6) POROPANGO 
7) PAPURA 

As you can see, when we modify the passed-in array, it does not impact the original array. Constrast this with the same operation with a struct:

function structToUpperCase(struct){
    for (var key in struct){
        struct[key] = struct[key].ucase();
    }
    return struct;
}

rainbow    = {red="Whero", orange="Karaka", yellow="Kowhai", green="Kakariki", blue="Kikorangi", indigo="Poropango", purple="Papura"};

rainbowInUpperCase = structToUpperCase(rainbow);

writeDump(var=rainbow, label="rainbow", format="text");
writeDump(var=rainbowInUpperCase, label="rainbowInUpperCase", format="text");

rainbow - struct

BLUE: KIKORANGI
GREEN: KAKARIKI
INDIGO: POROPANGO
ORANGE: KARAKA
PURPLE: PAPURA
RED: WHERO
YELLOW: KOWHAI

rainbowInUpperCase - struct

BLUE: KIKORANGI
GREEN: KAKARIKI
INDIGO: POROPANGO
ORANGE: KARAKA
PURPLE: PAPURA
RED: WHERO
YELLOW: KOWHAI

As you can see, because structs are passed by references, the argument in the function references the same struct as in the calling code, so changes to the argument are reflected in the original.

PHP: objects as arrays, and can CFML benefit from the concept?

G'day:
We had a situation at work the other day in which we needed a collection class, I think it was Properties (that's in the real estate sense of "property", not the OO one: I work for an online travel company, and this was part of a search result). This is distinct from a single Property which represents a single hotel / hostel / whatever.

Using CFML we would have used a record set to represent this, and in PHP we'd generally represent this as an array (of individual Property objects). However the collection of Properties needed other behaviour as well: storing the minimum price per night, highest rating percentage etc. So we're definitely wanting an object of some description to keep the array of properties and other... erm... properties (in the OO sense now) defined together in the one place.

A traditional approach might be this (pseudo code):

class Properties {
    Property[] properties;
    Decimal lowestPrice;
    Decimal highestRating;

    // and some accessors etc
}

And that would have been fine. However I rather liked the notion of instead of having to pluck the properties array out of the object before doing anything with it... just to use the object itself as an array. C# has an idea of Indexers, and I was pretty sure PHP had at least an Iterable interface, and perhaps some other bits and pieces that might work as well.

Indeed it does, and I was able to come up with an object which one can treat like an array. Almost.

// Collection.php

class Collection implements Iterator, Countable, ArrayAccess
{
    protected $collection;
    protected $index;

    public function __construct($array){
        $this->collection = $array;
    }

    public function getCollection() {
        return $this->collection;
    }

    public function current() {
        return $this->collection[$this->index];
    }

    public function key() {
        return $this->index;
    }

    public function next() {
        ++$this->index;
    }

    public function rewind() {
        $this->index = 0;
    }

    public function valid() {
        return array_key_exists($this->index, $this->collection);
    }

    public function count() {
        return count($this->collection);
    }

    public function offsetExists($index) {
        return array_key_exists($index, $this->collection);
    }

    public function offsetGet($index) {
        return $this->collection[$index];
    }

    public function offsetSet($index, $value) {
        echo "offsetSet() called with [$index], [$value]<br>";
        if (is_null($index)) {
            $this->collection[] = $value;
        }else{
            $this->collection[$index] = $value;
        }
    }

    public function offsetUnset($index) {
        unset($this->collection[$index]);
    }
}

I've highlighted the method implementation that each of the Iterator, Countable and ArrayAccess interfaces require. These methods allow me to traverse through the object - as one would in a foreach() loop; "count" the elements in the object (eg: using count()); and set elements into the object's array using array notation.

Having implemented these interfaces, I no longer need to extract the array from the object to use, I can access it via the object itself.

Here I populate a test collection with Maori numbers 1-4:

$numbers = ['tahi', 'rua', 'toru', 'wha'];
$numbersCollection = new Collection($numbers);
printf('Collection after initialisation: %s<br>', json_encode($numbersCollection->getCollection()));
echo '<hr>';

Output:
Collection after initialisation: ["tahi","rua","toru","wha"]

Then loop over the object using a normal foreach() loop:

foreach ($numbersCollection as $number) {
   echo "$number ";
}
echo '<hr>';

Output:
tahi rua toru wha

I can also set elements in the array via either a direct index reference:

$numbersCollection[4] = 'rima';
printf('Collection after setting [4]: %s<br>', json_encode($numbersCollection->getCollection()));

Output:

Collection after setting [4]: ["tahi","rua","toru","wha","rima"]

and even via the shorthand append syntax:

$numbersCollection[] = 'ono';
printf('Collection after setting []: %s<br>', json_encode($numbersCollection->getCollection()));

Output:

Collection after setting []: ["tahi","rua","toru","wha","rima","ono"]

All that's reasonably good!

However from here, the behaviour leaves a bit to be desired. Firstly here's an example of where PHP's array implementation demonstrates itself as being a bit wanting. I tested removing an element:

unset($numbersCollection[2]);
printf('Collection after unset(): %s<br>', json_encode($numbersCollection->getCollection()));

The intent here is to remove the third element: toru. Here's what we get:

Collection after unset(): {"0":"tahi","1":"rua","3":"wha","4":"rima","5":"ono"}

OK, that ain't good. It's cleared the value and removed the element, but it's not reindexed the "array", so there's a gap left in the indexes. As a result the JSON encoder has gone "array? that's not an array, sunshine... erm, I better make it an object" (notice the curly braces, not the square ones, and the explicitly stated keys).

Initially I thought this was a glitch in my implementation, so I tested on a native array:

unset($numbers[2]);
printf('Array after unset(): %s<br>', json_encode($numbers));

Output:

Array after unset(): {"0":"tahi","1":"rua","3":"wha"}

That's pants. Amateurs. This is an artifact of PHP not understanding that there's a difference between an indexed array (indexed via sequential numbers) and an associative array (indexed on arbitrary key), which is a fairly fundamental implementation shortfall.

And it turns out PHP's implementation of the ArrayAccess interface is a bit hamstrung too. Well the interface implementation itself is OK, but then PHP's internal usage of the interface is... lacking. Here I use a native array function on my collection:

$uppercaseNumbers = array_map(function($number){
    return strtoupper($number);
}, $numbersCollection);
printf('Collection used in array_map(): %s<br>', json_encode($uppercaseNumbers));

And this yields:

Warning: array_map(): Argument #2 should be an array in Collection.php on line 101
Collection used in array_map(): null

Because... whilst PHP offers the interface, it doesn't itself actually expect the interface in its array functions:

array array_map ( callable $callback , array $array1 [, array $... ] )

Note that it expects an array for the second (and subsequent) arguments. Not an ArrayAccess. If PHP is going to offer an interface to enable objects to have array behaviour... it really ought to have gone the whole hog, and implemented an ArrayImplementation interface, and then expect one of those in all its array functions.

This is the story of PHP: they seem to have the ideas, but they don't seem to know how to do a thorough job of planning and implementing the ideas.

---

If it was implemented thoroughly... could this general idea be useful for CFML? I'm thinking of one unified interface that describes what one needs for native CFML array functionality to be able to operate on a collection object? What would be needed?

// Array interface
interface {

    // access
    any function get(required numeric id);
    void function set(required numeric id, required any value);
    void function remove(required numeric id);
    numeric function size();
    
    // traversal
    void function reset();
    void function next();
    any function current();

}

That should provide all the hooks native CFML would need to be able to use an object as an array (eg: using native array access syntax, functions, methods, and general language constructs such as for() loops).

I think this would be a pretty handy addition to CFML? I think CFML has always offered reasonable OO opportunities for CFML code, but it itself is pretty lacking in the OO department when it comes to interacting with that code. This is an example of this: its array functionality only works with native arrays. It should work with any object that has the behaviour of an array. Something CFML has in common with PHP then. Another thing.


One thing my colleague/mate Brian pointed out (this was in the context of the PHP code, not this CFML idea) is that if we're using a dynamic language, should we need to implement actual interfaces? There's a good case for duck typing here. Provided a class implements all the requisite methods to be able to be treated as an array, should the runtime simply not allow an object to be used in place of an array.

EG: for an object to be used in a foreach() loop, it needs to provide a next() and get() method (maybe a getKey() and getValue() method. Provided the object has those methods, it should be usable in that situation. Even more so in the CFML situation: the class doesn't need to have the methods specified... provided the method is there at runtime, that should be fine (for my PHP readers, in CFML one can inject new methods into an object at runtime). This is true language dynamicism at work. This is where CFML (and PHP) should be positioning themselves: get rid of the ceremony, and just crack on with the business of being a modern dynamic language.

For PHP, they need to get their act together and understand the difference between an indexed array and an associative array. However this is a fundamental language problem that is unlikely to be addressed. For CFML... there's an opportunity here to make the language a bit more cooler.

Thoughts?

--
Adam

What should CFML's deleteAt() method return?

G'day:
This will be quick, as I'm out of time before I'm due to start work.

As I mentioned in my earlier article ("Weekend quiz: my answer (CFML version)"), Railo's (and ColdFusion's for that matter) Array.deleteAt() method returns a pointless boolean, rather than something useful. What do you think it should return?

Follow-up to a "not a bug" status from Adobe: arrayFindNoCase() returns false negatives

G'day:
About a year ago I raised a ticket observing that arrayFindNoCase() doesn't really work so reliably: 3316776. A coupla weeks ago it was marked as "not a bug", which I find rather bemusing - as there's definitely buggy behaviour - but not as bemusing as the explanatory feedback from Adobe.

arrayEach() could stand some improvement

G'day:
I'm going to lobby to get Saturday renamed "Sidetrackday". I sat down to write up a bunch of stuff one Ruby, and so far have written some CFML, some JS, some PHP and some Ruby... but not the Ruby I was intending do. Oh well.

I've touched in the past on my opinion that ColdFusion 10's new arrayEach() function is rather poorly/superficially implemented (here too), but today decided to check out how other languages deal with the same thing, and try to conclude what's the best approach for CFML to take.

So what's the story with arrayEach()?

Digression:

Those docs, btw, are frickin' useless:

• Where's the code example?
• Where's mention of the arguments passed to the callback? How many? What are they?
• See also "other closure functions". Should that be a hyperlink, or is it just meant to be a pointless exercise of stating the obvious?
• Category "Closure functions". Again: no link. And there isn't a category elsewhere in the docs "Closure Functions".
• The function argument doesn't need to be inline. It just needs to be a function reference.

There's more wrong about those docs that there is that is right. That's quite an achievement.

End of digression.

So in lieu of the docs explaining this, what does it do? Here's a baseline demonstration:

rainbow = ["Whero","Karaka","Kowhai","Kakariki","Kikorangi","Tawatawa","Mawhero"];

arrayEach(
    rainbow,
    function(v){
        writeOutput(v & "<br>");
    }
);

This outputs:

Whero
Karaka
Kowhai
Kakariki
Kikorangi
Tawatawa
Mawhero

Basically what arrayEach() does is loop over the passed-in array, and then passes the value of each element to the callback function. The callback function receives one argument, that value. That's all it receives. And it can return a value if it likes, but nothing is expecting the value, so for all intents and purposes, the callback's method signature is:

void function callback(Any value)

Superficially that's fine. However it's a very isolated-element-centric approach. What I mean is that as far as the callback goes, the value it receives - for all intents and purposes - is not an element of an array, it's simply a value. You cannot act on that value in its content in the array. What? Well let's say I want to transform the array's elements. A simple example would be to upper-case the elements of an array of strings. Maybe like this:

rainbow = ["Whero","Karaka","Kowhai","Kakariki","Kikorangi","Tawatawa","Mawhero"];

arrayEach(
    rainbow,
    function(v){
        return ucase(v);
    }
);

writeOutput(rainbow.toString());

Well... no. Remember how I said that nothing is listening for a return value? Well it's not. So you're returning your upper-cases element back to the ether. v is a disconnected value here: the callback doesn't get it as part of the array, it just gets it as a value.

OK, I'm very confused: arraySort() with a callback...?

G'day
I'm sitting at Shannon Airport awaiting my return flight to London, and killing time via Guinness and responding to comments on my blog. There are no little old catholic ladies here today to keep me bemused, indeed it's very quiet.

Anyway, this is the comment I'm currently trying to reply to:

I executed the example you have showed above for customised sorting using arraysort function, but i didn't get the same result. The output i got was the array that i defined in the code. I can't understand why its not happening for me.

 The code in question is this:

a = [
    {firstName="Witi", lastName="Ihimaera"},
    {firstName="Patricia", lastName="Grace"},
    {firstName="Alan", lastName="Duff"},
    {firstName="Lee", lastName="Tamahori"},    // OK: not an author
    {firstName="Keri", lastName="Hulme"}
];

arraySort(
    a,
    function (e1, e2){
        return compare(e1.lastName, e2.lastName) > 0;
    }
);
writeDump(a);

... however arrayContains() works much faster than arrayFind()

G'day:
This is a quick follow-up to that thing I mentioned the other day about the relative performance comparison of arrayFind() and listFind().

Arrays in Railo (appendix to the earlier 4-part series)

G'day:
Just when you thought it was safe... here's some more on arrays in ColdFusionRailo. I've already blathered on about the ins and outs of arrays in ColdFusion... well more like arrays in CFML as I cover some stuff in not only ColdFusion but also Railo and a bit on OpenBD too:

• Arrays in ColdFusion (part 1)
• Arrays in ColdFusion (part 2)
• Arrays in ColdFusion (part 3)
• Arrays in ColdFusion (part 4)

But a comment from Gert against one of my blog posts was tucked away in the back of my mind... Railo does some of its own thing when it comes to arrays (and structs, lists, etc).

Good things about Railo (#1) - CFLOOP

G'day, this is just a quick one.

In my last post I was discussing looping over arrays, and how Adobe messed-up their implementation of looping over an array with <cfloop> a bit (3321646).  Overnight, Gert from Railo added a comment, drawing my attention to Railo's handling of same.  And I gotta say I'm impressed.

Arrays in CFML (part 4)

G'day:
As the title suggests, this is part four of a series of articles I've written about arrays in CFML. So far I've covered what an array is and how to create them, plus two articles covering all the functions in CFML which specifically work with arrays.

• Arrays in CFML (part 1)
• Arrays in CFML (part 2)
• Arrays in CFML (part 3)

As well as these functions that are specifically implemented to maintain & manipulate arrays, there are a bunch of other functions relating to other data types that use arrays. I'm going to cover those in this article. I'll break them down by the data-type they relate to.  I'm not going to do indepth coverage of how the functions work, because I'll eventually cover them when I give the various other data types the same treatment I'm giving arrays here.  EG: I'll discuss structFindKey() thoroughly when I do an exploration into "Structs in ColdFusion" (OK, so you've been fore-warned... there's more of this to come! ;-).  I'm just gonna dwell on the array`ed-ness of the relevant functionality.

Arrays in CFML (part 3)

G'day:
This article finishes up the coverage of array functions in CFML (there's more to discuss than just the functions: I'll get to that stuff in yet another article). Most of the array functions were covered in an earlier article, and I'm just finishing of the rest of them here. The functions here haven't been put in a separate article for a good reason: it's just that I'm daft and forgot to put then in the earlier article that should have included them. I dunno how I missed them of the list initially, as I based my "todo list" on the list of functions on the "Array Functions" page from the CF docs. Like I said: I'm just daft. Anyway... here they are.

Arrays in CFML (part 2)

G'day:
In this article, which is part two in an ongoing series, I'll discuss the various functions CFML ships with for working with arrays. The first part described what an array is, and how to create them.

The online docs cover each of these functions too, but the coverage is pretty superficial. They cover all the syntax options and give a code example, but they don't go beyond that. Having been around the block a few times (and thus... err... ending up back where I started.. hmmmmm...), I've picked up the odd but of insight, and have explored some of the idiosyncrasies of CFML that isn't covered in the docs. I shall try to discuss why one might want to use a given function, rather than just cataloguing that it exists (which is all the official docs seem to set out to do, at times).

The docs are good for something though: they do act as a good lookup of what's available, so go have a look at then after yer done here.

Anyway, enough waffle...

Complex data-types in CF, and how they're not copied by reference

G'day
I've got part 2 of the array discussion underway, but I'm just going to digress slightly to write some notes up about how CFML deals with assigning and passing-around complex objects.

Again there's probably nothing new here for most bods out there, but hopefully it'll be useful to some people.

OK, so first up, there are a number of data types in CFML:

• string (including lists)
• date
• boolean
• numeric (including both integer and floating point)
• array
• struct
• query
• XML
• objects (specifically native CFML ones, ie: component instances)
• [others]

Included in "[others]" are stuff like file handles, images, spreadsheets, FTP connections, and stuff like that... there's a whole bunch of them, and I won't try to list all of them because I'll forget something, which'll mean someone will need to add a comment to point this out. So I'm just saving us all some time.

Arrays in CFML (part 1)

G'day:
I'm going to write up my understanding of how arrays work in CFML. This is a mundane topic in any programming language, but it's also absolutely critical, and having a good handle on how they work is very important. I will say up front that if you're a seasoned CFML developer that there probably won't be anything in here you don't already know.

Why do CFML arrays start at 1 instead of 0?

G'day
A few weeks ago whilst investigating something completely unrelated, I came across an interesting thread on StackOverflow, in which someone had asked why ColdFusion arrays start at index 1 rather than index 0 like other languages do.

This was an old thread, but my thoughts on it were a bit different from the others thusfar offered, so I wrote up my own point of view. Whilst writing it I started to think "I wish I had a blog for this sort of thing, rather than posting it on an old thread on StackOverflow"... and it was that thought that culminated in me finally starting this blog.

Anyway, I realise it's a bit of a duplication, but I'm "reclaiming" the post for myself. Here it is.