Railo adds more iteration functions: *some() and *every()

G'day:
Starting with ColdFusion 10 and some-version-or-other of Railo 4 (Railo moves so quickly I can't keep track of when features are added), CFML has been adding collection iteration functions which leverage callback functions to provided the functionality for each iteration. Examples are arraySort(), listFilter(), structEach() (the coverage in each CFML dialect for each collection data type is not fully comprehensive nor do they necessarily both implement the same functions the other does). ColdFusion added in map and reduce functions (detailed in my article "ColdFusion 11: .map() and .reduce()"), now Railo has added those in, and gone one better: now adding some() and every() functionality too. In this article I'll recap each of the sort / filter / each / map / reduce functionalities, and have a look at some() and every().This is a Railo-centric article. The code will not run on ColdFusion without modification.

First: recap.

sort()

A sort() operation is fairly obvious: it sorts the collection. The iterator function takes two arguments, and it returns how those values compare: -1 if the first is less than the second; 0 if they're the same; 1 if the second is higher than the first (so same as how the compare() CFML function works). How the callback arrives at that result is arbitrary; it can do anything it likes. Here's an example using both a function and a method of the array object to sort some Maori numbers:

numbers = [
    {maori="wha", digit=4},
    {maori="toru", digit=3},
    {maori="rua", digit=2},
    {maori="tahi", digit=1}
]

arraySort(numbers, function(v1,v2){
    return sgn(v1.digit - v2.digit)
})
dump(numbers)
echo("<hr>")

numbers.sort(function(v1,v2){
    return sgn(v2.digit - v1.digit)
})
dump(numbers)

This - predictably - sorts the array first by the substructs' digit key value ascending, then by the digit key value descending. Here the formula is the same: taking the sign of the difference of the two digits; the difference between ascending and descending is just whether I subtract v1 from v2 or vice-versa.

I noticed something interesting about the sorting process, but that will be covered in my next article, as I have not yet worked out WTF is going on.

filter()

A filter operation iterates over a collection, and for each element of the collection calls a callback. Here the callback's job is simply to return true or false. If the value is true, the element is kept in the collection; if it's false it's discarded. Here's an example in which some people offer their favourite colours (in English for a change). We then filter so that only people who selected a unique favourite colour is included (so Carol with her selection of "green"). In the second example we filter so that the people that share a favourite colour with someone else is retained:

answers = {
    angela    = "purple",
    bob        = "orange",
    carol    = "green",
    dave    = "purple",
    emma    = "orange",
    frank    = "purple"    
}

uniqueAnswers = structFilter(answers, function(key, value, struct){
    return arrayLen(structFindValue(struct, value, "all")) == 1
})
dump(uniqueAnswers)
echo("<hr>")

sharedAnswers = answers.filter(function(key, value, struct){
    return arrayLen(structFindValue(struct, value, "all")) > 1
})
dump(sharedAnswers)

Output:

Struct
CAROL	string green

---

Struct
ANGELA	string purple
BOB	string orange
DAVE	string purple
EMMA	string orange
FRANK	string purple

Here I am demonstrating that the entire struct is passed into the callback as the third argument (along with first the key, then second the value of that key). This means filter callbacks can base their processing on the whole struct, not just the key/value pair for that iteration.

each()

An each() iteration does not "return a value" from the callback, per se, it just loops over the collection and runs the callback each iteration. The callback can do anything one likes at all. In this sample code I first demonstrate the each() function, and the arguments passed to the callback; then use the arrayEach() function to output each element of the array; then the each() array method to output the array elements in uppercase.

numbers = ["tahi","rua","toru","wha"]

each(["single"],function(value,index,array){
    dump(arguments)
})
echo("<hr>")

arrayEach(numbers, function(v){
    echo (v & "<br>")
})
echo("<hr>")

numbers.each(function(v){
    echo(v.toUpperCase() & "<br>")
})

Scope Arguments
value	1	string single
index	2	number 1
array	3	Array 1 string single

---

tahi
rua
toru
wha


---

TAHI
RUA
TORU
WHA


The arguments passed in are the element value, its index in the array, and the whole array.


map() and reduce() are new to Railo.

map()

A map operation iterates over the collection, and - as expected - executes a callback for each element of the collection. The callback should return a value which becomes that element's equivalent value for a new collection. An example will make that more clear:

answers = {
    angela    = "purple",
    bob        = "orange",
    carol    = "green",
    dave    = "purple",
    emma    = "orange",
    frank    = "purple"    
}

individualsAnswerUniqueness = map(answers, function(key,value,struct){
    return {colour=value,factor=arrayLen(structFindValue(struct, value, "all"))}
})
dump(individualsAnswerUniqueness)
echo("<hr>")

Here we have the favourite colours again, but we create a new struct which shows the "uniqueness" of each person's favourite colour within the group (lower is better):

Struct
ANGELA	Struct COLOUR string purple FACTOR number 3
BOB	Struct COLOUR string orange FACTOR number 2
CAROL	Struct COLOUR string green FACTOR number 1
DAVE	Struct COLOUR string purple FACTOR number 3
EMMA	Struct COLOUR string orange FACTOR number 2
FRANK	Struct COLOUR string purple FACTOR number 3

The returned collection has the same element keys as the initial one, but new values for each element based on the callback.

Railo implements the controversial generic map() function, as well as type-specific functions like arrayMap() and structMap(), as well as map() methods on the various collection data types. I understand map() is going to be removed, being replaced with collectionMap() ("map()" caused problems in WireBox, as discussed yesterday: "Built-in functions, UDFs, methods, and function expressions... with the same name").

reduce()

This is probably the most complicated of the iteration functions as it builds a new value, based on the iteration process. It can be used to "flatten" a collection, which I've demonstrated before, but it can just be used to generate any completely different data structure, as demonstrated here:

answerTally = answers.reduce(function(previousValue,key,value,struct){
    if (!structKeyExists(previousValue, value)){
        previousValue[value] = {
            tally    = 0,
            people    = []
        }
    }
    previousValue[value].tally++
    arrayAppend(previousValue[value].people, key)
    return previousValue
}, {})
dump(answerTally)

(This assumes the same colour picks as the earlier examples).

This one outputs this:

Struct
green	Struct PEOPLE Array 1 string CAROL TALLY number 1
orange	Struct PEOPLE Array 1 string EMMA 2 string BOB TALLY number 2
purple	Struct PEOPLE Array 1 string FRANK 2 string DAVE 3 string ANGELA TALLY number 3

Basically it loops over the colour-pick collection and creates a summary object which tallies things up from the perspective of colour, not person: it lists who likes the given colour, and gives a tally as well.

The reduction process starts with a base value, and then each iteration takes that value as a starting point, and returns a new value. Here I am building on the passed-in value, but equally the value returned from each iteration could be a completely new value; an example would be summing all the elements in an array:

numbers = [1,2,3,4]
sum = numbers.reduce(
    function(prev,current){
        return prev+current;
    },
    0 // this is the starting point
)
dump([numbers,sum])

Array
1	Array 1 number 1 2 number 2 3 number 3 4 number 4
2	number 10

And now the stuff new to both Railo and CFML in general.

some()

The some iteration function returns a boolean. It returns true if some of the elements in the collection meet the test defined by the callback. Here's an example:

param name="URL.showTelemetry" default=false; // need a semicolon here for some reason

numbers = ["tahi","rua","toru","wha","rima","ono","whitu","waru","iwa","tekau"]

some = numbers.some(function(v,i,a){
    if (URL.showTelemetry) echo("Checking: #i# #v#<br>")
    return v[1] == URL.letter
})
echo("Does #numbers.toString()# contain any numbers starting with '#URL.letter#'? #some#<hr>")

Here I take some use input, and check to see if any of the numbers (we're back to Maori again) start with that letter. I also have an optional switch to output some telemetry, showing how the some() process iterates. Here's a basic run, just checking for the first letter being "w":

Does [tahi, rua, toru, wha, rima, ono, whitu, waru, iwa, tekau] contain any numbers starting with 'w'? true

No surprises then: Maori's fairly heavy on W usage, so any of wha, whitu or waru (btw, "wh" in Maori is pronounced more like an "f" in Japanese, not like in English. Like half way between an F and an H; it's quite hard for an English-only-speaker like me to voice) fit the bill there.

Let's have a look at what happens here with some telemetry being output:

Checking: 1 tahi
Checking: 2 rua
Checking: 3 toru
Checking: 4 wha
Does [tahi, rua, toru, wha, rima, ono, whitu, waru, iwa, tekau] contain any numbers starting with 'w'? true

So that's good news: it only iterates until it finds the first case that fulfills the requirement, then exits. There's no point continuing after it's found at least one element that matches the "some" criteria.

Also note in here I'm using a cool Railo trick: one can access a string via array notation: here v is the value of the element - eg "wha" - so v[1] is "w". Cool!

every()

every() is the spiritual opposite of some(): it returns true if all the collection elements match the criteria dictated by the callback. Again, this is determined by iterating over the collection, and calling the callback for each element until the first definitive result is returned: in this case a false. Here's an example:

every = numbers.every(function(v,i,a){
    if (URL.showTelemetry) echo("Checking: #i# #v#<br>")
    return v.length() >= URL.length
})
echo("Are all of #numbers.toString()# at least #URL.length# characters long? #every#<hr>")

This is similar to the previous one, except this time we check to see whether all the values have a length greater-than or equal-to the threshold. If we run this code with a length value of 3, we get this test run:

Checking: 1 tahi
Checking: 2 rua
Checking: 3 toru
Checking: 4 wha
Checking: 5 rima
Checking: 6 ono
Checking: 7 whitu
Checking: 8 waru
Checking: 9 iwa
Checking: 10 tekau
Are all of [tahi, rua, toru, wha, rima, ono, whitu, waru, iwa, tekau] at least 3 characters long? true

Obviously here it needs to iterate over everything to get a true value. On the other hand if we check for 4 instead of three, we exit quite quickly:

Checking: 1 tahi
Checking: 2 rua
Are all of [tahi, rua, toru, wha, rima, ono, whitu, waru, iwa, tekau] at least 4 characters long? false

"rua" doesn't fulfill the requirement of the callback, so that's it: the result is false.

Railo has implemented both a type-specific function for every() (eg: arrayEvery()), as well as a method on the array type, as demonstrated above.

What's next?

Railo still hasn't implemented any iteration functions for query objects (for that matter nor has ColdFusion... I dunno why this wasn't just an obvious requirement from the outset?), and Railo-specifically has not implemented them for lists. I am split on this. The dogmatist in me thinks "like it or not, CFML has a 'collection' data type which is a list, so for the sake of completeness and uniformity of the language, list-oriented iteration functions should be implemented along with the rest". The more pragmatic side of me thinks "yeah, but lists were a shite concept from the outset, so the less time spent 'enhancing' them the better". And it seems the community is split on this, from what I've read.

Micha is currently asking for input into how the query iteration functions should be implemented ('Query "closure" functions'), so go over to the Jira ticket and have a read and put your oar in if you have an opinion. Even if you don't use Railo, odds-on when Adobe get around to implementing these for ColdFusion, they will probably (/hopefully) follow Railo's lead, so go offer your input even if yer not a Railo user.

Sorry there was a lot of recap in this article compared to the earlier CF11 article coverings its new functions (which also had recap of the the older iteration functions). Hopefully it wasn't too dull. I really like these new functions, and can't wait to upgrade our production environment to be able to use them.

Nice work Railo. And, hey, nice work Adobe for their equivalent efforts.

--
Adam