G'day:
I couldn't think of a title for this one, so: that'll do.
A few days ago Tom King posted this on the CFML Slack channel:
I'm sure I've asked this before, but has anyone got a good function to take complex values and parse out to a curl style string? i.e, https://trycf.com/gist/0c2672ac4a02d4c52d1e3ee1c9a408e7/lucee5?theme=monokaiThe code in the gist is thus:
original = {
'payment_method_types' = ['card'],
'line_items' = [
{
"price_data" = {
"product_data" = {
"name" = "Something"
},
"currency" = 'gbp',
"unit_amount" = 123
},
"quantity" = 1
}
]
};
writeDump(var=original, label="Original");
wantedResult["payment_method_types[0]"] = 'card';
wantedResult["line_items[0][price_data][product_data][name]"] ='Something';
wantedResult["line_items[0][price_data][currency]"] = 'gbp';
wantedResult["line_items[0][price_data][unit_amount]"] = 123;
wantedResult["line_items[0][quantity]"] = 1;
writeDump(var=wantedResult, label="Want this:");
So he wants to flatten out the key-path to the values. Cool. I'll give that a bash. Note: we've actually already solved this, but I'm now going back over the evolution of my solution, following my TDD steps. This is more a demo of using TDD than it is solving the problem: don't worry about how much code their is in this article, it's mostly incremental repetition, or just largely copy-and-pasted-and-tweaked tests. I will admit that for my first take on this I just wrote the one test that compared my result to his final expectation, and it took me far more time to sort it out than it ought to. I did the whole thing again last night to prep the code for this article, and using more focused TDD I solved it faster. Obviously I was benefiting from already kinda remembering how I did it the first time around, but even the first time around I could see how to do it before I even wrote any code, I just pantsed around making mistakes the first time due to not focusing. TDD on my second attempt helped me focus.
Last things first: the final test
I've got the "client's" final requirement already, so I'll create a test for that now, and watch my code error because it doesn't exist (my code, that is). I'll link the test case description through to the current state of the code in Github. I'll tag each step.
it("fulfils the requirement of the puzzle", () => {
input = {
"payment_method_types" = ["card"],
"line_items" = [
{
"price_data" = {
"product_data" = {
"name" = "Something"
},
"currency" = "gbp",
"unit_amount" = 123
},
"quantity" = 1
}
]
}
expected = {
"payment_method_types[0]" = "card",
"line_items[0][price_data][product_data][name]" = "Something",
"line_items[0][price_data][currency]" = "gbp",
"line_items[0][price_data][unit_amount]" = 123,
"line_items[0][quantity]" = 1
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
This errors with No matching function [flattenStruct] found, because I ain't even started yet! Note: for this exercise, I'm just building the function in the test class itself, given it's just one function, and it only needs to exist to fulfil this blogging/TDD exercise.
This test will keep erroring until I've finished. This is fine. I am not going to look at that test code again until I have finished the task though.
It handles simple value key/value pairs
I'm not going to be crazily pendantic with the TDD here. For example in this case I'm rolling in "creating the function and solving the first step of the challenge" into one step. I've already got a test failing due to the lack of the function existing, so I can watch it's progress for some of the micro-iterations I am missing out. So for this step I'm gonna iterate over the struct, and deal with the really easy cases: top level key/value pairs that are simple values and do not require any recursion to work. This is also the "bottom" level of the recursion I'll move on to, so know I already need it anyhow. Maybe making the decision that I need to use recursion to sort this out is pre-emptive, but I'm buggered if I know any other way of doing it. And taking recursion as a given, I always like to start with the lowest level solution: the bit that doesn't actually recurse.
Here's my test for this:
it("handles top-level key/values with simple values", () => {
input = {
"one" = "tahi",
"two" = {"second" = "rua"},
"three" = {
"third" = {"thrice" = "toru"}
}
}
expected = {
"[one]" = "tahi"
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
And the implementation to make it pass:
function flattenStruct(required struct struct) {
return struct.reduce((flattened, key, value) => {
if (isSimpleValue(value)) {
return {"[#key#]" = value}
}
return flattened
}, {})
}
That's lovely, and the test passes, but there's a bug in it which I didn't notice the first time around.
It handles multiple top-level key/values with simple values
it("handles multiple top-level key/values with simple values", () => {
input = {
"one" = "tahi",
"first" = "tuatahi",
"two" = {"second" = "rua"},
"three" = {
"third" = {"thrice" = "toru"}
}
}
expected = {
"[one]" = "tahi",
"[first]" = "tuatahi"
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
That fails with Expected [{[one]={tahi}, [first]={tuatahi}}] but received [{[one]={tahi}}]. It's cos I'm a dropkick and not appending each iteration to the previous:
return {"[#key#]" = value}
// should be
return flattened.append({"[#key#]" = value})
Once I fixed that: the two TDD tests are green. Obviously the initial one is still broken cos we ain't done.
It handles complex values
Now all I needed was to add some structs to the sample data for the test:
it("handles complex values", () => {
input = {
"one" = "tahi",
"first" = "tuatahi",
"two" = {"second" = "rua"},
"three" = {
"third" = {"thrice" = "toru"}
}
}
expected = {
"[one]" = "tahi",
"[first]" = "tuatahi",
"[two][second]" = "rua",
"[three][third][thrice]" = "toru"
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
And write a quick implementation:
function flattenStruct(required struct struct) {
flatten = (flattened, key, value, _, prefix="") => {
var prefixedKey = "#prefix#[#key#]"
if (isSimpleValue(value)) {
return flattened.append({"#prefixedKey#" = value})
}
return flattened.append(value.reduce(
function (flattened, key, value, _, prefix=prefixedKey) {
return flattened.append(flatten(flattened, key, value, _, prefix))
},
{}
))
}
return struct.reduce(flatten, {})
}
This warrants some explanation:
- We need to extract all the logic we currently have within the outer function into its own function. We need to do this because when we recurse, we need it to call itself, so it needs a name.
- The recursion works by saying to each level of the recursion "the previous level had this key, so append your keys to this", so we implement this, and default the prefix to be an empty string to start with.
- We're leveraging a nice feature of CFML here in that one can pass additional arguments to a function, and "it'll just work", so we pass that after all the other arguments. We're not using the fourth parameter to the reduce callback which contains the original collection being iterated over, so we reflect that by calling the parameter _ (this is a notional standard for this sort of thing).
- When we get to the "bottom" of the recursion we will always have a simple value for the value (otherwise we're not done recursing), so we prepend its key with the parent key (which will be all the ancestor keys combined).
I also now needed to update my previous test case expectations to include the structs coming out in the result too. In hindsight, I messed up there: I should only have had test data for the case at hand, rather than having some more data that I knew was specifically going to collapse in a heap.
But it also has a bug. The first test completely broke now: Can't cast Complex Object Type Struct to StringUse Built-In-Function "serialize(Struct):String" to create a String from Struct
I find the situation here a bit frustrating. The signature for the callback function for Struct.reduce is this:
function(previous, key, value, collection)And that's what I have in the code above.
However. I had overlooked that the signature for the callback function for Array.reduce is this:
function(previous, value, key, collection)It had never occurred to me that key, value and value, key were reversed between the two. Sigh. So my single handling of complex values there doesn't work because I'm passing Array.reduce the key and the value in the wrong order.
It's reasonably easy to fix though.
It handles arrays
I've added an array into the test data:
it("handles arrays values", () => {
input = {
"one" = "tahi",
"first" = "tuatahi",
"two" = {"second" = "rua"},
"three" = {
"third" = {"thrice" = "toru"}
},
"four" = ["fourth", "quaternary", "wha", "tuawha"]
}
expected = {
"[one]" = "tahi",
"[first]" = "tuatahi",
"[two][second]" = "rua",
"[three][third][thrice]" = "toru",
"[four][1]" = "fourth",
"[four][2]" = "quaternary",
"[four][3]" = "wha",
"[four][4]" = "tuawha"
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
And added logic to check if the value is a struct or an array before recursing. The variation just being to expect the callback parameters to be in a different order:
function flattenStruct(required struct struct) {
flatten = (flattened, key, value, _, prefix="") => {
var prefixedKey = "#prefix#[#key#]"
if (isSimpleValue(value)) {
return flattened.append({"#prefixedKey#" = value})
}
if (isStruct(value)) {
return flattened.append(value.reduce(
function (flattened={}, key, value, _, prefix=prefixedKey) {
return flattened.append(flatten(flattened, key, value, _, prefix))
},
{}
))
}
if (isArray(value)) {
return flattened.append(value.reduce(
function (flattened={}, value, index, _, prefix=prefixedKey) {
return flattened.append(flatten(flattened, index, value, _, prefix))
},
{}
))
}
}
return struct.reduce(flatten, {})
}
With this amendment all the TDD tests pass, but the original test - which I was kinda expecting to also pass now - wasn't. I had a coupla bugs in my implementation still, basically from me "not reading the requirements"
It qualifies the keys correctly and zero-indexes the arrays
Yes yes, very bad to fix two things at once, I know. I didn't notice that Tom's requirements were to present the flattened keys like this (taken from the previous test): four[0], whereas I'm doing the same one like this: [four][1]. So I'm not supposed to be putting the brackets around the top-level item, plus I need to present the arrays with a zero index. This is easily done, so I did both at once. I figure I'm kinda OK doing this cos I am starting with a failing test afterall. And I also added a more complex test case in (mixing arrays, structs, simple values some more) to make sure I had nailed it:
it("qualifies the keys correctly and zero-indexes the arrays", () => {
input = {
"one" = "tahi",
"first" = "tuatahi",
"two" = {"second" = "rua"},
"three" = {
"third" = {"thrice" = "toru"}
},
"four" = ["fourth", "quaternary", "wha", "tuawha"],
"five" = [
{"fifth" = "rima"},
["tuarima"],
"quinary"
]
}
expected = {
"one" = "tahi",
"first" = "tuatahi",
"two[second]" = "rua",
"three[third][thrice]" = "toru",
"four[0]" = "fourth",
"four[1]" = "quaternary",
"four[2]" = "wha",
"four[3]" = "tuawha",
"five[0][fifth]" = "rima",
"five[1][0]" = "tuarima",
"five[2]" = "quinary"
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
And the fix:
function flattenStruct(required struct struct) {
flatten = (flattened, key, value, actual, prefix="") => {
var offsetKey = isArray(actual) ? key - 1 : key
var qualifiedKey = prefix.len() > 0 ? "[#offsetKey#]" : offsetKey
var prefixedKey = "#prefix##qualifiedKey#"
if (isSimpleValue(value)) {
return flattened.append({"#prefixedKey#" = value})
}
if (isStruct(value)) {
return flattened.append(value.reduce(
function (flattened={}, key, value, actual, prefix=prefixedKey) {
return flattened.append(flatten(flattened, key, value, actual, prefix))
},
{}
))
}
if (isArray(value)) {
return flattened.append(value.reduce(
function (flattened={}, value, index, actual, prefix=prefixedKey) {
return flattened.append(flatten(flattened, index, value, actual, prefix))
},
{}
))
}
}
return struct.reduce(flatten, {})
}
Quite simply:
- If the data we are reducing is an array, we reduce the index by one;
- and we only put the brackets on if we've already got a prefix value.
- We're using that _ parameter we were not using before, so I've given it a name.
Now everything works:
Or does it?
It handles the collection being an arguments scope object
I handed this implementation (or something very similar) to Tom with a flourish, and a coupla hours later he was puzzled about what sort of object the arguments scope is. It behaves like both a struct and an array. For some things. And it's ambiguous in others. When Tom was passing an arguments scope to this function, it was breaking again:
The arguments scope passes an isArray check, but its reduce method handles it like a struct, so they key being passed is the parameter name, not the positional index like we'd be expecting with an array that it's just claimed to us that it is.
Let's have a test for this:
it("handles arguments objects", () => {
input = {
"one" = "tahi",
"first" = "tuatahi",
"two" = {"second" = "rua"},
"three" = {
"third" = {"thrice" = "toru"}
},
"four" = ["fourth", "quaternary", "wha", "tuawha"],
"five" = [
{"fifth" = "rima"},
["tuarima"],
"quinary"
],
"six" = ((sixth, senary) => arguments)("ono", "tuaono", [6])
}
expected = {
"one" = "tahi",
"first" = "tuatahi",
"two[second]" = "rua",
"three[third][thrice]" = "toru",
"four[0]" = "fourth",
"four[1]" = "quaternary",
"four[2]" = "wha",
"four[3]" = "tuawha",
"five[0][fifth]" = "rima",
"five[1][0]" = "tuarima",
"five[2]" = "quinary",
"six[sixth]" = "ono",
"six[senary]" = "tuaono",
"six[2][0]" = 6
}
actual = flattenStruct(input)
expect(actual).toBe(expected)
})
Note I'm using CFML's IIFE feature to call an inline function expression there.
I am not entirely happy with my fix for this, but I've gaffer-taped it up:
function flattenStruct(required struct struct) {
flatten = (flattened, key, value, actual, prefix="") => {
var offsetKey = isArray(actual) && isNumeric(key) ? key - 1 : key
var qualifiedKey = prefix.len() > 0 ? "[#offsetKey#]" : offsetKey
I'm just checking if the "key" is numeric before I decrement it.
There's perhaps one more case I should put in there. This will silently ignore anything other than simple values, arrays or structs. I had a mind to throw an exception at the end if we'd not already returned, but I didn't get around to it. This solved the need we had, so that's all good.
Hopefully you found another incremental TDD exercise useful, and also found the problem we were trying to solve an interesting one as well. I did.
Righto.
--
Adam
