Tuesday, 8 January 2013

Weird behaviour with CFHTTP and JSON

G'day:
Here's some useless information to file away wherever it is you file away such stuff.

I was code reviewing some of my colleague Dennis's (who has no online presence that I am aware of, so he doesn't get a link) work yesterday, and when processing an HTTP response, he had this sort of thing:

response = httpResponse.fileContent.toString();

My reaction was along the lines of "whoa there Nelly... it's just some JSON, so it's already intrinsically a string: we don't need to make it into a ReallyReallyString by toString()-ing it".

Well I'm glad I like being proven to be wrong, because that's what happened next.


Dennis had already been down the road that I was heading down, seen his logical expectations being dashed, and revised his code to turn the HTTP response into a string. Because it wasn't a string.

What CF (this is CF 9.0.1, which is relevant in this case) was giving us was a java.io.ByteArrayOutputStream. O of course it was. Because that's what the CFHTTP request had requested, obviously. Except that it hadn't. All it had requested was a JSON string. And that is exactly what it received. And then CF did the thing it does second best: monkeyed with something it had no reason to monkey with, and messed up in the process (footnote: the thing it does best is "make hard things easy". But it also excels in doing really illogical sh!t sometimes, too).

How CF handles JSON responses to HTTP requests gobsmacks me slightly.  Here's some code, and its output:

<cfhttp url="http://localhost:8902/shared/CF/CFML/tags/protocol/http/json/target.cfm" result="response">
<cfset responseBody = response.fileContent>
<cfoutput>
<table border="1">
    <tbody>
        <tr><th>HTTP Response</th><td><cfdump var="#response#" label="HTTP response"></td></tr>
        <tr><th>Response Body</th><td><cfdump var="#responseBody#" label="Response body"></td></tr>
        <tr><th>Java Class</th><td>#responseBody.getClass().getName()#</td></tr>
        <tr><th>isObject()</th><td>#isObject(responseBody)#</td></tr>
        <tr><th>isSimpleValue()</th><td>#isSimpleValue(responseBody)#</td></tr>
        <tr><th>isJson()</th><td>#isJson(responseBody)#</td></tr>
        <tr><th>isStruct()</th><td>#isStruct(responseBody)#</td></tr>
        <tr><th>toString()</th><td>#responseBody.toString()#</td></tr>
        <tr><th>Raw output</th><td>#responseBody#</td></tr>
        <tr><th>deserializeJson()</th><td><cfdump var="#deserializeJson(responseBody)#" label="deserializeJson()"></td></tr>
    </tbody>
</table>
</cfoutput>

And target.cfm (the target of the HTTP call) is simply this:

<cfcontent type="application/json">{"white":"ma","black":"mangu","red":"whero"}

And the output is this:
HTTP Response
HTTP response - struct
Charset[empty string]
ErrorDetail[empty string]
Filecontent
HTTP response - object of java.io.ByteArrayOutputStream
Class Namejava.io.ByteArrayOutputStream
Methods
MethodReturn Type
close()void
reset()void
size()int
toByteArray()byte[]
toString(java.lang.String)java.lang.String
toString(int)java.lang.String
toString()java.lang.String
write(int)void
write(byte[], int, int)void
writeTo(java.io.OutputStream)void
Parent Class
HeaderHTTP/1.0 200 OK Content-Type: application/json Connection: close Date: Tue, 08 Jan 2013 08:28:30 GMT Server: JRun Web Server
Mimetypeapplication/json
Responseheader
HTTP response - struct
Connectionclose
Content-Typeapplication/json
DateTue, 08 Jan 2013 08:28:30 GMT
ExplanationOK
Http_VersionHTTP/1.0
ServerJRun Web Server
Status_Code200
Statuscode200 OK
TextNO
Response Body
Response body - object of java.io.ByteArrayOutputStream
Class Namejava.io.ByteArrayOutputStream
Methods
MethodReturn Type
close()void
reset()void
size()int
toByteArray()byte[]
toString(java.lang.String)java.lang.String
toString(int)java.lang.String
toString()java.lang.String
write(int)void
write(byte[], int, int)void
writeTo(java.io.OutputStream)void
Parent Class
Java Classjava.io.ByteArrayOutputStream
isObject()YES
isSimpleValue()NO
isJson()YES
isStruct()NO
toString(){"white":"ma","black":"mangu","red":"whero"}
Raw output{"white":"ma","black":"mangu","red":"whero"}
deserializeJson()
deserializeJson() - struct
blackmangu
redwhero
whitema

What a mess. What were you thinking, ColdFusion?

Observations I can make here:
  • No, the response was not a java.io.ByteArrayOutputStream, never was, never was intended to be.
  • It is simultaneously not a simple value, but is JSON. That's not possible: JSON is a string format, as we all know.
  • Thankfully CF manages to untangle itself, and if one treats the value as a string, it seems to automatically call toString() and one gets a string. As was the initial intent.
  • This is all caused by setting the MIME type to be application/json (which is the correct thing to do). If the MIME type is omitted, CF doesn't monkey, and one gets a string back.
What I want to know is why CFHTTP took it upon itself to do any monkeying with the response body anyhow? I guess the idea is that it detected that the MIME type had the word "application" in it instead of "text", so decided... um... well I dunno what the logic was, actually.

ColdFusion guys, two things:
  1. CFML is a loosely typed language. Don't go looking around for data typing that isn't there;
  2. CFHTTP is for making HTTP requests. It's not for "making HTTP requests and then monkeying with the results". That would be the <cfhttpandthenmonkeywiththeresult> tag. My point being even if it was appropriate for the data to be treated as some form of binary, that is not CFHTTP's job. It's my code's job.
  3. (OK, three things). If you were to be monkeying with JSON, then surely doing a deserializeJson() call on it would be the sensible thing to do? Who the hell wants a ByteArrayOutputStream if one is expecting JSON?
That said, they might have already twigged to this, as this behaviour no longer exists in CF10. CF10 just returns a string. I couldn't find a bug that referenced this that they specifically fixed, so I don't know if this was by design or just a happy coincidence. I tested on CF8 as well, and it behaves like CF9 does.

To be all BBC about things and show balance and impartiality: Railo just returns a string. OpenBD follows ColdFusion's lead.

The bottom line here is... given if one just treats this ByteArrayOutputStreamas a string, it works like a string... do we really need the toString() call? I can see arguments both ways: if it walks like a string, and... err... quacks like... a... string (OK, that was a bad metaphor to corrupt, but you get the point... it's just OK to treat it as if it's a string). Or one could be pedantic (and clear) about things, and demonstrate in the code that we know it's a ByteArrayOutputStrea, but we definitely want it to be a string now.  Thoughts?

Must dash... doctor's appt in 15min (which I will be late for).

--
Adam