Showing posts with label PHP. Show all posts
Showing posts with label PHP. Show all posts

Friday 30 December 2016

PHP: using SoapClient to consume a SOAP web service

G'day:
Yeah, OK, just don't ask why I'm needing to consume SOAP web services. It's Adobe's fault. Not because of ColdFusion for a change, but because an app they offer which only exposes its API via SOAP. Because it's still 2006 at Adobe, it seems.

For the last few weeks, this has been running through my head:



But anyway.

Fighting with the Adobe web service was a right pain in the arse, but I won't go into that - it's not useful info to share - but I did have to monkey around with how PHP consumes SOAP web services, and piecing it together took a chunk of googling for me to get my brain around it, so I thought I'd write it up here.

I hasten to add that most of the hassle getting it to work was because of the shitty Adobe web service being unhelpful, so it's perhaps not a challenge for most other people.

I cannot deal with the Adobe web service in this article as it's subscription-only, and it's kinda work-related stuff I'm not allowed to expose here. So I knocked together my own web service to consume. I used CFML for this as all it takes to create a SOAP web service in CFML is to give a method an access qualifier of remote, and have it web-accessible. ColdFusion handles everything else.

Here's the web service code (I'll keep the CFML code to a minimum, but it's all on GitHub anyhow):

import me.adamcameron.accounts.*;

component wsversion=1 {

    remote Invoice function getById(numeric id) returnformat="wddx" {
        var address = new Address(1, "London", "United Kingdom", "E18");

        var account = new PersonalAccount(2, "Adam", "Cameron", "1970-02-17", address);

        var penguin = new Product(3, "Penguin", 4.56);
        var pangolin = new Product(7, "Pangolin", 8.90);
        var platypus = new Product(11, "Playtpus", 12.13);

        var lines = [
            new InvoiceLine(14, penguin, 15, 16.17),
            new InvoiceLine(18, pangolin, 19, 20.21),
            new InvoiceLine(22, platypus, 23, 24.25)
        ];

        invoice = new Invoice(id, account, lines);

        return invoice;        
    }

}

Note that I should not have to state the wsversion there, but for some reason I could not get the default - version 2 - to work, so I had to force it to be version 1. I didn't look into why it didn't work on version 2, as I really didn't want to spend more time than necessary on the CFML code for this exercise.

Other than that, this code is straight forward: it returns an Invoice object which comprises a Person (with an Address), and an array of InvoiceLines each of which have a Product. the modelling for those is the same as in the receiving PHP code, further down.

On the PHP side of things, I just call this code to consume it:

<?php

namespace me\adamcameron\accounts;

require __DIR__ . '/model.php';

$wsdl = "http://localhost:8516/cfml/webservices/soap/accounts/public/Invoices.cfc?wsdl";


$options = [
    'trace' => true
];

$client = new \SoapClient($wsdl, $options);

$invoice = $client->getById(2011);

var_dump($invoice);
echo "==============================" . PHP_EOL . PHP_EOL;
var_dump($client->__getLastResponse());

So it's simply a matter of creating a SoapClient with the WSDL URL and some options. I'm using the trace option here so I can do that call to __getLastResponse.

Let's have a look at the raw SOAP response first (warning: it's pretty turgid reading):

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <soapenv:Body>
        <ns1:getByIdResponse xmlns:ns1="http://public.accounts.soap.webservices.cfml" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
            <getByIdReturn xmlns:ns2="http://accounts.adamcameron.me" xsi:type="ns2:Invoice">
                <account xsi:type="ns2:PersonalAccount">
                    <address xsi:type="ns2:Address">
                        <country xsi:type="xsd:string">United Kingdom</country>
                        <id xsi:type="xsd:double">1.0</id>
                        <localPart xsi:type="xsd:string">London</localPart>
                        <postcode xsi:type="xsd:string">E18</postcode>
                    </address>
                    <dateOfBirth xsi:type="xsd:string">1970-02-17</dateOfBirth>
                    <firstName xsi:type="xsd:string">Adam</firstName>
                    <id xsi:type="xsd:double">2.0</id>
                    <lastName xsi:type="xsd:string">Cameron</lastName>
                </account>
                <id xsi:type="xsd:double">2011.0</id>
                <items xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" soapenc:arrayType="xsd:anyType[3]" xsi:type="soapenc:Array">
                    <items xsi:type="ns2:InvoiceLine">
                        <count xsi:type="xsd:double">15.0</count>
                        <id xsi:type="xsd:double">14.0</id>
                        <price xsi:type="xsd:double">16.17</price>
                        <product xsi:type="ns2:Product">
                            <description xsi:type="xsd:string">Penguin</description>
                            <id xsi:type="xsd:double">3.0</id>
                            <rrp xsi:type="xsd:double">4.56</rrp>
                        </product>
                    </items>
                    <items xsi:type="ns2:InvoiceLine">
                        <count xsi:type="xsd:double">19.0</count>
                        <id xsi:type="xsd:double">18.0</id>
                        <price xsi:type="xsd:double">20.21</price>
                        <product xsi:type="ns2:Product">
                            <description xsi:type="xsd:string">Pangolin</description>
                            <id xsi:type="xsd:double">7.0</id>
                            <rrp xsi:type="xsd:double">8.9</rrp>
                        </product>
                    </items>
                    <items xsi:type="ns2:InvoiceLine">
                        <count xsi:type="xsd:double">23.0</count>
                        <id xsi:type="xsd:double">22.0</id>
                        <price xsi:type="xsd:double">24.25</price>
                        <product xsi:type="ns2:Product">
                            <description xsi:type="xsd:string">Playtpus</description>
                            <id xsi:type="xsd:double">11.0</id>
                            <rrp xsi:type="xsd:double">12.13</rrp>
                        </product>
                    </items>
                </items>
            </getByIdReturn>
        </ns1:getByIdResponse>
    </soapenv:Body>
</soapenv:Envelope>

Wot a bloody mouthful!

PHP takes all that in its stride, and rehydrates that into an object:

object(stdClass)#2 (3) {
    ["account"]=>
    object(stdClass)#3 (5) {
        ["address"]=>
        object(stdClass)#4 (4) {
            ["country"]=>
            string(14) "United Kingdom"
            ["id"]=>
            float(1)
            ["localPart"]=>
            string(6) "London"
            ["postcode"]=>
            string(3) "E18"
        }
        ["dateOfBirth"]=>
        string(10) "1970-02-17"
        ["firstName"]=>
        string(4) "Adam"
        ["id"]=>
        float(2)
        ["lastName"]=>
        string(7) "Cameron"
    }
    ["id"]=>
    float(2011)
    ["items"]=>
    array(3) {
        [0]=>
        object(stdClass)#5 (4) {
            ["count"]=>
            float(15)
            ["id"]=>
            float(14)
            ["price"]=>
            float(16.17)
            ["product"]=>
            object(stdClass)#6 (3) {
                ["description"]=>
                string(7) "Penguin"
                ["id"]=>
                float(3)
                ["rrp"]=>
                float(4.56)
            }
        }
        [1]=>
        object(stdClass)#7 (4) {
            ["count"]=>
            float(19)
            ["id"]=>
            float(18)
            ["price"]=>
            float(20.21)
            ["product"]=>
            object(stdClass)#8 (3) {
                ["description"]=>
                string(8) "Pangolin"
                ["id"]=>
                float(7)
                ["rrp"]=>
                float(8.9)
            }
        }
        [2]=>
        object(stdClass)#9 (4) {
            ["count"]=>
            float(23)
            ["id"]=>
            float(22)
            ["price"]=>
            float(24.25)
            ["product"]=>
            object(stdClass)#10 (3) {
                ["description"]=>
                string(8) "Playtpus"
                ["id"]=>
                float(11)
                ["rrp"]=>
                float(12.13)
            }
        }
    }
}

That's cool, but if we look at the SOAP response, we do have the object information in there too. Here's an extract:

xsi:type="ns2:Invoice"&gt;
    &lt;account xsi:type="ns2:PersonalAccount"&gt;
        &lt;address xsi:type="ns2:Address"&gt;
            &lt;country xsi:type="xsd:string"&gt;United Kingdom&lt;/country&gt;
            &lt;id xsi:type="xsd:double"&gt;1.0&lt;/id&gt;
            &lt;localPart xsi:type="xsd:string"&gt;London&lt;/localPart&gt;
            &lt;postcode xsi:type="xsd:string"&gt;E18&lt;/postcode&gt;
        &lt;/address&gt;
        &lt;dateOfBirth xsi:type="xsd:string"&gt;1970-02-17&lt;/dateOfBirth&gt;
        &lt;firstName xsi:type="xsd:string"&gt;Adam&lt;/firstName&gt;
        &lt;id xsi:type="xsd:double"&gt;2.0&lt;/id&gt;
        &lt;lastName xsi:type="xsd:string"&gt;Cameron&lt;/lastName&gt;
    &lt;/account&gt;
    &lt;id xsi:type="xsd:double"&gt;2011.0&lt;/id&gt;
    &lt;items xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" soapenc:arrayType="xsd:anyType[3]" xsi:type="soapenc:Array"&gt;
        &lt;items xsi:type="ns2:InvoiceLine"&gt;
            &lt;count xsi:type="xsd:double"&gt;15.0&lt;/count&gt;
            &lt;id xsi:type="xsd:double"&gt;14.0&lt;/id&gt;
            &lt;price xsi:type="xsd:double"&gt;16.17&lt;/price&gt;
            &lt;product xsi:type="ns2:Product"&gt;
                &lt;description xsi:type="xsd:string"&gt;Penguin&lt;/description&gt;
                &lt;id xsi:type="xsd:double"&gt;3.0&lt;/id&gt;
                &lt;rrp xsi:type="xsd:double"&gt;4.56&lt;/rrp&gt;
            &lt;/product&gt;
        &lt;/items&gt;
        &lt;items xsi:type="ns2:InvoiceLine"&gt;
            &lt;!-- etc &gt;


Hidden away in there we see there's an Invoice, PersonalAccount, Address, InvoiceLines and Products.

We can leverage this information to point PHP at classes to use when rehydrating the response, which is cool. We give the SoapClient a type map:

$options = [
    'trace' => true,
    'typemap' => [[
        'type_ns' => 'http://accounts.adamcameron.me',
        'type_name' => 'Address',
        'from_xml' => ['\me\adamcameron\accounts\Address', 'createFromXml']
    ],[
        'type_ns' => 'http://accounts.adamcameron.me',
        'type_name' => 'Account',
        'from_xml' => ['\me\adamcameron\accounts\Account', 'createFromXml']
    ],[
        'type_ns' => 'http://accounts.adamcameron.me',
        'type_name' => 'Product',
        'from_xml' => ['\me\adamcameron\accounts\Product', 'createFromXml']
    ],[
        'type_ns' => 'http://accounts.adamcameron.me',
        'type_name' => 'InvoiceLine',
        'from_xml' => ['\me\adamcameron\accounts\InvoiceLine', 'createFromXml']
    ],[
        'type_ns' => 'http://accounts.adamcameron.me',
        'type_name' => 'Invoice',
        'from_xml' => ['\me\adamcameron\accounts\Invoice', 'createFromXml']
    ]]
];

$client = new \SoapClient($wsdl, $options);

This maps the SOAP type to a PHP class, and method to use to rehydrate the object.

Running the code again with the typemap yields a fully-modelled result:

object(me\adamcameron\accounts\Invoice)#3 (3) {
    // ...
    ["account"]=>
    object(me\adamcameron\accounts\Account)#5 (5) {
        // ...
        ["address"]=>
        object(me\adamcameron\accounts\Address)#9 (5) {
            // ...
        }
    }
    ["items"]=>
    array(3) {
        [0]=>
        object(me\adamcameron\accounts\InvoiceLine)#13 (4) {
            // ...
            ["product"]=>
            object(me\adamcameron\accounts\Product)#15 (3) {
                // ...
            }
            // ...
        }
        [1]=>
        object(me\adamcameron\accounts\InvoiceLine)#14 (4) {
            // ...
            ["product"]=>
            object(me\adamcameron\accounts\Product)#17 (3) {
                // ...
            }
            // ...
        }
        [2]=>
        object(me\adamcameron\accounts\InvoiceLine)#16 (4) {
            // ...

            ["product"]=>
            object(me\adamcameron\accounts\Product)#19 (3) {
                // ...
            }
            // ...
        }
    }
}

(I've elided the irrelevant stuff from that one).

So that's pretty handy.

I do wish I could provide the typemap with each call though if I wanted to, not just when creating the SoapClient object. Oh well: it was no huge hardship.

There's no trick on the class side of things at all. Here's the InvoiceLine class:

<?php

namespace me\adamcameron\accounts;

class InvoiceLine {

    public $id;
    public $product;
    public $count;
    public $price;
        
    function __construct(int $id, Product $product, int $count, float $price){
        $this->id = $id;
        $this->product = $product;
        $this->count = $count;
        $this->price = $price;
    }
    
    static function createFromXml ($xml){
        $sxe = is_string($xml) ? new \SimpleXMLElement($xml) : $xml;
        $obj = new InvoiceLine(
            (int)$sxe->id,
            Product::createFromXml($sxe->product),
            (int)$sxe->count,
            (float)$sxe->price
        );
        return $obj;            
    }
}


OK the one trick is that the SoapClient only passes the "top level" object's XML through to the top level method. So in this case SoapClient calls InvoiceLine->createFromXml as per the type map, but from there it's up to me to do the same for any other objects this one uses, for example a Product in this case.

Oh and the other trick is that because of this, I need that XML / string differentiator at the beginning of the method. SoapClient passes in a string, but once I've converted that to XML, that's what the other calls will receive (ie: Product::createFromXml receives XML, not a string). That took me a while to nut-out, but that's just me being a div.

Oh yeah, and don't forget to explicitly cast the values going into yer properties, otherwise you'll end up with XML in there, not the actual values. That threw me for a while as well :-/

All the rest of the PHP code for this is on GitHub too.

The problem I had with the Adobe web service is that it didn't bother including the type attributes in its responses, other than on the container element (so in my example the array of InvoiceItems would have had a type on it). This means I had to pass the entire collection container into a handler method which then read the XML tag names and hand-cranked a switch based on that as to what rehydration method to use. I'm gonna try to contrive a non-business-sensitive version of that to show how I dealt with it.

Now... I've received orders from a mate that I'm expected at the pub in an hour. So I had better get cracking. Sorry to make you think about SOAP. And Adobe.

Righto.

--
Adam

Friday 16 December 2016

PHP: adolescent weirdness with arrays

G'day:
This was put on my radar the other day, by Alexander Lisachenko:


Well put. The code concerned is thus:

<?php
$A = [1 => 1, 2 => 0, 3 => 1];
$B = [1 => 1, 3 => 0, 2 => 1];

var_dump($A > $B);
var_dump($B > $A);


var_dump($A < $B);
var_dump($B < $A);

And it outputs...

C:\temp>php array.php
bool(true)
bool(true)
bool(true)
bool(true)

C:\temp>

Well that's quite... contrary.

PHP Team member Kalle Sommer Nielsen explained that this sort of comparison (between arrays) is "unsupported" in PHP.

Further conversation ensued, which you can read on Twitter, rather than me repeating it here.

My thoughts on this are that it's completely fine for PHP to not support this sort of comparison, but in that case the way it should deal with it is actively not support it: raise an IllegalOperationException or some such (some extension of LogicException, anyhow). It should not just issue forth illogical and misleading results. Kalle explains some of the background to the situation, which was helpful.

Another issue I have is that if we conclude that asking "is this array greater than that array" doesn't simply result in an error, and it must answer "true" or "false", then the answer ought to be false. Kalle contended it didn't matter, but my parallel is "cheese is greater than blue (true or false)?". It's a nonsensical question, But the answer is not "true".

The conclusion was: well it's done now... one shouldn't try to do these things, and whilst PHP might not handle it in an ideal fashion, it's a sufficient edge-case to not warrant remedial action. All fair enough.

I had a quick look at what PHP does with various comparisons:

<?php

$i1 = 5;
$i2 = 7;
echo "$i1 > $i2: " . booleanAsString($i1 > $i2) . PHP_EOL;
echo "$i1 < $i2: " . booleanAsString($i1 < $i2) . PHP_EOL;

$f1 = 5.5;
$f2 = 7.7;
echo "$f1 > $f2: " . booleanAsString($f1 > $f2) . PHP_EOL;
echo "$f1 < $f2: " . booleanAsString($f1 < $f2) . PHP_EOL;

$s1 = "a";
$s2 = "z";
echo "$s1 > $s2: " . booleanAsString($s1 > $s2) . PHP_EOL;
echo "$s1 < $s2: " . booleanAsString($s1 < $s2) . PHP_EOL;

$s1 = "A";
$s2 = "a";
echo "$s1 > $s2: " . booleanAsString($s1 > $s2) . PHP_EOL;
echo "$s1 < $s2: " . booleanAsString($s1 < $s2) . PHP_EOL;

$b1 = true;
$b2 = false;
echo sprintf("%s > %s: ", booleanAsString($b1), booleanAsString($b2)) . booleanAsString($b1 > $b2) . PHP_EOL;
echo sprintf("%s < %s: ", booleanAsString($b1), booleanAsString($b2)) . booleanAsString($b1 < $b2) . PHP_EOL;

$o1 = (object) ["i"=>5];
$o2 = (object) ["i"=>7];
echo sprintf("%s > %s: ", json_encode($o1), json_encode($o2)) . booleanAsString($o1 > $o2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($o1), json_encode($o2)) . booleanAsString($o1 < $o2) . PHP_EOL;

$a1 = ["i"=>5];
$a2 = ["i"=>7];
echo sprintf("%s > %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 > $a2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 < $a2) . PHP_EOL;


$a1 = [5];
$a2 = [7];
echo sprintf("%s > %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 > $a2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 < $a2) . PHP_EOL;

$a1 = [5=>5];
$a2 = [7=>7];
echo sprintf("%s > %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 > $a2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 < $a2) . PHP_EOL;

$a1 = [1 => 1, 2 => 0, 3 => 1];
$a2 = [1 => 1, 3 => 0, 2 => 1];
echo sprintf("%s > %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 > $a2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 < $a2) . PHP_EOL;

$a1 = [1 => 1, 2 => 0, 3 => 1];
$a2 = [1 => 1, 3 => 1, 2 => 0];
echo sprintf("%s > %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 > $a2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 < $a2) . PHP_EOL;


function booleanAsString($expression){
    return $expression ? "true" : "false";
}

I don't think any of that lot needs explaining: it's straight forward enough. There's a few baseline control expressions which apply the operator to numeric operands, which is clearly gonna have predictable results. Then I check with strings (including case differences), booleans, and then arrays (indexed and associative). The output:

5 > 7: false
5 < 7: true
5.5 > 7.7: false
5.5 < 7.7: true
a > z: false
a < z: true
A > a: false
A < a: true
true > false: true
true < false: false
{"i":5} > {"i":7}: false
{"i":5} < {"i":7}: true
{"i":5} > {"i":7}: false
{"i":5} < {"i":7}: true
[5] > [7]: false
[5] < [7]: true
{"5":5} > {"7":7}: false
{"5":5} < {"7":7}: false
{"1":1,"2":0,"3":1} > {"1":1,"3":0,"2":1}: true
{"1":1,"2":0,"3":1} < {"1":1,"3":0,"2":1}: true
{"1":1,"2":0,"3":1} > {"1":1,"3":1,"2":0}: false
{"1":1,"2":0,"3":1} < {"1":1,"3":1,"2":0}: false

Things work reasonably sensibly until we get to associative arrays, at which point PHP starts spouting nonsense.

I had a look around for something in the docs saying "don't do this", but what I did find kinda indicates there might indeed be a problem here (at least in the docs, if not in PHP's behaviour).

I've extracted this from "Comparison with Various Types":


Array with fewer members is smaller, if key from operand 1 is not found in operand 2 then arrays are uncomparable, otherwise - compare value by value (see following example)
(obviously that's my emphasis)

But this isn't so. See this example:

$a1 = ["a" => 1, "b" => 0, "c" => 1];
$a2 = ["a" => 1, "c" => 0, "b" => 1];
echo sprintf("%s > %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 > $a2) . PHP_EOL;
echo sprintf("%s < %s: ", json_encode($a1), json_encode($a2)) . booleanAsString($a1 < $a2) . PHP_EOL;

All the keys are present in both: "a", "b", "c", but the comparison doesn't work. I suspect it's the case of  poor wording in the docs, and it's down to whether the keys are in the same order too.

Also I'm not sure that suggesting they're "uncomparable" is helpful wording here. To me that indicates one would expect an exception if one tried. It should say "then the result is unreliable, and the operation should not be used".

I also note there's a big, highlighted warning about comparing floats:



Perhaps the same treatment should be given to array comparisons. If it tripped-up Mr Lisachenko - who seems to be pretty bloody good & experienced PHP dev - then it's gonna trip-up a lot of people.

Bottom line I agree that the handling here is "unfortunate", but it is pretty predictable given how PHP tends to handle not-obvious things, so one kinda oughta expect it. Equally it could be remediated by a documentation improvement, and I don't think it's worth anyone putting any effort into to "fixing" it.

Righto.

--
Adam

Friday 25 November 2016

PHP: decorating async GuzzleHttp calls - more methods

G'day:
Here's yet another article about decorating an async GuzzleHttp adapter. So far in the series I've got this lot:
To date my adapter has only had the one method to decorate: the get method. I also need to add more methods like post and put, which have their own considerations when doing decoration. Let's have a look.

Firstly I need some end points to test against. I decided I'd better shuffle-on from those CFML based ones I had, and use PHP instead. So I've rejigged them all as follows:

data.php

class Person{
    public $id;
    public $firstName;
    public $lastName;

    function __construct($id, $firstName, $lastName){
        $this->id = $id;
        $this->firstName = $firstName;
        $this->lastName = $lastName;
    }
}

$people = [
    new Person(1, "Donald", "Duck"),
    new Person(2, "Donald", "Cameron"),
    new Person(3, "Donald", "Trump"),
    new Person(4, "Donald", "Wearsyatroosas")
];

getById.php

$id = $_GET["id"];

require __DIR__ . "/data.php";

$filteredPeople = array_filter($people, function($person) use ($id) {
    return $person->id == $id;
});
$person = array_shift($filteredPeople);

$result = (array) $person;

sleep(5);
$result["retrieved"] = (new DateTime())->format("Y-m-d H:i:s");

header("type:application/json");
echo json_encode($result);

create.php

require __DIR__ . "/data.php";

$newPerson = new Person(
    uniqid(),
    $_POST['firstName'],
    $_POST['lastName']
);

$result = (array) $newPerson;

$result["retrieved"] = (new DateTime())->format("Y-m-d H:i:s");

header("type:application/json");
echo json_encode($result);

update.php

$id = $_GET["id"];
parse_str(file_get_contents("php://input"),$put_vars);

require __DIR__ . "/data.php";

$filteredPeople = array_filter($people, function($person) use ($id) {
    return $person->id == $id;
});

if (empty($filteredPeople)){
    http_response_code(400);
    throw new InvalidArgumentException("No Person with ID $id found");
}

$person = array_shift($filteredPeople);

$newPerson = new Person(
    $person->id,
    $put_vars['firstName'],
    $put_vars['lastName']
);

$result = [
    'before' => (array) $person,
    'after' => (array) $newPerson,
    'updated' => (new DateTime())->format("Y-m-d H:i:s")
];

header("type:application/json");
echo json_encode($result);

That lot is all pretty perfunctory. There was a coupla PHP things I didn't know about before hand, namely:
  • how to extract the body for a put request. I was initially expecting them to be in the $_POST array for some reason, but that makes no sense.
  • how to set the HTTP status. I'd never had to do that before with PHP (we use Silex / Symfony, and we just use API methods to do same).

(readers need to remember I'm fairly new to PHP ;-)

The next update is to the Adapter interface:

namespace me\adamcameron\testApp\adapter;

use GuzzleHttp\Promise\Promise;

interface Adapter {
    public function get($url, $parameters) : Promise;
    public function post($url, $body) : Promise;
    public function put($url, $body, $parameters) : Promise;
}

I've added the two new methods there. This now breaks all my Adapter implementations.

First I need to add the actual method implementations to the base adapter: the ones that actually makes the calls:

namespace me\adamcameron\testApp\adapter;

use GuzzleHttp\Client;
use GuzzleHttp\Promise\Promise;

class GuzzleAdapter implements Adapter {

    private $client;

    public function __construct() {
        $this->client = new Client();
    }

    public function get($url, $parameters) : Promise {
        $fullUrl = sprintf("%s?%s", $url, http_build_query($parameters));

        $response = $this->client->requestAsync("get", $fullUrl);

        return $response;
    }

    public function post($url, $body) : Promise {
        $options = ['form_params' => $body];

        $response = $this->client->requestAsync("post", $url, $options);

        return $response;
    }

    public function put($url, $body, $parameters) : Promise {
        $fullUrl = sprintf("%s?%s", $url, http_build_query($parameters));
        $options = ['form_params' => $body];

        $response = $this->client->requestAsync("put", $fullUrl, $options);

        return $response;
    }
}

Nothing surprising there. Note that each of the new methods have different argument requirements. This becomes relevant later.

Now I need to add equivalent methods to the decorators. I'll start with the caching one as that's easy: we have no caching requirements for POST or PUT requests, so the methods just pass through to the underlying adapter:

public function post($url, $body) : Promise {
    return $this->adapter->post($url, $body);
}

public function put($url, $body, $parameters) : Promise {
    return $this->adapter->put($url, $body, $parameters);
}


See: easy.

The logging adapter was more of a challenge. The first pass of the implementation looked like this sort of thing:

public function get($url, $parameters) : Promise {
    $encodedParameters = json_encode($parameters);
    $this->logger->logMessage(sprintf("%s: Requesting for %s", $this->thisFile, $encodedParameters));

    $response = $this->adapter->get($url, $parameters);

    $response->then(function($response) use ($encodedParameters) {
        $body = $response->getBody();
        $this->logger->logMessage(sprintf("%s: Response for %s: %s", $this->thisFile, $encodedParameters, $body));
        $body->rewind();
    });

    return $response;
}

public function post($url, $body) : Promise {
    $logDetails = json_encode($body);
    $this->logger->logMessage(sprintf("%s: Requesting for %s", $this->thisFile, $logDetails));

    $response = $this->adapter->post($url, $body);

    $response->then(function($response) use ($logDetails) {
        $body = $response->getBody();
        $this->logger->logMessage(sprintf("%s: Response for %s: %s", $this->thisFile, $logDetails, $body));
        $body->rewind();
    });

    return $response;
}

public function post($url, $body, $parameters) : Promise {
    $logDetails = json_encode([
        'parameters' => $parameters,
        'body' => $body
    ]);
    $this->logger->logMessage(sprintf("%s: Requesting for %s", $this->thisFile, $logDetails));

    $response = $this->adapter->put($url, $body, $parameters);

    $response->then(function($response) use ($logDetails) {
        $body = $response->getBody();
        $this->logger->logMessage(sprintf("%s: Response for %s: %s", $this->thisFile, $logDetails, $body));
        $body->rewind();
    });

    return $response;
}

That's fine, but look at how much similarity there is in there:
  • We log some stuff
  • We peform a request to a method
  • We have a resolution handler
What we log is different for all three methods, but the handling after that is the same. So I figured there was a refactoring opportunity there. One slight challenge is that which I mentioned above: each method passes different arguments. That'll be tricky to refactor into one method. Well so I thought. Enter PHP's variadic functions. These allow me to specify a method signature that has a concrete argument list, then one last argument that captures any other arguments passed into the function when it's called.

This way I have three congruent parts:
  • capture some stuff to log. The composition of what to log is method-specific, but that's fine
  • call a method
  • with some arguments

I was able to factor-out this commonality into a helper method:

private function performLoggedRequest($method, $logDetails, ...$requestArgs) : Promise {
    $this->logger->logMessage(sprintf("%s: Requesting for %s", $this->thisFile,  $logDetails));

    $response = call_user_func_array([$this->adapter, $method], $requestArgs);

    $response->then(function($response) use ($logDetails) {
        $body = $response->getBody();
        $this->logger->logMessage(sprintf("%s: Response for %s: %s", $this->thisFile, $logDetails, $body));
        $body->rewind();
    });

    return $response;
}

From there, it's just a matter of refactoring all three public methods to call this helper method:

public function get($url, $parameters) : Promise {
    $logDetails = json_encode($parameters);

    return $this->performLoggedRequest(__FUNCTION__, $logDetails, $url, $parameters);
}

public function post($url, $body) : Promise {
    $logDetails = json_encode($body);

    return $this->performLoggedRequest(__FUNCTION__, $logDetails, $url, $body);
}

public function put($url, $body, $parameters) : Promise {
    $logDetails = json_encode([
        'parameters' => $parameters,
        'body' => $body
    ]);

    return $this->performLoggedRequest(__FUNCTION__, $logDetails, $url, $body, $parameters);
}

For each call to performLoggedRequest, all the passed-in arguments after $logDetails end up in a single array, as $requestArgs All because of the ... in the method signature. Cool.

This enables use to call call_user_func_array. with both a dynamic method name, and a dynamic number of arguments in that array.

I then needed to do much the same with the StatusToExceptionAdapter:

public function get($url, $parameters) : Promise {
    return $this->request(__FUNCTION__, $url, $parameters);
}

public function post($url, $body) : Promise {
    return $this->request(__FUNCTION__, $url, $body);
}

public function put($url, $parameters, $body) : Promise {
    return $this->request(__FUNCTION__, $url, $parameters, $body);
}

private function request($method, ...$args) : Promise {
    return call_user_func_array([$this->adapter, $method], $args)
        ->then(function (Response $response) {
            return $this->handleThen($response);
        })
        ->otherwise(function ($exception) {
            $this->handleOtherwise($exception);
        });
}

Easy!

Finally a test. I used the same basic test rig as last time, so I'll just focus on the individual tests here. I also ran the equivalent test of getById as before: no change there so I'll not bother reproducing it.

The test for post was as follows:

function testCreate(PersonRepository $personRepository, LoggingService $loggingService) {
    $personDetails = [
        'firstName' => 'Donald',
        'lastName' => 'McLean'
    ];

    $loggingService->logMessage(sprintf("Test: calling create(%s)", json_encode($personDetails)));

    $response = $personRepository->create($personDetails);
    $body = (string) $response->wait()->getBody();

    $loggingService->logMessage(sprintf("Test: called create(): [%s]", $body));
}

That's all obvious. The log output for this was:

[2016-11-24 20:04:11] testPost.INFO: Test: calling create({"firstName":"Donald","lastName":"McLean"}) [] []
[2016-11-24 20:04:11] testPost.INFO: LoggingAdapter: Requesting for {"firstName":"Donald","lastName":"McLean"} [] []
[2016-11-24 20:04:11] testPost.INFO: LoggingAdapter: Response for {"firstName":"Donald","lastName":"McLean"}: {"id":"583747bb1a6d9","firstName":"Donald","lastName":"McLean","retrieved":"2016-11-24 20:04:11"} [] []
[2016-11-24 20:04:11] testPost.INFO: StatusToExceptionAdapter: non-exception status encountered: 200 [] []
[2016-11-24 20:04:11] testPost.INFO: Test: called create(): [{"id":"583747bb1a6d9","firstName":"Donald","lastName":"McLean","retrieved":"2016-11-24 20:04:11"}] [] []

We can see no caching was called (good), but the approrpiate logging and StatusToException checks were made. And we got a new record ID back after the insert. Excellent.

The test of the update had two things to do: doing an update of an existing record (happy path) then trying to update a non-existent record (which should yield an exception):

function testUpdate(PersonRepository $personRepository, LoggingService $loggingService) {
    testUpdateByPhase("Valid ID", 3, $personRepository, $loggingService);
    testUpdateByPhase("Invalid ID", -1, $personRepository, $loggingService);
}

function testUpdateByPhase($phase, $id, PersonRepository $personRepository, LoggingService $loggingService) {
    $personDetails = [
        'firstName' => 'Donald',
        'lastName' => 'Corleone'
    ];

    $logDetails = [
        'id' => $id,
        'details' => $personDetails
    ];

    $loggingService->logMessage(sprintf("Test (phase: %s): calling update(%s)", $phase, json_encode($logDetails)));

    $response = $personRepository->update($id, $personDetails);
    try {
        $body = (string) $response->wait()->getBody();
        $loggingService->logMessage(sprintf("Test (phase %s): called update(): [%s]", $phase, $body));
    } catch (Exception $e) {
        $previous = $e->getPrevious();
        $loggingService->logMessage(
            sprintf(
                "Test (phase %s): call to update failed: "
                . "Class: %s; "
                . "Code: %s; "
                . "Message: %s"
                . "Previous class: %s"
                . "Previous message: %s"
                , $phase, get_class($e), $e->getCode(), $e->getMessage()
                , get_class($previous), $previous->getMessage()
            )
        );
    } finally {
        $loggingService->logMessage(sprintf("Test (phase %s): complete", $phase));
    }
}

That's more code, but most of it is error handling, and there's nothing noteworthy. The log shows the outcome. First the valid update:

[2016-11-24 20:04:11] testPost.INFO: Test (phase: Valid ID): calling update({"id":3,"details":{"firstName":"Donald","lastName":"Corleone"}}) [] []
[2016-11-24 20:04:11] testPost.INFO: LoggingAdapter: Requesting for {"parameters":{"id":3},"body":{"firstName":"Donald","lastName":"Corleone"}} [] []
[2016-11-24 20:04:11] testPost.INFO: LoggingAdapter: Response for {"parameters":{"id":3},"body":{"firstName":"Donald","lastName":"Corleone"}}: {"before":{"id":3,"firstName":"Donald","lastName":"Trump"},"after":{"id":3,"firstName":"Donald","lastName":"Corleone"},"updated":"2016-11-24 20:04:11"} [] []
[2016-11-24 20:04:11] testPost.INFO: StatusToExceptionAdapter: non-exception status encountered: 200 [] []
[2016-11-24 20:04:11] testPost.INFO: Test (phase Valid ID): called update(): [{"before":{"id":3,"firstName":"Donald","lastName":"Trump"},"after":{"id":3,"firstName":"Donald","lastName":"Corleone"},"updated":"2016-11-24 20:04:11"}] [] []
[2016-11-24 20:04:11] testPost.INFO: Test (phase Valid ID): complete [] []

Success!

And the invalid ID:

[2016-11-24 20:04:11] testPost.INFO: LoggingAdapter: Requesting for {"parameters":{"id":-1},"body":{"firstName":"Donald","lastName":"Corleone"}} [] []
[2016-11-24 20:04:11] testPost.INFO: StatusToExceptionAdapter: remapped status encountered: 400 [] []
[2016-11-24 20:04:11] testPost.INFO: Test (phase Invalid ID): call to update failed: Class: me\adamcameron\testApp\exception\BadRequestException; Code: 400; Message: A response of 400 was received from the service Previous class: GuzzleHttp\Exception\ClientExceptionPrevious message: Client error: `PUT http://php.local:8070/experiment/guzzle/update.php?id=-1` resulted in a `400 Bad Request` response: <br /> <font size='1'><table class='xdebug-error xe-uncaught-exception' dir='ltr' border='1' cellspacing='0' cellpadding (truncated...)  [] []
[2016-11-24 20:04:11] testPost.INFO: Test (phase Invalid ID): complete [] []


That worked as expected too. Although I now note I should have returned a 404 there, not a 400. Oops. But it's doing what I told it to do, anyhow. And pleasingly, the StatusToExceptionAdapter did its job.

And that's about it. I learned a few new things whilst investigating this one, which is good.

That's the end of my foray into decorating GuzzleHttp adapters. I'll get onto something different next.

Righto.

--
Adam

Wednesday 23 November 2016

PHP: decorating async GuzzleHttp calls - multiple decorators

G'day:
Here's yet another article about decorating an async GuzzleHttp adapter. So far in the series I've got this lot:

Over the course of those articles, I've created three decorators:

  • logging - wraps the async call in "before" and "after" logging;
  • exception handling - remapping GuzzleHttp's default exceptions to HTTP-response-status-specific exceptions;
  • caching - caching successful responses so future calls for the same info won't need to hit the underlying  web service.

These all mirror real-world requirements I have.

I've tested them all separately, but the theory with decoration is that one should be able to further-decorate an already-decorated object. I've decided to give this a go.

Firstly I've revised my approach somewhat. Previously I was baking my endpoint into the adapter as that was convenient for how i was demonstrating the code. This isn't a good approach as the adapter should just focus on making the calls and handling them, not having a position on the destination of the calls. So I've removed the endpoint from the basic adapter:

use GuzzleHttp\Client;
use GuzzleHttp\Promise\Promise;

class GuzzleAdapter implements Adapter {

    private $client;

    public function __construct() {
        $this->client = new Client();
    }

    public function get($url, $parameters) : Promise {
        $fullUrl = sprintf("%s?%s", $url, http_build_query($parameters));

        $response = $this->client->requestAsync("get", $fullUrl);

        return $response;
    }
}

Now I pass the URL into the call to get, and append the passed-in parameters to make a complete URL. In real life I'd probably also be optionally passing in some headers, and perhaps some GuzzleHttp options, but that's just noise for the purposes of what I'm doing here.

Also note I've gone all PHP7 for a change, and added in some stricter typing. To that end, I'm also enforcing an interface on these classes now:

use GuzzleHttp\Promise\Promise;

interface Adapter {
    public function get($url, $parameters) : Promise;
}

In dynamic languages like PHP and CFML I generally try to shy away from being to forceful about type checking as it doesn't really add much, and sometimes gets in the way. But I think in this situation it makes some sense. And I also want to start getting into the swing of things of writing PHP 7 code too.

I've also changed my approach to calling the adapter somewhat. Now that the endpoint has been moved out of the adapter, I need it to go somewhere else. I don't want to be passing it in with every get call. We use the Repository Pattern at work to wrap up details of external service implementation, so I've run with that here.

use GuzzleHttp\Promise\Promise;
use me\adamcameron\testApp\adapter\Adapter;

class PersonRepository {

    private $adapter;

    private static $baseUrl = "http://cf2016.local:8516/cfml/misc/guzzleTestEndpoints/";
    private static $getByIdUrl = "getById.cfm";
    private static $getByNameUrl = "getByName.cfm";
    private static $returnStatusCodeUrl = "returnStatusCode.cfm";

    public function __construct(Adapter $adapter)
    {
        $this->adapter = $adapter;
    }

    public function getById($id) : Promise {
        $response = $this->adapter->get(
            self::$baseUrl . self::$getByIdUrl,
            ["id" => $id]
        );

        return $response;
    }

    public function getByName($firstName, $lastName) : Promise {
        $response = $this->adapter->get(
            self::$baseUrl . self::$getByNameUrl,
            [
                "firstName" => $firstName,
                "lastName" => $lastName
            ]
        );

        return $response;
    }

    public function returnStatusCode($statusCode) : Promise {
        $response = $this->adapter->get(
            self::$baseUrl . self::$returnStatusCodeUrl,
            ["statusCode" => $statusCode]
        );

        return $response;
    }
}

Notes on this one:

  • The repo knows about the endpoints it calls. Usually we'd probably have the base URL as a constructor param so we can easily change version of the web service, but that's just noise here, so I've "hard-coded" it.
  • I've added a new method - getByName - that I've not used before. I just wanted to have a second method which had a different method signature from the others I'd previously used. This was mostly for an experiment that I was gonna do but decided against for this article, but left the method in anyhow.
For the sake of completeness, here's the revised endpoint code:

getById.cfm

<cfscript>
cfcontent(type="application/json");

writeLog(file="testApp", text="[ID: #URL.id#] request received");

sleep(5000);
include "./data.cfm";

record = queryExecute(
    "SELECT id, firstName, lastName FROM people WHERE id = :id",
    {id=URL.id},
    {dbtype="query", query="people", maxrows=1}
);

result = structNew("ordered");
result["id"] = record.id;
result["firstName"] = record.firstName;
result["lastName"] = record.lastName;
result["retrieved"] = now().dateTimeFormat("HH:nn:ss.lll");

writeOutput(serializeJson(result));
writeLog(file="testApp", text="[ID: #URL.id#] response returned");
</cfscript>


getByName.cfm

<cfscript>
cfcontent(type="application/json");

writeLog(file="testApp", text="[firstName: #URL.firstName#][lastName: #URL.lastName#] request received");

sleep(2500);
include "./data.cfm";

record = queryExecute(
    "SELECT id, firstName, lastName FROM people WHERE firstName = :firstName AND lastName = :lastName",
    {firstName=URL.firstName, lastName=URL.lastName},
    {dbtype="query", query="people", maxrows=1}
);

result = structNew("ordered");
result["id"] = record.id;
result["firstName"] = record.firstName;
result["lastName"] = record.lastName;
result["retrieved"] = now().dateTimeFormat("HH:nn:ss.lll");

sleep(2500);

writeOutput(serializeJson(result));
writeLog(file="testApp", text="[firstName: #URL.firstName#][lastName: #URL.lastName#] response returned");
</cfscript>

data.cfm

<cfscript>
people = queryNew("id,firstName,lastName", "integer,varchar,varchar", [
    {id=1, firstName="Donald", lastName="Duck"},
    {id=2, firstName="Donald", lastName="Cameron"},
    {id=3, firstName="Donald", lastName="Trump"},
    {id=4, firstName="Donald", lastName="Wearsyatroosas"}
]);
</cfscript>

returnStatusCode.cfm

Remains unchanged:

<cfset sleep(2000)>
<cfheader statusCode="#URL.statusCode#">
<cfoutput>The request resulted in a #URL.statusCode# status code @ #now().dateTimeFormat("HH:nn:ss.lll")#</cfoutput>

The only thing to note really is that these endpoints are kinda data-driven now.

And I have a test of all this. It's long-winded, so I'll go through it section by section, before presenting it again in its entirety.

Firstly - and the only really important bit for the purposes of this article - is setting our test objects:

$loggingService = getLoggingService();
$personRepository = getPersonRepository($loggingService);

// ...


function getLoggingService() : LoggingService {
    $ts = (new DateTime())->format("His");
    $loggingService = new LoggingService("all_$ts");

    return $loggingService;
}

function getPersonRepository(LoggingService $loggingService) : PersonRepository {
    $guzzleAdapter = getGuzzleAdapter();
    $loggingAdapter = getLoggingAdapter($guzzleAdapter, $loggingService);
    $statusToExceptionAdapter = getStatusToExceptionAdapter($loggingAdapter, $loggingService);
    $cachingAdapter = getCachingAdapter($statusToExceptionAdapter, $loggingService);

    $personRepository = new PersonRepository($cachingAdapter);

    return $personRepository;
}

function getGuzzleAdapter() : GuzzleAdapter {
    $guzzleAdapter = new GuzzleAdapter();

    return $guzzleAdapter;
}

function getLoggingAdapter(Adapter $adapter, LoggingService $loggingService) : LoggingAdapter {
    $loggingAdapter = new LoggingAdapter($adapter, $loggingService);

    return $loggingAdapter;
}

function getStatusToExceptionAdapter(Adapter $adapter, LoggingService $loggingService) : StatusToExceptionAdapter {
    $exceptionMap = [
        400 => '\me\adamcameron\testApp\exception\BadRequestException',
        503 => '\me\adamcameron\testApp\exception\ServerException'
    ];

    $statusToExceptionAdapter = new StatusToExceptionAdapter($adapter, $exceptionMap, $loggingService);

    return $statusToExceptionAdapter;
}

function getCachingAdapter(Adapter $adapter, LoggingService $loggingService) : CachingAdapter {
    $cachingService = new CachingService($loggingService);
    $cachingAdapter = new CachingAdapter($adapter, $cachingService);

    return $cachingAdapter;
}

That looks like a chunk of code (well: it is), but it's mostly just compartmentalising some calls. Notes:

  • we need a LoggingService and a PersonRepository for our tests.
  • The PersonRepository needs an Adapter.
  • For our purposes we are decorating the baseline GuzzleAdapter with:
    • a LoggingAdapter. We only want to log calls made to the service, so we put this closest to the baseline Adapter.
    • a StatusToExceptionAdapter. We only want to cache successful calls, and intrinsically previously successful calls will not need the logic in this adapter, so put this next.
    • a CachingAdapter. This goes closest to the calling code so that there's a minimum of complexity if a successful call has been previously made.

The order that the decorations are layers is important, as hopefully I've explained above. The ordering needs to be considered on a case-by-case basis, determined by the purpose of each decorator, and how one decorator might interrupt the execution of other ones; eg: if the StatusToExceptionAdapter throws an exception, the CachingAdapter never does its work. And depending on what those two do, the GuzzleAdapter itself might not be needed, so the LoggingAdapter is skipped too.

From there I test each of the three endpoints again, in much the same way I did before, but noting how the decorators combine where appropriate.

makeRequestsToGetById($personRepository, $loggingService);
logTestSeparator($loggingService);

makeRequestsToGetByName($personRepository, $loggingService);
logTestSeparator($loggingService);

makeRequestsToReturnStatusCode($personRepository, $loggingService);
logTestSeparator($loggingService);

Testing getById I just check the caching still works:

function makeRequestsToGetById(PersonRepository $personRepository, LoggingService $loggingService){
    $id = 4;

    makeRequestToGetById($id, "Uncached", $personRepository, $loggingService);
    pause(5, $loggingService);
    makeRequestToGetById($id, "Cached", $personRepository, $loggingService);
}

function makeRequestToGetById($id, $phase, PersonRepository $personRepository, LoggingService $loggingService){
    $startTime = time();

    $loggingService->logMessage("Test (phase: $phase): calling getById($id)");
    $response = $personRepository->getById($id);
    $loggingService->logMessage("Test (phase: $phase): call to getById complete");
    $body = (string) $response->wait()->getBody();
    $loggingService->logMessage("Test (phase: $phase): Body: $body");

    $duration = time() - $startTime;
    $loggingService->logMessage("Test (phase: $phase): Process duration: {$duration}sec");
}


Sample output:

[2016-11-22 20:39:21] all_203921.INFO: Test: Getting not-yet cached results [] []
[2016-11-22 20:39:21] all_203921.INFO: Test (phase: Uncached): calling getById(4) [] []
[2016-11-22 20:39:21] all_203921.INFO: LoggingAdapter: Requesting for {"id":4} [] []
[2016-11-22 20:39:21] all_203921.INFO: Test (phase: Uncached): call to getById complete [] []
[2016-11-22 20:39:26] all_203921.INFO: LoggingAdapter: Response for {"id":4}: {"id":4,"firstName":"Donald","lastName":"Wearsyatroosas","retrieved":"20:39:26.161"} [] []
[2016-11-22 20:39:26] all_203921.INFO: StatusToExceptionAdapter: non-exception status encountered: 200 [] []
[2016-11-22 20:39:26] all_203921.INFO: CachingService: put called with cd51039fde58b80cf261c3f1a99da3ef [] []
[2016-11-22 20:39:26] all_203921.INFO: Test (phase: Uncached): Body: {"id":4,"firstName":"Donald","lastName":"Wearsyatroosas","retrieved":"20:39:26.161"} [] []
[2016-11-22 20:39:26] all_203921.INFO: Test (phase: Uncached): Process duration: 5sec [] []
[2016-11-22 20:39:26] all_203921.INFO: Test: waiting 5sec [] []
[2016-11-22 20:39:31] all_203921.INFO: Test: waited 5sec [] []
[2016-11-22 20:39:31] all_203921.INFO: Test (phase: Cached): calling getById(4) [] []
no log entry here, because the LoggingAdapter was never hit: the CachingAdapter took care of it
[2016-11-22 20:39:31] all_203921.INFO: Test (phase: Cached): call to getById complete [] []
[2016-11-22 20:39:31] all_203921.INFO: CachingService: get called with cd51039fde58b80cf261c3f1a99da3ef [] []
[2016-11-22 20:39:31] all_203921.INFO: Test (phase: Cached): Body: {"id":4,"firstName":"Donald","lastName":"Wearsyatroosas","retrieved":"20:39:26.161"} [] []
[2016-11-22 20:39:31] all_203921.INFO: Test (phase: Cached): Process duration: 0sec [] []

Also note this demonstrates the LoggingAdapter at work too. Specifically note that there's only a log entry for the request when the web service is actually being called, not when it's coming from cache, further down.

Next I test getByName. It's exactly the same test as the getById one, so I won't bore you with it. It demonstrates the same thing.

Lastly I test returnStatusCode:

function makeRequestsToReturnStatusCode(PersonRepository $personRepository, LoggingService $loggingService) {
    makeRequestToReturnStatusCode200("First", $personRepository, $loggingService);
    pause(2, $loggingService);
    makeRequestToReturnStatusCode200("Second", $personRepository, $loggingService);
    logTestSeparator($loggingService);
    makeRequestToMappedStatusCode("First", $personRepository, $loggingService);
    $loggingService->logMessage("Test: Repeating an erroring call will not touch cache");
    makeRequestToMappedStatusCode("Second", $personRepository, $loggingService);
}

The test here is twofold. First I test that requests that are not intercepted and don't error are logged and cached correctly:

function makeRequestToReturnStatusCode200($phase, PersonRepository $personRepository, LoggingService $loggingService){
    $statusCode = 200;
    $loggingService->logMessage("Test (phase: $phase): calling returnStatusCode($statusCode)");
    $response = $personRepository->returnStatusCode($statusCode);
    $loggingService->logMessage("Test (phase: $phase): call to returnStatusCode complete");
    $body = (string) $response->wait()->getBody();
    $loggingService->logMessage("Test (phase: $phase): Body: $body");
}

Sample log output:

[2016-11-22 20:39:41] all_203921.INFO: Test (phase: First): calling returnStatusCode(200) [] []
[2016-11-22 20:39:41] all_203921.INFO: LoggingAdapter: Requesting for {"statusCode":200} [] []
[2016-11-22 20:39:41] all_203921.INFO: Test (phase: First): call to returnStatusCode complete [] []
[2016-11-22 20:39:43] all_203921.INFO: LoggingAdapter: Response for {"statusCode":200}: The request resulted in a 200 status code @ 20:39:43.319 [] []
[2016-11-22 20:39:43] all_203921.INFO: StatusToExceptionAdapter: non-exception status encountered: 200 [] []
[2016-11-22 20:39:43] all_203921.INFO: CachingService: put called with 8670b5e29a709e36ae186bce7715de68 [] []
[2016-11-22 20:39:43] all_203921.INFO: Test (phase: First): Body: The request resulted in a 200 status code @ 20:39:43.319 [] []
[2016-11-22 20:39:43] all_203921.INFO: Test: waiting 2sec [] []
[2016-11-22 20:39:45] all_203921.INFO: Test: waited 2sec [] []
[2016-11-22 20:39:45] all_203921.INFO: Test (phase: Second): calling returnStatusCode(200) [] []
[2016-11-22 20:39:45] all_203921.INFO: Test (phase: Second): call to returnStatusCode complete [] []
[2016-11-22 20:39:45] all_203921.INFO: CachingService: get called with 8670b5e29a709e36ae186bce7715de68 [] []
[2016-11-22 20:39:45] all_203921.INFO: Test (phase: Second): Body: The request resulted in a 200 status code @ 20:39:43.319 [] []

The second test of returnStatusCode test a remapped status to make sure it works to spec:

function makeRequestToMappedStatusCode($phase, PersonRepository $personRepository, LoggingService $loggingService) {
    $statusCode = 400;
    $loggingService->logMessage("Test (phase: $phase): calling returnStatusCode($statusCode)");
    $response = $personRepository->returnStatusCode($statusCode);
    $loggingService->logMessage("Test (phase: $phase): call to returnStatusCode complete");
    try {
        $response->wait();
        $loggingService->logMessage("Test (phase: $phase): failed to throw expected exception");
    } catch (Exception $e){
        logException($e, $loggingService);
    }
}

Sample logging:

[2016-11-22 20:39:45] all_203921.INFO: Test (phase: First): calling returnStatusCode(400) [] []
[2016-11-22 20:39:45] all_203921.INFO: LoggingAdapter: Requesting for {"statusCode":400} [] []
[2016-11-22 20:39:45] all_203921.INFO: Test (phase: First): call to returnStatusCode complete [] []
[2016-11-22 20:39:47] all_203921.INFO: StatusToExceptionAdapter: remapped status encountered: 400 [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Request errored with 400  [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Class: me\adamcameron\testApp\exception\BadRequestException  [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Message A response of 400 was received from the service  [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Previous class GuzzleHttp\Exception\ClientException  [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Previous code 400  [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Previous message Client error: `GET http://cf2016.local:8516/cfml/misc/guzzleTestEndpoints/returnStatusCode.cfm?statusCode=400` resulted in a `400 Bad Request` response: The request resulted in a 400 status code @ 20:39:47.359   [] []
[2016-11-22 20:39:47] all_203921.INFO: Test: Repeating an erroring call will not touch cache [] []
[2016-11-22 20:39:47] all_203921.INFO: Test (phase: Second): calling returnStatusCode(400) [] []
[2016-11-22 20:39:47] all_203921.INFO: LoggingAdapter: Requesting for {"statusCode":400} [] []
[2016-11-22 20:39:47] all_203921.INFO: Test (phase: Second): call to returnStatusCode complete [] []
[2016-11-22 20:39:49] all_203921.INFO: StatusToExceptionAdapter: remapped status encountered: 400 [] []
[2016-11-22 20:39:49] all_203921.INFO: Test: Request errored with 400  [] []
[2016-11-22 20:39:49] all_203921.INFO: Test: Class: me\adamcameron\testApp\exception\BadRequestException  [] []
[2016-11-22 20:39:49] all_203921.INFO: Test: Message A response of 400 was received from the service  [] []
[2016-11-22 20:39:49] all_203921.INFO: Test: Previous class GuzzleHttp\Exception\ClientException  [] []
[2016-11-22 20:39:49] all_203921.INFO: Test: Previous code 400  [] []
[2016-11-22 20:39:49] all_203921.INFO: Test: Previous message Client error: `GET http://cf2016.local:8516/cfml/misc/guzzleTestEndpoints/returnStatusCode.cfm?statusCode=400` resulted in a `400 Bad Request` response: The request resulted in a 400 status code @ 20:39:49.392   [] []

So:
  • The status code is converted to the correct exception.
  • The initial log entry is made for the request, but not the one for the response (see the log extract from the getById tests for that one.
  • Nothing is cached or attempted to be fetched from cache at any point because the appropriate handlers are never invoked.

That'sit really. I'm happy with that.

Oh, I use a coupla other helper functions in that lot. For completeness here they are:

function logException(Exception $e, LoggingService $loggingService) {
    $previous = $e->getPrevious();
    $loggingService->logMessage(sprintf("Test: Request errored with %s%s", $e->getCode(), PHP_EOL));
    $loggingService->logMessage(sprintf("Test: Class: %s%s", get_class($e), PHP_EOL));
    $loggingService->logMessage(sprintf("Test: Message %s%s", $e->getMessage(), PHP_EOL));
    $loggingService->logMessage(sprintf("Test: Previous class %s%s", get_class($previous), PHP_EOL));
    $loggingService->logMessage(sprintf("Test: Previous code %s%s", $previous->getCode(), PHP_EOL));
    $loggingService->logMessage(sprintf("Test: Previous message %s%s", $previous->getMessage(), PHP_EOL));
}

function logTestSeparator(LoggingService $loggingService) {
    $loggingService->logMessage("==================================================================");
}

function pause($seconds, LoggingService $loggingService) {
    $loggingService->logMessage("Test: waiting {$seconds}sec");
    sleep($seconds);
    $loggingService->logMessage("Test: waited {$seconds}sec");
}

Indeed the code for everything here is up at GitHub, but I'm changing it with each article I write, so it gets out of date quickly. Here's the current commit, anyhow, and that's the code I've been using for this article.

I've got yet another article to write on this, but I haven't even worked out how I'm gonna write the code yet, so that might be a day or so still.

Righto.

--
Adam

Tuesday 22 November 2016

PHP: decorating async GuzzleHttp calls - caching

G'day:
This continues my series of exercise in decorating GuzzleHttp calls. Recap:

The last piece of the puzzle I need to deal with is caching. We need to decorate our web service adapter with a caching tier so that we don't continually pester it over the wire for info we have already asked for.

The treatment here is traditional:
  • we get a request for data  based on an ID;
  • we check the cache for data stored against that ID;
  • if we have it, we returned the cached version;
  • if not, we call the web service;
  • and before returning the result, we cache it for "next time".
I've implemented a stub caching service which simply sticks stuff in an array. This'll do for the purposes of testing:

class CachingService {

    public $cache;

    public function __construct(){
        $this->cache = [];
    }

    public function contains($id){
        return array_key_exists($id, $this->cache);
    }

    public function get($id){
        echo "Getting $id from cache" . PHP_EOL . PHP_EOL;
        return $this->cache[$id];
    }

    public function put($id, $value){
        echo "Putting $id into cache" . PHP_EOL . PHP_EOL;
        return $this->cache[$id] = $value;
    }
}

In reality we'd use redis or something like that, but it's the interface that matters here, not the implementation. Note I'm also outputing a message when the cache methods are hit, just so we can tell what's going on when the code is running.

Next I have a CachingGuzzleAdapter which is the decorator for a GuzzleAdapter. This has a chunk of code in it, so I'll break it down:

class CachingGuzzleAdapter {

    private $adapter;
    private $cache;

    public function __construct($adapter, $cache) {
        $this->adapter = $adapter;
        $this->cache = $cache;
    }

    // ...

}

No surprises here. The decorator takes the adapter it's decorating, as well as an instance of the cache service it'll be talking to. I probably shoulda called that $cachingService, now that I look at it. We're taking the same approach here as we did with the other decorators (in the earlier articles): all the guzzling is offloaded to the adapter here; this class only deals with the caching side of things.

To that end, our get method is very simple:

public function get($id){
    if ($this->cache->contains($id)) {
        return $this->returnResponseFromCache($id);
    }
    return $this->returnResponseFromWebService($id);
}

If it's in the cache: use that; if not: go get it. The details of how to get the thing from cache or how to get things from the adapter are hidden in helper functions.

Here's the tricky bit: getting the response back from cache:

private function returnResponseFromCache($id) {
    $p = new Promise(function() use (&$p, $id){
        echo "GETTING FROM CACHE" . PHP_EOL;
        $cachedResult = $this->cache->get($id);

        $newResponse = new Response(
            $cachedResult['status'],
            $cachedResult['headers'],
            $cachedResult['body']
        );

        $p->resolve($newResponse);
    });

    return $p;
}

Here we:

  • get some data from the cache. This method only gets called if the data is there, so we can dive straight in.
  • We create a new Guzzle Response object, and put the data back into it.
  • We resolve a promise with that as its value. We need to return a promise here because that's what the get method we're decorating returns.

I'm not entirely comfortable with the self-referential bit around $p, but that was from the docs as the recommended way of doing this. So be it.

If the stuff wasn't in cache, we need to make an actual web service call:

private function returnResponseFromWebService($id){
    $response = $this->adapter->get($id);

    $response->then(function(Response $response) use ($id) {
        echo "PUTTING IN CACHE" . PHP_EOL;

        $detailsToCache = [
            'status' => $response->getStatusCode(),
            'headers' => $response->getHeaders(),
            'body' => $response->getBody()->getContents()
        ];
        $this->cache->put($id, $detailsToCache);
    });

    return $response;
}

This is straight forward:

  • make the call;
  • in its resolution handler grab the data and bung it in the cache.
  • Note that I'm only caching the significant bits of the response, not the response itself. My original version of this cached the entire response object: because the cache is just an in-memory array I can do that; in reality I'd need to serialise it somehow, so I wanted to make a point of only caching the data here.

That's it!

Of course I need to test this, so I wrote a test rig, similar to the ones I've already done:

$endPoint  = "http://cf2016.local:8516/cfml/misc/guzzleTestEndpoints/getById.cfm?id=";

$guzzleAdapter = new GuzzleAdapter($endPoint);
$cache = new CachingService();
$adapter = new CachingGuzzleAdapter($guzzleAdapter, $cache);

printf("Getting not-yet cached results @ %s%s", (new DateTime())->format("H:i:s"), PHP_EOL . PHP_EOL);
makeRequests($adapter);

echo "===================================================" .PHP_EOL . PHP_EOL;

sleep(10);

printf("Getting results again (from cache) @ %s%s", (new DateTime())->format("H:i:s"), PHP_EOL . PHP_EOL);
makeRequests($adapter);


function makeRequests($adapter){
    $startTime = time();
    $ids = ["001", "002"];
    $responses = [];
    foreach ($ids as $id) {
        echo "Making requests" . PHP_EOL . PHP_EOL;
        echo "Requesting: $id" . PHP_EOL;
        $responses[] = $adapter->get($id);
        echo "Requests made" . PHP_EOL . PHP_EOL;
    }
    echo "Fetching responses" . PHP_EOL . PHP_EOL;
    foreach ($responses as $response){
        $body = (string) $response->wait()->getBody();
        echo "Body: $body" . PHP_EOL . PHP_EOL;
    }
    echo "Responses fetched" . PHP_EOL . PHP_EOL;
    $duration = time() - $startTime;
    echo "Process duration: {$duration}sec" . PHP_EOL . PHP_EOL;
}

This just:
  • calls the adapter for a coupla IDs,
  • waits 10sec
  • and gets the same data again.
The expectation is that the first call will take around 5sec as that's how long each web service call takes:

<cfscript>
cfcontent(type="application/json");

writeLog(file="testApp", text="[ID: #URL.id#] request received");

sleep(5000);

writeOutput(serializeJson({"id"=URL.id, "retrieved"=now().dateTimeFormat("HH:nn:ss.lll")}));
writeLog(file="testApp", text="[ID: #URL.id#] response returned");
</cfscript>

It'll only be 5sec in total not 10sec because the calls are being made asynchronously, remember?

The second round of calls should take no time at all because the adapter will be able to get the results from cache. In theory. Let's see:

>php testCachingAdapter.php
Getting not-yet cached results @ 09:24:25

Making requests

Requesting: 001
Requests made

Making requests

Requesting: 002
Requests made

Fetching responses

PUTTING IN CACHE
Putting 001 into cache

PUTTING IN CACHE
Putting 002 into cache

Body: {"retrieved":"09:24:30.453","id":"001"}

Body: {"retrieved":"09:24:30.457","id":"002"}

Responses fetched

Process duration: 5sec

===================================================

Getting results again (from cache) @ 09:24:40

Making requests

Requesting: 001
Requests made

Making requests

Requesting: 002
Requests made

Fetching responses

GETTING FROM CACHE
Getting 001 from cache

Body: {"retrieved":"09:24:30.453","id":"001"}

GETTING FROM CACHE
Getting 002 from cache

Body: {"retrieved":"09:24:30.457","id":"002"}

Responses fetched

Process duration: 0sec


>

Hurrah! As predicted the first call takes 5sec; the second takes 0sec. And the intermediary telemetry confirms that the cache is being used correctly. Also note the timestamps on the returned data reflect when the initial requests were made, not when the second round of calls were made.

That was pretty easy this time. It took me a while to find the code for returning a resolved promise with the Response value in it, but wiring everything up was simple.

As an exercise I'm now gonna wire up a CachedLoggedErrorMappedGuzzleAdapter. Or perhaps a LoggedErrorMappedCacheGuzzleAdapter. Not sure. But it's all a difference in wiring, and that's it really. I'll write that up next.

Righto.

--
Adam

Monday 21 November 2016

PHP: decorating async GuzzleHttp calls - handling exceptions a different way

G'day:
The previous article ("PHP: decorating async GuzzleHttp calls - handling exceptions") detailed how I've used a simple decorator to intercept 4xx & 5xx HTTP responses from my web service and remap Guzzle's default exceptions and use my own ones.  It was basically just this:

public function get($status){
    return $this->adapter
        ->get($status)
        ->otherwise(function($exception){
            $code = $exception->getCode();

            if (!array_key_exists($code, $this->exceptionMap)) {
                throw $exception;
            }
            throw new $this->exceptionMap[$code](
                "A response of $code was received from the service",
                $code,
                $exception
            );
        });
}


Notes:
  • get is a wrapper for a call to Guzzle, and it returns a GuzzleHttp\Promise
  • if the promise rejects (which it will on a 4xx or 5xx response), then I have a handler to check if it's a status that I'm interested in, and if so return (well: throw) a different exception.
  • If not, I just rethrow the original exception.
Simple enough and that works nicely. Read the article though, for more detail on this.

Along the way to that solution I ballsed something up in a way I still don't quite understand and I cannot replicate now, so abandoned that easy solution and approached things a different way. SO I've now got two ways of doing the same thing, and that seems liek a cheap win for me as far as blog content goes, so yer hearing about both of 'em ;-)

If I could not work out how to intercept Guzzle's own exception handling, I decided to replace it.

Reading through the docs I found one could switch off Guzzle's exception handling via the http_errors option. That was no use in itself (as it turns out), but looking at the matches for http_errors in the docs, I got down to the Middleware section, which detailed how the error handler (the one that pays attention to the http_errors setting) is one of the default middelware that Guzzle configures if no bespoke middleware is specified. It also details how one can set one's own middleware. Ergo: one can set one's own error handling middleware.

First things first, I took their own bespoke middleware example and modified it to handle my error remapping code (as per the previous article). As I was only really copying and pasting still into their example, I'll just present the final job:

private function handleHttpErrors($exceptionMap)
{
    return function (callable $handler) use ($exceptionMap) {
        return function ($request, array $options) use ($handler, $exceptionMap) {
            return $handler($request, $options)->then(
                function (ResponseInterface $response) use ($request, $handler, $exceptionMap) {
                    $code = $response->getStatusCode();

                    if ($code < 400) {
                        return $response;
                    }

                    if (array_key_exists($code, $exceptionMap)){
                        throw new $exceptionMap[$code](
                            (string)$response->getBody(),
                            $code
                        );
                    }
                    throw RequestException::create($request, $response);
                }
            );
        };
    };
}

Notes:
  • most of this is copy and paste from the docs and I have no idea why it is the way it is, so... erm... well there you go.
  • The rest just checks the response for an "error" situation;
  • if it's not just return it... the next middleware will continue with it.
  • If it is an exceptional situation: consult the exception map for how to handle it (which exception to throw);
  • or if there's no remapping consideration, just throw the same exception one would normally throw (this was pinched from a different part of the Guzzle source code: Middleware.php).
That's all lovely, but I still need to get it into the middleware stack. Remember this is just a decorator, and it doesn't itself have its own Guzzle client... that's hidden away in the base Guzzle Adapter. But I need to have access to its middleware stack.

Here I cheat slightly and add a method to the base GuzzleAdapter:

namespace me\adamcameron\testApp;

use GuzzleHttp\Client;

class GuzzleAdapter {

    private $client;
    private $endPoint;

    public function __construct($endPoint){
        $this->endPoint = $endPoint;
        $this->client = new Client();
    }

    public function get($id){
        $response = $this->client->requestAsync(
            "get",
            $this->endPoint . $id,
            ["http_errors" => true]
        );

        return $response;
    }

    public function getHandlerStack()
    {
        return $this->client->getConfig('handler');
    }

}

This means the ExceptionHandler decorator can call that method, and get a reference to the handler stack. One caveat with this is that all decorators now need a proxy for this method too. This kinda suggests to me I need to have a GuzzleAdapter interface which specifies which methods decorators need to implement, but I've not got around to that yet.

Now from the ExceptionHandler decorator, I can horse around slightly, replacing the default exception handling with my own version:

public function __construct($adapter, $exceptionMap) {
    $this->adapter = $adapter;
    $this->exceptionMap = $exceptionMap;

    $this->replaceClientErrorHandler($exceptionMap);
}

// ...

private function replaceClientErrorHandler($exceptionMap)
{
    $stack = $this->adapter->getHandlerStack();

    $stack->remove('http_errors');

    $stack->unshift($this->handleHttpErrors($exceptionMap));
}

The handler stack doesn't have a "set" method that works on a named handler like $stack->set('http_errors', $this->handleHttpErrors($exceptionMap)), hence having to remove the default and then stick my own back in. But that's OK I guess.

And that all works.

I have the same test code as I had in the previous article, and it outputs much the same results.

My question is now... which of these approaches is better?

  • The previous article's code is simpler, and easier to follow as one needs to know less about the inner workings of Guzzle.
  • However letting Guzzle handle exceptions in one way, then intercepting that and going "actually now I'll do it meself thanks" seems like doubling up on work a bit.

I think from the perspective of code maintenance, the previous approach is probably "better" in the long run. But I'll just give both options to the chief, and see what he reckons. As my lunch break is about to be over, I think he'll be reckoning "get back to work, Cameron", so I better do that.

Righto.

--
Adam