Unit testing (PHP): keeping sight of the intent of the tests

G'day:
An article with some actual code in it! That'd become a rarity. This is actuallya real-world challenge I'm faced with in backfilling some unit tests. Yeah, I know I'm an advocate of TDD so that should mean there's no "backfilling" of tests: they all get done up front. But for reasons I won't go into... I'm backfilling testing for this function.

Here's the function:

public static function loadValidatorMetadata(ClassMetadata $metadata)
{
    $metadata->addPropertyConstraint('languageId', new Assert\Range(
        ['min' => 1, 'max' => self::$maxLegacyLanguageId]
    ));

    $metadata->addPropertyConstraint('destinationId', new Assert\Range(['min' => 1]));

    $metadata->addConstraint(new Assert\Callback('validateArrivalDate'));
    $metadata->addConstraint(new Assert\Callback('validateNights'));

    $metadata->addPropertyConstraint('firstName', new Assert\NotBlank());
    $metadata->addPropertyConstraint('firstName', new Assert\Length(
        ['min' => 1, 'max' => GroupEnquiryRepository::SHORT_STRING_MAX_LENGTH]
    ));
    $metadata->addPropertyConstraint('surName', new Assert\NotBlank());
    $metadata->addPropertyConstraint('surName', new Assert\Length(
        ['min' => 1, 'max' => GroupEnquiryRepository::SHORT_STRING_MAX_LENGTH]
    ));

    $metadata->addPropertyConstraint('email', new Assert\NotBlank());
    $metadata->addPropertyConstraint('email', new Assert\Email());

    $metadata->addPropertyConstraint('groupTypeId', new Assert\Range(['min' => 1]));
    $metadata->addPropertyConstraint('groupSize', new Assert\NotNull());
    $metadata->addPropertyConstraint('groupSize', new Assert\Range(
        ['min' => GroupEnquiryService::MIN_GROUPS_THRESHOLD, 'max' => BookingService::MAXIMUM_PEOPLE]
    ));
    $metadata->addPropertyConstraint('youngestAge', new Assert\Range(
        ['min' => 1, 'max' => GroupEnquiryService::MAX_AGE]
    ));

    $metadata->addPropertyConstraint('telephoneCountryCode', new Assert\NotBlank());
    $metadata->addPropertyConstraint('telephoneCountryCode', new Assert\Length(
        ['min' => 1, 'max' => GroupEnquiryRepository::SHORT_STRING_MAX_LENGTH]
    ));
    $metadata->addPropertyConstraint('phone', new Assert\NotBlank());
    $metadata->addPropertyConstraint('phone', new Assert\Length(
        ['min' => GroupEnquiryService::MIN_TELEPHONE_NUMBER_LENGTH]
    ));
    $metadata->addConstraint(new Assert\Callback('validateMobileCountryCode'));
    $metadata->addConstraint(new Assert\Callback('validateMobile'));

    $metadata->addConstraint(new Assert\Callback('validateAdditionalInfo'));
}

First things first: yeah, normally I'd not write a function that's 40-odd lines long, but in this case there's very little actual logic, plus it's only setting some validation config. BTW this is using the Symfony 2 validation framework (Symfony: validation), as implemented by the Silex microframework (ValidatorServiceProvider: validating objects).

To provide context, we have a class - basically a validation bean - thus:

class GroupEnquiry
{
    public $languageId;
    public $destinationId;
    public $arrivalDate;
    public $nights;
    public $firstName;
    public $surName;
    public $email;
    public $telephoneCountryCode;
    public $phone;
    public $mobileCountryCode;
    public $mobile;
    public $groupTypeId;
    public $groupSize;
    public $youngestAge;
    public $additionalInfo;
}

And that loadValidatorMetadata() function provides the constraints for Symfony to enforce. As you can see: there's range validation, existence validation, length validation, date validation, and a coupla compound constraints implemented via callbacks.

So, anyway, I need to write the testing for loadValidatorMetadata(). Initially I was staring at the code going and going "how the hell?".

Then I thought: I'll assert the addConstraint() and addPropertyConstraint() methods are called for the correct properties, and the correct number of times. But that seemed unhelpful. What's the point of checking that some constraint is applied to a property, but not checking what sort of constraint. IT all seemed too contrived.

Then I thought that the constraints themselves don't need testing here because I'm either testing our callbacks elsewhere, or Symfony have done their own testing. In theory this is a fine notion, but in reality it does not actually help me test our validation code works.

I was stumped as to how to test this stuff, without basically replicating the function in parallel for test. And that clearly sucks as far as effort and common sense goes. It was the wrong approach.



Then I stepped back form the keyboard, and thought "what actually are we trying to test here?" We're wanting to test that all the various validation constraints we need to apply to a GroupEnquiry object are actually enforced. Basically that the validation of the GroupEnquiry object... well... works.

So the most expedient way of doing this is to not test this method directly, but instead create some test objects and... well... validate them! Basically I'll create a "happy path" object which I know passes the validation. Then in each of a series of test variations, initialise one of the properties with an invalid value (according to whatever constraints we have), and make sure the validation fails. If the validation fails when we expect it to: this validator metadata is configured properly, and we can say that the function is A-OK, and tested.

This is pretty easy to do with PHPUnit: it provides for the concept of a DataProvider, which is a function which returns an array of test data, which each element of the array being passed into the test method in turn. This means in my case my DataProvider might return a 20-element array, which means my test gets called 20 times. This is tidier than having 20 separate tests (which, ultimately, will share a lot of code). I haven't done the code yet, but I'll see if it's worth posting later on once it's done.

Anyway, this whole conjecture is something I've been thinking about recently sometimes the tests we write are too focused on the minutiae of the logic in the methods we test, and perhaps are looking too closely at that to see the bigger picture. At the end of the day the purpose of the tests is not to test that the logic works; it's to test that the business value as been fulfilled. I guess this is make more stark when not doing TDD... when doing TDD the tests evolve with each logic variation put into place, so it's natural and correct to be testing each branch and variation of iterations. It just seems odd to be doing that sort of thing when doing post-hoc testing.

I guess that's my lesson here. The strategy for back-filling tests might not take the same approach as when doing TDD from the outset. So the eventual tests might be approached and coded slightly differently.

Righto... enough talking about it... time to actually do it...

--
Adam