Do PACT Provider Verification tests replace Controller tests?
In this article, we are going to talk about PACT tests, which is a form of Consumer-Contract-Driven-Testing. For more information on what PACT is, please visit this link
Scope of a Provider Verification Test
A frequent question that came up in a couple of client engagements where we had used PACT testing was regarding the scope of the Provider verification test: just how much of the application gets tested as a part of such a test?
If you were to follow the official documentation and suggestions provided there, I’d say the options are really wide open for you and your team to choose:
- Stub all classes underneath your Controller class and test the responses adhere to the contract that the client requires from the provider
- Stub only external API calls / database interactions, but run the request through the entire app otherwise. The two stubbed components then would have their own PACT tests or DAO tests, respectively.
The first approach is what a previous engagement I was with chose to do, where as on another engagement of mine, our team had chosen to go the second route.
With the latter approach, we had an in-memory database that was populated with the data the State that was required from the provider, and stubbed out external dependencies (like payment gateway clients). And we let the request run it’s course.
This worked well for us. Since PACT was a tool to “deploy-with-confidence” running the client’s request through the entire application provided us a tad more of the said confidence.
However, the first approach that we used on another engagement worked worked perfectly well too. The choice of which approach to take will have to be made by you and your team, and by the nature of your project.
Personally, I prefer the second approach. PACT is often a tool I would like to rely on to ensure the “happiest -customer-flow” is well-tested right before a deployment to production. A PACT consumer contract from the client is should usually has (preferably, only) the happiest business flow as a part of it’s positive tests. Thereby, having a real request flow through as much of a system and asserting on an expected response would deliver a whole lot more value to me and the team than a test where downstream classes are mocked out.
Why? Because when we mock, we make assumptions that may or may not be true. It’s all dandy when all the you own the code on both sides of the mock (say, the when a *Service class mocks a DAO in it’s tests), but since in the PACT Provider test the request comes from an external team, assumptions could be unwittingly broken.
What about the unit tests for the controller?
Now to answer the question if we still need a separate controller-test when using second approach as mentioned above, the answer is clear. Provider verification tests are not your controller tests when using this method, because here you are almost testing all layers except calls external dependencies. It makes it very similar to an integration test.
You will absolutely need a separate controller-test to unit test your controllers. And yes, in these tests, all downstream classes will be mocked out.
We need these unit controller-tests, because in the case your provider tests fail, you need to use them and also the maybe the unit-tests for your domain classes to pin-point the source of the issue. This is possible only if you have a separate controller-test.
But what happens if you were to choose the first approach, where you stub all downstream classes in a Provider Verification test?
You might have noticed a Provider Verification test with all the downstream classes mocked out is essentially not very different from a controller test with all it’s downstream dependencies mocked out.
Do we still need a controller-test then?
It’s a very valid case not to have them, and then treat Provider Verification tests as the controller-tests. There is so much more less code to maintain, and also, it ensures that the controller has only the bare minimum code to satisfy the client needs. This approach does have very compelling arguments.
So what’s the problem, you might ask?
For me, I would still write a controller-test. And I have two reasons for this, one practical, and another philosophical.
From a practical stand-point, I find it a bit difficult to use the provider verification tests to do TDD. For starters, the feedback cycle is longer because, my first green build would only be at the end of writing the entire controller. There really is no step-by-step approach.
The loss for us is that we miss a controller unit test that details the various capabilities of the controller. With a step-by-step approach, you usually tend to have different tests in the controller unit test class for each function of the controller (for eg, the conditions by which an incoming request might be invalid)
Of course, you could pass down the request to another class (and unit-test that), making the controller as thin as possible, but then, I believe this is indirection for the sake of it. Another option is to pass the object directly to the business-logic containing classes; and I would recommend against it, because from a conceptual point of view, classes having business logic should not be dealing with request objects. That’s a clear violation of separation of concerns.
I would do all the request object validation and transformation at the controller, and all the business classes would receive the appropriate inputs for it to do it’s job. Essentially my controller is my anti-corruption layer before the inputs reach my business objects. Keep the classes having business-logic, focused on business problems.
My second reason is more philosophical, and I will admit it is a weak argument.
I prefer that a code repository for a service / application should contain all code needed for it to be run / tested, within the repository. This is however, not possible, if PACT contracts are stored outside the repository like in a PACT broker. The provider tests (which serve the dual role of a controller-test) will need them to test the controller.
Granted, almost all projects these days will download dependencies from remote source before compilation and running tests, and by that logic, PACT contracts could be considered just another dependency to download. But there’s a nagging feeling for me that this is not right. It’s like I’ve split the code up and stored some parts of my code outside the repository. It’s not like every other dependency.
So this is why I would personally prefer to still have controller tests in my projects.
I’m open to hearing your opinions, too; mostly because this is something I’m quite on the wall with.
