Layered Testing

Many software systems will reach a scale where achieving adequate coverage of all provided features via traditional manual testing will prove virtually impossible, especially in a reasonable time scale.

A strong suite of automated tests, coupled with the adoption of a shift left mindset, can provide a solution to this problem. This approach can mean a system is under almost constant testing, at least for critical journeys and functionality, without which users would be adversely affected.

As with most aspects of software engineering, its imperative for this testing infrastructure to have well defined structure to avoid the evils of spaghetti code. Code quality shouldn't be something only in developers mind when writing production code, its as important for maintaining good quality tests.

This structure starts by identifying the different categories of tests that you will write, what they are designed to achieve, and how they fit into the bigger picture.

Unit Tests

Unit tests are defined by having a single source of failure, namely the class that is being tested. This obviously discounts the tests themselves having bugs, whilst developers often gravitate to wanting the test themselves to be wrong when they fail, this isn't the case as often as we may want it to be.

This single source of failure is maintained by all functional dependencies of the class being tested being mocked. The distinction being drawn here with functional dependencies is to avoid model classes and such like also having to be mocked, if the dependency offers functionality that could cause the test to fail then it should be mocked.

Unit tests should be run as frequently as possible and at least as often as change is merged into a shared branch, for this to have value the tests must follow FIRST principles of being fast, independent and repeatable.

Unit tests are therefore your fist line of defence against bugs being introduced into a code base and represent the furthest left it is possible to introduce automated testing. Indeed when following a Test Driven Development (TDD) methodology the tests exist even before the code they will be testing.

Integration Tests

Within a code base classes do not in fact operate independently, they come together in a grouping to perform a purpose within your overall system. If unit testing should fulfil the role of defending against bugs being introduced inside a class's implementation, then integration testing should act as a line of defence against them being introduced within the boundaries and interactions between classes.

By their very nature integration tests don't have a single reason to fail, any class within the group you are testing has the potential to cause a test to fail. Where in our unit testing we made extensive use of mocking to simulate functionality, in our integration testing we are looking to include as much real functionality as possible.

This makes integration tests harder to debug but this is a necessary price to pay to validate all the individual sub-systems of your code interact properly.

Whilst even the most ardent advocate of TDD wouldn't write integration tests prior to implementation, like unit tests integration tests should be fast and run frequently.

Automated UI Tests

In the majority of cases software is provoked into performing some operation based on input from a user. Implementing automated UI testing is an attempt to test functionality as that user, or at least in as close an approximation of the user as possible.

As with integration tests, these automated UI tests will have multiple reasons to fail, in fact they are likely to have as many reasons to fail as there are sources of bugs in the system being tested.

Although it is necessary to engineer an environment for these test to run, and not all functionality will lend itself to being tested in this way, these tests should contain virtually no mocking.

Automated UI testing is never going to lend itself to being as fast as integration testing and unit testing. For this reason they should be structured such that they can be easily sub-divided based on what they are testing. This will allow their execution to be targeted on the areas of change within a code base and the critical paths that cannot break.

They are likely to be run less frequently but they serve as a health check on the current suitability of the code for deployment. They are also work horses for the mundane, freeing up the time of precious manual resource to concentrate on testing more abstract qualities of the code.

These three areas by no means cover all the possible types of test you may wish to write, one notable exception being testing performance via Non Functional Tests (NFT). However they do demonstrate how a well thought through testing architecture consists of many layers each with a different purpose and objective.