Why you should be automating acceptance testing
Before we talk about acceptance testing, let’s clarify first what we mean exactly and how it differs from unit testing.
Acceptance testing refers to the process of validating that an application delivers the business value expected by the customer, as well as preventing any defects that may break preexisting functions of the application.
Acceptance testing differs from unit testing in that unit tests are developer-facing, not business-facing. Acceptance testing is about testing whole stories at a time against a running version of the application in a production-like environment. Conversely, the aim of unit testing is to show that a single part of the application does what the programmer intends it to; this is by no means the same as asserting that a user has what they need.
When it comes to automating acceptance tests, too many project managers and developers still think that it is easier to do it manually. In these projects manual testing is the norm and often represents the only form of testing adopted by a team. The reasons that are often put forward to justify this are:
- It is too expensive to create and maintain a suite of automated acceptance tests.
- Unit test suites created through test-driven development are enough to protect against regressions.
Our experience, however, has been that the cost of a properly created and maintained automated acceptance test suite is much lower than that of performing frequent manual acceptance and regression testing, or that of the alternative of releasing poor-quality software.
In addition, the cost of performing manual acceptance tests every time that a new release of the application is available can also be an important constraint, and that’s especially true if the application is complex and new releases come out frequently.
Finally, we have also found that automated acceptance tests catch serious problems that unit or component test suites, however comprehensive, could never catch.
Manual testing, of course, has its place: exploratory testing, usability testing, user acceptance testing, showcasing. But human beings are simply not equipped to work effectively at the mundane, repetitive but complex tasks that manual regression testing requires of them. Poor-quality software is the inevitable outcome of such a poor-quality process.
For this reason, when used as a guarantee of the behavior of the system from a user’s perspective, automated acceptance testing offers an invaluable defense against regression problems that will arise during the life of any application of any complexity.