The dynamic test anti-pattern
Branching and configuration belie uncertainty.
March 29, 2014Are your step definitions 10+ lines long? Are you relying too much on state to test? Do you get confused when you read your tests back?
I believe that opinionated specifications are the best tests. They focus in on detail and cut through lines of integration that make tests slow and brittle. They restrict degrees of freedom, making it easier to diagnose failing tests. They are better.
The following are anti-patterns of un-opinionated testing that I want to present and correct.
Reading settings from the application
It is far better for a test to have, in plain English, the intent of the test written out.
This seems to solve this problem well, so long as your test statements are declarative and clear. You need to say "The headline is green" instead of, "The headline is the correct color" to pull this off, but it's worth the effort.
Making testing decisions at run-time
Specific tests for each situation, pointing at different parts of the application is a better choice here. A section for ads enabled/disabled, or separate tests for admins versus 'regular' users.
Avoiding if statements in tests makes them readable. An if statement in a test step is a fast route to a contradiction with its phrasing.
Relying upon third-party functionality
Separate the successful integration of a component into an integration test, and run them with every build.
Store a 'known good' representation for your own purposes as a test-mode mock. Update the mock when the integration test fails.
Configuration of a system at test time
Permanent areas of the system exemplifying each case should exist. A completely static systems with repeatable configuration remain the best option.
Of course, if that's the goal of your test, you should test the configuration layer from an end-user perspective.
Calculating values
There is not much to say here, but that if you know 2+2=5, why is your test re-calculating that value every time it needs it? Use final values; they are unambiguous and not subject to change. If they do change in the system, it's time to use that big brain of yours to recalculate their values and re-verify the work.
Conclusion
These anti-patterns are just a handful of ways where tests can get out of hand. I believe we can remove them with a bit of deep thought and some engineering muscle.