In this post, we list bug count alternatives that could help you in evaluating software tester’s work. The post is aligned with the Black Box Software Testing Foundations course (BBST) designed by Rebecca Fiedler, Cem Kaner, and James Bach.
It is very tempting to use the reported bug count as a tester work assession method. This could lead to proxy measurement fallacy.
People optimize what we measure them against, at the expense of what we don’t measure.
What alternatives we could use to assess the quality of software tester work?
Documenting Test Work
We do not mean documents in the form of test cases with detailed steps description. We refer here to Session-Based Test Management [Bach] documentation. In that document, tester starts with a charter that describes basic guidelines for a timeboxed session. Here is an example for WordPress application:
Explore Posts using Web Browser to discover that user could access only it’s own Posts.
After we have charters, we write notes with the following tags:
Idea – during exploring, we got an idea about the next charter, testing technique, risk, application feature.
Bug – we think that we found a bug, so we described not in great detail, just enough information that we could later get back to it.
Question – we note our confusion with a question
Opportunity – idea for the next chapter
Fact – we learned something new about the application.
Coaching Other Testers
How tester transfers knowledge to fellow testers?
Are bug reports written using RIMGEA or some other method?
How testers samples tests to mitigate regression testing risks?
Using idea and opportunity testing note types, so tester explores found bug to find more critical bugs.
We listed ideas on how to better assess the quality of software tester’s work. Remember that people optimize what we measure them against, at the expense of what we don’t measure.