TL;DR
Now that we know the structure of exploratory testing sessions in Session-Based Test Management [Bach], let’s explore the exploratory testing session report debriefing between test manager and tester. This is a part of the Exploratory Testing Pathway. Many thanks to Marcel who sublimed this great resource on his blog, That’s the buffet table.
The Parties
Prevailing parties for session-based test management debriefing session are test manager and tester that did the exploratory testing session. Because the exploratory testing session has the structure in session-based test management, the test manager debriefing session also has the structure (checklist) that could help test the manager in the evaluation of the tester exploratory testing report.
Checklist
Charter
The Charter is usually prepared by the test manager. In this part, we check if the testing session was inlined with the defined Charter.
- Did you review relevant approved session reports?
- Does it match the bulk of the testing that was actually done?
Areas
Areas help us to direct our exploratory testing session so we could find valuable information in defined charter duration time.
- Is there at least one O/S keyword (unless it’s not applicable)?
- Is the build keyword accurate?
- Is there at least one strategy keyword?
- Is there at least one product area, as specific as meaningful, to specify?
Duration
Duration is crucial so we could tailor exploratory testing session scope (area) based on actual duration data.
Is the duration code in line with the actual duration?
Was the session continuous and uninterrupted?
TBS
TBS stands for three types of tasks in session-based test management: test design and execution, bug investigation, and test setup.
- Have the TBS definitions been followed?
- Have the TBS precedence rules been followed?
- Do the TBS numbers relate to On Charter work only?
Opportunity
The opportunity helps us to determine was an exploratory testing session on track with the Charter. This number is usually high when we have little knowledge about the application that we test.
- If the opportunity number is over 0%, what was the opportunity?
- If the opportunity number is over 25%, consider modifying the Charter.
- If the opportunity number is over 50%, modify the Charter for this session, and consider doing a new session based on the original Charter.
Data Files
- If there were no data files, why not?
- If there were data files, were they original or re-used? If re-used, were they modified in any way? If so, how do they now relate to other sessions that refer to the same data?
- Is there an associated test coverage outline that should be referenced?
Test Notes
- Are they comprehensible?
- In conjunction with the Charter, do they answer the question, “what happened in this test session?”
- Do they include information about coverage, oracles, and strategy?
- Is there anything in the notes that can be re-used in a future session? If not, that may be okay, but remember: Part of the reason for the notes is to build a better plan for testing.
- Is this section free of issues and bugs?
Bugs
- Is enough information included to reproduce the problem?
- Is the section free of test notes and issues?
Issues
- If there are no issues, does that mean there was no confusion, no remaining questions, and no obstacles in the path of testing?
- Do any issues require actions to be taken?
- Is the section free of test notes and bug reports?
Overall
- Will metrics based on this session sheet faithfully represent the testing that was done?
- Do the results of this session suggest the need for another session?
- Should this session be extended and amended?
Remember
This checklist is only giving the structure for the debriefing sessions—for example, part on the opportunity percentage. The test manager still needs to know the application domain to determine opportunity percentage numbers. And most of that is connected with tacit and mimeomorphic knowledge.
Comments are closed.