TL;DR

To have precise measurements, we must determine the measurement scope. The post is aligned with the Black Box Software Testing Foundations course (BBST) designed by Rebecca Fiedler, Cem Kaner, and James Bach.

It is crucial to define a measurement scope because as the scope broadens, more measurement variables are introduced. More variables cause a higher probability of measurement error.

We gave an example for code coverage measurement. In code coverage, we could have the following scopes:

• statement
• branch
• path

So we could have 100% statement coverage and still miss a lot of known bugs because our branch coverage could be at the same time, 50%.

Or what do we include in the coverage report? Do we include all libraries used in our project? In that case, our 100% statement coverage could significantly drop. If library developers had not written unit tests, for that library code, we have 0% statement coverage. Does our code coverage tool have an option to include multiply source code folders?