TL;DR
We give concrete examples of what does it mean to do a complete testing. The post is aligned with the Black Box Software Testing Foundations course (BBST) designed by Rebecca Fiedler, Cem Kaner, and James Bach.
It seems that WordPress also knows that it is not possible to test Everything. My current installation that was installed automatically from Pagely, version 5.3.2, has broken Add Media feature. Currently, I can not add an image to this post body. When I selected the Set feature image, it opened add an image that looks different than usual. The summary is that I lost thirty minutes of my valuable time. Below you will find the solution to this problem.
To Test Everything, you would have to:
- Test every possible combination of inputs to every combination of variables
- Test every possible sequence through the program
- Test every possible timing of inputs (check for timeouts and races)
- Test every interrupt at every point it can occur
- Test every hardware/software configuration/version, including configurations of servers that are not under your control
- Test for the interference of other programs operating at the same time
- Test every way in which any user might try to use the program
The solution to the problem described above? I switched to Safari. It seems that WordPress (or Google) did not test this version of WordPress on Chrome Version 79.0.3945.130 (Official Build) (64-bit). This is my second recent big fail of Chrome. First was that AmazonPrime English cc subtitles have not been displayed. When I tried to find out the Chrome version, Chrome auto-updated to Version 80.0.3987.122 (Official Build) (64-bit). This is an example of how it is hard to test every software configuration/version. This version fixed the issue 🙂
We suggest Dr. Cem Kaner excellent paper on The Impossibility Of Complete Testing to find out more about this software testing problem.
