In the previous post, we commented on our answers for exercise Specification For Simple Function. Let’s move on and discuss how software fault, error, and failure differ from each other. We will introduce you to software testing based on the remarkable book, Introduction To Software Testing by Paul Ammann and Jeff Offutt.
The theoretical limitation of software testing is that testing can not show the absence of failures.
The problem of finding all failures in the program is undecidable [Ammann and Offutt].
Many testing professionals consider a successful test a test that finds a failure. The problem with that statement is that this is level two software testing thinking.
Validation Vs. Verification
These software testing definitions are from the book Introduction Software Testing. I found them useful, but it is ok to use different definitions.
Validation is a process of evaluation software at the end of development to ensure compliance with intended usage. It is usually done by non-developers with strong domain knowledge.
Verification is the process of determining whether the products of a given phase of the software development process fulfills requirements established in the previous phase. This is more technical activity and usually is done by developers and technical testers.
It is essential to state that if verification passes, validation might still fail.
Fault, Error, And Failure
Software fault is a
static defect in the software. Software failure is an incorrect internal software
state that is a manifestation of software fault. Software failure is product external unexpected
behavior concerning expected behavior (listed in requirements or common sense behavior). The fault is always a developer (human) software design mistake.
Let’s examine our Elixir
union function from exercise on a simple function specification. Here is our
union function with error handling:
Let’s introduce a
fault that will cause wrong
error state, and
failure for specific
The developer made a software design mistake by relying on Elixir List concatenation operator
union implementation and forgetting to add
Enum.uniq() the function that removes duplicate
List elements. That design mistake is a
failure that causes
error function state because there is only one
union function state, final List that contains elements of both
Input test data
[1, 2, 3] and
[4, 5, 6] does not trigger failure because the result is actual
union of those two lists:
[1, 2, 3, 4, 5, 6].
But input test data
[1, 2, 3] and
[3, 4, 5] triggers failure because the result is
[1, 2, 3, 3, 4, 5] but requirement states that
[1, 2, 3, 4, 5 ].
failures could only be identified by
failures could be caught both by