TL;DR
In the previous post, we commented on our answers for exercise Specification For Simple Function. Let’s move on and discuss how software fault, error, and failure differ from each other. We will introduce you to software testing based on the remarkable book, Introduction To Software Testing by Paul Ammann and Jeff Offutt.
Limitation
The theoretical limitation of software testing is that testing can not show the absence of failures.
The problem of finding all failures in the program is undecidable [Ammann and Offutt].
Many testing professionals consider a successful test a test that finds a failure. The problem with that statement is that this is level two software testing thinking.
Validation Vs. Verification
These software testing definitions are from the book Introduction Software Testing. I found them useful, but it is ok to use different definitions.
Validation is a process of evaluation software at the end of development to ensure compliance with intended usage. It is usually done by non-developers with strong domain knowledge.
Verification is the process of determining whether the products of a given phase of the software development process fulfills requirements established in the previous phase. This is more technical activity and usually is done by developers and technical testers.
It is essential to state that if verification passes, validation might still fail.
Fault, Error, And Failure
Software fault is a static defect in the software. Software error (failure update on 16.10.2020) is an incorrect internal software state that is a manifestation of software fault. Software failure is product external unexpected behavior concerning expected behavior (listed in requirements or common sense behavior). The fault is always a developer (human) software design mistake.
Example
Let’s examine our Elixir union function from exercise on a simple function specification. Here is our union function with error handling:
Let’s introduce a fault that will cause wrong error state, and failure for specific test data:
The developer made a software design mistake by relying on Elixir List concatenation operator ++ as union implementation and forgetting to add Enum.uniq() the function that removes duplicate List elements. That design mistake is a failure that causes error function state because there is only one union function state, final List that contains elements of both Lists.
Input test data [1, 2, 3] and [4, 5, 6] does not trigger failure because the result is actual union of those two lists: [1, 2, 3, 4, 5, 6].
But input test data [1, 2, 3] and [3, 4, 5] triggers failure because the result is [1, 2, 3, 3, 4, 5] but requirement states that union is [1, 2, 3, 4, 5 ].
Remember
Faults and failures could only be identified by developers, while failures could be caught both by developers and testers.
Comments are closed.