Taxonomy is the practice and science (study) of classification of things and concepts, including the principles that underlie such classification [source Wikipedia]. First time when I herd of how to apply taxonomy in software testing was in BBST Test Design course. You can find more details in Dr. Cem Kaner paper, Bug Taxonomies: Use Them to Generate Better Tests.
In my testing I often have to test CRUD functionality. I searched for CRUD (create, retrieve, update, delete) bug taxonomy list, but Google did not list any relevant result.
In this post I will try to do bug taxonomy for CRUD acronym. CRUD acronym is often used and I think that many testers would found that list very useful. In the end, I will map CRUD acronym on REST API methods.
In this example we have object with mandatory and optional attributes. One of the attributes must have unique value. Client side could create any Object (no any check is performed) and server has to do all checks. Context of bug taxonomy are object attribute values.
Create object:
- missing mandatory attribute
- attribute does not belong to object
- duplicate attribute
- wrong attribute value type
- wrong attribute value range
- wrong attribute value format
- existing value for unique attribute value
Retrieve object:
- missing unique attribute value
- unique attribute value does not exist
- wrong type of unique attribute value
- missing attribute in response
Update object:
- missing unique attribute value
- unique attribute value does not exist
- update of unique attribute value to existing value
- wrong type of unique attribute value
- duplicate attribute
- wrong attribute value type
- wrong attribute value range
- wrong attribute value format
- attribute does not belong to object
Delete object:
- missing unique attribute value
- unique attribute value does not exist
- wrong type of unique attribute value
- object already deleted
Notice the redundancy in Retrieve, Update and Delete.
This could be easily mapped to REST API:
Create -> POST
Retrieve -> GET
Update -> PUT
Delete -> DELETE
When I start testing RSET API, I first automate tests for CRUD bug taxonomy. But that taxonomy must be also used by project management in order to negotiate REST API protocol. For every object, we must know:
- attributes
- mandatory attributes
- unique attributes
- attribute value type
- attribute value range
- attribute value format
Otherwise, RSET API could easily be used for malicious data manipulation. If you found that I missed some bug taxonomy, please let me know and I will add it to the list.
Karlo – I did a similar analysis on a web application about 5 years ago and this approach looks interesting. Do the attributes pertain only to the REST protocol itself or to underlying business rules? For example, where would a check for string buffer overflow fit in the taxonomy, where the mandatory attribute is actually the maximum string length for a text field? I don't think that attribute would be transmitted as part of the REST protocol itself. Thanks
Hi Jeff, thanks for your feedback!
No, attributes do not pertain only to REST API protocol. For example, analysis could also be applied to database connection (jdbc, odbc,…) between client and server.
Check for string buffer overflow would fit in attribute value range check.
Server would check is string attribute in defined range, to prevent overflow failure.
Although, having maximum string length for a text field as protocol attribute is not good business rule practice. That means that we trust client how big is his data. That type of protocol error was a cause for hearthbleed openssl bug.
Regards, Karlo.
I like it. It may be helpful to include attribute format. When I am thinking about CRUD testing I also think about the format of attributes (e.g. must begin with AD and end with 0, must be date yyyy/mm/dd…). This may be just semantic because the attribute can be a business defined type and not just a value type (number, alpha, alpha-numeric…). The business type could include the value and format, but I normally think of the separately.
Hi Charles, thanks for your feedback!
I agree, checking attribute value format is added to the list.
Regards, Karlo.