TL;DR
This is first post in what should be series of post. Idea is: How to kill UI regression testing. Idea started at CAST 2016 lighting talk and I extended it more at Testival 2016 open session slot where I get excellent feedback from the audience. The goal is to write material that will try to explain this idea in constructive, thoutfll way with experiments feedback that will suport or fail it.
First step, lets detect the symptoms, or reason why you are using UI regression testing.
You have a test plan that only contains a huge list of test cases. Every test case has useful name, and not so useful steps description. Context is a document with a hundred of pages. You are proud, because you wrote that document, and you are the only person that read it.
You have a automated test suite of selenium web driver tests, that have more that a hundred automated test cases, and in order to run it, you need several hours and several Jenkins instances. You have a number of failed tests, and those fails are false positives (test failed but this does not indicate problem in your product).
If this is the case, that means that developers DO NOT WRITE TESTS in their development framework (e.g. Angular, Ruby on Rails, Django,…). Note here that I do not mention unit tests, but DEVELOPMENT FRAMEWORK TESTS (DEFRATE).
There are many DEFRATE broken developers. They know only part of their framework, an usually they are not interested in testing part.
And because of that, you are developers safety net, and you are not doing software testing.
Because software testing is:
Every physician has first to determine the symptoms, in order to determine disease. I defined the symptoms for disease called UI regression testing.
Next week I will talk about DEVELOPMENT FRAMEWORK TESTS (DEFRATE).
