TL;DR

This is third part in series how to kill UI regression testing. It is about git source control system and its web application part, github, bitbucket or gitlab. I have already wrote about this topic in Product moving parts as a source for test strategy, and Micro tracking changes for regression test. In this post, I will extend my thoughts from those two posts.

So git helps you to determine which part of application feature could be affected by the code change. Precondition is that you need to be able to read the code. Just read and understand functionality in that code. Many software testers started in software testing because their are afraid of code. I think that to be able to read the code could be great booster for software tester career.  And by reading pull request, you have very valuable mentor, the author of code himself.

Soon, you will find different code change patterns. For example, change in function signature in helper module could break more features than change in one module method. You can learn about that very quickly, the most important is that you should stop be afraid of reading the code.

And basically, you are doing code review. You will have questions and by asking them you could find issues before code is deployed and put in action.

Next post will be about Jenkins as an example of continuous integration tool.

by

TL;DR

This is second post about idea how to kill UI regression testing. In first post I described symptoms for UI regression testing that I would like to kill. I will describe development framework tests that are precondition for killing UI regression testing.

Development framework tests (DEFRATE) are automated tests that are written by developers. Here are reasons why I think that should be the case for every development project.

You have a developer that is a “rock star” in some development framework, lets take for example Ruby on Rails (you can replace that with development framework on your project). Lets see what Ruby on Rails framework offers in testing front.

Here is official documentation on testing Ruby on Rails application. Here is additional ruby testing toolbox. Every software developer that claims Ruby on Rails rockstar title, should be master in any of those ruby testing frameworks.

But which tests should developer create?

Developer test should create tests in DEFRATE that CHECK their code design.

Developer should check all flows that he created in its mind. Those code flows must PAST through the RUNTIME (lets say processor instructions). Tests should follow DEFRATE practice and use all features. For example, developer wrote some code that communicates with database. Essential part of test should be code that mocks database. Good DEFRATE enables easy mock code creation. And when somebody changes code that is mocked in a test, mocked code will fail and detect REGRESSION in code. Lowest and most quickest way to detect regression change.

Side effect of DEFRATE tests are tools that enables you to have code coverage by running those tests. Code coverage information is excellent source of information for you test strategy on UI level. And remember, code line coverage is just one testing coverage out of 101 testing coverage metrics. Here is excellent resource about testing coverage [Kaner: Software Negligence and Testing Coverage]

Google testing blog is also excellent resource for DEVELOPERS how to write good DEFRATE tests.

And the best way to learn testing framework is to read a book about it and do ALL the exercises listed in a book.

But my developers do not write DEFARTE tests! So I must stick with my UI regression testing. Ok, but now you know metrics that enables you to provide information about software quality in your project. And you have directions how to make quality in your project much better.

In next post, I will discuss source control tools that can help you to kill UI regression testing.

by

TL;DR

This is first post in what should be series of post. Idea is: How to kill UI regression testing. Idea started at CAST 2016 lighting talk and I extended it more at Testival 2016 open session slot where I get excellent feedback from the audience. The goal is to write material that will try to explain this idea in constructive, thoutfll way with experiments feedback that will suport or fail it.

First step, lets detect the symptoms, or reason why you are using UI regression testing.

You have a test plan that only contains a huge list of test cases. Every test case has useful name, and not so useful steps description. Context is a document with a hundred of pages. You are proud, because you wrote that document, and you are the only person that read it.

You have a automated test suite of selenium web driver tests, that have more that a hundred automated test cases, and in order to run it, you need several hours and several Jenkins instances. You have a number of failed tests, and those fails are false positives (test failed but this does not indicate problem in your product).

If this is the case, that means that developers DO NOT WRITE TESTS in their development framework (e.g. Angular, Ruby on Rails, Django,…). Note here that I do not mention unit tests, but DEVELOPMENT FRAMEWORK TESTS (DEFRATE).

There are many DEFRATE broken developers. They know only part of their framework, an usually they are not interested in testing part.

And because of that, you are developers safety net, and you are not doing software testing.

Because software testing is:

Every physician has first to determine the symptoms,  in order to determine disease. I defined the symptoms for  disease called UI regression testing.

Next week I will talk about DEVELOPMENT FRAMEWORK TESTS (DEFRATE).

by