TL;DR

This post will list pieces of evidence why you should not waste money on alleged testing automation.

This post was provoked by the TestBash home 2020 presentation playback by Angie Jones: Revisited: How To Get Automation Included In Your Definition Of Done.

In her presentation, Angie gives a cookbook on how to squeeze “testing automation” into a tight sprint schedule:

1. Collaborate
2. Automate strategy
3. The incremental build of “automation.”

I agree with those three points, but I do not agree that we need dedicated professionals for this. In this presentation, Angie did not explain what is the benefit of “testing automation” or what is “testing automation.” I will try to explain why it is not such a thing as testing automation.

Let’s Make Things More Complicated

The main issue with today’s software development is that it is broken in teams. So we waste a great amount of money, energy, and thinking about how to sync those teams. The idea of testing automation is to make things even more complicated by introducing a new team that will write automated tests. There is no such thing as automated testing. Either you write code that is part of the final product, or either you are testing the overall product. I have a filling that testers consider that writing code is just dumping source code. There is a lot of thinking and probing involved in the testing concept written in code. Some developers decided to write code that helps them to prove that concept. Some people call it automated testing.

In CIA’s The Art Of Simple Sabotage, there is the part of how to sabotage work in companies.

Managers and Supervisors: To lower morale and production, think of the worst boss you’ve had and act like that. Be pleasant to inefficient workers; give them undeserved promotions. Discriminate against efficient workers; complaint unjustly about their work. When possible, refer all matters to committees for “further study and consideration.” Attempt to make the committees as large and bureaucratic as possible.

A great example is when your developers are grouped around technology. In our excellent development team, we have frontend and backend developers. Most of the time, the frontend team is waiting for the backend team to finish the rest API endpoint. Then we try to squeeze frontend work at the end of the sprint (along with testing work). And the reason why we do not require our developers to learn Javascript and Ruby (or Python, Java, Elixir,…) is that this would make things more complicated 🙂

Testing Automation Mith

There is no such thing as testing automation. If you are a tester, investing your time in learning how to use “testing ” frameworks (in any language) is a waste of time. Those frameworks are created by developers for developers. They built those frameworks to help them to develop complex applications. The application could be Linux service, mobile app, web application, or Tesla car controller for the brake system. Using those frameworks, developers can exercise their production code using shortcuts provided by the “testing framework.”

If you would like to learn how to write code, start the software developer learning path.

Software testers do not write code during testing activity.

If you write code, you are a software developer. And using some of the frameworks is also a waste of time. Like frameworks that exercise browser. Code written in those frameworks is very sensitive to any code change. Another problem is that the user would never use your web application as the script written in the browser automation framework would.

Rest API frameworks are also problematic because you need to have running webserver to exercise those tests. And this adds time to execution times because you run your tests through HTTP. Every modern web development framework enables you to exercise server-side code without an HTTP web server, resulting in much faster test execution times.

How To Test As Developer

When you write code as a developer, if you are a good developer, you would use several frameworks. Those frameworks are code organized in APIs, created by other developers that had a similar problem as you. The excellent developer is supposed to write a lot of code that handles alternative flows. Usually, those flows are called error flows. Many of those alternative flows are from the framework used by the developer.

The developer would exercise that alternative flows either by “testing framework” or just by using the application in a development environment. The testing framework is handier, because, only by running one cmd, you can exercise a lot of code executions that they develop.

The developer could do that on any level. It could use just one code unit (class, module, function), or several groups in cooperation (integration). But this is their responsibility. Still, the strategy is to exercise all possible alternative flows that ARE DOCUMENTED in framework documentation.

The second development strategy is to use code coverage framework that supports report for statement and branch coverage. Strive to have a 90% statement and branch coverage is not an impossible task.

And this is their test automation definition of done. It is part of the development effort.

Real Software Testing

A software tester is someone who finished (or is interested in completing) Black Box Software Testing courses. This is a start. In those courses (held by several providers), software tester will get all needed information on how to further elevate software testing learning path (For example, Rapid Software Testing).

Software tester would think about alternative flows not mentioned in developer frameworks. They notice that the development framework did not document important alternative flow, would think about essential combinations of alternative flows, or would think about risk analysis (BBST course materials have further information).

Regression Test

In my consultancy work, I realized that 90% of tester work is related to regression testing.

We need to check all scenarios for this release because we do not know how changes in code base affect our application.

And this was the main reason why “testing automation” was born. Let’s recap:

We do not know what changed, therefore, we will check everything.

There is the much smarter way to track changes in the application codebase. If you have not been living in a cave for the last 20 years, you probably heard of git, pull request, GitHub, GitLab, and Bitbucket. Using those tools, you exactly know what changed.

A simple IDE search feature or smart grep could easily list were in the codebase is changed function is used.

If developers do not have unit tests, they could still run code runs of application parts that changed. And check how that change affects application parts used by that code.

Conclusion

Developers could write code that helps them to check how code changes affect the whole application. They could still do that without using unit tests. They know which part of the app changed, and they rerun just that part of the code. The strategy is to have 90% for statement and branch coverage. Missing code strategy is to read the documentation (or in case of open source, actual code) of the framework that they use.

Software testers are moving from regression checkers. They help developers to reveal possible missing code; they help in creating examples for user stories; for instance, mapping workshops, they identify user scenarios using the application as the real user. In document: Cem Kaner, Elisabeth Hendrickson & Jennifer Smith-Brock: Managing the proportion of testers to (other) developers (page 14.), you can find how testers could contribute moving from regression testing checkers.

What to do with existing “tester automation specialists”? Put them on the software developer path. Teach them how to use development frameworks that create production code. Give them the option to use testing frameworks for unit or integration tests. Make them forget about learned “software testing frameworks”. They should become real code contributors.