Application variability tour in action

Reading Time: 2 minutes

TL;DR

My first contact with software testing was in factory and control schools of software testing. At that time I did not know that fact. In this post I will explain how I needed to make a shift from those schools, in order to deliver better software testing to my clients.

In my current software testing project, I got question: “Do you have all the requirements in order to start your testing activities?”. I said: “For start, I only need the application.”

By that I meant, I will start to explore the application based on product tours created by James Bach and expanded by Michael Kelly [source]. This paper is required reading for BBST Test Design course [source].

The main idea is that you can learn about the application by doing application tours. I will explain how variability tour helped me to identify potential problem in the application.

When you are doing application tours, from tools you only need a pen and paper. Simple as that. What counts are notes that you will put in your notebook.

Idea behind the variability tour is to find things that you can change in your product, and then try to change them. These are the moving parts of the application. And you will start to think about the application data that are part of those changes. What gets challenging here is that you realize that moving parts are connected and dependant on each other.

For better understanding this problem, see James Lyndsay’s black-box testing machines [source].

I tested a feature which purpose was to move application item through various states. This is known as State pattern [source]. Item has states and actions that are used to move item through those states.

I was doing coverage test of that feature. Create an item and move it through all available states. A variability tour. And I found an interesting feature that could be a possible issue. In every state, item had a same list of possible actions. There was a helping action, move an item to previous state. Very useful for the coverage test.

And I noticed that order of list of actions changed when item was returned to previous state from the order when item was first time in that state.

This was not the issue by itself, but by knowing that this feature exist, we could reveal some other important issue. This is example of over featuring the application. And developer did not know that this feature is implemented.

That was definitely issue for me as a tester because I needed to be extra careful to not make a mistake while I was doing coverage of item state transitions.

Conclusion

In my country, fresh graduates are not attracted by software testing, because they know about software testing from factory and control schools of software testing. By presenting them with application tours, maybe you will change their opinion about software testing.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to smart test minor version Ruby on Rails upgrade?

Reading Time: 1 minute

TL;DR

In this post I will explain what should be checked after Ruby on Rails minor version upgrade. Minor version upgrades are usually connected with security releases.

As I am subscribed to Ruby on Rails security Google group, when I receive information about latest security issue, I need to act very fast. Update must be pushed to production almost immediately, so there is no time for extensive regression testing.

How can we do quick test and be sure that everything still works as before upgrade?

Heuristic 1. Rails upgrade was extensively regression tested.

That heuristic proved itself to be always true.

Heuristic 2. Inspect Gemfile.lock to be sure that only Rails gem is upgraded.

How do we actually upgrade Ruby on Rails?

Edit gemfile:gem 'rails', '4.2.5.2'
bundle update rails

Investigate Gemfile.lock changes using git diff to see what else was updated beside Rails. If some other gem (not part of Rails) was also updated, check, using Google search, possible issues for that gem that are connected with Rails upgrade.

Heuristic 3. Search Google for Ruby on rails upgrade to n.n.n.n version issues (bugs, problems)

Conclusion.

For minor Ruby on Rails upgrades, using those three heuristics, you can do regression test in smart and quick way.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Be careful with docker upgrade

Reading Time: 1 minute

TL;DR

These days, updating software that we use is very easy. Installation scripts are automated and tested. In this post I will describe the issue caused with docker upgrade and how I analysed issue in order to resolve it.

In one of my previous post, I described how to create docker image for headless selenium chrome testing. And that worked for some time.

Then I needed to upgrade docker from version 1.9.1 to 1.10.2 because of this Docker 1.9.1 hanging at build step “Setting up ca-certificates-java” issue.

After upgrade, Chrome refused to start in docker image.

After I logged in to headless chrome docker container using configuration described in previous post:

>docker-compose run web bash
>google-chrome
>"Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted"

Lets try to downgrade docker. But how to do that? I remembered that every docker machine has docker version:

docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default - virtualbox Running tcp://192.168.99.100:2376 v1.10.2 
dev * virtualbox Running tcp://192.168.99.101:2376 v1.9.1

Which means that with docker upgrade, only default machine is automatically upgraded.

So I switched to dev machine:

eval "$(docker-machine env dev)"

And using docker-compose I was able to run again cucumber headless chrome test:

docker-compose run web /bundle/bin/cucumber features/feature_name

One last issue. docker tool was upgraded, so when you run:

docker ps -a
Error response from daemon: client is newer than server (client API version: 1.22, server API version: 1.21)

Here comes docker version manager to the rescue. Install it and run its help:

dvm help

Conclusion

Docker is complex technology, it consists of great number of moving parts. Be very carefull when you decide to upgrade docker, especially when around early versions like 1.10.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather