How browser automation helped me to find UI issue

Reading Time: 1 minute
image credit: www.macforensicslab.com

TLDR

There is important misunderstanding that Context driven school of testing is against test automation and programming in general. In following example I will give an example that will help to refute such opinion.

Here is one context driven principle in action:

“Automated testing is not automatic manual testing: it’s nonsensical to talk about automated tests as if they were automated human testing.”

I am writing browser automation framework for one of my client. Web application is one page application with “heavy” javascript involvement. Framework contains script that does following:

Given that I sign in
When I open xy application tab
Then there is xy amount shown
And I log out.

I wrote that script with following mission, to check how javascript behaves in repetitive action.
After I ran script, I was not able to login to application. UI interaction is blocked, which is not unusual when you have javascript driven UI. Browser refresh action was shortcut to make application operational again.

I tried to reproduce the issue without the script, but without success. Difference between human actions and script actions were in timings. Log out step was done in script much faster. And that was cause of issue.

While application tab was still loading, script did log out action (interrupt), which cause unhandled exception in javascript.

This example supports context driven principle, that automated testing extends human testing, it does not replace it.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Run Chrome with extension on SauceLabs

Reading Time: 1 minute

TLDR

In this post I will explain how to run chrome with extension on SauceLabs.

For one of my client I need to create webdriver automation scripts for web application that is javascript heavy. In order to identify html elements, I am using testability ids! chrome extension. We are also using SauceLabs in order to be able to test web application on chrome/windows7 combination.

Here is technology stack: cucumber->page-objects->watir-webdriver->webdriver->ruby

Heavy googling did not helped immediately because of this issue. Scroll to Ruby example. Can you spot the documentation UI issue? I also used git history feature to get important information. In the end I was able to see code example by looking at page source.

Here is the gist:

So, without the exploratory testing of ruby ChromeDriver documentation I would not be able to create automation script.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Could you guess what I discovered with load test?

Reading Time: 2 minutes
Credit: www.yachtingmonthly.com

TLDR
In this post I will describe how I discovered important product information using load test. And nobody involved in this product expected that we will find out this information.

For one of my client I performed and load test with goal to discover how much concurrent users could get satisfactory response times on production environment.

I coded load test script with agreed scenario using Grinder Load Testing Framework. I created test result reports using Grinder Analyzer. Both tools are opened sourced. Every measured request was POST http request with simple http authentication. Data format was json.

In order to prepare the load test, I had a meeting with all important stakeholders: project lead, test lead and developer lead. This is important, especially when you plan to do load test on production environment. I presented them my test strategy and requested feedback.

We agreed on user scenario where most important requirement was to have user/requests per second ratio equal to 1. Second important thing was to agree on load test time execution window time frame. With those information,  I was ready to execute load test on production environment.
I checked test script and test report by first running load test on test environment.

I run the test from my office in early hours and when I came to client site I got immediate questions from developer lead.

What do you know about transactions with date from the past? I got a call from person xy!
These are transactions generated by load test, I put that date in test data.

So first revealed information was that transaction date was taken from incoming data, not generated by application in real time. Which is bad because you must never trust the user data.

But who is person xy? Was asked by developer lead, not me. Why he has access to our production environment? I thought about Monty Python’s sketch: Nobody expects the Spanish inquisition.
We revealed second important information, who has access to production site and has business value in production data.

Because of person xy request, developer lead decided to delete all load test generated data. Which is not good pattern because second run of load test would have database with more data than test before it. And that also could reveal some interesting information (missing database indexes) about production site in next run of load test.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Export/import feature wrong design issue

Reading Time: 1 minute
credit: wikipedia.org

TLDR
In this post I will give an example for wrong design issue of export/import feature with the advocacy why is this an issue.

In order to prepare test data for performance test, I created test data using web application on test environment. This environment was used to check does performance grinder script works as expected. As web application had export/import feature, my plan was to export test data from test environment, and import them to production environment. My heuristic was that export/import feature was implemented as inverse function [wikipedia source].

But that heuristic failed. I needed to edit exported files in order to be able to import data back again in the application using import feature. Using HICCUPPS [Bolton as source] consistency analysis, this is an issue because:

  • comparable product (like blogger) use export/import feature in that way
  • reasonable user would expect that import/export feature is inverse action
  • this is export/import first purpose
So, If you find that export/import feature is not inverse function, you should raise that as an issue using HICCUPPS analysis as advocacy for this bug.
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Selenium webdriver issue report mapped to RIMGEA

Reading Time: 2 minutes
credit: seleniumhq.org

TLDR
In this post I will map instructions how to report selenium webdriver issue to RIMGEA acronym. Those instructions should be used in your project.

Selenium webdriver, also known as Selenium 2.0, introduced webdriver API that made driving a browser natively locally or remotely more transparent. As I was using selenium behind clients corporate firewall, I thought that I found an webdriver issue. As I finished AST Bug Advocacy  course, I used knowledge and craft learned and gained at that course to report an issue.

Selenium project has rules how to report an webdriver issue. In that way, they get better community bug reports. By following those instructions, we are using A from RIMGEA acronym, and that stands for Articulate. Bug report will be articulated by following those instructions.

In order to be sure that this is really webdriver issue, user should first search project frequently asked questions. This is to filter out wrong webdriver usage and configuration issues.

You need to describe the issue as accurate as possible by answering following questions:

  • What is the problem? By describing the problem, you are Articulating your bug report.
  •  Does it happen every time you run the test? If not, how frequently? Are you able to Replicate the issue?
  • Are others on your team experiencing the problem? This information will make your report more General.
  • A link to a github gist of any interesting stack traces. This will help to Replicate the issue. By why github gist? Gist is a simple way to share snippets and pastes with others. All gists are Git repositories, so they are automatically versioned, forkable and usable from Git. Using gists for exception stack trace will make you report more readable (Articulated).
  • Are you able to reproduce the failure on other browsers? This will make report more General.
  • Do you have other changes in your code that might be affecting this test? This will help you to Isolate the issue.
  • Which OS, browser and version of the browser are you using? This is vital. This information makes your report more General.
  • Ideally, a minimal test case against a public site we can run. Here are some templates you can use to create a test case: Ruby. Isolate your replication steps.
  • How to run the test (if there’s any special set up needed). Replicate information.
  • It is also very helpful if you can create a reduced test case. Isolate steps important for the issue.

If you think that bug reporting process in your company or project is not satisfactory, I suggest that you start with selenium webdriver template and tailor it to your project needs.

                •  
                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather