Category Archives: selenium-webdriver

Selenium webdriver Net::ReadTimeout (Net::ReadTimeout) exception

Reading Time: 1 minute

TL;DR

In this post I will explain how I resolved this exception in context of selenium webdriver testing in headless mode using xvfb on linux.

You finally managed to set up linux and xvfb server, your selenium webdriver settings are correct, and you successfully run your first scenario in headless mode! Astonishing achievement when I add fact that you only used open source technology.

You have your cucumber feature files with scenarios, written according every best BDD practice. Although, one feature file has more scenarios than other. Feature that it tests is rather complicated.

You set up jenkins job, and find out in the next morning of run, that only that feature file has failures. You examined execution html log, and find out this cryptic exception:

screen-shot-2017-01-07-at-11-56-08-am

Network timeout!? But site that you test is ready and online.

What is going on under the hood of selenium webdriver tests.

Selenium webdriver is http server. It accepts http connections, according to webdriver protocol that is http based. Your language selenium webdriver bindings start server for you when you create driver and your test script sends webdriver protocol commands to webdriver server. Simple as that.

Above exception means that your selenium webdriver server is not responding. Why? In context of headless xvfb linux test run, this could be caused by various reasons.

Solution.

I noticed that exception appeared always on same scenario, so I split long cucumber feature file in two. If you have properly written feature file according to BDD practices, this should not affect your test suite.

I guess My heuristic is that selenium webdriver server could be up for some limited time and that with long feature file I hit that limit.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Value of UI automation tests

Reading Time: 1 minute

TL;DR

After series of posts about How to kill UI regression testing, maybe you got impression that I am against UI browser automation test scripts. In this post, I will put my thoughts about the value of UI automation tests.

First some context. Here are requirements for excellent UI automation tests:

  • Language that enables you to write less code. Something like Ruby.
  • selenium web driver implementation in that language
  • framework just above selenium webdriver. For example watir-webdriver.
  • Page object pattern implementation
  • test framework, something like rspec. For writing awesome asserts.
  • gherkin implementation. Something like  cucumber. No, I have not changed my mind stated here. But we have different context here.
  • continuous integration tool. Something like jenkins
  • headless browser. Something like xvfb or sauce labs.

What is the value of UI automation test scripts? They represent executable documentation for your product. They help you to answer questions about complicated business scenarios. For example facebook  setting who can contact me.

Your testers will need to have a skill of writing DRY tests using those requirements. So when you have new tester on board, by reading those tests script, he can quickly learn application features.

Those tests would be run on jenkins and run should be triggered manually. Tests will provide you information is some automated feature changed. It is not wise to use those results for you regression testing strategy.

Because those checks, or asserts only check what is coded. Nothing else. And UI is for human users, not for test scripts. So human should always do smart regression testing , using its brain, eyes, ears and fingers.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Be careful with docker upgrade

Reading Time: 1 minute

TL;DR

These days, updating software that we use is very easy. Installation scripts are automated and tested. In this post I will describe the issue caused with docker upgrade and how I analysed issue in order to resolve it.

In one of my previous post, I described how to create docker image for headless selenium chrome testing. And that worked for some time.

Then I needed to upgrade docker from version 1.9.1 to 1.10.2 because of this Docker 1.9.1 hanging at build step “Setting up ca-certificates-java” issue.

After upgrade, Chrome refused to start in docker image.

After I logged in to headless chrome docker container using configuration described in previous post:

>docker-compose run web bash
>google-chrome
>"Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted"

Lets try to downgrade docker. But how to do that? I remembered that every docker machine has docker version:

docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default - virtualbox Running tcp://192.168.99.100:2376 v1.10.2 
dev * virtualbox Running tcp://192.168.99.101:2376 v1.9.1

Which means that with docker upgrade, only default machine is automatically upgraded.

So I switched to dev machine:

eval "$(docker-machine env dev)"

And using docker-compose I was able to run again cucumber headless chrome test:

docker-compose run web /bundle/bin/cucumber features/feature_name

One last issue. docker tool was upgraded, so when you run:

docker ps -a
Error response from daemon: client is newer than server (client API version: 1.22, server API version: 1.21)

Here comes docker version manager to the rescue. Install it and run its help:

dvm help

Conclusion

Docker is complex technology, it consists of great number of moving parts. Be very carefull when you decide to upgrade docker, especially when around early versions like 1.10.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to automate check for youtube video subtitle

Reading Time: 1 minute

TL;DR

In this post I will explain how I implemented check that youtube video contains specific language subtitle.

As part of application acceptance test, I needed to implement UI check that confirms that selected youtube video contains selected subtitle language. Getting the value of youtube video available subtitles from youtube player would be hard (even not manageable) task. So I decided to use different approach.

I remembered that while “search videos by subtitle language” feature was developed, developers mentioned (in github issue conversation) that this feature will be based on youtube api. So why not use this same code in my automated check?

The risk is here that we are using api that could return different list of available subtitles than one that are obtained using youtube UI elements. But knowing that Google is very good at testing their product features, I labeled that risk as very low.

youtube video subtitles
youtube video subtitles

Here is cucumber step definition:

It is important to state that I found on this blog post how to get attribute value using watir-webdriver.

Always use your tools wisely. Do not force one way of achieving something if there is more elegant way.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to automate page with dynamic elements

Reading Time: 1 minute

TL;DR

In this post I will explain how to examine dynamic html elements in order to be able to automate web page that contains such an element.

 

Many of modern web applications uses heavily some of the popular Javascript frameworks (Ember, Angular are some of them). In that way, your web application is more interactive and dynamic.

For example, user does some action and it gets message that disappears after few seconds. Very effective from the UX point, but how to automate that scenario?

The problem here is how to examine html structure of disappearing message in order to find out element id (unlikely to be present) or class attribute?

In order to do that, you can use chrome developer tools, the sources panel. You need to debug javascript code that shows and removes application message. You can do that very simply with javascript debugger pause button.

 

Here is what you should do:

  1. open your web application in Chrome
  2. open Chrome developer tools
  3. open Sources tab
  4. make action on your web application that will trigger dynamic application message
  5. when message appears, hit the pause button
  6. go to elements tab and examine the message html attributes
  7. when you are finished, go back to sources tab and press continue button

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to send shift and special characters in Selenium Webdriver with Ruby

Reading Time: 1 minute

TL;DR

In order to automate one browser scenario, I needed to send ampersand character. It seems that this is not an easy action in selenium webdriver. So, instead testing the web application, I spent some time in order to find out a hack how to send ampersand key to browser in programmatic way.

My first reaction was: Is this for real, selenium-driver ate ampersand character!? Google as helpful, I found following blog post how to resolve this issue. Solution is in Python, so I rewrote it in Ruby.

Conclusion

Instead of trying to find the solution how to automatically send ampersand in ruby to browser, instead I could do that “manually” and continue the real testing activity. So, my dear testers, when you hear developers claiming that they test their application, prove them wrong. They just try to find solutions how to use automation framework, and that has nothing with sapient activity, software testing!

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Tweeting with Firefox is different than tweeting with Chrome

Reading Time: 1 minute

TL;DR

In this post I will explain how simple change in testing environment influenced my testing of simple feature.

Feature is rather simple, tweet something interesting from the application. As this feature is tested, I was creating automated script for it. This feature is important for my client, so I decided that it should be part of regression testing suite.

I developed script for Chrome, and I also decided to check it using Firefox. Remember that in RIMGEA we learned that G stands for Generalize. We need to test our feature by changing the environment, in this case browser itself.

And in Firefox script failed! Why? Because Firefox tweet feature has one extra step in domain of twitter application.

 

firefox_tweet_3

 

It is clear that Firefox requires one extra button click. Since prior to that click, user clicks on “Log in and tweet” button, this is not consistent with that button. User expects that with one click he does two actions, log in to twitter and tweet that item.

chrome_tweet_2

 

 

With this post I amplified on real case how important is to vary your testing environment because in that way you could find interesting issues within the application.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to automate remember me feature using selenium webdriver and cucumber

Reading Time: 2 minutes

TL;DR

 

In this post I will explain problems I encountered and how I resolved them while I was designing automated test for remember me feature.

 

Context of this post is automation of end to end feature that are visible to user via the browser. You will see remember me checkbox on your login page, but that is the point where visibility of this feature ends.

 

How remember me feature actually works?

If you do not check remember me, then when you close browser after you have finished using web application, next time you come back to the application, you will need to login in again.

 

If you check remember me, then when you close browser and come back again to your application, you will be automatically logged in.

Magic behind this feature are cookies. Cookie contains session for your application and information how long that session in cookie is valid. Server side creates the cookie, but browser must act based on the information in that cookie.

 

So when you test remember me feature, you actually test browser and web application features.

 

First issue in test design

 

In order to have faster execution of browser automation tests, I do not close browser between scenarios (Watirmelon first page object anti pattern). But in order to test remember me feature, you need to close the browser. Here is the code that should be put in cucumber env.rb file that resolves this issue:

 

Second issue is scenario run order dependency

 

But that solution introduces another anti pattern, now scenario run from remember me feature file must be always run in same order:

 

 

That means you can not use following cucumber run option:

 

cucumber –order random :5738 remember_me.feature

Here is step that loads cookies:

 

Third issue selenium webdriver cookie domains

In my first implementation, in env.rb file, I just saved all browser cookies to file. But when I loaded those cookies, i got following exception:

 

 

Note: By observing that exception I learned that Firefox webdriver is actually Firefox extension written in javascript. That extension is automatically installed in Firefox before test run.

And here is the line of code that produces exception:

 

 

Observe comment in line 8 🙂 For me it implies that developer did not know how to properly implement cookie domain check. But as it is open source project, we will forgive him.

 

The issue here was that Google Analytics cookies had subdomain:

.app.domain.com

as cookie domain value. As application domain was:

app.domain.com

Google Analytics cookie domain was rejected in firefox webdriver. So before saving cookie values in file, I deleted Google Analytics cookies.

 

Conclusion

 

I am moving this blog from Blogger. While I was publishing this post, I discovered that WordPress needs plugin in order to embed gists to posts. Blogger had that feature out of the post.

 

Finding proper WordPress heuristic: install one that has more stars.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather