Category Archives: headless

headless, selenium-webdriver, testing tool

Selenium webdriver Net::ReadTimeout (Net::ReadTimeout) exception

Leave a comment
Reading Time: 1 minute

TL;DR

In this post I will explain how I resolved this exception in context of selenium webdriver testing in headless mode using xvfb on linux.

You finally managed to set up linux and xvfb server, your selenium webdriver settings are correct, and you successfully run your first scenario in headless mode! Astonishing achievement when I add fact that you only used open source technology.

You have your cucumber feature files with scenarios, written according every best BDD practice. Although, one feature file has more scenarios than other. Feature that it tests is rather complicated.

You set up jenkins job, and find out in the next morning of run, that only that feature file has failures. You examined execution html log, and find out this cryptic exception:

screen-shot-2017-01-07-at-11-56-08-am

Network timeout!? But site that you test is ready and online.

What is going on under the hood of selenium webdriver tests.

Selenium webdriver is http server. It accepts http connections, according to webdriver protocol that is http based. Your language selenium webdriver bindings start server for you when you create driver and your test script sends webdriver protocol commands to webdriver server. Simple as that.

Above exception means that your selenium webdriver server is not responding. Why? In context of headless xvfb linux test run, this could be caused by various reasons.

Solution.

I noticed that exception appeared always on same scenario, so I split long cucumber feature file in two. If you have properly written feature file according to BDD practices, this should not affect your test suite.

I guess My heuristic is that selenium webdriver server could be up for some limited time and that with long feature file I hit that limit.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
cucumber, headless, learn testing, page object gem, ruby, selenium-webdriver, testing tool, UI regression

Value of UI automation tests

2 Comments
Reading Time: 1 minute

TL;DR

After series of posts about How to kill UI regression testing, maybe you got impression that I am against UI browser automation test scripts. In this post, I will put my thoughts about the value of UI automation tests.

First some context. Here are requirements for excellent UI automation tests:

  • Language that enables you to write less code. Something like Ruby.
  • selenium web driver implementation in that language
  • framework just above selenium webdriver. For example watir-webdriver.
  • Page object pattern implementation
  • test framework, something like rspec. For writing awesome asserts.
  • gherkin implementation. Something like  cucumber. No, I have not changed my mind stated here. But we have different context here.
  • continuous integration tool. Something like jenkins
  • headless browser. Something like xvfb or sauce labs.

What is the value of UI automation test scripts? They represent executable documentation for your product. They help you to answer questions about complicated business scenarios. For example facebook  setting who can contact me.

Your testers will need to have a skill of writing DRY tests using those requirements. So when you have new tester on board, by reading those tests script, he can quickly learn application features.

Those tests would be run on jenkins and run should be triggered manually. Tests will provide you information is some automated feature changed. It is not wise to use those results for you regression testing strategy.

Because those checks, or asserts only check what is coded. Nothing else. And UI is for human users, not for test scripts. So human should always do smart regression testing , using its brain, eyes, ears and fingers.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
cucumber, docker, headless, learn testing, selenium-webdriver

Be careful with docker upgrade

Leave a comment
Reading Time: 1 minute

TL;DR

These days, updating software that we use is very easy. Installation scripts are automated and tested. In this post I will describe the issue caused with docker upgrade and how I analysed issue in order to resolve it.

In one of my previous post, I described how to create docker image for headless selenium chrome testing. And that worked for some time.

Then I needed to upgrade docker from version 1.9.1 to 1.10.2 because of this Docker 1.9.1 hanging at build step “Setting up ca-certificates-java” issue.

After upgrade, Chrome refused to start in docker image.

After I logged in to headless chrome docker container using configuration described in previous post:

>docker-compose run web bash
>google-chrome
>"Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted"

Lets try to downgrade docker. But how to do that? I remembered that every docker machine has docker version:

docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default - virtualbox Running tcp://192.168.99.100:2376 v1.10.2 
dev * virtualbox Running tcp://192.168.99.101:2376 v1.9.1

Which means that with docker upgrade, only default machine is automatically upgraded.

So I switched to dev machine:

eval "$(docker-machine env dev)"

And using docker-compose I was able to run again cucumber headless chrome test:

docker-compose run web /bundle/bin/cucumber features/feature_name

One last issue. docker tool was upgraded, so when you run:

docker ps -a
Error response from daemon: client is newer than server (client API version: 1.22, server API version: 1.21)

Here comes docker version manager to the rescue. Install it and run its help:

dvm help

Conclusion

Docker is complex technology, it consists of great number of moving parts. Be very carefull when you decide to upgrade docker, especially when around early versions like 1.10.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
cucumber, docker, headless, testing tool, webdriver

Headless browser testing in docker container

3 Comments
Reading Time: 2 minutes

TL;DR

In this post you will find out how to run headless chrome browser tests on your development machine using prepared and configured docker machine.

Docker enables you to run on your development machine minimal linux machine. You are in control which software will be installed in that linux machine. All linux machine configuration is stored in two files. Docker uses cache, so only changes in machine configuration are rerun.

Install Docker

First, you need to install Docker for your operating system. Fire up docker terminal after you installed it.

Type: docker-machine ls

Output:

Screen Shot 2016-02-14 at 11.11.42 AM

 

docker machine is linux (special docker flavour), that has docker software that will run your docker containers. It runs in virtualbox that is automatically installed with docker.

You can ssh to it:

 

Screen Shot 2016-02-14 at 11.19.15 AM

 

Clone this github repository that contains docker files needed for creating and running docker image. Follow the README instructions. And that it is, you can run headless chrome tests on your development machine. While those tests are running, you can use your development machine for other tasks.

Dockerfile

Here is official Dockerfile reference documentation. Dockerfile is source code for creating docker images. But I would like to save you time that I spent investigating the official documentation, so you will be ready to create your own docker images very quickly.

Important note: in order to be able to create your own docker images, previously you have successfully configured and installed linux, connected to linux machine, and you know your way around in linux shell.

You do not have to create Dockerfile from scratch. There is great Dockerfile repository, called Docker hub. In search box input what you need, e.g headless chrome, and there is high probability that somebody already have prepared what you are looking for.

If this is not the case, you can download Dockerfile and change it according to your needs. Check in Dockerfile reference what every docker command actually do.

docker-compose.yml

This is second file from docker ecosystem. It is a source code for creating docker containers and their interconnections based on docker machines. For example, WordPress site needs (according to good architecture practice) php container and mysql container.

Line 4 in docker-compose.yml maps current directory (your ui automation project folder) to web docker container app folder. That is the magic which enables docker to run your cucumber tests in docker linux container that is ready with headless chrome, something that is not possible on osx.

Volumes are important docker feature, because without them, docker container does not remember any changes.

Additional magic is that bundle install is run on docker-compose build only when Gemfile is changed.

For example:

  1. docker-compose run web bash
  2. cd ~
  3. touch text.txt
  4. exit
  5. docker-compose run web bash
  6. ls -al ~

There is no text.txt file!

Here is composer reference documentation.

Here are instructions how to start testing using headless chrome:

  1. in docker-composer.yml, replace {project_name} with name of your ui automation project folder name.
  2. copy Dockerfile, docker-entrypoint.sh and docker-composer.yml to your ui automation project folder.
  3. start docker terminal, cd to ui automation project folder.
  4. run docker-compose build –force-rm
  5. run your tests: docker-compose run web /bundle/bin/cucumber -f html -o chrome.html features/feature_name.feature

chrome.html file is created in your local folder, in root folder of ui automation repository!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
AMI Linux, Firefox, headless, testing tool

Runing Firefox on AMI Linux instance using Jenkins

Leave a comment
Reading Time: 3 minutes

TL;DR

In this post I will give step by step instructions for setting up Jenkins on AMI Linux host. Jenkins will run selenium webdriver tests on Firefox in headless mode.

Install Jenkins

1. `sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo`
2. `sudo rpm –import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key`
3. `sudo yum install jenkins`
4. `sudo yum remove java`
5. check that `java -version` is something like OpenJDK 64-Bit Server VM
6. check that jenkins runs on 8081
`sudo cat /etc/sysconfig/jenkins | grep JENKINS_PORT`
JENKINS_PORT=”8081″
7. `sudo service jenkins start`
8. `curl http://localhost:8081` should return jenkins home html page

Control Jenkins

`sudo service jenkins start/stop/restart`
log file `/var/log/jenkins/jenkins.log`
config file `/etc/sysconfig/jenkins`

Rvm

  1. `sudo yum install git`
  2. `sudo -u jenkins /bin/bash –login`
  3. `curl -sSL https://get.rvm.io | bash`
  4. `rvm install ruby_from_dot_ruby-version_file`

Firefox

Install it as jenkins user:
`sudo -u jenkins bash`

Installation instructions.

xvfb

`sudo yum install xorg-x11-server-Xvfb`

Check xvfb

first terminal:
`Xvfb :1 -screen 0 1280x768x24`

another terminal:
`export DISPLAY=:1`
`firefox`

**Note**
It is known that after sudo yum update, xvfb will fail with following error.

`Xvfb: symbol lookup error: Xvfb: undefined symbol: pixman_glyph_cache_create`

And console with firefox will report:

`Xvfb: symbol lookup error: Xvfb: undefined symbol: pixman_glyph_cache_create`

In order to resolve this issue, you need to run:

`rm /usr/local/lib/libpixman-1.so.0`

 

Jenkins security

Follow instructions from paragraph Password protect Jenkins.

## Jenkins plugins

Paragraph Install Jenkins plugins.
Install rvm plugin.

 

Configure Jenkins to run cucumber tests

0. become jenkins user: `sudo -u jenkins /bin/bash –login`
in `/var/lib/jenkins` folder, create ssh key.
copy `/var/lib/jenkins/.ssh/id_rsa.pub` to github deploy keys.
1. login to jenkins web console (before create your account)
2. create new job
3. project name: unique_project_name_by_your_decision
4. check discard old builds, strategy log rotation, max # of builds to keep 10.
5. source code management is git, your repository url is your git repository *.git
6. credentials: none
7. branches to build: */master (or whatever you like)
8. Build environment, check run the build in a RVM-managed environment
9. Implementation: e.g. ruby-2.1.2@gemsetname
10. ssh agent check
11. select credentials jenkins
12. add credential jenkins if not present
13. scope: global, type: ssh private key, username: jenkins, private key from a file on jenkins master: /var/lib/jenkins/.ssh/id_rsa
14. add build step: execute shell, `cd $WORKSPACE`
15. add build step: execute shell, `bundle install`
16. add build step: execute shell, `cucumber features -f html – o reports/report_$BUILD_TAG.html`
17. save
18. `ssh to AMI linux instance
19. `sudo -u jenkins /bin/bash –login`
20. Only first time create your test configuration files that are not present in git repository. E.g. application.yml file (repository README.md has content instructions ) in
`vi /var/lib/jenkins/jobs/unique_project_name_by_your_decision/workspace/config/application.yml`
21. You can check cucumber execution report in workspace/reports

**Tip and tricks**

Once you have one jenkins job, in order to create another one, just use feature `copy existing item` in New Item feature. Edit configuration as appropriate and hit OK. Run the job that will fail. After that, do action from step 21. and task is ready.

Jenkins rest api

1. first obtain csrf crumb
`curl http://jenkins_username:jenkins_password@10.30.105.232:8081/crumbIssuer/api/json`
you will get:
`{“crumb”:”f4828a00ccfe682b59b3977f8fdc8134″,”crumbRequestField”:”.crumb”}`
2. start jenkins job:
`curl http://jenkins_username:jenkins_password@10.30.105.232:8081/job/job_name/build?token=jenkins_username_api_token -H “.crumb:f4828a00ccfe682b59b3977f8fdc8134” –data “”`
jenkins_username, password and api token are you jenkins credentials.

Runing firefox in headless mode

`sudo Xvfb :10 -ac` – start Xvfb server.
`export DISPLAY=:10`

This should be part of Jenkins job configuration.

 

Ant that it is, your AMI linux is ready to run your cucumber selenium webdriver tests in headless mode.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather