Web page cache heuristic that failed

Reading Time: 1 minute

TL;DR

In this post I will describe my heuristic for web page cache testing that failed and why it failed.

Web page cache is mechanism that stores parts of web page that does not change a lot in browser local storage. Hard part is to figure which part of page is not changing very often. A lot of scientific effort have been put in this problem.

I got a Jira card that clearly stated a story with cache usage.

During the creation of business object, cache all referenced data so when user will edit that object, all referenced data is already available.

I did exploratory test around this story, using my brain, eyes, ears and fingers. There were two paths to feature create business object. First path worked as described in user story, but second path did not. When I opened the create business object feature, all data had already been cached.

My first heuristic was that this is an issue. But after I explored the pull request connected with this card and discovered that all code changes were in one place I realized that my cache heuristic failed. Developers are using framework that helps them to write DRY code.

As create business feature component was reusable, first path to that feature already filled cache with data.

If you are in doubt how to enhance your testing craft, BBST is excellent starting point.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

What I learned on Testival #27 meetup

Reading Time: 1 minute

TL;DR

This post is report about what I learned at Testival #27 meetup and how I provoked that learning.

Last week we had Testival #27 meetup. Attendant metric was 15/25, which is excellent 0.6! You probably have seen something like this metric in your test reports. 0.6? So what that number means? We recently moved to meetup platform which exposed Testival to local community. 15 testers showed, and 25 applied to meetup. We have total of 65 members in Testival group. This is great improvement for Testival.

Sponsor of this meetup was Repsly, and we met again in HUB385. Repsly is hiring for a QA position. After short introduction of all participants,  we started with lectures and lighting talks.

My takeaways.

  1. Ana showed hers PageObject implementation in javascript. Ruby has PageObject gem with lot of out of the box helpers that you need to develop by yourself in javascript.
  2. Zeljko showed javascript source code for browser automation in various framework. It is less readable than Ruby code and it seems that Ruby gem ecosystem is stronger than one in javascript.
  3. Between two sessions I heard interesting question, how to check large tables using selenium (I always mingle in listening mode in every meetup), and Zeljko mentioned that he successfully used dedicated htl parser for that purpose, not the Watir-webdriver implementation.
  4. I was very impressed how Scratch evolved in past few years.
  5. There is jenkins plugin that enables you to keep history and version of your application builds.
  6. Over beers I learned on real example how is very hard to manage team of programmers. There is no silver bullet, managing a group of programmers is very hard.

Ruby : Javascript in browser automation context 3:0.

Here is small gallery:

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to change docker default route on Centos7

Reading Time: 2 minutes

TL;DR

In this post I will explain how to resolve following issue. You are connected via ssh to your centOS box. You successfully installed docker on that box and you started docker daemon using systemctl. Bang!, you are disconnected from your centOS box.

This happened to me. I had successfully installed docker on several boxes without a problem. Reality check, I just scratched the surface of my docker knowledge.

Reality check. This centos box was different than previous ones. I found out that it is in different network. Luckily, there was another box in same network that I can connect to. I ssh to that box and then from that box to the original centOS box. I  am in!

I was able to log in because they were in same network subnet. My laptop was in different range. What to do in order not to be bitten by this issue?

Before you start docker daemon with systemctl, run following cmd:

[ ~]$ route -n

route shows/manipulates routing table. Routing table describe where IP packets should go. Picture above shows ok situation. docker and linux box have different destination for packets.

In above example, docker took 172.17.0.0 by default [source]. It reserves 65536 IP addresses for containers.

In my case, centOS box was in range 172.29., and docker took 172.29.1.0 so from outside, docker took all the traffic, and my laptop was automatically disconnected.

How to change docker cidr range.

Before you start your docker daemon:

  1. run route -n
  2. check your box eth0 destination
  3. edit docker systemctl start file:
vi /usr/lib/systemd/system/docker.service

4. put docker containers in different ip range

Here is example how to set docker ip range from 172.29.1.0 – 172.29.1.255 [cidr calculator]:

ExecStart=/usr/bin/dockerd -bip 172.29.1.1/24

5. Now is save to start docker daemon:

# systemctl daemon-reload
# systemctl start docker
#systemctl status docker
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Docker behind a proxy

Reading Time: 1 minute

TL;DR

In this post I will explain how to set up docker when you are behind corporate proxy.

All Internet tutorials about any technology take assumption that you are not behind a proxy. Working for my client that must use corporate proxy, I spent a lot of working hours discovering how to use docker behind  a proxy.

You first need to find out value for corporate proxy. In corporate environment, this could be rather tricky action, because that value is not always publicly known.

  1. Set docker systemd settings to use proxy.

Use official docker instructions [source]. With this setting, docker daemon nows how to get images from the Internet.

2. Dockerfile.

If you need to tailor your own image, you need to use Dockerfile. In order to enable Dockerfile commands to get stuff from the Internet, you need to set ENV HTTP_PROXY variable.

After FROM directive in Dockerfile, put following settings:

FROM image_name
ENV http_proxy <HTTP_PROXY>
ENV https_proxy <HTTPS_PROXY>

3. Some tools do not use environment variable http_proxy.

There are some tools that made a wrong design decision where they do not use environment variable http_proxy, but they require that command switch must be used in order to set proxy value.

I broke my teeth on following tools:

  • pip
  • npm

Here is simple heuristic. If command from Dockerfile gets timeout (usually it appears to hang when it can not get to Internet), consult documentation for this command to find out how to set proxy value.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to replace curl data-binary value to form-data upload parameters

Reading Time: 1 minute

TL;DR

In this post I will explain how master Joda visited me in my dream and asked me important question about one of my previous blog posts: How to measure upload speed of web application upload form [link]. The post provided right information about curl parameter that enables you to see upload speed. But curl example for upload file is wrong!

I found that is wrong when I tried to measure upload speed in real application. And realized that file is not uploaded in application. By observing the copy pasted curl, I noticed that there was not Content-Length header. Bingo, lets create Chrome developer tools issue ticket[link].

Please read comments, because I learned a lot about curl upload file feature. Google developers who answered my ticket did not just dismissed my wrong curl upload file heuristic, they took extra step and educated me!

So, in order to use curl upload file feature, you need to transform something like this:

--data-binary $'------WebKitFormBoundaryoeJQRnHeojsIBayU\r\nContent-Disposition: form-data; name="fileType"\r\n\r\n5\r\n------WebKitFormBoundaryoeJQRnHeojsIBayU\r\nContent-Disposition: form-data; name="file"; filename="elixir-websetup.exe"\r\nContent-Type: application/x-msdownload\r\n\r\n\r\n------WebKitFormBoundaryoeJQRnHeojsIBayU--\r\n'

into this:

-F file=@elixir-websetup.exe -F fileType=5

You can now data-binary parameter and its value from curl.

It is important to have check mechanism for upload file. Use your application features to check is file really uploaded. curl can also help by observing application returned value.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

What is happening in our application?

Reading Time: 1 minute

TL;DR

In this post, I will try to elaborate why is often expected from me as a tester to create meaningful application error messages.

So, you are using some web application, and you get something like this( thanks to Evil Tester):

Screen Shot 2017-01-14 at 2.04.02 PM

And that error message does not tell you anything about your action or input. At least they put disclaimer: “That is all we know”.

While I am testing application, I find a lot of such not informative error messages. Why I find them and developers not? Because I know how to make reasonable scenarios that trigger those errors. And to come up with such scenarios and test data is not something of interest of regular developer. In my experience they find this very boring and not creative job. But I think that finding alternative flows by exploring api that you use is very creative and important job.

Developers just have different meaning for definition of DONE. Or they are just lazy.

First line of defense is that user will never do that. But you took BBST Bug advocacy, and you have already uncornered your test scenarios making it more probable in real life.

After I talk to project leader and provide him information about low quality error messages, I put extra work on my shoulders. Karlo, as you already found those error messages, you will create meaningful error messages.

The problem here is that I can only provide error messages for scenarios that I covered. But low level mechanics of the application, which includes inter module cooperation based on api contracts, this is developer responsibility.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Selenium webdriver Net::ReadTimeout (Net::ReadTimeout) exception

Reading Time: 1 minute

TL;DR

In this post I will explain how I resolved this exception in context of selenium webdriver testing in headless mode using xvfb on linux.

You finally managed to set up linux and xvfb server, your selenium webdriver settings are correct, and you successfully run your first scenario in headless mode! Astonishing achievement when I add fact that you only used open source technology.

You have your cucumber feature files with scenarios, written according every best BDD practice. Although, one feature file has more scenarios than other. Feature that it tests is rather complicated.

You set up jenkins job, and find out in the next morning of run, that only that feature file has failures. You examined execution html log, and find out this cryptic exception:

screen-shot-2017-01-07-at-11-56-08-am

Network timeout!? But site that you test is ready and online.

What is going on under the hood of selenium webdriver tests.

Selenium webdriver is http server. It accepts http connections, according to webdriver protocol that is http based. Your language selenium webdriver bindings start server for you when you create driver and your test script sends webdriver protocol commands to webdriver server. Simple as that.

Above exception means that your selenium webdriver server is not responding. Why? In context of headless xvfb linux test run, this could be caused by various reasons.

Solution.

I noticed that exception appeared always on same scenario, so I split long cucumber feature file in two. If you have properly written feature file according to BDD practices, this should not affect your test suite.

I guess My heuristic is that selenium webdriver server could be up for some limited time and that with long feature file I hit that limit.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Meet Blitzy: not so simple HTTP load tester in Elixir

Reading Time: 2 minutes

TL;DR

In this post I present Blitzy, not so simple HTTP load tester in Elixir. This is my open source contribution to original Blitzy, simple HTTP load tester in Elixir.

My journey with Elixir started six months ago. With proper book from The Pragmatic Bookshelf, this was fun experience. I made myself to do all exercises, but in the end I skipped Macro exercises. How to test my Elixir practical knowledge? The best way is to contribute to some of Elixir open source project.

For one of my clients, I need to do load, duration and concurrent test. I previously used Grinder, Java Load Test Framework. Driven by pure curiosity, I searched for load testing tool in Elixir, and I found Blitzy.

It has simple feature set, do load test for simple http get. By exploring the code and using Programming Elixir 1.3 as a reference, I found out that I can read and build Elixir project code. So, why not add features to Blitzy that I need for my current project.

My first contribution was to update dependencies, and do some refactor because some libs changed their interfaces. This was need just to build Blitzy for current set of features.

In closing session of CAST 2016, there was discussion how can we help to enhance software testing. My contribution, lets create software testing tool, that can be easy to extend, develop and maintain. Another pointer for Blitzy contribution.

Power of Blitzy is that is programmed in Elixir, which uses Erlang virtual machine. Downsize of Erlang is rather cryptic programming language, it feels that you are writing mathematical formulas and proofs, and not a program. But there is Elixir, that resolves that issue.

Blitzy codebase is small, because it uses Erlang virtual machine concurrency and code distribution out of the box features.

I remember that most of the code for Grinder was about those features, distribution and concurrency.

Check github repository, and happy load testing.

I wrote tests for Blitzy, currently line coverage is more that 80%, but I have not wrote most important test that include data. How about that metric preachers? Important unit test are on to do list.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Technology junky

Reading Time: 2 minutes

TL;DR

In this blog post I will describe my TV upgrade that should last for next 10 years.

This summer was Football UEFA Euro competition year in France. Heuristic is that you could by new TV models with significant discount. I bought my last TV in 2008, and it was plasma. TV technology have dramatically changed in those eight years and I did know where to start. My Christmas started earlier this year. I asked my friend, who is real TV junky, for help. He needed just one input, my budget. With my budget of 1500 eur, he started his research. Believe me, it is not an easy task. After few pointers and one week, he hit a jackpot, LG-49UH8507, in my budget range.

This is my short review wearing a testers hat. Basically, I bought computer system . Every computer needs operating system.

WebOS 3.0

You can find technical specification here. From user perspective it is very fast and reliable. UX is pretty good, menus and actions are strait forward. I enabled auto upgrade which is done in sleep mode. When I turn it on, I get notification in upper right corner about the upgrade.

Network

WiFi and Ethernet.

I use ethernet for streaming services. Big surprise was when my Iphone 6 offered to connect to TV while I was watching Youtube. It is called screen sharing.

Remote

It has magic mouse feature, which means mouse pointer that I steer remotely. Remote UX is not so good, because I mostly use it in dark room and I can not guess button that I need based on its position or texture structure.

Display

UltraHD which means 4K resolution. This is Netflix Dardevil in 4K (click for larger image):

img_1231

Entertainment and education

I watch 90% of my time Netflix or HBOGo. Up till last update, Netflix had stability issue. I always exited application, but when I reentered it, it always crashed. HBOGo has streaming issue with new items, but it is stable. Netflix has much batter UX experience when I continue watching where I stopped previously. No streaming issues at all even with 4K items.

There is also youtube player, so I can educate myself from the conform of my sofa.

I am very satisfied with my new gadget. For the same price, I also got LG G3 smartphone, which was big surprise! And salesperson was very educated about the product, he confirmed all technical specs that I knew thanks to my friend.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Cleanup order in selenium headless test

Reading Time: 1 minute

TL;DR

In this post I describe on real example how to use important question for every software tester: “Did I do it in proper order?”

For one client I created environment for running selenium webdriver test suite in headless mode for firefox browser. I use docker image that contains latest Jenkins, xvfb server and firefox browser. I was not able to start chrome with xvfb because chrome has much higher security. More about that environment in following posts.

When I run test suite, I got following cryptic exception:

Failure/Error: Unable to find matching line from backtrace
     Selenium::WebDriver::Error::UnknownError:
       Failed to decode response from marionette
     # ./spec/spec_helper.rb:93:in `block (2 levels) in <top (required)>'

Line that triggered exception was:

@driver.quit

With help of this excellent blog post,  I find how to ask right question: “Did I use proper order of calling different api methods?”

I first closed headless driver (which is xvfb server), and then I tried to close selenium webdriver. That lost connection to its X server manager (xvfb).

After I first closed selenium webdriver, and after it I closed headless xvfb server, test suite worked as expected.

When you investigate what went wrong, use this simple question:

“Did I use proper order?”

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Blog that makes software testing interesting and exciting.