Category Archives: testing tool

Be careful with your testing tools

Reading Time: 2 minutes

TL;DR

In this post I will describe how I got different test results using two testing tools Bug Magnet and Counter strings.

Bug Magnet is handy Chrome Extension developed by Gojko Adzic.

It is: “Convenient access to common boundaries and edge cases for exploratory testing.

Counterstings is implementation of James Bach algorithm that quickly reveals character position in string.

In my test, I used them to check maximal number of allowed characters in input form. This test should be simple, right?

One interesting thing happened. I first used Bug Magnet and inserted string with 128 characters. String was accepted and test failed because maximal allowed number should be 64 characters. I did second test with counterstrings. Boundary of 64 characters was successfully detected. What!?

What happened?

For me, any test should provoke sapient thinking process. In modern web applications, input form could be implemented in two ways:

  • html 5 standard, where all input constraint checks are done by browser code according to that standard

You can recognize this by inspecting input text box (Chrome developer tools) and if you see something like this:

<form action="/action_page.php">
  Username: <input type="text" name="usrname" maxlength="10"><br>
  <input type="submit" value="Submit">
</form>

then input textbox is implemented in pure html5.

  • using one of popular javascript frameworks (Angular for example, and that was the case with application that I was testing).

Then there is javascript code triggered on keyboard typing action. That code usually manipulates DOM tree to create better looking results.

Application created using modern javascript framework.

Bug Magnet is implemented in javascript, and it manipulates DOM tree in order to insert the test data. Be careful, because application user does not do that. Application user types directly into input box. Doing that manipulation, Bug Magnet managed to avoid string length check.

Counterstring uses Ruby on Rails code to generate counterstring on server side. It uses javascript to copy/paste counterstring to user clipboard. My input was done with copy/paste, something more closer to application user.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to expose your reading list using rss

Reading Time: 1 minute

TL;DR

This post is about how I manage blog posts reading list. It list tools that I use on daily basis to collect and share what I read.

My first try was twitter. I use it for collecting my blog post reading list by following software testers that blog about software development and testing topics. Here is my following list.

But very soon, twitter failed me. As they started pivoting their business model, very soon I received in my twitter feed a lot of noise (shares, likes, advertisement, flame discussions).

Then I turned to rss Google reader. RSS is old technology, and you can check if web site supports it by just adding /rss at the end of site url. No noise, no flame wars, just useful data. Notifications about new blog posts. But Google decided to shut down Reader (no advertisements rings a bell).

Rss reader market is not so big, and after short investigation, I found Newsblur . It has browser and mobile desktop and I use paid option (I think 27 us$ per year).

I have around twenty feeds that I am following (security, software development and software testing).

Another issue is how to share what I have red to support the author? I know, there is Twitter, Facebook. But my Facebook friends are mostly not connected with software development.

Here is how I share my reading list. Newsblur has it own share feature, one click away. It is called blurblogs. What I share, directly goes to my blurblogs page. That page also has rss feed. I expose it on this blog first page.

rss is old and very useful technology, give it a try.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

vim, show me your colors!

Reading Time: 2 minutes

TL;DR

This post is about vim and how you can start using it in your daily work.

At one point in my testing path, I decided that I should start using vim. I opened Google, and start my search journey. I searched for:

“vim development platform”

What got my intention was search result:

spf13-vim – The Ultimate Vim Distribution

This is “spf13-vim is a distribution of vim plugins and resources for Vim, Gvim and MacVim. Which means that you need to install vim by yourself, and this distribution will add a set of useful vim plugins for development. Adding new plugin is also simplified (although, you need to know linux cat basics to do that).

For me, important one are related to ruby language. Here is one rspec file, opened in mvim:

Yes, you have coloured syntax. For me, this is very important, because with colours, I make less syntax errors!

Ok, you are set to go and you expenses so far are 0.

To get to know basic, run vimtutor (linux and mac os have it, for windows read this).

Here is set of vim cmd’s that I use on daily basics:

  • fire up vim: cd to your development folder and type:
    mvim &
  • enter command mode
  • :
  • start folder explorer vim plugin (note, tab will complete your commands, so you can type Expl, then hit tab, and vim will do autocomplete
  • :Explore
  • open file in new tab
  • select file with arrow keys, then press t
  • close file
  • :q!
  • go to file bottom
  • G
  • go to file top
  • gg
  • go to end of line
  • $
  • go to start of line (this is zero)
  • 0
  • delete line
  • dd
  • undo action
  • u
  • start edit mode (you know that you are in edit mode when you see this in the bottom of vim

  • i
  • exit edit mode (you need to do that in order to do all other commands listed here
  • esc
  • delete character on the right
  • x
  • copy paste action. Note, if you start typing after you hit p, copied item is cleared from the buffer.
  • v, then with arrow keys make a selection, y, go to position when you want to paste, p, text will be pasted on the right of cursor
  • search
  • /, what you search, enter, found text will be highlighted, with n you move to next found item.

 

That it is. This is small set of features what vim can do, but for me it is enough to be productive!

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Bottom up Heuristic

Reading Time: 1 minute

TL;DR

This post is about heuristic that I discovered while I wrote rspec ruby test script. Heuristic can be applied in any situation when you investigate errors reported on file line number.

One of my test frameworks includes Ruby where I write test scripts in rspec. I like rspec because it represents domain specific language (DSL) that makes testing code more readable and maintainable.

So yo have been in situation when you run your test suite, and you get test run report that contains line numbers where your check failed.

You need to update your test script. If you start your edit from top to bottom, your script will be no more aligned with test report. Now you have to run test suite again in order to align it again, and that takes time.

It is time for bottom up heuristic. Start your edits from bottom up directions, and you will keep your test report with test script. You will save some test run cycles and become slightly more productive.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Randomize your test data

Reading Time: 1 minute

TL;DR

This post is about one important testing method that should be automated, test data randomization.

You should randomize test data in automated script because this will make your automated test runs slightly more user likely because application that you are testing will be used by different users that will use different data inputs.

Also, it is very usual that application business rules are data driven. Based on data values, feature behaves differently. Using test data randomization, you can discover and test hidden business rules.

Here is my real example. I randomized account number, and soon realized that feature behaves non deterministic. One attribute output values were different from run to run. I could not define the rule.

I asked developers in dedicated slack channel about this behavior, and they explained that this is client business rule. Based on account first digit, created record was labeled differently (output attribute value). Of course, that business rule was not documented in any form.

In my ruby testing scripts, I use Faker gem. Happy test data randomization!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Docker behind a proxy

Reading Time: 1 minute

TL;DR

In this post I will explain how to set up docker when you are behind corporate proxy.

All Internet tutorials about any technology take assumption that you are not behind a proxy. Working for my client that must use corporate proxy, I spent a lot of working hours discovering how to use docker behind  a proxy.

You first need to find out value for corporate proxy. In corporate environment, this could be rather tricky action, because that value is not always publicly known.

  1. Set docker systemd settings to use proxy.

Use official docker instructions [source]. With this setting, docker daemon nows how to get images from the Internet.

2. Dockerfile.

If you need to tailor your own image, you need to use Dockerfile. In order to enable Dockerfile commands to get stuff from the Internet, you need to set ENV HTTP_PROXY variable.

After FROM directive in Dockerfile, put following settings:

FROM image_name
ENV http_proxy <HTTP_PROXY>
ENV https_proxy <HTTPS_PROXY>

3. Some tools do not use environment variable http_proxy.

There are some tools that made a wrong design decision where they do not use environment variable http_proxy, but they require that command switch must be used in order to set proxy value.

I broke my teeth on following tools:

  • pip
  • npm

Here is simple heuristic. If command from Dockerfile gets timeout (usually it appears to hang when it can not get to Internet), consult documentation for this command to find out how to set proxy value.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to replace curl data-binary value to form-data upload parameters

Reading Time: 1 minute

TL;DR

In this post I will explain how master Joda visited me in my dream and asked me important question about one of my previous blog posts: How to measure upload speed of web application upload form [link]. The post provided right information about curl parameter that enables you to see upload speed. But curl example for upload file is wrong!

I found that is wrong when I tried to measure upload speed in real application. And realized that file is not uploaded in application. By observing the copy pasted curl, I noticed that there was not Content-Length header. Bingo, lets create Chrome developer tools issue ticket[link].

Please read comments, because I learned a lot about curl upload file feature. Google developers who answered my ticket did not just dismissed my wrong curl upload file heuristic, they took extra step and educated me!

So, in order to use curl upload file feature, you need to transform something like this:

--data-binary $'------WebKitFormBoundaryoeJQRnHeojsIBayU\r\nContent-Disposition: form-data; name="fileType"\r\n\r\n5\r\n------WebKitFormBoundaryoeJQRnHeojsIBayU\r\nContent-Disposition: form-data; name="file"; filename="elixir-websetup.exe"\r\nContent-Type: application/x-msdownload\r\n\r\n\r\n------WebKitFormBoundaryoeJQRnHeojsIBayU--\r\n'

into this:

-F file=@elixir-websetup.exe -F fileType=5

You can now data-binary parameter and its value from curl.

It is important to have check mechanism for upload file. Use your application features to check is file really uploaded. curl can also help by observing application returned value.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Selenium webdriver Net::ReadTimeout (Net::ReadTimeout) exception

Reading Time: 1 minute

TL;DR

In this post I will explain how I resolved this exception in context of selenium webdriver testing in headless mode using xvfb on linux.

You finally managed to set up linux and xvfb server, your selenium webdriver settings are correct, and you successfully run your first scenario in headless mode! Astonishing achievement when I add fact that you only used open source technology.

You have your cucumber feature files with scenarios, written according every best BDD practice. Although, one feature file has more scenarios than other. Feature that it tests is rather complicated.

You set up jenkins job, and find out in the next morning of run, that only that feature file has failures. You examined execution html log, and find out this cryptic exception:

screen-shot-2017-01-07-at-11-56-08-am

Network timeout!? But site that you test is ready and online.

What is going on under the hood of selenium webdriver tests.

Selenium webdriver is http server. It accepts http connections, according to webdriver protocol that is http based. Your language selenium webdriver bindings start server for you when you create driver and your test script sends webdriver protocol commands to webdriver server. Simple as that.

Above exception means that your selenium webdriver server is not responding. Why? In context of headless xvfb linux test run, this could be caused by various reasons.

Solution.

I noticed that exception appeared always on same scenario, so I split long cucumber feature file in two. If you have properly written feature file according to BDD practices, this should not affect your test suite.

I guess My heuristic is that selenium webdriver server could be up for some limited time and that with long feature file I hit that limit.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Meet Blitzy: not so simple HTTP load tester in Elixir

Reading Time: 2 minutes

TL;DR

In this post I present Blitzy, not so simple HTTP load tester in Elixir. This is my open source contribution to original Blitzy, simple HTTP load tester in Elixir.

My journey with Elixir started six months ago. With proper book from The Pragmatic Bookshelf, this was fun experience. I made myself to do all exercises, but in the end I skipped Macro exercises. How to test my Elixir practical knowledge? The best way is to contribute to some of Elixir open source project.

For one of my clients, I need to do load, duration and concurrent test. I previously used Grinder, Java Load Test Framework. Driven by pure curiosity, I searched for load testing tool in Elixir, and I found Blitzy.

It has simple feature set, do load test for simple http get. By exploring the code and using Programming Elixir 1.3 as a reference, I found out that I can read and build Elixir project code. So, why not add features to Blitzy that I need for my current project.

My first contribution was to update dependencies, and do some refactor because some libs changed their interfaces. This was need just to build Blitzy for current set of features.

In closing session of CAST 2016, there was discussion how can we help to enhance software testing. My contribution, lets create software testing tool, that can be easy to extend, develop and maintain. Another pointer for Blitzy contribution.

Power of Blitzy is that is programmed in Elixir, which uses Erlang virtual machine. Downsize of Erlang is rather cryptic programming language, it feels that you are writing mathematical formulas and proofs, and not a program. But there is Elixir, that resolves that issue.

Blitzy codebase is small, because it uses Erlang virtual machine concurrency and code distribution out of the box features.

I remember that most of the code for Grinder was about those features, distribution and concurrency.

Check github repository, and happy load testing.

I wrote tests for Blitzy, currently line coverage is more that 80%, but I have not wrote most important test that include data. How about that metric preachers? Important unit test are on to do list.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Technology junky

Reading Time: 2 minutes

TL;DR

In this blog post I will describe my TV upgrade that should last for next 10 years.

This summer was Football UEFA Euro competition year in France. Heuristic is that you could by new TV models with significant discount. I bought my last TV in 2008, and it was plasma. TV technology have dramatically changed in those eight years and I did know where to start. My Christmas started earlier this year. I asked my friend, who is real TV junky, for help. He needed just one input, my budget. With my budget of 1500 eur, he started his research. Believe me, it is not an easy task. After few pointers and one week, he hit a jackpot, LG-49UH8507, in my budget range.

This is my short review wearing a testers hat. Basically, I bought computer system . Every computer needs operating system.

WebOS 3.0

You can find technical specification here. From user perspective it is very fast and reliable. UX is pretty good, menus and actions are strait forward. I enabled auto upgrade which is done in sleep mode. When I turn it on, I get notification in upper right corner about the upgrade.

Network

WiFi and Ethernet.

I use ethernet for streaming services. Big surprise was when my Iphone 6 offered to connect to TV while I was watching Youtube. It is called screen sharing.

Remote

It has magic mouse feature, which means mouse pointer that I steer remotely. Remote UX is not so good, because I mostly use it in dark room and I can not guess button that I need based on its position or texture structure.

Display

UltraHD which means 4K resolution. This is Netflix Dardevil in 4K (click for larger image):

img_1231

Entertainment and education

I watch 90% of my time Netflix or HBOGo. Up till last update, Netflix had stability issue. I always exited application, but when I reentered it, it always crashed. HBOGo has streaming issue with new items, but it is stable. Netflix has much batter UX experience when I continue watching where I stopped previously. No streaming issues at all even with 4K items.

There is also youtube player, so I can educate myself from the conform of my sofa.

I am very satisfied with my new gadget. For the same price, I also got LG G3 smartphone, which was big surprise! And salesperson was very educated about the product, he confirmed all technical specs that I knew thanks to my friend.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather