How to properly write automated check

Reading Time: 1 minute

So you are in position to use some programming language in order to automate test. Welcome to the rabbit hole of software testing! If you would like to know more about software development in software testing, start with this excellent blog from Michael Bolton.
Now that you know difference between checking and testing, I will give an rspec example of proper automated check in REST API test.
In my example, REST API response is in JSON format. In case of successful response, message is:

{response = ‘success’}

and in case of some error, response message is:

{response=’error’, error=’error message’}.

How should we code in rspec response message check when we expect successful response?
First option:

response_message[‘response’].should eq(‘success’)

What is wrong with this check? What if we receive error message? Failure in that case would be:

expected response_message[‘success’] to be ‘success’ but ‘error’ received.

Do you know from test fail message what caused error?
Automated test has to be designed in following manner:

  • you are able to determine cause of fail directly from fail message
  • test has to be designed so that checks one and only one thing
  • test is independent from other tests. In rspec world that means it always produce same result when we use rspec option –order=random
In this example, proper check is:

response_message[‘error’].should be_nil

because in case of test failure, message is:

expected response_message[‘error’] to be nil but received
“message that explains reason for failure”
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

curl vs. Ruby http client gem on practical example

Reading Time: 2 minutes

In my current testing project one part of testing strategy is to test REST api endpoints of web application independently for user interface. Application is in early stage and documentation is in great volatile state. There is testing environment dedicated to testers and there is new release deployed on it almost every second day. I choose curl and Ruby httpclient gem as testing tools and assistance in this task. In this post I will explain usage context for each tool.
As documentation for api endpoints is still very volatile, I need a way to explore each of api endpoints. REST API endpoints are explored using following inputs:

  • method type: HEAD, GET, POST, PUT, DELETE
  • headers: set restricted to headers sent by latest Chrome version
  • parameters: query for GET and parameters for POST, PUT and DELETE
Exploration context
In order to explore api endpoints I used Chrome developer tools (network tab with copy as curl feature) and curl which is bundled with Maverick OS X. Reason for that is because using curl I can easily remove, add and change value for every REST api call input parameter and its value.

Regression context

When I stop testing api endpoint using curl (using for my stop testing criteria api endpoint feature coverage using endpoint inputs), then I code in Ruby application automation testing framework using httpclient, json_spec (api endpoints input and output are in json format) and rspec as testing framework. In order to achieve flexibility as in curl, I had to investigate httpclient gem documentation and had to test developed application automation testing framework output by directing traffic through ZAP proxy. Using proxy I discovered that httplib gem is smart enough to automatically populate Cookie header, based on previously received Set-Cookie headers (there where duplicate Cookie headers, one set by automation framework and one set by httpclient). This is super functionality if you are developer of HTTP client, but for tester this reduces their flexibility to trigger errors. For example, I created a test case where I would like to know information does application properly handles http sessions stored in cookie header. Fortunately, there is property that turns off automatic cookie population.
Another example was that I was not able to send duplicate parameters (with same name) as POST parameters.
As in real life, every tool has its usage context, so testers should wisely choose which tool to use in which context in their testing job. 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather