How to set regression test scope for HTTPS only access?

Reading Time: 2 minutes

TL;DR

In this post I will provide an example how to set the scope of system regression test in order to achieve coverage of features that need to be tested.

Context

The trigger for regression system test was HTTPS only access to client Ruby on Rails application. Prior to that feature, it was possible to use both HTTP and HTTPS protocols. Developers use TDD concept, and prior to my test, all developer tests passed on CircleCI environment.

There was also set of selenium-webdriver tests, but those tests do not cover all application features.

My strategy was to include them in the regression test. All test passed. But I had not finished regression test yet.

I did not know all application features at that time. So I started risk analysis, which features could fail if HTTPS only protocol was introduced to web application. Let’s call OWASP for help.

There is transport layer cheat sheet. Reading through the rules, I pinpointed rules that were potential risk for application functionality:

  • Do Not Mix TLS and Non-TLS Content – because browser (modern browsers) will AUTOMATICALLY prohibit access to non HTTPS urls.
  • Use a Certificate That Supports Required Domain Names – if this is not the case for your application, browser will present to user a security error

First risk could be mitigated by using the application in Chrome and observing javascript console for mixed content errors. Could I automate that task? First thought would be: write selenium-webdriver test suite that covers all the features! But I do not have that time. Was there a simpler way?

My heuristic was to search all the code base for keywords HTTP and IFRAME.

grep -H -r ‘iframe’ * | grep http:// | grep -v elements.txt | grep -v ‘README.md’ | less

That piped command searched through the all code base in terminal and returned code that uses mixing content.

And we discovered additional issue, it was not possible to immediately set all HTTP urls to HTTPS protocol. Those urls were referencing external applications, like blog. For example, in order to set this blog to HTTPS protocol, I need to buy another plan that costs more money. And I need to have a certificate for tentamen.eu domain. Which brings us to second risk.

Use a certificate that supports required domain names. And this is environment dependant test. This risk was mitigated on my testing environment, but I should also check it on production environment (production is hosted on different domain).

Doing risk analysis is fun. You will learn something new and the most importantly, you will properly set scope for your regression test.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to get page_url value in ruby page-object gem

Reading Time: 1 minute

TL;DR

In this short post I will provide code example for undocumented page-object gem feature, how to get value of page object page_url attribute in page object instance.

If you do browser test automation, I advise you to use following software stack:

  • Ruby language
  • selenium-webdriver gem
  • watir-webdriver gem
  • page-object gem
  • cucumber

 

And most of your code base should be written in cucumber feature files, step definitions and page definitions using page-object framework. Doing that, your code base will be highly maintainable, readable and DRY.

All of mentioned frameworks are open sourced, and they do not document all of their features (although I am very satisfied with the documentation). Sometimes you need to dive in source code and spec tests of those frameworks.

Page object gem has a lot of useful helper methods at your disposal. page_url returns page url that is defined in class definition. For example:

But in page-object gem wiki page there is no example how to actually get page_url value.

To save your time browsing through gem code base, here is how you can do that:

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to automate page with dynamic elements

Reading Time: 1 minute

TL;DR

In this post I will explain how to examine dynamic html elements in order to be able to automate web page that contains such an element.

 

Many of modern web applications uses heavily some of the popular Javascript frameworks (Ember, Angular are some of them). In that way, your web application is more interactive and dynamic.

For example, user does some action and it gets message that disappears after few seconds. Very effective from the UX point, but how to automate that scenario?

The problem here is how to examine html structure of disappearing message in order to find out element id (unlikely to be present) or class attribute?

In order to do that, you can use chrome developer tools, the sources panel. You need to debug javascript code that shows and removes application message. You can do that very simply with javascript debugger pause button.

 

Here is what you should do:

  1. open your web application in Chrome
  2. open Chrome developer tools
  3. open Sources tab
  4. make action on your web application that will trigger dynamic application message
  5. when message appears, hit the pause button
  6. go to elements tab and examine the message html attributes
  7. when you are finished, go back to sources tab and press continue button

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How BBST Test Design course helped me with Netflix setup

Reading Time: 2 minutes

TL;DR

In this post I will explain how I applied knowledge gained in BBST Test Design course in order to quickly resolve Netflix issue on my Sony BDP-S4500 Blu-ray player.

 

Last year, for my birthday, I treated myself with Sony BDP-S4500 Blu-ray player. After unboxing it, I was very excited because I realized that this gadget is much more than a Blu-ray player. Notice white button with red letters: NETFLIX.

I connected player to my TV and router for cable Internet, configured it, and I pressed netflix button from pure tester curiosity. After 20 seconds, I got expected message: “Feature not available” because at that time Netflix was not available in Croatia.

On January 6th 2016., six months later, CES 2016 started in Las Vegas. Netflix announced that expands its service to 190 countries. Luckily for me, Croatia is one of those countries!

I have read several posts from Ben Simo about Netflix issues that he encountered. And I envy him a lot on those issues.

I used my Mac in order to signup for Netflix service. I choose plan for 10 euros. One of the features of that plan is ability to use Netflix on two devices. I encountered no issues with signup process.

I switched to player and hit Netflix button again. Feature was again not available. Ok, now we have a problem here. But with Netflix or player device?

BBST Test design kicks in (BBST Test design slides, page 174, Initial states)! Feature not available was returned without asking me for any information. My heuristic was that player was “stucked” in state where Netflix availability check was done. And did not initiate new check.

How to reinitiate new Netflix check?

First, I updated player software to latest version. No change. Should I reset player settings? But I would like just to reset customer information data, particularly data that states that Netflix is not available for player public IP address. How to do that? I should consult player claims (documentation). Fastest way to do that is Google.

After googling for 5 minutes, I found the solution on this link, posted by user BrianP.

 

Do Initialize Personal Information.
1. On the supplied remote, press the HOME button.
2. Go to Setup.
3. Under Setup, select Resetting.
4. Under Resetting, select Initialize Personal Information.

 

Sony hid Initialize Personal Information, as submenu of Reseting menu. Which is probable inconsistency with user expectations. As a user, I was afraid to select Reseting in order not to start factory reset of player device.

 

Action 5. was click again on Netflix button, and I was in! Sign in, remember me, and Daredevil and 12 Monkeys season 1. episode 1. were ready on my eight year old Panasonic Viera series plasma TV.

netflix_viera

I have watched three shows so far, without any issues. Maybe is a good thing not be early adopter of such a cool service.

 

Conclusion.

Take BBST test design course, but do not forget to practice it not only in your daily job, but also in daily life! And write a blog post about it.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to pet a cat

Reading Time: 1 minute

TL;DR

In this post I will explain how I used my software testing craft in order to learn how to pet a cat.

Picture shows Sivka. She enjoys more than any other cat in my neighborhood to be pet. The issues is that starting the pet session is not an easy task. Sometimes I managed to do that, but most of the time she run away from me.

So I switch on my software testing skills. I compared the environment when I failed and when I succeeded in petting session. I crunched the testing data taking into account fact that Sivka could easily get frighten.

At that time I read excellent blog post from Berine Berger: The Inefficiency of Gifts and Software Metrics. In that blog post there is a link to Evaluating test plans using rubrics and in that paper he explains quality assessments.

Quality assessment are:

  • judged based on characteristics
  • empirical –  verified by observation
  • experiential – verified by experience
  • usually reported in words
  • usually contrasted with quantitative

In order to start petting session I needed to approach her much slower with my hand. But how much slower? I do not have any tools that could help me to measure the approach speed of my hand towards the Sivka. So I used quality assessment and real time feedback. I started my approach with one velocity, and adapted that velocity if I noticed that Sivka started to get frighten.

Now I know what is the approach speed of my hand. I do not have quantity value of that speed (to be written in test script or test case), but that speed is now written subconsciously in my brain, the greatest software testing tool!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather