Oracle exercise on real example

Reading Time: 2 minutes

TL;DR

This post is example how to apply oracle heuristic to identify is there a problem. Disclaimer: this blog post is not about some fancy new software testing framework. Pure software testing craft.

You are still here after disclaimer? Great!

Oracles are simply the principle or mechanism by which we recognise a problem. [Ref.]

Please read the article, it is well written and easy to comprehend. Another quality of excellent software tester.

In order to know how to use oracles in software testing, you need to practice. I hope that this example will help you.

I am “forced” to use Microsoft Word in order to create documentation for one project. I decided to insert images from external documentation using “Insert from URL feature”. In that way, when external documentation changes, link would either break or would automatically point to new image.

I clicked in Word Insert menu, then on image icon. After several minutes, i realized that there is no “Insert from URL option”.

I searched with Google to find quick answer:

Go to Insert – Quick Parts – Field…

Then you will get select box with a lot of options, one of them is insert image from URL (why we should bother to put it as first option in the list).

What!? I will repeat that because it sounds like sentence from Monty Python’s Flying Circus sketch:

Go to Insert – Quick Parts – Field…

Hmm…, do we have a problem here? I am calling oracle consistency heuristics Comparable Product into help.

We expect the system to be consistent with systems that are in some way comparable. That might include other products in the same product line, or from the same company. The consistency-with-past-versions (History) heuristic is arguably a special case of this more general heuristic. Competitive products, services, or systems may be comparable in dimensions that could help to discover a problem. Products that are not in the same category but which process the same data (as a word processor might use the contents of a database for a mail merge) are comparable for the purposes of this heuristic. A paper form is comparable with a computerized input form designed to replace it. Indeed, any product with any feature may provide some kind of basis for comparison, whereby someone might recognize a problem or a suggestion for improvement [Ref.].

Lets check Google Docs.

Click Insert menu option. First suboption is Image icon, click on it, there is option window with option By URL. It took me four seconds to find it.

So this word option is not consistent with comparable product because in comparable product is much easier to insert image. Proof that Google docs has better UX than Microsoft Word.

And you can use this as selling pitch for this issue to your product manager.

I once presented oracle consistency heuristics to software testers. Feedback was: Oh, this is fancy and great, but we DO NOT HAVE TIME TO DO THAT!

Then I asked them contra question: How much time you spent in your bug triage sessions?

A lot.

With oracle heuristics, you are first filter for bug. If you can not find inconsistency in listed heuristics, than you will not report this issue. And your bug triage sessions will be much shorter.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

UI check automation suggests important project issues

Reading Time: 1 minute

TL;DR

This blog post is my experience about UI test automation applied in various projects.

First disclaimer, this post is not against UI check automation. If not used as a testing hammer, it can help towards better product quality.

How to recognize UI automation as marker for important project issue? If project testing pyramid morphs into testing coan [source: Watirmelon].

  1. skilfull session based testing is replaced with manual repeating of instructions listed in test cases documents.
  2. all automation checks are in UI level, and represent end-to-end checks.

This points to important project issues:

  1. lack of skilfull testing
  2. knowing test automation framework, usually selenium based, is sexier that skilfull testing

What can you do? Start learning and practicing resources listed in point 1. This will help your project to use testing pyramid and help you to fight your desire for ice cream!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Ruby on Rails bottom up security – Cross site scripting (XSS) check

Reading Time: 1 minute

TL;DR

This post explains how to check your Rails application source code for cross site scripting (XSS) attack.

Cross site scripting means that your application accepts html code as user input. Biggest issue is <script> tag, that allows user to execute javascript code in the context of your application. Second one is <img> tag that can also be used for code execution.

Rails by default escapes all input, which means that html code will be transformed, so browser will not interpret it as html:

<script>alert("Session based test management");</script>` => `&lt;script&gt;alert(&quot;Session based test management&quot;);&lt;/script&gt;

But some applications, such as github, allow users to have text formatting options.

Dirty way is to allow html input (github is using markdown language), and Rails have api methods for that:

html_safe

raw

This is ok as long there is not direct user input as parameter of that method (for your editor implementation, you also want to use markdown). Never trust your users!

Use this for code check:

grep -H -r 'html_safe' * | less

grep -H -r 'raw' * | less

There is one more important xss security attack vector. When you open link in new tab, application from that new tab can control, using javascript, application in original tab.

Use this for source code check:

grep -H -r 'target="_blank"' * | less

and make sure that link tag also has this option:

rel="noopener"

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Exploratory session of Pena Palace

Reading Time: 2 minutes

TL;DR

Using this excellent post by Marcel Gehlen , I am learning about exploratory software testing.  I created github wiki where I put notes about every resource listed in that post. This post is practical exploratory session of Pena palace located in Sintra, Portugal.

My vacation for end of this summer was Portugal tour, organized by Mondo Travel agency. Part of that visit was Pena national palace, located in Sintra town. As I just had read about session based test management by James Bach, I decided to practice it on Pena palace.

# CHARTER

Mission: Cover every walking path allowed to tourists in Pena palace and document interesting parts using Iphone 6s camera.

Note: When you do testing coverage, it is very important to state in report which coverage was done. Cem Kaner listed 101 testing coverage types, so please read it in order to know how complex is test coverage problem. By stating properly your testing mission, it is easier to estimate how much testing sessions is required.

I stated that I would do every walking path allowed to tourists. So no sneaking to restricted areas. Here are some other possible testing coverage types:

  • investigate every wall picture
  • investigate every palace window
  • investigate every palace door
  • investigate every tiles

# START

I know exact time that is when I took first Pena photo.

# TESTER

Karlo Smid

TASK BREAKDOWN

# DURATION

90 minutes

Note: This was “hard” requirement, because if I had exceeded that time, my group would have waited for me.

Timestamp of last picture

# TEST DESIGN AND EXECUTION
90%

# BUG INVESTIGATION AND REPORTING
0%

# SESSION SETUP
10%

As a group, we got info about Pena Palace from our guide. Also, I checked my testing tools, Iphone 6s and Iphone 6s smart battery case.

#CHARTER VS. OPPORTUNITY
80/20

Note: Kitchen looked very interesting. I would have definitely investigate it more if I had had more time.

# DATA FILES

# TEST NOTES

I managed to walk all available paths and document all items of my interest. When you go on vacation that is organized by agency, your are in the group and you need to adapt to given time. This is tradeoff. Positive thing is that you meet new people that have something in common: like to travel!

# BUGS

None

# ISSUES

None

In this post you learned about test coverage and how to apply session based test management during your leisure time. Have you noticed that exploratory word was striked through in TL;DR? James statement is that every testing is exploratory, so there is no need for exploratory word. And I agree with that statement based on my practical experience in last month when I applied my latest knowledge of exploratory testing.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather