All posts by Karlo Smid

Be careful with software updates: example for osx text to speech feature

Reading Time: 1 minute

TL;DR

This blog post is about how OSX update affected text to speech feature that I use as  proof reading aid for my blog posts. I will propose a testing charter based on this experience.

As I am not english native speaker, in order to avoid basic grammar errors in blog posts, I always reply blog post using OSX text to speech feature. This proved to be very useful.

In wordpress editor, (html page) I selected blog post (using keyboard) in paragraph chunks, then I clicked right mouse (using touchpad two finger gesture) and selected speech => Start speaking.

No speech.

In order to decide “Do we have a problem here?”, I used oracle comparable product (FEW HICCUPS by M. Bolton), OSX Notes.

Using same steps as in WordPress, there was speech.

Idea, WordPress editor is html/javascript editor, so ttx can not read html?

Idea, what recently changed with my laptop?

Yesterday I upgraded OSX to Sierra version.

Lets check if state of ttx OSX feature (in the end, this is just another program) is affected with upgrade – BBST Test design (by Kaner and Fiedler).

System Preferences => Accessibility => Speech

Oh there is unselected option, “Speak selected text when the key is pressed”. I enabled Command + Z combination.

Go to wordpress editor, select blog post paragraph, hit command + Z, it works!

Try option with context menu described previously, it worked!

We are taking for granted that complex upgrades, such is OSX upgrade, should work out of the box. In this example, ttx speech was not broken, but was set to initial state because of new configuration option.

I described oracle heuristic comparable product (one C from FEW HICCUPS), and initial values  quick test idea.

You can learn about them in BBST Foundations and Test Design courses.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

What I learned at Testival #32 meetup

Reading Time: 2 minutes

TL;DR

This post is about my  Testival #32 meetup learning experience.

Sponsor of this meetup was Degordian, digital agency  where there is no cure for their curiosity 🙂

They recently moved to new location, and this was my first experience of “Apple” style working place. And it was great experience. When looking such premises only through photos, it may give impression of show off.  But every detail in this place is made with reason. The reason is to stimulate your brain for new ideas and new learning experience! Great job!

 

We started with usual introduction of 14 attendants. One newcomer felt like she does not belong there, because of her educational background. But this was good thing, because we had at the meetup several people with diverse educational background. Software testing is not confirming that stream of binary code works/does not work.

Testing is a “meta” activity. It’s not just a task, but a task that generates new tasks (by finding bugs that should be fixed or finding new risks that must be examined). It’s a task that can never be “completed” yet must get “done.” [James Bach]
Testing is an investigation based on, concerned with, or verifiable by observation or experience conducted to provide stakeholders with information about the quality of the product or service under test. [Cem Kaner]

Marko Kruljac from Degordian presented “Integrating Jenkins with your GitHub Pull Request Workflow”.

Marko talks about Jenkins/github workflow.

 

As I have done similar for several clients, I was curious how they cope with that, because Jenkins offers several options through its great Plugin community.

I learned that for github the best option is Github pull request builder. It also made me think how important is to select Jenkins plugin name in order to present what is actually do.

After pizza and beer break, we had three lighting talks.

Zeljko Filipin from Wikimedia presented unicode zero-width non-joiner character. In Ruby interactive console he merged two “emoji” with that character in order to make new one (e.g. man + pan = cook).

Next day I found that with that character you can make very scary things, like publish your own What’s App application in Google play. Read more about that in Gojko Adzic excellent blog post.

Second lighting talk was from Vilim Stubičan from Degordian talked about his learning process (while he was solving Rubik cube with one hand 🙂 ).

I presented the most important software tester tool, a notebook. Read again James and Cem definitions of software testing to know the reason for that fact.

P.S.

During the testing session in preview mode of this post, I found important issue with one photo because it contained Degordian wifi password. Which automation would caught that issue?

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Ruby on Rails bottom up security – mass assignment

Reading Time: 2 minutes

TL;DR

Mass assignment is security risk where user can create/update data attributes that is not allowed to update.

Here is an example. Imagine application that registers your employees working hours. When user logs in it sets start time, and when it logs out it sets end time. Pretty simple feature.

User login form has username/password input fields. Imagine that user can temper its login timestamp using login request. How? Your employee friend is skilfull tester, and he knows how to send POST request using Postman tool. Using Chrome developer tools he/she finds out the login attributes and now he tries to guess login timestamp attribute:

  • createtime
  • logintime
  • login_time
  • create_time, …

Those names set with date values in the past (he/she wants his friend to work less) are sent using curl (no need to know cookie!). Heuristic to know when correct time attribute is guessed is very simple. There is another url endpoint, or even login response, that will return login time.

How is that possible!? Ruby on Rails got its popularity because is has a lot of default features that made developer work much easier. One of those features is to automatically accept all http request input parameters that match to available ORM (object relational model) object attributes.

Creation timestamp is an example of attribute that should be set by application, not the user input.

Remember, never trust the user input. And hacker loves default framework features.

Ruby on Rails in current version does not allow mass assignment. Every input parameter must be listed in

permit

method in order to be accepted by ORM.

Using super power tool grep, you should search your Rails codebase for this:

grep -H -r 'permit' * | less

Using your knowledge about the application, you need to conclude (look ma, no automation here!) are those parameter allowed to be listed in permit method in the first place.

I also strongly advise communication with your developers 🙂 in order to make the decision.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Oracle exercise on real example

Reading Time: 2 minutes

TL;DR

This post is example how to apply oracle heuristic to identify is there a problem. Disclaimer: this blog post is not about some fancy new software testing framework. Pure software testing craft.

You are still here after disclaimer? Great!

Oracles are simply the principle or mechanism by which we recognise a problem. [Ref.]

Please read the article, it is well written and easy to comprehend. Another quality of excellent software tester.

In order to know how to use oracles in software testing, you need to practice. I hope that this example will help you.

I am “forced” to use Microsoft Word in order to create documentation for one project. I decided to insert images from external documentation using “Insert from URL feature”. In that way, when external documentation changes, link would either break or would automatically point to new image.

I clicked in Word Insert menu, then on image icon. After several minutes, i realized that there is no “Insert from URL option”.

I searched with Google to find quick answer:

Go to Insert – Quick Parts – Field…

Then you will get select box with a lot of options, one of them is insert image from URL (why we should bother to put it as first option in the list).

What!? I will repeat that because it sounds like sentence from Monty Python’s Flying Circus sketch:

Go to Insert – Quick Parts – Field…

Hmm…, do we have a problem here? I am calling oracle consistency heuristics Comparable Product into help.

We expect the system to be consistent with systems that are in some way comparable. That might include other products in the same product line, or from the same company. The consistency-with-past-versions (History) heuristic is arguably a special case of this more general heuristic. Competitive products, services, or systems may be comparable in dimensions that could help to discover a problem. Products that are not in the same category but which process the same data (as a word processor might use the contents of a database for a mail merge) are comparable for the purposes of this heuristic. A paper form is comparable with a computerized input form designed to replace it. Indeed, any product with any feature may provide some kind of basis for comparison, whereby someone might recognize a problem or a suggestion for improvement [Ref.].

Lets check Google Docs.

Click Insert menu option. First suboption is Image icon, click on it, there is option window with option By URL. It took me four seconds to find it.

So this word option is not consistent with comparable product because in comparable product is much easier to insert image. Proof that Google docs has better UX than Microsoft Word.

And you can use this as selling pitch for this issue to your product manager.

I once presented oracle consistency heuristics to software testers. Feedback was: Oh, this is fancy and great, but we DO NOT HAVE TIME TO DO THAT!

Then I asked them contra question: How much time you spent in your bug triage sessions?

A lot.

With oracle heuristics, you are first filter for bug. If you can not find inconsistency in listed heuristics, than you will not report this issue. And your bug triage sessions will be much shorter.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

UI check automation suggests important project issues

Reading Time: 1 minute

TL;DR

This blog post is my experience about UI test automation applied in various projects.

First disclaimer, this post is not against UI check automation. If not used as a testing hammer, it can help towards better product quality.

How to recognize UI automation as marker for important project issue? If project testing pyramid morphs into testing coan [source: Watirmelon].

  1. skilfull session based testing is replaced with manual repeating of instructions listed in test cases documents.
  2. all automation checks are in UI level, and represent end-to-end checks.

This points to important project issues:

  1. lack of skilfull testing
  2. knowing test automation framework, usually selenium based, is sexier that skilfull testing

What can you do? Start learning and practicing resources listed in point 1. This will help your project to use testing pyramid and help you to fight your desire for ice cream!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Ruby on Rails bottom up security – Cross site scripting (XSS) check

Reading Time: 1 minute

TL;DR

This post explains how to check your Rails application source code for cross site scripting (XSS) attack.

Cross site scripting means that your application accepts html code as user input. Biggest issue is <script> tag, that allows user to execute javascript code in the context of your application. Second one is <img> tag that can also be used for code execution.

Rails by default escapes all input, which means that html code will be transformed, so browser will not interpret it as html:

<script>alert("Session based test management");</script>` => `&lt;script&gt;alert(&quot;Session based test management&quot;);&lt;/script&gt;

But some applications, such as github, allow users to have text formatting options.

Dirty way is to allow html input (github is using markdown language), and Rails have api methods for that:

html_safe

raw

This is ok as long there is not direct user input as parameter of that method (for your editor implementation, you also want to use markdown). Never trust your users!

Use this for code check:

grep -H -r 'html_safe' * | less

grep -H -r 'raw' * | less

There is one more important xss security attack vector. When you open link in new tab, application from that new tab can control, using javascript, application in original tab.

Use this for source code check:

grep -H -r 'target="_blank"' * | less

and make sure that link tag also has this option:

rel="noopener"

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Exploratory session of Pena Palace

Reading Time: 2 minutes

TL;DR

Using this excellent post by Marcel Gehlen , I am learning about exploratory software testing.  I created github wiki where I put notes about every resource listed in that post. This post is practical exploratory session of Pena palace located in Sintra, Portugal.

My vacation for end of this summer was Portugal tour, organized by Mondo Travel agency. Part of that visit was Pena national palace, located in Sintra town. As I just had read about session based test management by James Bach, I decided to practice it on Pena palace.

# CHARTER

Mission: Cover every walking path allowed to tourists in Pena palace and document interesting parts using Iphone 6s camera.

Note: When you do testing coverage, it is very important to state in report which coverage was done. Cem Kaner listed 101 testing coverage types, so please read it in order to know how complex is test coverage problem. By stating properly your testing mission, it is easier to estimate how much testing sessions is required.

I stated that I would do every walking path allowed to tourists. So no sneaking to restricted areas. Here are some other possible testing coverage types:

  • investigate every wall picture
  • investigate every palace window
  • investigate every palace door
  • investigate every tiles

# START

I know exact time that is when I took first Pena photo.

# TESTER

Karlo Smid

TASK BREAKDOWN

# DURATION

90 minutes

Note: This was “hard” requirement, because if I had exceeded that time, my group would have waited for me.

Timestamp of last picture

# TEST DESIGN AND EXECUTION
90%

# BUG INVESTIGATION AND REPORTING
0%

# SESSION SETUP
10%

As a group, we got info about Pena Palace from our guide. Also, I checked my testing tools, Iphone 6s and Iphone 6s smart battery case.

#CHARTER VS. OPPORTUNITY
80/20

Note: Kitchen looked very interesting. I would have definitely investigate it more if I had had more time.

# DATA FILES

# TEST NOTES

I managed to walk all available paths and document all items of my interest. When you go on vacation that is organized by agency, your are in the group and you need to adapt to given time. This is tradeoff. Positive thing is that you meet new people that have something in common: like to travel!

# BUGS

None

# ISSUES

None

In this post you learned about test coverage and how to apply session based test management during your leisure time. Have you noticed that exploratory word was striked through in TL;DR? James statement is that every testing is exploratory, so there is no need for exploratory word. And I agree with that statement based on my practical experience in last month when I applied my latest knowledge of exploratory testing.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Ruby on Rails bottom up security – authentication and session management check

Reading Time: 1 minute

TL;DR

This post is about checking “The Gates” of your Rails application.

Every web application is a set of urls. Some of them are publically available and some are available only to you (e.g. your bank account page should be available only to you and your spouse).

Modern web application authentication works as follows:

  • there is log in page where you enter your username/password
  • backend checks that combination
  • if this combination is valid, backend returns in Set-Cookie header long, unique, hard to guess string of characters. This is session string.
  • Browser takes that value and sends it in Cookie header in all following requests.
  • Backend checks cookie value and it needs to be same as one assigned to username/password combination
  • there should be logout endpoint that removes session string form username/password combination.

More about Ruby on Rails session security can be found here.

If you want to start your own session management, this could go wrong in infinity number of combinations.

So what can you do? Use Devise gem with following options:

  • store session in database
  • session must expire after some user inactivity time
  • log out feature must be implemented.
  • rotate devise key value (most frequently means more security).

In future security audits you only need to check devise configuration, that devise if updated with (possible) latest security patches and that devise security key rotation is active.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Learning risk management by example

Reading Time: 2 minutes

TL;DR

After taking state of the art software learning courses, I concluded that best way to comprehend knowledge is to learn by using examples of presented  materials.

How to measure quality of a course or workshop? For me, one metric are examples used in course or workshop. Of course, not THE NUMBER of examples, but my subjective measure how examples helped me to understand theory. And what is more important, how those examples help me to remember what I learned.

My examples of such courses are Rapid Software Testing  by Satisfice or BBST courses by AST foundation.

Today is SATURDAY and it was RAINY morning, and I drove to my hometown Zabok in EARLY morning. Security of my trip was jepordised on third stop light. Traffic lights were off. This was intersection with one direction road and intersection had road signs.

Should I wait for traffic lights be repaired? In that case, I will be late.

Since it was SATURDAY, EARLY RAINY morning, the traffic on the intersection was very light and it was intersection with one direction road, I crossed the intersection SAFER than on working day.

Ok, that was example, but what is the topic?

Your project is using a lot of 3rd party software components. Those components could have important security fixes. Deployment of new version for 3rd party component requires testing.

You are at YOUR crossroad with jeopardised security, this time of your product.

Should you upgrade? Will your sprint be late because of it? Do we need to deploy? Can project be secure enough without the patch?

This is security risk analysis for your product in the context of 3rd party component security fix. You need to create questions and answers in the context of your product and related to software security domain.

Can you give us any example?

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Ruby on Rails bottom up security – sql injection check

Reading Time: 2 minutes

TL;DR

This post will explain how to check your Ruby on Rails code base against sql injections [Wikipedia].

After you have read Wikipedia source link about sql injections, you are ready to proceed. It is important to state that Ruby on Rails offers one of the best (if not the best framework) out of the box sql injection protection.

But sometimes, developer needs to be able to override this protection for some “Twin Peaks” feature requests. In that case, even experienced Rails developer could make sql injection attack possible. On the other spectrum, if your company can only afford junior Rails developers, this check is a must.

Here is how you should hunt such protection overrides. First, head to Rails SQL Injection site. This page lists many query methods and options in ActiveRecord which do not sanitize raw SQL arguments and are not intended to be called with unsafe user input. You will use this site as a reference.

Open your terminal, go to root folder of Rails project repository, and use this cmd:

grep -H -r 'search for activerecord method' * | less

You need to replace search term with active record method name. Here you need to be creative in order to filter out results that do not represent active record method.

Here is the punch line:

Never, ever, trust the user input!

Which means that user input must not be directly used in active record methods. Again, refer to Rails SQL injection site in order to learn how to recognize code that is prone to sql injection.

Wait, but I am using Windows that do not have grep! Well, Google is your friend, there is a number of open source tools that will help you.

We also need to check all custom SQL queries using command:

grep -H -r 'execute' * | less

grep -H -r 'params\[' * | less

Same as for the active record methods, user input must not be directly used in custom SQL queries.

p.s. Twin Peaks feature request makes developer to have experience  very similar to experience of watching Twin Peaks: “The Return” [source].

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather