Category Archives: learn testing

It is TestBash Brighton time!

Reading Time: 3 minutes

TL;DR

This is my personal cheat sheet for TestBash Brighton 2017 testing gathering.

Based on my previous conference experiences, I realized that I need a one stop shop reminder for conference activities. I will print out this blog post. Here it goes.

  1. Wednesday Pre-TestBash Meetup, 29-30 Surrey street, Brighton, BN1 3PA

Starts at 7.00PM but it does not designates ending time. I can be there at 9.30PM so I will need to check meetup page when I arrive.

My Google maps route:

Pool Valley Coach Station, Brighton BN1 1NJ, UK => 52 Regency Square, Brighton BN1 2FF, UK => 29 Surrey St, Brighton BN1 3PB, UK

2. Thursday workshop day

a. Morning

Agile Exploration Workshop::Alessandra Moreira

Teams practicing any flavor of agile development have different views on the role of a tester. As a test practitioner and hiring manager, I found one of the challenges we face as in the Software Test industry is that although Exploratory Testing makes its way to many resumes and conversations, not many people know what it really is, are unsure how to perform it in an agile context, and are unaware of how to execute it skillfully.

In this hands-on workshop, we will discuss what exploratory testing is by definition and will put it in practice by testing real software. We will examine how it is a perfect fit in an agile environment, and how to master the skills necessary to become an expert exploratory tester – in any team.

Hope for takeaway

My current testing project has zero documentation and by documentation I do not mean word document. I mean documentation that can be get by using various tools. For example, my current issue is that I need to test an api. In Ruby on rails project, I would use

rake routes

but there is no comparable command in client framework. I will try with security scanners, in order to find all routes with brute force. I hope that I will get some practice and ideas how to solve similar issues. In this workshop description I am confused with Agile Exploration title that switches to Exploratory Testing. I will definitely need clarification on that matter.

b. Afternoon

SOLD OUT – Critical Thinking for Testers, Programmers, and Managers::Michael Bolton

Critical thinking is the kind of thinking that specifically looks for problems and mistakes. Regular people don’t do a lot of it. However, if you want to be a great tester, you need to be a great critical thinker. The good news is that critical thinking is not just innate intelligence or a talent <— > it’s a learnable and improvable skill. Michael Bolton shares the specific techniques and heuristics of critical thinking and presents realistic testing puzzles that help you practice and increase your thinking skills.

Rapid critical thinking begins with four questions—Huh? Really? And? So?—that kickstart your brain and help you to analyze plans, specifications, risks, causes, effects, bugs, and anything else that puzzles you. Join Michael for this interactive, hands-on session and practice your critical thinking skills.

Hope for takeaway

To improve my critical thinking skills. To be able to develop my own testing puzzles.

3. TestBash Meetup – sponsored by QASymphony – 6PM conference location

3. Friday Pre-TestBash Run, 7AM, Palace pier

There is no Palace Pier in Google maps so I will need to check the actual location during Thursday. It is not clear what is run ending point, I hope that it is Conference location. My plan is to walk at pace of 4km/hour.

4. Friday conference day

Nine 30 minutes sessions will probably drain my intellectual energy by I plan to stay focus for those 270 minutes. No alcohol on Wednesday night and a minimum of seven hour of deep sleep. My accommodation heuristic was to pick the cheapest Booking offer. In Birmingham that was bad decision because I got glass braking noise wakeup at 4 AM. Hope Brighton does not do so heavy bottle recycling.

99 second talks

This how you need to do it:

Think of something to talk about
Ensure you are in a quiet location
Grab your mobile phone in landscape mode
Press record
Talk or do something for 99 seconds
Stop recording
Email the file or a place to download it to Rosie – rosie@softwaretestingclub.com

I am not sure how this will work. Shall we watch already recorded talks, or talks will be filmed on stage?

5. Hindsight TestBash Meetup 6PM, Conference location

Game meetup. Eric Davis wrote a great post about games for software testers: Using Games to Make You a Better Tester. I am bringing Batman Fluxx and Game of Set. I can also present Mastermind that requires only pen, paper and somebody to play with.

6. Saturday Open Space event by Richard Bradshaw 9.00AM

I always learn a lot at open space events. In Croatia we have annual Testival open space conference, I was at TestRetreat 2016, organized by Matt Heusser. Why this works? Because you are in the same place with smart testers and you can discuss whatever you like. By that, you learn a lot!

That it is, full steam pack of learning opportunities.  My tool is notebook with four pencils (red, green, blue and black).

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Randomize your test data

Reading Time: 1 minute

TL;DR

This post is about one important testing method that should be automated, test data randomization.

You should randomize test data in automated script because this will make your automated test runs slightly more user likely because application that you are testing will be used by different users that will use different data inputs.

Also, it is very usual that application business rules are data driven. Based on data values, feature behaves differently. Using test data randomization, you can discover and test hidden business rules.

Here is my real example. I randomized account number, and soon realized that feature behaves non deterministic. One attribute output values were different from run to run. I could not define the rule.

I asked developers in dedicated slack channel about this behavior, and they explained that this is client business rule. Based on account first digit, created record was labeled differently (output attribute value). Of course, that business rule was not documented in any form.

In my ruby testing scripts, I use Faker gem. Happy test data randomization!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Scrum testing heuristic: Is feature usable?

Reading Time: 1 minute

TL;DR

This post will describe how I test in project that uses scrum. Tool is Jira and context is early development phase.

We have in JIRA workflow QA phase.

Frontend has code reviews and developers are able to test (there is no automation) what they code using browser.

Here is how I test in described context.

Feature is usable when user can do main flow described in jira card, for example, create business object.

I use browser and Chrome developer tools. Cards have some story description, so I ask for clarification in card comment section. Also, I observe pull request (angular framework) to see which modules changed. I am looking for feature side effects,  when pull request contains changes not directly related to the card.

I approve card when feature is usable. Remember that context is early development phase, and scrum is iterative process, so there is no point to stop the feature if it is usable. For found issues or missing features, I create new card and bug or task, and I connect those two cards.

In BBST foundations lecture 2, you can learn how your testing differs depending on your context.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Web page cache heuristic that failed

Reading Time: 1 minute

TL;DR

In this post I will describe my heuristic for web page cache testing that failed and why it failed.

Web page cache is mechanism that stores parts of web page that does not change a lot in browser local storage. Hard part is to figure which part of page is not changing very often. A lot of scientific effort have been put in this problem.

I got a Jira card that clearly stated a story with cache usage.

During the creation of business object, cache all referenced data so when user will edit that object, all referenced data is already available.

I did exploratory test around this story, using my brain, eyes, ears and fingers. There were two paths to feature create business object. First path worked as described in user story, but second path did not. When I opened the create business object feature, all data had already been cached.

My first heuristic was that this is an issue. But after I explored the pull request connected with this card and discovered that all code changes were in one place I realized that my cache heuristic failed. Developers are using framework that helps them to write DRY code.

As create business feature component was reusable, first path to that feature already filled cache with data.

If you are in doubt how to enhance your testing craft, BBST is excellent starting point.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to replace curl data-binary value to form-data upload parameters

Reading Time: 1 minute

TL;DR

In this post I will explain how master Joda visited me in my dream and asked me important question about one of my previous blog posts: How to measure upload speed of web application upload form [link]. The post provided right information about curl parameter that enables you to see upload speed. But curl example for upload file is wrong!

I found that is wrong when I tried to measure upload speed in real application. And realized that file is not uploaded in application. By observing the copy pasted curl, I noticed that there was not Content-Length header. Bingo, lets create Chrome developer tools issue ticket[link].

Please read comments, because I learned a lot about curl upload file feature. Google developers who answered my ticket did not just dismissed my wrong curl upload file heuristic, they took extra step and educated me!

So, in order to use curl upload file feature, you need to transform something like this:

--data-binary $'------WebKitFormBoundaryoeJQRnHeojsIBayU\r\nContent-Disposition: form-data; name="fileType"\r\n\r\n5\r\n------WebKitFormBoundaryoeJQRnHeojsIBayU\r\nContent-Disposition: form-data; name="file"; filename="elixir-websetup.exe"\r\nContent-Type: application/x-msdownload\r\n\r\n\r\n------WebKitFormBoundaryoeJQRnHeojsIBayU--\r\n'

into this:

-F file=@elixir-websetup.exe -F fileType=5

You can now data-binary parameter and its value from curl.

It is important to have check mechanism for upload file. Use your application features to check is file really uploaded. curl can also help by observing application returned value.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

What is happening in our application?

Reading Time: 1 minute

TL;DR

In this post, I will try to elaborate why is often expected from me as a tester to create meaningful application error messages.

So, you are using some web application, and you get something like this( thanks to Evil Tester):

Screen Shot 2017-01-14 at 2.04.02 PM

And that error message does not tell you anything about your action or input. At least they put disclaimer: “That is all we know”.

While I am testing application, I find a lot of such not informative error messages. Why I find them and developers not? Because I know how to make reasonable scenarios that trigger those errors. And to come up with such scenarios and test data is not something of interest of regular developer. In my experience they find this very boring and not creative job. But I think that finding alternative flows by exploring api that you use is very creative and important job.

Developers just have different meaning for definition of DONE. Or they are just lazy.

First line of defense is that user will never do that. But you took BBST Bug advocacy, and you have already uncornered your test scenarios making it more probable in real life.

After I talk to project leader and provide him information about low quality error messages, I put extra work on my shoulders. Karlo, as you already found those error messages, you will create meaningful error messages.

The problem here is that I can only provide error messages for scenarios that I covered. But low level mechanics of the application, which includes inter module cooperation based on api contracts, this is developer responsibility.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Technology junky

Reading Time: 2 minutes

TL;DR

In this blog post I will describe my TV upgrade that should last for next 10 years.

This summer was Football UEFA Euro competition year in France. Heuristic is that you could by new TV models with significant discount. I bought my last TV in 2008, and it was plasma. TV technology have dramatically changed in those eight years and I did know where to start. My Christmas started earlier this year. I asked my friend, who is real TV junky, for help. He needed just one input, my budget. With my budget of 1500 eur, he started his research. Believe me, it is not an easy task. After few pointers and one week, he hit a jackpot, LG-49UH8507, in my budget range.

This is my short review wearing a testers hat. Basically, I bought computer system . Every computer needs operating system.

WebOS 3.0

You can find technical specification here. From user perspective it is very fast and reliable. UX is pretty good, menus and actions are strait forward. I enabled auto upgrade which is done in sleep mode. When I turn it on, I get notification in upper right corner about the upgrade.

Network

WiFi and Ethernet.

I use ethernet for streaming services. Big surprise was when my Iphone 6 offered to connect to TV while I was watching Youtube. It is called screen sharing.

Remote

It has magic mouse feature, which means mouse pointer that I steer remotely. Remote UX is not so good, because I mostly use it in dark room and I can not guess button that I need based on its position or texture structure.

Display

UltraHD which means 4K resolution. This is Netflix Dardevil in 4K (click for larger image):

img_1231

Entertainment and education

I watch 90% of my time Netflix or HBOGo. Up till last update, Netflix had stability issue. I always exited application, but when I reentered it, it always crashed. HBOGo has streaming issue with new items, but it is stable. Netflix has much batter UX experience when I continue watching where I stopped previously. No streaming issues at all even with 4K items.

There is also youtube player, so I can educate myself from the conform of my sofa.

I am very satisfied with my new gadget. For the same price, I also got LG G3 smartphone, which was big surprise! And salesperson was very educated about the product, he confirmed all technical specs that I knew thanks to my friend.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Cleanup order in selenium headless test

Reading Time: 1 minute

TL;DR

In this post I describe on real example how to use important question for every software tester: “Did I do it in proper order?”

For one client I created environment for running selenium webdriver test suite in headless mode for firefox browser. I use docker image that contains latest Jenkins, xvfb server and firefox browser. I was not able to start chrome with xvfb because chrome has much higher security. More about that environment in following posts.

When I run test suite, I got following cryptic exception:

Failure/Error: Unable to find matching line from backtrace
     Selenium::WebDriver::Error::UnknownError:
       Failed to decode response from marionette
     # ./spec/spec_helper.rb:93:in `block (2 levels) in <top (required)>'

Line that triggered exception was:

@driver.quit

With help of this excellent blog post,  I find how to ask right question: “Did I use proper order of calling different api methods?”

I first closed headless driver (which is xvfb server), and then I tried to close selenium webdriver. That lost connection to its X server manager (xvfb).

After I first closed selenium webdriver, and after it I closed headless xvfb server, test suite worked as expected.

When you investigate what went wrong, use this simple question:

“Did I use proper order?”

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Search in project for method usage using only bash script

Reading Time: 1 minute

TL;DR

This post is about simple bash shell scripts that finds all files that use particular method.

In my previous post: Product moving parts as source for test strategy, I described how I use github pull request in order to discover which part of application changed in order to create regression test strategy.

Code that changed could be some helper method or .css and .jpg assets that are used in various places in code base. And those places are not part of pull request. So I need an automated way to find all places where is that helper method used.

For that purpose I use simple bash script. You need to know loop programming concept and a unix utility commands cat and grep.

Here is bash script:

#!/bin/bash
for i in `cat pull_request_items.txt`
do
  echo $i
  grep -H -r "$i" * | grep -v cache | grep -v manifest
done

And content of pull_request_items.txt

our_overview_gettingstarted.png
tour_overview_lessonhighlights.png
tour_overview_originals.png

Script reads items from txt file, and each row value is searched in project codebase using grep utility. Search is recursive in all subfolders.

Output contains files that contain searched items.

Manual part was to copy/paste from pull request to pull_request_items.txt file and do some editing in order to clean not important pull request information.

Why not use some fancy editor like sublime? Because presented utilities are installed on almost every linux machine in the world and i can use this script out of the box.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Outlook broke important business feature

Reading Time: 1 minute

TL;DR

This post is about Outlook broken business feature, carbon copy recipients.

In e-mail, the abbreviation CC indicates those who are to receive a copy of a message addressed primarily to another (CC is the abbreviation of carbon copy). The list of recipients in copy is visible to all other recipients of the message. In common usage, the To field recipients are the primary audience of the message, CC field recipients are others to whom the author wishes to send the message publicly, and BCC field recipients are the others to whom the message is sent.[source]

Carbon copy feature in email is de facto a standard business rule. If you are implementing email client, you should implement cc feature in that way.

And here is Microsoft 2016 cc feature:

issues_from_te_void_2 issues_from_te_void

And here is received email:

issues_from_te_void_2

You can not distinguish to: recipients from cc: recipients.

Business impact.

I received such an email, and replied to project manager to explain who is responsible for subject of his email. He replied (not very pleasantly, but that is the different issue) with highlighted to:  in message conversation automatically attached in message reply, and that I should know the cc: rule. So in message reply, you can distinguish between to: and cc: fields. But not in original mail.

Can I move to other email client? Answer is no, because Outlook is mandatory tool in clients organization and I am obligated to use it.

Update after Facebook feedback (Thanks Vanja and Igor!)

This issue is not consistent with Outlook history, because in previous Outlook versions that worked. Also, Gmail is not adequate alternative email client for this feature, because gmail also does not show cc field.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather