Category Archives: BBST

BBST, bug advocacy, learn testing

What is happening in our application?

1 Comment
Reading Time: 1 minute

TL;DR

In this post, I will try to elaborate why is often expected from me as a tester to create meaningful application error messages.

So, you are using some web application, and you get something like this( thanks to Evil Tester):

Screen Shot 2017-01-14 at 2.04.02 PM

And that error message does not tell you anything about your action or input. At least they put disclaimer: “That is all we know”.

While I am testing application, I find a lot of such not informative error messages. Why I find them and developers not? Because I know how to make reasonable scenarios that trigger those errors. And to come up with such scenarios and test data is not something of interest of regular developer. In my experience they find this very boring and not creative job. But I think that finding alternative flows by exploring api that you use is very creative and important job.

Developers just have different meaning for definition of DONE. Or they are just lazy.

First line of defense is that user will never do that. But you took BBST Bug advocacy, and you have already uncornered your test scenarios making it more probable in real life.

After I talk to project leader and provide him information about low quality error messages, I put extra work on my shoulders. Karlo, as you already found those error messages, you will create meaningful error messages.

The problem here is that I can only provide error messages for scenarios that I covered. But low level mechanics of the application, which includes inter module cooperation based on api contracts, this is developer responsibility.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing, testing tool

I constructevly object to Gherkin language

5 Comments
Reading Time: 2 minutes

TL;DR

In this post, I give constructive example why using Gherkin as communication tool  is not productive. At CAST 2016, I gave 5 minute lightning talk “I constructively object to UI automation. This first part is about using Gherkin language as communication tool.

I sad in lighting talk (thanks Neil):

Screen Shot 2016-08-31 at 10.32.22 PM

That statement went to Twitter, and I got reasonable question:

Screen Shot 2016-08-31 at 10.32.34 PM

I consider “pretty sloppy” remark as “highly creative”.

I am giving example why I think that Gherkin language is bad for communication. And I welcome you to prove me wrong.

It is important to state that James Bach already have spoken his mind about this topic: Behavior-Driven Development vs. Testing

So here is example. Let’s write start of Shakespeare Hamlet in Gherkin:

Feature: ACT I SCENE I. Elsinore. A platform before the castle.

Scenario: FRANCISCO at his post meets BERNARDO

Given that Francisco is at his post

When Bernardo approaches in dark

Then Francisco asks who is there

Scenario: Bernardo replaces Francisco at his post

Given that is midnight

When Bernardo offers to replace Francisco at his post

Then Francisco accepts this greatfully

Scenario: Type of guard

Given that Francisco had quiet guard

When Bernardo asks him about his guard

Then Bernardo confirms that he had quiet guard.

And here is original play:

ACT I

SCENE I. Elsinore. A platform before the castle.

FRANCISCO at his post. Enter to him BERNARDO
BERNARDO
Who's there?
FRANCISCO
Nay, answer me: stand, and unfold yourself.
BERNARDO
Long live the king!
FRANCISCO
Bernardo?
BERNARDO
He.
FRANCISCO
You come most carefully upon your hour.
BERNARDO
'Tis now struck twelve; get thee to bed, Francisco.
FRANCISCO
For this relief much thanks: 'tis bitter cold,
And I am sick at heart.
BERNARDO
Have you had quiet guard?
FRANCISCO
Not a mouse stirring.

Notice the difference?

When we try to fit interesting user stories in Gherkin Given When Then format, we lose a lot of beautiful and important information. I read a lot of blog posts “How to effectively written scenarios in Gherkin format”.

For example: Describe every scenario in ONLY three steps, Given When Then !?

But scenario, not feature, is core of every application. They describe how application features, building blocks of scenario, are combined together to provide VALUE for the USER. This is how application will be used by REAL users. Do you really think that users think in Given When Then format?

This is like we have a hammer and we will make Eiffel tower with only hammer, here are the instructions.

Hamlet in gherkin is not a Hamlet. Why not read actual Hamlet in order to be able to test it?

Another point. Gherkin was introduced as communication tool. Gherkin is documentation communication tool.

So here is alternative that I propose. As I am testing various projects, for me the best sources of communication are:

  • application under test
  • tester that provides information about application and is happy to share it with others
  • developers that implemented that information into the application
  • git (github, bitbucket, gitlab) – to find out what changed in application

And when I communicate with them (or just listen their conversations or read slack chats) I always get UP TO DATE, LATEST information about application under test.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing

Stairs help for blind people

Leave a comment
Reading Time: 1 minute

TL;DR

This post is follow up on my post about traveling to CAST 2016 as software tester. In that post I mentioned that I noticed at Paris Charles De Gaulle Airport metal pattern at the top of the stairs. Using Google, I tried to find out the purpose of that pattern, but with no result. Then community kicked in and I got response about the purpose of that pattern (tanks to another_one and Bruno Prsa).

That this pattern helps blind people to identify where stairs begin. This  blog post describes testing techniques that could help me to get to that conclusion by myself.

Michael Kelly article: Taking a tour through test country presents application tours that helps to get to know with the application. Two tours could have helped me to identify purpose of the pattern: user and scenario tours.

Kelly states:

The first is the user tour. In this tour, you attempt to imagine five users for the product and the information they would want from the product or the major features they would be interested in. The second tour is the scenario tour. Here, try to imagine five realistic scenarios for how the users identified in the user tour would use this product.

In BBST Test Design, you can learn about James Bach Heuristic Test Strategy Model where one of test technique is user testing.

Are you ready to enhance your testing craft?

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing

Alternative flow in my neighborhood store

1 Comment
Reading Time: 1 minute

TL;DR

In this post I will explain how I tested alternative business flow in my neighborhood store. No computers, no automation, just human interaction.

Few months ago, my local store put a big sign with a business flow:

“If you find that item price do not match, or it item does not have a price tag, we will charge you only half the price.”

For one of my items I do not remember the exact price, but my heuristic is that this item should never cost more than 6 $. And this time price on the register was more than that amount.

So I reported my issue, and pointed to the big sign on the wall (consistency heuristic with claims). They confirmed me, but immediately started with excuses, why the price did not matched. I did not hesitate, show me my money!

So they put me aside, because that alternative flow required some processing of my invoice through their system. Lady at the cash register did not know how to do that, so they asked for the manager. In the end I got my money back.

Conclusion

When you find an issue in product that you are testing, always report that issue. They will use against you various psychology tricks to persuade you that this is not an issue.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing

Application variability tour in action

Leave a comment
Reading Time: 2 minutes

TL;DR

My first contact with software testing was in factory and control schools of software testing. At that time I did not know that fact. In this post I will explain how I needed to make a shift from those schools, in order to deliver better software testing to my clients.

In my current software testing project, I got question: “Do you have all the requirements in order to start your testing activities?”. I said: “For start, I only need the application.”

By that I meant, I will start to explore the application based on product tours created by James Bach and expanded by Michael Kelly [source]. This paper is required reading for BBST Test Design course [source].

The main idea is that you can learn about the application by doing application tours. I will explain how variability tour helped me to identify potential problem in the application.

When you are doing application tours, from tools you only need a pen and paper. Simple as that. What counts are notes that you will put in your notebook.

Idea behind the variability tour is to find things that you can change in your product, and then try to change them. These are the moving parts of the application. And you will start to think about the application data that are part of those changes. What gets challenging here is that you realize that moving parts are connected and dependant on each other.

For better understanding this problem, see James Lyndsay’s black-box testing machines [source].

I tested a feature which purpose was to move application item through various states. This is known as State pattern [source]. Item has states and actions that are used to move item through those states.

I was doing coverage test of that feature. Create an item and move it through all available states. A variability tour. And I found an interesting feature that could be a possible issue. In every state, item had a same list of possible actions. There was a helping action, move an item to previous state. Very useful for the coverage test.

And I noticed that order of list of actions changed when item was returned to previous state from the order when item was first time in that state.

This was not the issue by itself, but by knowing that this feature exist, we could reveal some other important issue. This is example of over featuring the application. And developer did not know that this feature is implemented.

That was definitely issue for me as a tester because I needed to be extra careful to not make a mistake while I was doing coverage of item state transitions.

Conclusion

In my country, fresh graduates are not attracted by software testing, because they know about software testing from factory and control schools of software testing. By presenting them with application tours, maybe you will change their opinion about software testing.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing, test design

How BBST Test Design course helped me with Netflix setup

Leave a comment
Reading Time: 2 minutes

TL;DR

In this post I will explain how I applied knowledge gained in BBST Test Design course in order to quickly resolve Netflix issue on my Sony BDP-S4500 Blu-ray player.

 

Last year, for my birthday, I treated myself with Sony BDP-S4500 Blu-ray player. After unboxing it, I was very excited because I realized that this gadget is much more than a Blu-ray player. Notice white button with red letters: NETFLIX.

I connected player to my TV and router for cable Internet, configured it, and I pressed netflix button from pure tester curiosity. After 20 seconds, I got expected message: “Feature not available” because at that time Netflix was not available in Croatia.

On January 6th 2016., six months later, CES 2016 started in Las Vegas. Netflix announced that expands its service to 190 countries. Luckily for me, Croatia is one of those countries!

I have read several posts from Ben Simo about Netflix issues that he encountered. And I envy him a lot on those issues.

I used my Mac in order to signup for Netflix service. I choose plan for 10 euros. One of the features of that plan is ability to use Netflix on two devices. I encountered no issues with signup process.

I switched to player and hit Netflix button again. Feature was again not available. Ok, now we have a problem here. But with Netflix or player device?

BBST Test design kicks in (BBST Test design slides, page 174, Initial states)! Feature not available was returned without asking me for any information. My heuristic was that player was “stucked” in state where Netflix availability check was done. And did not initiate new check.

How to reinitiate new Netflix check?

First, I updated player software to latest version. No change. Should I reset player settings? But I would like just to reset customer information data, particularly data that states that Netflix is not available for player public IP address. How to do that? I should consult player claims (documentation). Fastest way to do that is Google.

After googling for 5 minutes, I found the solution on this link, posted by user BrianP.

 

Do Initialize Personal Information.
1. On the supplied remote, press the HOME button.
2. Go to Setup.
3. Under Setup, select Resetting.
4. Under Resetting, select Initialize Personal Information.

 

Sony hid Initialize Personal Information, as submenu of Reseting menu. Which is probable inconsistency with user expectations. As a user, I was afraid to select Reseting in order not to start factory reset of player device.

 

Action 5. was click again on Netflix button, and I was in! Sign in, remember me, and Daredevil and 12 Monkeys season 1. episode 1. were ready on my eight year old Panasonic Viera series plasma TV.

netflix_viera

I have watched three shows so far, without any issues. Maybe is a good thing not be early adopter of such a cool service.

 

Conclusion.

Take BBST test design course, but do not forget to practice it not only in your daily job, but also in daily life! And write a blog post about it.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing

How to pet a cat

Leave a comment
Reading Time: 1 minute

TL;DR

In this post I will explain how I used my software testing craft in order to learn how to pet a cat.

Picture shows Sivka. She enjoys more than any other cat in my neighborhood to be pet. The issues is that starting the pet session is not an easy task. Sometimes I managed to do that, but most of the time she run away from me.

So I switch on my software testing skills. I compared the environment when I failed and when I succeeded in petting session. I crunched the testing data taking into account fact that Sivka could easily get frighten.

At that time I read excellent blog post from Berine Berger: The Inefficiency of Gifts and Software Metrics. In that blog post there is a link to Evaluating test plans using rubrics and in that paper he explains quality assessments.

Quality assessment are:

  • judged based on characteristics
  • empirical –  verified by observation
  • experiential – verified by experience
  • usually reported in words
  • usually contrasted with quantitative

In order to start petting session I needed to approach her much slower with my hand. But how much slower? I do not have any tools that could help me to measure the approach speed of my hand towards the Sivka. So I used quality assessment and real time feedback. I started my approach with one velocity, and adapted that velocity if I noticed that Sivka started to get frighten.

Now I know what is the approach speed of my hand. I do not have quantity value of that speed (to be written in test script or test case), but that speed is now written subconsciously in my brain, the greatest software testing tool!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST

BBST Instructor: week 1

Leave a comment
Reading Time: 1 minute
credit: http://www.servantsofgraceapologetics.org/

TL;DR

In this post I will provide guidelines how to cope with information overload as BBST Instructor in first week of a course.

First you should create label in your email client which helps you to easy find emails of other instructors and link to Google spreadsheet of course tasks list.

Go through the task list and get to know with tasks that you picked. Fieldstone link and Instructor’s manual are your tools. Be sure to do them before the deadline that is stated in the spreadsheet.

Then, subscribe in moodle to all forums in the form of daily digest mail. Otherwise, you will look like a truck in the picture. Schedule a time when you will go through that email every day.

Week 1 tasks that are common through all BBST courses are:

  • philosophy of quizzes
  • preview lesson 2
Preview lesson 2 is a great task because you need to read through instructor’s manual and you recap what are learning goals for that lesson!
You should also get opportunity to provide feedback in forum discussion, especially ones related to lesson task and when you detect that group or student having issues with grasping lesson learning objectives.
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST

BBST Instructor: week 0

1 Comment
Reading Time: 2 minutes
credit: http://www.keepcalm-o-matic.co.uk/

TL;DR

Being BBST Instructor is a challenging task. In this and following blog posts I will try to provide missing link for BBST Instructors, guidelines based on my experience that could help you to be effective BBST Instructor.

It is important to state the context, I am talking about BBST Instructor for courses created by Association for software testing organization. But you will say: AST is context driven! They do not believe in guidelines and best practices. Lets think on this guidelines as BBST Instructor framework, version 0.1. You can always fork those guidelines and tailor them. You can do pull request of your version and AST members will evaluate it and probably accept your pull request.

Preconditions for becoming BBST Instructor:

Toolbox:
  • BBST Instructor’s manual (provided at Instructor course)
  • AST Fieldstone projects (provided when you apply as instructor)
  • Instructors task list
Week 0
Yes, your work starts one week before course start. You need to help lead instructor to check the course materials in moodle. As process of setting up new course is based on existing template, all link would probably work. You should set your focus on dates present in materials. Pick one lesson, announce your work in mail list to other instructors, and do your work.

Also, keep in mind that you are tester, and as in any other test project, you will not have enough time. So it is not possible to provide feedback to every student discussion. It is not even possible to read all student posts!

So what can you do? In order to make a tradeoff regarding providing feedback, learn in instructor’s manual what is the learning goal of every lesson. Provide your feedback to some students that have trouble reaching that goal.

Commit your work in instructor’s task list. Be proactive! Lead instructor does not have time to assign you a task. Pick a task, put x in task list, read in fieldstone project where feedback is already provided and tailor your feedback if needed. Fieldstone project is set of materials that was created by previous instructors and that material will help you to make your  tasks.

Guess what? In that way I learned a lot about course material. Do you really want to miss that opportunity?

I have heard a lot that software testing craft is not properly acknowledged in software development community. By becoming BBST Instructor and teaching one of the AST BBST Instructor class, you will take software testers destiny in your hand. Food for thought. What would make bigger impact on software development community: a group of 50 BBST Instructors or group of 1000 BBST Instructors?

And there are companies that make a business from BBST courses. What do you think, where they would start looking for new BBST Instructors? 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
BBST, learn testing

My experience as AST BBST Co-instructor

1 Comment
Reading Time: 2 minutes
Credit: associationforsoftwaretesting.org

In this post I will give my experience as AST BBST instructor for course Instructor Course. I am member of Associations for Software Testing and I successfully passed Foundations and Instructor course. These are prerequisites to be eligible to take Instructor course. When I successfully finish Instructor course, I got detailed instructions how to enroll as AST BBST Instructor.

 It is important to state that BBST is a registered trademark of Kaner Fiedler & Associates LLC. Being AST BBST instructor is a volunteer activity.

BBST Instructor is the least demanding course, so in my context that I have eight hour daily job and family, it was a good starting point as instructor. I create commitment for myself and that is that I will read every student post related to course lab activities.

Lead instructors were Justin Rohrman and Markus Gärtner. Justin first send Google spreadsheet with instructor task list. I could apply for any of those tasks, and every task had a deadline stated. Using that information, I could manage my workload and decide (given that this was my first course as instructor) which task to choose. My first task was to review the student task list spreadsheet and is everything ok with moodle course site (links,  text, forums). I choose lesson preview tasks. I needed to give students brief introduction what is next lesson all about. I put first draft in instructors forum in order to get feedback from lead instructors.

As I was reading student post to lesson lab task, I also provide several feedbacks using my knowledge that I gained in Instructor course as a student. In the end we needed to provide for every student our pass/fail decision along with comments for fail decision.

I had several takeaways from being co instructor. First was that by providing lesson previews and student feedback I actually did a recap of lesson materials which helped me to better comprehend and extend my understanding of Instructor course materials. In total I spend 12 hours over the two and a half week course duration. Second was that by reading student responses I also extended my Instructor course knowledge. I met a Miagi-Do School of Software testing instructor, Michael Corum, and Lucas Dargis uTest community contributor, testers for which I heard before by reading their blog posts.

I recommend you to become AST BBST Instructor if you want to share your knowledge with other testers and to extend and recap your knowledge from AST BBST courses. However, you also need to put some commitments from your side, and that is your free time.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather