Report for Zagreb STC #6 Meetup

Reading Time: 3 minutes

Yesterday we held our 6th Zagreb Software Testing Club meetup. We have been gathering now for one year! You can read more on agenda meetup here
There where ten testers, three of them where first time at the meetup. We started with tester introduction and what he tested last week. Other testers had a lot of questions during that introduction, so we got debate that lasted for almost an hour. And that is what you should expect from every true tester!  
After pizza, Manuel started his presentation on software testing. It is based on his final faculty work, and it represents one of the rare software testing materials in Croatian language. For me this is excellent starting point for someone that does not know what is software testing. Unfortunately, in Croatia there is a lot of IT professionals that does not have a clue what is software testing. We will encourage Manuel to repeat this presentation on next Viaqa (hopefully with new conference name) conference.
During the presentation, almost for every presentation topic, we had a debate and conversation. I was very pleased with that. From discussion I concluded that we are working in different ‘software testing schools‘. For now we have factory and agile representatives. At work my company legacy is Factory school, and I am trying to introduce Context driven school. In my free time, on my Elance projects, I am practicing knowledge from Context driven school. 
As I do not totally agree with Manuel presentation, here are my comments.

  • unit testing is software design technique, better known as Test Driven Development. Developer use this technique to better design his subsystem. Subsystem can not be done from the first try (unless you are Dijkstra fan). It is created in small iterations, with every iteration committed into version control, and changes covered with unit tests. Developer should prove that his subsystem works as he thought that it would work. Unit tests are that prove. Testers must also test that subsystem. Developer should prove with unit tests that reported bugs were fixed.
  •  code coverage with some percentage does not measure the quality of code. It is more important that developer creates unit tests that prove its subsystem design. And it is also important that testers test the subsystem using their brain and testing knowledge.
  • this video is very good explanation why mentioned metrics are dangerous in software testing.
  • list of testing methods is not complete. I encourage Context driven testing (by Caner and Bach).
  • I like chapter 2.9 because it mentions real testing. However, statement about negative sides of Agile testing is wrong. People who do not know how to apply Agile are responsible for problems with Agile. With chapter 2.9.2 I totally disagree. The truth about Exploratory testing could be find here. (Bolton)
  • Chapter 2.9.3 is about automated testing. I would like to state that automated testing is not only validation (pass/fail). I support idea (Bolton, Bach), that automated testing is a tool in executing testing idea.
  • Chapter 2.10 talks about Factory school. That school was attempt to implement in software testing processes from Toyota car manufacturing process. And I think that is wrong. There are also other software testing schools that were not mentioned. 
  • The most important software testing tool is our brain. I have experience with Mantis that I use in my company not only for bug reports, but also for task management. In my company it took 1.5 year to everybody accept Mantis for that purpose.

In overall I think that Manuel’s work is very good starting point if you need to prepare for some kind of testing certification.In the last minutes of meeting, Zeljko explained to us Cucumber scenario outlines. Zeljko also wrote about the meeting.

In the end, I would like to state that I am very pleased with direction of Software Testing Club meetup! Marko, a regular meetup attender, also wrote his thoughts on meetup discussions. 
See you in two months.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing in 140 characters (or less) #6

Reading Time: 1 minute

In discussions of software #testing, people often say to me “In the real world, I have to…” just before they describe some way of faking.
— Michael Bolton (@michaelbolton) September 11, 2012

bit.ly/Qdv3o5 @jhunterj @salvius23 And instead of thinking “pass”, think “appeared to meet some requirement to some degree.” #testing
— Michael Bolton (@michaelbolton) September 12, 2012

@michaelbolton One answer: testing that ignores problems and value.
— George Dinwiddie (@gdinwiddie) September 12, 2012

bit.ly/RIEj5g User experience is also about responsiveness, compatibility, support, elegance, flow. Are you #testing for those?
— Michael Bolton (@michaelbolton) September 12, 2012

Bugs are bad, but confusion, misunderstanding, and disagreement are the nests where bugs develop. Give priority to reporting nests. #testing
— Michael Bolton (@michaelbolton) September 13, 2012

If you think #testing is the bottleneck, another problem is that there are things around the bottle that are poorly understood.
— Michael Bolton (@michaelbolton) September 13, 2012

@aquintian If you’re a tester, it’s very unlikely that you have control over schedule, budget, staffing, release date, product scope (1/2)
— Michael Bolton (@michaelbolton) September 13, 2012

@aquintian or source code. So the best you can possibly do is to provide information to those who DO have control over those things. (2/2)
— Michael Bolton (@michaelbolton) September 13, 2012

@aquintian So the question arises, then:on what, exactly, do they base a *demand* for you to deliver a high quality product?
— Michael Bolton (@michaelbolton) September 13, 2012

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing in 140 characters (or less) #5

Reading Time: 1 minute

Feeling stuck when #testing? To get unstuck, try changing something, anything, big or small, /right now/. Stand up; doodle; change fonts.
— Michael Bolton (@michaelbolton) August 25, 2012

Photo radar doesn’t prevent speeding; it tells us if speeding happens. Awareness of it helps some remember to be conscientious. 1/2 #testing
— Michael Bolton (@michaelbolton) September 5, 2012

In the same way, checking doesn’t prevent bugs; it tells us about some bugs, and may remind some of us to be careful. 2/2 #testing
— Michael Bolton (@michaelbolton) September 5, 2012

“Passing a test” means “meeting expectations”, but we don’t often take time to examine those expectations in the first place.
— Jon Bach (@jbtestpilot) September 5, 2012

Testers; slaying beautiful hypotheses with ugly facts.
— Ben Simo (@QualityFrog) September 6, 2012

Testers: Disturb people. Disturb their thinking. Ask questions that disturb settled lines of thought; provoking new ideas.
— Ben Simo (@QualityFrog) September 6, 2012

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Test case challenge from uTest forum

Reading Time: 3 minutes


Last week I found at the uTest forum a test challenge. Details about the challenge could be found hereThe challenge has two main objectives and a set of explicit requirements. First part of challenge is to identify the test cases and second part is to group them, so customer could easily understand and rerun them.

uTest is growing testing company and I think that they created this challenge in order to attract good testers. My goal is opportunity to learn testing by testing my current testing knowledge and practice. By posting my solution I will receive feedback from the test challenge author, Lucas Dargis, and other testers interested in this challenge.

I choose following approach for this test challenge. In order to structure and group test cases, I first created the state machine model of the application. My drawing of state machine model is here. More about using models in software testing could be found in this great Michael Bolton blog post. So I would like to emphasize that this is just one of the possible models of the application, and it represents just some part of the application under test. This model tries to capture  implicit functional requirements not listed in testing challenge. If the application under test was alive, by using that application I would expand this model. But it is a good starting point for functional testing. 
Here is explanation of the state machine model. It has three states and six transitions.
states = [ ‘DISCRETE’, ‘CONTINUOUS’, ‘ARRAY’ ]
Every state represents what application displays in its output filed. For example application is in ‘DISCRETE’ state means that user entered valid discrete number, pressed ‘Go’ button, and output filed displays only that valid discrete number. Application remains in ‘DISCRETE’ state. Two possible transitions could happened.
Model has six transitions.
transitions = [ ‘no input or discrete’, ‘no input or continuous’, ‘no input or array’, ‘continuous’, ‘discrete’, ‘array’ ]
Transition name is what type of input user used before entering the ‘Go’ button.
On previous example on user action, depending on the starting state and assuming that application is in ‘DISCRETE’ state (we can assume that any of possible states is starting application state), the transition triggered by ‘Go’ button is ‘no input or discrete’.
Following spreadsheet contains test cases. It contains four sheets:
test case groups = [ discrete numbers, array numbers, continuous numbers, state display ]
First three groups represents checking the explicit  requirements. However, fourth group is based on state machine model transitions. For example, by observing the model I created test case ‘stay in ‘DISCRETE’ state’. My goal with that test case is to observe application behavior when only discrete numbers or empty input are used in a row for some time. Other test case could be what is application behavior when transits to array state? Would application be able to detect invalid array input? Or would application properly display just array input? 

Using application state machine model tester could come up with test cases that are connected with state transitions. That means that tester could easily structurally test case execution order. When application has states, test cases for checking explicit requirements are not sufficient. It is important to have a tool that can help testers to group test case execution in particular orders. State machine model of application under test is one of the tools. Using it with test automation, tester could easily execute those state transition test cases.
Also, by presenting the customer application state machine model, customer gets the visualization of the application. Observing that picture, customer gets the idea how application actually works. 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing in 140 characters (or less) #4

Reading Time: 1 minute

Making GUI test automation less fragile often means making it resistant to change it should detect, and therefore less valuable. #Paradox
— Ben Simo (@QualityFrog) August 11, 2012

@jarilaakso It doesn’t necessarily make a tester worse, but a tester drawn to or obsessed with coding is probably not studying testing.
— James Marcus Bach (@jamesmarcusbach) August 12, 2012

@mheusser There is more to testing than finding bugs.Building trust and collaboration with the team is half the battle that is to be won
— Mohinder Khosla (@mpkhosla) August 13, 2012

Want a reasonable first-order measurement for an exploratory #testing session? Ask: how many new leaf nodes did you add to your mind map?
— Michael Bolton (@michaelbolton) August 22, 2012

Don’t take bit.ly/O4Kzynas a rigorous measure, but discovering new coverage areas is a valuable outcome of exploratory #testing.
— Michael Bolton (@michaelbolton) August 22, 2012

A well-framed story about bit.ly/O4Kzynis a fine *qualitative* way to measure the productivity of exploratory #testing sessions.
— Michael Bolton (@michaelbolton) August 22, 2012

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

My contribution in spreading what is software testing

Reading Time: 1 minute

In order to become better tester, I read a lot about software testing. My primary source are testers that I follow on twitter. They are my “testing learn” twitter list. I also have ‘programming learn’ twitter list. And on that list I came across on following blog post:

http://qualitypoint.blogspot.com/2010/04/software-testing-terms.html


My obligation was to comment on that post. I have this blog post because I am not sure if author will approve my comment.


Hi!


This blog post is only useful if you want to pass ISQTB exam. It will not help you to test software in such manner to help team to produce better product. For example:
“It will help to make sure that we are not having any test cases for the requirements which are not asked by the customer.”
It is not possible to write down all the requirements. What about implicit requirements? For example:
System will not crash down if I try to login with Croatian characters.”


This post is example how process from automobile industry were unsuccessfully copied to software industry. In order to understand what is software testing, start with this video.

http://www.youtube.com/watch?v=ILkT_HV9DVU


Regards, Karlo.  

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing in 140 characters (or less) #3

Reading Time: 1 minute

@QualityFrog That’s true; /a performance/ doesn’t pass or fail. A performance focuses attention on interesting stuff. @testobsessed
— Michael Bolton (@michaelbolton) July 19, 2012

My eye saw ‘profs’ at first. RT @jamesmarcusbach: “Testing cannot demonstrate the absence of bugs. NOTHING can do that, not even proofs.”
— Michael Bolton (@michaelbolton) July 21, 2012

“What we see depends mainly on what we look for.”-John Lubbock #testing #quality ow.ly/cpEg1
— Gerald Weinberg (@JerryWeinberg) July 22, 2012

@imccowatt Taleb says that failure prevention is never noticed, and that’s “monstrously unfair”.
— Michael Bolton (@michaelbolton) July 23, 2012

@imccowatt @paulholland_twn Exactly: oracles can be used generatively, in the moment, or retrospectively. #testing
— Michael Bolton (@michaelbolton) July 23, 2012

The implicit theme of Rapid Software #Testing is: testing work doesn’t have to suck, neither in the doing nor the results. @jamesmarcusbach
— Michael Bolton (@michaelbolton) July 25, 2012

If you’re worried about regression, ramp up review, research, pairing, refactoring, #testing, investigation. Checks alone won’t do. #agile
— Michael Bolton (@michaelbolton) July 27, 2012

Worry about regression implies worry about weak knowledge of the code, excessive speed, sloppiness. Consider fixing those. #agile #testing
— Michael Bolton (@michaelbolton) July 27, 2012

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing in 140 characters (or less) #2

Reading Time: 1 minute

Scientists don’t like anomalies in the same way devs and testers don’t like bug – @c_wiedemann #CAST2012
— Anand and Komal (@testinggeek) July 17, 2012

Software testing is boring – for non-testers. We have to find ways to expose effects of testing & make ppl aware of it. @mgaertne #cast2012
— lisacrispin (@lisacrispin) July 17, 2012

Everybody is a tester; it just happens that we’re really good at it. @testobsessed #CAST2012
— Markus Gärtner (@mgaertne) July 17, 2012

I believe testers “make stuff up” a lot less than testers uncover and make explicit things that have been latent and implicit. #CAST2012
— Michael Bolton (@michaelbolton) July 17, 2012

Saying “programmers can’t test” is silly; they do it all the time. Like everyone else, they can’t see ALL errors in their own work. #testing
— Michael Bolton (@michaelbolton) July 17, 2012

@michaelbolton We uncover ways in which value is destroyed, in ways that have often not been considered
— Iain McCowatt (@imccowatt) July 17, 2012

@imccowatt Yes. I’d say s/destroyed/threatened, though; the latter encompasses the former. Oracles aren’t perfect; testers aren’t judges.
— Michael Bolton (@michaelbolton) July 17, 2012

@imccowatt Yes; not judge and jury, but exercising judgment about requirements, interpretations, risks, value,…
— Michael Bolton (@michaelbolton) July 17, 2012

Note every tester should code. We end up taking awesome testers and turn some into awful programmers @testobsessed #CAST2012
— Michael Larsen (@mkltesthead) July 17, 2012

@mkltesthead At least as bad, maybe worse: starting with an awful programmer and turning him into an awful tester. @testobsessed
— Michael Bolton (@michaelbolton) July 17, 2012

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Croatian character on trip to Guadalajara

Reading Time: 2 minutes

At the start of this year, my friend and fellow tester Zeljko was working in Mexico, Guadalajara. You can read about his experience here. He provided a Picasa foto album, and, because he is a tester, Zeljko asked a question in this photo comment.
Here is my answer.
As I know that Željko has Croatian letter Ž at the start of his name, and that lettrer was printed as Å½, my first assumption was that Bikla application does not work with utf-8 encoding. First, I wanted to replicate the issue.
I first found one web page with letter Ž. UTF-8 hex code for letter Ž is c5bd (two bytes). In order to check has letter Ž been acctualy encoded in web page source using utf-8 encoding, save page source as html, open it in vim, switch vim to hex mode (:%!xxd), and search for c5bd. In right column, you will see text corresponding to letter Ž (but in vim hex mode it will be displayed as ..).
Other option is to check is your browser encoding set to utf-8 and if you can see letter Ž, then you know that letter has proper uft-8 hex code c5db.
In order to reproduce Bikla issue, after loading web page, I started to change my browser encoding. First in top-down list in Chromium is Western (ISO-8859-1). And browser changed Ž into Å½. Bingo, but this time I was lucky, but skilled tester would not guess, by try to use heuristics (more on heuristics you can find here) in order to determine Bikla page encoding from the first try (there is 37 encoding supported in chromium). I also confirmed that Zeljko’s name was properly encoded in the Bikla database. Question for Zeljko: how was your name entered into Bikla datbase?
Here is my heuristic. Go to google, google for ‘spanish character encoding’ and try those encoding. Google returned following encodings:
utf-8, iso-8859-1, iso-8859-e15.
So, this time, if I will go on the encoding list from left to right, I will get the Bikla encoding from the first try, but without any luck needed.
As I tester I identified following:
the issue: Ž into Å½
testing tools: vim, chromium, google
use of heuristic, how to reproduce issue
Knoweledge: character encodings.
Conclusion: I would inform Bikla that they should implement utf-8 encoding into their application. Their decision would be based on the fact does wrong printed names affects Bikla business.
For testers: find one more Croatian letter from the picture that was wrongly decoded.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing in 140 characters (or less)

Reading Time: 2 minutes

In this blog post I will put all important testing tweets. My plan is to use this blog post as repository of testing tweets. All those tweets are important for every tester to become even better tester. I am using “embed this tweet” tweeter feature.



The first principle of critical thinking: things could be different. That’s a core principle in #testing, too. No coincidence.
— Michael Bolton (@michaelbolton) July 14, 2012


RT @jamesmarcusbach “Testers need to know the difference between observing a phenomenon and knowing the whole truth about it.” #testing
— Michael Bolton (@michaelbolton) July 14, 2012

Usabliity BASICS: the user wants to accomplish tasks, NOT exercise functions in your product. Learn and honour the difference. #testing
— Michael Bolton (@michaelbolton) July 14, 2012

“It is necessary to relax your muscles when you can. Relaxing your brain is fatal–Stirling Moss, British racing driver #thinking
— Gerald Weinberg (@JerryWeinberg) July 15, 2012

Arrange software teams so that most of the communication necessary to develop a feature happens within a single team.
— Dale Emery (@dhemery) July 15, 2012

Arrange software teams so that each team is downstream from its own work.
— Dale Emery (@dhemery) July 15, 2012

@adamgoucher – first heuristic – build your own framework and tool chain because problem is specific to the project #CAST2012
— Anand and Komal (@testinggeek) July 16, 2012

Selecting tool set and language which make sense – don’t use same language as development for the sake of it – @adamgoucher #CAST2012
— Anand and Komal (@testinggeek) July 16, 2012


Turn off all the third parties components / tests in functional test environment – explore feature switch / toggle @adamgoucher #CAST2012
— Anand and Komal (@testinggeek) July 16, 2012

Refresh machine, get VMs and run tests on different machines to identify if it’s too much attached to machine @adamgoucher #CAST2012
— Anand and Komal (@testinggeek) July 16, 2012

Start with study, not plan. #CAST2012 keynote
— Markus Gärtner (@mgaertne) July 16, 2012

Tripp Babbitt: “A focus on costs always increases them.”Me: Still, *consider* /opportunity cost/.#CAST2012 #testing
— Michael Bolton (@michaelbolton) July 16, 2012

Surely the people doing the work are those who should design the work. If so, managers’ role is to test & tune the work; remove blocks.
— Michael Bolton (@michaelbolton) July 16, 2012

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Blog that makes software testing interesting and exciting.