Report on Testival 2016

Reading Time: 2 minutes

TL;DR

This post is my report on Testival 2016 that was held last weekend in Rijeka STEPRI center.

 

First, a few metrics.

We had in total 32 participants, with female/male ratio 12/20!

As this is free event, dropout count is important metrics for us. 15 registered testers did not show, and did not let us know that they will not show. Which means that we wasted a lot of food.

Testival was first time international event:

Serbia: 4

Hungary: 1

Austria: 2

USA: 1

Liberland: 1

We are very excited about that metric!

Here are links to Facebook photo albums (you need to be log in in Facebook): Irja, Zeljko, Zeljko, Zeljko and me.

Here is my observation. Testers were not shy, they jumped in open session proposals without fear and hesitation:

img_1739

My impression is that several testers problems spawned through all themes.

I got a chance to further model my thinking around idea: “How to kill regression user interface automated testing”, and got excellent feedback and deeper understating of the problem. New material for blog post.

For me, new thing was Splunk product.

And this event introduced for the first time context driven testing school by Ben Simo, that has years of experience of applying context driven principles in his daily work.

Mirjana introduced exploratory testing, and how to transition in daily work from test case step descriptions towards exploratory testing.

My opinion what could be done better for next Testival:

Get proactive help in organising process.

Switch Testival to be Thursday/Friday event.

Add workshops on Thursday day.

More coffee.

Add additional sponsors.

In order to fight against food waste, symbolic price for registration for which value would be given back in form of software testing book or something similar.

Here is audience feedback:

img_1754

 

Again, without our sponsors, this event will not be possible:

screen-shot-2016-09-06-at-9-08-52-pm

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to get full stack trace during gunicorn exception investigation

Reading Time: 2 minutes

TL;DR

This blog post will explain how to get full exception stack trace during the gunicorn start up issue. gunicorn is python native application server for running django application.

So you changed something in your django settings.py configuration, and gunicorn failed to start. In my case, I did not get full exception stack trace, which means I did not know what was the root cause for this issue.

Traceback (most recent call last):

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 196, in run

web_1 |     self.sleep()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 346, in sleep

web_1 |     ready = select.select([self.PIPE[0]], [], [], 1.0)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

web_1 |

web_1 | During handling of the above exception, another exception occurred:

web_1 |

web_1 | Traceback (most recent call last):

web_1 |   File "/usr/local/bin/gunicorn", line 11, in <module>

web_1 |     sys.exit(run())

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/wsgiapp.py", line 74, in run

web_1 |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/base.py", line 192, in run

web_1 |     super(Application, self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/base.py", line 72, in run

web_1 |     Arbiter(self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 218, in run

web_1 |     self.halt(reason=inst.reason, exit_status=inst.exit_status)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 331, in halt

web_1 |     self.stop()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 381, in stop

web_1 |     time.sleep(0.1)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
 Google led me to this usefull blog post.
gunicorn slweb.wsgi:application --preload
 And official gunicorn documentation explains –preload option:
Load application code before the worker processes are forked.
Because first exception describes worker spawn failure, loading application before worker spawn may trigger more detailed exception.
And that was true:
django.core.exceptions.ImproperlyConfigured: 'oracle' isn't an available database backend.
Try using 'django.db.backends.XXX', where XXX is one of:

    'base', 'mysql', 'oracle', 'postgresql_psycopg2', 'sqlite3'
 The root cause was wrong name for oracle driver.
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Android back arrow icon UX issue

Reading Time: 1 minute

In this post I will present a UX issue from a very famous engineer cultured company.

Every user interface should be developed in such manner that it is self descriptive. It should be self documented, so menu items or input fields should have labels that clearly describe their purpose. Tooltips must not be some fancy technology, but they should contains small pieces of business logic that describe how is this part of form connected with business logic.

Icons should also have that purpose. If I want to save something, please put floppy disk icon (thank you Microsoft for that mnemonic).

And then we have Android back icon. What is first mental image in your head when you see that icon? What is the purpose of that icon?

For me: I will first go right in menu, and when I get there, Android will AUTOMATICALLY make a U turn and then I will go to the left in application menu.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Feedback on my objection to gherkin language

Reading Time: 2 minutes

TL;DR

In this post I will explain how not to engage in twitter war, but with simple politeness, provoke useful feedback about important topic.

Twitter war is very serious thing. Neil Studd talked about this problem at CAST 2016 lightning talk sessions. During his talk, simple idea popped up in my head. It was time for the experiment.

In my post I constructively object to Gherkin language I expressed my opinion on Gherkin language (which is first part on objection on using UI automation as regression purpose thing).

I always tweet my blog post, and this time I mentioned mr.  George Dinwiddie, because he asked my opinion, also via Twitter.

Here was his first reaction:

screen-shot-2016-09-10-at-6-26-07-pm

He used word strawman, and as I did not know that word, i looked it up:

A straw man is a common form of argument and is an informal fallacy based on giving the impression of refuting an opponent’s argument, while actually refuting an argument that was not advanced by that opponent.[1][Wikipedia]

So I learned new english word, and he objected that I am using gherkin for wrong purpose.

Here was my response:

screen-shot-2016-09-10-at-6-36-01-pm

So I did not engage in twitter war, I politely asked for a blog post. And we have thoughtful response about gherkin language: Using a good tool for the wrong thing. And that was experiment about idea during the Neil lighting talk.

And Ben Simo also responded:

screen-shot-2016-09-10-at-6-40-12-pm

So now we know true purpose of Gherkin language.

And in my original post I forgot to mention how I get information from business people, what product should do so I can create my testing mission. I get them at the beginning of github issues, as several issue comments, written in plain english language, with references to google docs and pictures about the design of the feature.

Using that feedback, here is my follow up on gherkin objection.

We have product examples that could be automate and run so we have immediate evidence that product actually behaves  as this example documents. Which is reasonable purpose.

The problem is:

  • when only those examples are used in testing the product.
  • they are used for regression testing.
  • they are used for UI regression testing (end to end testing) as only automated tests on project

Here is what I suggest:

  • do not stop testing using “manual” (highly intellectual activity)
  • create test mission “using tools at our disposal, lets find out what exactly changed in product from previous release” and do “manual” testing in that part of product. There is no need for automated regression testing on UI level.

As I find this matter highly important for testing community altogether, I would really appreciate your feedback on this matter.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

I constructevly object to Gherkin language

Reading Time: 2 minutes

TL;DR

In this post, I give constructive example why using Gherkin as communication tool  is not productive. At CAST 2016, I gave 5 minute lightning talk “I constructively object to UI automation. This first part is about using Gherkin language as communication tool.

I sad in lighting talk (thanks Neil):

Screen Shot 2016-08-31 at 10.32.22 PM

That statement went to Twitter, and I got reasonable question:

Screen Shot 2016-08-31 at 10.32.34 PM

I consider “pretty sloppy” remark as “highly creative”.

I am giving example why I think that Gherkin language is bad for communication. And I welcome you to prove me wrong.

It is important to state that James Bach already have spoken his mind about this topic: Behavior-Driven Development vs. Testing

So here is example. Let’s write start of Shakespeare Hamlet in Gherkin:

Feature: ACT I SCENE I. Elsinore. A platform before the castle.

Scenario: FRANCISCO at his post meets BERNARDO

Given that Francisco is at his post

When Bernardo approaches in dark

Then Francisco asks who is there

Scenario: Bernardo replaces Francisco at his post

Given that is midnight

When Bernardo offers to replace Francisco at his post

Then Francisco accepts this greatfully

Scenario: Type of guard

Given that Francisco had quiet guard

When Bernardo asks him about his guard

Then Bernardo confirms that he had quiet guard.

And here is original play:

ACT I

SCENE I. Elsinore. A platform before the castle.

FRANCISCO at his post. Enter to him BERNARDO
BERNARDO
Who's there?
FRANCISCO
Nay, answer me: stand, and unfold yourself.
BERNARDO
Long live the king!
FRANCISCO
Bernardo?
BERNARDO
He.
FRANCISCO
You come most carefully upon your hour.
BERNARDO
'Tis now struck twelve; get thee to bed, Francisco.
FRANCISCO
For this relief much thanks: 'tis bitter cold,
And I am sick at heart.
BERNARDO
Have you had quiet guard?
FRANCISCO
Not a mouse stirring.

Notice the difference?

When we try to fit interesting user stories in Gherkin Given When Then format, we lose a lot of beautiful and important information. I read a lot of blog posts “How to effectively written scenarios in Gherkin format”.

For example: Describe every scenario in ONLY three steps, Given When Then !?

But scenario, not feature, is core of every application. They describe how application features, building blocks of scenario, are combined together to provide VALUE for the USER. This is how application will be used by REAL users. Do you really think that users think in Given When Then format?

This is like we have a hammer and we will make Eiffel tower with only hammer, here are the instructions.

Hamlet in gherkin is not a Hamlet. Why not read actual Hamlet in order to be able to test it?

Another point. Gherkin was introduced as communication tool. Gherkin is documentation communication tool.

So here is alternative that I propose. As I am testing various projects, for me the best sources of communication are:

  • application under test
  • tester that provides information about application and is happy to share it with others
  • developers that implemented that information into the application
  • git (github, bitbucket, gitlab) – to find out what changed in application

And when I communicate with them (or just listen their conversations or read slack chats) I always get UP TO DATE, LATEST information about application under test.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather