Writing time: 1.5 hour
TL;DR
A real case example of test design based on user testing.
I have to be honest, I do not remember when I last time had done user testing. But after one situation from Van Gogh museum in Amsterdam, I realized that I have been doing it wrong.
BBST Test design states:
User testing - This testing is done by the types of people who would typically use your product.
Ok, there are 7 billion people on earth!? So the real question is how to sample those 7 billion people to people classes (groups) that will typically use your application. Smart consultant will say: “It depends on the context of your application.”
Here is the context:
Application provides additional information about some of Van Gogh’s masterpieces. I was exploring masterpiece: View from Theo’s Apartment:
Here is short video of the application in action.
[embedyt] https://www.youtube.com/watch?v=z2nDbUSJLCM[/embedyt]
There was a tablet with installed application next to the painting. One feature was:
- drop down photo of apartment view on appropriate part on the painting in order to reveal historical information
Question: Who are people who would typically use this application?
While you are thinking (I will provide my testing strategy and end of post), I will describe my revelation about user testing.
I was exploring application feature with drop down interaction. IT TOOK ME ABOUT 45 seconds to figure out that I should use drop down! After I figured that out, a primary school pupil stopped next to me. I clicked on next feature of the app that also involved drop down. I gestured pupil to use the app. He needed 2 SECONDS to figure out that feature includes drop down.
Takeaways
Tester is typical application user in very rare ocasions. Be aware of that fact.
My try on typical users of Touch Van Gogh Application.
Van Gogh Amsterdam museum is very busy place. In 2015, there were 1.9 million visitors. Those visitors were:
- students, local and international
- tourists that like art (like me)
My heuristics is that 90% of them use smartphone. So we still have high probability of application usage. But who will try to use the application? There was no queue in front of the tablet (as compared to queue in front of famous Sunflowers). So actual sample of app users is very low. Who are those people? Me and the pupil. I am software tester, and I like to explore applications, Touch Van Gogh application is excellent exploration example. But who was that pupil? Why was he engaged?
Heuristic: Touch application will be used by children. How to test this heuristic before application deployment is another challenge.
What about other possible users? People that are into the art, that will seek more information about masterpieces. Are they experienced smartphone users? Do they know what is drag and drop, are they familiar with application phone gestures [source]?
Goal of this post is to present that to define typical user of the application is not an easy task. You need to use heuristics and you need to test them. And this is why I am into software testing!
Extra content 🙂
This is official link of Touch Van Gogh application, it contains link to Google Play and App Store. Click on App Store [https://itunes.apple.com/en/app/touch-van-gogh/id673141174] results in HTTP 400, bad request. Bug Magnet.
