Category Archives: RIMGEA

Tweeting with Firefox is different than tweeting with Chrome

Reading Time: 1 minute

TL;DR

In this post I will explain how simple change in testing environment influenced my testing of simple feature.

Feature is rather simple, tweet something interesting from the application. As this feature is tested, I was creating automated script for it. This feature is important for my client, so I decided that it should be part of regression testing suite.

I developed script for Chrome, and I also decided to check it using Firefox. Remember that in RIMGEA we learned that G stands for Generalize. We need to test our feature by changing the environment, in this case browser itself.

And in Firefox script failed! Why? Because Firefox tweet feature has one extra step in domain of twitter application.

 

firefox_tweet_3

 

It is clear that Firefox requires one extra button click. Since prior to that click, user clicks on “Log in and tweet” button, this is not consistent with that button. User expects that with one click he does two actions, log in to twitter and tweet that item.

chrome_tweet_2

 

 

With this post I amplified on real case how important is to vary your testing environment because in that way you could find interesting issues within the application.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Black hole and RIMGEA

Reading Time: 1 minute
credit: bug.hr

TL;DR

In this post I will explain what black hole and RIMGEA acronym have in common.

dr.sc. Dario Hrupec is a speaker at following Testival tester’s event (free admission is here). I found Dario’s work interesting and connected with software testing when I read his article about black holes (croatian version is here and Google translate is here).

Every object in order to be able to “escape” from other object, needs to be accelerated to the escape velocity speed. Black hole is an object that has so strong gravitational field that even light speed is not enough for light to escape that object. So we see that object as black hole.

Luckily for us, in human context, black hole is rare event. In the RIMGEA acronym, G stands for generalize. In order to generalize issue, you need to uncorner your corner case. Which means find corner values that are not at extreme borders. Values that are more probable in user context.

Would you report black hole as an astronaut security risk on a trip from Earth to Moon?
And on a trip from Earth to another galaxy? I would raise black hole as a security risk only in later case.

If you ask yourself: Am I a good tester? Try to revalidate your issue reports. Issue report is one of the crucial tester artifact, it is a written proof how good tester are you. Always generalize your issue report. Uncorner your corner cases. Do not report black hole as a security issue on your project. Unless you are preparing for intergalactic travel.

If you want to learn more about RIMGEA acronym, here are BBST free resources.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Selenium webdriver issue report mapped to RIMGEA

Reading Time: 2 minutes
credit: seleniumhq.org

TLDR
In this post I will map instructions how to report selenium webdriver issue to RIMGEA acronym. Those instructions should be used in your project.

Selenium webdriver, also known as Selenium 2.0, introduced webdriver API that made driving a browser natively locally or remotely more transparent. As I was using selenium behind clients corporate firewall, I thought that I found an webdriver issue. As I finished AST Bug Advocacy  course, I used knowledge and craft learned and gained at that course to report an issue.

Selenium project has rules how to report an webdriver issue. In that way, they get better community bug reports. By following those instructions, we are using A from RIMGEA acronym, and that stands for Articulate. Bug report will be articulated by following those instructions.

In order to be sure that this is really webdriver issue, user should first search project frequently asked questions. This is to filter out wrong webdriver usage and configuration issues.

You need to describe the issue as accurate as possible by answering following questions:

  • What is the problem? By describing the problem, you are Articulating your bug report.
  •  Does it happen every time you run the test? If not, how frequently? Are you able to Replicate the issue?
  • Are others on your team experiencing the problem? This information will make your report more General.
  • A link to a github gist of any interesting stack traces. This will help to Replicate the issue. By why github gist? Gist is a simple way to share snippets and pastes with others. All gists are Git repositories, so they are automatically versioned, forkable and usable from Git. Using gists for exception stack trace will make you report more readable (Articulated).
  • Are you able to reproduce the failure on other browsers? This will make report more General.
  • Do you have other changes in your code that might be affecting this test? This will help you to Isolate the issue.
  • Which OS, browser and version of the browser are you using? This is vital. This information makes your report more General.
  • Ideally, a minimal test case against a public site we can run. Here are some templates you can use to create a test case: Ruby. Isolate your replication steps.
  • How to run the test (if there’s any special set up needed). Replicate information.
  • It is also very helpful if you can create a reduced test case. Isolate steps important for the issue.

If you think that bug reporting process in your company or project is not satisfactory, I suggest that you start with selenium webdriver template and tailor it to your project needs.

                •  
                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                How I failed as RIMGEA tester

                Reading Time: 2 minutes
                credit: picardfacepalm.com

                In this post I would like to give an example how I failed as a tester. I registered my blog to be collected on Ministry of testing feeds. Everything worked in order for months. After I published one blog post, I noticed that it was not published on feed list. My blog is also registered on AST blog feed, and that feed listed my blog post. In order to find out what is going on, I send an email to Rosie, the owner of Ministry of testing page.

                She replied that post is being fed, but it is not currently displaying as the latest post is over a week old and more recent posts have replaced it.

                Facepalm! I failed as an RIMGEA tester! And the actual blog post was about RIMGEA technique. I did not do RIMGEA analysis of my issue and because of that I spent Rosie’s valuable time to do analysis for me.
                I instantly remembered that one week before I accidentally published (instead of saving it) that post. I reverted it to draft state again, but publish time stamp was still present.

                Here is what I should have done.

                R stands for replicate the issue. One of important parts of replicating it is to try to remember the exact steps that led to issue. As this is usually hard, we need to observe all relevant information related to the issue.
                By observing blog post attributes, I would noticed that below post schedule attribute is post publish time stamp. Post that has never been publish do not have that time stamp. After that I would just need to compare the oldest post time stamp on Ministry of testing feed and figure out that my not shown post is to old to be shown on testing feed.
                So by just using the first part of RIMGEA, replication, I would identified what is the real cause of the described issue.

                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                Bank lockdown of my company financial data, RIMGEA analysis

                Reading Time: 3 minutes
                Credit: simpleicon.com

                I found out that my company financial data is lockdown by a Bank. Not all data, but using bank Internet banking service, I can only search for transactions that are no older than ninety days.
                I will use this application behavior as my exercise of RIMGEA acronym for bug reporting.

                Given that I am logged in
                And I go to Reports–Transactions
                And I select account in HRK
                And I identify payment transaction with negative value
                And I select date from older than 90 days
                And I select date to today
                When I click on accept
                Then search result does not contain transactions older than 90 days

                I think that this is a bug of Internet banking system, and I will try to prove that assumption using RIMGEA acronym. RIMGEA is acronym that helps testers to write better bug reports.

                R is for replicate. I successfully replicated described application behavior. As issue variable is time, I used following follow up tests. I run the search first day. I remembered the transaction that was exactly ninety days old (I had to do it on several consecutive days until I found that transaction. So test lasted several days). I did the search the next day, and that transaction was not present in search result.

                I is for isolate. Eliminate unnecessary steps and report only one issue. This issue is present for all my accounts (accounts differ in currency). So here is core of the issue:

                Given that I am in transaction report menu
                When I select any account
                And I select date from older than 90 days
                And I select date to today
                When I click on accept
                Then search result does not contain transactions older than 90 days.

                M is for maximize. I did not maximize described issue. I was just able to replicate the issue. I varied my actions by creating pdf version of search results, by printing search results and using search results navigation buttons. I entered for starting date date that is four month before current date and for ending date date that is three months before current date. I varied program settings by changing account date and time format. I varied my environment by using Firefox browser.

                G is for generalize. We need to replicate issue for different values of variables involved in this feature, less extreme values of those variables, or on broader range of systems. In this context I replicated issue for all three values of account type variable.

                E is for externalize. Lets bring this issue in front of the stakeholders. I contacted application helpdesk. I was prepared for the answer: “This is a feature, not a bug”. I looked in the application manual the fact that search only looks for data 90 days in the past. I did not find that information. Helpdesk confirmed the behavior. Quote: “We are aware of this lack of functionality. We are preparing our next release in which this behavior will not be present.”

                A is for the articulate. Write bug report in professional manner. So here is is.

                Bug report.

                Summary: Not searchable transaction older than 90 days.
                Type: coding error or documentation mismatch
                Steps:

                Given that I have transaction in previous year
                And that I am in transaction report menu
                When I select any account
                And I select date from older than 90 days
                And I select date to today
                When I click on accept
                Then search result does not contain transactions older than 90 days.

                Reproducibility: yes
                Severity: small
                Base on my experience as customer, it is very rare case that I will need transaction older than 90 months. There is alternative way (snail mail) to obtain that data from a bank.

                priority: set by project manager. From helpdesk response I figured that priority is set to low.
                component: transaction reporting in JohnDoe application
                release number: 1.04.008
                version number: Build 15176
                configuration:
                Latest Chrome and Firefox.
                Account date format: DD.MM.GGGG
                Account time settings: HH:MM:SS
                Value format: 1.000,00

                Suppose that you are developer or project manager, would I get your interest with this bug report?

                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                Report a bug well: add iPhone number feature

                Reading Time: 2 minutes

                In previous post I described how to Externalize a bug on add iPhone number issue. We are almost

                Mantis bug report fields. source: http://wiki.scribus.net/

                finished with our RIMGEA bug analysis. What is left is to create bug report itself.

                AST Bug Advocacy course teaches us that we must write bug report well, clear and dispassionate. We must:

                • make the report easy to understand
                • keep our tone neutral and non antagonistic because angry and blaming reports discredit  the reporter

                For this bug report, (here is issue description) we will use following fields:

                • summary – summary must be short and precise, it must interest the bug report reader for this bug report as soon as possible. First two words of summary must create explosion in reader attention for this bug report.
                Summary: Missing send message icon

                • report type: Type of this report, e.g. coding error, design issue, documentation mismatch, suggestion, query.
                Report type: regression functional issue
                • steps. Steps produced after we filter steps from replicate phase using Isolate tactic.

                Given that I am in All Contacts

                When I touch blue + in upper right corner
                And I touch add phone option
                And I enter my mobile phone number: +385 92 1762 984
                And I touch Done in upper right corner

                Then number label is home by default
                And there is no message icon
                • bug reproducibility: yes/no
                Can you reproduce the bug: yes, in iOS 7.1.2 and in iOS 7.0.3 simulator
                • severity: assigned by tester, with values: small, medium or large
                Severity: medium
                • priority: assigned by project manager.
                Priority: left blank
                • component: the visible item under test
                Component: iPhone add contact
                • release number: software release version
                Release number: iOS 7.1.2
                • Version (build) identifier
                Version (build) number: 11D257
                • Configuration: h/w and s/w configs under which the bug was found and replicated. Configs related to function that manifests the issue.

                Configuration: iPhone 4s with two keyboards, English (default) and Croatian. Language is English. Messages settings: iMessages: on, Send as SMS: on, Send And Receive: my mobile number, use my Apple ID for iMessages.

                That is it. I tried to file the bug report to Apple bug reporter, but I constantly receive following error message.

                Click on email message is not helpful, because I do not have configured email client on my laptop. Here is bugreportsupport@apple.com href content:



                Testers feedback

                I presented Externalize part of RIMGEA at Zagreb Software Testing Club meetup #15 First impression was that tester would need to spend a lot of time just for one bug report.
                I explained that tester does not have to note whole RIMGEA process, it only needs to produce final report. Other parts could be written in testers mind. Especially when tester masters the RIMGEA process. But RIMGEA notes could be very helpful learning tool for junior testers and that time could be counted as part of mentoring process.

                Now is time to practice and use RIMGEA in your daily work. Please provide your progress as comments to this post.

                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                Externalize add iPhone number feature

                Reading Time: 2 minutes
                Taken from http://www.quickmeme.com/Hungry-Yoda

                In previous post I described generalization of add iPhone number feature. Now is time to bring this issue in front of people that matter, stakeholders.

                AST Bug advocacy teaches us that we need to externalize issue. That means:

                • What are the consequences of this failure?
                • Is comparative data available (product history and competitors issues)?
                • Have people written about problems like this?
                • What benefits does this failure interfere with?
                • WHO WOULD CARE ABOUT THIS FAILURE AND WHY?


                Lets answer those question in context of add iPhone number feature.
                What are the consequences of this failure?
                User adds new contact which is mobile number. But default contact category is home. If user does not change default category to mobile or iPhone, send message icon will not be present next to the new number. User will not be able to send message from contact list (or call list) in quickest way.
                Is comparative data available?
                Yes. Using iOS simulator for iOS 7.0.3, that failure is not reproducible.
                Have people written about problems like this?
                Yes, but not a lot. Using Google or Yahoo, I only found one complain. I tried to search apple bug database, but this is not possible (say what!?). Before judge, read more about this here.
                What benefits does this failure interfere with?
                HICCUPS oracle analysis is helpful to determine that.
                History – in iOS 7.0.3 SIMULATOR, quick message icon was present, regardless of category.
                Comparable product – Using SauceLabs service, I created Android 4.4 virtual machine and confirmed that adding contact with home category (default category number is mobile) it is possible to send message to that number using send message quick icon.
                User expectation. By searching the internet, I have not found a lot of complains. So, user expectation is broken, but not in such extent that could threaten Apple public image.
                Read more about HICCUPS in provided link. 
                Who would care about this failure and why?
                Product owner because of history inconsistency and sales department because comparable (and competitors) product inconsistency.
                We have two strong facts that could help us to sell this bug (make it fix by developers).
                In next post, I will write how to compile all gathered information so far in credible and meaningful bug report.
                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                Generalize add iPhone number feature

                Reading Time: 1 minute
                Depicted generalization [source: Wikipedia]

                In previous post, I explained how to maximize issue. In this post I will explain why is important to generalize issue and will apply generalize mechanism on add iPhone number feature failure.

                Wikipedia states that generalization of a concept is an extension of the concept to less-specific criteria. It is a foundational element of logic and human reasoning. Generalizations posit the existence of a domain or set of elements, as well as one or more common characteristics shared by those elements.
                For tester, concept is failure. AST Bug Advocacy course taught me how to generalize a failure. We need to uncorner our corner cases. This could be done by showing failure (or its absents) under less extreme conditions. Second tactic is to show failure (or not) on broad range of systems.

                I have already done that in maximization post. During the assignments in Bug Advocacy course, I noticed that Generalize and Maximize from RIMGEA acronym usually overlap.

                In order to uncorner my test case, I used “+385921” number because that number is the shortest number that contains Croatian international code 385, and local mobile operator code, 92. 1 is for the shortest none existing number. None existing because I wanted to be sure that failure is not connected with possible iOS ability to know that number is mobile by doing digit analysis. Failure was reproducible using that number.

                I run test on iOS 7.03 simulator. Failure was not reproduced on that environment. That gives us indication that this could be regression issue.

                In next post I will try to explain how to Externalize the issue.

                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                Maximze add iPhone number feature

                Reading Time: 2 minutes
                Spectacular Tesla Coil (Wikimedia Commons)

                In previous post we described how to isolate failure in bug report. Next task is to do follow up testing in order to try to demonstrate this failure in more spectacular way. We need to Maximize (RIMGEA) it.

                AST Bug Advocacy course teaches that in order to Maximize the failure we can do four types of follow up testing:

                • vary our actions
                • vary the program’s settings
                • vary the stored data you give the program
                • vary the environment

                1. Vary our actions

                By changing default “home” phone number category I discovered that there are ten possible categories: work, iPhone, mobile, main, home fax, work fax, pager, other, custom. Only iPhone and mobile categories offer send message icon. Nothing spectacular here, but we discovered new information about the product.

                By exporting contact that contains work, iPhone and mobile categories to vcf file format, and importing back that information, nothing changed in that contact. Again, nothing spectacular, but we discovered vcf file format for work, iPhone and mobile categories.

                work category format:
                TEL;type=WORK;type=CELL:(2 )

                iPhone category format
                TEL;type=IPHONE;type=CELL;type=VOICE:(3 )

                mobile category format
                TEL;type=CELL;type=VOICE:(4 )

                Additional types? And the rule how those extra types are added is not obvious? What is going on here?
                I am stopping my analysis here, because this is another failure. Remember, we are investigating here message icon presence.
                In vary stored data technique, we will investigate how contact changes when we change vcf file format.

                2. Vary the program’s settings

                I changed two iOS settings related to contact application.
                Settings/Phone/dial assistant/off caused no change in failure manifestation.
                Settings/Messages/imessage/off caused no change in failure manifestation. Settings/Messages/sms/off when imessage not available also caused no effect in failure manifestation.

                3. Vary environment

                I used following number:

                “+385921”

                On Simulator for iPhone (Retina 4 inch) and iPad with iOS 7.0.3 when I added that number under home category, message icon was present. Not so spectacular as Tesla Coil, but we found inconsistency!

                4. Vary stored data

                I did following tests:

                • home category with valid mobile number – no message icon
                • mobile category with land line number – message icon is present
                • +++++, #####,  *******, no effect on failure

                Then I varied exported vcf data. I imported edited vcf file using email.
                I discovered that when vcf row has several types then types priority starts from left to right. For example,
                TEL;type=IPHONE;type=WORK;type=VOICE:(3 ) belongs to iPhone category, and
                TEL;type=WORK;type=IPHONE;type=VOICE:(3 ) belongs to work category.

                I have found nothing spectacular here.

                Maximizing the failure was the greatest effort so far. In next post, I will try to Generalize the failure.

                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

                Isolate add iPhone add number feature

                Reading Time: 1 minute
                Isolated Jabuka island in Adriatic sea.

                In previous post we described R (Replicate from RIMGEA acronym) on iPhone add number feature. I is for isolate.
                AST Bug Advocacy course teaches that in order to isolate a bug, we need to:

                • eliminate unnecessary steps
                • have one failure per report


                Lets eliminate all unnecessary steps. Our goal is to be still able to replicate the issue with that minimal number of steps. We can remove all Given steps that describe precondition.

                So, instead of:

                Given that I unlock iPhone
                And I was not in search mode in contacts
                And I touch Phone icon
                And I touch Contacts icon
                And I touch blue + in upper right corner

                we can have:

                Given that I am in All Contacts
                When I touch blue + in upper right corner

                One failure per report.
                In original report, observation Then section describes two symptoms instead of one: message icon is present for some time and then it disappears and progress iOS icon is shown for some time (indicates that some background task is running). Furthermore, in original report I entered number that is already listed under “my number” contact. When I entered number that is not already present in iPhone contacts, message icon does not show at all (and there is no progress icon).

                This is indication that we report two issues instead of one. Original Then section:

                Then message icon is present right to number
                And number label is home by default
                And I see iOS progress icon right to mobile network indicator
                And message icon disappears after some time

                is replaced with:

                Then number label is home by default
                And there is no message icon

                We showed how to isolate a bug in our bug report. In next post, we will try to Maximize the failure.

                Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather