Introduction
I have found an issue in iMessage iPhone application service which is part of Messages Apple application. In this post I will describe iMessage behaviour and use consistency oracles for test reporting. Consistency oracles for test reporting is one of the topic that you can learn and practice at BBST foundations testing course.
The issue.
One of my contacts also use iPhone. I successfully received iMessage from that contact, but my immediate reply on that message failed with red exclamation symbol and red text “Not Delivered” next to my reply. At that time I thought that GMS network problem is the reason for that. When the issue repeated again, this time I clicked on red exclamation sign. Pop up message appeared with text “This iMessage was not delivered.” and options Try again, Send as Text message and Cancel. Send as text message successfully send my reply with notification that this is text message. This time I knew that my contact’s iPhone was at servicing and that he put his SMS card to phone that do not have iMessages application.
Consistency oracles
How to explain to programmers (or other stakeholders) that this is bad? I will use seven consistency oracles for test reporting.
- consistent within product – is this iMessage behaviour consistent with behaviour of comparable functions or functional patterns within the iPhone? I choose for comparable function Messages text message service. I send to my home fixed line, with not enabled SMS service, text message. Messages application notified this as Text message, sending time, and green text message. There was no information that action failed. So iMessage behaviour is not consistent with this behaviour.
- consistent with comparable products – is this iMessage behaviour consistent with behaviour of comparable functions or functional patterns within the afreesms web application? I entered my GSM phone number without last number and application reported in pop up window “Ops, something went wrong.” Here we have consistent iMessage behaviour.
- consistent with history – present behaviour consistent with past behaviour. I noticed described iMessage behaviour in period of two weeks (iPhone servicing time period). I have not upgraded iOS in that period, so I can not apply this consistency.
- consistent with our image – behaviour consistent with an image the organisation wants to project. Apple wants to project image to public as technology leader, inovation leader, product design leader, business solution leader. Not immediately knowing why sending of iMessage message failed, threatens Apple image as business solution leader. Behaviour is not consistent with this oracle.
- consistent with claims – behaviour consistent with documentation, specifications, or ads. Support document for iMessage claims following for the case when iMessage is not available: On iPhone, if iMessage is unavailable, the message may be sent as SMS or MMS, depending on the content. Carrier messaging rates may apply. You can adjust this behavior on your iPhone at Settings > Messages > Send As SMS. My iPhone messages settings has iMessages on, MMS on, send as SMS off (send SMS message when iMessage is unavailable. Carrier messaging rates may apply). So we have consistent behaviour of iMessage with this oracle consistency.
- consistent with standards or regulations -behavior consistent with externally-imposed requirements. Since iMessage is messaging application developed in house by Apple using its own Apple push notification service protocol, this protocol is considered as “externally-imposed” protocol. Behaviour is consistent with this claim, because failed message send is reported in iMessage application.
- consistent with user’s expectations -behaviour consistent with what we think users want. Regular user of iMessage application would know that click on exclamation mark would give him additional information about message failure, along with information how to resolve the issue. It would also know that sending SMS message is not primary purpose of iMessage application. We can discuss that Message setting send as SMS is on by default, but Apple decided not to go with defaulting that option to on because this involves additional cost and Apple first claim for iMessage application is: iMessage is an Apple service that lets you send unlimited messages over an active Wi-Fi (or cellular data*) connection to other iOS devices using iOS 5 or later or Macs using OS X Mountain Lion or later. So we have another consistency. And this consistency supports that iMessage is consistent with image consistency. Because standard Apple user business would not be threatened when sending iMessage fails.
- consistent with purpose – behaviour consistent with product or function’s apparent purpose. Purpose of product is to send messages from Message application for free and to notify user when sending of message fails. Another iMessage behaviour consistency, this time with purpose oracle.
