Reading Time: 2 minutes
Zagreb testers community held Meetup #3. For now we are at our schedule to have those gatherings every two months and we are very happy about that fact.
First, we distributed free copies of Testing Planet that we received from Software Testing Club. We are very grateful for that gesture!
First testing problem (was it accidental or intentionally?) was the room projector problem. Projector was at the ceiling of our room and we did not have its remote. On projected image we could not see Windows XP taskbar. We agreed that we have to discover the proper laptop resolution for the projector. First we blindly tried several laptop resolutions with no success. Than one tester suggested that we should use resolution with ratio similar to ratio of projector canvas. At that was solution of our problem!
|Where is the taskbar?
Topic of meeting was performance testing. Our fellow testers from Asseco gave presentation about testing in their organisation, and they finished it with live presentation of performance test of their product using Jmeter.
They tailored testing process from Microsoft Solution Framework. Before that, there was no dedicated testing team in their organisation. Developers were responsible for testing. They are continously working on testing process, but their experience for now is that they now have much less problems with their products in live production.
|Congratulations to projector problem resolver! (first from the left)
Mission of performance test was to discover reference system configuration for their application. Basic input is that response for the application most critical service must be under 300ms. They changed their application from java process and in house code to J2EE application. Tomcat is application server, Oracle is database and Linux is target os. Client interface are web services. Other testers commented that this architecture is very similar to architecture of products in their organisations.
First observation of performance test was that product on new J2EE architecture had slower response times than the stand alone java process application. Their experience with Tomcat is that commercial solutions like IBM Websphere are much better.
They created various scenarios in Jmeter. Important note is that they cooperated with developers in order to create Java client code for Jmeter scenarios! Testers in their organisation do not program.
During the live demo, we discussed (along with pizza that was kindly sponsored by Asseco fellow testers) several testers topics:
- communication/cooperation with developers in our organisations
- using Junit framework for system testing
- bug reporting (Bugzilla, Mantis)
- test reporting
- testing estimation
- sql injection
- mobile testing (automated vs. manual)
In the end I conclude that Zagreb STC MeetUp #3 was very productive and interesting gathering. If you would like to join for the next meeting, please subscribe to our Google group Zagreb STC, drop an introduction post about yourself, or propose something of your interest for the next meeting.
Reading Time: 1 minute
In video Dale Emery and Elisabeth Hendrickson demonstrated how to identify variables while exploring and use them to discover interesting things to test. As a demonstration they used Triangle Tester Exercise, application which determines based on user input for length of three triangle sides, is it possible to make valid triangle from those side values. Dale and Elisabeth listed possible testing variables for that application. I will list none testing variables.
None testing variables:
- label for side input field
- font type, size and color for information about the result of triangle side calculation algorithm.
- font type, size, color and background color for test data history header column.
- font type, size and color for input field.
- font type, size, color and background color for test data history column.
- color, line and fill pattern for drawing for valid triangle.
To identify none testing variables is also important information because we can detect is this static data candidate to become variable if we look from the application user or tester perspective. For user application could become more usable, and tester could better test the application.
For triangle application example, input field fill color as a variable. Red fill for invalid input, bright red color for invalid input for color blind users.
Reading Time: 3 minutes
I attended STP2010 conference at Las Vegas. I was honored to meet Gerald M. Weinberg. I bought his book “Perfect Software: And Other Illusions About Software Testing” and got his autograph at the first page. I finally managed to read the book. And the next day at my work I started applying his advice’s in my project.
My current project is to check the implementation of Electronic Health Care Record (EHCR) that is based on CEN ENV 13606:2000 and HL7v3 standards. My first focus is to check the functionality of the EHCR module that implements that functionality.
CEN standard gives CRIUD (Create, Retrieve, Insert, Update, Delete) functionality to patient health record. Retrieve gives all the history of patient record, functionality very similar to version tree feature of modern version control systems, or only portion of the patient history, based on the query time constraint.
Complexity of CEN is in the xml implementation of the CRIUD functionality. Basic unit of CEN is archetype and datatype, and we support around two hundred of them (e.g. one is patient demographic data). Every delete and update creates version tree of particular archetype. CRIUD actions on archetype could be done by several roles (e.g. physician) defined by permission rules. In the end, basic combinatoric mathematics gives us infinite number of test cases (checks).
With budget constrains (time and money) we had to decide what to test (actually check). Talking with developers I wanted to get the insight how the EHCR module was designed. First, I found out that they wrote very good subsystem documentation (features, architecture and data model). Project issue with documentation is that there is no unique central document management system, so no one from test team didn’t know about that documentation. ehcr module uses a number of CEN xml schemas that do the first level of CEN xml message validation. Schema validation was coded using third party library (xerces). Based on that information we created several test cases to check that functionality and confirmed that errors in xml messages regarding defined schema were properly detected. The possibility in wrong error detection was only if schema is not according to requirements. But requirement analyst checked all xml schemas (about 200 of them) using advanced xml editor.
We put our focus on checking the requirements implementation that was coded by developers (various xpath expressions and if..then..else logic). Checking coded values (codes from database) was first check candidate. We found errors in those checks because developers did not understood the requirements (or requirements changed) and they did not coded them in business logic (missing xpath expressions and if..then..else logic).
The hardest part were hidden requirements of CEN protocol. You just have to understand CEN protocol which is not so simple.
I will give one of the test cases. Demographic data contains three data items: mandatory date of birth and patient sex and optional date of death. Also, demographic data has its position in CEN (cen coordinates). Patient physician first creates demographic data with patient birthday and sex. After that inserts patient death date. He realizes that he made a mistake and deletes the death date. Patient changed its sex so physician updates patient sex and birthday. Now we have several versions of demographic data data items. Physician wants to update again birthday, but sends identification id (cuid) od sex data item instead of birthday. System reports an error code.
Another test case is when Physician updates whole demographic data, so whole demographic date info becomes obsolete.
So on system level we have an infinite number of such test cases.We discussed with developers those test cases in order to implement them in subsystem test because there are no any unit tests. In that way developers narrowed down the infinite number of test cases because they know subsystem code and how to implement those checks on lower level.
We created automated jython scripts as system test, to test various and complex CRIUD functionality, for several randomly picked archetypes. There was a significant number of reported issues. After developers implemented subsystem checks and done code review base on our input, all issues were resolved. But important information was that there was no such issues when we checked other archetypes!
Testers had a problem of infinite number of checks for CRIUD CEN functionality. First, testers narrowed down the number of checks by checking only test cases that will be triggered in real usage of CEN protocol. What is real usage of CEN protocol, testers concluded based on real data gathered from the production subsystems that already gathered similar data. They explained those test cases to developers, so developers could implement checks in subsystem code.
Reading Time: 2 minutes
On Software Testers Speak Up Meeting #1 I met a colleague tester that is responsible in his organization for writing user technical documentation. At that time, I was strongly against idea that testers should do that activity, because testers are responsible for testing the system. So my advice to the fellow tester was to convince his project manager that testers should stop writing project documentation.
But his response was that testers in his organization are the most qualified for writing user documentation, and user documentation is one of the most important artifacts of the agreement with the customer.
They had a situation when customer database administrator dropped production database, instead of applying a database patch (stupid mistake according to every database administrator that knows its craft). User documentation that explicitly stated how database patching should be done, played significant role when customer wanted to blame testers organization.
Because of that information I had been thinking for several weeks after testers speak up meeting. For testers organization, customer documentation is integral part of their product, not just the system itself!
So I decided to ask for help. Twitter was my tool of choice.
@michaelbolton I met a tester. He doesn’t do testing because he has to write customer documentation all the time. How should I advise him?
I know about Michael Bolton and his testing magic for a long time. I met him at the STP2010 in Las Vegas where I attended his Rapid Software Testing course. Great thing about him is that he is doing free consultancy. Here is his answer.
@karlosmid When writing customer documentation, I did plenty of #testing. Often, I found way more bugs than “testers” did.
@karlosmid So, for your friend: 1) NOTICE that you’re testing. 2) Report problems, give repro procedure, explain /why/ they’re problems.
And immediately all become clear. I had an emotion about testers writing user documentation. That emotion got me thinking about fellow tester problem. I asked for help, and advice was immediately clear: “As a tester, your are glue in your organization”. That means ask a lot questions about every line that you put in user documentation. When you write user documentation, interact with others, ask questions about documentation stuff that bothers you. Don’t be just a dummy that writes it down.
Today, there was also one important Michael tweet that connects with his advices about writing user documentation:
Read his blog post to further enhance your user documentation writing and questioning skills.
Reading Time: 2 minutes
Yesterday, we held Software Testers Speak Up Meeting #1. Zeljko Filipin wrote about that event on his blog.
I would like to put some of my notes.
There were 16 people from various companies (200+ employees), and one tester from very small company (Zeljko).
For me It was a great event. How do I know that it was a great event? For that I use my own heuristic:
If discussions on event last for three hours without a break, and people starts topics in a meaningful flow, than that is a great event.
I put some topics on the whiteboard just for start.
We started discussion with question: “Why did you become a tester?”
First issue emerged, testers from big companies have one common problem: they all have to write documentation. I asked two questions: who reads that documentation, and do they mean by documentation word documents?.
Should (and which) documentation must be produced, depends on the company process (not only necessary software development process). But remember, when tester writes Word document, he does not do his job, and that is testing.
Question about requirements specification emerged next. We all agreed that no one have ever got satisfactory requirements specification.
Testing process was next topic, and CMMI come along with that discussion. I briefly explained testing process in my company and how we use mantis as a supporting tool for that process.
How testers cope with the deadlines provoked counter questions: “What is deadline?” and “What should be done by that deadline?” On statement QA is testing I explained that testers are not QA police. Whole organization should take care about quality. Testers only provide information about product quality.
Testers from big companies all agreed that their software development process does not contain unit testing, TDD, code reviews.
Pizza time had arrived, and I discussed about JMeter. I introduced grinder, by explaining how I am currently using it in order to test the capacity of our new mq broker configuration with Oracle database.
In the end, whiteboard looked like this.
Our plan is to do #2 SpeekUp meeting in two months, this time with some presentations.