 |
| Credit: www.yachtingmonthly.com |
TLDR
In this post I will describe how I discovered important product information using load test. And nobody involved in this product expected that we will find out this information.
For one of my client I performed and load test with goal to discover how much concurrent users could get satisfactory response times on production environment.
I coded load test script with agreed scenario using Grinder Load Testing Framework. I created test result reports using Grinder Analyzer. Both tools are opened sourced. Every measured request was POST http request with simple http authentication. Data format was json.
In order to prepare the load test, I had a meeting with all important stakeholders: project lead, test lead and developer lead. This is important, especially when you plan to do load test on production environment. I presented them my test strategy and requested feedback.
We agreed on user scenario where most important requirement was to have user/requests per second ratio equal to 1. Second important thing was to agree on load test time execution window time frame. With those information, I was ready to execute load test on production environment.
I checked test script and test report by first running load test on test environment.
I run the test from my office in early hours and when I came to client site I got immediate questions from developer lead.
What do you know about transactions with date from the past? I got a call from person xy!
These are transactions generated by load test, I put that date in test data.
So first revealed information was that transaction date was taken from incoming data, not generated by application in real time. Which is bad because you must never trust the user data.
But who is person xy? Was asked by developer lead, not me. Why he has access to our production environment? I thought about Monty Python’s sketch: Nobody expects the Spanish inquisition.
We revealed second important information, who has access to production site and has business value in production data.
Because of person xy request, developer lead decided to delete all load test generated data. Which is not good pattern because second run of load test would have database with more data than test before it. And that also could reveal some interesting information (missing database indexes) about production site in next run of load test.
