TL;DR
In last post I described how I was hit with my Bank business merging process. At that time I thought that process won, but human come to a rescue. And what is business process that does not use help from expensive automation tool?
As I spent all my cash, I went to my bank (luckily it is in my neighborhood), to get some more cash. At the front desk I recognized familiar face, a bank employee that always had a solution for my bank problems. I did money transaction and tried my luck. “Excuse me, my wrong card is blocked, could you maybe unblock it (last time different bank employee put my request in dev/null)”. Ok, no problem, your card is unblocked, and the old one is blocked. Hurra 🙂
Lesson learned. When you are testing, always ask for second opinion (or ask for second help).
I also have my business account at the same bank. Yesterday I got notification from Pagely and AWS that payment for their services was unsuccessful. I called my card help desk, and operator explained me that Pagely is on bank black list. Armed with second opinion heuristic, I waited for two hours and called same number. Different operator answered, and she put a lot more effort to investigate what happened.
As part of bank merge operation, starting from THIS MONTH, my business card was put under fancy (and probably expensive) tool scan that AUTOMATICALLY blocks fishy transaction. For now they blocked Pagely (my heuristic is that they really do not know who is Pagely) and Amazon. I asked operator for clarification. She said that AWS was blocked :).
Here is my heuristic. New tool is not the problem. Problem is that tool is not fine tuned by HUMANS. It runs with white list. Very likely Amazon is white listed but AWS is blocked. They test in production and will do the fine tuning using data from the customers that complain (me).
Do you still think that testing in production for bank business is an good test strategy?