Search in project for method usage using only bash script

TL;DR

This post is about simple bash shell scripts that finds all files that use particular method.

In my previous post: Product moving parts as source for test strategy, I described how I use github pull request in order to discover which part of application changed in order to create regression test strategy.

Code that changed could be some helper method or .css and .jpg assets that are used in various places in code base. And those places are not part of pull request. So I need an automated way to find all places where is that helper method used.

For that purpose I use simple bash script. You need to know loop programming concept and a unix utility commands cat and grep.

Here is bash script:

#!/bin/bash
for i in `cat pull_request_items.txt`
do
  echo $i
  grep -H -r "$i" * | grep -v cache | grep -v manifest
done

And content of pull_request_items.txt

our_overview_gettingstarted.png
tour_overview_lessonhighlights.png
tour_overview_originals.png

Script reads items from txt file, and each row value is searched in project codebase using grep utility. Search is recursive in all subfolders.

Output contains files that contain searched items.

Manual part was to copy/paste from pull request to pull_request_items.txt file and do some editing in order to clean not important pull request information.

Why not use some fancy editor like sublime? Because presented utilities are installed on almost every linux machine in the world and i can use this script out of the box.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Outlook broke important business feature

TL;DR

This post is about Outlook broken business feature, carbon copy recipients.

In e-mail, the abbreviation CC indicates those who are to receive a copy of a message addressed primarily to another (CC is the abbreviation of carbon copy). The list of recipients in copy is visible to all other recipients of the message. In common usage, the To field recipients are the primary audience of the message, CC field recipients are others to whom the author wishes to send the message publicly, and BCC field recipients are the others to whom the message is sent.[source]

Carbon copy feature in email is de facto a standard business rule. If you are implementing email client, you should implement cc feature in that way.

And here is Microsoft 2016 cc feature:

issues_from_te_void_2 issues_from_te_void

And here is received email:

issues_from_te_void_2

You can not distinguish to: recipients from cc: recipients.

Business impact.

I received such an email, and replied to project manager to explain who is responsible for subject of his email. He replied (not very pleasantly, but that is the different issue) with highlighted to:  in message conversation automatically attached in message reply, and that I should know the cc: rule. So in message reply, you can distinguish between to: and cc: fields. But not in original mail.

Can I move to other email client? Answer is no, because Outlook is mandatory tool in clients organization and I am obligated to use it.

Update after Facebook feedback (Thanks Vanja and Igor!)

This issue is not consistent with Outlook history, because in previous Outlook versions that worked. Also, Gmail is not adequate alternative email client for this feature, because gmail also does not show cc field.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Value of UI automation tests

TL;DR

After series of posts about How to kill UI regression testing, maybe you got impression that I am against UI browser automation test scripts. In this post, I will put my thoughts about the value of UI automation tests.

First some context. Here are requirements for excellent UI automation tests:

  • Language that enables you to write less code. Something like Ruby.
  • selenium web driver implementation in that language
  • framework just above selenium webdriver. For example watir-webdriver.
  • Page object pattern implementation
  • test framework, something like rspec. For writing awesome asserts.
  • gherkin implementation. Something like  cucumber. No, I have not changed my mind stated here. But we have different context here.
  • continuous integration tool. Something like jenkins
  • headless browser. Something like xvfb or sauce labs.

What is the value of UI automation test scripts? They represent executable documentation for your product. They help you to answer questions about complicated business scenarios. For example facebook  setting who can contact me.

Your testers will need to have a skill of writing DRY tests using those requirements. So when you have new tester on board, by reading those tests script, he can quickly learn application features.

Those tests would be run on jenkins and run should be triggered manually. Tests will provide you information is some automated feature changed. It is not wise to use those results for you regression testing strategy.

Because those checks, or asserts only check what is coded. Nothing else. And UI is for human users, not for test scripts. So human should always do smart regression testing , using its brain, eyes, ears and fingers.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

It is not always so obvious

TL;DR

In this post I will provide an example how we should be very careful during the root cause analysis. Because sometimes you need to dig deeper to find the root.

During the CAST 2016, I visited Dr. Sun Yat-Sen Classical Chinese Garden. It consists of part where entrance is free, and close part with admission and included park guidance. TripAdvisor said that you can spent there 2 hours. As this was rather small park, my thought was I am done in 30 minutes. But guided tour proved me wrong.

My first impression was with water that was muddy. I was rather disappointed. Despite that,  kai fish were happily swimming.

Before the tour I got free garden tour flyer. I skimmed it and found this about garden water:

The cloudy, jade green pond-water creates a tranquil atmosphere and reflects buildings, rocks and plants. A special clay pond liner intentionally creates the opaque colour. Jade is the favorite Chinese gemstone which symbolizes wealth and purity.

How about that, water was muddy on purpose so all the garden artifacts could be better reflected in it!

Every garden item has a story and symbolic meaning, rocks, buildings, window shapes. Two hours passed very fast, and I spend observing objects in water for some time.

Elements of a scholar’s  garden:

  • buildings represent human element
  • rocks symbolize rugged landscapes and sculptural elements
  • water reflects buildings, rocks and plants
  • plants represent nature, their symbolic meaning has seasonal value
  • words as naming of buildings and views

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to kill UI regression testing: continuous integration

TL;DR

Continuous integration (CI) is last element that successfully kills your UI regression testing. In this post I will explain how CI system should fit in software development process.

There is always lively debate what should be automated in software development process. Pipeline elements of continuous integration should definitely be automated.

What are elements of pipeline continuous integration? Every command of operating system that you run as software tester is that element. And software tester usually run several of those commands in a pipeline as their daily task. Those os commands should be put in a script, and that script should become running element of your continuous integration tool (e.g. Jenkins).

Script can be Dockerfile or bash shell script. It depends on the technology that you use. The goal is to have one click button in your CI tool that will do following:

  • get the latest code from git branch (e.g. integration)
  • run development framework tests. Whole suite must run in less than 10 minutes. Otherwise, these are not development framework tests
  • deploy the application
  • run database migration and fixture files (hopefully, your development framework offers those features out of the box. Otherwise, find appropriate tool.)

Whole process must be done in less that 10 minutes.

Now you are ready to do you regression test strategy that includes:

  • observing changed files from last deploy using git pull requests and commits
  • investigating development framework test results, including test coverage report (be sure to know what that coverage means)
  • decide which end to end tests should be executed.

That concludes How to kill UI regression testing series. You will probably say, but I can not implement that in my project. That is ok, but now you know where should your project streamline in order to have better quality.

You have measure points for your project that can you use when you are asked:

  • Why we missed that critical bug?
  • How can we improve quality of our project?

On the other side, you can use this information in your interview for testing position or you can get information about the quality of a project that you are starting with.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Be careful with django fixture files

TL;DR

In this post I will explain the issue that can happen during the importing django fixture files into testing database.

Django fixture files enable developer to prepare testing data. We usually put in those files data that is static and essential for data model. For example, list of countries.

The problem is that developer needs to put by hand id fields that must be unique and sometimes also present in other fixture file (related data).

Some context. In the early development, you have several developers working on the feature. They install database on their development machine. Now we have one testing database, and several development databases.

Developer imports other fixture files and creates fixture files related with his features. Now it needs to include his fixtures to source control. The easiest way is to use django command, dumpdata which dumps whole database model. His fixture files contain already imported data from other developers.

Development process continues, which means that data will be added to developer local databases.

It is time for another fixture dump, and import of that dump fails on testing databases.

The reason is that ids in fixture files from several developers are not in sync. Quick and dirty solution is to drop the database. Which is maybe applicable at the early stage of development and on test database. But when we have production data, dropping the database is not the solution.

Solution? All developers should import (and resolve any conflicts using plain sql) fixture files from version control  into their local development database before they do dumpdata with new fixture files.

In that way, version control becomes central point for all database fixture files.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to kill UI regression testing: git source control

TL;DR

This is third part in series how to kill UI regression testing. It is about git source control system and its web application part, github, bitbucket or gitlab. I have already wrote about this topic in Product moving parts as a source for test strategy, and Micro tracking changes for regression test. In this post, I will extend my thoughts from those two posts.

So git helps you to determine which part of application feature could be affected by the code change. Precondition is that you need to be able to read the code. Just read and understand functionality in that code. Many software testers started in software testing because their are afraid of code. I think that to be able to read the code could be great booster for software tester career.  And by reading pull request, you have very valuable mentor, the author of code himself.

Soon, you will find different code change patterns. For example, change in function signature in helper module could break more features than change in one module method. You can learn about that very quickly, the most important is that you should stop be afraid of reading the code.

And basically, you are doing code review. You will have questions and by asking them you could find issues before code is deployed and put in action.

Next post will be about Jenkins as an example of continuous integration tool.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to kill UI regression testing: development framework tests

TL;DR

This is second post about idea how to kill UI regression testing. In first post I described symptoms for UI regression testing that I would like to kill. I will describe development framework tests that are precondition for killing UI regression testing.

Development framework tests (DEFRATE) are automated tests that are written by developers. Here are reasons why I think that should be the case for every development project.

You have a developer that is a “rock star” in some development framework, lets take for example Ruby on Rails (you can replace that with development framework on your project). Lets see what Ruby on Rails framework offers in testing front.

Here is official documentation on testing Ruby on Rails application. Here is additional ruby testing toolbox. Every software developer that claims Ruby on Rails rockstar title, should be master in any of those ruby testing frameworks.

But which tests should developer create?

Developer test should create tests in DEFRATE that CHECK their code design.

Developer should check all flows that he created in its mind. Those code flows must PAST through the RUNTIME (lets say processor instructions). Tests should follow DEFRATE practice and use all features. For example, developer wrote some code that communicates with database. Essential part of test should be code that mocks database. Good DEFRATE enables easy mock code creation. And when somebody changes code that is mocked in a test, mocked code will fail and detect REGRESSION in code. Lowest and most quickest way to detect regression change.

Side effect of DEFRATE tests are tools that enables you to have code coverage by running those tests. Code coverage information is excellent source of information for you test strategy on UI level. And remember, code line coverage is just one testing coverage out of 101 testing coverage metrics. Here is excellent resource about testing coverage [Kaner: Software Negligence and Testing Coverage]

Google testing blog is also excellent resource for DEVELOPERS how to write good DEFRATE tests.

And the best way to learn testing framework is to read a book about it and do ALL the exercises listed in a book.

But my developers do not write DEFARTE tests! So I must stick with my UI regression testing. Ok, but now you know metrics that enables you to provide information about software quality in your project. And you have directions how to make quality in your project much better.

In next post, I will discuss source control tools that can help you to kill UI regression testing.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to check font size on a web page?

TL;DR

In this short blog post I will share with you a tool that could help you in checking font size on a web page.

I had to check application change that involved changing font sizes on application web page. In BBST foundations course I learned how to create test strategy. My test strategy was that I need to find a tool that will help me to do that check.

I searched Chrome extensions, and found WhatFont extension. My heuristic to quickly evaluate how good is that tool:

  1. simple and intuitive user interface
  2. more than 1000 ratings
  3. rating average > 3.5 starts

If those three are satisfied, chrome extension is a good tool. For WhatFont those three criteria are true.

After you enable extension, you just click on a font and you will get all needed information about it.

screen-shot-2016-10-13-at-10-34-44-am

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to kill UI regression testing: the simptoms

TL;DR

This is first post in what should be series of post. Idea is: How to kill UI regression testing. Idea started at CAST 2016 lighting talk and I extended it more at Testival 2016 open session slot where I get excellent feedback from the audience. The goal is to write material that will try to explain this idea in constructive, thoutfll way with experiments feedback that will suport or fail it.

First step, lets detect the symptoms, or reason why you are using UI regression testing.

You have a test plan that only contains a huge list of test cases. Every test case has useful name, and not so useful steps description. Context is a document with a hundred of pages. You are proud, because you wrote that document, and you are the only person that read it.

You have a automated test suite of selenium web driver tests, that have more that a hundred automated test cases, and in order to run it, you need several hours and several Jenkins instances. You have a number of failed tests, and those fails are false positives (test failed but this does not indicate problem in your product).

If this is the case, that means that developers DO NOT WRITE TESTS in their development framework (e.g. Angular, Ruby on Rails, Django,…). Note here that I do not mention unit tests, but DEVELOPMENT FRAMEWORK TESTS (DEFRATE).

There are many DEFRATE broken developers. They know only part of their framework, an usually they are not interested in testing part.

And because of that, you are developers safety net, and you are not doing software testing.

Because software testing is:

screen-shot-2016-10-08-at-5-45-50-pm

Every physician has first to determine the symptoms,  in order to determine disease. I defined the symptoms for  disease called UI regression testing.

Next week I will talk about DEVELOPMENT FRAMEWORK TESTS (DEFRATE).

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Blog that makes software testing interesting and exciting.