Risks Related To Software Upgrade

TL;DR

In the last post, we presented a Recipe For Creating A Bug Taxonomy. The idea was from this excellent paper: Bug Taxonomies: Use Them to Generate Better Tests [Vijayaraghavan, Kaner].  From this same paper, today, we present an example of risks related to software upgrade error. Many thanks to Marcel, who sublimed this great resource on his blog, That’s the buffet table.

Software Upgrade

Regression testing is the testing method to check whether our software regress from the working state to a less developed state. For that, we develop several test suites on various levels:

• unit
• module
• system

We even try (unsuccessfully) to automate several ends to end scenario tests.

But there is the smarter way to do it by identifying risks related to software upgrades.

Server Side Failures

• A common error is the failure to back up before the upgrade.
• Accidentally over-writing the product database file during the upgrade
• Non-removal of staging files before upgrade may lead to corruption of the application
• Failure to update or reset correct file permissions after the upgrade process, this causes some pages to show “Unauthorized to view” errors when the user clicks on a page link
• Many software upgrade processes look for folders with standard names. For example, Ruby On Rails expects all configuration files in config the directory path. Any deviations from the standards pose the risk of an incomplete install/upgrade
• Some upgrades corrupt the web application by changing the default file types to newer file
  types. The newer file type may not be compatible with clients that use it.
• Files upgraded successfully but did not to make changes go ”live” after the upgrade!
• Failure to check the OS compliance of the host server before the upgrade
• Failure to verify the host server’s software and hardware requirements before the upgrade
• Insufficient disk space available for the upgrade process and the upgrade
  stalls before completion
• Failure to update older and outdated content before an upgrade or site re-design
• Risk of mistakenly listing outdated and discontinued data by over-writing new files
  with older ones.
• “We ran two programs at the same time that will not run together,”
• Upgrades performed without checking inter-compatibility between existing or newer software processes within the system.
• Post upgrade “internal glitches” have caused transactions to be processed. They generally occur due to new but mismatched data feed installs, convoluted linking due to the addition of new links within the shopping cycle, older links not removed, and new links installed without targets.
• Upgrades to some parts of the system may cause selective failures independent or related sections of the system. A common issue has been upgrades to client information databases, causing user authentication failures due to lockouts and denial of login processes access.
• A fix to one bug causes another! A common problem in conventional software too.
• A “newer look” or “fresh look” after an upgrade may not always mean an error-free look for the site, “newer look” changes the GUI and functionality, and this leads to newer problems both in terms of functionality, usability, and technical glitches leading to blackouts.
• Another important risk is the risk of security problems caused by poor installation and incomplete installation that results in some security features being turned off.
• Software upgrades sometimes set all options to ‘default’ automatically after the installation is complete. This, in turn, may overwrite any existing customized options. This leads to a change in system behavior and settings.

Client-Side Response to Server Side Software Upgrade

• Browser incompatible with the new upgraded server-side shopping cart

Conclusion

When you have a list of upgrade risk failures, you should try to automate the failure tasks. Moder software upgrade tools are great to help. Examples are:

• AWS
• Heroku
• Gigalixir
• Docker
• Kubernetes