Category Archives: security testing

Do not keep your credit card and PIN together in your wallet heuristic

Reading Time: 1 minute


This post is about testing heuristic : “Do not keep your credit card and PIN together in your wallet”.

Heuristic is commonsense rule (or set of rules) intended to increase the probability of solving some problem [WolframAlpha]. Heuristic is fallible.

Captcha is [WolframAlpha]

a type of computer-administered test, usually in the form of distorted text or images, aimed at determining whether the respondant is a human or computer; used as a security measure on many websites to block automatically generated spam, since computers should be unable to respond correctly

Here is how we can apply this heuristic on capcha problem. Developer finished his captcha code. You hit Chrome developer tool Inspect feature on Captcha element, and you see this:

There is captcha question “odaberi kokice”, and three answers, radio values 1, 2 and 3 with appropriate image.

Can you apply credit card and PIN heuristic here?

Credit card is question, and PIN is radio image name. Image name contains PIN value. So it is possible, using simple algorithm, to automated answer to this captcha. In this example:

if captch_question is odaberi kokice the select img/captch-kokice.png.

In proper captcha, img src must not contain easy decodable captcha answer.

<img src="0efc2e4ab9e04bc9dc66833dbb98505438c26f5557713e53c69090b586e62c4ceff814a5def8174f5c6d417aec5c2d2d1829fafaa9d12b461b3b0fff0ab894a4.png">

Also, having more than three captcha answers helps in your fight against crawlers, spiders and robots.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to smart test minor version Ruby on Rails upgrade?

Reading Time: 1 minute


In this post I will explain what should be checked after Ruby on Rails minor version upgrade. Minor version upgrades are usually connected with security releases.

As I am subscribed to Ruby on Rails security Google group, when I receive information about latest security issue, I need to act very fast. Update must be pushed to production almost immediately, so there is no time for extensive regression testing.

How can we do quick test and be sure that everything still works as before upgrade?

Heuristic 1. Rails upgrade was extensively regression tested.

That heuristic proved itself to be always true.

Heuristic 2. Inspect Gemfile.lock to be sure that only Rails gem is upgraded.

How do we actually upgrade Ruby on Rails?

Edit gemfile:gem 'rails', ''
bundle update rails

Investigate Gemfile.lock changes using git diff to see what else was updated beside Rails. If some other gem (not part of Rails) was also updated, check, using Google search, possible issues for that gem that are connected with Rails upgrade.

Heuristic 3. Search Google for Ruby on rails upgrade to n.n.n.n version issues (bugs, problems)


For minor Ruby on Rails upgrades, using those three heuristics, you can do regression test in smart and quick way.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to set regression test scope for HTTPS only access?

Reading Time: 2 minutes


In this post I will provide an example how to set the scope of system regression test in order to achieve coverage of features that need to be tested.


The trigger for regression system test was HTTPS only access to client Ruby on Rails application. Prior to that feature, it was possible to use both HTTP and HTTPS protocols. Developers use TDD concept, and prior to my test, all developer tests passed on CircleCI environment.

There was also set of selenium-webdriver tests, but those tests do not cover all application features.

My strategy was to include them in the regression test. All test passed. But I had not finished regression test yet.

I did not know all application features at that time. So I started risk analysis, which features could fail if HTTPS only protocol was introduced to web application. Let’s call OWASP for help.

There is transport layer cheat sheet. Reading through the rules, I pinpointed rules that were potential risk for application functionality:

  • Do Not Mix TLS and Non-TLS Content – because browser (modern browsers) will AUTOMATICALLY prohibit access to non HTTPS urls.
  • Use a Certificate That Supports Required Domain Names – if this is not the case for your application, browser will present to user a security error

First risk could be mitigated by using the application in Chrome and observing javascript console for mixed content errors. Could I automate that task? First thought would be: write selenium-webdriver test suite that covers all the features! But I do not have that time. Was there a simpler way?

My heuristic was to search all the code base for keywords HTTP and IFRAME.

grep -H -r ‘iframe’ * | grep http:// | grep -v elements.txt | grep -v ‘’ | less

That piped command searched through the all code base in terminal and returned code that uses mixing content.

And we discovered additional issue, it was not possible to immediately set all HTTP urls to HTTPS protocol. Those urls were referencing external applications, like blog. For example, in order to set this blog to HTTPS protocol, I need to buy another plan that costs more money. And I need to have a certificate for domain. Which brings us to second risk.

Use a certificate that supports required domain names. And this is environment dependant test. This risk was mitigated on my testing environment, but I should also check it on production environment (production is hosted on different domain).

Doing risk analysis is fun. You will learn something new and the most importantly, you will properly set scope for your regression test.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Selenium RemoteWebDriver over http

Reading Time: 1 minute


In this post I will explain how I connected knowledge from two testing domains, security and UI automation, in order to achieve proper UI automation stack configuration.

 In order to be able to use various browser/os combinations for ui automation, I am using SauceLabs, cloud based testing provider for web and mobile. SauceLabs is running Selenium Server instances to which you are connecting using RemoteWebDriver.

Here is how to do that in Ruby:

What bothered me is that UI test would open http connection and testing infrastructure would be vulnerable to man in the middle attack. By investigating the Selenium documentation, it is not possible to run Selenium Server with https.

What is next? SauceLabs has its own product, Sauce Connect.

“Sauce Connect is a secure tunneling app which allows you to execute tests securely when testing behind firewalls via a secure connection between Sauce Labs’ client cloud and your environment.”

That means you are still using http towards Selenium sever, but over secure tunnel.

“Data transmitted by Sauce Connect is encrypted through industry-standard TLS, using the AES-256 cipher.”

What is interesting is SauceLabs official statement:

“You should use Sauce Connect whenever you’re testing an app behind a firewall. Sauce Connect is not required to execute scripts on Sauce.”

This should be paraphrased:

You should use Sauce Connect when your testing scripts are using sensitive corporate data.

Never put aside your testing domain knowledge, you should always combine them, especially when clients security is at stake.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

When you should restart any server instance?

Reading Time: 2 minutes
image credit:


In order to know when you should restart your server process (web server, database server, or any other type of server), you need to understand how program works. In this post I will explain basics of program and used libraries and how to detect on linux when you must restart your server.

I am very frustrated with Windows operating system. You need to restart it almost for everything, important security issues, regular updates. Pain is even greater when there is some security product which just pop ups notification: your system will reboot immediately!

And in the beginning, there were stories that unix, and later linux is much better, because you do not need to restart it. As I was learning and using unix/linux, I found that this was true. There were notifications that new updates are installed, but there was no need for machine restart. As I learned about linux, I found out that this was not true.

If you are running linux instances as part of publicly available web application, you must walk extra mile. You will probably set silent automatic updates for you linux instance. But, you will have to do extra check to see do you need to restart your server processes.

Every server process is a program. It consists of executable part and a number of libraries. When you start your server process, executable and all required libraries (on linux those are files with .so extension) are loaded into memory assigned to that process. So if there was an update of libraries used by your server process, and those libraries are loaded into process memory space, you will need to restart your server instance in order to load new version of those libraries.

For linux, there is utility that can help. lsof lists all opened files. It has useful option, DEL, which list all opened files that are marked for deletion.

This example shows that postgres still uses deleted version of libcrypto library.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Heroku ACTION REQUIRED: Potential security vulnerability in Ruby and YAML parsing

Reading Time: 1 minute

I received email from with that subject on April 3rd, 2014 at 22.55 local time. I was expecting that email. How many of you who have Rails application deployed on Heroku, patched your application by following instructions in security heroku email?
I followed instructions by typing them in my Mac terminal, and found out that instructions are wrong.
First command from email:

heroku run “ruby -rpsych -e ”p Psych.libyaml_version.join(‘.’)”” -a application_name   

should be replace with:

heroku run `ruby -rpsych -e ‘p Psych.libyaml_version.join(“.”)’ -a application_name

I do not explicitly use Psych gem so I found out that cmd:

git commit –allow-empty -m “upgrade ruby version”

should be replaced with:

git commit –allow-empty -m ‘upgrade ruby version’

I had to upgrade my Ruby from ruby-2.0.0-p247 to ruby-2.0.0-p451. For ruby-2.0.0-p247, libyaml version was 1.4.0 and by upgrading Ruby, I only managed to upgrade libyaml to version 1.5.0. I do not want to upgrade Ruby to ruby-2.1.1 because of following issue.

If you do not parse user yaml input in your application, then this issue does not affect you.
This blog post shows what security risks you should consider when you deploy your application at 3rd party cloud application platform. You can learn more about risk based testing in lecture 2 of BBST test design course.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

‘Hacking’ Rails session

Reading Time: 1 minute

One of important security attack vector in web applications is cookie session content. Rails security guide gives all relevant information which data could be stored in cookie session.
I will explain how you could obtain cookie session object content (I learned about that from this blog post). First, get the application cookie. Hit F12 in Chrome, switch to network tab, log in to application that you are testing and select POST authentication request. In response object find Set-Cookie header, and copy cookie value. Cookie ends with ‘;’ character.
Start irb, Ruby interpreter. First decode cookie string value, and then de-marshal it. Here is Ruby code:  

1: require ‘base64’
2: plain = Base64.decode64(cookie)
3: data = Marshal.load(plain)

Tip: if you get exception in third line, you need to include with require statement package that contains reported class.

data is Ruby hash object that represents Rails session.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather