How to properly check result of orderby method using random data?

In current project we are testing api endpoints that are developed using OData protocol. Microsoft implemented framework that implements Odata protocol and it helps experienced developers to expose system data very quickly and efficiently.

We are developing regression test suite for Odata REST api endpoints using following technology stack:

• Ruby
• httpclient gem
• rspec gem

One of important features of regression test suite is robustness on application data changes. In this post I will present evolution of check for orderby Odata filter. We first improperly designed that check making it not robust on application data changes.

Here is first, not data robust version:

it “default orderby” do
        response = @api_client.get @customers_list_url, query={“$orderby” => “Name”, “$top” => 2}, ‘Cookie’ => @cookie
        json_response = parse_json response.content
        expect(json_response[‘value’][0][‘Name’]).to eq ‘Game of Thrones’ 
        expect(json_response[‘value’][1][‘Name’]).to eq ‘Lord of the Rings’
end

and second version, robust on data changes:

it “default orderby” do
        response = @api_client.get @customers_list_url, query={“$orderby” => “Name”, “$top” => 2}, ‘Cookie’ => @cookie
        json_response = parse_json response.content
        expect(json_response[‘value’][0][‘Name’]).to be <= json_response[‘value’][1][‘Name’]
end

In your automated check, always check for what feature actually do, do not check for hardcoded test data values that are result values of the feature.