Randomization Testing of Filter Handler in Postman

Sometimes, as a test engineer, you need to make multiple requests of a single API method with different parameters

A special case of such API method could be a filter or a search handler, which makes requests with pre-selected parameters. In the interface it might looks like this:

Each parameter has predefined values. They could be hardcoded on the frontend or got from other handlers.

Thus, for making a desired request we firstly need somehow to collect the parameters.

After that we can proceed to testing and apply different techniques of test design:

Let’s focus on the last one, because the aim of the article is to show how to operate with handlers, variables and collections in Postman rather than finding defects. COVID-19 Rich Data Services free API was taken as an example.

Our step-by-step algorithm is as follows:

  1. Decide which parameters we need to collect for filtering request;
  2. Understand how to collect that parameters;
  3. Create precondition requests with scripts;
  4. Run preconditions;
  5. Run filter handler with randomized parameters, received from previous steps.

1. Decide which parameters we need to collect for filtering request

According to the documentation the filter handler can accept a lot of parameters, but we will limit ourselves to a few:

http://{host}/rds/api/query/{catalogId}/{productId}/select?collimit={number}&count={boolean}&format={id}&inject={boolean}&limit={number}&metadata={boolean}&where={query}

2. Understand how to collect that parameters

3. Create precondition requests with scripts

For making a real filter request catalogId and productId must match, but since our goal is to make a synthetic request with any combinations, it is OK to collect these parameters without matching. To overcome inconsistency we will run collection of preconditions for a few times.

After parsing requested data from response, we need to set them to an environment variables:

let jsonData = pm.response.json();// Collect and store data only if it existsif (jsonData.catalogs.length > 0) {    let allCatalogs = [];    let allDataProducts = [];    jsonData.catalogs.forEach(        element => allCatalogs.push(element.id)    );    // Method of parsing data depends on requested data schema    for (var i = 0; i < allCatalogs.length; i++) {        jsonData.catalogs[i].dataProducts.forEach(            element => allDataProducts.push(element.id)        );    }pm.environment.set("allCatalogs", allCatalogs);pm.environment.set("allDataProducts", allDataProducts);}

For the format parameter we will set all available values into an environment variable. Unfortunately, if we manually set data into Postman variables as array type, it will set as string (data type saves into variable’s value only by pm.environment.set() script). We should take this behavior into account in the next steps.

4. Run preconditions

We should run preconditions separately from our main request to reduce the load and avoid extra queries not affecting the final result.

Before start we have no data in out environment:

In the Collection Runner we check only precondition handles:

The result of the run will fill environment variables with data:

5. Run filter handler with randomized parameters

A part of randomized parameters will get from environment variables, the other part we will calculate in precondition script. For boolean and integer parameters we are lucky to use Dynamic variables instead of writing our own code:

To randomize parameters we can just take a random item from array which get from environment variable:

let preconditionItemFromArray = preconditionArray => {    let tempArray = pm.environment.get(`${preconditionArray}`);    // https://stackoverflow.com/a/5915122    return tempArray[Math.floor(Math.random() * tempArray.length)];};

Now, when you run requests in the Collection Runner, combination values of Iterations and Delay could simulate primitive load testing (especially considering random requests):

I highly recommend to set «Request timeout in ms» in Postman settings to prevent runner’s freezing if some requests will execute unexpectedly long.

This collection can be downloaded from GitHub.

Quality assurance engineer