Many Ways to Speed up Autotests

As a test engineer who has to maintain a bunch of autotests, you will definitely face the task of speeding up the execution of your test suite.

Many Ways to Speed up Autotests
  1. Parallelisation;
  2. Less retries;
  3. No unconditional expectations;
  4. Fail fast;
  5. Cancel subsequent steps when the first one fails;
  6. No knowingly failed tests;
  7. Filter tests;
  8. Bundle tests;
  9. Keep Node and test framework up to date;
  10. Less dependencies and installations;
  11. Preconditions via API, not via UI;
  12. Authorize from cache;
  13. Hardcode precondition data.

1. Parallelisation

It is the most essential way. All other ways are just a neat tuning, that might not dramatically speed up the execution time as parallelisation does.

  • Some test runners are still do not support parallelisation;
  • Some mean by parallelism only tests inside test files — when all tests inside a file are run in parallel;
  • Others can run in parallel test files — test files are run in parallel, but tests in a single file are run in order — the most convenient way for UI automation.

2. Less retries

Retries are good to avoid flakiness, but retrying more than one time may not be a good idea.

3. No unconditional expectations

Get rid of pause and timeout functions — they just slow down tests for specified time.

4. Fail fast

If you see multiple simultaneous failures in different tests during a test run, that is a sign of problems with the test environment, there is no reason to execute more tests.

5. Cancel subsequent steps when the first one fails

This way is similar to «4. Fail fast», but refer to an individual test (test file).

6. No knowingly failed tests

Are you familiar with this situation, when after a test run with a few failed tests developers or QA say: «It’s fine, these tests always fail»?

7. Filter tests

Most modern test runners are allowed to filter tests by filenames or test titles/descriptions:

8. Bundle tests

This way is best applicable for CI/CD and combines items «1. Parallelisation» and «7. Filter tests».

Bundle tests
  • This method does not make sense for a small number of tests — it is a desperate way for mature projects;
  • You will have to create a system to join test reports from different bundles;
  • You will have to pay for computational resources: additional cloud instances or CI/CD agents — not everyone can afford it.

9. Keep Node and test framework up to date

Every release of Node.js has performance improvements ⇒ the code of your tests will execute a little bit faster.

10. Less dependencies and installations

This item is about the infrastructure of your tests.

11. Preconditions via API, not via UI

For browser/UI testing most of all preconditions like authorization and generation test data should be done through the API. Opening a browser is quite an «expensive» operation compared to HTTP requests. As much you can do through API instead of UI, the faster your test suite will be.

12. Authorize from cache

If your test requires authorization, it is not necessary to authorize each time in preconditions. During a test run you can authorize only the first time, put authorization credentials (cookies or tokens) into a cache file and take it from there by other tests.

Authorize from cached credentials

13. Hardcode precondition data

Attention, this is a controversial way! It depends on your project and requires full awareness of what you are doing.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andrey Enin

Quality assurance engineer: I’m testing web applications, APIs and doing automation testing.