Principles of Writing Automated Tests

While working on test automation in different projects, I’ve learned that there are not enough static analyzers and code formatters for writing good tests. The team had to have an agreement on how the tests should be written.

If any team member will write tests with his own ideal vision it will be a mess. Some patterns, applicable to unit tests on Java or Python, do not fit for integration tests on JavaScript and vice versa. Tests of a certain type should be consistent and should conform to acceptable rules for the project.

A presented compilation of principles is based on years of experience and have proven their effectiveness in real projects. They are mostly applicable for end-to-end tests (API and UI) on JavaScript/TypeScript and corresponding test frameworks: mocha, Jest, WebdriverIO, Playwright and the like. Some principles may overlap or even conflict with each other, some may be controversial — common sense will judge it, at least it all depends on the context of the testing project.

1. No tests without assertions

There should not be test steps without checks.

Do not make test steps like this:

test("Should open menu", async () => {
  await page.locator('.button').click();
});

Each test step has to have an assertion:

test("Should open menu", async () => {
  await page.locator('.button').click();
  const locator = await page.locator('.dropdown-menu');
  await expect(locator).toBeVisible();
});

2. No assertions in before or after hooks

beforeAll, beforeEach, afterAll, afterEach hooks should not have assertions. Preconditions/postconditions should contain only pure actions (for example: authorization). Checks should be done inside tests.

If you still need to check something in preconditions/postconditions, then use try…catch and/or throw errors.

3. No actions without expectations

After all actions in tests: clicks, hovers, gotos, etc. should be an assertion with expectation to check that the action was definitely committed.

test("Should open menu", async () => {
  await page.locator('.button').click();
  await page.locator('.dropdown-menu').waitFor({ state: 'visible' });
});

Or:

test("Should open menu", async () => {
  await page.locator('.button').click();
  const locator = await page.locator('.dropdown-menu');
  await expect(locator).toBeVisible();
});

The second example is valid because expect(locator).toBeVisible() contains conditional expectation, see № 4.

4. No unconditional expectation

Do not add pause and timeouts for N seconds between action and assertion to prevent flakiness — it only slows down the tests.

Instead of unconditional expectation:

it('Should open menu', async () => {
  const button = await $('.button');
  await button.click();
  await browser.pause(3000);
  const menu = await $('.dropdown-menu');
  await menu.isDisplayedInViewport();
});

Use wait for something (for some element’s state):

it('Should open menu', async () => {
  const button = await $('.button');
  await button.click();
  const menu = await $('.dropdown-menu');
  await menu.waitForExist({timeout: 3000});
  await menu.isDisplayedInViewport();
});

The second example works faster in case of passing the test and will fail for obvious and unambiguous reasons.

5. No commented test

If the test should be turned off, it should be skipped by test framework feature (skip), not by commented code.

Instead of:

// test("Should have a menu", async () => {
//  const locator = await page.locator('.dropdown-menu');
//  await expect(locator).toBeVisible();
// });

Do:

test.skip("Should have a menu", async () => {
  const locator = await page.locator('.dropdown-menu');
  await expect(locator).toBeVisible();
});

The number of skipped tests will be presented in the test report.

If the test is outdated and/or not needed it should be deleted without regret.

6. No hanging locators

Tests should not contain lines of code with «meaningless» locators:

test("Should do something", async () => {
  await page.locator('.button');
  …

The code in the tests has to do something: perform actions and/or assertions.

7. One expect for each test step

Test steps should be short and each step should check only one thing.

Do not put more than one or two assertions inside one test step.

Do not try to do everything and/or check everything in a single step.

The more «atomic» will be test steps, the mode intelligible will be test reports and test logs.

8. Do not put await inside expect

One operation inside another operation leads to a complication.

Instead of:

test("Should have title on the button", async () => {
  expect(await page.locator('.button')).toHaveText(/Menu/);
});

Do:

test("Should have title on the button", async () => {
  const button = await page.locator('.button');
  expect(button).toHaveText(/Menu/);
});

It is more verbose, but less chance to forget about await.

9. Do not reload the page, reopen it

Refreshing a page by a standard command (page.reload() for Playwright or browser.refresh() for WebdriverIO) is not a good idea — it makes the test flaky.

Instead of:

test("Should have something after reload", async () => {
  await page.reload();
  …
});

Get the current page URL and just open it:

test("Should have something after reload", async () => {
  const uri = await page.url();
  await page.goto(uri);
  …
});

This makes tests robust.

This pattern also applies for goBack() and goForward() methods, but unfortunately does not fit for SPA web applications in which the state of the page can differ from the URL.

10. Do not check URLs through includes

Do not use string.prototype.includes() for string comparison in assertions, because includes() returns true or false. When your check fails, you will get a report that false is not true — and no more details.

Instead of:

test("Should have corresponding URL", async () => {
  const uri = await page.url();
  await expect(uri.includes('example')).toBeTruthy();
});

Use appropriate method:

test("Should have corresponding URL", async () => {
  const uri = await page.url();
  await expect(uri).toHaveURL(/example/);
});

Or builtin assertions in case of an unusual checks:

test("Should have corresponding URL", async () => {
  const uri = await page.url();
  await expect(uri).toEqual(expect.stringContaining('example'));
});

This pattern applies for checking any strings and affects the readability and clarity of test reports.

11. Avoid regexp in checks

Checks with regular expressions make tests too sensitive and do not add much reliability to the tests, but make it difficult to analyze after failures.

There are two exceptions:

• regexp for checking URLs;
• regexp for date and time.

Both types of this kind of data are suitable for checking by regexp.

If your testing project includes IDs of a specific domain which can be attributed to some pattern, then testing them by regexp is also OK.

12. Wrap clicks and expectations into a promise

Instead of:

await page.locator('.button').click();

const response = await page.waitForResponse('https://example.com/');

await expect(response.ok()).toBe(true);

Do:

const [response] = await Promise.all([
  page.waitForResponse('https://example.com/'),
  page.locator('.button').click(),
]);

await expect(response.ok()).toBe(true);

Promise.all prevents a race condition between clicking and waiting for something. The first example is likely to be extremely flaky.

13. Do not use global variables for page object methods

Isolate tests/steps from each other. Do not use global variables which are used and rewritten by multiple test steps in a single test suite.

Instead of:

const myPageObject = new MyPageObject(page);

test('Should do something', async () => {
  await myPageObject.doSomething();
  …
});

test('Should have something', async () => {
  await myPageObject.haveSomething();
  …
});

Do:

test('Should do something', async () => {
  const myPageObject = new MyPageObject(page);
  await myPageObject.doSomething();
  …
});

test('Should have something', async () => {
  const myPageObject = new MyPageObject(page);
  await myPageObject.haveSomething();
  …
});

If variables are not rewritten, it reduces the probability of rewriting them incorrectly or asynchronously — it increases the overall stability of the tests.

14. Do not scatter test cases

The same functionality should be checked the same way everywhere.

For example, instead of having test-1.spec.ts test where you check a banner through expect A, and having test-2.spec.ts test where you check the same banner (but perhaps in the other page) through expect B, test that banner by both expects (A and B) in each of the tests (test-1.spec.ts and test-2.spec.ts).

15. Do not mix different kind of tests

If you want to check API and UI for a single user action — do two tests: API test and UI test.

If you want to check UI functionality and check the layout by screenshot simultaneously — do two tests: UI test and screenshot test.

If you want to check end-to-end API scenarios and to check JSON schemes simultaneously — do two integration API tests each of which makes certain checks.

16. Use linters and formatters from the testing project

If a directory with tests is located inside a testing project or tests are located in separate repository, or if tests are written by dedicated autotest engineers or by developers, tests should inherit linter and formatter rules from the testing (parent) project.

Tests will be closer to the testing code (and autotest engineers will be closer to developers), if ESLint and Prettier rules are the same.

Useful remark:

• Tests often contain many JSON objects, therefore, it is very necessary to have permission to use trailing commas — it simplifies diff and code review.

Your own principles for your own project could be dramatically different.