Principles of Writing Automated Tests

While working on test automation in different projects, I’ve learned that there are not enough static analyzers and code formatters for writing good tests. The team had to have an agreement on how the tests should be written.

Principles of Writing Automated Tests

1. No tests without assertions

There should not be test steps without checks.

test("Should open menu", async () => {
await page.locator('.button').click();
});
test("Should open menu", async () => {
await page.locator('.button').click();
const locator = await page.locator('.dropdown-menu');
await expect(locator).toBeVisible();
});

2. No assertions in before or after hooks

beforeAll, beforeEach, afterAll, afterEach hooks should not have assertions. Preconditions/postconditions should contain only pure actions (for example: authorization). Checks should be done inside tests.

3. No actions without expectations

After all actions in tests: clicks, hovers, gotos, etc. should be an assertion with expectation to check that the action was definitely committed.

test("Should open menu", async () => {
await page.locator('.button').click();
await page.locator('.dropdown-menu').waitFor({ state: 'visible' });
});
test("Should open menu", async () => {
await page.locator('.button').click();
const locator = await page.locator('.dropdown-menu');
await expect(locator).toBeVisible();
});

4. No unconditional expectation

Do not add pause and timeouts for N seconds between action and assertion to prevent flakiness — it only slows down the tests.

it('Should open menu', async () => {
const button = await $('.button');
await button.click();
await browser.pause(3000);
const menu = await $('.dropdown-menu');
await menu.isDisplayedInViewport();
});
it('Should open menu', async () => {
const button = await $('.button');
await button.click();
const menu = await $('.dropdown-menu');
await menu.waitForExist({timeout: 3000});
await menu.isDisplayedInViewport();
});

5. No commented test

If the test should be turned off, it should be skipped by test framework feature (skip), not by commented code.

// test("Should have a menu", async () => {
// const locator = await page.locator('.dropdown-menu');
// await expect(locator).toBeVisible();
// });
test.skip("Should have a menu", async () => {
const locator = await page.locator('.dropdown-menu');
await expect(locator).toBeVisible();
});

6. No hanging locators

Tests should not contain lines of code with «meaningless» locators:

test("Should do something", async () => {
await page.locator('.button');

7. One expect for each test step

Test steps should be short and each step should check only one thing.

8. Do not put await inside expect

One operation inside another operation leads to a complication.

test("Should have title on the button", async () => {
expect(await page.locator('.button')).toHaveText(/Menu/);
});
test("Should have title on the button", async () => {
const button = await page.locator('.button');
expect(button).toHaveText(/Menu/);
});

9. Do not reload the page, reopen it

Refreshing a page by a standard command (page.reload() for Playwright or browser.refresh() for WebdriverIO) is not a good idea — it makes the test flaky.

test("Should have something after reload", async () => {
await page.reload();

});
test("Should have something after reload", async () => {
const uri = await page.url();
await page.goto(uri);

});

10. Do not check URLs through includes

Do not use string.prototype.includes() for string comparison in assertions, because includes() returns true or false. When your check fails, you will get a report that false is not true — and no more details.

test("Should have corresponding URL", async () => {
const uri = await page.url();
await expect(uri.includes('example')).toBeTruthy();
});
test("Should have corresponding URL", async () => {
const uri = await page.url();
await expect(uri).toHaveURL(/example/);
});
test("Should have corresponding URL", async () => {
const uri = await page.url();
await expect(uri).toEqual(expect.stringContaining('example'));
});

11. Avoid regexp in checks

Checks with regular expressions make tests too sensitive and do not add much reliability to the tests, but make it difficult to analyze after failures.

  • regexp for checking URLs;
  • regexp for date and time.

12. Wrap clicks and expectations into a promise

Instead of:

await page.locator('.button').click();

const response = await page.waitForResponse('https://example.com/');

await expect(response.ok()).toBe(true);
const [response] = await Promise.all([
page.waitForResponse('https://example.com/'),
page.locator('.button').click(),
]);

await expect(response.ok()).toBe(true);

13. Do not use global variables for page object methods

Isolate tests/steps from each other. Do not use global variables which are used and rewritten by multiple test steps in a single test suite.

const myPageObject = new MyPageObject(page);

test('Should do something', async () => {
await myPageObject.doSomething();

});

test('Should have something', async () => {
await myPageObject.haveSomething();

});
test('Should do something', async () => {
const myPageObject = new MyPageObject(page);
await myPageObject.doSomething();

});

test('Should have something', async () => {
const myPageObject = new MyPageObject(page);
await myPageObject.haveSomething();

});

14. Do not scatter test cases

The same functionality should be checked the same way everywhere.

15. Do not mix different kind of tests

If you want to check API and UI for a single user action — do two tests: API test and UI test.

16. Use linters and formatters from the testing project

If a directory with tests is located inside a testing project or tests are located in separate repository, or if tests are written by dedicated autotest engineers or by developers, tests should inherit linter and formatter rules from the testing (parent) project.

  • Tests often contain many JSON objects, therefore, it is very necessary to have permission to use trailing commas — it simplifies diff and code review.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andrey Enin

Quality assurance engineer: I’m testing web applications, APIs and doing automation testing.