Statistics of Assertions in Playwright Test Suite

When the number of tests (test files) for my current work project passed over a hundred, it became interesting to see what kind of assertions are the most frequently used.

Introductory notes about the testing project:

My testing project is a modern client-server application. Backend (API) is tested separately. Frontend (on React) is full of business logic and has a multimodal interface:

UI mockup; background image source

Automated tests cover more than 80% of the functionality and are the main component of regression testing.

Playwright test suite has:

• 103 test files;
• 955 tests;
• 601 expect methods (in 97 test files);
• There are 50 negative checks among them (assertions has .not before matchers) ~ 8.5% of expects are negative;
• No assertions in page object models, but there are 185 awaits for locators;
• No assertions in beforeAll or afterAll hooks (see №2 in Principles of Writing Automated Tests);
• All assertions are executable (no if-statements in tests, see №7 in Principles of Writing Automated Tests);
• No soft assertions;
• All tests run for 14 min in 4 threads in CI.

I need some clarification on it: I use waitFor() method in page objects for the waiting locator’s specific state which may considered as a kind of assertion because the test will fail if the locator does not satisfy the condition. That is why some tests (test files) can have none of expect assertions because they consist only of page objects with waitFor() like this:

Page object example:

export class Dungeon {

  private gemstone: Locator;

  constructor(page: Page) {
    this.gemstone = page.locator('.mineral_crystal');
  }

  async waitForGem(visibleState: boolean): Promise<void> {
    await this.gemstone.first().waitFor(state: 'visible');
  }
}

Test example:

test('Should have a gem', async () => {
  const dungeon = new Dungeon(page);
  await dungeon.waitForGem(true);
});

That is why I added waitFor() to the top list of assertions along with Genetic Assertions.

Assertions top list:

1. .toBe— 328
2. waitFor() — 185
3. .toStrictEqual — 67
4. .toContain — 54
5. .toEqual — 39
6. .toHaveURL — 34
7. .toHaveAttribute — 22
8. .toBeChecked — 15
9. .toHaveClass — 14
10. .toBeGreaterThanOrEqual — 10
11. .toBeNull — 6
12. .toMatch — 4
13. .toBeTruthy — 4
14. .toBeGreaterThan — 2
15. .toBeFalsy — 2

Assertions top list: expects + waitFor()

Results are very expectable:

• A little over 60% — are checking data for some value;
• About a quarter — are waiting for something (locators/text);
• The remaining 10–15% — are URLs/titles matching, uncommon checks for CSS attributes, etc.

That is very similar to previous projects I have worked on.

Read more about Platwright assertions: Assertions.