Hidden Gems of Playwright

The main tools for writing UI tests are: browser, IDE and testing framework documentation.

Playwright’s documentation is rich and intricate at the same time. Sometimes you can make a check or perform an action one way, and after a while find that it could be done in another way — just because you look into the API section instead of Docs or vice versa.

I picked up a few «features» that excited me when I discovered them:

• maxFailures
• Custom Expect Messages
• test.slow
• expect.soft
• test.step
• test.fixme
• Explore Selectors in Playwright Inspector

maxFailures

Docs

Let’s start from a config file: after reaching the set number of maximum failures, Playwright will stop testing and exit with an error.

The maxFailures parameter is an ultimate way to speed up a feedback loop of your CI/CD in case of a broken test environment. If too many tests fail simultaneously, it often means that something is wrong with the environment, not with test cases — it is preferable to stop the testing and investigate the reason for the mass failure, than to wait for the completion of all remaining tests.

const config: PlaywrightTestConfig = {
  maxFailures: 10,
};

Or, if you pass CI option (as environment variable) to determine the run in a continuous integration server (TeamCity, Jenkins, Travis CI, Drone, etc.):

const config: PlaywrightTestConfig = {
  maxFailures: process.env.CI ? 10 : 0,
};

Example of env.CI = true in a build configuration in TeamCity

Custom Expect Messages

Docs

Any assertion (expect() function) can be endowed with its own error message. This allows to describe checks more precisely, which is a great asset for the test report.

await test('Should have proper page title', async () => {
  await expect(page, 'Should have "Home" in title').toHaveTitle(/Home/);
});

Example of custom expect error message in HTML report

test.slow

Docs

Declaration of a «slow» test will triple the default timeout. If your playwright.config.ts file has timeout: 10000, then a specified test will be allowed to run 3 * 10000 = 30000 ms. It is very handy when only one test in a group of tests has long execution.

test('Test name', async ({ page }) => {
  test.slow();
});

expect.soft

Docs

Playwright will terminate test execution in case of a failed assertion, but soft assertions prevent this behavior. Test with soft assertions will continue to run, but in the end it will be marked as failed. This may be useful if you have a minor checks inside a big test.

await test.step('Should have proper page title', async () => {
  await expect.soft(page, 'Should have "Home" in title').toHaveTitle(/Home/);
});

Example of HTML report with failed soft assertion — all test steps are passed, but the entire test was marked as failed

I personally am against having soft assertions in tests — tests must be unambiguous and obvious (soft assertions are hiding problems instead of highlighting), but anyway this feature could be counted as a nice gem.

test.step

Docs

As an old user of testing frameworks like mocha and jest, I am used to dividing a big test on a bunch of small tests and logically grouping them with describe() method inside one test file:

test.describe('Home page toolbar', async () => {
  test('Should have proper page title', async ( => {…});
  test('Should have toolbar', async () => {…});
  test('Should have proper toolbar title', async () => {…});
});

Playwright has an additional way of writing test with step() method:

test('Home page toolbar', async ({ page }) => {
  await test.step('Should have proper page title', async () => {…});
  await test.step('Should have toolbar', async () => {…});
  await test.step('Should have proper toolbar title', async () => {…});
});

Both approaches has pros and cons:

Describe:

• Each test in describe() is clearly represented in reporters;
• Each test in describe() is isolated from others. Thanks to the insulation tests inside the describe() can be run as sequentially, as parallel.

Step:

• Steps are hidden in console reporters (list, line and dot), but can be shown in the «Test Steps» section in HTML reporter. For some engineers this may be acceptable — no extra data in reporter, more steps and actions could be included in one test, — but for others this can be considered as less informative reports and non-atomic tests.
• Recorded videos and traces in case of failure are recorded within the test() — that means you can playback a full test with all steps. This is highly convenient for debugging. This debugging method (through videos and traces) is almost impossible for atomic tests inside describes, because artifacts would be extremely short and will not contain data about previous test’s «steps».

Example of HTML report of identical tests: one with describe and child tests, another with child steps

Furthermore, you can combine describes and steps inside a single test file to gather all the power of Playwright’s test runner.

test.fixme

Docs

Marking a test with the «fixme» method will skip it during testing. It is the easiest and fastest way to skip tests (or a group of tests if it stands under describe()).

test('Should have proper page title', async () => {
  test.fixme();
  await expect(page, 'Should have "Home" in title').toHaveTitle(/Home/);
});

Example of skipped test in a list reporter

Explore Selectors in Playwright Inspector

Docs

Sometimes, when you try to choose the right selector for a locator, it is not easy to do it from the first time, especially if the page layout is complicated. For such cases you can test your locators in the «Explore» input in Playwright Inspector.

Example of exploring selector

For thoughtful debugging you can add await page.pause() on a desired line of the code — it will stop test execution and will not close Inspector window after timeout.