Advantages and Disadvantages between Cypress and Playwright

As a test engineer I have been using WebdriverIO with Selenium WebDriver for frontend automation for a few years.

Recently I got an opportunity to move away from Selenium and therefore had to compare alternatives (inside my usual stack of TypeScript and Node.js):

• Cypress, as the most hyped automation tool;
• Playwright, as a more high-level tool close to the browser.

Puppeteer did not hit my shortlist because it is only an API (Chrome DevTools Protocol) to control Chromium and it needs additional testing framework (Jest, Mocha or something else). WebdriverIO can also work by DevTools Protocol, but I wanted to try something different.

Disclaimer: all listed pros and cons are subjective and based on my own experience.

Cypress

Advantages:

1. Luxury dev experience! Running and debugging tests in the app is extremely convenient.
2. Brilliant documentation! Cypress has the best documentation I have ever met. During the installation process and writing the first tests, I used only documentation and examples from it.
3. All in one instrument. Test runner, assertion library, HTTP request library and many more are already built in Cypress. No need to choose different tools for building testing infrastructure. It also supports plugins for more functionality.
4. Chai as an assertion library (unlike Playwright with humble Jest expects).
5. Getting elements and interactions with them have a built-in «wait until» for the actionable state of the element.
6. Easy access to network activity. No need for any external proxies to stub network requests and responses — it is also built in Cypress.
7. Ready-made support of custom commands (and even types for you custom commands if you use TypeScript).
8. Auto-making screenshots on failures.

Curious features:

1. There is an Electron as default «browser». Despite the fact that the Electron is based on Chromium it is not the same environment as a pure browser. For example, my tested web app started behaving unstable and sending console errors, so I had to switch to Chromium — lucky that Cypress supports cross-browser testing in Firefox and Chrome-like browsers.
2. It watches for uncaught exceptions from your web app during the test run. It could be useful if your project has zero tolerance to console errors.
3. Tests can work only in one tab. You can not test muli-tab scenarios in Cypress.
4. Cypress uses its own environment variables. If you are used to using Node.js’s process.env variables, you have to switch for Cypress’ ones and include each in the config file.

Disadvantages:

1. Cypress includes tons of dependencies. Your node_modules directory will get heavier by several tens of pounds.
2. It is forbidden to use console.log for debugging. You have to use cy.log which accepts only a string.
3. All the tests are somehow merged during the test run. It means that you can not use variables with the same names in different test files.
4. Cypress did not start from the first time inside CI — I got a weird error in TeamCity.

Playwright

Advantages:

1. Inspector — a GUI tool for debugging tests. Not as good as Cypress has, but still worthwhile.
2. Fast work in headless mode (I did not make accurate performance measurements, but it just feels faster than Cypress).
3. Auto-waiting of elements before performing some actions.
4. Visual regression testing is working out of the box (in Cypress you have to use a plugin).
5. Detailed HTML report, which could be useful both at local run and in CI.
6. Auto-making screenshots, videos and traces on failures — all of them are great for debugging failed tests, especially in Trace Viewer.

Curious or controversial features:

1. Test generator — a GUI tool for record tests. Of course, the generated results can not be used for real tests: chosen selectors are odd and need to be replaced, and expects should be added manually later. But it could be handy for creating rough drafts for simple tests.
2. There is no support for custom commands, as Cypress and WebdriverIO have. Playwright pushes you to use Page Object Model instead of creating custom syntactic sugar around your testing infrastructure.
3. Most of page’s and locator’s methods work only with the first element on the page. You have to seek out special methods which are allowed to work with multiple elements. Playwright pushes you to choose extremely unambiguous selectors, while Cypress or WebdriverIO do not limit you this way.
4. P̶l̶a̶y̶w̶r̶i̶g̶h̶t̶ ̶d̶o̶e̶s̶ ̶n̶o̶t̶ ̶a̶l̶l̶o̶w̶ ̶p̶a̶s̶s̶i̶n̶g̶ ̶a̶n̶ ̶a̶r̶b̶i̶t̶r̶a̶r̶y̶ ̶h̶e̶a̶d̶e̶r̶ ̶t̶o̶ ̶t̶h̶e̶ ̶t̶e̶s̶t̶ ̶p̶a̶g̶e̶ (sorry, I figured it out, it is possible). But anyway, I could not pass a single header Cookies with multiple cookies, I had to set each cookie one by one. The same strange thing with localStorage: I could not pass only a single localStorage to the new browser’s context, because setting a storage state implies the object of both mandatory cookies and localStorage (and not just one thing). On the one hand, this is strictly right, but on the other hand it is an excessive restriction.

Disadvantages:

1. Сonfusing documentation. Without a doubt a Playwright’s documentation is good, but looking for lore takes time and examples are quite poor. After the installation process I had to use Google and Stackoverflow to figure out some of the details.
2. Despite the fact that Playwright allows API testing, its request method is not enough advanced. For example, it does not support disabling follow redirects (but Cypress does).
3. Screenshots, videos and traces on failures are made inside one test() function. It means that if you have a few tests inside test.describe(), you will get videos and traces for each nested test, but not for a whole root test function (steps have the same behavior). It is a shame that so many artifacts can not be assembled into one piece.
4. TeamCity is not supported out of the box — you have to write your own reporter for CI builds.

Playwright in Debug Mode

Playwright is a more technical tool, which works closer to the browser and has precise commands. Cypress is a more flexible tool with an expectable API for testers, which allows to develop tests easy and fast enough.