Selection Criteria of JavaScript Test Framework

When a frontend team starts building a testing infrastructure, there is always a debate about the choice of testing tools.

Selection Criteria of Test Framework
  • Ava — test runner;
  • CodeceptJS — end-to-end testing framework;
  • Cypress — end-to-end browser testing framework;
  • Hermione — test runner for browser testing based on Mocha and WebdriverIO;
  • Jasmine — testing framework;
  • Jest — testing framework;
  • Karma — test runner;
  • Mocha — test runner;
  • Nightwatch — end-to-end testing framework;
  • Node Tap — a Test-Anything-Protocol library (mostly for any kind of unit tests);
  • Playwright — end-to-end testing framework;
  • Protractor — deprecated end-to-end test framework for Angular applications;
  • Puppeteer — headless Chrome Node.js API (actually, it is not a testing tool);
  • SelenideJS — browser testing framework;
  • TestCafe — end-to-end browser testing framework;
  • WebdriverIO — end-to-end testing framework.
  1. Do you need unit or integration/end-to-end testing?
  2. Browser support (including mobile browsers);
  3. Documentation;
  4. Community (articles, video tutorials, stackoverflow answers);
  5. Support by developers (reactions on GitHub issues) and frequency of updates;
  6. Parallelisation (the power of test runner);
  7. Retries (the power of test runner);
  8. Skips and conditional skips (the power of test runner);
  9. Assertions (the power of assertion library);
  10. Reports (variety of console and HTML reports);
  11. WebSocket support;
  12. Selenium (WebDriver) support? DevTools Protocol support? Or both?
  13. Working with tabs (for example Cypress does not support that);
  14. Mocks/stubs;
  15. Proxy HTTP requests/responses;
  16. Support HTTP API requests;
  17. Setting cookies, localStorage and user-agents;
  18. Visual/screenshot testing;
  19. Debug mode;
  20. Headless and headed modes (for browsers);
  21. Integration with IDE (for example Playwright can run test in VSCode);
  22. Run on localhost;
  23. Run in CI/CD systems;
  24. Run a single test;
  25. Filter tests (ability to grep a bunch of tests);
  26. Logging the test execution process;
  27. Making screenshots and videos in case of failure (for example Playwright can even record traces);
  28. Distribution size (the number of dependencies in package.json);
  29. Maintainability (and how quickly new team members will get used to the project);
  30. Open Source and/or license.
  • Cypress;
  • Playwright;
  • WebdriverIO.
  • Jest;
  • Mocha;
  • And an additional HTTP request library for Node.js, like GOT.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andrey Enin

Quality assurance engineer: I’m testing web applications, APIs and doing automation testing.