Playwright Productivity Hack: How --only-changed Can Save Your Day

Introduction

• Brief overview of Playwright:

  • Playwright is a testing library for automating web browsers. It supports Chromium, Firefox, and WebKit browsers, providing a unified API for interacting with web pages across different browsers.

  • Read more in this blog -> Playbook Blog

• Importance of quick iterations in testing with --only-changed

  1. Focus on the test case at hand

  2. Focus on affected test cases

  3. Less time wasted in iterating all the test cases or selecting and iterating the test cases

Understanding the --only-changed option

• What it does:

  • Improves Efficiency: By running only the necessary tests, you can significantly reduce the time it takes to execute your test suite, especially for large projects with many tests.

  • Reduces Noise: Focusing on changed tests can help you identify the root cause of issues more quickly, as you can eliminate irrelevant tests from the equation.

• When to use it:

  • Large Test Suites: For projects with large test suites, using --only-changed can dramatically reduce the time it takes to run all your tests.

  • Continuous Integration: As part of your continuous integration pipeline, using --only-changed can help you reduce the time it takes to run your tests and identify issues early in the development process.

Implementing --only-changed

• Basic syntax

    npx playwright test --only-changed #basic
  

• Advance use cases

  1. Iterating test cases against a branch: main, development

  2. Iterating test cases since the a specific commit

      npx playwright test --only-changed=<branch-name>
      npx playwright test --only-changed=<commit-number>
     

Benefits of using --only-changed

• Use when you've changed constants and wanted to see the affected test cases

• When you want to run tests that you changed since last commit

• Run tests faster for affected changes in your new branch

  • Example you're on a new branch new-branch and want to run test cases which are just in your new branch against your main

  • npx playwright test --only-changed=main will only run test cases which are different from the latest commit on main branch

• Running test cases since a commit in timeline

  • Example if you want to run test cases which were affected since last two commits; copy the commit id from your history

  • git log -2 to check the second last commit

  • npx playwright test --only-changed=e2245ff672bb618c485b793c3c258064a67f6025

    

Potential pitfalls and how to avoid them

• Incorrect Change Detection:

  • Dependency Changes: Changes in dependencies or external libraries might not be reflected accurately if they don't directly impact the test files.

  • Formatting Changes: Minor formatting changes (e.g., whitespace, indentation) might lead to unnecessary test re-runs, especially if your version control system strictly enforces formatting rules.

• Flaky Tests: Flaky tests (tests that fail intermittently) can lead to false positives or negatives, making it difficult to determine if a change is actually causing a problem.

• Dependency Conflicts: Conflicting dependencies between tests can cause unexpected behaviour and make it difficult to pinpoint the root cause of changes.

Conclusion

This article provides a comprehensive guide on using the --only-changed option in Playwright to speed up test iterations by running only the tests affected by recent changes. It explains the importance of quick iterations in testing, the setup required, and the benefits of this option. Additionally, it covers practical examples, including running tests changed since the last commit or specific branches, and addresses potential pitfalls and solutions. Encouragement is given to integrate this option into regular workflows for efficient testing.

Ending Note

--only-changed if used the right way in your organisation can save a lot of efforts and running and iterating tests faster. Good planning and well defined use case for your project would really help your team to excel.

Join Community

You can read more about this on DevAssure’s Community Page. Click on the banner below to redirect to Community Page.