Automated Cypress Cloud Flaky Test Scorecard
Every Monday, WebRun opens Cypress Cloud, compiles the week's flaky test report across every project, updates a Notion test health page with the top offenders, and logs every flaky test to Google Sheets so the trend is visible and nobody has to dig through run history to find what keeps failing intermittently.
How do I track flaky Cypress tests across my projects?
WebRun compiles your Cypress Cloud flaky test report every Monday, across every project. It updates a Notion test health page with the top offenders and logs every flaky test to Google Sheets, so intermittent failures get tracked as a trend instead of being rediscovered by whoever hits them next.
- Flaky tests are tracked as a trend instead of rediscovered repeatedly
- Chronically flaky tests stand out after a few weeks of logging
- The test health page stays current without a manual review
Built for QA teams · test engineers · frontend teams · software engineering teams
What does WebRun do on every run?
The exact actions WebRun takes, in order - in plain language, so you can adjust anything.
-
WebRun signs in and gets to work
Opens
cloud.cypress.ioin a real browser with your saved login - no setup, no API keys. -
1
Cypress Cloud - compile the week's flaky test report
- Open Cypress Cloud and pull the week's flaky test report across every project
- Capture the test name, project, and flake rate for each
- Rank them by how often they flaked
Done when Every flaky test this week is listed with its project and flake rate.
-
2
Notion - update the test health page
notion.so
WebRun opens Notion to update the test health page. - Update the test health page with this week's top flaky tests
- Show the flake rate and project for each
- Keep the page as the current at a glance summary
Done when The Notion page reflects this week's top flaky tests.
-
3
Google Sheets - log each flaky test to a sheet
google.com
WebRun opens Google Sheets to log each flaky test to a sheet. - Log every flaky test as a new row with its project and flake rate
- Keep prior weeks in place so the trend is visible
- Note any test that has been flaky for 3 weeks running
Done when This week's flaky tests are logged as new rows in the sheet.
How is each run configured?
Secure by default
Connect once, stays signed in
WebRun signs in once and keeps each session in a persistent environment, so every run picks up right where it left off.
Every action is checked against this policy before it runs.
Questions, answered
Will it fix or quarantine the flaky tests?
No. WebRun only reports which tests are flaky. Fixing or quarantining a test is always done by an engineer.
How is a test marked flaky?
It's the flaky test detection Cypress Cloud already runs. WebRun reports what Cypress Cloud found, it does not calculate its own flake score.
Can I see which tests are chronically flaky?
Yes. Because every week's flaky tests are logged as new rows, a test that keeps showing up week after week stands out in the sheet.
Related templates
Put this on autopilot.
Turn it on in minutes - or have our team set it up for you.