Automated AB Tasty Stale Test Alerts
Every Monday, WebRun opens AB Tasty, checks every running test's start date and current sample size against its planned duration, logs any test that has run long past plan to Airtable, and posts the chase list into Notion for the owner to close out.
How do I catch AB Tasty tests that have run past their planned end date?
WebRun checks every running AB Tasty test's planned duration and sample size each Monday, logs any test still running long past plan to Airtable, and posts the chase list in Notion with each owner named. Experimentation teams close out lingering tests instead of leaving them running indefinitely with no one accountable.
- Lingering tests get a named owner and a chase entry
- Nothing runs indefinitely without someone accountable for closing it
- The longest overdue tests are always at the top of the list
Built for experimentation teams · product managers · conversion rate optimization teams
What does WebRun do on every run?
The exact actions WebRun takes, in order - in plain language, so you can adjust anything.
-
WebRun signs in and gets to work
Opens
abtasty.comin a real browser with your saved login - no setup, no API keys. -
1
AB Tasty - check test duration and sample size
- Open AB Tasty and list every currently running test
- Read each test's start date, planned duration, and current sample size
- Flag any test running past its planned end date
Done when Every running test's duration is checked against its plan.
-
2
Airtable - log the overdue tests
airtable.com
WebRun opens Airtable to log the overdue tests. - Open the experiment tracker base in Airtable
- Log each overdue test with its owner and how far past plan it is
- Keep the running list so nothing gets chased twice
Done when Every overdue test is logged with its owner.
-
3
Notion - post the chase list
notion.so
WebRun opens Notion to post the chase list. - Open the experiment chase list page in Notion
- Post each overdue test with its owner and days past plan
- Put the longest running ones at the top
Done when This week's chase list is published in Notion.
How is each run configured?
Secure by default
Connect once, stays signed in
WebRun signs in once and keeps each session in a persistent environment, so every run picks up right where it left off.
Every action is checked against this policy before it runs.
Questions, answered
Does WebRun stop the overdue test itself?
No. It only reads test duration and sample size and flags what's overdue. Ending a test and shipping the result stays the owner's call.
What counts as overdue?
Any test still running past the planned duration you set when it launched. Tests still within plan are not chased.
Will a test get chased more than once?
It stays on the Airtable log so it's tracked, but the Notion list is meant to prompt one clear close out action, not repeat nagging.
Related templates
Put this on autopilot.
Turn it on in minutes - or have our team set it up for you.