Flaky Test Hunter
By n o a h · Published · Updated
Finds out why a test passes sometimes and fails others, checking timing, shared state, ordering and environment, then fixes the cause instead of adding retries.
Prompt template
## Flaky Test Hunter Find the real cause of a flaky test and fix it. - Test name and code: [textarea: Paste the test and what it exercises] - Failure output: [textarea: Paste one failing run and one passing run if you have them] - How often it fails: [text: 1 in 10 locally or 1 in 3 on CI] - Where it fails: [dropdown: Only on CI, Only locally, Both, Unknown] - Test framework and runner: [text: Jest or pytest or Playwright] - Recent changes: [textarea: Anything merged near when it started failing] Sort suspects into four buckets and test each in order. Timing: sleeps, unawaited promises, animations, timeouts, clocks and time zones. Shared state: globals, database rows, files, caches and singletons left by earlier tests. Order and parallelism: does it pass alone but fail in the suite, or fail when run in a different order or with other workers. Environment: CPU speed, network, random seeds, locale, resource limits and version drift. For each bucket say what evidence in the output points toward or away from it, and give a command that would confirm it, such as running 50 times, running in isolation, or fixing the seed. Name the most likely cause, then give the fix at the root: wait on a condition rather than a delay, isolate state, or control the clock. Reject retries and longer timeouts unless nothing else is possible, and say so.