Pipeline-Native Models for Detecting and Minimising Flaky Tests

Abstract

Flaky tests - non-deterministic pass/fail outcomes on unchanged code—undermine CI/CD reliability. We present a pipeline-native framework that combines Bayesian outcome modelling, controlled test re-ordering, coverage- and change-aware triage, and a light-weight machine-learning predictor to identify likely flakiness early and cheaply. A cost-aware policy allocates re-runs and restricts quarantine to time-boxed, auto-revalidated cases. Minimisation emphasises hermetic builds, deterministic time/network abstractions, concurrency isolation, and automated fixes (e.g., seeding, teardown guards). The design is grounded in empirical evidence on dominant causes (async waits, concurrency, order dependency) and industrial practice at scale. The result is reduced noise, preserved bug-finding power, and faster feedback cycles in CI/CD.