Flaky Tests Python Packages

astral-dev-toolchain-cargo-nextest

A next-generation test runner for Rust.

pytest-sentry

Track flaky tests in Sentry

testopshub

基于 pytest 的测试可信度治理实践：批量执行、终端输出结构化解析、Allure 桥接、CI 六步门禁（测试数下限/增量类型检查/依赖分级）拦截删测试换绿；跨平台进程树清理与 Redis 队列可靠性语义。支持 HTTP 与串口/Telnet（模拟验证）协议用例。

pytest-html-plus

Generate Actionable, automatic screenshots, unified Pytest HTML report in less than 3 seconds — no hooks, merge plugins, no config, xdist-ready.

verdict-qa-mcp

Claude Code QA agent with memory, plus a read-only MCP server (verdict-qa-mcp on PyPI): baseline → delta runs (NEW/REGRESSED), a fact harness so the model judges while the system measures, signed run history, flaky quarantine with expiry, and an exit-code release gate for CI. Ships six scored eval fixtures — misses published.

retracesoftware

Turn failed Python tests and CI runs into replayable debug sessions.

flakemap

Find the flaky tests in your CI history: JUnit XML or .NET TRX from past runs in, a ranked, statistically honest flake report out.

tracereports

Self-hosted test reports that tell you why a test failed: steps, screenshots, browser network and AI diagnosis that groups failures by root cause. pytest · Playwright · Selenium · JUnit · Go.

flakelens

Separate real LLM failures from flaky ones. Repeat-run evals with Wilson confidence intervals. Zero deps.

waitless

Automatic UI stabilization for Selenium using DOM, network, animation, and layout signals.

pytest-reporter-plus

Generate Actionable, automatic screenshots, unified Pytest HTML report in less than 3 seconds — no hooks, merge plugins, no config, xdist-ready.

pytest-acereports

Ace Reports is a unified test reporting solution for **.NET, Python, and Java**. Automatically capture API requests and responses, track retries and flaky tests, collect logs and screenshots, and generate self-contained HTML reports — all with minimal setup and no manual reporting instrumentation.

lcb-gate

Statistical pass/fail gating for flaky tests and LLM evals: Wilson lower-bound verdicts, paired A/B, pytest plugin, stdlib only

flakestat

Find flaky tests in any language - pytest, Jest, go test, JUnit. Local CLI, no SaaS, no account.

pytest-probability

A semantic bug won't reproduce in one run — and can't hide from ten. pytest plugin for nondeterministic code: N runs per case, pass fractions, flaky detection, token cost.

passrate

hyperfine, but for pass rates. Is it better, or did you get lucky? Wilson intervals, Fisher's exact test and early stopping that doesn't cheat. 0 deps.

pytest-together

Run things at the same instant in a test, and fail if they did not actually overlap.

radarflake

Flaky test detection, historical tracking, and AI-assisted root cause analysis for pytest. Works with zero API cost, optional Groq/Gemini/Ollama or your own key.

testmatrix-hub

基于 pytest 的测试可信度治理实践：批量执行、终端输出结构化解析、Allure 桥接、CI 六步门禁（测试数下限/增量类型检查/依赖分级）拦截删测试换绿；跨平台进程树清理与 Redis 队列可靠性语义。支持 HTTP 与串口/Telnet（模拟验证）协议用例。

pytest-unflakable

Official Unflakable Plugins for Python

mendwright

Playwright & Selenium selectors that repair themselves when a site changes. Fixes from memory first, a 1 GB local model when needed, and every fix checked on the live page.

pytest-cloudreport

pytest plugin for test analytics and flaky-test detection. Free local HTML reports out of the box, or upload to cloudreport.dev for cloud history and team dashboards.

limen-eval

The same-configuration noise floor of an evaluation: verdict flakiness, sign-stability rulings, and a CI gate over repeated identical runs

qualflare-pytest

Native pytest plugin for Qualflare — results, per-attempt retry history with every attempt's error, flakiness, steps and attachments. Supports pytest-xdist, pytest-rerunfailures and pytest-bdd. Makes no network calls: writes a standardized report directory that qualflare-cli collect uploads.