Ai Reliability Python Packages

evalseal

Git-style receipts for LLM evals: measure variance, capture provenance, detect flaky verdicts, and verify results in CI

clinical-ai-reliability

Open-source reliability framework for medical and health AI systems.

lintlang

Static analysis for AI agent configs, tool descriptions, and system prompts — catches vague tool descriptions, missing stop conditions, and schema gaps before they reach runtime. Zero-LLM, deterministic checks, built for CI.

boothpy

A lightweight, zero-dependency Python checkpoint layer for structured LLM output reliability.

fidelis-memory

Zero-LLM agent memory for Claude Code and AI agents: local-first BM25, dense-vector, and reciprocal-rank-fusion retrieval. Returns original passages verbatim by default. Available on PyPI as fidelis-memory. Apache-2.0.

agentverity

Qualify repeated categorical AI-agent evidence before it becomes a regression reference

bounded-loops

Bounded Loops & Graphs: 69 agent loops (65 keyless), 7 reference graphs, independent gates, declared bounds, verifiable receipts. Paper: arXiv:2609.27871

hermes-jailbench

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

hermes-rubric

Evidence-first LLM-as-judge scoring for AI artifacts — papers, PRs, prompts, cold emails: synthesizes a rubric, collects quoted-evidence citations, scores only against that evidence, and hedges on thin evidence. Every dimension ties to a file:line or quote, with reproducibility receipts. 7 backends.

pygate-ci

quick-gate-python (PyPI: pygate-ci) is a deterministic Python CI quality gate that normalizes Ruff, Pyright, and pytest results into one fail-fast decision, attempts bounded auto-repair, and escalates with machine-readable evidence for humans or agents when it cannot finish safely. A gate-and-escalate wrapper, not a lint dashboard.

claude-router

claude-router is a local prompt router that picks the right Claude model tier and prepends the right scaffold using local embeddings before you call the API. A deterministic routing layer for eval, research, content, and review prompts that helps teams stop overspending on Sonnet and Opus when Haiku plus structure is enough.

rule-audit

Static analyzer for AI system prompts: parses a prompt into normative rules and reports contradictions, coverage gaps, priority ambiguities, and absolute-rule edge cases - no LLM calls. Deterministic pure-Python lint with CLI, Python API, and CI exit codes. pip install rule-audit

hermes-prime-mcp

Drift-prevention session-init convention card for fresh Claude Code sessions. Injects a self-contained card so a new session opens with its grounding triggers, calibration rules, and tool-map in scope — instead of re-deriving them at minute 30. Per-project, marker-anchored, idempotent, reversible. Bash installer plus MCP server.

failroute

Static detection of failure-routing anti-patterns in Python (swallowed exceptions, silent fallback returns, masked exceptions), with an empirical study of how often findings are real bugs. AST-based, SARIF, PyPI.

agentassert-abc

Runtime behavioral contracts for AI agents: define YAML contracts, check adapter outputs, and raise explicit contract violations. Agent Behavioral Contracts (ABC).

agent-convergence-scorer

CLI and Python library for lexical stability checks on repeated short agent outputs: pairwise exact agreement, token overlap, divergence point, and an order-independent score. Zero runtime dependencies.

csv-quality-gate

csv-quality-gate is a command-line data quality gate that runs CSV preflight validation, failing fast before an ML or LLM pipeline ingests broken, incomplete, duplicated, or junk input. It checks missing columns, empty files, empty cells, and duplicate rows, returning pass, warn, or fail with matching exit codes. Stdlib-only, CI-ready.

agent-kickstart

A guided onboarding project for people new to Claude Code — no coding or terminal experience required. Asks a few questions, proposes real starter projects shaped around what you care about, and begins one with you.

langstate

Inspectable context compression for LLM conversations: turn older history into visible scaffold state, keep recent turns verbatim, and check named facts with lexical receipts.

hermeneutic

Turn corrections from AI agent logs into guidance for similar tasks. Local memory with semantic retrieval, prompt-context hooks, and response checks.

intent-verify

intent-verify is a deterministic, zero-LLM CLI that checks whether a repo's source still lexically covers the acceptance items in a markdown spec, INTENT.md, or handoff doc, returning verified, partial, or missing. A fast guardrail for catching spec-vs-code drift before review, release, or handoff. Lexical coverage, not semantic proof.

zer0dex

A local dual-layer memory pattern for AI agents: a compact, human-readable markdown index paired with semantic retrieval from a local vector store, queried before each message. For cross-project recall where flat memory files or vector-only RAG fall short. Local-first. Reference implementation.

langquant

Explicit, inspectable conversational state outside the chat transcript for local LLMs.

zer0lint

zer0lint is a memory-extraction health diagnostic for mem0 configs and HTTP memory endpoints. It flags silent failure modes where ingestion reports success but facts never survive the LLM extraction step, then generates a stronger extraction prompt validated on your own model. Works over HTTP with any add/search memory API.