Llm Validation Python Packages

boothpy

A lightweight, zero-dependency Python checkpoint layer for structured LLM output reliability.

llmvalidate

A comprehensive Python framework for evaluating LLM-extracted structured data against ground truth labels. Supports binary classification, scalar values, and list fields with detailed performance metrics, confidence-based evaluation, and statistical uncertainty quantification via non-parametric bootstrap confidence intervals.

t1-t2-protocol

MCP server for heterogeneous AI validation — T1 structured reasoning, T2 cross-model evaluation, and deterministic checksum. Python stdlib-only.

logicshield

Deterministic validation firewall that verifies AI-generated proposals against ground-truth state using immutable rules. Zero dependencies.

dual-llm-bench

Setting benchmarking standards for the LLM security!

llmtestr

A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern

llmtest-framework

pytest for LLM apps - Test for grounding failures, prompt injection, safety violations, and regressions

tef-estimator

FAIR cyber risk quantification toolkits by Laura Voicu: agent-based control simulation (FAIR-CAM), threat event frequency estimator (PyPI), LLM classification validator (PyPI), Monte Carlo risk engine with IRIS benchmarks.

formwork

Runtime semantic constraints and targeted repair for structured LLM output — declare which fields your code owns and which the model may invent.