Faithfulness Python Packages

rag-benchmarking

RAG Benchmarking — Framework-agnostic RAG/agentic-AI evaluation harness. Faithfulness, agentic metrics, EU AI Act Article 15 accuracy evidence. Apache 2.0.

openagent-eval

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

groundguard

Verify LLM output against your source documents. Catch hallucinations in RAG pipelines and agentic workflows before they reach users.

auditai-cli

Developer-first LLM/RAG safety audits for CI/CD — faithfulness, relevancy, prompt injection (BYOK OpenAI/xAI). pip install auditai-cli

faithscore

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models