Deepeval Python Packages

evaltrack

Pytest plugin that records, gates and tracks your LLM evals in the CI you already have.

mendmark-evals

Mutation testing for agent evals. Find the tool-use failures your tests miss before they reach production.

cnllm

Python toolkit for Chinese LLMs, with flexible batch capacity, structured real-time visulization and automated accumulation for streaming, and explicit feedback on vendor-native parameter validation.

truescore

Statistics for LLM-judged evaluations: bias-corrected scores, real confidence intervals, and comparisons that hold up

faultmap

Automatically discover where and why your LLM is failing — embedding-space clustering + statistical hypothesis testing to surface input slices with elevated failure rates and audit test suite coverage gaps.

auditai-cli

Developer-first LLM/RAG safety audits for CI/CD — faithfulness, relevancy, prompt injection (BYOK OpenAI/xAI). pip install auditai-cli