Retrieval Evaluation Python Packages

retrieval-eval

Your RAG test suite decays every time you re-chunk. Drift detection that tells you which relevance labels are still true after a chunker or corpus change and re-anchors the ones it can, plus deterministic retrieval metrics and a portable, TREC qrels-compatible judgment format.

toolkit-rag-quality

Retrieval regression testing in CI: trec_eval-equal metrics, per-query diffs, significance tests and leakage checks.

smallevals

Small Language Models Evaluation Suite for RAG Systems

arxiv-embedding-benchmark

Benchmark embedding models on academic paper similarity and retrieval tasks.