Llm As Evaluator Python Packages

langtest

Deliver safe & effective language models

vllm-judge

A tiny, lightweight library for LLM-as-a-Judge evaluations on vLLM-hosted models.

nlptest

Deliver safe & effective language models

openjury

An sdk and framework for evaluating and comparing multiple model outputs using configurable LLM-based jurors

llm-antibodies

Antibodies for LLMs hallucinations (grouping LLM as a judge, NLI, reward models)

xfinder

[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation

antibodies-rafaelsandroni

Antibodies for LLMs hallucinations (grouping LLM as a judge, NLI, reward models)