Model Comparison Python Packages

regscorepy

This is the repo for a python package that does model comparison between different regression models.

jaxns

Probabilistic Programming and Nested sampling in JAX

rankweave

Dependency-free Python library for retrieval fusion (RRF), rank evaluation, statistical comparison, and TREC benchmarking, with an auditable CLI.

decishift

Open-source ML behavioral regression testing for comparing decision-system versions, identifying changed actions, attributing shifts, and gating releases with Decision Contracts.

llmswap

Ask, compare, and synthesize across 10 LLM providers. Python SDK, CLI, local Arena, MCP, and privacy-aware

promptseal

Regression testing for prompts, agents, and models. Know what breaks before you switch.

cli-modelarium

Statistically rigorous LLM comparison CLI for terminal-first developers. Compare 12 cloud providers + local models side-by-side with bootstrap confidence intervals, significance testing (paired t-test, McNemar), LLM as judge, hallucination detection, and CI/CD assertions.

lazygrid

Automatic, efficient and flexible implementation of complex machine learning pipeline generation and cross-validation.

octopus-ml

A collection of handy ML and data visualization and validation tools. Go ahead and train, evaluate and validate your ML models and data with minimal effort.

bayes-compass

COMPASS: Comparison Of Models using Probabilistic Assessment in Simulation-based Settings

madys

MADYS: isochronal parameter determination for young stellar and substellar objects

frontiertrials

Private, browser-local blind comparisons of AI products using your own work. No API keys.

llm-bench-cost

Rank LLM APIs by cost-effectiveness against Arena ELO scores · 按 Arena ELO 性价比对 LLM API 实时定价排名

depth-lens

Cost-vs-accuracy CI for LLM ops. Pick the cheapest API tier, compare self-hosted vLLM vs cloud APIs on one Pareto, and grade free-form output with an LLM-as-judge - all on your own data with Wilson 95% CIs.

dissectml

The missing middle layer between EDA and AutoML — unified pipeline from deep data understanding to model comparison in 3 function calls. Deep EDA, 36-model battle arena, statistical significance testing, target leakage detection, and publication-ready HTML reports.

bfas

Brute Force Architecture Search

model-compare

model-compare evaluates AI models side‑by‑side based on user tasks, rating accuracy, creativity, and efficiency to guide model choice.