Benchmarks Python Packages

sqloxide

Python bindings for sqlparser-rs

528K 210 26

pytdc

Therapeutics Commons (TDC): Multimodal Foundation for Therapeutic Science

263K 1K 225

nemo-gym

Evaluate and improve models and agents using environments

160K 1K 386

optiprofiler

Benchmarking optimization solvers.

28K 16 5

openadapt-evals

Windows Agent Arena and GUI-agent evaluation for OpenAdapt. Live support surface. Install the product with pip install openadapt; this repo is the eval harness, including WAA.

6K 4 4

bohrin

Check the checker: an open standard for testing the graders of RL environments, with proof before every accusation.

4K 2 0

avalanche-lib

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

3K 2K 321

langtest

Deliver safe & effective language models

3K 561 54

autor3search

Autonomous AI-driven performance optimization for any Python repository

2K 0 0

screamingface

ScreamingFace is an open toolkit for composing model fusions: several models answering together, graded against a real research benchmark. Every run reproduces from a single line, on your own keys.

2K 15 0

ia-router

Distribute your tasks across official AI CLIs (Claude, Codex, Agy) using objective metrics from Arena and Artificial Analysis. Apache-2.0.

2K 1 0

fab-clientlib

Flatland Benchmarks (FAB) is an open-source web-based platform for running Benchmarks to foster Open Research

1K 3 4

memory-path-engine

Structured memory for agents: weighted retrieval and replayable evidence paths

1K 3 0

cpu-perf

A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.

1K 1K 87

flowserv-core

Reproducible and Reusable Data Analysis Workflow Server

1K 2 0

benchstats

Statistical testing & reporting CLI tool and Python package. Primarily for benchmark results comparison, but you can plug in any data source with ease.

1K 5 0

genome-memory

Auditable memory layer for AI agents: zero-LLM-call local ingest (~10ms/msg, air-gapped), matches Mem0 on accuracy at ~1000x lower ingest cost, bi-temporal belief-state, MCP server. Honest LoCoMo/LongMemEval benchmarks. Open source (Apache-2.0).

990 10 4