Language Model Python Packages

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

langsmith

LangSmith Client SDK Implementations

swebench

SWE-bench: Can Language Models Resolve Real-world Github Issues?

swesmith

[NeurIPS 2025 D&B Spotlight] Scaling Data for SWE-agents

open-clip-torch

An open source implementation of CLIP.

lm-eval

A framework for few-shot evaluation of language models.

loralib

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

g4f

The official gpt4free repository | various collection of powerful language models | opus 4.6 gpt 5.3 kimi 2.5 deepseek v3.2 gemini 3

esperanto

A unified interface for various AI model providers

nucliadb-utils

NucliaDB, The AI Search database for RAG

nucliadb-protos

NucliaDB, The AI Search database for RAG

nucliadb-telemetry

NucliaDB, The AI Search database for RAG

nucliadb-models

NucliaDB, The AI Search database for RAG

nucliadb-sdk

NucliaDB, The AI Search database for RAG

instructorembedding

[ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings

nucliadb-dataset

NucliaDB, The AI Search database for RAG

nucliadb

NucliaDB, The AI Search database for RAG

mblm

A hierarchical LM that scales to training on context windows of +5M tokens

optimum-rbln

⚡ A seamless integration of HuggingFace Transformers & Diffusers with RBLN SDK for efficient inference on RBLN NPUs.

pylate

Late Interaction Models Training & Retrieval

nidx-protos

NucliaDB, The AI Search database for RAG

txtai

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

nidx-binding

NucliaDB, The AI Search database for RAG

hypernix

Ai training toolkit