Language Model Python Packages

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

langsmith

LangSmith Client SDK Implementations

swebench

SWE-bench: Can Language Models Resolve Real-world Github Issues?

swesmith

[NeurIPS 2025 D&B Spotlight] Scaling Data for SWE-agents

open-clip-torch

An open source implementation of CLIP.

lm-eval

A framework for few-shot evaluation of language models.

instructorembedding

[ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings

loralib

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

g4f

The official gpt4free repository | various collection of powerful language models | opus 4.6 gpt 5.3 kimi 2.5 deepseek v3.2 gemini 3

nucliadb-telemetry

NucliaDB, The AI Search database for RAG

nucliadb-protos

NucliaDB, The AI Search database for RAG

nucliadb-utils

NucliaDB, The AI Search database for RAG

rwkv

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

nucliadb-models

NucliaDB, The AI Search database for RAG

pylate

Late Interaction Models Training & Retrieval

nucliadb-sdk

NucliaDB, The AI Search database for RAG

nucliadb-dataset

NucliaDB, The AI Search database for RAG

nucliadb

NucliaDB, The AI Search database for RAG

txtai

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

optimum-rbln

⚡ A seamless integration of HuggingFace Transformers & Diffusers with RBLN SDK for efficient inference on RBLN NPUs.

pyvhdlmodel

An abstract language model of VHDL written in Python.

nidx-protos

NucliaDB, The AI Search database for RAG

opennmt-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

asreview

Active learning for systematic reviews