Large Language Models Python Packages

ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

xgrammar

Fast, Flexible and Portable Structured Generation

onnx-ir

Efficient in-memory representation for ONNX, in Python

redisvl

Redis Vector Library. The AI-native Python client for Redis.

fla-core

🚀 Efficient implementations for emerging model architectures

flash-linear-attention

🚀 Efficient implementations for emerging model architectures

gliner

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts)

chronos-forecasting

Chronos: Pretrained Models for Time Series Forecasting

haystack-ai

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

megatron-core

Ongoing research training transformer models at scale

langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

llm-sandbox

Lightweight and portable LLM sandbox runtime (code interpreter) Python library.

reasoning-gym

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

tokencost

Easy token price estimates for 400+ LLMs. TokenOps.

agentscope

Build and run agents you can see, understand and trust.

lightrag-hku

[EMNLP2025] LightRAG: Simple and Fast Retrieval-Augmented Generation

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

llm-guard

The Security Toolkit for LLM Interactions

adaptive-classifier

A flexible, adaptive classification system for dynamic text classification

evalplus

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

langflow-base

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

camel-ai

🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

langflow

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

multilspy

multilspy is a lsp client library in Python intended to be used to build applications around language servers.