Large Language Models Python Packages

ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

xgrammar

Fast, Flexible and Portable Structured Generation

onnx-ir

Efficient in-memory representation for ONNX, in Python

redisvl

Redis Vector Library (RedisVL) -- the AI-native Python client for Redis.

haystack-ai

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

chronos-forecasting

Chronos: Pretrained Models for Time Series Forecasting

gliner

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts)

llm-sandbox

Lightweight and portable LLM sandbox runtime (code interpreter) Python library.

fla-core

🚀 Efficient implementations for emerging model architectures

tokencost

Easy token price estimates for 400+ LLMs. TokenOps.

flash-linear-attention

🚀 Efficient implementations for emerging model architectures

llm-guard

The Security Toolkit for LLM Interactions

langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

agentscope

Build and run agents you can see, understand and trust.

llamafactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

lightrag-hku

[EMNLP2025] "LightRAG: Simple and Fast Retrieval-Augmented Generation"

megatron-core

Ongoing research training transformer models at scale

camel-ai

🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

reasoning-gym

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

langflow

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

langflow-base

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

textgrad

TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients. Published in Nature.

tensorzero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.