Large Language Models Python Packages

ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

xgrammar

Fast, Flexible and Portable Structured Generation

onnx-ir

Efficient in-memory representation for ONNX, in Python

redisvl

Redis Vector Library (RedisVL) -- the AI-native Python client for Redis.

haystack-ai

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

llamafactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

gliner

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts)

llm-sandbox

Lightweight and portable LLM sandbox runtime (code interpreter) Python library.

fla-core

🚀 Efficient implementations for emerging model architectures

flash-linear-attention

🚀 Efficient implementations for emerging model architectures

chronos-forecasting

Chronos: Pretrained Models for Time Series Forecasting

langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

tokencost

Easy token price estimates for 400+ LLMs. TokenOps.

lightrag-hku

[EMNLP2025] "LightRAG: Simple and Fast Retrieval-Augmented Generation"

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

reasoning-gym

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

llm-guard

The Security Toolkit for LLM Interactions

megatron-core

Ongoing research training transformer models at scale

agentscope

Build and run agents you can see, understand and trust.

langflow-base

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

langflow

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

megatron-fsdp

Ongoing research training transformer models at scale

textgrad

TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients. Published in Nature.

agentdojo

A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents.