Llama Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

strands-agents-tools

A set of tools that gives agents powerful capabilities.

torchao

PyTorch native quantization and sparsity for training and inference

unsloth

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

unsloth-zoo

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

aider-chat

aider is AI pair programming in your terminal

llamafactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components

liger-kernel

Efficient Triton Kernels for LLM Training

gptcache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

strands-agents-builder

An example agent demonstrating streaming, tool use, and interactivity from your terminal. This agent builder can help you to build your own agents and tools.

ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

transformerlab

The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.

tensorzero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

liger-kernel-nightly

Efficient Triton Kernels for LLM Training

astrbot

AI Agent Assistant & development framework that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw alternative. ✨

paddlenlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

fuzy-jon

Fuzzy-JSON is a compact Python package with no dependencies, designed to address the pesky JSONDecodeError that sometimes occurs when utilizing OpenAI's powerful call function.

xinference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

strands-agents-mcp-server

This MCP server provides documentation about Strands Agents to your GenAI tools, so you can use your favorite AI coding assistant to vibe-code Strands Agents.