Prompt Engineering Python Packages

mlflow-skinny

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow-tracing

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

partial-json-parser

Parse partial JSON generated by LLM

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

banks

LLM prompt language based on Jinja. Banks provides tools and functions to build prompts text and chat messages from generic blueprints. It allows attaching metadata to prompts to ease their management, and versioning is first-class citizen. Banks provides ways to store prompts on disk along with their metadata.

arize-phoenix

AI Observability & Evaluation

arize-phoenix-otel

AI Observability & Evaluation

outlines

Structured Outputs

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

judgeval

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

promptflow-devkit

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-core

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-tracing

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

llm-guard

The Security Toolkit for LLM Interactions

headroom-ai

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.

promptflow-tools

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-azure

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

ell-ai

A language model programming library.

agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

langchain-decorators

syntactic sugar 🍭 for langchain

opik-optimizer

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.