Multimodal Python Packages

pyod

A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.

rerun-sdk

Visualize, query, and stream to train on multimodal robotics data.

daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

mteb

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

mlx-audio

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

vortex-data

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

jina

☁️ Build multimodal AI applications with cloud-native stack

bentoml

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

docarray

Represent, send, store and search multimodal data

ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

a11-kit

A streaming action runtime for AI agents, model serving, and multimodal APIs

lamindb

Open-source data management for multimodal AI. Query, trace, and govern with a lineage-native, format-agnostic lakehouse for agents and teams. Supports biological formats and registries - by the creators of Scanpy. 🍊YC S22

granular

Fast dataset format and loader

lamindb-core

Open-source data management for multimodal AI. Query, trace, and govern with a lineage-native, format-agnostic lakehouse for agents and teams. Supports biological formats and registries - by the creators of Scanpy. 🍊YC S22

vlmrun-hub

A hub for various industry-specific schemas to be used with VLMs.

datachain

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

vllm-omni

A framework for efficient model inference with omni-modality models

mmcls

OpenMMLab Pre-training Toolbox and Benchmark

mmpretrain

OpenMMLab Pre-training Toolbox and Benchmark

abstractframework

Write once. Generate everything. A modular, open-source ecosystem for building durable, observable, multimodal AI systems. Text, voice, image, video, music — one unified interface, any provider, any model, local or cloud.

achatbot

An open source chat bot architecture for voice/vision (and multimodal) assistants, local(CPU/GPU bound) and remote(I/O bound) to run.

airunner

Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows