Multimodal Python Packages

mteb

MTEB: Massive Text Embedding Benchmark

pyod

A Python library for anomaly detection across tabular, time series, graph, text, and image data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.

rerun-sdk

Visualize, query, and stream to train on multimodal robotics data.

daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

vortex-data

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

vlmrun-hub

A hub for various industry-specific schemas to be used with VLMs.

bentoml

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

mlx-audio

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-R1, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

docarray

Represent, send, store and search multimodal data

jina

☁️ Build multimodal AI applications with cloud-native stack

torchscale

Foundation Architecture for (M)LLMs

img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

datachain

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

vllm-omni

A framework for efficient model inference with omni-modality models

mmcls

OpenMMLab Pre-training Toolbox and Benchmark

stability-sdk

SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)

weblinx

WebLINX is a benchmark for building web navigation agents with conversational capabilities

mmpretrain

OpenMMLab Pre-training Toolbox and Benchmark

achatbot

An open source chat bot architecture for voice/vision (and multimodal) assistants, local(CPU/GPU bound) and remote(I/O bound) to run.

metaxy

Pluggable sample-level metadata versioning for incremental multimodal pipelines.

tsflex

Flexible time series feature extraction & processing

videodb

VideoDB Python SDK