Multimodal Python Packages

pyod

A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.

mteb

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

rerun-sdk

Visualize, query, and stream to train on multimodal robotics data.

daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

mlx-audio

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

bentoml

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

vlmrun-hub

A hub for various industry-specific schemas to be used with VLMs.

vortex-data

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

docarray

Represent, send, store and search multimodal data

ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

jina

☁️ Build multimodal AI applications with cloud-native stack

torchscale

Foundation Architecture for (M)LLMs

img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

videodb

VideoDB Python SDK

datachain

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

stability-sdk

SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)

mmcls

OpenMMLab Pre-training Toolbox and Benchmark

vllm-omni

A framework for efficient model inference with omni-modality models

mmpretrain

OpenMMLab Pre-training Toolbox and Benchmark

granular

Fast dataset format and loader

lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

tsflex

Flexible time series feature extraction & processing

cybertask

An advanced AI assistant that leverages the capabilities of ChatGPT API, Gemini Pro, AutoGen, and open-source LLMs, enabling it both to engage in conversations and to execute computing tasks on local devices.