Multi Modal Python Packages

modelscope

ModelScope: bring the notion of Model-as-a-Service to life.

agentscope

Build and run agents you can see, understand and trust.

docarray

Represent, send, store and search multimodal data

medmnist

[pip install medmnist] 18x Standardized Datasets for 2D and 3D Biomedical Image Classification

pyvalhalla

Open Source Routing Engine for OpenStreetMap

dalle-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

py-data-juicer

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

pyvalhalla-weekly

Open Source Routing Engine for OpenStreetMap

transfusion-pytorch

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

cn-clip

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

aicu-scanner

Black-box security scanner for LLM applications — prompt injection, safety bypass, credential leakage

brainles-preprocessing

preprocessing tools for multi-modal 3D brain imaging

byaldi

Use late-interaction multi-modal models such as ColPali in just a few lines of code.

qwen

My personal implementation of the model from "Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities", they haven't released model code yet sooo...

deepke

[EMNLP 2022] An Open Toolkit for Knowledge Graph Extraction and Construction

switch-transformers

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

mychatgpt

mychatgpt is a small and useful Python package that provides utils to create OpenAI's GPT conversational agents. This module allows users to have interactive chat with GPT models and keeps track of the chat history. Useful in Python projects as Copilot agent.

pyrhubarb

A Python framework for multi-modal document understanding with Amazon Bedrock

move-dl

MOVE (Multi-Omics Variational autoEncoder) for integrating multi-omics data and identifying cross modal associations

mm-poe

Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Models

tiny-gptv

Simple Implementation of TinyGPTV in super simple Zeta lego blocks

pyrhubarb-mcp

A Python framework for multi-modal document understanding with Amazon Bedrock

kosmosx

The Next Generation Multi-Modality Superintelligence

rt2

Democratization of RT-2 "RT-2: New model translates vision and language into action"