Multi Modal Learning Python Packages

open-clip-torch

An open source implementation of CLIP.

x-clip

A concise but complete implementation of CLIP with various experimental improvements from recent papers

cn-clip

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

open-clip-torch-any-py3

An open source implementation of CLIP.

simple-clip

A minimal, but effective implementation of CLIP (Contrastive Language-Image Pretraining) in PyTorch

ddd-dataset

A detection/segmentation dataset with labels characterized by intricate and flexible expressions. "Described Object Detection: Liberating Object Detection with Flexible Expressions" (NeurIPS 2023).

imml

A Python package for integrating, processing, and analyzing incomplete multi-modal datasets.

chug

Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.

nevax

The open source implementation of "NeVA: NeMo Vision and Language Assistant"

elysium-knowledge-repository

Elysium Knowledge Repository is an open source initiative to embed all of Humanity's multi-modal knowledge and wisdom.