Multimodal Learning Python Packages

autobotlib

An autoML for explainable text classification.

multimodal-transformers

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data

proto-clip-toolkit

Code release for Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning | IROS 2024

navit-torch

My implementation of "Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution"

immuno-compass

Generalizable AI predicts immunotherapy outcomes across cancers and treatments

ak-ssl

A Self-Supervised Learning Library

coin-clip

Coin-CLIP: fine-tuned with a vast collection of coin images from CLIP using contrastive learning. It enhances feature extraction for coins, boosting image search accuracy. This model merges Visual Transformer (ViT) with CLIP's multimodal learning, optimized for numismatic applications.

pykale

Knowledge-Aware machine LEarning (KALE): accessible machine learning from multiple sources for interdisciplinary research, part of the 🔥PyTorch ecosystem. ⭐ Star to support our work!

topicnet

Interface for easier topic modelling.

autort-swarms

Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents"

slp

Utils and modules for Speech Language and Multimodal processing using pytorch and pytorch lightning

pali3

pali3 - Pytorch

mmae

Package for Multimodal Autoencoders in TensorFlow / Keras

kosmosg

kosmosg - Pytorch