Multimodal Learning Python Packages

autobotlib

AutoBOT: Explainable AutoML for texts

immuno-compass

Generalizable AI predicts immunotherapy outcomes across cancers and treatments

multimodal-transformers

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data

navit-torch

navit - Pytorch

topicnet

Interface for easier topic modelling.

ak-ssl

A Self-Supervised Learning Library

proto-clip-toolkit

Code release for Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning | IROS 2024

pykale

Knowledge-Aware machine LEarning (KALE): accessible machine learning from multiple sources for interdisciplinary research, part of the 🔥PyTorch ecosystem. ⭐ Star to support our work!

coin-clip

Coin-CLIP: fine-tuned with a vast collection of coin images from CLIP using contrastive learning. It enhances feature extraction for coins, boosting image search accuracy. This model merges Visual Transformer (ViT) with CLIP's multimodal learning, optimized for numismatic applications.

slp

Speech, Language and Multimodal Processing models and utilities in PyTorch

autort-swarms

Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents"

pali3

pali3 - Pytorch

mmae

Package for Multimodal Autoencoders in TensorFlow / Keras

kosmosg

My implementation of the model KosmosG from "KOSMOS-G: Generating Images in Context with Multimodal Large Language Models"