Multimodality Python Packages

big-sleep

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

fedot

Automated modeling and machine learning framework FEDOT

fonduer

A knowledge base construction engine for richly formatted data

emmental

A deep learning framework for building multimodal multi-task learning systems.

fuse-med-ml

A python framework accelerating ML based discovery in the medical field by encouraging code reuse. Batteries included :)

tfce-mediation

Fast regression and mediation analysis of vertex or voxel MRI data with TFCE

medpalm

Towards Generalist Biomedical AI

swarms-torch

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

mirasol-pytorch

Implementation of 🌻 Mirasol, SOTA Multimodal Autoregressive model out of Google Deepmind, in Pytorch

pali-torch

Democratization of "PaLI: A Jointly-Scaled Multilingual Language-Image Model"

pydance

DANCE: a deep learning library and benchmark platform for single-cell analysis

tenderness

tenderness is a fast library for synthetic, deterministic document rendering from text and images

tmi-viewer

Fast regression and mediation analysis of vertex or voxel MRI data with TFCE

navit-torch

My implementation of "Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution"

pali3

Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"

sygra

SyGra - Graph-oriented Synthetic data generation Pipeline

fedot-example-test

Automated modeling and machine learning framework FEDOT

nmtpytorch

Sequence-to-Sequence Framework in PyTorch

mmca

The open source community's implementation of the all-new Multi-Modal Causal Attention from "DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention"

mmqqa

Experiments around using Multi-Modal Casual Attention with Multi-Grouped Query Attention

mmca-mgqa

Experiments around using Multi-Modal Casual Attention with Multi-Grouped Query Attention

mmmgqa

Experiments around using Multi-Modal Casual Attention with Multi-Grouped Query Attention