Vision Language Model Python Packages

mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

colpali-engine

The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.

lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

mlx-tune

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.

x-anylabeling-cvhub

Effortless data labeling with AI support from Segment Anything and other awesome models.

vidore-benchmark

Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.

thepipe-api

Get clean data from tricky documents, powered by VLMs.

urban-worm

Workflow of reproducible multimodal inference for urban environment evaluation.

x-anylabeling

Effortless data labeling with AI support from Segment Anything and other awesome models.

lica-gdb

GDB: GraphicDesignBench - A real-world benchmark for evaluating AI on graphic design tasks

kani-vision

Kani extension for supporting vision-language models (VLMs). Comes with model-agnostic support for GPT-Vision and LLaVA.

visionzip

Official repository for VisionZip (CVPR 2025)

llava-torch

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

mbodied

Seamlessly integrate state-of-the-art transformer models into robotics stacks

nerif

LLM powered Python

lica-gdb-helm

GDB: GraphicDesignBench - A real-world benchmark for evaluating AI on graphic design tasks

gptparse

Document parser for RAG

openscenesense-ollama

Offline video analysis using Ollama models and local Whisper

ps3-torch

Scaling Vision Pre-Training to 4K Resolution

kani-multimodal-core

Core shared libraries for multimodal Kani extensions.

kanoa

AI-powered interpretation of data science outputs with multi-backend support (Molmo, Gemini, Claude, OpenAI)

unsloth-mlx

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.

german-ocr

High-performance German document OCR - Local & Cloud with GPU/CPU support

mlx-vlm-nell

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.