Vision Language Model Python Packages

mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

colpali-engine

The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.

lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

x-anylabeling-cvhub

X-AnyLabeling: A lightweight, efficient, and unified cross-platform desktop application for annotating text, image, video, and multimodal data, combining versatile built-in tools with state-of-the-art AI models and flexible multi-format export.

thepipe-api

Get clean data from tricky documents, powered by vision-language models ⚡

optivision-rag

Visual-token compression for VLM document retrieval: 448 KB to 3.95 KB per page (113.5x) at 86.7% of baseline nDCG@5. ColPali/ColSmol, PyTorch, Qdrant.

mbodied

Seamlessly integrate state-of-the-art transformer models into robotics stacks

aicu-scanner

Black-box security scanner for LLM applications — prompt injection, safety bypass, credential leakage

mlx-tune

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.

discern-vl

Ask a vision model a question, get the answer from its logits in one forward pass. No decoding, nothing trained. The confidence is the logit gap in nats.

kanoa

AI-powered interpretation of data science outputs with multi-backend support (Molmo, Gemini, Claude, OpenAI)

glance-vlm

Ask an open vision-language model typed questions about an image and get probabilities back, on your own machine.

pdfhell

Adversarial PDFs that break AI document readers. Procedural ground truth, not LLM-as-judge.

urban-worm

Urban-WORM (Workflow Of Reproducible Multimodal Inference) is a user-friendly high-level interface that is designed for adding rich and meaningful captions for crow-sourced data with geotags using multimodal models.

pagemark

PDF to Markdown with vision-language models — bring your own OpenAI-compatible endpoint. Detects and recovers from VLM failures instead of passing them through.

mlx-serve

Local inference server for Apple Silicon — hot-swaps MLX models (LLM, vision, embeddings, TTS, STT) via OpenAI API

mlx-reason-ct

Native MLX port of NVIDIA NV-Reason-CT for 3D CT reasoning on Apple Silicon

vidore-benchmark

Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.

cv-suite

Computer vision CLI for dataset work: convert annotations, auto-label, caption, classify, OCR, generate images and clean datasets.

intern-decision-mlx

Screenshot + questions in, calibrated typed decisions out. Intern-Decision on Apple silicon: 0.9 s per 1080p screenshot on an 8 GB M2 MacBook Air, offline.

nerif

LLM-powered Python

fast-lightonocr

High-performance Rust inference engine for LightOnOCR-2-1B using ONNX Runtime.

llmog

framework for using llms on object grounding