Ggml Python Packages

gguf

LLM inference in C/C++

xinference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

transcribe-cpp-native

ggml speech-to-text inference for 16+ model families

cyllama

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

transcribe-cpp

ggml speech-to-text inference for 16+ model families

cyllama-vulkan

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-cuda12

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-rocm

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-sycl

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

stable-diffusion-cpp-python

stable-diffusion.cpp bindings for python

clip-cpp

CLIP inference in plain C/C++ with no extra dependencies

smartloop

Smartloop Command Line interface to process documents using LLM

pyllama-server

Python wrapper for llama.cpp server - OpenAI API compatible backend with auto device detection

grammatica

Grammatica is a GBNF (GGML BNF) grammar builder that exposes the full richness of the GBNF notation system using a syntax that reflects the resulting grammar.