mteb

Monthly downloads

698.4K

Weekly downloads

174.2K

GitHub stars

3.5K

Downloads in the 30 and 7 days to October 10, 2026. Refreshed weekly.

Package Information

Latest Version

2.26.2

Requires Python

<3.15,>=3.11

Licence

Apache-2.0

Last Release

3 hr ago

First Release

June 30, 2022

Total Releases

819

No reported malware or high/critical advisories · Details

The current version has no OSV.dev malware reports and no critical or high-severity vulnerabilities.

Package Insights

What's this?

Downloads by OS

Linux 18.6K

Windows 46

macOS 31

Python Versions

3.13

71.4% 13.3K

3.12

18.5% 3.4K

3.10

6.7% 1.3K

3.11

2.9% 544

3.14

0.5% 89

Top Countries

13.7K

3.7K

886

325

319

264

259

192

168

10.

164

Weekly Downloads

Last 7 days

174,214

Last 30 days

698,398

Peak week (to 2026-10-10)

174,214

2026-09-25 – 2026-10-10. Enable JavaScript for the interactive chart.

GitHub Stars

Current

3,454

Earliest (2026-09-26)

3,436

Change

+18

2026-09-26 – 2026-10-11. Enable JavaScript for the interactive chart.

Dependencies

datasets >=2.19.0
numpy <3.0.0,>=1.23.0
polars >=1.0.0
pydantic >=2.11.0
pytrec-eval-terrier >=0.5.6
requests >=2.26.0
rich >=13.0.0
scikit-learn >=1.4.0
scipy >=1.14.0
sentence-transformers >=3.0.0
torch >=2.0.0
tqdm >1.0.0
transformers <6.0.0,>=4.40.0
typing-extensions >=4.5.0

165 optional dependencies

accelerate[llama-nemotron-embed-vl-1b-v2]
extra == "llama-nemotron-embed-vl-1b-v2"
accelerate>=1.1.0[wemm]
extra == "wemm"
accelerate>=0.26.0[visrag-ret]
extra == "visrag-ret"
accelerate>=0.26.0[fusion-embedding]
extra == "fusion-embedding"
accelerate[nemotron-colembed-vl-v2]
extra == "nemotron-colembed-vl-v2"
accelerate[llama-nemotron-colembed-vl]
extra == "llama-nemotron-colembed-vl"
bm25s>=0.2.6[bm25s]
extra == "bm25s"
boto3>=1.34.0[boto3]
extra == "boto3"
cachetools>=5.2.0[leaderboard]
extra == "leaderboard"
codecarbon<3.0.0,>=2.0.0[codecarbon]
extra == "codecarbon"
cohere==5.14.0[cohere]
extra == "cohere"
colpali-engine>=0.3.12[colpali-engine]
extra == "colpali-engine"
colpali-engine>=0.3.18[evie]
extra == "evie"
datasets[audio]
extra == "audio"
datasets<5.0.0,>=3.3.0[languagebind]
extra == "languagebind"
datasets>=4.0.0[video]
extra == "video"
einops>=0.8.0[jina]
extra == "jina"
einops>=0.8.0[jina-clip]
extra == "jina-clip"
einops>=0.8.1[nomic]
extra == "nomic"
einops>=0.8.0[radio]
extra == "radio"
einops>=0.8.0[cosmos-embed1]
extra == "cosmos-embed1"
einops>=0.8.0[dasheng]
extra == "dasheng"
einops>=0.8.0[internvideo2]
extra == "internvideo2"
faiss-cpu>=1.12.0[faiss-cpu]
extra == "faiss-cpu"
fastapi>=0.110[api]
extra == "api"
flagembedding==1.3.4[flagembedding]
extra == "flagembedding"
flash-attn>=2.6.3[nemotron-colembed-vl-v2]
extra == "nemotron-colembed-vl-v2"
flash-attn>=2.6.3[flash-attention]
extra == "flash-attention"
flash-attn>=2.6.3[llama-nemotron-colembed-vl]
extra == "llama-nemotron-colembed-vl"
flash-attn>=2.6.3[llama-nemotron-embed-vl-1b-v2]
extra == "llama-nemotron-embed-vl-1b-v2"
flash-linear-attention>=0.5.1[topk-embed]
extra == "topk-embed"
google-genai>=1.7.0[google-genai]
extra == "google-genai"
gradio==6.0.1[leaderboard]
extra == "leaderboard"
gritlm>=1.0.2[gritlm]
extra == "gritlm"
huggingface-hub>=0.32.0[xet]
extra == "xet"
imagebind-unofficial>=0.1.0[imagebind]
extra == "imagebind"
kernels<0.15,>=0.14.1[topk-embed]
extra == "topk-embed"
languagebind==0.1.0[languagebind]
extra == "languagebind"
librosa>=0.10.0[languagebind]
extra == "languagebind"
librosa>=0.10.0[fusion-embedding]
extra == "fusion-embedding"
llm2vec<0.3.0,>=0.2.3[llm2vec]
extra == "llm2vec"
matplotlib>=3.9.4[leaderboard]
extra == "leaderboard"
model2vec>=0.3.0[model2vec]
extra == "model2vec"
msclap>=1.3.4[msclap]
extra == "msclap"
muq==0.1.0[muq]
extra == "muq"
octen>=0.4.0[octen]
extra == "octen"
open-clip-torch>=2.24.0[internvideo2]
extra == "internvideo2"
open-clip-torch==2.31.0[open-clip-torch]
extra == "open-clip-torch"
openai>=1.41.0[openai]
extra == "openai"
opentelemetry-exporter-otlp-proto-http>=1.27[api]
extra == "api"
opentelemetry-instrumentation-fastapi>=0.48b0[api]
extra == "api"
opentelemetry-sdk>=1.27[api]
extra == "api"
peft>=0.15.2[jina-v4]
extra == "jina-v4"
peft>=0.11.0[peft]
extra == "peft"
peft>=0.11.0[omniretriever]
extra == "omniretriever"
peft>=0.15.2[jina-clip]
extra == "jina-clip"
peft>=0.13.0[internvideo2]
extra == "internvideo2"
pillow>=10.0.1[image]
extra == "image"
pillow>=10.0.1[fusion-embedding]
extra == "fusion-embedding"
playwright>=1.49.0[og]
extra == "og"
plotly<6.0.0,>=5.24.0[leaderboard]
extra == "leaderboard"
prometheus-client>=0.20[api]
extra == "api"
protobuf>=3.0.0[internvideo2]
extra == "internvideo2"
protobuf>=3.0.0[siglip]
extra == "siglip"
protobuf>=3.0.0[tipsv2]
extra == "tipsv2"
ps3-torch>=0.1.3[ps3]
extra == "ps3"
pydantic-settings>=2.0[api]
extra == "api"
pygithub>=1.60[github]
extra == "github"
pylate>=1.3.1[pylate]
extra == "pylate"
pymongo>=4.15.5[leaderboard]
extra == "leaderboard"
pystemmer>=2.2.0.3[bm25s]
extra == "bm25s"
qwen-omni-utils[qwen-omni-utils]
extra == "qwen-omni-utils"
qwen-vl-utils>=0.0.14[wemm]
extra == "wemm"
qwen-vl-utils>=0.0.14[eager-embed]
extra == "eager-embed"
qwen-vl-utils>=0.0.14[qwen-vl]
extra == "qwen-vl"
salesforce-lavis>=1.0.2[blip2]
extra == "blip2"
sauerkrautlm-colpali>=0.1.0[sauerkrautlm-colpali]
python_full_version < "3.14" and extra == "sauerkrautlm-colpali"
sentence-transformers>=5.4.1[nemotron-3-embed]
extra == "nemotron-3-embed"
sentence-transformers>=6.0.0[colnanovdr]
extra == "colnanovdr"
sentence-transformers<5.4.0,>=3.1.0[jina]
extra == "jina"
sentence-transformers>=5.4.0[multimodal-sbert]
extra == "multimodal-sbert"
sentence-transformers>=6.0.0[pplx-embed-v2]
extra == "pplx-embed-v2"
sentence-transformers>=5.0.0[sparse-encoder]
extra == "sparse-encoder"
sentence-transformers>=6.0.1[topk-embed]
extra == "topk-embed"
sentence-transformers>=5.7.0[wemm]
extra == "wemm"
sentencepiece>=0.2.0[siglip]
extra == "siglip"
sentencepiece>=0.2.0[internvideo2]
extra == "internvideo2"
sentencepiece>=0.1.99[visrag-ret]
extra == "visrag-ret"
sentencepiece>=0.2.0[tipsv2]
extra == "tipsv2"
soundfile>=0.13.1[msclap]
extra == "msclap"
soundfile>=0.13.1[imagebind]
extra == "imagebind"
soundfile>=0.13.1[fusion-embedding]
extra == "fusion-embedding"
speechbrain>=0.5.12[speechbrain]
extra == "speechbrain"
tenacity>9.0.0[voyage-v]
extra == "voyage-v"
tencentcloud-sdk-python-common>=3.0.1454[youtu]
extra == "youtu"
tencentcloud-sdk-python-lkeap>=3.0.1451[youtu]
extra == "youtu"
tiktoken>=0.8.0[ark]
extra == "ark"
tiktoken>=0.8.0[openai]
extra == "openai"
timm==1.0.15[ps3]
extra == "ps3"
timm<1.1.0,>=1.0.15[radio]
extra == "radio"
timm<1.1.0,>=1.0.15[timm]
extra == "timm"
timm<1.1.0,>=1.0.15[visrag-ret]
extra == "visrag-ret"
timm<1.1.0,>=1.0.15[jina-clip]
extra == "jina-clip"
timm>=1.0.9[internvideo2]
extra == "internvideo2"
torch-vggish-yamnet==0.2.1[torch-vggish-yamnet]
extra == "torch-vggish-yamnet"
torchaudio>=2.0.0[internvideo2]
extra == "internvideo2"
torchaudio[audio]
extra == "audio"
torchcodec[video]
extra == "video"
torchcodec>=0.1.0[languagebind]
sys_platform == "linux" and extra == "languagebind"
torchvision>=0.22.1[jina-v4]
extra == "jina-v4"
torchvision>=0.22.0[llama-nemotron-colembed-vl]
extra == "llama-nemotron-colembed-vl"
torchvision>=0.22.0[llama-nemotron-embed-vl-1b-v2]
extra == "llama-nemotron-embed-vl-1b-v2"
torchvision>=0.22.0[nemotron-colembed-vl-v2]
extra == "nemotron-colembed-vl-v2"
torchvision>=0.22.1[ps3]
extra == "ps3"
torchvision>=0.16.0[visrag-ret]
extra == "visrag-ret"
torchvision>=0.16.0[cosmos-embed1]
extra == "cosmos-embed1"
torchvision>=0.22.1[colqwen3]
extra == "colqwen3"
torchvision>=0.22.1[evie]
extra == "evie"
torchvision>=0.22.1[fusion-embedding]
extra == "fusion-embedding"
torchvision>0.2.1[image]
extra == "image"
torchvision>=0.22.1[jina-clip]
extra == "jina-clip"
transformers<4.52.0,>=4.51.0[omniretriever]
extra == "omniretriever"
transformers<5.0.0[arctic-embed-m-v2]
extra == "arctic-embed-m-v2"
transformers>=5.0.0[colnanovdr]
extra == "colnanovdr"
transformers>=4.57.0[colqwen3]
extra == "colqwen3"
transformers<6.0.0,>=5.14.1[colvec1-1]
extra == "colvec1-1"
transformers<5[cosmos-embed1]
extra == "cosmos-embed1"
transformers<=4.44.2[e5-v]
extra == "e5-v"
transformers>=4.56.0[embeddinggemma]
extra == "embeddinggemma"
transformers<6.0.0,>=5.13.1[evie]
extra == "evie"
transformers<4.52.0[gme]
extra == "gme"
transformers[image]
extra == "image"
transformers<5[internvideo2]
extra == "internvideo2"
transformers<5.0.0[jina]
extra == "jina"
transformers<5.0.0,>=4.52.0[jina-clip]
extra == "jina-clip"
transformers<5.0.0,>=4.52.0[jina-v4]
extra == "jina-v4"
transformers<4.46,>=4.40.0[languagebind]
extra == "languagebind"
transformers==4.51.0[llama-embed-nemotron]
extra == "llama-embed-nemotron"
transformers==4.49.0[llama-nemotron-colembed-vl]
extra == "llama-nemotron-colembed-vl"
transformers>=4.56.0[llama-nemotron-embed-vl-1b-v2]
extra == "llama-nemotron-embed-vl-1b-v2"
transformers<4.53,>=4.49.0[moca]
extra == "moca"
transformers>=4.48.0[modernbert]
extra == "modernbert"
transformers>=5.3.0[multimodal-sbert]
extra == "multimodal-sbert"
transformers>=5.2.0[nemotron-3-embed]
extra == "nemotron-3-embed"
transformers==5.0.0[nemotron-colembed-vl-v2]
extra == "nemotron-colembed-vl-v2"
transformers==4.47.1[nemotron-rerank]
extra == "nemotron-rerank"
transformers==4.42.4[nvembed]
extra == "nvembed"
transformers>=5.4.0[pplx-embed-v2]
extra == "pplx-embed-v2"
transformers>=4.57.0[qwen-vl]
extra == "qwen-vl"
transformers<5.0.0,>=4.57.0[radio]
extra == "radio"
transformers<5.13.0,>=5.9.0[topk-embed]
extra == "topk-embed"
transformers<5[transformers-v4]
extra == "transformers-v4"
transformers>=5[transformers-v5]
extra == "transformers-v5"
transformers<6.0.0,>=5.4.0[uembed]
extra == "uembed"
transformers<4.53,>=4.49.0[visrag-ret]
extra == "visrag-ret"
transformers>=5.2.0[wemm]
extra == "wemm"
uvicorn>=0.27[api]
extra == "api"
vertexai==1.71.1[vertexai]
extra == "vertexai"
vllm>=0.11.1[vllm]
extra == "vllm"
volcengine-python-sdk==3.0.2[ark]
extra == "ark"
voyageai<2.0.0,>0.3.0[voyage-v]
extra == "voyage-v"
voyageai<2.0.0,>0.3.0[voyageai]
extra == "voyageai"
wav2clip==0.1.0[wav2clip]
extra == "wav2clip"
xformers>=0.0.29[arctic-embed-m-v2]
extra == "arctic-embed-m-v2"
xformers>=0.0.29[xformers]
extra == "xformers"

Used By

gritlm 2.3K/mo
seb 2.2K/mo
byaldi 1.5K/mo
autorag-research 1.2K/mo
vidore-benchmark 551/mo
llm2vec-gen 291/mo
said-lam 281/mo
ullme 235/mo
said-lam-gpu 162/mo
post-instruct 71/mo

View all 14 dependents

Repository details

Forks

719

Issues

281

Last Commit

6 days ago

Contributors

389

Open PRs

Archived

GitHub Topics

Benchmark Clustering Information Retrieval Sentence Transformers Sts Text Embedding Retrieval Neural Search Semantic Search Sbert Text Classification Bitext Mining Reranking Multilingual Nlp Mteb Low Resource Nlp Multimodal Embeddings Evaluation

Data and sources

Tracking Since

September 26, 2026

Data Updated

October 11, 2026

Data Freshness

Fresh

Keywords

deep learning text embeddings embeddings multimodal benchmark retrieval information retrieval

Sources and methodology

Package Information

Package Insights

Downloads by OS

Python Versions

Top Countries

Weekly Downloads

GitHub Stars

Dependencies

Used By

Repository details

Data and sources

Health Score

Maintenance

Category

Momentum

Python Support

License Type

Release Cadence

Contributor Risk

Additional Metrics