Multilingual Nlp Python Packages

mteb

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

codetext

[EMNLP 2023] The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation

low-resource-nlp-toolkit

A Python toolkit for African language pre-processing, emotion labels, evaluation, and routing.

vocabulous

Bootstrapping Language Detection from Noisy & Ambiguous Data

workrb

WorkRB: Work Research Benchmark

padie-extended

An open-source Python package that uses AI to predict Nigerian languages, including English, Pidgin, Yoruba, Hausa, and Igbo.

quinductor

A multilingual data-driven method for generating reading comprehension questions