Language Identification Python Packages

lingua-language-detector

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

fasttext-predict

fasttext with wheels and no external dependency, but only the predict method (<1MB)

py3langid

Faster, modernized fork of the language identification tool langid.py

simplemma

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

split-lang

✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and budoux

eldc

The fastest high-accuracy natural language detector. Written in C.

glotscript

[LREC 2024] 🖋 Resource and Tool for Writing System Identification

eld

Fast and accurate natural language detection. Detector written in Python. Nito-ELD, ELD.

fastlangid

fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)

rosette-api

Babel Street Analytics Client Library for Python

langdetect-py

Port of Google's language-detection library to Python.

fastspell

Targetted language identifier, based on FastText and Hunspell.

textpipe

textpipe: clean and extract metadata from text

fastspell-dictionaries

Targetted language identifier, based on FastText and Hunspell.

polars-whichlang

Language identification plugin for polars

lidbox

End-to-end spoken language identification (LID) on TensorFlow

seqtolang

Multi Langauge Documents Langauge identification

whatlangid

Lightning Fast Language Prediction powered by FastText and langid.

codeswitch

CodeSwitch is a NLP tool, can use for language identification, pos tagging, name entity recognition, sentiment analysis of code mixed data.

pylade

PyLaDe - Language Detection tool.

geolid

Geographically-informed language identification

afrolid

AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages..