Language Identification Python Packages

lingua-language-detector

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

fasttext-predict

fasttext with wheels and no external dependency, but only the predict method (<1MB)

py3langid

Faster, modernized fork of the language identification tool langid.py

simplemma

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

split-lang

✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and budoux

eld

Fast and accurate natural language detection. Detector written in Python. Nito-ELD, ELD.

fastlangid

fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)

rosette-api

Babel Street Analytics Client Library for Python

glotscript

[LREC 2024] 🖋 Resource and Tool for Writing System Identification

langdetect-py

Port of Google's language-detection library to Python.

fastspell

Targetted language identifier, based on FastText and Hunspell.

textpipe

textpipe: clean and extract metadata from text

fastspell-dictionaries

Targetted language identifier, based on FastText and Hunspell.

polars-whichlang

Language identification plugin for polars

lidbox

End-to-end spoken language identification out of the box.

seqtolang

Multi Langauge Documents Langauge identification

codeswitch

CodeSwitch is a NLP tool, can use for language identification, pos tagging, name entity recognition, sentiment analysis of code mixed data.

whatlangid

This project is build on top of whatthelang and langid

pylade

PyLaDe - Language Detection tool.

afrolid

AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages..

geolid

Geographically-informed language identification