Language Identification Python Packages

lingua-language-detector

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

py3langid

Faster, modernized fork of the language identification tool langid.py

simplemma

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

split-lang

✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and budoux

eld

Fast and accurate natural language detection. Detector written in Python. Nito-ELD, ELD.

rosette-api

Babel Street Analytics Client Library for Python

fastlangid

fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)

eldc

The fastest high-accuracy natural language detector. Written in C.

glotscript

[LREC 2024] 🖋 Resource and Tool for Writing System Identification

langdetect-py

Port of Google's language-detection library to Python.

fasttext-new

Maintained drop-in replacement for the archived fastText Python package: Rust core, GIL-free batch predict, training, exact parity with C++ fastText

fastspell

Targetted language identifier, based on FastText and Hunspell.

fastspell-dictionaries

Targetted language identifier, based on FastText and Hunspell.

langaccess

Classifies the language access a website provides, separating the site's own writing from a translation widget's output, and returns the evidence behind each verdict.

textpipe

Textpipe: clean and extract metadata from text

codelang-detect

A fast, lightweight, regex-based programming language detector for Python.

polars-whichlang

Language identification plugin for polars

seqtolang

Multi-Langauge Identification

lidbox

End-to-end spoken language identification out of the box.

pylade

PyLaDe - Language Detection tool.

taglid

A word-level Language Identification (LID) tool for Tagalog-English (Taglish) text

codeswitch

CodeSwitch is a NLP tool, can use for language identification, pos tagging, name entity recognition, sentiment analysis of code mixed data.

whatlangid

This project is build on top of whatthelang and langid

afrolid

AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages.