Natural Language Processing Python Packages

huggingface-hub

The official Python client for the Hugging Face Hub.

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

datasets

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

nltk

NLTK Source

sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

thinc

🔮 A refreshing functional take on deep learning, compatible with your favorite libraries

spacy

💫 Industrial-strength Natural Language Processing (NLP) in Python

spacy-loggers

📟 Logging utilities for spaCy

htmldate

Fast and robust date extraction from web pages, with Python or on the command-line

usaddress

:us: a python library for parsing unstructured United States address strings into address components

unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

gensim

Topic Modelling for Humans

textblob

Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

lingua-language-detector

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

autogluon-tabular

Fast and Accurate ML in 3 Lines of Code

autogluon-core

Fast and Accurate ML in 3 Lines of Code

pythainlp

Thai natural language processing in Python

autogluon-features

Fast and Accurate ML in 3 Lines of Code

spark-nlp

State of the Art Natural Language Processing

autogluon-common

Fast and Accurate ML in 3 Lines of Code

flaml

A fast library for AutoML and tuning. Join our Discord: https://discord.gg/Cppx2vSPVP.

stanza

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages