Downloads in the 30 and 7 days to October 10, 2026. Refreshed weekly.
Package Information
Package Insights
- High downloads but no recent maintenance activity
((week_daily_avg - month_daily_avg) / month_daily_avg) * 100More insights and metrics
Downloads by OS
Python Versions
Top Countries
Weekly Downloads
GitHub Stars *
Dependencies
- anyascii >=0.3.2
- blingfire ==0.1.8
- boto3
- charset-normalizer >=3.2.0
- fasttext-wheel ==0.9.2
- fsspec >=2023.6.0
- jq
- jsonpath-ng
- msgspec >=0.14.2
- necessary >=0.4.3
- nltk >=3.9.1
- numpy <2
- omegaconf >=2.3.0
- platformdirs >=4.2.0
- python-dotenv >=0.19.0
- pyyaml
- requests
- rich
- s3fs ==2023.6.0
- smart-open >=7.0.4
- tokenizers <=0.19.1,>=0.15.0
- tqdm
- uniseg
- zstandard >=0.23.0
32 optional dependencies
-
beautifulsoup4>=4[code]extra == "code"
-
black>=22.6.0[dev]extra == "dev"
-
brotli[trafilatura]extra == "trafilatura"
-
cchardet>=2.1.7[trafilatura]python_full_version < "3.11" and extra == "trafilatura"
-
detect-secrets==1.4.0[code]extra == "code"
-
dolma[trafilatura]extra == "trafilatura"
-
dolma[all]extra == "all"
-
dolma[resiliparse]extra == "resiliparse"
-
fasttext-wheel==0.9.2[lang]extra == "lang"
-
fastwarc[warc]extra == "warc"
-
faust-cchardet>=2.1.18[trafilatura]python_full_version >= "3.11" and extra == "trafilatura"
-
flake8>=5.0[dev]extra == "dev"
-
flake8-pyi>=22.8.1[dev]extra == "dev"
-
flake8-pyproject>=1.1.0[dev]extra == "dev"
-
htmldate>=1.4.3[trafilatura]extra == "trafilatura"
-
ipdb>=0.13.0[dev]extra == "dev"
-
ipython>=8.4.0[dev]extra == "dev"
-
isort>=5.10.1[dev]extra == "dev"
-
lingua-language-detector>=2.0.0[lang]extra == "lang"
-
mypy==0.971[dev]extra == "dev"
-
py3langid>=0.2.2[trafilatura]extra == "trafilatura"
-
pycld2[lang]extra == "lang"
-
pygments[code]extra == "code"
-
pytest>=5.2[dev]extra == "dev"
-
regex[pii]extra == "pii"
-
regex[code]extra == "code"
-
resiliparse[resiliparse]extra == "resiliparse"
-
trafilatura>=1.6.1[trafilatura]extra == "trafilatura"
-
types-dateparser[dev]extra == "dev"
-
types-pyyaml[dev]extra == "dev"
-
url-normalize[warc]extra == "warc"
-
w3lib[warc]extra == "warc"