News Crawler Python Packages

trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

news-please

news-please - an integrated web crawler and information extractor for news that just works

fundus

A very simple news crawler with a funny name

argos-scout

Argos: Tech Scout - AI 기술 동향 자동 추적 슬랙봇

koreanewscrawler

Crawl the korean news

webcorpus

Generate large textual corpora for almost any language by crawling the web

shutterstock-analysis

The internet's very first python package supports analyzing the Shutterstock public data, which helps creators optimize their creative portfolio and earn more income with less effort.

ur-gadget

A Python package that helps capture news updates from top Vietnamese news sites

newshound

This Python package can be used to systematically extract multiple data elements (e.g., title, keywords, text) from news sources around the world in over 50 languages.

vnnews

A Python package that helps capture news updates from top Vietnamese news sites