Scraper Python Packages

firecrawl-py

The API to search, scrape, and interact with the web at scale. 🔥

selectolax

Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.

rnet

An ergonomic, censorship-resistant Python HTTP Client

firecrawl

The API to search, scrape, and interact with the web at scale. 🔥

newspaper3k

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

crawlee

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

google-play-scraper

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

spider-client

Python, Javascript, and Rust libraries for the Spider Cloud API.

tiktoklive

The definitive Python library to receive livestream events (comments, gifts, etc.) in realtime from TikTok LIVE.

snscrape

A social networking service scraper in Python

scrapfly-sdk

Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.

zenrows

SDK to access ZenRows API directly from Python. We handle proxies rotation, headless browsers and CAPTCHAs for you.

twikit

Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

cinemagoer

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people and characters

twscrape

Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.

scweet

Scrape tweets, profiles, followers and following from Twitter/X, no API key needed. Python library with smart multi-account pooling, proxy support and async.

wreq

An ergonomic, censorship-resistant Python HTTP Client

scrapetube

A YouTube scraper for scraping channels, playlists, and searching 🔎

outscraper

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

misoreports

A comprehensive Python library for downloading Midcontinent Independent System Operator (MISO) public reports into pandas dataframes.

perplexity-webui-scraper

An advanced, high-performance Python client, MCP server, and REST API for reverse-engineering Perplexity AI's WebUI.

imdbpy

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people and characters

app-store-scraper

Single API ☝ App Store Review Scraper 🧹

homeharvest

Python package for scraping real estate property data