Scraper Python Packages

selectolax

Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.

firecrawl-py

🔥 Search, scrape, and clean the web for AI agents.

newspaper3k

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

firecrawl

🔥 Search, scrape, and clean the web for AI agents.

google-play-scraper

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

crawlee

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

spider-client

Python, Javascript, and Rust libraries for the Spider Cloud API.

scrapfly-sdk

Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.

rnet

An ergonomic Python HTTP Client with TLS fingerprint

twikit

Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

zenrows

SDK to access ZenRows API directly from Python. We handle proxies rotation, headless browsers and CAPTCHAs for you.

snscrape

A social networking service scraper in Python

cinemagoer

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people, characters and companies

wreq

An ergonomic Python HTTP Client with TLS fingerprint

twscrape

2025! X / Twitter API scrapper with authorization support. Allows you to scrape search results, User's profiles (followers/following), Tweets (favoriters/retweeters) and more.

misoreports

A comprehensive Python library for downloading Midcontinent Independent System Operator (MISO) public reports into pandas dataframes.

scrapetube

A YouTube scraper for scraping channels, playlists, and searching 🔎

tiktoklive

The definitive Python library to receive livestream events (comments, gifts, etc.) in realtime from TikTok LIVE.

perplexity-webui-scraper

An advanced, high-performance Python client, MCP server, and REST API for reverse-engineering Perplexity AI's WebUI.

outscraper

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

imdbpy

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people, characters and companies

is-crawler

Crawler detection from User-Agent strings in 50 ns. Issues and pull requests welcome!

app-store-scraper

Single API ☝ App Store Review Scraper 🧹

homeharvest

Python package for scraping real estate property data