Web Scraper Python Packages

firecrawl-py

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

tls-client-python

High-performance Python binding for bogdanfinn/tls-client via CFFI – zero-copy, panic-proof, full TLS fingerprint control

crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

cbbpy

A Python-based web scraper for NCAA basketball.

lightnovel-crawler

Generate and download e-books from online sources.

basketball-reference-web-scraper

NBA Stats API via Basketball Reference

docpull

Local-first Python CLI and MCP server for versioned, cited context from web and document sources.

hexium-browser

Stealth antidetect browser for Playwright — patched Chromium 151 with C++ persona, persistent profiles, humanize & GeoIP. launch() in, Chrome out.

wre-client-akamai

Headless Akamai & Kasada solver that runs without a browser, and the reversing toolkit it was built with

pywebscrapr

An open-source Python web scraping tool. Supports both image scraping and text scraping.

fetchv2-mcp-server

A robust MCP server for fetching and extracting web content using Trafilatura. Optimized for AI agents with clean markdown output.

vigorish

MLB data web scraper, Python CLI application

facebook-page-scraper

Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV

cbse-results-scraper

Parinaam is a command-line Python application that scrapes and downloads student reports of CBSE Class XII Results from http://cbseresults.nic.in.

gli99

Web scraper for gifcities.org

strava-cz

High level API pro interakci s webovou aplikaci Strava.cz udelane v Pythonu

senpwai

A less annoying way to download anime

sabrmetrics

A library of web-scraping software for popular SABRmetrics websites.

markdown-crawler

A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG

wre-client-example

Headless Akamai & Kasada solver that runs without a browser, and the reversing toolkit it was built with

wre-client-kasada

Headless Akamai & Kasada solver that runs without a browser, and the reversing toolkit it was built with

linktrace

Async web crawler with rate limiting, robots.txt support, and broken link tracking