Webscraping Python Packages

htmldate

Fast and robust date extraction from web pages, with Python or on the command-line

14.2M 157 38

firecrawl-py

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

5.5M 190K 10K

requests-cache

Persistent HTTP cache for python requests

4.4M 2K 166

patchright

Undetected Python version of the Playwright testing and automation library.

3.9M 2K 114

camoufox

🦊 Anti-detect browser

934K 13K 1K

scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

929K 87K 9K

cloakbrowser

Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 30/30 tests passed.

884K 32K 3K

firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

716K 190K 10K

zenrows

SDK to access Zenrows API directly from Python. We handle proxies rotation, headless browsers and CAPTCHAs for you.

237K 21 11

autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

218K 8K 831

kameleo-local-api-client

Anti-detect browser for web scraping and automation. Engine-level fingerprint masking for Chromium and Firefox. Self-hosted, Docker-ready. Integrates with Selenium, Playwright, and Puppeteer via SDKs in Python, JavaScript, and C#.

161K 212 30

libzim

Libzim binding for Python: read/write ZIM files in Python

49K 110 29

invisible-playwright

Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.

45K 3K 302

stealth-requests

Undetected web-scraping & seamless HTML parsing in Python!

44K 567 57

gpt-researcher

An autonomous agent that conducts deep research on any data using any LLM providers

42K 30K 4K

homeharvest

Python package for scraping real estate property data

25K 751 175

clearcote

Open-source stealth Chromium with engine-level fingerprint spoofing - de-Googled, drop-in Playwright, fully buildable and verifiable from source.

17K 196 28

shardx

Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ProxyShard team. Engine-level fingerprint spoofing in Chromium 152 (WebGL / WebGPU / Client Hints / fonts / TLS), 220+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.

16K 1K 138

hyper-sdk

Python SDK for Bot Protection Bypass - Automate Akamai, Incapsula, Kasada, and DataDome. No browsers required. Solve challenges and generate valid sensors/cookies via API.

15K 70 4