Scraping Websites Python Packages

requests-html

Pythonic HTML Parsing for Humans™

outscraper

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

cfscrape

A Python module to bypass Cloudflare's anti-bot page.

webskrap

Python scraping framework built on Playwright.

scrapemm

LLM-friendly scraper for media and text from social media and the open web.

crawlbase

Fast python library for the Crawlbase API

tgcore

TgCore • Designed for complex systems. Made simple. A fluent DSL for building Telegram bots, APIs, and AI workflows.

g-pygle

A tool for searching the entire web with the Google technology

proxycrawl

ProxyCrawl Python library for scraping and crawling

hexium-browser

Stealth antidetect browser for Playwright — patched Chromium 151 with C++ persona, persistent profiles, humanize & GeoIP. launch() in, Chrome out.

scrape-search-engine

Search anything on the different Search Engine's it will collect all the links.

google-services-api

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

nintendeals

Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).

multporn

A python library used to scrape and download from Multporn.net

nicovideo-py

ニコニコ動画に投稿された動画の情報を取得するライブラリです。（動画をダウンロードすることはできません。）

aioprox

aioprox – Asynchronous proxy manager for Python. Fetch, test, and filter HTTP/SOCKS proxies with optional latency measurement and custom sources.

scrapy-proxy-headers

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

google-maps-reviews

Python SDK that allows fetching Google Maps Reviews from any place. Fetch reviews from any business on Google Maps without limitations.

fakepilot

Python package for scraping Trustpilot.

bitscrape

A modular, async Python web scraping framework — simple for a first spider, capable of distributed crawling, storage, search-ranking, and observability for everything after that.

lst

Declarative Scraping Library

indoquake

A Latest Earthquake Detection Package Taken Based on BMKG | Meteorological, Climatological, and Geophysical Agency

pnu-libgh

GitHub scraping tool and library

korean-news-crawler

Python Library for Crawling News Artircles in Korean Top 10 News Websites with Utilities