PyRank

Crawling Python Python Packages

Python packages with the GitHub topic crawling-python. Sorted by monthly downloads, with stars and forks.
D4Vinci
scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

929K 87K 9K
thewebscraping
wrapper-tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

88K 161 12
Galarzaa90
tibia-py

API to parse tibia.com content into python objects.

2K 40 16
wael-sudo2
facebook-page-info-scraper

Free Facebook pages MetaData Scraping Library - Unlimited Calls

507 68 8
WwwwwyDev
crawlipt

The script for selenium in python. Make automated testing easier! 使用json脚本驱动selenium

333 133 2
proxymesh
scrapy-proxy-headers

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

333 4 0
helviojunior
filecrawler

File Crawler index files and search hard-coded credentials

267 38 9
WwwwwyDev
crawlist

A universal solution for web crawling lists. 抓取网页列表的通用解决方案

267 103 1
D4Vinci
iflow-mcp-d4vinci-scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

236 87K 9K
spicyparrot
kafka-scrapy-connect

A custom library that integrates Scrapy with Kafka.

218 12 1
esequiel378
proxy-randomizer

This library helps you sfetly crawle apis and web pages

191 8 1
arrester
endabyss

Recon Crawling Endpoint and Parameter Scan Tool

147 2 0
hpcao299
mcqueen

A Collection Of Information-Gathering Tools 🌐

136 2 0
elecbrandy
pw-simple-scraper

simple web scraper using by playwright

131 0 0
sergioburdisso
solidscraper

Easy to use JQuery-Like API for Web Scraping/Crawling.

126 3 0
mike-gee
webtranspose

Web scraping API for building AI applications.

123 41 2
Musubi-ai
musubi-scrape

Musubi: A convenient crawling tool for collecting web text data in Python.

117 1 0
opsysdebug
newscrap

NewsCrap adalah alat scraping berita Google berbasis Command Line Interface (CLI) yang dirancang untuk riset, investigasi, dan pengumpulan data OSINT. Dengan fitur canggih seperti rotation proxy, scheduling otomatis, dan multi-format export, alat ini memudahkan pengumpulan data berita secara efisien dan andal.

55 57 13
D4Vinci
scrapling-zhng

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

50 87K 9K