PyRank

Spiders Python Packages

Python packages with the GitHub topic spiders. Sorted by monthly downloads, with stars and forks.
scrapinghub
spidermon

Scrapy Extension for monitoring spiders execution.

25K 562 100
proxymesh
scrapy-proxy-headers

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

333 4 0
Tyrone-Zhao
crawlerutils

Utils for programming web crawler

189 11 2
x1ah
cumtbscore

日常小脚本，懒人欢乐多。

154 102 26
victormartinez
shub-cli

A CLI for dealing with the features of ScrapingHub

92 16 0
harootune
scrapy-dynamic-spiders

Dynamically generate spider subclasses based on existing spiders, including editing crawl rules and per-spider custom settings. Run crawls sequentially using crochet. Do both at the same time.

43 0 1