PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Scrapy Python Packages

Python packages with the GitHub topic scrapy. Sorted by relevance, with stars and monthly downloads.
scrapy
itemadapter

Common interface for data container classes

2.6M 69 13
scrapy-plugins
scrapy-playwright

🎭 Playwright integration for Scrapy

1.2M 1K 160
Luqman-Ud-Din
random-user-agent

A package to get list of user agents based on filters such as operating system, software name etc..

397K 103 12
scrapfly
scrapfly-sdk

Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.

256K 55 15
eliasdabbas
advertools

advertools - online marketing productivity and analysis tools

168K 1K 242
scrapy-plugins
scrapy-zyte-api

Zyte API integration for Scrapy

106K 41 22
hellock
icrawler

A multi-thread crawler framework with many builtin image crawlers provided.

79K 924 179
scrapy-plugins
scrapy-splash

Scrapy+Splash for JavaScript integration

78K 3K 456
jxlil
scrapy-impersonate

Scrapy download handler that can impersonate browser' TLS signatures or JA3 fingerprints.

39K 233 27
rmax
scrapy-redis

Redis-based components for Scrapy.

39K 6K 2K
clemfromspace
scrapy-selenium

Scrapy middleware to handle javascript pages using selenium

27K 952 352
alecxe
scrapy-fake-useragent

Random User-Agent middleware based on fake-useragent

26K 688 94
scrapy-plugins
scrapy-zyte-smartproxy

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

25K 365 91
scrapy-plugins
scrapy-crawlera

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

23K 365 91
TeamHG-Memex
scrapy-rotating-proxies

use multiple proxies with Scrapy

15K 774 158
ScrapingAnt
scrapingant-client

ScrapingAnt API client for Python.

10K 43 5
City-Bureau
city-scrapers-core

Core functionality for City Scrapers projects

9K 8 10
Boris-code
feapder

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单,功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度

8K 4K 543
my8100
logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

7K 92 24
shengchenyang
ayugespidertools

使 scrapy 开发不用在意 item,pipeline,middleware 等通用场景下模块的编写,解放开发者的双手。

7K 100 16
ScrapeOps
scrapeops-scrapy

Scrapy extension that gives you all the scraping monitoring, alerting, scheduling, and data validation you will need straight out of the box.

7K 38 13
orangain
scrapy-s3pipeline

Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.

6K 76 12
TikHub
tikhub

Modern Python SDK for the TikHub social-media data API.

5K 614 72
scrapingbee
scrapy-scrapingbee

JavaScript support and proxy rotation for Scrapy with ScrapingBee.

5K 152 6
    • Data from PyPI, GitHub, ClickHouse, and BigQuery