Scrapy Extension Python Packages

ayugespidertools

使 scrapy 开发不用在意 item，pipeline，middleware 等通用场景下模块的编写，解放开发者的双手。

scrapy-wayback-machine

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

scrapy-fieldstats

A Scrapy extension to generate a summary of fields coverage from your scraped data.

spider-info-webservice

Scrapy extension for monitoring spider.

scrapy-spiderstats-extension

Scrapy Spider Stats to MongoDB Extension

scrapy-slackbot

A Scrapy extension to send notification to the Slack channel.

scrapy-folder-tree

A scrapy pipeline which stores files using folder trees.