Scrapyd Python Packages

logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

scrapeops-scrapy

Scrapy extension that gives you all the scraping monitoring, alerting, scheduling, and data validation you will need straight out of the box.

aio-scrapy

A high-level Web Crawling and Web Scraping framework based on Asyncio

scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:

gerapy

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

spider-admin-pro

spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具，SpiderAdmin的升级版

spiderkeeper

admin ui for scrapy/open source scrapinghub

spidermanager

admin ui for scrapy/open source scrapinghub

scrapyduler

Scrapyd launcher module that schedules scrapy spiders by time

spider-info-webservice

Scrapy extension for monitoring spider.

scrapyd-heroku

Wrapper for running Scrapyd in Heroku or locally as a service

scrapyq

Scrapyd queue management using Redis.

scrapyc

Simple client to scrapyd. Done right.

gerapy-team

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

scrapyd-egg-checksum

Get the checksum of eggs in case of building distributed scrapy clusters

scrapyd-dash

A dashboard for scrapy project using scrapyd and logparser

scrapyd-mongodb

Scrapyd Queue Management with MongoDB

scrapyd-ui

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:

scrapymon

Simple Web UI for Scrapy spider management via Scrapyd