Crawling Framework Python Packages

ruia

Async Python 3.6+ web scraping micro-framework based on asyncio

acrawler

🔍 A powerful web-crawling framework, based on aiohttp.

exoskeleton

A library to create a bot / spider / crawler.

sneakpeek-py

Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

wsltools

Web Scan Lazy Tools - Python Package

spideyx

SpideyX - A Web Reconnaissance Penetration Testing tool for Penetration Testers and Ethical Hackers

spriterx

基于python协程池、用法灵活的高性能爬虫框架