Crawl Python Packages

tavily-python

The Tavily Python SDK allows for easy interaction with the Tavily API, offering the full range of our search, extract, crawl, map, and research functionalities directly from your Python programs. Easily integrate smart search, content extraction, and research capabilities into your applications, harnessing Tavily's powerful features.

xhs

基于小红书 Web 端进行的请求封装。https://reajason.github.io/xhs/

air-web

A lightweight package for crawling the web with the minimalist of code.

nudecrawler

Crawl telegra.ph searching for nudes!

sitecrawl

Simple Python module to crawl a website and extract URLs

webclaw

Official Python SDK for the Webclaw web extraction API

crawldiff

git log for any website. Track what changed on any site with AI-powered summaries.

crawlist

A universal solution for web crawling lists

renfepy

🚂 Train search CLI program and library written in Python

font-obfuscator

No description available

preparser

a slight preparser to help parse webpage content or get request from urls,which supports win, mac and unix.

fcrawler

Python application that can be used to copy files of a given file type from a folder directory.

universal-utils

python utils

xhs-fork

基于小红书 Web 端进行的请求封装。https://reajason.github.io/xhs/

spideyx

SpideyX a multipurpose Web Penetration Testing tool with asynchronous concurrent performance with multiple mode and configurations.