Content Extraction Python Packages

argus-search

Retrieval platform for AI agents — 14 search providers, 12-step content extraction, dead URL recovery, site capture, and local corpus storage. One API.

article-extractor

Pure-Python article extraction library and HTTP API - Extract clean content from web pages as Markdown or HTML

extractnet

A fork of Dragnet that also extract author, headline, date, keywords from context, as well as built in metadata extraction all in one package

llamaindex-agent-toolbox

LlamaIndex integration for Agent Toolbox API — 13 tools for AI agents: search, extract, screenshot, weather, finance, news, translate, geoip, whois, dns, pdf-extract, and qr code generation

anything2md

Convert documents to Markdown using Cloudflare Workers AI toMarkdown.

lurkers

Convenient API + CLI to fetch web content (HTML, YouTube, Twitter, RSS) for agents

tikara

The metadata and text content extractor for almost every file type.

fetchv2-mcp-server

A robust MCP server for fetching and extracting web content using Trafilatura

langchain-agent-toolbox

LangChain integration for Agent Toolbox API — 13 tools for AI agents: search, extract, screenshot, weather, finance, news, translate, geoip, whois, dns, pdf-extract, and qr code generation

jsuite

Tools for parsing and manipulating JATS XML documents.

webpeel

Fast web fetcher for AI agents — smart extraction, stealth mode, structured data

docscraperforai

A Python library for scraping technical documentation with ease — supports single-page and domain-wide crawling, multiple output formats (Markdown, JSON, TXT), and proxy-friendly setups. Ideal for AI, research, and content generation workflows.

peduncle

content extraction from html