url-normalization
Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters
A deterministic SEO metadata library for content platforms. Generate canonical URLs, titles, descriptions, robots directives, Open Graph, Twitter Cards, and JSON-LD from content entities. One Rust core with Python, Node.js, and Rust APIs.
Remove clutter from URLs and return a canonicalized version