url-checker
Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters
A tool to scrape a website and check for the broken links.
A utility to determine the content type of a URL.
