boilerplate-removal
HTML to RAG-ready Markdown. A 150 KB learned filter removes nav, footers, cookie banners, related links and comments; beats Trafilatura on held-out WCXB (F1 0.902 vs 0.860). CPU-only, ~36 ms/page.
Pareto-optimal models for cleaning the web — fast, encoder-based main-content extraction from HTML.
Remove boilerplate from scraped markdown before it reaches an LLM
Heuristic text extraction from news sites in Python3
