ai-crawlers
User-agent parsing, crawler detection and AI bot blocking for Django.
See which AI crawlers read your site and which AI chat products send you visitors. CLI, middleware, JS snippet.
Check whether a website is reachable and readable by AI crawlers, and score it. Two scores, never averaged. Blocking model training costs no points.
Grade how ready a website is for AI agents & LLM crawlers — llms.txt, AI-bot policy, structured data, SSR content & more. Free web tool + CLI + embeddable badge.
