Know which crawlers are hitting your site
Dashboard logs summarize user agents, client types, scores, decisions, and paths so teams can see crawler behavior before enforcing blocks.
Website crawler blocker
Website crawler blocking with route logs, install snippets, and search-bot separation.
Dashboard logs summarize user agents, client types, scores, decisions, and paths so teams can see crawler behavior before enforcing blocks.
The browser snippet collects lightweight signals that help identify webdriver, empty plugin surfaces, missing language, and static HTML clients.
Middleware can protect pricing, content, docs, search, and API routes while leaving public pages open during rollout.
FAQ
It can if configured badly. Voidek classifies known search engines separately so Googlebot can stay allowed while unwanted automation is challenged.
No. Start in monitor mode, then choose which crawler classes should be logged, challenged, or blocked.