FocusWeb AI Crawler Monitor

FocusWeb AI Crawler Monitor is an observation-only plugin that helps you understand how AI crawlers interact with your WordPress site. It never blocks or alters requests — it simply logs matching visits and gives you a dashboard to review them.

Detected bots (built in, configurable):

• GPTBot (OpenAI)
• ChatGPT-User (OpenAI, on-demand)
• ClaudeBot (Anthropic)
• Claude-Web (Anthropic)
• PerplexityBot
• Google-Extended
• CCBot (Common Crawl)
• Bytespider (ByteDance)
• Amazonbot
• Applebot-Extended

Features:

• Lightweight init-hook based detection, no server log file access required.
• Custom database table with indexes for fast dashboard queries.
• Dashboard with summary cards, a Chart.js time-series chart, a most-crawled URL table, and a paginated raw log with CSV export.
• Configurable log retention with a daily cleanup cron job.
• Optional IP anonymization (masks the last octet/segment) for GDPR-friendly logging, enabled by default.
• Excludable URL path patterns (e.g. /wp-admin/*, /checkout/*).
• AI Referral Traffic tracking: detects human click-throughs from AI answer engines (ChatGPT, Perplexity, Microsoft Copilot, Google Gemini, Claude, Meta AI, You.com) via the Referer header or matching utm_source parameters, and captures the source query when the referring platform exposes one.
• Built-in llms.txt generator: produces suggested content from your site title, tagline, pages, and posts for manual placement at your site root.
• Fully functional on shared/managed hosting — no filesystem writes outside the database, no special server permissions needed.