Documentation

Getting Started

AI Bot Tracker installs like any WordPress plugin. No technical configuration is required — detection begins the moment you activate it.

Installation

1. In your WordPress admin, go to Plugins → Add New
2. Search for "AI Bot Tracker"
3. Click Install Now, then Activate

That's it. The plugin creates its database tables, deploys a honeypot trap, and starts logging bot visits immediately. No API keys, no external accounts, no setup wizards.

Your First Dashboard View

After activation, click the AI Bot Tracker menu item in your WordPress sidebar. The dashboard shows a real-time summary of all detected AI bot activity on your site. If you just installed, give it 24–48 hours for bots to visit — most WordPress sites see their first AI crawler within a day.

Free vs. Paid

The free version (Monitor tier) includes bot detection, honeypot traps, and the analytics dashboard. Paid tiers unlock response strategies, auto-blocking, GeoIP tracking, and more. See Licensing & Tiers for the full comparison.

Dashboard & Analytics

The dashboard is your command center for understanding AI bot activity on your site. It updates in real time as new visits are logged.

Stat Cards

Four summary cards appear at the top of the dashboard:

• Total Bot Visits — all detected AI bot requests in the selected time period
• Unique Bots — number of distinct AI bot user agents detected
• Honeypot Hits — bots that followed the hidden honeypot link
• Blocked Visits — requests blocked or responded to with a non-standard strategy (Protect+ tiers)

Charts

The dashboard includes two primary visualizations:

• Bot Distribution (pie chart) — breakdown of visits by bot identity (e.g., GPTBot, ClaudeBot, Bytespider)
• Daily Activity (bar chart) — bot visits per day over the selected time range

Time Range

Use the time range selector to filter data: last 24 hours, 7 days, 30 days, or a custom range. All stat cards and charts update to reflect the selected period.

Visits Log

Below the charts, a detailed log table shows each individual bot visit with the bot name, URL visited, timestamp, response strategy applied, and IP hash. The log is searchable and sortable.

Honeypot Detection

Honeypot detection is the core of AI Bot Tracker. It catches bots that ignore robots.txt by placing an invisible link on your site that only automated crawlers will follow.

How It Works

1. AI Bot Tracker injects a hidden link into your pages. The link is invisible to human visitors (hidden via CSS) but appears in the raw HTML that bots parse.
2. When a bot follows this hidden link, it reveals itself as a crawler that doesn't respect standard conventions.
3. The visit is logged and, on paid tiers, the bot can be automatically blocked or responded to with a custom strategy.

Zero False Positives

Because the honeypot link is visually hidden and not linked from any navigation or sitemap, real users never follow it. This makes honeypot detection a zero-false-positive detection method.

Auto-Generated Path

By default, AI Bot Tracker generates a random honeypot path (e.g., /_ai-honeypot/f8a3b1/). This path changes on first activation and remains consistent thereafter.

Custom Honeypot Paths Protect+

Protect tier and above can configure up to 5 custom honeypot paths. Useful for targeting specific crawler behaviors or testing detection coverage across different site sections.

robots.txt Trap Protect+

The robots.txt trap is a second detection layer that catches bots ignoring your robots.txt directives. It works alongside honeypot detection to identify non-compliant crawlers.

How It Works

1. You configure up to 5 custom trap paths in Settings → robots.txt Trap.
2. AI Bot Tracker automatically injects these paths as Disallow directives into your site's robots.txt file via the WordPress robots_txt filter.
3. Compliant bots read robots.txt and avoid these paths. Non-compliant bots visit them anyway — and get caught.

Why Both Honeypot and robots.txt Trap?

Honeypots catch bots that follow hidden links. robots.txt traps catch bots that explicitly ignore Disallow rules. Some bots do one but not the other. Running both gives you broader detection coverage.

Response Integration

When a bot hits a robots.txt trap path, the same response strategies apply — log, block, tarpit, rate-limit, decoy, or shadowban. Trap hits are logged alongside honeypot hits in your detection dashboard.

Response Strategies

When AI Bot Tracker detects a bot, it can respond in one of six ways. Each strategy serves a different purpose depending on your goals.

Choosing a Strategy

For most sites, start with Log Only to understand your bot traffic, then move to Block 403 or Shadowban for bots you want to deny. Use Tarpit against aggressive crawlers that hit your site hundreds of times per day. Decoy Content is a creative option for content sites concerned about their text being used to train AI models.

Auto-Blocking Protect+

Auto-blocking automatically applies your chosen response strategy to bots based on their behavior, without manual intervention.

Tiered Thresholds

• Unknown bots — blocked immediately on their first honeypot hit. If a bot follows a hidden link, it's crawling without permission.
• Known bots — blocked after a configurable number of honeypot hits (default: 3, minimum: 2). This allows legitimate crawlers like Googlebot a grace period in case of accidental honeypot hits.
• Whitelisted bots — never auto-blocked, regardless of behavior. Use this for bots you explicitly want to allow (e.g., your own monitoring tools).

Block Duration

Auto-blocks last 30 days by default. After the block expires, the bot can resume crawling. If it hits the honeypot again, it gets re-blocked. You can adjust the duration or set blocks to permanent in settings.

Notifications

Enable email notifications to get alerted each time a bot is auto-blocked. See Email Alerts for setup details.

Per-IP Rules & Whitelists Protect+

Per-IP rules let you assign specific response strategies to individual IP addresses. This gives you fine-grained control over how your site responds to each crawler.

How It Works

Navigate to the Honeypot tab and scroll to the Per-IP Rules section. You can add up to 200 rules, each mapping an IP hash to a response strategy. Rules take priority over auto-blocking and the default response strategy.

Privacy-First Design

AI Bot Tracker stores IP hashes, not raw IP addresses. When you create a rule, you enter the IP address and the plugin immediately hashes it. The original IP is never stored in the database.

Common Use Cases

• Whitelist your own crawlers — set strategy to "Log Only" for your monitoring tools, uptime checkers, or staging server
• Block known bad actors — assign "Block 403" to IPs that are aggressively scraping your content
• Tarpit specific bots — slow down crawlers from a known IP range without blocking them outright

GeoIP Location Tracking Optimize+

GeoIP tracking adds geographic context to every bot visit, showing you where in the world your crawlers are coming from.

Setup

1. Create a free account at maxmind.com and generate a GeoLite2 license key
2. In AI Bot Tracker, go to Settings → GeoIP
3. Enter your MaxMind license key and click Download Database

The GeoLite2 database is stored locally in your WordPress installation. It auto-updates weekly via WP-Cron, so your location data stays current.

What You Get

• Country + city for each bot visit (when available)
• Geographic Distribution — a pie chart showing the top countries by bot visits
• Visits by Country — a detailed table with visit counts, unique bots, and honeypot hit rates per country

Why It Matters

Geographic data helps you identify patterns. Most major AI companies operate crawlers from US and European data centers. If you see heavy crawling from unexpected regions, it could indicate rogue scrapers rather than legitimate AI services.

Crawl Analytics & URL Insights Optimize+

Crawl Analytics goes beyond simple visit counting. It shows you exactly which pages bots are targeting, how efficiently they crawl, and where your crawl budget is being wasted.

URL-Level Data

Every URL on your site gets its own crawl profile: total bot visits, unique bots, first and last seen dates, and which response strategy was applied.

Crawl Velocity

Track how many pages bots are crawling per hour or per day. Sudden spikes often indicate a new bot has discovered your site or an existing crawler has ramped up its activity.

Crawl Health Cards

Four diagnostic metrics help you evaluate crawl quality:

• 404 Waste — bots requesting pages that don't exist
• Redirect Waste — bots hitting URLs that redirect (301/302)
• Parameter URLs — bots crawling parameterized URLs that may be duplicates
• Stale Re-crawls — bots re-crawling unchanged content

Sitemap Coverage

Compare which URLs appear in your XML sitemap versus which URLs bots are actually crawling. This reveals content gaps (pages in your sitemap that bots never visit) and crawl waste (pages not in your sitemap that bots keep hitting).

Email Alerts Protect+

Stay informed about important bot activity without constantly checking the dashboard.

Alert Types

• Honeypot Intrusion — triggered when a new bot hits your honeypot trap for the first time
• Auto-Block Notification — sent when a bot is automatically blocked after exceeding the threshold
• Crawl Anomaly — fires when bot activity spikes to 3x or more above your site's normal baseline

Configuration

Enable or disable each alert type independently in Settings → Alerts. Alerts are sent to the WordPress admin email address by default. You can add additional recipients.

Email Delivery

AI Bot Tracker sends alerts using WordPress's built-in wp_mail() function. If your site's emails aren't reaching you reliably, install an SMTP plugin like WP Mail SMTP to route emails through a proper mail server.

AI Discoverability (llms.txt) Protect+

AI Bot Tracker can serve llms.txt and llms-full.txt files that help AI systems understand your site's content and permissions.

What Are llms.txt Files?

The llms.txt standard is an emerging convention for websites to provide structured information to large language models. Think of it as a machine-readable summary of your site — what it contains, how AI systems should interact with it, and what permissions apply.

Setup

1. Create a WordPress page with the slug llms-txt for the summary version
2. Optionally create a page with the slug llms-full-txt for the detailed version
3. AI Bot Tracker automatically serves these as plain text at /llms.txt and /llms-full.txt with CORS headers enabled

Block markup is stripped automatically — the plugin extracts clean plain text from your page content.

AI Agent Discovery (ai-plugin.json) Optimize+

The ai-plugin.json manifest tells AI agents what your site offers and how to interact with it programmatically.

What It Does

AI Bot Tracker serves a JSON manifest at /ai-plugin.json following the OpenAI plugin schema. This file includes your site name, description, contact email, legal info URL, and a pointer to your API specification. AI agents that support plugin discovery can read this file to understand your site.

Configuration

Go to Settings → AI Agent Discovery to configure the contact email and legal info URL included in the manifest. The rest is auto-generated from your WordPress site metadata. CORS headers are enabled so AI agents can fetch the manifest cross-origin.

Anonymous Telemetry Protect+

AI Bot Tracker includes an optional anonymous telemetry system that helps us understand how the plugin is being used. Telemetry is off by default and only available on paid tiers.

What Gets Sent

When enabled, a weekly report is sent to our servers containing:

• Plugin version, WordPress version, PHP version
• Active feature flags (which features you have enabled)
• Aggregate bot count (total, not per-bot)
• License tier and site locale

Each report is tied to a randomly generated UUID — not to your site URL, domain name, or any personally identifiable information.

What Is Never Sent

• Your site URL or domain name
• Visitor IP addresses or bot logs
• Page content or crawled URLs
• Any personal data about you or your visitors

How to Enable or Disable

Go to Settings → Telemetry and toggle the checkbox. You can also enable it via the one-time admin notice that appears after license activation. Disabling telemetry immediately stops all data transmission and unschedules the weekly cron job.

REST API Scale

The REST API gives you programmatic access to all AI Bot Tracker data. Use it to build custom dashboards, integrate with monitoring tools, or export data to your analytics pipeline.

Authentication

All authenticated API requests require an API key. Generate one in Settings → API. Pass the key via the X-AI-Tracker-Key header:

curl -H "X-AI-Tracker-Key: your-api-key" \
  https://yoursite.com/wp-json/ai-tracker/v1/stats

Key Endpoints

Rate Limits

API requests are limited to 60 requests per minute per API key. Exceeding this returns a 429 response with a Retry-After header.

OpenAPI Spec

A full OpenAPI 3.0 specification is available at /wp-json/ai-tracker/v1/openapi.json when the REST API is active. Import it into tools like Postman or Swagger UI for interactive exploration.

Backup & Restore Scale

Export your complete AI Bot Tracker dataset as a JSON file and import it on another site or restore it after a migration.

Export

Go to Settings → Backup and click Export Data. The JSON export includes:

• All bot visit logs
• Plugin settings and configuration
• Per-IP rules and whitelists
• Alert configuration
• Blocked IP records

Sensitive data (API keys, MaxMind license key) is excluded from exports for security.

Import

Upload a previously exported JSON file in Settings → Backup. Choose one of two import modes:

• Merge — adds imported records to your existing data without overwriting anything
• Replace — overwrites all existing data with the imported dataset

Licensing & Tiers

AI Bot Tracker uses a four-tier licensing model. The free Monitor tier is available on WordPress.org. Paid tiers unlock advanced features.

Activating Your License

1. After purchasing, you'll receive a license key via email
2. In WordPress, go to AI Bot Tracker → Settings → License
3. Enter your license key and click Activate

Your license is tied to one site. To transfer it, deactivate on the current site first, then activate on the new one.

Billing & Renewals

All payments are processed by Lemon Squeezy (our Merchant of Record). Manage your subscription, update payment methods, or cancel through your Lemon Squeezy customer portal. After cancellation, your paid features remain active until the end of your current billing period, with a 7-day grace period. After that, the plugin reverts to Monitor (free) functionality — your data is preserved, but paid features are deactivated.

FAQ & Troubleshooting

Emails not being delivered?

AI Bot Tracker sends alerts via WordPress's wp_mail() function. Many hosts don't configure server-side email properly. Install WP Mail SMTP (or a similar plugin) to route emails through a reliable SMTP server like Gmail, SendGrid, or Mailgun.

GeoIP database not downloading?

Double-check your MaxMind license key. You need a free GeoLite2 account at maxmind.com. Make sure your server can make outbound HTTPS requests (some hosting firewalls block this). If the auto-download fails, you can manually download the GeoLite2-City database and upload it via SFTP.

Honeypot not catching any bots?

Give it 24–48 hours. Bots need to visit your site and discover the hidden honeypot link. If you still see no hits after a few days, check that your caching plugin isn't stripping the honeypot link from cached pages. Exclude /_ai-honeypot/ and any custom honeypot paths from your cache.

Does it work with caching plugins?

Yes. AI Bot Tracker is compatible with all major caching plugins (WP Super Cache, W3 Total Cache, LiteSpeed Cache, WP Rocket, etc.). Just make sure to exclude your honeypot paths from the cache so the detection mechanism can trigger on each bot request.

Does it slow down my site?

No. AI Bot Tracker adds less than 1ms of overhead per request. The honeypot link injection is a lightweight DOM operation, and bot detection only runs when a request matches a known user-agent pattern or hits a honeypot path. Normal visitors experience no measurable impact.

How do I know which tier I need?

• Monitor (Free) — you want to see what's happening. No blocking, no responding.
• Protect ($69/yr) — you want to take action. Block, tarpit, shadowban, or serve decoy content.
• Optimize ($129/yr) — you want deep analytics. GeoIP tracking, crawl velocity, URL-level insights.
• Scale ($249/yr) — you need API access, backup/restore, or manage multiple sites.