How to use UAWatch to check if bots can access your website

UAWatch monitors which AI crawlers, search engines, and SEO tools can access your website. It checks daily and emails you the moment anything changes — so you find out about access problems before they affect your rankings or AI visibility.

Getting started

1

Sign in

Enter your email address on the sign-in page. We'll send you a magic link — no password needed. Click the link in the email to access your dashboard.

2

Add a site

From your dashboard, enter the full URL of the site you want to monitor using https:// (e.g. https://yourdomain.com or https://www.yourdomain.com) and click Subscribe. You'll be taken to Stripe to complete payment — £10/month per site.

3

First check runs immediately

As soon as your subscription is confirmed, UAWatch runs an immediate check against all 124 bots. Results are usually ready within 90 seconds.

4

Check daily, alert on change

UAWatch checks your site every night between 01:00 and 03:00 UTC. You'll only receive an email if something changes — a bot that was allowed is now blocked, or vice versa. No news is good news!

Understanding your results

Each bot is tested in two ways:

The Access column combines both signals into an overall verdict:

The Reason column gives a plain English explanation — for example: "Blocked by server (403) — likely WAF or Cloudflare rule". This tells you where the block is coming from so you know what to fix.

Important: UAWatch tests using user-agent spoofing from our server IP. Firewalls like Cloudflare that verify both the user agent and the IP address may show false negatives — the bot appears allowed here but is blocked for real crawlers. If you see a 403 alongside an allowed robots.txt status, this is the most likely cause.

Bot categories

UAWatch monitors 124 bots across 9 categories:

AI Crawler
Trains on or indexes content for AI models. GPTBot, ClaudeBot, Google-Extended etc.
AI Search
Powers AI answer engines. PerplexityBot, DuckAssistBot, OAI-SearchBot etc.
AI Agent
Autonomous AI agents. ChatGPT Agent, Google-Agent.
Search Engine
Traditional search indexing. Googlebot, Bingbot, Applebot, Bravebot etc.
Training Data
Bulk dataset collection. CCBot, AI2Bot, img2dataset.
Social Media
Link preview fetchers. Twitterbot, LinkedInBot, Pinterest, TikTok, Reddit etc.
SEO Tool
SEO platform crawlers. Ahrefs, Semrush, Screaming Frog, Majestic, Moz etc.
Data Extraction
Commercial scraping services. Diffbot, Brightbot, Scrapy.
Productivity
Tool integrations. Atlassian.

Why does it matter if AI crawlers are blocked? If bots like GPTBot, ClaudeBot, or PerplexityBot can't access your site, your content won't appear in AI-generated answers. As AI search grows, this directly affects your visibility in tools like ChatGPT, Claude, and Perplexity.

Full Bot List

Here's the full list of all 124 bots that UAWatch checks each night:

AI Agents / Bots / Crawlers

Search Engines

Data Extraction Bots

Productivity & Training Data Bots

SEO Tool Bots

Social Media Bots

The dashboard tabs

Re-checking manually

You can trigger a manual check at any time using the Re-check now button on the results page. This is rate-limited to once per hour per site to avoid overloading your server.

Email alerts

Alerts are sent to the email address you signed up with. You'll receive one email per site per night — but only if something changed. The email shows a table of what changed, from what status to what, with a direct link back to your dashboard.

Managing your subscription

Need help?

If you have a question or something isn't working as expected, email us at support@uawatch.com.