Are AI bots actually reading llms.txt files?

This live experiment tracks whether major AI bots like GPTBot, ClaudeBot, and PerplexityBot are reading llms.txt files across the web.

Each time an AI crawler accesses an llms.txt file, we log an anonymous signal from that site. This helps us understand which bots are actually checking for AI-specific permissions and content rules.

—

Websites are participating in the experiment

Bot Sites Seen On Last Seen
Visual overview of the llms.txt crawler experiment

What is this experiment?

Read more about this project and how we're tracking AI crawler access to llms.txt files.

Read the overview
WordPress plugin to log llms.txt crawler access

Join via WordPress Plugin

Install the free plugin to create a llms.txt file and track crawler behavior.

Get the plugin