# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # Allow all crawlers, including AI search bots (GPTBot, ChatGPT-User, # PerplexityBot, ClaudeBot, anthropic-ai, Google-Extended, Bingbot). User-agent: * Allow: / Sitemap: https://urlpipe.dev/sitemap.xml