EN / RU / πŸ€–
← Back to essays
Β· Essay Β· 1 min

🐭 Cat-and-Mouse Game: Perplexity vs. Websites

Cloudflare revealed how Perplexity continues the cat-and-mouse game with websites, scraping content and ignoring robots.txt.

🐭 Cat-and-Mouse Game: Perplexity vs. Websites

Cloudflare has revealed how Perplexity continues the cat-and-mouse game with websites: scraping content by disguising its bots as regular users, ignoring robots.txt, and using third-party IPs.

This is yet another episode in the eternal struggle between AI agents and services trying to restrict access to their data. Some seek new loopholes, others build barriers β€” and so it goes in circles.

Who will win?

πŸ”— Source: https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/

🐭 Cat-and-Mouse Game: Perplexity vs. Websites β€” illustration