rvachev.orgEN / RU / 🤖
← Back to essays
· Essay · 1 min

🐭 Cat-and-Mouse Game: Perplexity vs. Websites

Cloudflare revealed how Perplexity continues the cat-and-mouse game with websites, scraping content and ignoring robots.txt.

🐭 Cat-and-Mouse Game: Perplexity vs. Websites

Cloudflare has revealed how Perplexity continues the cat-and-mouse game with websites: scraping content by disguising its bots as regular users, ignoring robots.txt, and using third-party IPs.

This is yet another episode in the eternal struggle between AI agents and services trying to restrict access to their data. Some seek new loopholes, others build barriers — and so it goes in circles.

Who will win?

🔗 Source: https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/

🐭 Cat-and-Mouse Game: Perplexity vs. Websites — illustration

Newsletter

Get new posts by email

A weekly digest of what I write about AI, software and products. Unsubscribe anytime.