Killswitch: Add per-function short-circuit mitigation primitive
The article explains the use of Anubis, a Proof-of-Work system, to protect websites from AI-driven scraping that can cause downtime. Anubis aims to deter large-scale scrapers by imposing computational costs while minimizing impact on individual users. It currently requires JavaScript, which may conflict with privacy plugins, and future improvements will focus on better detection of headless browsers.
- ▪Anubis is a Proof-of-Work system designed to prevent AI companies from aggressively scraping websites.
- ▪It uses a method similar to Hashcash to make mass scraping more computationally expensive.
- ▪The system currently requires modern JavaScript, which may block users with privacy plugins like JShelter.
- ▪Anubis is a temporary solution while developers work on better fingerprinting techniques for headless browsers.
- ▪The website running Anubis is version 1.25.0 and was created by Techaro with mascot design by CELPHASE.
Opening excerpt (first ~120 words) tap to expand
Making sure you're not a bot! Loading...You are seeing this because the administrator of this website has set up Anubis to protect the server against the scourge of AI companies aggressively scraping websites. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.Anubis is a compromise. Anubis uses a Proof-of-Work scheme in the vein of Hashcash, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.Ultimately, this is a placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Kernel.
Discussion
0 commentsMore from Kernel
