Skip to content
Tech News
← Back to articles

Who does Anubis actually stop?

read original more articles
Why This Matters

This article highlights the ineffectiveness of Anubis, an HTTP proxy that imposes proof-of-work challenges to block access, revealing that it fails to stop sophisticated AI-based scraping while unfairly burdening human users and those with limited devices. The discussion underscores the ongoing challenges in balancing web security measures with accessibility and the unintended consequences of anti-bot tactics.

Key Takeaways

I have been working on a patch to the Linux kernel to support $ORIGIN for the interpreter ( PT_INTERP ) via bpf in binfmt_misc [thread].

Of course I’m leveraging an LLM to help me do this! To pre-seed the context of the LLM, I asked it to read the https://lore.kernel.org/ thread.

Uh oh. Looks like they have adopted Anubis, which is an HTTP proxy that requires proof-of-work before allowing access to the resource.

Did this really do anything?

Unfortunately, no.

My AI diligently came up with anubis-fetch, which you can find at https://github.com/fzakaria/anubis-fetch. The tool tries to natively solve the proof of work or, as a last resort, will launch Chromium to visit the URL.

This tool also impersonates a real Chrome TLS/JA3 fingerprint natively via req so it clears passive Cloudflare blocking too. ☝️

# HTML to stdout $ anubis-fetch https://lore.kernel.org/linux-mm/some-thread/T/ # readable plain text $ anubis-fetch --text https://lore.kernel.org/linux-mm/some-thread/T/

So who did we stop?

The exact adversary Anubis targets defeats it trivially.

... continue reading