GitHub - TecharoHQ/anubis: Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers
github.com
Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers - TecharoHQ/anubis

I just started using this myself, seems pretty great so far!

Clearly doesn’t stop all AI crawlers, but a significantly large chunk of them.

  • @[email protected]English
    2•3 months ago

    So if you try to access a website using this technology via terminal, what happens? The connection fails?

    • DrewEnglish
      3•3 months ago

      If your browser doesn’t have a Mozilla user agent (I.e. like chrome or Firefox) it will pass directly. Most AI crawlers use these user agents to pretend to be human users

      • @[email protected]English
        1•3 months ago

        What I’m thinking about is more that in Linux, it’s common to access URLs directly from the terminal for various purposes, instead of using a browser.

        • DrewEnglish
          1•3 months ago

          If you’re talking about something like curl, that also uses its own User agent unless asked to impersonate some other UA. If not, then maybe I can’t help.