sábado, 15 de agosto de 2026

The Jungle You Don't See: The Automated Fauna of the Internet and Its War of Attrition


We like to think of the internet as a highway: people coming and going, websites waiting for visitors, and the occasional clash of opinions on the shoulders. It is a comfortable image—and a false one. What lies before us looks much more like a hyper-connected digital jungle, and more than half of its inhabitants aren't people, but autonomous lines of code: programs that read, copy, surveil, protect, attack, and disguise themselves without anyone watching.

Behind the scenes, an asymmetric, automated war of attrition is being waged for control of three resources: information, bandwidth, and computing power. In this war, the fauna mutates constantly because it needs to survive, hunt, or protect its territory. What follows is an attempt at cartography: first a census of the species, then their combat tactics, and finally a look at what all of this means when you publish a simple blog post.


An Uncomfortable Fact to Start With

Data from cybersecurity firms agrees on one thing: more than half of the traffic circulating on the internet is not human. According to the Bad Bot Report 2026 by Imperva/Thales *, automated traffic accounted for 53% of the total in 2025 (up from 51% the previous year). And of that total, according to Help Net Security's coverage of the report **, 40% were malicious bots and barely 13% were benign automation. Put plainly: for every good bot, there are roughly three bad ones. The postcard image of a clean highway is quite a long way from that.

• *   https://www.imperva.com/blog/bad-bot-report-2026-bots-agentic-age/
• **  https://www.helpnetsecurity.com/?p=368471
An umbrella term for all of this is automated traffic: bots, crawlers, automated scripts, and botnets. Within this category, you'll find everything from the mail carrier who delivers your content where it needs to go, to the burglar trying doorknobs on every house down the block.


The Census: Digital Biological Castes

The ecosystem can be organized into five castes, each with its own evolutionary niche and function:

•    Beneficial and operational: Googlebot, Discord and Telegram previews, aggregators.

•    Monitoring: UptimeRobot, Pingdom, and company.

•    Defensive: Cloudflare, reCAPTCHA, firewalls.

•    Extractive: Bytespider, GPTBot, residential scrapers.

•    Predatory: Scattered Spider, botnets, vulnerability scanners.

Let's go through them one by one.

1. The Beneficial and Operational Fauna: Explorers of the Light

These are the agents that keep the ecosystem alive: they index content and allow information to circulate in an orderly fashion. They aren't looking to destroy. They aim to siphon data to process it elsewhere, and they are the ones sustaining the information economy.

•    Search engine crawlers and spiders (Googlebot, Bingbot, DuckDuckBot): They continuously traverse web pages to feed search results.

•    AI agents and LLM indexers (GPTBot, ClaudeBot, PerplexityBot): They crawl the web to feed the knowledge bases and language models of artificial intelligences. Here, the boundary becomes blurry, because depending on how you look at them, they fit into this caste just as much as the extractive one. I'll come back to that below.

•    Link and SEO validators (AhrefsBot, SemrushBot, Moz): They analyze link health and page structure for optimization metrics.

•    Aggregators and feed scrapers: Scripts that devour RSS feeds and XML structures from newspapers and blogs to power news readers, financial trading platforms, or automated channels.

•    Preview generators (Twitterbot, Discordbot, TelegramBot): Micro-crawlers that spring into action the instant a human pastes a link into a chat or social network. Their sole purpose is to read the website's Open Graph tags to render the visual card you subsequently see on screen.
Here I'll return to what I left pending regarding AI indexers, because the Imperva report adds a third category that doesn't fit neatly into any single house: AI agents. They no longer limit themselves to browsing sites; they interact with them, retrieve data, execute workflows, and act on behalf of people. This is the fauna mutating in real time: the next species isn't coming to read, it's coming to act. And it blurs the line between legitimate and malicious even further, because both use the exact same interfaces and follow the exact same logic.


2. The Monitoring and Performance Fauna: Guardians and Analysts

Automated software that keeps watch to ensure systems run smoothly and don't crash. This is iterative administrative traffic: repetitive, routine, almost bureaucratic.

•    Uptime monitors (UptimeRobot, Pingdom): They send requests every few minutes to verify that a site is still online. They are the everyday network checks: a tireless, repeating "Are you still there?"

•    Analytics and telemetry: Invisible scripts that measure load speeds, JavaScript errors, and browser performance for every visitor.

3. The Defensive Fauna: The Invisible Shields

Automated systems that protect infrastructure from external attacks and filter out unwanted traffic.

•    WAF (Web Application Firewall): Blocks common attacks before they ever touch the main server.

•    Bot challenges (Cloudflare Turnstile, reCAPTCHA): Intercepts suspicious visits and forces them to solve logic puzzles or cryptographic proofs before displaying content.

•    Anti-DDoS systems: Diverts avalanches of artificial traffic designed to knock servers offline via denial-of-service.

4. The Extractive Fauna: Camouflaged Parasites

Here, the fauna turns murky. These are bots dedicated to massive data extraction, such as Bytespider—ByteDance/TikTok's crawler—and, more broadly, web scrapers and mass-crawling bots operating through proxies.

The trick lies in the mechanism. To avoid being blocked by firewalls, they operate through residential proxy networks. Instead of firing millions of requests from a single Amazon Web Services server (easy to identify and ban), they rent or infect home connections: a household router, a Smart TV. Every request appears to come from a real user sitting in their living room, making surgical IP blocking extraordinarily difficult: to stop the parasite, you have to risk blocking the family next door, too.


5. The Predatory Fauna: Alphas of the Underworld

At the top of the food chain sit the code mafias: automated programs dedicated to hunting down security flaws, stealing data, or saturating resources.

•    Sniffers and vulnerability scanners (Nmap, Nikto, OpenVAS): The sharks of the network. They swim in constant circles across the internet's entire IP range looking for outdated servers, open ports, vulnerable plugins, neglected WordPress installations, or misconfigured hidden directories (the classic /wp-admin/). They have no fixed target; they are just looking for any unlocked door.

•    Brute-force and credential stuffing bots: They attempt to guess passwords using massive automated combinations against login forms, or directly test billions of leaked passwords in seconds.

•    Spam bots: They sweep the web in search of comment sections and contact forms to flood them with fake commercial links or scams.

•    Malicious content scrapers: They automatically copy entire blogs to replicate them on fraudulent clone sites. Yes, blogs just like this one.

•    Cells like Scattered Spider: They use botnets not only to launch DDoS attacks that take down servers through overload, but also as infrastructure for credential stuffing or as proxies for advanced ransomware attacks.

Tactics of the Hidden War

The internet's old peace treaty was the robots.txt file: a courtesy by which websites asked bots not to enter certain rooms, and bots—out of habit and reputation—complied. That treaty has collapsed. It worked as long as the cost of ignoring it was low and the benefit of respecting it was reasonable; it works poorly when data is worth its weight in gold. The hunger for training material for artificial intelligences and the monetization of cybercrime pushed things into open combat. It comes down to asymmetric incentives: the party crawling risks nothing, while the party being crawled pays for the bandwidth and compute. A gentlemen's agreement between parties with nothing at stake lasts only as long as goodwill does.

What is left in its place? A regime of faits accomplis, with no shared contract. On the website side, behavioral and reputation-based blocks replace polite requests for good conduct (the walls I describe below), alongside paywalls and licensing for AI crawler access, and traps for anyone taking what wasn't given to them. On the other side, thicker disguises. Nobody writes the rulebook in a plain text file anymore; each defender writes it with whatever they have on hand.

These are the three primary tactics.

Tactic 1: Identity Spoofing (User-Agent Spoofing)

Aggressive scrapers and indexers no longer introduce themselves by their real names in HTTP headers. They configure their scripts to clone the exact signatures (User-Agents) of common human browsers—something along the lines of Mozilla/5.0 Windows NT 10.0; Chrome/120.0.... By blending in with human traffic, they force administrators to rely on complex behavioral analysis tools to unmask them. The bot walks into the party wearing a guest's suit and tie.

Tactic 2: Data Poisoning

This is the webmasters' resistance. Knowing that AI bots ignore blocks and scrape sites anyway, developers set digital "bear traps."

•    How it works: They inject text invisible to humans (matching the background color) or hidden sections packed with contradictions, subtle malicious code, or flat-out false information.

•    What it achieves: When the bot siphons the web page, it feeds garbage into its language model, degrading the intelligence of the AI that harvested it.
It is a curious defense: the very thing that harms the invader is precisely what the invader chose to steal.

Tactic 3: Cryptographic Walls (Turnstile and Behavioral Challenges)

Defenders (WAFs, Cloudflare) have abandoned the interactive CAPTCHAs of the past because AIs now solve them better than humans do. Today's defense relies on silent telemetry challenges and background cryptographic proofs. While the page loads, the visitor's browser solves lightweight cryptographic puzzles and submits signals about its environment and behavior, which the system uses to estimate whether there is a person on the other end (in some cases, right down to mouse micro-movements). If a bare script is on the other side (for instance, a Python bot using Requests), the server slams the door in its face instantly.

 

Perpetual Crossfire

The internet operates today under this constant crossfire. Every millisecond, an average website fends off hundreds of micro-attacks or extraction attempts while simultaneously holding the door open for the bots that feed it: the search engines that make it discoverable, the previews that make it presentable, and the monitors that sound the alarm if it goes down.

 

What Happens When You Publish a Post

All of this comes down to earth in the simplest of gestures. You publish a post on your blog, drop a link on another platform, and within minutes—sometimes seconds—a small fraction of this invisible fauna has already slipped quietly past to read it. Before your first flesh-and-blood reader ever shows up, it has likely already been read by a preview generator, a feed aggregator, a search engine crawler, an AI indexer, and—with a bit of bad luck—a scraper hauling it off to paste somewhere else.

If this fauna is already reading, copying, and classifying everything we write, the underlying question ceases to be technical: who, exactly, are we writing for? For the human who might arrive, for the machine that definitely arrived, or for both at once—knowing full well that nobody invited the second one.


Minimal Glossary

•    Automated traffic: All traffic generated by programs rather than people (bots, crawlers, scripts, botnets).

•    Iterative administrative traffic: Repetitive maintenance and surveillance requests (uptime monitors, network checks).

•    Feed scraping / aggregators: Automated reading of RSS and XML feeds to reuse content.

•    Web scrapers and mass crawlers: Crawling bots that traverse sites to extract or index data; the most aggressive ones, like Bytespider, operate via proxies.

•    Residential proxies: Rented or infected home internet connections used to disguise the origin of requests.

•    Botnet: A network of remotely controlled devices used for DDoS attacks, credential stuffing, or as intermediaries in cyberattacks.

•    Credential stuffing: Testing leaked passwords on a massive scale against login forms.

•    User-Agent spoofing: Falsifying browser identification to pass as human.

•    Data poisoning: Planting false or hidden content to contaminate whatever extracts it.

 

Co-created with my AI assistant.

 

Supplementary entries on the topic:

The Silent Trench: An Apology for the Blog in the Era of Mass Algorithms
https://reuniendoletras.blogspot.com/2026/08/the-silent-trench-apology-for-blog-in.html

The Longevity of Blogging: On Das Bücherregal
https://reuniendoletras.blogspot.com/2026/07/the-longevity-of-blogging-on-das.html

No hay comentarios.:

Publicar un comentario