Bots Now Outpace Humans in Cloudflare’s Web Page Traffic Data as AI Agents Reshape the Internet

Bots now generate more web page requests than humans in Cloudflare’s latest Radar data, marking a major shift in how the open web gets used. The company’s Cloudflare Radar dashboard shows automated traffic ahead of human traffic for HTTP requests to HTML content.

The key detail is important: this does not mean bots now produce more total internet activity across every category. Cloudflare’s metric focuses on requests to HTML pages, not video streaming, mobile app usage, gaming, email, or total bandwidth.

Even with that limitation, the change matters. Web pages remain the foundation of search, publishing, ecommerce, documentation, and online services. If machines now request more of those pages than people do, publishers, advertisers, security teams, and search platforms need to rethink how they measure traffic and protect content.

What The Latest Cloudflare Data Shows

Cloudflare’s current bot-versus-human view shows bots accounting for the majority of HTTP requests to HTML content. Human traffic now represents the smaller share of that same category.

The shift follows a trend that Cloudflare had already documented in its 2025 Radar Year in Review. That report found that non-AI bots started 2025 ahead of human-generated HTML requests, while AI bots averaged 4.2% of HTML requests during the year.

Cloudflare CEO Matthew Prince had previously warned that AI agents could push bot activity beyond human activity sooner than expected. The latest figures suggest that the crossover has already arrived in a key slice of web traffic.

Key Details At A Glance

Main finding	Bots now lead humans in Cloudflare’s HTTP requests to HTML content metric
Measured category	Bot vs. human requests to HTML pages
What it does not measure	Total internet bandwidth, app usage, streaming, gaming, or email
Main driver	Traditional crawlers, scraping systems, bad bots, and newer AI agents
Publisher impact	More machine visits, less reliable engagement signals, and growing content-access pressure
Security impact	More automation to classify, rate-limit, block, or monetize

Imperva Also Saw Bots Pass Humans In 2024

Cloudflare is not the only company reporting a bot-heavy web. Imperva said in its 2025 Bad Bot Report that automated traffic reached 51% of all web traffic in 2024, surpassing human activity for the first time in a decade.

Imperva also said bad bots made up 37% of all internet traffic. That includes automated activity used for account takeover, scraping, spam, fraud, inventory hoarding, fake registrations, and other forms of abuse.

The two datasets measure traffic differently, but they point in the same direction. Automation has moved from background noise to a dominant force on the web.

Why AI Agents Are Accelerating The Shift

Traditional bots include search crawlers, monitoring tools, uptime checkers, scrapers, and malicious automation. The newer pressure comes from AI systems that browse, summarize, compare, retrieve, and act on behalf of users or companies.

A person shopping for a product may visit a few pages. An AI agent handling the same task can query hundreds or thousands of pages in a short time. That difference changes the traffic profile of the web, even when the final decision still belongs to a human.

Cloudflare’s year-end traffic analysis also showed how quickly AI crawlers became visible in HTML traffic. They still formed a smaller share than traditional bots in 2025, but the growth pattern gave publishers and infrastructure providers a warning.

Why This Matters For Publishers And Website Owners

For publishers, the bot-heavy web creates a business problem. A page view from an AI crawler does not equal a loyal reader, a newsletter signup, or a subscriber. It can still consume server resources and extract value from original reporting.

For advertisers, the change can distort analytics. If bot activity inflates page requests or weakens engagement quality, campaign reports may look stronger than real human attention supports.

Welp, that happened faster than I predicted. Thought it would be end of 2027, then early 2027, but agentic traffic growing so fast that bots have now passed human traffic online for the first time in the Internet's history. https://t.co/2zX5bHdhsa

— Matthew Prince 🌥 (@eastdakota) June 3, 2026

For website owners, the operational cost also grows. More automated requests can raise hosting costs, strain origin servers, and force teams to invest more in bot detection, rate limiting, and access control.

Security Teams Face A Bigger Automation Problem

Not every bot is harmful. Search crawlers, monitoring services, accessibility tools, and approved AI crawlers can serve legitimate purposes. The problem is that harmful automation often hides among normal-looking web requests.

The Imperva bot report warned that AI has made bot creation easier for less skilled attackers. That raises the volume of automated attacks and makes some bot behavior harder to detect with older rules.

Security teams should therefore focus less on simple bot labels and more on behavior. Login abuse, rapid scraping, abnormal user-agent patterns, repeated checkout attempts, suspicious API access, and unusual session behavior deserve closer attention.

What Website Owners Should Do Now

• Separate trusted crawlers from unknown or suspicious bots.
• Monitor HTML request patterns, not just total traffic.
• Review analytics for signs of inflated machine traffic.
• Use rate limits on sensitive paths such as login, search, checkout, and API endpoints.
• Require stronger protection for account creation and password reset pages.
• Review robots.txt, but do not rely on it as a security control.
• Decide which AI crawlers can access content and which should be blocked.

Publishers should also review whether their content strategy still assumes human-first discovery. If AI systems summarize pages before users click through, websites may receive fewer visits even when their content influences more answers.

Cloudflare Pushes Toward Paid And Permissioned Crawling

Cloudflare has responded to the AI crawler surge with tools that let site owners manage and monetize automated access. Its Pay Per Crawl system lets participating content owners set payment terms for crawler access.

The idea reflects a growing debate across the web. AI companies need fresh content, but publishers want control, attribution, traffic, or compensation when machines consume their work at scale.

The same Pay Per Crawl model also shows where the market may go next. Instead of a simple allow-or-block approach, publishers may start treating crawler access as a licensed product.

The Web Is Becoming Machine-First In Some Places

The latest Cloudflare Radar data does not prove that humans have left the internet. It shows that machines now drive the majority of requests in a major category of web page traffic.

That distinction matters, but it does not make the trend less important. The open web now serves two audiences at once: people who read pages and machines that fetch, rank, summarize, compare, and reuse them.

The next phase of web strategy will depend on how companies handle that split. Publishers will need better crawler controls, advertisers will need cleaner measurement, and security teams will need stronger bot defenses. The old assumption that web traffic mostly represents human attention no longer works.

FAQ