Marketers face dilemma around rising bot and AI web traffic

“We’re less interested in blocking automated traffic than in classifying it,” said Chad Keller, co-founder of Mellow Sleep, a direct-to-consumer pillow brand. “AI assistants are already sending us traffic that converts several times better than our site average. The blunt instrument of blocking every non-human agent would cost us more than it saves.”

Related Insights

The Business of AI

In Graphic Detail: New data shows publishers face growing AI bot, third-party scraper activity Read More

British retailer John Lewis has seen AI agentic searches rise from 0.3% to 2.5% of all web visits in the space of a year, Reuters reported last week; the company is attempting to position its web properties to attract more in the hope of increasing AI-enabled e-commerce sales.

For other advertisers the rising tide poses a threat.

“Bot activity can inflate impressions and conversions, make campaigns look like they’re performing better than they are, muddy lookalike audiences and eat into frequency caps,” said Mallory Beck, vp, client services at full-service agency Lighthouse Creative.

David Dweck, president at media agency GoFish, said that the agency’s e-commerce clients had, on average, seen bot traffic increase by 80% year-on-year. That had become a problem for clients relying on retargeting — the practice of directing digital ads towards users after they’ve engaged with a website — because their tools inadvertently funnelled media spend towards “users” that were, in fact, bots.

“Those actions got muddied, or poisoned, with a lot of bot traffic,” explained Dweck, who declined to name the clients affected. The issue had caused costs-per-acquisition to briefly spike. “We saw a degradation in performance starting in Q4 last year and dragging into this year,” he added.

The problem had been exacerbated by AI agents, he said, because they can add items to carts or sign up to newsletters, tricking filters designed to catch bots merely visiting and bouncing off web pages.

In response, Dweck said GoFish’s clients had narrowed the retargeting parameters they were using, increasing the average CPM by 20% (Dweck didn’t provide financial specifics). Others saw the bot traffic as a final straw for programmatic and opted to shift budget away from retargeting and into social platforms wholesale. “Since 2023… we’ve seen a big shift out of programmatic and back into places where people are more comfortable like Google, Amazon, Meta,” he said.

GoFish’s clients aren’t alone in that response. Megan Herling, executive director of media, at LERMA/, said in an email that the agency’s clients had cut down on their use of retargeting in recent years as performance has steadily declined. “Bots are part of the problem, but also privacy updates and data opt outs have reduced scale significantly,” she said.

Three other agency buyers told Digiday their clients had experienced similar problems. “If you’re not paying attention, you can end up spending real budget against activity that isn’t coming from real prospects,” said Beck.

Nola Ladd, brand media supervisor at Collective Measures, said three clients had been affected by the issue. Similar to GoFish’s experience, Ladd said the rise in bot traffic had provided “the final nail in the coffin” for retargeting for one client, which shifted its media spend towards retail media and social platforms. She declined to name the brand.

Increased bot traffic to publishers could also reduce the performance of more typical programmatic campaigns, a concern remedied, according to Tim Lathrop, vp  of platform digital at Mediassociates, by the use of pre-bid verification and inclusion lists.

Ladd said Collective Measures was steering other clients towards programmatic targeting that relied on first-party data or a retail media network like Walmart. “First party data is king,” she said.

Rising bot traffic is causing other issues. Colin Maduzia, svp of experience and product strategy at RPA, said it was leading to a rise in fraudulent purchases, as well as confusing analytics and attribution efforts.

Related Insights

AI Revenue Generation

When bots look like buyers: agentic traffic causing new publisher headaches Read More

Meanwhile marketers are briefing agencies to spend more time scrutinizing web visitor behavior, and having to upgrade filtering systems, with the aim of keeping bots out, while letting legitimate AI agents like those sought by John Lewis and Mellow Sleep through. 

“I have to imagine that as this evolves, there will be an increasing level of resources dedicated to filtering out the real and the fake,” said Justin Scarborough, head of programmatic at Crossmedia.

Whether the tools and systems available to marketers will be able to counter the increasing complexity of both AI agents and bots is an open question.

Lighthouse Creative’s Beck, however, advised that brands focus on users exhibiting “higher intent actions” and become pickier about which behaviors they directed marketing efforts towards.

“The answer is being more rigorous about what we consider a meaningful signal,” she said.