Measuring Google’s AI Overviews’ impact: Why keyword data is more telling than CTRs for publishers

It’s a clear indicator that an increasing volume of content is now being summarized by Google’s AI feature, according to Semrush president Eugene Levin.

Some publishers had their share of keywords with AI Overviews increase significantly in the last six months. For example, ProPublica.org’s share of keywords triggering AI Overviews increased 204% in that time. BuzzFeed.com’s was up 172%. Rolling Stone and Vanity Fair’s sites were both up 164%. NewRepublic.com was up 156% in that time, per Semrush’s data. 

While search traffic continues to wildly fluctuate across different publishers, there is nothing yet to show that AI Overviews is tanking referral traffic. Though it certainly isn’t helping drive any additional traffic. 

“There’s no statistical evidence that says that, in aggregate, Google generates less traffic when there is an AI Overview. It generates roughly the same amount of traffic,” Levin said.

From AI Overviews’ launch in May 2024 to February 2025, ProPublica’s organic global traffic was up 51.4%, BuzzFeed’s was up 8.2%, Rolling Stone was down 11.8%, Vanity Fair was down 19.7% and NewRepublic.com ‘s traffic was up 59.6%, per Semrush’s data. 

Other reports, however, paint a more worrying picture of the wider impact. Last week Tollbit — a marketplace for publishers and AI firms — released a report that revealed that AI search bots are sending on average 95.7 percent less referral traffic than traditional Google search. It had gathered data from 140 websites including both national and local news sites. (Tollbit’s report did not include Google AI Overviews or its Gemini AI model.)

Publishers and analytics companies told Digiday they don’t have any real visibility into the impact of AI Overviews on their traffic. That’s because Google only provides data on overall search referral traffic, not click-throughs from AI Overviews, which provide generated summaries of information from multiple sources to answer a user’s Google search query.

One publishing exec said that their Google Search referral traffic appeared “stable,” so they couldn’t tell if there was a negative impact from Overviews. And because the data was lumped together, they couldn’t see if links in AI Overviews have a higher or lower click-through rate than normal search results.

“All we can do is look at the aggregate and be like, ‘I don’t think it’s a negative, but I don’t know for sure,’” said Paul Bannister, chief strategy officer at Raptive, which sells ads for independent sites.

Google said in a press briefing last October that embedding direct links within generated answers in Overviews was boosting publisher traffic, but didn’t share any data to back that up.

The winners and losers from the introduction (and expansion) of AI Overviews for now, boils down to how publishers’ content surfaced in Google search before AI Overviews, and what keywords those publishers were targeting, Levin said.

For example, one publisher’s page would be the main source in Google’s “featured snippet” (the few-line blurb that appears on the search results page). But now, Google’s AI Overviews shows at least three sources in a similar space on the search results page, Levin said.

“You share this space with more people. So if you were the one who was in a featured snippet, then you would probably lose traffic. But if you were the one who was not in a featured snippet, you will probably gain traffic,” Levin said.

Publishers are doing this calculation too. In a Q4 2024 earnings call on Feb. 25, Ziff Davis CEO Vivek Shah said the company was comparing the click-through rates of search queries a year ago to similar ones that now include AI Overviews. Shah said they found that AI Overviews results were present in just 12% of the company’s top queries and the click-through rate didn’t change.

In his Q3 2024 letter to shareholders, IAC CEO Joey Levin said the company, which owns Dotdash Meredith, found that AI Overviews had appeared in about 20% of DDM’s searches but the impact on overall traffic was “minimal.”

However, publishers need to be mindful that this method also has its limitations. While it’s one of the “only measures we have … it’s laced with all the same problems we have with Google already,” which includes a lack of transparency and a volatile search environment, another publishing exec told Digiday, under the condition of anonymity.

The reliability of measuring Google’s AI Overviews’ impact on publishers’ sites by looking only at search keywords “is absolutely questionable,” the exec added.

And the introduction of AI Mode to AI Overviews last week, which keeps users within the AI assistant on the page where they can ask it follow-up questions, could change things yet again if rolled out widely. 

“Let’s be clear — these tools aren’t ‘Search’ engines as we’ve come to know them, they are designed to be powerful ‘Answer’ engines,” said Paul Hood, AI media consultant and former News UK exec. “In my view the future lies not in relying on AI for traffic but in forming alliances that benefit both parties — turning this challenge into an opportunity for growth.”