CMOs might be pushing ahead on AI, but lack of measurement’s holding them back

CMOs are measuring the benefits of AI across a range of metrics: time saved, payroll cost reductions or consumer sentiment. Each has their pros and cons, but few can be applied across the breadth of a business. And so far, a standard dashboard hasn’t yet emerged.

“We do have measures that give us some sense of how effective it is,” said Cheryl Guerin, evp of brand strategy and innovation at Mastercard. “I haven’t been able to say, ‘Oh, I’ve saved X amount of money because [of AI].” 

Without a common means of backing up the value of AI investments, marketers are taking a more cautious approach to the technology. They’re operating within the bounds of what’s provable, rather than what’s possible.

Dashboard signals 

For Bryan Demaranville, CEO of L.A.-based jewelry brand Melinda Maria, payroll and customer service are the success metrics he’s tracking to determine if the AI juice is worth the squeeze. Demaranville estimates he’s generating savings of $1.2 million in payroll costs annually and a 60% increase in customer service efficiency via AI-powered tools. 

Implementing the tools hasn’t resulted in layoffs, he said, but it has frozen hires and freed up staff from administrative tasks. 

Alicia Hale, svp of growth at B2B SaaS firm Demandbase, told Digiday the company had put AI to use across marketing and sales functions. 

The company’s sales team have used AI tools to speed up elements of the outreach process, allowing the firm to continue growing without having to hire as many new staffers, she said. “We’ve been able to keep our hiring flat on the business development side of the house, because instead of having to go and ask for additional resourcing, we can scale the high-quality personnel that we have,” said Hale.

By contrast, the in-house advertising team’s use of gen AI creative tools has been measured in the volume of assets generated. To put some numbers to it, the platform has saved 366 hours in the last 90 days by using AI writing tool Jasper for content generation. AI has also helped improve the brand’s ad efficiency, allowing DemandBase to reinvest $250,000 back into the platform.

Because of that, the precise cost-benefit ratio across Demandbase’s overall marketing operation remains unclear to Hale and her colleagues. “We’ve all gotten caught up in this AI frenzy that the diligence isn’t being done as well as it could be,” said Hale. “It’s going to be something that we all have to begin to focus on very quickly.”

Zip, a buy now, pay later financial brand, has been experimenting with AI in campaign creation, digital twinning, brainstorming and creative production. Jinal Shah, chief customer officer at Zip, said the biggest measurable difference is the company’s ability to scale leadership and employee productivity. 

Meanwhile, Mastercard has been leveraging AI-powered tools for the last few years. Four years ago, Mastercard launched a proprietary digital marketing engine that listens to social media to identify trends, alerts social media personnel and automatically selects and launches creative campaigns in real-time, according to Guerin. The financial brand also uses generative AI to recommend, regenerate and edit sonic tracks for creative campaigns. 

In all cases, calculating ROI in terms of revenue generation or marketing cost savings, allowing marketers to make apples-to-apples comparisons, has been more difficult than the initial setup. 

“It’s hard to put a value on this right now other than productivity. I have more hours in my day—things like that,” Shah said. 

When it comes to standardization in measurement and impact of AI investments, CMOs and C-suite execs may need a different set of expectations that aren’t directly tied to financial value, Gartner analyst Nicole Greene, said in an email.

“Part of the challenge is that AI can’t be viewed as a straight technology investment,” she said. “You can measure that with operational metrics, but in today’s volatile world we need to move from productivity to proving growth and financial impact.”

In-house slowdown

The lack of clear means of measuring gen AI ROI may be holding marketers back from further deployment of the tech. As of last year, most of the industry still hadn’t fully adopted AI in campaigns, according to the Interactive Advertising Bureau’s Internet Advertising Revenue Report. According to its April report, 70% of agencies, brands and publishers are still only using it in limited ways due to messy data, security concerns and disconnected tools. 

This time last year, buy now, pay later fintech company Klarna made headlines after announcing it had accounted for a 37% cost savings of roughly $10 million per year in its ideation, image creation and translation efforts. This year, however, CEO and co-founder Sebastian Siemiatkowski, told Bloomberg that Klarna is now prioritizing human support, ensuring customers can always speak to a real person, flicking at the limits of artificial intelligence.

Klarna isn’t alone in its change of heart. Through 2027, Gartner predicts 60% of gen AI projects will be abandoned after proof of concept in part due to inadequate AI-ready data, lagging governance, risk controls, steep costs or unclear business value.

The lack of a clear way of measuring their own AI investments has meant that CMOs otherwise keen to in-house marketing capabilities are still reliant on agency partners. Demandbase, for example, still works with creative and PR agencies. Hale said there’s no current plans to reduce their briefs.

“We really see AI as a way to extend the capabilities of our internal team members,” said JJ Kaye, evp and chief marketing officer at Capital Bank. “I’m specifically hiring people now with AI skill sets. [But] for the moment, I don’t see it replacing agencies.”

In-housing remains one of the industry’s key tectonic shifts, and it’s a change that most industry observers expect AI to accelerate. But in the absence of comparable ROI metrics for AI, few marketers can justify cutting out their agency partner—at least for now. (Increasingly, tech titans—namely Meta—have been vying for more marketing spend, rolling out AI-powered tools with plans to fully automate AI ads by 2026.) 

All said, it’s a work in progress as marketers expect the tools to advance and greater efficiency to unfold sooner rather than later. Call it deja vu, in which marketers compare the AI learning curve to that of social media or SEO, said Zip’s Shah.

“As a marketer, I will never fall into that debate about something as fundamental as AI,” she said. “You fundamentally have to accept that this is changing how we work.”