OpenAI’s measurement gaps are keeping ChatGPT ads budgets at test level

It’s far from a universal problem, though. Execs from AdRoll, Jellyfish and another independent U.S. ad agency said their execs hadn’t seen click discrepancies. For Web Guide Partner, though, it’s more about the timing of those insights. Where conversions do register on ChatGPT, the agency sees them arrive 24 to 36 hours late, against a few hours on Google. Reporting carries no demographic or prompt data.

These problems only apply to advertisers who have switched conversion tracking on, and many haven’t. Doing so means signing OpenAI’s tracking terms, which legal teams at a lot of advertisers won’t, citing liability and unease about what the company might do with their customer data. One U.S. independent ad agency said that applies to almost all of its clients, across every vertical, though when pressed few could point to the clauses they object to. The agency thinks those brands forget they already hand the same data to Google and Meta. The difference is that advertisers have had years to get comfortable with Google and Meta’s terms, whereas OpenAI isn’t offering much room to shape its own. It told Jellyfish from the outset it doesn’t accept red lines. 

Maybe some of the hesitation comes from marketers confusing conversation tracking with the company training its models on their data, agency execs have mused. Regardless, the same concerns are also slowing the uptake of custom audiences, which require advertisers to upload their customer lists. OpenAI’s conversions API could close some of these attribution gaps, but enterprise clients are slow to implement it and smaller ones often lack the people to set it up properly.  

That leaves OpenAI in something of a measurement catch-22: advertisers need to give it better data for its conversion tracking to improve, but some of the advertisers with the most data, are the least willing to hand it over. And without enough reliable conversion data, OpenAI can’t yet offer the kind of optimization that makes Google and Meta so useful to performance marketers.

“The transition now that we are able to optimize to CPA, rather than focusing on CPC and leading metrics, has been pivotal,” said Jellyfish’s Victor Batista, senior paid media director. “I think with time, as they gain confidence in their measurement suite, we will see further evolution, and platform confidence will improve as more data is analyzed across partners.”

Between tracking that misses conversions and brands that won’t turn it on, many advertisers are still optimizing campaigns toward clicks or impressions. That’s starting to change as conversion-optimized bidding becomes more commonplace, giving advertisers some ability to bid toward cost per acquisition. Jellyfish’s Batista said the bigger shift would come as OpenAI builds out optimization toward metrics such as CPA or return on ad spend, which requires OpenAI to have confidence in its own numbers first. Until then, advertisers are bidding with little to go on. Accuracast gets “low bid” warnings telling it to pay more without saying how much, and those warnings sometimes vanish after a few days, leaving the agency unsure whether to trust them. Delivery is just as hard to read. Accuracast’s marketers saw one campaign serve thousands of impressions one week and fail to reach 100 the next. Even a well-set bid doesn’t guarantee a result advertisers can learn from. 

“The biggest issue we’re seeing is around inventory — it isn’t predictable, and there’s definitely room to serve more,” Divecha said. “There’s no obvious explanation for the change.”

Which is why marketers are craving more third-party measurement on their ads. Indeed, Mile Marker’s vp of biddable centre of excellence Monica Shukla said it was important to establish an independent measurement framework before testing the platform, rather than taking its own reporting at face value.

“One of the things I feel is important, is to establish what our measurement is going to look like before the test, and then go back to it to define your KPIs, instead of just using the platform-first metrics and data at face value,” she said.

How far that gets depends on which partners OpenAI lets in and what data it gives them. It has signed up mobile measurement partners including AppsFlyer and Adjust, which most marketers welcome given so much ChatGPT use happens in the app. That said, the exec from the independent agency said the wave of smaller vendors announcing integrations are all as limited as going direct. Indeed, Accuracast’s Divecha said a verification partner would confirm ads were seen without showing whether they worked.

The cost of these issues shows up in budgets. One exec said they have clients spending $10 million a month on Google, and put less than $100,000 a month into ChatGPT despite wanting to spend more. Accuracast’s Divecha said the unanswered questions stop the agency moving clients from test budgets between £10,000 to £20,000 up to £100,000.

“I think we would need solid verifiable information to know that it works because right now, the way the model works is you’re paying for impressions and visibility,” Divecha said.

OpenAI declined Digiday’s request for comment.