Ad execs enter crucial phase of Google’s Privacy Sandbox experimentation

Meanwhile, other ad tech firms, preferring anonymity, told Digiday their own evaluations are slated to start next month. They’re doing this because it’s all part of playing by the U.K.’s Competitions and Markets Authority and Google’s rulebook on sandbox testing.

During this crucial stretch, ad tech vendors are focused on the Protected Audiences API, the Topics API, and the Attribution Reporting API. Their attention will be particularly honed on getting a feel for how valuable these alternatives could be for publishers and advertisers.

For publishers, this means keeping a close eye on ad prices — specifically, how those prices change and why. Ideally, Criteo and Adform want to attribute price changes to the effectiveness of the ads, not just spending fluctuations.

If a publisher’s ad price is looking good because more advertisers are participating and paying higher prices for ad space, that seems positive. However, if the overall demand from advertisers drops significantly during a testing period because their ads aren’t bringing in enough returns (ROAS), then even with a good eCPM, the publisher will end up making less money.

Since the testing period involves only a small portion of ad traffic, (one percent to be precise) – it will take some time for advertisers to accurately measure how effective their campaigns are and adjust their budgets accordingly. This process could take weeks or even months.

As a spokesperson for Magnite explained: “It is accurate that we will not make any changes to the core technical set up that impacts the auction mechanics during the testing period. If for some reason we had to do so, we’d restart the test to have eight clean weeks of data.”

That’s why this sustained testing period is crucial.

“If you don’t have a predictable amount of demand going to traffic in Chrome that has Privacy Sandbox labels then it’s impossible really to see statistically significant results on CPMs over time,” said Criteo’s chief product officer Todd Parsons. 

The same goes for advertisers. Keeping ad prices stable is critical.

This ensures that ad tech vendors can accurately assess the effectiveness of advertising using those APIs in generating revenue. The more stable these prices are, the better they can prevent any influence from changes in ad performance that might otherwise lead advertisers to shift their budgets around.

“This is really about making sure that we can drive campaign outcomes as measured by constant return on ad spend,” said Criteo’s svp of go-to-market Nola Solomon. 

At the end of the period, whatever those outcomes may be, they won’t be conclusive.

Especially when they’re based on a limited amount of traffic without third-party cookies and don’t fully capture the Sandbox’s potential depth and breadth. Remember, the Sandbox is only partially ready. For instance, purchasing occurs on a CPM basis, rather than the more detailed cost-per-acquisition or cost-per-click metrics.

Despite these limitations, ad execs finally have something tangible to work with. It’s a chance for them to gain valuable insights on how to tackle a complex issue that has been clouded in uncertainty and speculation. And yes, that includes the U.K.’s Competition and Markets Authority.

The regulator wants ad tech vendors testing the sandbox to submit results by June 15 based on several weeks of continuous testing where no changes are made to the test environment, including not adding new publishers or other vendors into the test over the period. 

“Our plan is to run our stable test starting April 1 and to run it for all of April and May,” said a senior ad tech vendor, who exchanged candor for anonymity. “We’ll then take two weeks to compile our results to submit to the CMA. We hear other ad tech companies have similar plans and timing.”

Given this, nothing they get from these tests has to be conclusive; it just has to be informative. That’s sufficient for now.

“The sandbox system is only partially ready for optimization,” said Jochen Schlosser, chief technology officer at Adform. “Evaluation will be done on reach and CPM trading, not on the more focused use cases around minimizing CPA or increasing ROAS where — according to market perception — the downstream activation of Sandbox campaigns will happen.”

Ever since Google took the ax to those third-party cookies on one percent of traffic in Chrome, companies like Criteo and Adform have been preparing for this moment. They’ve been diligently setting up the technical infrastructure, ensuring they receive the correct sandbox labels, and debugging their technology — all in anticipation of this so-called “stable testing period.”

“We want to be able to see that the budgets flowing through the Privacy Sandbox perform for advertisers’ return on ad spend as an outcome, and publishers’ CPM as an outcome,” said Parsons. “It’s about seeing whether that balance can be achieved. That’s fundamental to the way we think here at Criteo.”

There aren’t many other companies that can say the same. 

If there were, there would likely be more momentum around the sandbox. That’s not to say Criteo and Adform are the only ones entering this key testing phase — clearly, they’re not.

But it’s more like a trickle of activity than a tidal wave. Some ad tech vendors didn’t even know what the stable testing period was when asked by Digiday. 

Crazy as this sounds, it’s also understandable. 

Developing the tech to support these tests isn’t cheap nor easy, requiring heaps of cash and time that many ad tech vendors can ill-afford to waste, especially on something that has so many question marks over it.

“Whilst many of us might love the Privacy Sandbox to work perfectly so other browsers could take on the same workflow and integrate to enable everyone to use the same privacy first solutions — it feels naïve to think that will happen in the short term,” said Tom Bottomley, head of programmatic at digital marketing agency Journey Further. “As an industry we gravitate towards standardization, but maybe the antidote to cookieless is not a one answer solution.”