WTF are shadow blocklists and how they can take a bite out of publisher revenue
Shadow blocklists aren’t as widely known, or openly controversial, in the industry as keyword blocklists, but they could just as easily damage publisher revenue during a market downturn where any fall off could have tough consequences.
The set-up of a shadow blocklist is complex, and as its name suggests, slightly murky. Here’s an explainer on what we know about them, how they function and their potential threats.
WTF is a shadow blocklist?
The first thing to know is that shadow blocklists aren’t a literal list. They are a catch-all term to describe those instances where a buyer — in a bid to avoid funding rogue publishers — uses algorithms to do so without letting the publisher know the reason. A publisher either knows that a buyer has stopped bidding on their inventory but doesn’t know why or they don’t know it’s happening because the ad tech vendors they sell through aren’t flagging the issue. Either way, publishers lose money.
Tech site XDA Developers recently saw revenue drop 30% overnight when Google’s demand-side platform DV360 stopped buying ads on its site, said Boris Shterev, head of marketing at PubGalaxy, the ad management firm the publisher hired to fix the problem. The investigation started in October and it took five months of trial and error before Google’s DSP started bidding again in February. Even then, the investigators couldn’t say for sure what they did to get off the shadow blocklist, said Shterev.
Why are shadow blocklists a tough nut to crack for publishers?
Many publishers aren’t close enough to demand-side platforms to ask why they’ve stopped bidding on inventory. Getting those answers isn’t straightforward, as XDA Developers’ execs discovered. “Google may own both sides of the marketplace, but they’re separate entities so a publisher can’t just go to the side they sell ads through and ask why the demand side isn’t working,” said Shterev.
And even if publishers were able to talk to both sides of Google’s marketplace, there’s a chance that they still wouldn’t be able to. “Google doesn’t want to give out the algorithm for how their systems work because people would find ways to cheat it,” said Shterev.
Shadow blocklists aren’t just a Google problem, however. It’s just easier for publishers to spot when Google does stop bidding on their ads because it accounts for a large slice of their online ad revenues.
“It’s why whenever we’re analyzing drops in performance for publishers we’re looking at all the different factors that could impact it because sometimes the impact [of shadow blocklists] isn’t visible,” said Mirela Kolarova, manager of yield and account management teams at PubGalaxy.
What should I do if I find my ads are being inexplicably blocked?
The first thing to do is to check that the site isn’t doing anything visibly wrong or nefarious to win ad revenue. Its a long and arduous process that saw PubGalaxy testing a medley of theories, from analyzing bot traffic coming to the XDA Developers site, to getting agencies to bid on traffic in order to see how the pages were being labeled by Google’s DSP.
Eventually, the investigators found what they think caused the problem. The two companies that managed the affiliate and comment aspects of the site were also serving ads on it as part of the deal. That meant that whenever Google — or another ad tech vendor — decided not to bid on ads sold by those businesses because the page contained objectionable content they were sent a notice. Usually, those notices are passed on to the publisher. But neither the affiliate or comment companies usually help sell publisher inventory and so hadn’t thought to pass those notices on to XDA Developers, said Kolarova. They could have been receiving those notifications “for years” and not know it, she said.
“Once the publisher was aware of the page violations they resolved everything,” she said. “We had a day of no page violations and then after that, we got our demand back.
Start with who is selling
It’s a problem that could be stunting revenue for many unsuspecting publishers. In fact, there are two other publishers working with PubGalaxy on similar investigations, said Kolarova. Other ad tech execs say the XDA Developers investigation spotlights concerns on the publisher’s side. The term is new, said one exec, but the problem is a “real thing.”
“Ultimately, the story of XDA was over monetization and a lack of governance and controls on who is selling,” said John Donahue, CEO at media consultancy WLxJS. “If a publisher is concerned over who is buying them, they should start with who is selling them and place governance on that.”
Should advertisers care?
It depends on how much an advertiser cares about the impressions they’ve won and lost. Most just aren’t close enough to those impressions for shadow blocklists to be a concern now. As Shterev explained: “I don’t think many advertisers really know how to evaluate the sites they’re buying from. They rely on platforms to do it for them.”
Member ExclusiveGlobal Publishing Summit Recap: prepare for a world without cookies
If you were listening in to our Global Publishing Summit on February 24-26, you’re probably still processing the wealth of insights, tips and strategies shared by all the speakers we heard from over the course of the summit. If you weren’t able to attend, fear not: we’re here to distill the key talking points from […]
Member ExclusiveMedia Buying Briefing: Black-owned media companies step to the forefront of the upfront
Mediabrands is bringing together a raft of its biggest clients — about 20 Black-owned and Black-targeted media players — for an Equity upfront.
‘They’re solving a problem’: The Washington Post readies its Zeus platform for the buy side
Zeus currently offers more than 4 billion impressions per month across its participants' owned-and-operated sites.
SponsoredHow publishers are maximizing retention after the COVID-19 subscription surge
Michael D. Silberman, senior vice president of strategy, Piano For many publishers, 2020 was a good year for subscriptions, and the trend has continued into 2021. For example, over the last month, The New York Times grew active news subscriptions by 48%, and Insider has doubled its subscriber base to just over 100,000 in the […]
Why independent Black-owned media companies are not participating in agency multicultural marketplaces
Advertising agencies are launching new multicultural PMPs but some Black-owned media companies are refraining from joining.
‘We don’t have visibility’: Google’s ad targeting limits expose publishers with reliance on open programmatic market and first-party data weakness
Most digital publishers connect to Google’s ad tech in some way, but those reliant on open programmatic ad exchanges, and without robust first-party data solutions, could be hurt by Google's data decisions.