WTF are shadow blocklists and how they can take a bite out of publisher revenue
Shadow blocklists aren’t as widely known, or openly controversial, in the industry as keyword blocklists, but they could just as easily damage publisher revenue during a market downturn where any fall off could have tough consequences.
The set-up of a shadow blocklist is complex, and as its name suggests, slightly murky. Here’s an explainer on what we know about them, how they function and their potential threats.
WTF is a shadow blocklist?
The first thing to know is that shadow blocklists aren’t a literal list. They are a catch-all term to describe those instances where a buyer — in a bid to avoid funding rogue publishers — uses algorithms to do so without letting the publisher know the reason. A publisher either knows that a buyer has stopped bidding on their inventory but doesn’t know why or they don’t know it’s happening because the ad tech vendors they sell through aren’t flagging the issue. Either way, publishers lose money.
Tech site XDA Developers recently saw revenue drop 30% overnight when Google’s demand-side platform DV360 stopped buying ads on its site, said Boris Shterev, head of marketing at PubGalaxy, the ad management firm the publisher hired to fix the problem. The investigation started in October and it took five months of trial and error before Google’s DSP started bidding again in February. Even then, the investigators couldn’t say for sure what they did to get off the shadow blocklist, said Shterev.
Why are shadow blocklists a tough nut to crack for publishers?
Many publishers aren’t close enough to demand-side platforms to ask why they’ve stopped bidding on inventory. Getting those answers isn’t straightforward, as XDA Developers’ execs discovered. “Google may own both sides of the marketplace, but they’re separate entities so a publisher can’t just go to the side they sell ads through and ask why the demand side isn’t working,” said Shterev.
And even if publishers were able to talk to both sides of Google’s marketplace, there’s a chance that they still wouldn’t be able to. “Google doesn’t want to give out the algorithm for how their systems work because people would find ways to cheat it,” said Shterev.
Shadow blocklists aren’t just a Google problem, however. It’s just easier for publishers to spot when Google does stop bidding on their ads because it accounts for a large slice of their online ad revenues.
“It’s why whenever we’re analyzing drops in performance for publishers we’re looking at all the different factors that could impact it because sometimes the impact [of shadow blocklists] isn’t visible,” said Mirela Kolarova, manager of yield and account management teams at PubGalaxy.
What should I do if I find my ads are being inexplicably blocked?
The first thing to do is to check that the site isn’t doing anything visibly wrong or nefarious to win ad revenue. Its a long and arduous process that saw PubGalaxy testing a medley of theories, from analyzing bot traffic coming to the XDA Developers site, to getting agencies to bid on traffic in order to see how the pages were being labeled by Google’s DSP.
Eventually, the investigators found what they think caused the problem. The two companies that managed the affiliate and comment aspects of the site were also serving ads on it as part of the deal. That meant that whenever Google — or another ad tech vendor — decided not to bid on ads sold by those businesses because the page contained objectionable content they were sent a notice. Usually, those notices are passed on to the publisher. But neither the affiliate or comment companies usually help sell publisher inventory and so hadn’t thought to pass those notices on to XDA Developers, said Kolarova. They could have been receiving those notifications “for years” and not know it, she said.
“Once the publisher was aware of the page violations they resolved everything,” she said. “We had a day of no page violations and then after that, we got our demand back.
Start with who is selling
It’s a problem that could be stunting revenue for many unsuspecting publishers. In fact, there are two other publishers working with PubGalaxy on similar investigations, said Kolarova. Other ad tech execs say the XDA Developers investigation spotlights concerns on the publisher’s side. The term is new, said one exec, but the problem is a “real thing.”
“Ultimately, the story of XDA was over monetization and a lack of governance and controls on who is selling,” said John Donahue, CEO at media consultancy WLxJS. “If a publisher is concerned over who is buying them, they should start with who is selling them and place governance on that.”
Should advertisers care?
It depends on how much an advertiser cares about the impressions they’ve won and lost. Most just aren’t close enough to those impressions for shadow blocklists to be a concern now. As Shterev explained: “I don’t think many advertisers really know how to evaluate the sites they’re buying from. They rely on platforms to do it for them.”
New app launches through Apple hoping to win with ‘zero-party data’ when others haven’t
Caden's new app lets users connect data from their Uber, Amazon, Netflix and other accounts in exchange for money. Will it take off?
‘The next level for us’: The New York Times eyes longer play sessions for games in subscription drive
The games division is focusing on finding new ways to mine the inherent competitive nature of games like encouraging people to play multiple games in a single session or through new achievements and rewards for progression.
Digiday+ Research: Video ads are a growing business for publishers large and small
Video advertising's potential rings most true among small publishers, while data points to video advertising already reaching its potential among large publishers — but also not, at the same time.
SponsoredIn a cookieless world, publishers are embracing new approaches to personalized UX
Asaf Shamly, CEO and co-founder, Browsi With user experience at the forefront of many publishers’ minds, the eventual deprecation of third-party cookies is bound to wreak havoc for those who haven’t quite figured out how to adjust their ad model to the coming change. The problem is well defined at this point: They can’t afford, […]
How The Guardian’s Luis Romero is selling the legacy U.K. publication in the U.S.
The Guardian U.S.'s Luis Romero is working to grow the U.K.-based publishers' ad business in North America while combating the pervasive brand safety problem facing all news publishers.
In graphic detail: Publishers’ full year 2022 earnings
Looking back at 2022, the hits to publishers' revenue were partially staunched, but by the end of the year nearly all areas of the business felt the impact of the economic downturn.