Why is third-party data still often wrong?

In this edition of “Internet Mysteries,” we examine why third-party data still has trouble identifying people’s most basic characteristics.

The problem
At the Digiday Programmatic Summit in November, Matt Rosenberg, then CMO of ChoiceStream, highlighted the imperatives that third-party data providers operate under.

“Advertisers need scale, and as a data vendor, if you can’t provide that, no one will buy your segment,” he said.

But in order to reach scale, accuracy often gets sacrificed. As Rosenberg put it: “If you can get 300,000 people in a group with 95 percent confidence that they belong there, or 30 million people in a group with 60 percent confidence, well, it might not be such a hard decision to relax your model a bit, especially when no one is set up to audit you.”

In one study, ChoiceStream found that a particular data vendor had identified 84 percent of users as both male and female. While that case was an extreme outlier, ChoiceStream also examined the two vendors that were least likely to identify people as both male and female. By getting the third-party data internally from the vendors and syncing IDs across datasets, ChoiceStream found about a third of the time the two vendors disagreed on what gender an individual was. Similarly, Mediasmith research found that data from four of 11 vendors wasn’t much better than chance in targeting age and gender.

Why does this still happen?
According to several sources, these inaccuracies persist because of poor incentives. In a thirsty quest for scale, marketers abandon their concerns about certainty. And middlemen are paid based on their number of transactions, and not upon their accuracy. While advertising’s tech capabilities may have gotten much more sophisticated in recent years, priorities haven’t changed as quickly.

“Unless the agency gets pressure from the brand to hold the data company’s feet to the fire, you get a system that is running on its inertia, but that is not really doing the job it is supposed to do,” said an ad tech exec.

What can be done?
Sources emphasized that the quality of data vendors varies widely — and that their data isn’t always crap.

“The key to navigating this landmine of good and bad vendors is to hire the right marketing agency with the right skills, experience and methodology of their own for determining the highest quality sources of inventory and data,” said Mark Paci, COO of Huddled Masses.

Marcus Pratt, Mediasmith vp of insights and tech, noted that in recent years some vendors have expanded their offerings to include data packages at various levels of scale and accuracy, and that these vendors have been more upfront about which packages have the least accuracy. So if advertisers want more accuracy, they can choose the product that best suits them.

Other sources stressed that advertisers need to more persistent in questioning data firms’ methodologies. One source suggested checking third-party data against proprietary surveys or monitoring targeted campaigns with tools from Nielsen and comScore, though these services also prompt their own fees and tradeoffs.

Some brands can also skirt data vendors by relying on their own first-party data. However, many brands don’t have that much useful first-party data to begin with.

Theoretically, a data management platform (DMP) could allow brands to link their own data to third-party data, which could give them a chance to validate the accuracy of third-party sources. But using a DMP creates another middleman with its own set of incentives and costs, and not all marketers are up to speed on how to gather and interpret data in a DMP to make informed buying decisions.

“The thing is,” a programmatic buyer said, “as an advertiser, you have to ask yourself how many asterisks are you willing to accept.”