Email newsletters have a measurement problem

Pete Sena, chief creative officer of digital marketing agency Digital Surgeons, said he automatically discounts the subscriber numbers publishers report, knowing they can be inflated by multiple email sign-ups by the same person or ISPs blocking delivery.

“It is a problem,” he said. “I don’t trust that number, because we all know lists come and go. I don’t believe [publishers] overtly lie, but I do believe there’s a little bit of omission of how valid are those emails.”

Not only that, but publishers don’t share information like what device the subscriber is using and what time of day it is when they open the email. “You get very limited metrics,” he said.

Adding to the problem, MailChimp, the company that serves many email newsletters, has technological limits in how it measures open rates, which can lead to underestimates of how much email newsletters are read.

Then there’s the issue of publishers fluffing their subscription numbers by using come-ons like contests, which may yield email addresses if not necessarily loyal readers.

Yet these issues are missing from the frothy coverage of newsletters like TheSkimm, with its “massive subscriber list” (it claims 4 million) and the “exceptional” Lenny Letter (it claims 400,000 subscribers).

Newsletters face something of a chicken-and-egg problem. Right now, buyers are OK with the lack of independent, third-party measurement because they’re not buying newsletters on their own, so there’s little incentive for the publisher to provide it. Newsletters are usually part of a larger ad buy (and sometimes thrown in for free), or a sponsorship, and as such, they’re low risk to the advertiser.

In the case of Quartz, for example, it largely relies on reach and performance data from MailChimp when it comes to measuring its popular email newsletter, the Daily Brief (which it says goes to more than 200,000 subscribers), and clients are fine with that, the publisher said.

Carlos Watson, co-founder and CEO of Ozy, another publisher with a big focus on newsletters, pointed out that measurement of web pages is inflated, and that newsletters offer a way to connect with consumers in a more personal way. Advertisers and publishers do need to be creative to avoid alienating consumers in such a personal medium, though, he pointed out.

As long as there’s no outside audit, newsletters’ financial growth will be limited, though.

“We see them as part of something bigger, and it’s the something bigger we value,” said Doug Rozen, chief digital and innovation officer at OMD. “No advertiser is using newsletters as their own marketing vehicle. It’s not a campaign in and of itself. It’s an added premium.”

The Skimm also benefits from the cool factor; it’s just the kind of “micro content” a lot of advertisers are looking for now, and a lot of young media buyers are readers themselves, said Lisa Cucinotta, director of social strategy and business development for Horizon Media.

Sena, while skeptical of publisher-provided subscriber numbers, says newsletters are worth doing anyway because they’re still a better way to reach a specific audience than a broad-reach campaign, even if the advertiser is reaching fewer people than the publisher would have him believe.

Dave Helmreich, COO of LiveIntent, a leader in newsletter ad sales, with more than 1,300 publisher clients, as expected, said the medium has a lot going for it. The fact that subscribers have to take multiple steps to opt in makes them valuable to advertisers, and the medium is immune to issues like viewability, clutter and ad blocking that dog online display advertising. “There is no ad blocker for email,” he said. He wouldn’t share dollar figures but said LiveIntent’s revenue has been increasing 50 percent or more a year for the past few years, and he expects that to continue.

But he admits there is room for improvement in measurement as buyers demand more data to understand how to value its performance. Demand to measure ad viewability is coming, and the traditional way of doing that on the web doesn’t work in email, for example.

“Every buyer wants to understand how to pay for it, measure it, how it looks compared to things they’ve been doing for years,” Helmreich said.