Now that publishers understand their relationships with their audience are important, they’ve begun asking readers directly what they think of their products and content.
At the beginning of July, Business Insider started adding one-question surveys to the bottom of the paywalled stories it distributes to Business Insider Prime members. Last month, Axios sent an email to newsletter subscribers asking how likely they were to recommend the news publisher’s newsletters to someone they knew (it sent a second out Sunday night), hoping to establish a Net Promoter Score for its newsletters.
The asks for direct feedback, along with more established efforts like The Athletic’s, are part of a yearslong transition away from scale-oriented metrics such as pageviews or unique visitors in favor of numbers that measure the scope of the audience’s engagement with their content. Today, most of that measurement is still done in the background: measuring things like how long site visitors spend on pages, how frequently a story is shared, or whether it converts readers into newsletter subscribers.
But volunteered feedback is different enough that it needs to be gathered and treated differently. Some publishers such as Axios are funneling the data to their product teams, hoping to set a baseline that can be used to see a trend line in how people regard what they make. Others, such as The Athletic, use reader feedback about stories to help figure out how to distribute or package content on a personalized basis.
But in all cases, the feedback publishers solicit from readers is being thrown into a larger stew of metrics. Though a Net Promoter Score might be easy to understand, or a list of the stories readers deemed most valuable is easy to compile, all of the publishers using user-provided feedback say that they cannot use it as a standalone signal that shapes product, editorial or audience decisions.
“I think [this trend] makes sense with revenue, but you’re not just trying to attract drive-by traffic anymore,” said Gwen Vargo, the director of reader revenue at the American Press Institute. “Value is a proxy of if someone’s going to pay for it or not.”
“Because it’s a singular number, it’s easy to understand, but you don’t want it to become the new pageviews of 2019,” Vargo went on, talking about Net Promoter Scores. “You don’t want them to be not looking at other things.”
Though the one-question surveys mentioned above bear superficial similarities, the language of each of publisher’s message differs in a meaningful way.
BI Prime members are asked how “valuable” the story was, while Athletic readers are asked, “What did you think of this story?” Back in 2015, Mic readers that got to the bottom of stories saw a widget that asked, “Was this story worth your time?”
That people were identifying with the icons’ faces made the feedback less valuable for its intended purpose. But the replies remain one of the signals The Athletic uses in its strategy. This year, the site began using the data to figure out which kinds of stories specific readers like, then serving more of those kinds of stories to subscribers using newsletters or mobile push notifications. A reader that prefers biographical back stories on athletes, for example, might be sent more of those than a newsier or more analytical item, using their story feedback as a signal.
The key, Hansmann said, is to measure and analyze lots of signals at once, including comments, scroll depth, time spent, shares and other things. Finding the balance, Hansmann said, without offering specifics on the Athletic’s process, is the key.
“If you over-optimize for one variable, then everything becomes the same,” Adam Hansmann, Athletic co-founder, said. “If you over-optimize for raw clicks, then you get a pile of mock drafts. If you optimize for emoji, then you get positive backstories. No one piece of data is the end-all.”
In the case of BI Prime, the purpose of its survey is to measure what readers find valuable and to reinforce the message that their input, along with their satisfaction with the product, matters.
Claudius Senst, the head of consumer subscriptions at Business Insider, said BI will take time gathering data before using it to inform any product changes. But it has already gotten one unexpected benefit out of it. BI Prime readers who answer are given an anonymous feedback form where they can leave additional details, and many respondents have used that field to share questions that the story they read did not answer. That has yielded several ideas for followup stories, Senst said.
Ad-supported publishers have tried to incorporate that kind of feedback into their strategies too. Mic first put a one-question survey at the foot of stories in 2015, and within a few months of deploying it, people discovered there was little correlation between how valuable the story was deemed and how many pageviews it got, or how often it was shared, one former employee said.
That seemed important enough that a small team began trying to figure out a new aggregate metric that could be shared with Mic’s newsroom, a combination of pageviews, share counts, how valuable the content was deemed. Persistent pressure to grow revenue made it hard to make the project a priority, and a metric was never ultimately shipped, that source said.
As publishers focus more on concepts such as consumer journey or consumer experience, Vargo said she expects publishers will work to gather more of these kinds of insights from their audience. They will have to do so with a light touch. “I always say, ‘Don’t make them work too hard,'” Vargo said.
An earlier version of this story said that BI Prime users who responded neutrally were given a chance to follow up. All respondents are given that opportunity.
‘Halloween is when Christmas ends’: A look at publishers’ pre-Black Friday commerce content playbooks
Publishers' Black Friday coverage plans are starting earlier and earlier but commerce teams are evolving to meet the demand.
How social media managers are coping with the Twitter debacle
Twitter – once a stable and trusty workhorse for social media strategists – now resembles the most wildly unpredictable social platform in the marketing arsenal.
‘A big reset in 2023’: After Big Tech’s mass layoffs, job candidates face intense competition
Recruiters report that 'we've never seen a market quite like this' as tens of thousands of employees flood the market.
SponsoredWhy cookie deprecation is deflating performance and inflating costs for advertisers
With the full deprecation of third-party cookies on the horizon, advertisers and publishers are navigating a challenging and quickly evolving landscape. The sunset of the third-party cookie continues as usage and lifetimes fall. Their deprecation is preventing brands from effectively measuring the effectiveness of media campaigns in real-time at highly granular levels. As the industry […]
Martin Sorrell-backed S4S Ventures, Bertelsmann invest $10M in data asset management outfit as it blends new content, analytics-based marketing for clients
The recent explosion in content has created the need not only for more sophisticated tools to manage it, but better ways to attach data and analytics to the content in order to better optimize it at the right time for the right opportunity.
Member ExclusiveMedia Buying Briefing: Which media will buyers turn to in a soft local market in 2023?
Traditional media including broadcast and print are expected to be hit hard by revenue losses. What will save local from a deeper downward trend next year will be local ad spending on digital, digital out-of-home (OOH) media and connected TV.