How The New York Times moderates 12,000 comments a day
Last week, The New York Times announced it would expand the use of automation to open up more articles to reader comments. Using a system called Moderator developed with Jigsaw, the tech incubator from Google’s parent company Alphabet, the Times hopes to expand the number of stories open to comments from 10 percent today to 80 percent by the end of the year. But the tech wouldn’t work without the foundation the Times community desk laid.
That’s because Moderator has been trained by being fed 16 million Times comments that have been hand-moderated over the past 10 years. It uses those judgments to build an algorithm that will prioritize comments for moderation.
The editor of the community desk is Bassey Etim, who leads a team of 14 part- and full-time moderators who review about 12,000 comments a day. It’s a complicated process, so the Times can only open up just so many articles to comments and only for a limited time. Moderators aren’t just screening comments for curses and threats; they’re seeking to create a place where readers can discuss stories, exchange ideas at a high level and see all sides of a debate reflected. Etim calls it “content curation at a very high level.”
The job of the desk has gotten more complicated in the Trump era. There are just more comments, and in this heated political environment, it’s become harder to characterize what’s inflammatory. People have gotten more creative about name-calling those they disagree with, whether it’s “liberal snowflakes” or “followers of the Great Orange,” leaving the Times to figure out how to reflect the culture without letting too much meanness get through.
A lot of how the Times handles these depends on context. It’s OK to say snowflake if someone is discussing liberals generally, but not to attack a specific commenter. “Whereas things like ‘t-Rump’ and ‘Orange Menace’ are never allowed, just like we do not allow ‘DemoRats’ or ‘President O-Bozo,'” Bassey said.
Policing comments can be particularly hard on a day like last week’s congressional baseball practice shooting, which brings up a lot of different hot-button issues. With emotions running high, the risk is that a comment that goes too far will slip by the filters. Despite the high-pressure environment, Etim said the community desk has a low turnover rate, with several staffers who joined the desk before he did in 2008.
The desk is made up of people with a mix of editorial backgrounds. Etim looks for people who have strong knowledge of the news, understand ideas across the political spectrum and are able to put their own biases aside to foster a good discussion because moderators have to be able to handle comments on stories on any topic. He asks candidates to evaluate comments for inclusion and to explain their reasoning.
Still, it takes a full year for new employees to get up to speed, but they eventually develop a shorthand, which is how the Times is able to handle the volume it does by hand. “When you read that many comments, your brain starts to code how people talk; you start to know the commenters. Our moderators are familiar with hundreds, maybe a thousand commenters,” Etim said. The automation will help them get to more comments by filtering them. At the same time, the Times is aware of the possibility of bias creeping into algorithms that can change the tenor of comments.
Many publishers from Popular Science to Quartz have eschewed or given up on comments sections as they’ve become polluted by trolls and more people are sharing their opinions on social media. The Times polices comments on its social accounts, too, where it can’t control the process but it can respond to questions and comments. That job falls under Cynthia Collins, Times social media editor, who has nine social media editors working around the globe. “One of our most talked about stories on Facebook could have up to 5,000 comments,” she said.
But comments can build engagement, and engagement can lead people to becoming paying subscribers, which the Times is increasingly leaning on as ad revenue becomes harder to grow. The Financial Times says readers who comment are seven times more engaged than those who don’t, reading more, spending more time on-site and returning more often.
At such a politically polarized time, Etim sees the comments section as a way for people to be exposed to opposing points of view and challenge them without offending the people holding them. “There’s a lot of mind-changing in the comments. It’s really gratifying because the work we do allows people to have those conversations and understand a little bit of the common humanity that is easy to miss.”
One recent Times article on how climate change science divided a classroom in Ohio generated 857 comments from teachers, students, parents and other citizens. “We were basically hearing from every possible person who could be reading this story,” Etim said. “There was a respectful dialogue. That’s what makes the job really fun.”
Pinterest testing new co-sold, revenue-share ad model for publishers with Tastemade
Currently in an experimental phase, Tastemade is the first publisher to sign on and the brand that is funding this ad experiment is corn chip snack Fritos.
As publishers clean up automated supply chains, education-title Chegg cut ad resellers and saw no negative impact on revenue
"We were told as publishers that resellers were so important, but no [publisher] has communicated to me they removed resellers and lost X% lift."
Member ExclusiveThe Facebook ad boycott could help publishers swing the pendulum back to context
Publishers have a golden opportunity to shift thinking around the role context, broadly defined, should play in advertising.
SponsoredFour ways to adapt to the changing publisher ecosystem in 2020
By Neal Sinno, general manager Americas at GeoEdge For marketers, 2020 started out with so much promise — but this changed rapidly as the industry faced a global epidemic head-on. Not only did our own daily routines come to a screeching halt, for many of us our professional lives did as well. Almost as quickly […]
Patagonia: Boycotting Facebook ads will lead to an ‘even more thoughtful approach’ to its ad buying
Patagonia has pulled all paid ads from Facebook and Instagram until at least the end of July. The boycott will extend if the advertiser has seen three specific changes to how the social network handles hate speech.
How Substack has spawned a new class of newsletter entrepreneurs
As the media ecosystem contracts amid coronavirus, Substack has been thrust into an uncomfortable role — that of a savior.