Digiday Research: Attack of the chatbots

This research is based on unique data collected from our proprietary audience of publisher, agency, brand and tech insiders. It’s available to Digiday+ members. More from the series →

This is the third part of a research series on a set of emerging technologies in media and marketing. Read our other reports on virtual reality and augmented reality.

The 2016 election was a hotbed of media experimentation, and The New York Times was no wallflower. To cover the last 19 days of the race, it built a Facebook Messenger bot engineered to deliver insight from political reporter Nick Confessore.

Digiday Research surveyed 172 executives from media and marketing companies to uncover their approaches to chatbots — and the development of the market. Key findings:

  • About 47 percent of publishers and 31 percent of marketers use chatbots.
  • Fifty-two percent of publishers and marketers say chatbots are mostly deployed as information resources.
  • Facebook Messenger is the chatbot’s natural habitat, with 88 percent of both publishers and marketers deploying them on that platform.
  • According to marketers, chatbots conduct less than 20 percent of consumer interaction before passing the conversations off to human support.
  • The proportion of chat requests fulfilled and the number of chats initiated are the dominant ways chatbot performance is measured now.

The Times editorial team wrote a “choose your own adventure”-style script each day to simulate conversation with Confessore. Text prompts guided users through a pre-structured dialogue. “It was fully automated, but originally created by a human,” said Andrew Phelps, a member of Story[X], the Times’ innovation division. “So people still really identified with Nick the reporter, and it felt like a personal experience.”

Chatbots like these dot the publishing landscape: About 47 percent of publishers surveyed have used them. But they’re still far from intelligent.

“The illusion that HAL is out there, and the machine is alive is just that: an illusion,” said Derek Fridman, global executive experience director at Huge. “There’s machine learning taking place and algorithms making decisions, but in most cases, we’re scripting sequences.”

That’s fine for now, since 52 percent of media and marketing professionals say chatbots are predominantly employed as information resources.

“We’re really pushing brands to focus on utility, things that consumers are going to want to use on a regular basis,” said Adam Simon, director of strategy at IPG Media Lab.

“Anything that makes my life more convenient, I’m willing to give you everything up to my fingerprint for,” agreed Fridman. “With chatbots, it’s all, ‘Give me, give me, give me.’ I want to make a request. I want something back right away.”

When FT Labs created a chatbot integrated with the site search capability, they were doing just that — two years too soon. “We had to explain to users what [Slack] was,” said Chris Gathercole, head of the Financial Times’ FT Labs, of the now-ubiquitous workplace chat platform that housed the bot. “The project ran out of steam; we tried too early.”

But Gathercole is more optimistic about the future, especially around emerging voice environments: “That’s a good back end to support conversations.”

Educating users about new platforms is a persistent, chronic challenge both media and marketing professionals feel when pushing emerging tech. That’s why an overwhelming proportion of both publishers and marketers (88 percent) deploy chatbots on the most familiar platform of all: Facebook Messenger.

“We tend to make bets where we see opportunities to provide the best value for users in a space that’s familiar to them, but can be made new through innovation and technology,” said Brian Dell, director of Quartz Creative.

There are other clear reasons to employ chatbots in closed, private-message settings, especially given Microsoft Tay’s highly public, highly toxic Twitter malfunction. “A chatbot makes a lot more sense as a closed, one-to-one session, rather than broadcasting all of the problems consumers are having to everyone at all times,” said Michael Lebowitz, CEO of Big Spaceship.

This potential for public failure is vital to grasp, given that 38 percent of marketers who use chatbots use them for customer support purposes.

Even then, humans step in when the situation calls for a true conversationalist. Most marketers (91 percent) say the chatbots themselves conduct less than 20 percent of consumer interaction, passing the conversations off to human support when things go off-script.

The technology simply isn’t far along enough to absorb the brunt of customer interaction. Too many conversational dead ends (or misunderstandings) cause most unsupervised chatbot chats to spectacularly fail the Turing test.

This is fine, as long as marketers acknowledge the problem (and keep the conversations fairly private). “That would not be an error of the technology, but of putting too much trust in the system,” said Lebowitz.

Still, the ability to at least initiate many conversations points back to chatbots’ main strength: scaling one-to-one interactions. Forty-one percent of media and marketing professionals say chatbots have high or the highest potential in this area.

This is also reflected in how chatbot success is currently measured: 38 percent of marketers say the proportion of chat requests fulfilled, while 33 percent say it’s the number of chats initiated. Publishers aren’t so split: 66 percent say it comes down to the number of chats initiated. The focus, so far, is on connections, not engagement, reflecting a still-dominant industry preference for impressions.

But that doesn’t mean marketers are going all in on utility at the expense of entertainment with their bots. “Where it gets more interesting is how chatbots could become part of larger narratives,” said Lebowitz. “Years ago, we took over digital for the Skittles brand, and the conceit of our [Facebook] presence was that all of the posts would be issued from the character of The Rainbow. Could the brand be issuing funny-enough content in a one-to-one or one-to-few setup that it would be worth bringing that personification to a chatbot?”

According to IPG Media Lab, yes. The lab worked with ‘90s fizzy alcoholic hit Zima for its 2017 comeback, programming a bot emerged freshly from millennials’ halcyon days with no awareness of contemporary trends or current events. To assess its success, IPG Media Labs is looking past impressions to compare how the chatbot performs as a channel relative to other campaign channels.

“How does this perform on a cost basis against your mobile site or your Instagram feed,” said Simon. As for why consumers would even be interested in a branded chatbot without much utility, “it’s the same reason people are on Facebook in the first place: They’re killing time in line at the grocery store.”

Both media and marketing professionals still have a lot to learn about this new set of conversational interfaces, but all of the information they’re capturing through these early experiments will help accelerate the process.

“At the end of the day, we’re seeing our clients just want to collect as much data as possible about what their consumers’ wants, dreams and needs are,” said Fridman. “As the machine becomes more aware, we can begin to leverage all that data to be a hell of a lot smarter in the next generation from an AI standpoint.”


More in Marketing

The era of the in-depth brand and gaming creator partnership has arrived

To reach gamers outside of video games, brands have moved beyond one-off activations based on specific intellectual properties toward more fully integrated programs that span across all aspects of a creator’s community and fandom.

Companies seem determined to make everything a retail media network. How did we get here?

Brands are leveraging retail media to push the boundaries of where and how we can shop. How did we get here?

Sifting through ‘the noise’: AI tools for HR are evolving fast – here’s how to catch up

Like with all emerging tech, sorting the useful from the useless, is critical and time-consuming.