Editor’s note: Last weekend was the latest edition of my favorite journalism conference, the International Symposium on Online Journalism in Austin. You can catch up on what you missed through these two epic YouTube videos of the two days’ livestreams.
But there were two talks in particular that I thought Nieman Lab readers might be interested in seeing, from America’s two top newspapers, The New York Times and The Washington Post. Both Andrew Phelps, an editor on the Times’ Story[X] newsroom R&D team, and Joey Marburger, the Post’s director of product, spoke about how they were using bots in their news operations.
Today, we’re publishing transcripts (lightly edited for clarity) of their two talks. Below is Joey’s talk; Andrew’s is over here.
Where basically, like, robots aren’t supposed to kill you — until they try to kill you. Hopefully, a conversational journalism won’t ever try to kill you.
So I developed basically three quick laws. But they’re they’re pretty spot on to the Laws of Robotics, which is: We don’t want to spread false information. It should follow what a human journalist tells it to do, unless the human tells it to spread false information.
We’ve done a lot of experiments on bots. And we’re very excited about it, because it’s this great, simple experience, and the technology is getting so much better for it: AI’s getting better. big data’s more accessible. So we knew we wanted to try a bunch of things and see what’s out there, because it’s kind of hard to have a ton of successes when you’re on the bleeding edge.
I’m going to go over three bots, which are kind of our favorites, but we actually have almost 100 bots actually. Like 99 percent of them are internal, though.
So this is our most successful reader-facing bot: It’s called the Feels Bot. About 30 days prior to the U.S. presidential election, if you opted into it on our politics Facebook page, we would message you in the evening and ask you how you felt about the election. And it was just five emoji responses, from super angry to happy — and we would curate all that. Then in the morning, we would show you a graph of how people were feeling.
We knew that we had to have a cadence in alerting people but not annoying people, because we had already built a bot for that. It was just a general news bot which didn’t do very well — which we figured would happen. Even though there are a billion people on Facebook Messenger, I don’t think anyone’s built a bot that has that many users.
So this was really fun to work on — and it was curated by a human. It had a low user account — like less than 10,000 people. But the engagement — meaning people actually answered the question every day for 30 days — was greater than 65 percent, because it’s simple. It asks you a simple question; it was a very charged election. And, you know, if you ask people how they feel, turns out they’ll tell you, which is great.
So we’d generate these social cards from it and highlight a few. Some of the best responses we’d share on Twitter, put them up on our site. We generated these little graphics out of it were really fun, and we did this every day for 30 days, which is a great exercise. Empathy is a powerful driver in conversation.
Another thing we call our Virality Oracle is a Slack bot in a Slack channel — a public channel inside the Post — that is powered by a really amazing algorithm from our data science team. From the second that a story is published, it starts monitoring it and it knows within the first 30 minutes of publishing if it’s basically going to be viral. (It’s really “popular” — “viral” is kind of a loaded word.) And it notifies the channel, so we can maybe go in and add something of the story, or start writing off of it a little bit. We get about three to five stories like this in a day. And then it also models out a 24-hour traffic window, and then the bot also emails us to digest, so we can see like the lifecycle of stories. This is really a bot as a tool — so it’s like a service bot or utility bot. It’s very handy.
So this is actually the data behind the bot, which I’m not going to go into in super detail. Our prediction model is taking in all these data points — this bot is just eating and gobbling up. We ran it for a long time, almost a year, for the machine learning to get really accurate. And we ran it on every story published — about 300 stories a day.
And we found we’d add in a new metric and it would get a little better. And now we’re at about 80 percent competency.
Whenever a reporter starts a story, they actually put in when they plan to publish it. So what is the deadline — which can always be changed. So if you’re behind, it will tell you: Hey, you’re either really close to deadline or you missed your deadline. It personally messages you — it doesn’t, like, shame in a channel or anything. And it’s really funny when it messages Marty — which I think has maybe happened once.
So this is a pretty cool thing too, called Heliograf, which is another way to think of a bot. It’s not a conversational bot, but it takes in data points from a feed and can basically craft stories very simple short stories, based on templates. Anybody every play Mad Libs? You know, put in a noun, pick an adjective, whatever? This is kind of what that does.
So we used this for the Olympics and for elections. We published a story on every single Olympic event, because of Heliograf. And then for elections, we posted a story on every single race in the U.S. on Election Night, and generated newsletters, generated tweets. We did all sorts of fun stuff from it. So it was a bot that was helping us do better journalism.
Audio bots are super, super huge right now. Amazon doesn’t call Alexa a bot, even though pieces of it inside are a bot. They like to refer to it as like it’s an operating system, as audio AI.
Our politics Flash Briefing was one of our fastest-growing products last year. We caught the wave just right — there’s a reason that the Echo is out of stock on Amazon all the time. They’re actually outselling a lot of their other hardware. Jeff Bezos, our owner, is personally driving this road map, which also gives you an indicator of how successful it is. And it’s super fun.
But what we’re thinking about bots and how it plays into your day-to-day life and your habit is: Bots can do very simple tasks. It shouldn’t do everything, because then you’ve got a lot of cognitive overhead, it’s a lot of work. Sometimes you don’t know what to ask a bot, other than, like, “What’s the news?” So we’re thinking about — the future’s here. You can build these things — and actually now there are a lot of tools, you can build them pretty easily. Amazon has a tool called Lex, which — point-and-click, you can build a pretty robust bot without any code. So the future is here, it’s just not evenly distributed — which is a quote from William Gibson, another science fiction writer. And I think this is super true for bots. Bots aren’t totally new — they’re just getting more accessible. It’s almost becoming a household name.
So we think bots can fill all these spaces between platforms — like, on different platforms, but also they fill in these gaps a little bit between things. A bot could notify you to catch you up on where you left off in a story while you were listening to it on the train into work. You sit down your desktop, and it’s like: “Hey buddy, here’s where you were in that story.” It like fills that space a little bit. This is what we’re starting to work on a lot right now; we’re calling it a handoff bot.
I remember bringing this up in the newsroom — nobody really understood it. “Why would we do this?” Especially when you do the first one and it gets like five people that use it — you’re like, “We got to keep doing it!” And it turns out that you learn a lot from experimenting. When things are really simple and really hard, it’s very attractive to a designer and a product person. So we’ll be we’ll be iterating on bots for a long time to come.