The 2016 US Presidential election elevated the issue of social media bots and “fake news” to an unprecedented level of attention. Yet for all of the headlines it’s an issue that’s both complex and in many ways misunderstood. Continue reading “How Bots Are Threatening Online Discourse”
We’ve all seen the stories and allegations of Russian bots manipulating the 2016 U.S. presidential election and, most recently, hijacking the FCC debate on net neutrality. Yet far from such high stakes arenas, there’s good reason to believe these automated pests are also contaminating data used by firms and governments to understand who we (the humans) are, as well as what we like and need with regard to a broad range of things.
Hayder Casey, Pinterest engineering manager, Growth
More than 200 million people use Pinterest every month to find ideas to try, from recipes to gifts to decor for the home. Family and friends are often a big part of these plans, and that’s why nearly 1M Pins are shared to Facebook Messenger each week. Today we’re making it easier for Pinners to collaborate with others on Messenger by launching a new chat extension and bot.
Researchers estimate we will speak to chatbots more than we speak to our spouses by 2020. Obviously, companies that implement chatbots are doing something right. However, businesses still have a hard time determining whether or not their bots are up to snuff. While there are plenty of effective chatbots on the market, there are also many that don’t quite meet consumers’ needs. So how do you measure the success of your chatbot?
This is the dilemma facing an increasing number of companies that use chatbots as part of their customer experience. 80 percent of businesses want to implement a chatbot by 2020, but many still face the challenge of gauging the efficacy of the technology.
Google’s Chatbot Analytics platform recently opened up to all, but it is still necessary for businesses to develop and understand their own chatbot success metrics to effectively use the platform.
The process of defining the best KPIs for your company’s bot will depend on your business goals and the functions you want your bot to perform.
Here are seven metrics of success you can use to identify opportunities for improvement in your company’s chatbot.
The first question any prospective investor wants to know about a company is whether or not it makes money. Therefore, the best indicator of a chatbot’s value is its financial benefit.
There are many ways to evaluate a bot’s impact on revenue – the best one for your bot will depend on its purpose. Another interesting wrinkle is that your chatbot can have a knock-on effect on a number of areas.
For example, you can measure a customer service bot’s profitability growth by the amount of money it saves the company compared to maintaining a customer service team 24/7. But you will want to take the bot’s impact on customer service into account. If self-service rates are higher and clients are more satisfied, that will result in repeat customers and higher online sales, thus impacting top-line revenue growth.
Nirvana comes for businesses the moment a user gets exactly what they want from the chatbot without any human input.
If your chatbot’s goal is to change a user’s password, you would measure success by the percentage of user interactions that end with this as a result.
The self-service rate closely correlates the cost savings aspect of revenue growth – in other words, how much money did your chatbot save?
What better way to find out exactly how well your chatbot is doing than to ask the very people who use it?
Your chatbot can help you determine this metric by asking the key question for the Net Promoter Score – “On a scale of 1-10 how likely is it that you would recommend our chatbot to a friend/colleague?” As a lead indicator of growth, the NPS provides a crucial foundation for understanding your chatbot’s customer experience performance.
At this point, it’s worth reflecting on AARRR and its importance in measuring the success of your business.
The activation rate in the context of a chatbot refers to when a user responds to its initial message with a question or an answer which is relevant to your business goals.
For example, a chatbot designed to provide you with weather updates would receive an activation rate when you enter your location – thus allowing the bot to provide you with the information.
How can this KPI help? If for some reason people were not responding when the weather chatbot first reached out to them, the botmaster would be able to tinker with it to enable a more satisfactory outcome.
Unfortunately, even bots with the most robust natural language processing are unable to understand everything a user says.
These errors are a useful indicator for measuring whether or not you need to improve your chatbot’s matching.
Bear in mind there are three different triggers, each of which necessitates its own type of response.
There is first the simple confusion from the bot if it cannot understand a comment. A basic “Sorry, I didn’t understand that. Can you ask again in a different way?” response would suffice.
Second is if the user sends a number of messages which are outside the remit of your chatbot. After a couple of attempts, it would be worth programming your bot to relay a message that reminds the user of its exact purpose.
The final trigger is if the bot forces a user to speak to a customer service agent after the interaction. Each of these will tell you something different about how your chat agent is performing.
Once again referring to AARRR, the retention rate represents the percentage of users who return to the chatbot over a specified period of time.
This timespan would vary between the bots depending on their purposes. For example, a fitness chatbot would require daily interaction and would benefit from analyzing its 1-day retention.
Artificial intelligence/machine learning rate
How strong is the AI in your chatbot? You can use the percentage of user questions that are correctly understood to measure this.
Which leads us the million, if not billion dollar question — can my chatbot learn independently?
Chatbots with machine learning can measure progress by comparing the improvement in self-service rate over a period of time without human intervention.
An agent with robust machine learning will be able to continually run its own gap analysis to highlight potential areas of improvement.
The demand for chatbots among Millennials is clear. Consumers are asking for simple and effective customer service, but not every chatbot is capable of delivering on this promise without a few tweaks. In a market that is becoming increasingly crowded, these KPIs can help you keep your chatbot one step ahead of the pack.
Jordi Torras is CEO and founder of Inbenta, an artificial intelligence technology company.
Google’s chatbot analytics platform is now open to everyone, more than six months after its quiet debut during the company’s I/O developer conference. Called Chatbase, it’s intended to help developers better analyze and optimize their bots so they can improve conversion rates and accuracy — and avoid having users feel bots are useless.
Anyone can use Google’s Chatbase for free, similar to Google Analytics, and it’ll work across any platform, including Facebook Messenger, Kik, Slack, Viber, and Skype. But it’s more than messaging services where Chatbase could prove invaluable: With the rise of voice assistants like Google Assistant, Amazon Alexa, Samsung’s Bixby, and Apple’s Siri, understanding analytics will be important.
A product of Google’s Area 120 internal incubator, Chatbase currently has “hundreds” of companies using it, including Ticketmaster, HBO, and Viber. A spokesperson for the Rakuten-owned messaging service said in a statement: “We increased query volume by 35% for a popular stickers bot by optimizing queries with high exit rates. Chatbase has been immensely helpful … instead of combing through logs, we rely on its machine-learning capability to help prioritize required optimizations.”
Ofer Ronen, Chatbase’s team lead, told VentureBeat that since the platform’s early release, Google has learned that “building and analyzing bots can be challenging because the tools are relatively new and still maturing. Unlike websites and apps which are well understood, bot development is still establishing best practices.”
He went on to say: “An aspect that makes bots especially challenging is how open-ended they are: Users expect bots to handle a request containing any phrasing they choose. This is an area that Chatbase is especially focused on, by exposing popular requests to which a bot is not responding well.”
Google isn’t the only one in the analytics space for bots, as it competes against Dashbot, Botanalytics, BotMetrics, Manner, and others. But what might be an advantage to Google is what it’s done with Google Analytics, one of the top analytics tools for mobile and website developers. Ronen added that, besides the extensive array of things that could be tracked, Chatbot’s machine learning capabilities gives it leverage over the competition, clustering “similar problematic user messages. One example would be for finding and fixing ‘misses’, or alternate phrasing of supported actions that weren’t originally anticipated by the developer,” he said.
“Putting some of Google’s machine learning capabilities to work for our users is a clear differentiator, and our users are really excited about that.”
If Google is successful in positioning Chatbase as being platform-agnostic and the service becomes as widely used as its Analytics sibling, then the breadth of data that the company will receive around conversation, be it voice or text, will be enormous. That would not only allow Google to improve its bot ecosystem, but to see a significant boost in the machine learning space. Plus it may eventually lead to helping the company figure out ways to properly monetize bots — using a chatbot version of Google AdWords, perhaps?
Chatbase won’t give you the exact same metrics that you’d expect from a traditional analytics platform, although there are some overlaps. Among the data you’ll receive include the number of active users, sessions, and retention, while also comparing performance by platform.
Anyone can sign up for Chatbase. Those using Dialogflow, the service formerly known as API.ai, will automatically have access to Chatbase’s basic features within Dialogflow.
Bots are gaining lots of attention, thanks to the momentum in artificial intelligence and natural language processing. In fact, according to this Business Insider study, 80 percent of businesses are using, or intend to use, chatbots by 2020. While simplified rules-based bot builders like QNAMaker.ai make entering the realm of bots easy, truly conversational and awe-inspiring brand experiences need to come with a thoughtful and strategic approach. This means that before you jump in and start writing (or hiring out) a single line of code, you’ll want to do some planning. Let’s start with the most obvious question. Continue reading “3 questions marketers must answer before launching a chatbot”
When a person feels sick, they might start deciding whether it seems serious enough to visit a doctor. However, things like having to get to the doctor’s office, long delays in the waiting room, and the potential difficulty in getting an appointment could discourage that individual from getting prompt treatment. A new AI-powered chatbot called Ada could be the perfect solution for that predicament. Let’s take a look at how Ada and other telemedicine offerings could change the future of health care and what downsides the technology has. Continue reading “Chatbots can save you from trying to diagnose that cough yourself”
Facebook Messenger now has a plugin that lets visitors to a website engage in live chat with a human or bot without leaving that website. Called Customer Chat, the plugin is one in a series of major changes announced today as part of the release of version 2.2 of the Messenger Platform. The announcement was made by Messenger head of product Stan Chudnovsky on stage at Web Summit in Lisbon, Portugal. Continue reading “Facebook Messenger brings live chat and bots to websites”
“In its first phase, the Schibsted customer care assistant will be able to answer frequently asked questions and guide the user to the right answer. The main focus area will be aiding users with questions regarding subscriptions and logging them into our platform… The Schibsted customer care assistant is built in collaboration with Bakken & Bæck on its Orbit Artificial Intelligence (AI) platform. The assistant’s framework is built in Python, where the chatbot’s brain is represented as a finite-state machine. “
Quartz got some money from Knight last year to launch its own Bot Studio, building interactive tools/chat interfaces/general bot substrate for both itself and other newsrooms. (More here and here.) Today, Quartz announces the latest fruit of that effort — a Slack bot named Quackbot, built in collaboration with DocumentCloud:
Together we’re releasing Quackbot, which performs tasks useful to reporters, editors, and news producers right where so many of us work all day — inside Slack. In its first version, Quackbot can do a select few tricks that might prove handy in a modern newsroom…But we’re excited to collaborate with the rest of the journalism world to give Quackbot many more skills over time. Think of it as a fully hosted and friendly interface to open-source tools…
Journalist-programmers are an especially sharing lot. Sure, they’ll work night and day to scoop each other, but once the story’s published they’re happy to share how they did it — even sharing the tools they built. As a result, there are many dozens of useful tools available to programmers in newsrooms everywhere.
But there’s a catch: Not every newsroom has programmers. And even existing programmers might not have the time, skills, or resources to get a project’s code, put it on a server, and keep it working.
It’s in an early state, but a few of those launch features might still be useful to you:
1. It can take a screenshot of any webpage.
2. It will preserve any URL by telling the Internet Archive to save a copy of the page.
3. Given a topic, it can suggest some reliable sources of data.
4. If you provide Quackbot with a URL, it will identify any cringe-worthy clichés on that page.
Soon, Quackbot will also allow journalists to upload PDFs to DocumentCloud, extract text and charts from PDFs, monitor websites for changes, make quick charts, and more. We’re also inviting other journalists to bring their tools into Quackbot, making them readily available within Slack.
Once it achieves anatine maturity, “Quackbot will become a core feature of DocumentCloud, which will maintain the infrastructure and provide troubleshooting and support.”
The bot itself will be available for install this Thursday (such teases), and you can bug John Keefe about it at ONA.
Publishers increasingly offer agency services, and Quartz has gone beyond making ads to constructing a chatbot for Hewlett Packard Enterprise.
Named Hugo, the chatbot was incorporated into the branded-content series “Machines with Brains,” focused on how humans, technology and artificial intelligence intersect. Those who clicked through to the bot on their phones could learn more about the stories’ topics covered in the series and how HPE creates technology related to the series.
As seen in the video below, users can select topics like “artificial intelligence,” “cloud computing” and “Internet of Things.” Over a period of six weeks, 117,155 messages were served (after users selected topics) and users spent an average of two minutes with the bot, according to the publisher. Now the bot’s distribution has been widened to include Facebook Messenger, where it will roll out in the next few weeks. Quartz will also run retargeting to encourage return users.
The bot has also evolved to mine relevant articles across the web, not just Quartz and HPE content. The topics have widened to include energy, health care and communications. Users can now also type specific questions, a function that wasn’t on the first version. The bot’s new features are already accessible via Quartz’s U.S. app and will be available in Europe at the start of November. Quartz worked with HPE agency DigitasLBi on the effort.
“There is a lot of wasted time and effort in the current [marketing funnel] structure,” said Sean Mahoney, vp group director at DigitasLBi. “The challenge we gave to Quartz was how do you target the right people, not in a shotgun-blast way but in a way that’s conversational and useful.”
Quartz Creative had 12 people working on the bot, including developers, designers, user-experience specialists and analytics staff. “The process of creating value used to be to create the shiniest objects possible. The new model is to create something that might generate real value,” said Brian Dell, director of Quartz Creative.
For Quartz, bots are another way to differentiate from run-of-the-mill ads, said Jay Lauf, publisher and president of Quartz.
“Advertisers benefit because people spend more time with their messaging,” he said, “and from that, advertisers can learn more about their audiences because they’re explicitly expressing what they’re interested in, so that helps marketers deliver smarter, more relevant experiences.”
Conversational interfaces have become a larger part of our day-to-day lives. Many of us wake up in the morning and talk to Alexa or Google Home about the weather, ask Siri to “call mom” during our commute, and engage with several Slack apps and bots throughout the workday. But as the AI and bot industry matures, developers realize that users do not really care about AI or chat. What they really want is a better way to complete tasks. In this article, I will talk about how to optimize your conversational interface to do just that.
In my bot design book, I coined the term of a “conversational funnel.” Like a web or mobile user funnel, a conversational funnel models and measures the engagement, ease of use, and conversion of a user within a conversational interface.
Do you have an AI strategy — or hoping to get one? Check out VB Summit on October 23-24 in Berkeley, a high-level, invite-only AI event for business leaders.
Let’s imagine the conversational funnel for a bot that helps people buy a new laptop:
In each step of the funnel, users get closer to making the purchase, but fewer and fewer of them progress through the entire funnel. Many drop off — confused, distracted, or just tired of the process.
Never underestimate the power of making things easy to use. Companies like Lyft and Uber, and even Google and Amazon, were built on the foundation of providing an easier way to consume a pre-existing service or product. Conversational interfaces are no different. If they can become an easier, more pleasant way to identify a service or a product, they will be successful. Otherwise, they will fail.
Let’s explore ways to optimize a conversational funnel.
Buttons for simple choices
Here is a very common funnel failure we see with conversational interfaces today:
At this point, the user is already frustrated. They would rather go online and view the report on a web page. It is definitely not a pleasant or easy experience. The user isn’t satisfied, and the bot appears dense and unhelpful. But what if we change the interaction just a bit?
Now it is very clear what the user should do. Taking action is quick and contextual, whether interacting on web or mobile.
Pull-down lists for more choices
This problem of capturing the right information becomes even more complex when choosing from a large list of items:
Again, the user is frustrated. It is hard to spell out a specific item from a large list. Let’s make a small modification:
Now the experience is much easier. For example, Slack menus are auto-complete enabled, so it’s simple for the user to start typing and select the right item from a list. Again, improvements like these make the interaction more productive than the web or mobile app alternatives.
Forms for collecting structured data
When it comes to real-life examples, many bots must collect more structured information, which can be particularly cumbersome to collect with regular text.
Just reading this correspondence might make you tired. Capturing long structured user input is a nightmare when turned into plain conversation. So what should a bot do? Well, this week my team is introducing dialogs, a new Slack feature that allows developers to build interactive forms.
Once the user has clicked on the button, a well-structured form pops up in the conversation.
After the user fills in the form, the conversational interface can change accordingly, keeping the information captured in the context of the conversation.
As you can see, in this example the bot captured the structured data and brought the results back into the conversation, keeping the core value of bots in messaging apps: retaining a quick and contextual workflow.
Dialogs are new interactive modals that can capture multiple pieces of information and send them directly to your bot. You can use them to build a robust form inside Slack, or simply collect a single line of text.
Traditional conversation is a great hammer, but not everything’s a nail. Using rich interactions like buttons, menus, and dialogs make better performing conversational funnels and helps create a more pleasant and productive way to get things done. With these new emerging experiences, we take a large step toward making our lives simpler, more pleasant, and more productive.
Amir Shevat is head of developer relations at Slack.
When bots finally do take over the world, let’s hope they allow us to at least pick our own coffee. We might all be plugged into machines Keanu Reeves-style as in The Matrix, providing the power they need, but maybe we’ll still be allowed to partake of a fine blend intravenously.
Until then, there’s the Crema coffee bot for Slack. I tested it with a group of college students recently, and it works like a digital butler, not only asking everyone to weigh in on the flavors they like and picking the winner, but also shipping out the beans direct to your doorstep.
At first it didn’t seem like machine learning. It was more like machine annoying. With apologies to the fine folks at Crema (which is known for their coffee subscription service), the bot interrupted the team on Slack a few times, asking everyone to vote on coffee. There’s a little bit of hand-holding involved, because you have to select the size you want and configure a few other options at first, like where to send the package.
To be honest, I didn’t mind at all — I’m a coffee snob. The students didn’t really understand why they needed to vote, as evidenced by how long it took them to click a little vote button. Eventually, they played along. The bot collected the winners on its own.
The cool part — which is to say, the part that involves some machine learning — is that everyone also gets to rate the coffee after it arrives, and this rating system then feeds into an algorithm that the bot uses to pick the next batch of flavors. For example, let’s say everyone keeps rating a fruity flavor highly. The bot will make suggestions along those lines.
“The bot just asks a couple of questions upfront to ascertain how much coffee you need, how often,” explained Tyler Tate, the CEO at Crema. “The real team personalization piece starts with the initial voting of which coffee to get first, and then continues as the team later rates that coffee, votes on what to get next, etc. So if you add the Crema Slackbot to a channel everyone has access to, they’ll have the opportunity to vote.”
I really liked how it worked. You don’t have to do anything except vote and rate. There’s some fun involved, thinking back to whether you liked a flavor. And then the coffee just arrives in the mail. You don’t have to head out to Costco ever again.
The Slackbot encourages team dynamics — you can even start lobbying folks in the channel to vote for the blend you like, although the Crema bot doesn’t haven’t anything to do with that.
The bot doesn’t just collect votes and choose the popular one. It uses a concept called collaborative filtering, which might sound familiar if you have ever used Amazon. It’s a comparison between what you like and what everyone else likes. In music, you might say, a bot would recommend Coldplay to U2 fans not just because you listen to U2 constantly and might like Coldplay because of its similar sound, but because a lot of other people in general also listen to Coldplay.
All I can say about the Crema bot is — it worked. We started liking the coffee more and more, and I found the selections seemed to match up with the most people on the team. As is true of every great bot, this one not only acquired the right information but did all of the legwork so we didn’t have to do anything but unpack the coffee and start a new roast.