Answer Bot
Designing the conversation tree for an automated chatbot
The Design Process and My Role
While at the DFINITY Foundation, I was the first and lead administrator of the Support Help Center immediately following launch. One of the projects we implemented during that time was the “Answer Bot”, a support chat bot that would guide users through a series of questions and prompts that lead to technical information, video/written walkthroughs, or troubleshooting guides tailored to their queries and searches. Ideally, this would save the Support team time and bandwidth by minimizing the need for a human touch.
I was one of two individuals developing this chatbot from scratch, a process that required a lot of testing, time, and trial and error. My hand in the process was to not only conceptualize and design the architecture of this bot, but to test and analyze user data as needed, iterating on the design with the context clues tracked by our team and enhancing Answer Bot as time progressed. I gathered user data, then developed the bot around conclusions drawn over the course of 6 months, implementing new features and adjusting direction as necessary. This study walks through two releases of Answer Bot, and discusses some metrics that were tracked with either Google Analytics or Zendesk Explore, from June 2021 to June 2022.
Defining the Process: Zendesk Capabilities
Zendesk chat bots were designed to help end users sift through the content held throughout the website, using a series of multiple choice questions and/or automated suggestions. Our first implementation was through an early stage of Zendesk’s release for this feature, but our second implementation also added the use of Agent Transfers, where users could input their information into predefined forms, and internally-defined keywords, where users could type a word or phrase and be prompted with a more specific tree of questions or article suggestions. Both iterations could close their interactions with an option to turn their conversation into a Support Ticket, where they could add any text description or necessary links/photos. After every conversation, users could be prompted with a CSAT survey, allowing for feedback and tailored improvement.
Identifying Users & Mapping User Flow
Luckily, this project followed the same set of user personas as the general website would, so I had a very clear picture of the types of users that frequented the Help Center. They fell into three major categories: those troubleshooting staking/voting/custody, those seeking development and hardware help and information, and those who are brand new to web3 trying to learn more about what we did at the foundation.
The Answer Bot is primarily focused on helping “the Struggler” or those seeking solutions to various troubleshooting issues.
Initial Key Goals
Given the plethora of articles (75+) on the DFINITY Help Center, Answer Bot aims to supply information for any questions that have already been referenced or answered.
Goal #1: Provide Relevant Information
Goal #2: Guide Solution-Oriented Conversations
Answer Bot’s next step is to help users find an answer to their specific troubleshooting issues, whether that be entirely on its own or with the help of a Support Agent.
Although inevitably there are tickets that will be necessary, given the private nature of financial transactions, Answer Bot tries to alleviate the amount of presence a Support Agent must have on the process. Ideally, the user finds all help needed, and no ticket is necessary. In most cases, we’re aiming to minimize replies between the two parties.
Goal #3: Minimize Need for Support Tickets
Initial Strategy
As stated before, the reorganization of the 50+ (now 75+) articles was the first priority, to best serve our end users.
Usability Testing & Data Analysis
Market Analysis vs ours
Revisions and Strategy
As stated before, the reorganization of the 50+ (now 75+) articles was the first priority, to best serve our end users.
Goal #1: Agent Transfers
Version 2: Improvements
Success rate jumped from 8% -> nearly 20%, no human touch; 30% agent transfer rate, while this and other contributing factors allowed our 1-touch tickets to jump to over 40% consistently WOW
Final Design and Takeaways
Overall, the final design has seen major success. Webpage views rose over time to a peak of over 1500/month, and the amount of pages users read on our help center more than doubled. As more videos and articles have arisen, categories remained successful and subcategories grew as needed. A huge learning opportunity for me, this project was a large undertaking that pushed me to think a lot about the product we were creating and the different ways consumers interact with the Internet Computer, and tailor the experience on our site for the variety of individuals that come seeking resources.
The opportunity to bring some of my web developer experience into a role that didn’t necessarily require that was thrilling and challenging, and I really appreciated the amount of freedom working at a start up gave me in this situation. The redesign of this site required knowledge of content management systems like Zendesk, as well as a foundation in HTML, CSS, and JavaScript for coding the more complex features.