Title: Making Chatbots More Usable
Advisor: Shwetak Patel and Khai N. Truong
Supervisory Committee: Shwetak Patel (Co-Chair), Khai N. Truong (Co-Chair, Univeristy of Toronto), Leah Findlater (GSR, HCDE), James Fogarty, and Kurtis Heimerl
Abstract: Conversational agents, popularly called chatbots, received significant attention in the last two years. The major reason behind the success of these systems is that chatbots use the already familiar conversational interface. However, chatbots are still in their nascent stage: They have a low penetration rate as 84% of the Internet users have not used a chatbot yet. First, we conducted a study with 16 first-time chatbot users interacting with eight chatbots over multiple sessions on the Facebook Messenger platform. Analysis of chat logs and user interviews revealed several major problems with the current chatbots, including (a) mismatch between the chatbot's state of understanding (also called context) and the user's perception of the chatbot's understanding, (b) targeting chatbots specifically towards the Internet-savvy technically-advanced users, (c) limitations in natural language understanding technologies leading to dialog failures, and (d) unreasonably high expectation with chatbots. We focused on these four problems, respectively: (a) Convey: stands for CONtext View, is a window added to the chatbot interface, displaying the conversational context and providing interactions with the context values, which we evaluated with 16 participants; (b) FarmChat: is a multi-modal multi-lingual conversational agent, to meet the information needs of rural low literate farmers, and evaluated with 34 farmers in Ranchi, India; (c) Resilient Chatbot: explores user preferences for eight repair strategies taken from commercially-deployed chatbots (e.g., confirmation, providing options) as well as novel strategies explaining characteristics of the underlying machine learning algorithms, was evaluated with 216 Mturkers; (d) BigBlueBot: is a fun learning experience with the goals of teaching players about how chatbots work and generating feelings of empathy toward chatbots to increase desire to interact with them, which was evaluated with 88 MTurkers. To summarize, we propose several ways to make chatbots more usable.