A team of Emory University students won Amazon’s 2020 Alexa Prize, a global competition to create the most engaging chatbot to advance the field of artificial intelligence. The team earned $500,000 for taking first place with their chatbot named Emora.
The students designed Emora as a “social companion” to provide comfort and warmth to people interacting with Amazon’s voice-activated Alexa-enabled devices, whether they wanted to discuss movies, sports and their pets or their concerns for themselves and their families amid the COVID-19 pandemic.
The Emory team consisted of 14 students in Emory’s Laney Graduate School and Emory College of Arts and Sciences, led by graduate student Sarah Fillwock and faculty advisor Jinho Choi, assistant professor in the Department of Computer Science.
They christened their chatbot Emora because it sounds like a feminine version of “Emory” and is similar to a Hebrew term for a sage skilled in eloquence. Even as they celebrate their win, the Emory team is looking ahead to how they can apply the concepts they developed to benefit everything from education to people suffering from depression and social isolation.
“Congratulations to the team from Emory for their impressive work in making conversations between humans and Alexa more engaging,” said Prem Natarajan, Amazon vice president of Alexa AI’s Natural Understanding organization. “We are delighted that for the third year in a row, the winning team has set a new Alexa Prize record in terms of average ratings from users.”
The annual Alexa Prize, launched in 2016, challenges university students to make breakthroughs in the design of chatbots, or social bots — software apps that simplify interactions between humans and computers by allowing them to talk with one another. Emory used a unique strategy to beat out nine other universities and take the top spot in this year’s competition, the most hotly contested ever.
“Normally people think of a chatbot as being an intelligent assistant, to answer questions or provide a customer service,” Fillwock says. “We designed a more socially oriented chatbot that could actually show interest in an individual user and provide comfort to people if they wanted it.”
‘Emora cares for you’
The contest began with a beta phase in September. The Emory team consisted of 11 graduate students and three undergraduates. All of the students were members of either Choi’s Natural Language Processing Lab or the Intelligent Information Access Lab of Eugene Agichtein, professor of computer science.
“From the beginning, our team slogan has been ‘Emora cares for you,’” Choi says. “We didn’t set out to design technology that would impress people. We aimed to satisfy people. We also wanted to support human relations, especially family relations. We had components that encouraged people to talk about their family members and to keep in touch with them as much as possible.”
Fillwock strongly agrees with Choi’s philosophy that technology should serve to strengthen human bonds. While she was the official captain of the Emora team, her de-facto co-captain was James Finch, also a graduate student in Choi’s lab. The pair became friends at age 10 in their hometown of Grand Blanc, Michigan, when they met in a middle school math class. By age 13 they were officially a couple and they have been together since.
“When it works, it works,” Finch says.
The pair became engrossed in computer programming during high school. They attended Michigan State University together, working side by side in a lab aiming to program robots to speak more naturally with humans, their shared passion. They chose to come to Emory for graduate school because of Choi and Agichtein.
“Their work is complementary,” explains Fillwock, who began her graduate career in the Agichtein lab and is now with Choi’s group. “Together they form a powerhouse for research into natural language processing and intelligent data mining.”
When Fillwock and Finch finally got married last year they honeymooned in Seattle, so that afterwards they could attend an Amazon bootcamp to prepare Alexa Prize contestants. Each university team received a $250,000 research grant, Alexa-enabled devices and other tools, data and support from Amazon.
Striving to make the world a better place
Choi’s sense of purpose helped carry the team through the year-long project.
“Jinho begins every lab meeting with the question, ‘Why do we do natural language processing research?’” Finch says. “The correct answer is always, ‘To make the world a better place.’ He is an amazingly selfless human being who bases his actions on what is best for his students and society. He’s like a catalyst for positive development and I emulate him in every way that I can.”
Choi’s enthusiasm is also infectious.
Growing up in Seoul, South Korea, he knew by first grade that he wanted to design robots that could converse well with humans. “I read a lot of storybooks about robots and futuristic technology,” he recalls. “I asked my parents, ‘How come there aren’t talking robots all around us?’ They explained that the technology wasn’t yet advanced enough. From then on, I decided to be one of the people to help develop it.”
Conversational artificial intelligence is one of the most challenging problems in computer science. It’s one thing to feed a software system reams of facts that a chatbot can use to answer questions. It’s much more difficult to enable a chatbot to converse naturally with people.
Due to the time constraints, the Emory team chose a well-established technology, known as state machine, as the base for its platform. State machine is a behavioral mathematical model similar to a flowchart. Depending on what people say to the chatbot, the machine makes a choice about what path of a conversation to go down and keeps doing that based on probabilities for what most interests the person.
They quickly developed their own state-machine model, which gave the team more time to learn what people wanted to talk about, research those topics, and test and implement the best strategies to keep them engaged.
“That’s where the magic happens,” Finch says.
Chatbots that can continue conversations
They could also integrate machine-learning models to boost the system’s accuracy. Graduate student Ali Ahmadvand developed an intent-topic classifier while graduate student Zihao Wang built a response ranker. Other graduate students who contributed machine-learning models to the project include Ha He, who worked on semantic parsing; Liyan Xu, who focused on co-reference resolution; and Jiaying Liu, who did event extractions. Some of the Agichtein lab team members had competed in previous years in the Amazon contest, bringing experience that gave Emory another edge.
Individual team members developed chatbot content based on their interests. Graduate students Sergey Volokhin, Harshita Sahijwani, Ingyu Choi, Xiangjue Dong and Zihan Wang focused on movies, news, COVID-19, pets and travel, respectively. Ruixiang Qi, who graduated with a bachelor’s degree in May, and senior Chenxi Xu created a sports component. Instead of just spewing out facts about a topic, the students created conversational scripts. For example, the sports component started out with stats and then asked peoples’ opinions about their favorite teams and players.
Alexa users volunteered to test out the competing chatbots, which were not identified by their names or universities. A chatbot’s success was gauged by user ratings and how long users chose to continue conversations. Teams used this feedback to keep improving their chatbots.
“We had fun with it and tried some quirky topics that people likely hadn’t discussed with a chatbot before,” says Fillwock. She developed content about teleportation that proved popular.
Even more ordinary topics, such as families, were sometimes given a fun spin. Content focused on younger users, for instance, asked them to share silly stories about their siblings.
Emora had hundreds of conversations daily with users over the course of the competition. “We saw overwhelmingly that people wanted to talk with the chatbot in a human way, sharing experiences and opinions, rather than just getting information,” Finch says.
Responding to COVID-19
When COVID-19 started heating up, in mid-March, the team immediately responded. The platform they developed gave them the flexibility to adapt existing content and also quickly add COVID-19 to the conversation.
“Most people were not seeking objective information about the coronavirus,” Fillwock says. “They responded much more positively to questions about how they were doing and how the virus was affecting them and those around them. We tried to provide a supporting, encouraging environment for people.”
The strategy earned Emory a spot amid the five finalists in May. Ultimately, none of the competing teams earned a composite score of 4.0 from the final judges to take this year’s Grand Challenge Alexa Prize of $1 million. The Emory team’s average rating of 3.81, however, netted them the first-place prize of $500,000. A team from Stanford University came in second with an average rating of 3.17.
The Emory team is now looking at real-world applications for what they learned through the contest. One idea is a chatbot to help children develop reading comprehension by conversing about stories.
“Many parents are essentially homeschooling their children right now,” says Choi, who has two sons, ages 2 and 4. “My wife and I love to read to our children and discuss the lessons in stories with them, but we can’t do that all day. A chatbot might help children understand the wisdom in stories and how they can apply it to improve their lives and relationships.”
A longer-term goal is to develop chatbots to support people with depression and other forms of mental illness. “People with depression often try to avoid talking to others and may become socially isolated,” Choi says. “A chatbot could potentially help them gain confidence and improve their mood so they could better engage with people.”
Technology can divide people from one another, Choi notes, citing cases of people who become addicted to computer games.
The Emory team’s mission, however, is to help bring people together. “We cannot avoid technology,” Choi says. “It will continue to be here and to advance. We want to advance it in ways that help out humankind.”