Lydia Lai <lydialai@stanford.edu>
Pai-Rou Chen <pairouc@stanford.edu>
Gek Siong Low <geksiong@cs.stanford.edu>
Mak Imamura <imamura@stanford.edu>

CS 377B Final Project Report
English conversation practice agent for non-native speakers

Introduction

Many people from non-English speaking countries come to the United States to work or study. Although many of them are taking English language classes, online or otherwise, we feel that what they lack is an American friend with whom they can converse. For our project, we implemented an adaptive learning guide that can fill in the role of a helpful and patient conversation partner, and to explore the effectiveness and possibilities of such a system.

Our target users are adults from the age of 25 to 35 who are non-native speakers of English, and who have been learning English for several years. We therefore assume that the users already have rudimentary knowledge of the language, and are able to communicate (somewhat) in English. However, they do not have many opportunities to practice their conversational skills, especially with a native English speaker, outside of the classroom.

Design Goals

The goal of our learning guide is not to teach our users all the gory details of English grammar, nor is it to give them drills and exercises – these are adequately and probably much better handled by real-life teachers and existing language web sites.

The goals of our learning guide are:

·        To help the users gain more confidence in communicating in English

·        To expand their knowledge of American culture

·        To improve their conversational and social interaction skills with Americans through conversations with the learning guide

A conversational practice partner should not be lecturing to the learner like a teacher, so instead of fixed rigid lessons, users learn about American culture indirectly by interacting with the learning guide. To make the project feasible to implement in one quarter, we focused on conversations about Americans holidays. Future versions will be able to converse about other topics so as to provide a much more realistic user experience.

The pedagogical role of the learning guide is a friend whom the user can chat with. The character is active and dynamic – he likes sports and reads widely.  He is humorous, enthusiastic, helpful, easygoing, talkative, and bright.  He is also very patient, but if the learner insults him, he will be offended and insist upon an apology. Because of his dynamic personality, he is a very fun person to talk to. Users will find conversing with him a pleasant and stimulating experience.

Creative Focus

We decided that a focus on the curriculum and backstory was most appropriate for our character. We wanted to have enough content so that the conversation will be interesting and varied. We also selected a few holidays to go into greater depth so as to be able to sustain a longer interaction with the user.

To achieve our creative focus, we included more stories for each holiday and for the backstory of the character. We tried to author the dialogs in such a way as to encourage user responses, and added a lot of appropriate top-level and after-dialog keywords to handle likely user responses so that the conversation feels more natural, and less like talking to a story-telling robot. The character will also sometimes ask the user about the holidays directly

Valentine’s Day, Christmas and Thanksgiving were chosen to be special holidays to discuss in depth, and so have more stories than the other holidays. These stories are also more personal and rich, and provide for greater interaction with the user. We also authored some questions where some analysis of the user response is attempted, and appropriate replies generated, to give the illusion that the character does care about what the user says, instead of giving some generic response such as “I see” or “That’s interesting”.

We’ve also beefed up the small talk section in the backstory, and enhanced the sports and politics sections, moving them out to become proper stops, so as to provide some interesting alternatives to just solely discussing American holidays.

To provide a more relevant user experience, we also attempted to make our own animated character, because none of the existing characters look correct for our purposes. This wasn’t in our original creative focus.

Design Experience

We created the guide based on our design goals: helping English-as-a-second-language learners practice English conversation and conveying American culture by talking about American holidays.  To make the guide more convincing to the users, we decided our guide would be an American, a native speaker of English, at first. First we started to build his personality and backstory. Because the guide is supposed to be a human being and close to the target audience’s age, the work was pretty straightforward – trying to create human-like stories and personality to show that he is like a real person in the real world. We made the character a Stanford PhD student in American culture to provide a pretext for discussing the holidays. Unless someone tries to insult him, he is a person who likes to make friends and is friendly, talkative, and humorous.  Thus, he can earn the users’ trust so that the users will be willing to interact with him and be confident of talking to him.

Next, we started to build our curriculum. The structure of our curriculum is using alternative dialogs. Taking Valentine’s Day as an example, the basic design blueprint for each holiday is as follows: 

Basically, each holiday is a loop, consisting of only 2 steps. Step 0 has many dialogs about the holiday experience and information.  If the user wants to change the topic, he can take an active role by asking the guide to talk about something else. The guide will also ask the user if he would like to change the topic after some time. The guide then changes his behavior depending on the response he got from the user. We chose ten holidays to work on. Based on information we found from some web sites and our own personal experiences, we created the content by means of stories, either personal stories or history, about the holidays, yes/no questions to lead the interaction, and questions with follow-ups that provide general responses to user input. Therefore, the curriculum not only introduced the holidays but also incorporated how Americans actually celebrate these holidays into the stories.

In the process of building the application, we encountered some problems. First, we found that sometimes the guide would mispronounce some words. Also, since none of our group members is a native speaker of English, the guide’s English didn’t feel American enough. Therefore, we later decided to change our persona; the guide became a Taiwanese-American who immigrated to the States when he was 15 years old. This change provided a reasonable explanation for his mispronunciation and the way he talks. Because of this change in persona, we decided to create our own Asian-looking character animations. After we finished creating and revising most of the content, we also had two native speakers of English go over the dialogs to improve the English  of the content. Another problem was the character’s adaptivity and responsiveness, which will be discussed in the next section.

Results of User Testing

First, we want to mention that the user testing evaluations may have some bias because most of the testers were native speakers of English. Their comments may not fully represent our target audience. Other than the classmates’ evaluations, we also had some other non-native speakers to test the application. Most of the testers gave us positive feedback on the idea of using holidays as a framework to talk about American culture. They also liked our content and the guide’s personality. A tester said, “John’s personality was a great match for the task - I found him to have the most visible personality of any of the applications. His behaviors were appropriate (and funny, too).” However, most of them also felt that the guide was not so responsive and that he was not adaptive enough to the users.

From the users’ feedback, it seemed that the guide was more successful in the area of teaching American culture than in practicing English conversation with the users. Therefore, our group picked three holidays to work on the guide’s responsiveness. We added as many after-dialog keywords as possible to deal with users’ possible questions. We also shorten the length of the stories because some users’ complained that the character rambled on and on without pause. To adapt to users’ different levels of English, we added 8-second delay between each piece of dialog for the higher-level users to have time to type in questions or comments and for the lower-level users to comprehend what the guide just said. We also included possible vocabulary words into keyword sets. Thus, whenever the users encounter some difficult words that they didn’t understand, they can ask the guide.

In terms of our creative focus on curriculum, we think our curriculum successfully conveyed American culture by talking about the holidays and did a fairly good job on interacting with the users by asking them various questions. In addition, some of the holidays go in great depth for both content and adaptivity, such as Christmas, Valentine’s Day, and Thanksgiving. Other holidays also have a great amount of content such that we believe they are complete enough for the users’ to learn about these holidays. A user in Taiwan said, “The guide really taught me a lot about American culture and it was fun to talk to him.”