Do Multilingual Users Prefer Chat-bots that Code-mix? Let's Nudge and Find Out!

Предпочитают ли многоязычные пользователи чат-ботов, способных к смешению языков? Давайте мягко подтолкнём их и выясним!
Monojit Choudhury, Anshul Bawa, Pranav Khadpe, Pratik Joshi, Kalika Bali
2020-05-28

code-mixinghuman-in-the-loop chatbotlanguage proficiencymixed-method user studymultilingual chatbots
Despite their pervasiveness, current text-based conversational agents (chatbots) are predominantly monolingual, while users are often multilingual. It is well-known that multilingual users mix languages while interacting with others, as well as in their interactions with computer systems (such as query formulation in text-/voice-based search interfaces and digital assistants). Linguists refer to this phenomenon as code-mixing or code-switching. Do multilingual users also prefer chatbots that can respond in a code-mixed language over those which cannot? In order to inform the design of chatbots for multilingual users, we conduct a mixed-method user-study (N=91) where we examine how conversational agents, that code-mix and reciprocate the users' mixing choices over multiple conversation turns, are evaluated and perceived by bilingual users. We design a human-in-the-loop chatbot with two different code-mixing policies -- (a) always code-mix irrespective of user behavior, and (b) nudge with subtle code-mixed cues and reciprocate only if the user, in turn, code-mixes. These two are contrasted with a monolingual chatbot that never code-mixed. Users are asked to interact with the bots, and provide ratings on perceived naturalness and personal preference. They are also asked open-ended questions around what they (dis)liked about the bots. Analysis of the chat logs, users' ratings, and qualitative responses reveal that multilingual users strongly prefer chatbots that can code-mix. We find that self-reported language proficiency is the strongest predictor of user preferences. Compared to the Always code-mix policy, Nudging emerges as a low-risk low-gain policy which is equally acceptable to all users. Nudging as a policy is further supported by the observation that users who rate the code-mixing bot higher typically tend to reciprocate the language mixing pattern of the bot. These findings present a first step towards developing conversational systems that are more human-like and engaging by virtue of adapting to the users' linguistic style.
1
A mixed-method study with 91 bilingual users evaluated chatbot policies for code-mixing across multiple conversation turns.
2
A nudging policy—subtly cueing code-mixing and reciprocating only when users mix—was equally acceptable across users, but offered lower potential gains than always code-mixing.
3
Multilingual users strongly preferred chatbots capable of code-mixing over a chatbot that remained exclusively monolingual.
4
Self-reported language proficiency was the strongest predictor of users’ chatbot preferences.
5
Users who rated the code-mixing chatbot more highly typically reciprocated its language-mixing pattern, supporting reciprocal adaptation as a design strategy.

bilingual users interacting with multilingual conversational agents (chatbots)

user preferences and perceived naturalness of chatbots that code-mix or reciprocate users’ code-mixing, including the effects of nudging policies and language proficiency

Publication Details
Publication Date
2020-05-28
Journal
Publisher
ISSN
Cited by
43
Access Type
Author Information
Authors
Monojit Choudhury
Anshul Bawa
Pranav Khadpe
Pratik Joshi
Kalika Bali
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%