AI-assisted learningcustomer-support agentsgenerative AI conversational assistantheterogeneous treatment effectsworker productivity
Figures from the paper
Abstract (AI)
Abstract We study the staggered introduction of a generative AI–based conversational assistant using data from 5,172 customer-support agents. Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers. The effects vary significantly across different agents. Less experienced and lower-skilled workers improve both the speed and quality of their output, while the most experienced and highest-skilled workers see small gains in speed and small declines in quality. We also find evidence that AI assistance facilitates worker learning and improves English fluency, particularly among international agents. While AI systems improve with more training data, we find that the gains from AI adoption are largest for moderately rare problems, where human agents have less baseline experience but the system still has adequate training data. Finally, we provide evidence that AI assistance improves the experience of work along several dimensions: customers are more polite and less likely to ask to speak to a manager.
Key Findings
1
AI assistance facilitates worker learning and improves English fluency, with particularly strong benefits for international agents.
2
AI assistance improves work experience: customers become more polite and are less likely to ask to speak to a manager.
3
Access to a generative AI conversational assistant increases customer-support agents' productivity by 15% on average (issues resolved per hour).
4
Gains from AI adoption are largest for moderately rare problems, where human agents lack baseline experience but the AI has sufficient training data.
5
Productivity and quality effects are heterogeneous: less experienced and lower-skilled agents gain both speed and quality improvements, while most experienced/high-skilled agents gain small speed increases but small declines in quality.
Research Object
Generative AI–based conversational assistant deployed to customer-support agents
Research Subject
Effects of AI assistance on agent productivity, output speed and quality, heterogeneous impacts by experience/skill, worker learning and English fluency, problem-type dependence of gains, and customer interaction outcomes
Publication Details
Publication Date
2024-12-29
Journal
Publisher
ISSN
Cited by
844
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Beyond Computation: Information Technology, Organizational Transformation and Business Performance2000
Sparks of Artificial General Intelligence: Early experiments with GPT-42023
Artificial intelligence, firm growth, and product innovation2023
Generative AI at Work2023
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models2023