Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering

Генерация с дополнением поиска на основе графов знаний для ответов на вопросы в службе поддержки клиентов
Zhentao Xu, Missy Meine C. Dela Cruz, Matthew Guevara, Tie Wang, M. N. DESHPANDE, Xiaofeng Wang, Zheng Li, Manasi Deshpande
2024-07-10

customer service question answeringissue tracking ticketsknowledge graphsretrieval-augmented generationsub-graph retrieval
In customer service technical support, swiftly and accurately retrieving relevant past issues is critical for efficiently resolving customer inquiries. The conventional retrieval methods in retrieval-augmented generation (RAG) for large language models (LLMs) treat a large corpus of past issue tracking tickets as plain text, ignoring the crucial intra-issue structure and inter-issue relations, which limits performance. We introduce a novel customer service question-answering method that amalgamates RAG with a knowledge graph (KG). Our method constructs a KG from historical issues for use in retrieval, retaining the intra-issue structure and inter-issue relations. During the question-answering phase, our method parses consumer queries and retrieves related sub-graphs from the KG to generate answers. This integration of a KG not only improves retrieval accuracy by preserving customer service structure information but also enhances answering quality by mitigating the effects of text segmentation. Empirical assessments on our benchmark datasets, utilizing key retrieval (MRR, Recall@K, NDCG@K) and text generation (BLEU, ROUGE, METEOR) metrics, reveal that our method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU. Our method has been deployed within LinkedIn's customer service team for approximately six months and has reduced the median per-issue resolution time by 28.6%.
1
A six-month deployment within LinkedIn’s customer service team reduced median per-issue resolution time by 28.6%.
2
Knowledge-graph integration improves retrieval accuracy and answer quality by preserving service-structure information and reducing text-segmentation effects.
3
On benchmark datasets, the method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU.
4
The proposed method constructs a knowledge graph from historical customer-service issues, preserving intra-issue structure and inter-issue relationships for retrieval.
5
The system parses customer queries and retrieves related knowledge-graph subgraphs to support retrieval-augmented answer generation.

historical customer service technical-support issue tracking tickets and their knowledge-graph representation

retrieval accuracy and answer-generation quality for customer service question answering, including preservation of intra-issue structure and inter-issue relations

Publication Details
Publication Date
2024-07-10
Journal
Publisher
ISSN
Cited by
170
Access Type
Author Information
Authors
Zhentao Xu
Missy Meine C. Dela Cruz
Matthew Guevara
Tie Wang
M. N. DESHPANDE
Xiaofeng Wang
Zheng Li
Manasi Deshpande
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%