Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering
Генерация с дополнением поиска на основе графов знаний для ответов на вопросы в службе поддержки клиентов
2024-07-10
SCID: 54.1/kw86kgf2
Discuss with AI
customer service question answeringissue tracking ticketsknowledge graphsretrieval-augmented generationsub-graph retrieval
Figures from the paper
Abstract (AI)
In customer service technical support, swiftly and accurately retrieving relevant past issues is critical for efficiently resolving customer inquiries. The conventional retrieval methods in retrieval-augmented generation (RAG) for large language models (LLMs) treat a large corpus of past issue tracking tickets as plain text, ignoring the crucial intra-issue structure and inter-issue relations, which limits performance. We introduce a novel customer service question-answering method that amalgamates RAG with a knowledge graph (KG). Our method constructs a KG from historical issues for use in retrieval, retaining the intra-issue structure and inter-issue relations. During the question-answering phase, our method parses consumer queries and retrieves related sub-graphs from the KG to generate answers. This integration of a KG not only improves retrieval accuracy by preserving customer service structure information but also enhances answering quality by mitigating the effects of text segmentation. Empirical assessments on our benchmark datasets, utilizing key retrieval (MRR, Recall@K, NDCG@K) and text generation (BLEU, ROUGE, METEOR) metrics, reveal that our method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU. Our method has been deployed within LinkedIn's customer service team for approximately six months and has reduced the median per-issue resolution time by 28.6%.
Key Findings
1
A six-month deployment within LinkedIn’s customer service team reduced median per-issue resolution time by 28.6%.
2
Knowledge-graph integration improves retrieval accuracy and answer quality by preserving service-structure information and reducing text-segmentation effects.
3
On benchmark datasets, the method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU.
4
The proposed method constructs a knowledge graph from historical customer-service issues, preserving intra-issue structure and inter-issue relationships for retrieval.
5
The system parses customer queries and retrieves related knowledge-graph subgraphs to support retrieval-augmented answer generation.
Research Object
historical customer service technical-support issue tracking tickets and their knowledge-graph representation
Research Subject
retrieval accuracy and answer-generation quality for customer service question answering, including preservation of intra-issue structure and inter-issue relations
Publication Details
Publication Date
2024-07-10
Journal
Publisher
ISSN
Cited by
170
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest