A Re-evaluation of Knowledge Graph Completion Methods

Переоценка методов дополнения графов знаний
Yiming Yang, Zhiqing Sun, Partha Pratim Talukdar, Soumya Sanyal, Shikhar Vashishth
2020-01-01

Evaluation biasEvaluation protocolKnowledge graph completionLink predictionReproducible code
Knowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs.A vast number of state-of-the-art KGC techniques have got published at top conferences in several research fields, including data mining, machine learning, and natural language processing.However, we notice that several recent papers report very high performance, which largely outperforms previous state-of-the-art methods.In this paper, we find that this can be attributed to the inappropriate evaluation protocol used by them and propose a simple evaluation protocol to address this problem.The proposed protocol is robust to handle bias in the model, which can substantially affect the final results.We conduct extensive experiments and report performance of several existing methods using our protocol.The reproducible code has been made publicly available.
1
Extensive experiments reassess several existing KGC methods under the proposed protocol, with reproducible code publicly released.
2
Reported breakthroughs in knowledge graph completion are partly attributable to inappropriate evaluation protocols rather than genuine model improvements.
3
The paper proposes a simple evaluation protocol designed to correct evaluation problems and provide more reliable comparisons among KGC methods.
4
The proposed protocol is robust to model bias, which can substantially influence final performance results.

Knowledge graph completion methods

Evaluation performance and protocol robustness, including bias effects, of existing knowledge graph completion methods

Publication Details
Publication Date
2020-01-01
Journal
Publisher
ISSN
Cited by
171
Access Type
Author Information
Authors
Yiming Yang
Zhiqing Sun
Partha Pratim Talukdar
Soumya Sanyal
Shikhar Vashishth
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%