A Re-evaluation of Knowledge Graph Completion Methods
Переоценка методов дополнения графов знаний
2020-01-01
SCID: 54.1/22ntjtnv
Discuss with AI
Evaluation biasEvaluation protocolKnowledge graph completionLink predictionReproducible code
Figures from the paper
Abstract (AI)
Knowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs.A vast number of state-of-the-art KGC techniques have got published at top conferences in several research fields, including data mining, machine learning, and natural language processing.However, we notice that several recent papers report very high performance, which largely outperforms previous state-of-the-art methods.In this paper, we find that this can be attributed to the inappropriate evaluation protocol used by them and propose a simple evaluation protocol to address this problem.The proposed protocol is robust to handle bias in the model, which can substantially affect the final results.We conduct extensive experiments and report performance of several existing methods using our protocol.The reproducible code has been made publicly available.
Key Findings
1
Extensive experiments reassess several existing KGC methods under the proposed protocol, with reproducible code publicly released.
2
Reported breakthroughs in knowledge graph completion are partly attributable to inappropriate evaluation protocols rather than genuine model improvements.
3
The paper proposes a simple evaluation protocol designed to correct evaluation problems and provide more reliable comparisons among KGC methods.
4
The proposed protocol is robust to model bias, which can substantially influence final performance results.
Research Object
Knowledge graph completion methods
Research Subject
Evaluation performance and protocol robustness, including bias effects, of existing knowledge graph completion methods
Publication Details
Publication Date
2020-01-01
Journal
Publisher
ISSN
Cited by
171
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai6
WordNet: a lexical database for English1995
Knowledge Graph Embedding by Translating on Hyperplanes2014
DBpedia – A large-scale, multilingual knowledge base extracted from Wikipedia2015
Convolutional 2D Knowledge Graph Embeddings2018
Holographic Embeddings of Knowledge Graphs2016
TransG : A Generative Model for Knowledge Graph Embedding2016