SemEval-2021 Task 11: NLPContributionGraph - Structuring Scholarly NLP Contributions for a Research Knowledge Graph

Задача 11 SemEval-2021: NLPContributionGraph — структурирование научных вкладов в области обработки естественного языка для исследовательского графа знаний
Jennifer D’Souza, Sören Auer, Ted Pedersen
2021-01-01

NLP contribution graphSemEval-2021 shared taskresearch knowledge graphscholarly NLP contributionsscientific term and relation extraction
There is currently a gap between the natural language expression of scholarly publications and their structured semantic content modeling to enable intelligent content search.With the volume of research growing exponentially every year, a search feature operating over semantically structured content is compelling.The SemEval-2021 Shared Task NLPCONTRI-BUTIONGRAPH (a.k.a.'the NCG task') tasks participants to develop automated systems that structure contributions from NLP scholarly articles in the English language.Being the firstof-its-kind in the SemEval series, the task released structured data from NLP scholarly articles at three levels of information granularity, i.e. at sentence-level, phrase-level, and phrases organized as triples toward Knowledge Graph (KG) building.The sentencelevel annotations comprised the few sentences about the article's contribution.The phraselevel annotations were scientific term and predicate phrases from the contribution sentences.Finally, the triples constituted the research overview KG.For the Shared Task, participating systems were then expected to automatically classify contribution sentences, extract scientific terms and relations from the sentences, and organize them as KG triples.Overall, the task drew a strong participation demographic of seven teams and 27 participants.The best end-to-end task system classified contribution sentences at 57.27% F1, phrases at 46.41% F1, and triples at 22.28% F1.While the absolute performance to generate triples remains low, in the conclusion of this article, the difficulty of producing such data and as a consequence of modeling it is highlighted.
1
Participating systems classified contribution sentences, extracted scientific terms and relations, and organized them into knowledge graph triples.
2
Seven teams with 27 participants took part; the best end-to-end system achieved 57.27% sentence F1, 46.41% phrase F1, and 22.28% triple F1.
3
The SemEval-2021 NLPCONTRIBUTIONGRAPH task introduced automated structuring of NLP scholarly contributions for research knowledge graph construction.
4
The released annotations covered three granularities: contribution sentences, scientific term and predicate phrases, and phrase-based triples.
5
Triple generation remained substantially more difficult than sentence and phrase processing, highlighting challenges in creating and modeling contribution knowledge graphs.

NLP scholarly article contributions in English

The automated structuring of contribution sentences, scientific terms, predicates, and relations into research knowledge-graph triples

Publication Details
Publication Date
2021-01-01
Journal
Publisher
ISSN
Access Type
Author Information
Authors
Jennifer D’Souza
Sören Auer
Ted Pedersen
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%