SEARCHFORMER: Semantic patent embeddings by siamese transformers for prior art search
SEARCHFORMER: семантические представления патентов с использованием сиамских трансформеров для поиска уровня техники
2023-04-05
SCID: 54.1/sjucnyhs
Discuss with AI
BM25 rankingpatent prior art searchpatent-specific BERTsemantic patent embeddingssiamese transformers
Figures from the paper
Abstract (AI)
The identification of relevant prior art for patent applications is of key importance for the work of patent examiners . The recent advancements in the field of natural language processing in the form of language models such as BERT enable the creation of the next generation of prior art search tools. These models can generate vectorial representations of input text, enabling the use of vector similarity as proxy for semantic text similarity. We fine-tuned a patent-specific BERT model for prior art search on a large set of real-world examples of patent claims, corresponding passages prejudicing novelty or inventive step, and random text fragments, creating the SEARCHFORMER. We show in retrospective ranking experiments that our model is a real improvement. For this purpose, we compiled an evaluation collection comprising 2014 pairs of patent application and related potential prior art documents. We employed two representative baselines for comparison: (i) an optimized combination of automatically built queries and the BM25 ranking function, and (ii) several state-of-the-art language models , including SentenceTransformers optimized for semantic retrieval. Ranking performance was measured as rank of the first relevant result. Using t-tests, we show that the achieved ranking improvements of the SEARCHFORMER over the baselines are statistically significant ( α = 0 . 01 ).
Key Findings
1
An evaluation collection of 2,014 patent-application and potential-prior-art document pairs was compiled for retrospective ranking experiments.
2
SEARCHFORMER fine-tunes a patent-specific BERT model using patent claims, novelty- or inventive-step-prejudicing passages, and random text fragments for prior-art retrieval.
3
SEARCHFORMER improved the rank of the first relevant result compared with optimized query-plus-BM25 retrieval and state-of-the-art semantic language-model baselines.
4
The model represents patent text as vectors and uses vector similarity to support semantic prior-art search.
5
The ranking improvements over all baselines were statistically significant according to t-tests at α = 0.01.
Research Object
patent applications and potential prior art documents
Research Subject
semantic relevance and ranking performance for prior art retrieval
Publication Details
Publication Date
2023-04-05
Journal
Publisher
ISSN
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest