acceptodds
Under review as a conference paper at ICLR 2027

PAGE: Patent Assessment with LLM-Extracted Technical Feature Graphs

Abstract

General-purpose large language models (LLMs) have demonstrated remarkable capabilities in language understanding, reasoning and generation across a wide range of domains. Patent assessment, however, remains difficult for them, because they struggle to understand the complex structural information embedded in patent claims. We define patent assessment as three critical tasks that an examiner performs on a patent: novelty evaluation, prior-art retrieval and patent classification. Novelty evaluation is the most demanding of the three, because establishing a novelty conflict requires that every technical feature of the patent under examination, together with its properties, is covered by the prior art. Considering that patent claims are inherently written with a hierarchical structure, we propose PAGE (Patent Graph Examiner), a two-stage framework that addresses all three tasks using a unified representation. PAGE first devises an LLM to convert claims and prior-art texts into directed technical feature graphs, whose nodes represent technical features and typed edges encode composition, interaction, taxonomy and property relations. A lightweight graph neural network then encodes each graph using frozen text embeddings of features and relations, edge-aware graph attention, and attentive readout. Task-specific heads map the resulting graph embeddings to predictions for the three tasks: a Siamese interaction head for novelty evaluation, a contrastively trained dual encoder for prior-art retrieval, and a multilayer perceptron classifier for patent classification. Experiments on PatentMatch and CLEF-IP datasets show that PAGE effectively improves patent novelty evaluation and is equally effective for prior-art retrieval and patent classification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Models, Graph Representation Learning, Natural Language Processing, Patent Assessment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8FCkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PAGE separates patent-text processing from downstream prediction. An open-weight LLM extracts directed technical-feature graphs using four relation types, and a trained graph encoder combines frozen MiniLM node/edge embeddings with TransformerConv layers and attentive pooling. Task-specific models perform PatentMatch X/A classification, prior-art passage retrieval, and eight-section IPC classification on a CLEF-IP sample. The same graph schema and encoder architecture are reused, with encoder parameters trained separately for each task according to Appendix C. The strongest configuration reports 0.77 F1 for X/A classification versus 0.70 for a supervised BERT cross-encoder, and 0.641 retrieval MAP versus 0.445 for matched fine-tuned PatentSBERTa. The IPC macro-F1 improvement over fine-tuned MiniLM is much smaller: 0.628 versus 0.625.

Critical strengths
  1. Encoding components, properties and their relations fits the restrictions and nested structure of patent claims. Each unique text is extracted once and its graph is cached, so downstream models can reuse it. The prompts, schema and algorithms make the pipeline clear, and the results with five extractors show the approach does not depend on one LLM.
  2. Splitting PatentMatch by connected components keeps identical claims and passages from crossing splits. The fine-tuned PatentSBERTa retrieval baseline matches the objective, hard negatives, effective batch size, epoch budget and model-selection rule, so the 0.196 MAP gain is a real result. The supervised BERT comparison is also more informative than the large gains over directly prompted LLMs.
Critical weaknesses
  1. The method mostly integrates existing parts: LLM decomposition followed by graph learning appears in GraphEval, graph models for patent assessment in Gao et al., and feature-level analysis in FiNE-Patents, while TransformerConv, attentive pooling, Siamese features and InfoNCE are standard. The ablations do not show that the directed relational graph drives the gains, because replacing raw text with extracted feature spans changes the input as well as the structure, and the R-GCN and mean-pooling variants still keep the graph. A supervision- and capacity-matched classifier on the same extracted features, plus edge-removal or relation-shuffling controls, is needed, and the gap to few-shot LLMs cannot be credited to graphs alone since PAGE is trained on labeled data.
  2. Schema validation only shows that the output is well formed: there is no semantic audit, omission or error rate, fallback frequency or sensitivity analysis, although one missed restriction breaks the all-features coverage criterion and cross-references are skipped when parent claims are unavailable. The claims need bounding, since EPO category X can mean novelty or single-document inventive step, retrieval covers only 11,053 examiner-cited passages, classification uses eight broad IPC sections, and the classifier outputs a label without an evaluated feature-to-evidence alignment. The results have no uncertainty estimates, which matters for the 0.003 IPC macro-F1 margin, and no extraction cost is measured, so together with the incremental method and incomplete controls I recommend 2: weak rejection.
Questions for authors

1 What does the technical-feature graph add beyond existing LLM-to-graph and feature-based patent analyses, and can you compare against the same extracted node and relation texts and supervision without relational message passing, plus edge-removal or relation-shuffling controls?

2 What verifies semantic graph quality (numeric restrictions, negation, relations), including expert audits, invalid-output and single-node fallback rates per extractor, the effect of extraction errors, the handling of dependent claims without parents, and evidence that coverage is examined feature by feature rather than inferred from pooled embeddings?

3 Does the PatentMatch subset separate X citations for lack of novelty from those for single-document inventive step, how is the overlap of families and applications across splits handled beyond exact-text separation, and what confidence estimates support the main comparisons, especially IPC classification?

4 Please give the numeric training and extraction settings, the release plan for cached graphs and split indices, and the measured extraction and inference costs, and distinguish reusing one graph schema from sharing one learned encoder across tasks, since Appendix C trains separately.

Reviewer_rrMnOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PAGE, a framework for automating three patent assessment tasks: novelty evaluation, prior-art retrieval, and patent classification. It first uses an LLM to convert patent claims and prior-art texts into technical feature graphs. These graphs are then encoded using a GNN, followed by task-specific heads: a Siamese interaction head for novelty evaluation, a contrastively trained dual encoder for prior-art retrieval, and an MLP for patent classification.

Critical strengths
  1. The paper addresses novelty evaluation, prior-art retrieval, and patent classification using the same graph extraction framework. This shows the reusability of the representation.
  2. The method achieves substantial improvements over the baselines.
Critical weaknesses
  1. The paper mentioned novelty is decided feature by feature, but the proposed method only compares graph-level embeddings using an MLP. So the method relies on graph-level similarity rather than feature-level novelty checking, and it cannot identify which feature is missing.

  2. Patent novelty can depend on a small but significant improvement over existing inventions. The proposed graph-level comparison may overlook subtle differences in individual features. Here, novelty is reduced to matching one claim fragment against one passage. A passage can cover part of a claim while missing its actual new contribution. Also, dependent claims lose the features they inherit from the parent claim.

  3. Novelty evaluation is performed on predefined claim–passage pairs from PatentMatch where an examiner has already cited the prior art. The method does not retrieve prior art itself, and in practice, novelty-destroying documents may exist beyond those cited; therefore, retrieval and judgment should be evaluated together in a single pipeline.

  4. The graph uses only four edge types, which may fail to capture complex relationships between features, such as the order of steps or cause-and-effect relationships.

Questions for authors

Q1. The graph uses only four predefined edge types. How were these relations selected, and can they adequately represent complex relationships between technical features?

Q2. The patent classification task considers only IPC sections. Can the proposed graph representation also support more fine-grained patent classification, such as IPC subclasses or CPC categories?

Q3. Each edge connects only two features, which may lose information about relationships involving multiple components. For example, in Figure 2, the heat shield is positioned between the purification device and the fuel tank. How does PAGE preserve such relationships, and how much additional information do textual edge embeddings provide beyond the predefined relation types?

Reviewer_jRHfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PAGE, a two-stage framework for patent assessment that authors define as three tasks: novelty evalution, prior-art retrieval and patent classification . The PAGE framework consists of two stages. The first stage is an offline extraction stage, LLM converts each patent text into a technical feature graph with typed relationships, and the resulting graph are cached for reuse. The second stage is a trained learning stage, a forzen sentence encoder transforms the technical feature graph into node and edge feature matrices.

The trainable graph encoder the encodes these feature matrices into a graph representation vector. Finally three separate modules are independently trained for the respective tasks of novelty evaluation, prior-art retrieval and patent classification, each incorporating its own graph encoder. PAGE is validated on PatentMatch and CLEF-IP.

On novelty evaluation it markedly outperforms the same five open-weight LLMs used as direct judges, and in its best configuration also leads the BERT cross-encoder fine-tuned on the same claim–passage pairs. Each of the three component ablations, conducted on novelty evaluation, incurs a consistent drop.

On prior-art retrieval it ranks first on all five reported metrics, ahead of BM25, the zero-shot sentence encoders and the same patent-domain encoder fine-tuned under an identical protocol. On patent classification it is on par with the fine-tuned text encoder. The paper's contribution lies in separating "reading" and "judging": using LLM to extract explicit technical feature graph at once, and using trainable graph models to complete judgments on them.

Critical strengths
  1. this paper defines patent assessment as three critical tasks that an examiner performs on a patent: novelty evaluation, prior-art retrieval and patent classification.

  2. PAGE is proposed, (Patent Graph Examiner), a two-stage framework that addresses all three tasks using a unified representation.

Critical weaknesses
  1. Compressing each graph into a single vector inevitably risks information loss, yet the paper doesn't establish whether this representation preserves the key features and relations needed for novelty evaluation. Eq 5 and Eq 6 first use an attention-based readout to compress each graph into a single vector and then make a prediction based on the two vectors and their combinations (p. 5). Once a graph has been reduced to a single vector, the model no longer compares individual technical features directly. It is therefore important to establish whether information critical to novelty evaluation is lost during this process. The paper provides no targeted experiments to verify whether the graph representation retains the technical features and relations that determine novelty evaluation. This limits my confidence in the paper's attribution of performance gains to the introduction of graph representations: if critical information is already lost during compression, that attribution remains insufficiently supported.

  2. The rationale for defining the four relation types and the extent of their coverage are insufficiently justified. The paper restricts relations to four types: CONSISTS OF, INTERACTS WITH, IS A, and HAS PROPERTY (p. 4, lines 168–172). However, it doesn't explain how these categories were derived, what criteria were used to define them, or whether they adequately cover the relations found in patent text. These relation types are a key design choice in the proposed method, as they determine how text is converted into graphs and how relationships among technical features are represented. Without a clear explanation of their provenance and coverage, readers can't determine which types of patent text the method is suitable for, under what circumstances it may fail, or whether conclusions drawn from this representation are well-founded.

Questions for authors
  1. The abstract (p. 1, line 15) defines patent assessment in terms of three tasks: novelty evaluation, prior-art retrieval, and patent classification. However, the rationale for this categorization is not explained. Why were these three tasks selected, and what criteria were used to define the scope of patent assessment?

  2. The method is motivated by the premise that patent claims have a hierarchical structure. To what extent does this property hold across patent claims in practice, and are there types of claims for which this assumption does not apply? The paper provides no empirical evidence to support this premise. Could the authors provide further evidence?

  3. Could the paper articulate more clearly the fundamental research challenges underlying patent novelty assessment? The current motivation focuses primarily on the limitations of existing methods, leaving the intrinsic difficulties of the task itself unclear.

  4. The research problem focuses on novelty evaluation, whereas the proposed framework covers all three patent assessment tasks. These two scopes do not fully align. What motivated the inclusion of all three tasks, and are they all part of the intended research objectives?

  5. Each node contains two text fields, subject (the canonical name) and text (the original text span) (p. 4, lines 168–177), whereas p. 4, line 209 states only that txt(⋅)\mathrm{txt}(\cdot) denotes the textual content of a node or edge. For nodes, does txt(⋅)\mathrm{txt}(\cdot)​ refer to subject, text, or a combination of both?

  6. The Method section suggests that the three tasks are trained independently, sharing only the encoder architecture rather than its parameters. However, the descriptions in the abstract and pipeline diagram (e.g., task-specific heads and shared graph encoder) could be interpreted as indicating a multi-task architecture with a shared backbone. Does "shared" refer to the architecture or to the model parameters? I recommend using consistent terminology throughout the paper and revising the pipeline diagram to reflect the actual training procedure.

  7. The current ablation study focuses on replacing components of the encoder (p. 8, line 381), but does not isolate the contributions of specific graph features, particularly the node subject and edge context. Could the authors conduct targeted ablations to assess the contribution of each of these two sources of information separately?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.