acceptodds
Under review as a conference paper at ICLR 2027

Vibe Patenting: Agentic AI for Professional Patent Drafting

Abstract

Professional patent drafting requires more than long-form legal text generation: an agent must extract inventive concepts from heterogeneous technical evidence, plan a protection strategy, coordinate claims, the specification, embodiments, and figures, and repeatedly check support and consistency. We present Vibe Patenting, an agentic AI framework for this end-to-end professional workflow. The system combines a structured Problem–Insight–Solution–Effect (PISE) invention representation with specialized patent skills, realized either as reusable modules orchestrated by a meta-agent or as an integrated domain-specialized patent agent. Across more than one hundred generated drafts from ten technical reports, stronger models and greater inference-time reasoning generally receive higher LLM-judge scores, and agentic configurations outperform general-purpose chat in our system-level comparisons. We also place a separate patent-QA LLM judge inside a multi-round revision loop: structured judge feedback continues to improve judge-assessed quality after generic revision begins to saturate. Independent evaluation by a professional patent attorney confirms a substantial aggregate quality gain after QA-guided revision while revealing strong metric dependence in judge agreement and calibration. A second, independently instructed QA evaluator further shows that revision gains can transfer beyond the optimizing judge even as evaluator-specific preferences emerge. These results position professional patent drafting as a testbed for agentic AI built from structured domain representations, reusable workflows, inference-time reasoning, and external evaluation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
AI Agent, Patent Agent, Meta Agent, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_wdfMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors study patent-drafting agents that cover the complete generation process, from a technical report to claims, a specification, and figures. Their analysis, drafting, and consistency-checking modules share a representation of the invention's problem, insight, solution, and effect (PISE). The authors compare drafting configurations and test revision with and without feedback from QA1, the LLM evaluator that scores the drafts. A separately configured LLM evaluator, QA2, and a patent attorney also assess the Instant drafts before and after revision with QA1 feedback. Both give higher mean scores after revision. The correlations of QA1 and QA2 scores with the attorney's scores vary across drafting configurations.

Critical strengths

S1. External assessments reveal revision gains and weaknesses in the judges. Both QA2 and the attorney give Instant drafts higher mean scores after QA1-guided revision than before it, providing external evidence of improvement. Their scores are not reported for revision without QA1 feedback, so this evidence does not establish the added benefit of that feedback.

The authors examine the correlation of automatic and human evaluation separately for agent, pro, and instant drafts. Although this analysis does not establish that either LLM judge can replace professional assessment, it shows where the judges track the attorney's scores poorly, weaknesses that a correlation across all three configurations would hide.

Critical weaknesses

W1. The experiments leave the contributions of PISE, individual skills, and the evaluators unclear. AutoPatent already combines planning, specialized writing agents, and feedback from an examiner agent. In my reading, this empirical study of PISE and reusable patent skills can be illuminating even absent a new algorithm. The authors acknowledge, however, that they do not test PISE or individual skills separately, so their experiments cannot show which components are needed for the reported gains.

PatentScore evaluates several dimensions of patent-claim quality through LLM judgments and compares its scores with those of domain experts. The present paper discusses both PatentScore and PatClaimEval, but does not compare either with QA1/QA2 on the same drafts using shared criteria for claim quality. Do QA1/QA2 scores agree more closely with expert assessments of claim quality than these methods' scores do? Please clarify which comparisons are possible, given what each evaluator assesses and whether it requires reference claims.

W2. Professional validation relies on one attorney, and some evaluation materials remain unavailable. The authors acknowledge that the study is preliminary, but with only one attorney providing human scores, it cannot measure inter-annotator agreement among practitioners or establish whether other attorneys would give similar assessments.

Across the agent, pro, and instant drafts together, QA1's and QA2's overall scores correlate more strongly with each other than with the attorney's, whereas for agent drafts alone, both judges' scores have near-zero Pearson correlations with the attorney's (Figure 7). The pooled results therefore do not establish that either judge reliably ranks agent drafts in the same order as this attorney, and the sample of nine inventions provides too little evidence to extend that finding to other inventions. Agreement in score values is a separate question: Pearson correlation measures the strength of a relation between two variables, not the agreement between them. In Table 5, both judges give higher scores than the attorney on average, and QA2's scores are farther from the attorney's on average than QA1's.

The disclosed procedure and rubric explain what aspect of quality is being evaluated and how, but do not supply all the instructions needed to reproduce the LLM assessments: the authors state that "Proprietary system instructions and skill internals are not reproduced". It is also unclear whether QA2's instructions were fixed before the attorney's scores were examined. For revised Instant drafts, the reported attorney scores are limited to means, leaving readers unable to check how the improvement in those scores varies across inventions.

Questions for authors
  1. What materials and scores could the attorney see during evaluation (W2)? Please specify the order in which items were presented, the instructions, questions or prompts the attorney received, and whether the drafting configuration was concealed. Were QA2's instructions fixed before the attorney's scores were examined?
  2. Please share any existing results comparing how closely QA1/QA2, PatentScore, and PatClaimEval scores agree with expert judgments on the same drafts and overlapping criteria for claim quality (W1). If differences in required inputs or evaluation criteria prevent this comparison, please explain them. Correlations from different datasets would not establish which evaluator agrees more closely with experts.
  3. For QA1/QA2, please provide the full set of instructions and existing scores for individual drafts, so readers can assess agreement between each evaluator's scores and the attorney's within each drafting configuration (W2). If QA2 or the attorney scored drafts revised without QA1 feedback, please share those scores alongside the scores for drafts revised with QA1 feedback. This would show whether the advantage found by QA1 also appears in either external assessment. These requests are limited to existing materials and results. Evaluation by several attorneys would be valuable future work rather than a requirement for the rebuttal.
3 comments
AuthorsOpenReview ↗

Comparison with AutoPatent and clarification of contributions

We appreciate the reviewer's identification of AutoPatent as closely related work. AutoPatent establishes important mechanisms for patent generation, including specialized writing agents, hierarchical Patent Writing Guideline Tree (PGTree) planning, and Reference-Review-Augmented Generation (RRAG) with examiner feedback. We also acknowledge its published component ablations.

To strengthen the empirical comparison, we implemented an AutoPatent-inspired PGTree/RRAG pipeline as an installable skill within our meta-agent framework. The official AutoPatent repository provides examples and demonstration materials, while its complete agent implementation and training data have not been publicly released. Our implementation reconstructs the published workflow using prompt-based agents.

Both this reconstructed baseline and Vibe Patenting AgentEx used GPT-5.6 with Extra-High reasoning. We evaluated paired nine corresponding inventions, with five independent scoring repetitions per invention and evaluator.

Table R1. Five-axis patent-quality comparison (1–10, higher is better).

Quality dimensionAutoPatent QA1Vibe QA1AutoPatent QA2*Vibe QA2*
Prosecution Resilience4.875.585.536.87
Claim Strength5.366.626.137.00
Disclosure Strength6.027.806.077.64
Figure Quality4.497.824.387.20
Filing Readiness4.426.044.425.98
Overall mean5.036.775.316.94

Under the five-axis QA1 evaluations, Vibe Patenting achieves a 1.74-point higher mean, with higher scores across all nine inventions. The largest observed differences concern Figure Quality (+3.33) and Disclosure Strength (+1.78).

The two approaches address complementary aspects of patent drafting. AutoPatent focuses on hierarchical planning and iterative refinement of long patent descriptions. Vibe Patenting focuses on coordinating invention understanding and protection strategy across claims, embodiments, specifications, figures, and quality evaluation through a shared PISE representation.

We will expand Related Work to explain these complementary contributions and acknowledge AutoPatent's prior PGTree/RRAG results. We will also make the available evaluation records accessible.

We present this numerical comparison as exploratory system-level evidence.

AuthorsOpenReview ↗

Additional experiment: isolating the contribution of PISE

We appreciate the reviewer's suggestion to evaluate the individual contributions of PISE and specialized patent-drafting skills. We have completed a targeted ablation of PISE.

We compared our GPT-5.6 Medium agent with and without PISE. Both configurations preserve the professional skill-orchestration framework, including the Insight Builder, Claim-Tree Builder, Invention Knowledge Dossier, Patent Figure Generator, and other specialized drafting capabilities. Only the PISE-specific intermediate representation and workflow were removed.

This design allows us to investigate PISE's incremental contribution while retaining the agent's professional drafting capabilities.

Table R2. PISE ablation: quality improvement with PISE (1–10 scale).

Quality dimensionQA1 gainQA2 gain
Prosecution Resilience-0.08+0.36
Claim Strength+0.09+0.31
Disclosure Strength-0.06+0.23
Figure Quality+0.76+0.99
Filing Readiness+0.32+1.06
Overall+0.21+0.59

The agent with PISE achieves a mean score of 6.13 versus 5.92 under QA1, and 6.60 versus 6.01 under independent QA2.

The QA2 results are particularly encouraging: PISE improves the mean score in all five dimensions, with the largest benefits in Filing Readiness (+1.06) and Figure Quality (+0.99). These dimensions reflect the coordination and consistency of patent-package components.

The nine-invention paired analysis yields p=0.207 under QA1 and p=0.059 under QA2. It provides an encouraging, consistent directional result across all five quality axes.

The performance of the no-PISE configuration also demonstrates the effectiveness of the underlying professional skill orchestration. PISE provides an additional structured invention representation that can improve coordination across these existing capabilities.

We will add this ablation, the per-invention scores, and the associated evaluation protocol to the revised manuscript. Together with the previously reported AutoPatent comparison, these results distinguish the role of PISE from the contribution of the broader professional agent architecture.

AuthorsOpenReview ↗

Independent evaluation, agreement with professional assessments, and reproducibility

W2, Question 1. Blinding, evaluation timing, and information exchange.

The patent attorney was blinded to the drafting configuration. QA1, QA2, and the attorney performed their assessments independently, without exchanging scores, rankings, or feedback.

QA2 was instructed using the same professional rubric as the attorney. The shared rubric establishes common assessment criteria while preserving independent scoring outcomes.

EvaluationInformation-flow protocol
QA1Feedback partly used to improve drafts for the revision loop
QA2Independent assessment; no QA1 feedback or attorney scores supplied
AttorneyBlinded assessment; no QA1 or QA2 evaluation results supplied
Between evaluatorsNo exchange of scores or feedback

W2, Questions 2–3. Agreement with attorney evaluation.

We analyzed 27 identical invention-configuration conditions scored by QA1, QA2, and the attorney.

Table R3. Agreement with blinded attorney evaluations.

MetricQA1 vs. attorneyQA2 vs. attorney
Pearson correlation0.5980.603
Spearman correlation0.5200.558
Mean absolute error1.792.19
Mean score bias+1.79+2.19

Both AI evaluators achieve similar pooled correlations with the attorney. QA1 has lower absolute error in this sample.

We also examined the correlations within each drafting configuration.

Drafting configurationQA1–attorney rQA2–attorney r
Meta-agent-0.092-0.133
Higher-reasoning generalist0.7960.703
Instant generalist0.3320.868

These results demonstrate substantial configuration-dependent variation. We will report both pooled statistics and individual configuration results, distinguishing correlation from numerical score calibration.

W1/W2. Independent validation of QA-guided revision.

The completed three-evaluator analysis shows that improvements from QA1-guided revision transfer to independent QA2 and attorney evaluations.

Table R4. Quality differences across three evaluators.

ComparisonQA1QA2Attorney
QA1-guided revision minus initial draft+1.28+0.58+1.58
AgentEx minus QA1-guided revised draft+0.06+0.65+0.68

The attorney values are aggregate differences. QA2 rates the QA1-revised draft above the initial draft for eight of nine inventions.

The second comparison reveals an evaluator-specific effect. QA1 finds the two selected drafting conditions nearly equivalent, while QA2 and the attorney retain larger preferences for AgentEx.

These observations provide evidence of transferable improvement together with evaluator-specific optimization.

Comparison with generic revision.

Our original controlled experiment compares generic revision and QA1-guided revision under QA1. At round four, QA-guided revision scores +0.38 higher for Instant and +0.45 higher for Extra-High reasoning.

The existing independent QA2 and attorney evaluations cover the initial and QA1-guided revised drafts, but not the corresponding generic-revision outputs.

W1. Comparison with PatentScore and PatClaimEval.

PatentScore and PatClaimEval assess claim-level properties using different inputs, evaluation criteria, and potentially reference claims. Our evaluation addresses complete patent packages, including prosecution resilience, disclosure, figure quality, and filing readiness.

We do not have matched professional evaluations applying all three methods to the same claims under common criteria. Consequently, the correlations reported in their respective studies cannot establish which evaluator agrees more closely with patent attorneys.

Evaluation transparency.

We will expand the experimental documentation with available individual AI scores, evaluator instructions, rubric mappings, and condition-level results. We will also identify the attorney comparisons available only as aggregate scores.

Reviewer_6MzuOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Vibe Patenting, a system for end-to-end agentic patent drafting that transforms heterogeneous technical reports into coordinated patent packages (claims, specification, figures, supporting analyses). The central representational design is PISE (Problem-Insight-Solution-Effect), a 4-tuple invention state that is reused across downstream specialized skills (prior-art analysis, protection strategy, embodiment expansion, claim and specification drafting, figure generation, QA, revision). The paper compares three allocations of professional structure: generalist GPT-5.6 Sol chat at four reasoning levels (Instant through Extra-High), a meta-agent that composes reusable skills, and a custom patent agent that integrates workflow logic directly. A separate patent-QA LLM judge scores drafts on five dimensions (Prosecution Resilience, Claim Strength, Disclosure Strength, Figure Quality, Filing Readiness) with a 1-10 rubric and five independent repetitions per draft. The judge is also inserted into a 4-round revision loop and compared against generic revision without structured feedback. Across more than 100 drafts from 10 technical reports: Chat Instant 5.53 to Extra-High 6.58, agent 6.15 to 6.80, custom agent 7.02 on two patents. Judge-guided revision continues to improve past round 2 while generic revision saturates: Instant+QA 5.43 to 6.71 at round 4 (vs 6.33 generic), Extra-High+QA 6.26 to 7.18 (vs 6.73 generic), and QA-guided revision closes the Instant to Extra-High initial gap. Professional validation uses one patent attorney on 28 matched AI-draft conditions from 9 patents. QA1 to attorney overall Pearson r = 0.670 is misleading on its own: by dimension, Figure Quality is 0.819, Disclosure Strength 0.533, Claim Strength -0.077, Filing Readiness 0.305, Resilience 0.054, and the overall judge is biased by +1.76 points above the attorney. An independently instructed QA2 (attorney-aligned rubric, Extra-High reasoning) shows that the main revision gain (Instant to Instant times 4) transfers to the attorney (+1.58) and to QA2 (+0.58), while the headline AgentEx vs Instant times 4 comparison gives +0.06 under QA1 but +0.65 under QA2 and +0.68 under the attorney, which is direct evidence of judge-specific optimization narrowing QA1's perceived gap. Limitations section explicitly acknowledges the system-level nature of all comparisons, the lack of PISE or individual skill ablation, the single-attorney validation, the n=2 for the custom agent, and the differences between QA1 and QA2 in both rubric and reasoning effort.

Critical strengths

1. The three-way QA1-QA2-attorney cross-evaluation produces a direct, quantitative demonstration of judge-specific optimization in a revision loop. Section 4.4 reports that Instant to Instant times 4 (the revision loop optimized by QA1) is a transferable gain confirmed by all three evaluators (+1.28 under QA1, +0.58 under QA2 on 8 of 9 patents, +1.58 under the attorney). Critically, the AgentEx versus Instant times 4 comparison then shows QA1 at +0.06 (near-tie), QA2 at +0.65, and attorney at +0.68. The near-tie under QA1 is explained precisely because Instant times 4 was optimized against QA1, so QA1 sees the gap as closed; QA2 and the attorney both still see a substantial AgentEx advantage. This is the clearest kind of judge-overfitting evidence I have seen in a revision-loop paper: not only is there transfer, but there is also a quantifiable judge-specific component that only reveals itself under independent re-scoring. The paper also reports the per-dimension Claim Strength Pearson r = -0.077 against the attorney (Table 2), which is a striking honest negative result for what is arguably the most important dimension in patent drafting. These findings generalise beyond patent drafting: any LLM-judge-guided iterative revision pipeline needs this kind of multi-evaluator triangulation, and the paper's protocol is directly transportable.

Critical weaknesses

1. The paper's named contributions (PISE, specialized skills, custom patent agent) are never causally isolated, and the paper itself acknowledges this in Section 5. The three-system comparison (chat, meta-agent, custom agent) varies instructions, state handling, tools, and workflow integration simultaneously, so the custom agent's 7.02 overall advantage over chat Instant's 5.53 cannot be attributed to PISE, to the skill library, to the integrated workflow logic, or to the dedicated instructions, individually or in combination. For a paper whose Section 3 is largely about PISE as the core representational design principle, this is the central methodological gap: a reader cannot tell whether PISE buys anything on top of a strong system prompt that enumerates the same drafting stages, or whether the specialized skills on top of a flat generalist agent would reproduce most of the gain. The Limitations section correctly identifies this as "the highest-value follow-up" but the current submission does not contain it. For ICLR main track, this leaves the specific contribution of PISE and the skill library unsupported by evidence.

2. The sample sizes and the proprietary nature of the pipeline make the reported numbers hard to generalise. The main study uses 10 technical reports and more than 100 drafts, which sounds moderate but under the condition-split (eleven AI configurations plus one human reference across 10 patents) each cell has at most 10 observations, and the custom agent is only on 2 patents (acknowledged). Professional validation hinges on 1 patent attorney across 9 patents (28 matched draft-conditions), with AgentEx and Instant times 4 having only aggregate attorney means rather than per-condition scores. The LLM-judge, custom agent architecture, PISE-encoded knowledge, and attorney scores are all private; the AI Use Statement discloses that Figures 1-3 are AI-generated conceptual illustrations and the reproducibility statement does not promise code or data release. Taken together, (a) the gains cannot be attributed to any single named component, (b) the per-condition statistical power is low, (c) the single-evaluator attorney score cannot be checked against inter-attorney agreement, and (d) nothing in the pipeline can be reproduced by an external reader. The paper's own Limitations section honestly flags all of these, which is a strength of the writing but does not resolve the underlying evidence problem.

Questions for authors
  1. A fair PISE ablation. The simplest version of the controlled study you flag as "highest value" is: hold base model, reasoning effort, source reports, and system prompt structure fixed, run the drafting agent with and without the PISE intermediate state (replacing PISE with free-form notes or with four parallel independent summaries), and score the resulting drafts with QA1, QA2, and the attorney. Can you report this for at least the 2 custom-agent patents, or scope it to a subset of the full 10-report set? Without this, the PISE contribution is not supported.

  2. Attorney independence. Was the attorney blinded to which configuration produced each draft when scoring? Does the attorney know the names of the eleven AI configurations? If yes, the attorney scores may carry a configuration-identification signal that could bias the Instant to Instant times 4 vs AgentEx comparison.

  3. Inter-attorney reliability. With one attorney, we have no inter-rater reliability number against which to compare the QA1-attorney r = 0.670 and the Claim Strength r = -0.077. Even a single additional attorney rescoring the 28 matched conditions would convert the current attorney scores into a joint anchor and allow a direct human-human vs human-AI comparison. Is this feasible for a revision?

  4. Transfer across inventions. The 10 technical reports span multiple technical domains (Section 4). Are there within-domain subsets where the configuration ranking stays the same, versus domains where it reverses? If the ranking is dominated by a few patents, the aggregate results may hide important heterogeneity.

  5. Reproducibility. Will the final submission release (a) the QA1 and QA2 judge rubrics and full prompts, (b) the Vibe Patenting skill prompts, (c) the attorney scoring instructions, and (d) at least one representative anonymised technical report and the matched 11 drafts for external inspection? Without the first three, the AgentEx-vs-Instant times 4 comparison is not replicable by anyone outside your lab.

3 comments
AuthorsOpenReview ↗

Additional baseline

We thank the reviewer for highlighting the need for stronger architectural comparisons.

We have added an AutoPatent-inspired baseline following the published multi-agent pipeline: specialized section writers, a two-level Patent Writing Guideline Tree (PGTree), reference retrieval, and Reference-Review-Augmented Generation (RRAG) with examiner-guided subsection refinement.

The official AutoPatent repository currently provides examples and demonstration assets, while the original implementation and training data remain unavailable. We therefore reconstructed the published workflow as an installable skill within our meta-agent framework, including explicit role prompts, intermediate schemas, and documented implementation choices.

We ran this reconstruction using GPT-5.6 with Extra-High reasoning, matching our Vibe Patenting AgentEx configuration, and evaluated nine paired inventions, with five independent QA assessments per invention.

Quality dimensionAutoPatent-inspiredVibe Patenting
Prosecution Resilience4.875.58
Claim Strength5.366.62
Disclosure Strength6.027.80
Figure Quality4.497.82
Filing Readiness4.426.04
Overall mean5.036.77

Vibe Patenting achieves a 1.74-point higher recorded mean, with the same direction across all nine inventions. The largest differences appear in Figure Quality (+3.33) and Disclosure Strength (+1.78).

This experiment extends our analysis to a substantively different professional patent-agent architecture using the same base model and reasoning effort. We will provide the reconstructed skill and per-invention scores to support reproducibility.

The comparison is currently exploratory, while these results may be insightful at the complete-workflow level.

AuthorsOpenReview ↗

PISE ablation

We thank the reviewer for suggesting a direct ablation of PISE. We have now conducted this experiment with GPT-5.6 Medium, comparing our patent-drafting agent with and without PISE.

Importantly, both configurations retain the same professional skill-orchestration framework, including the Insight Builder, Claim-Tree Builder, Invention Knowledge Dossier, Patent Figure Generator, and other specialized patent-writing skills. We removed only the PISE intermediate representation and its associated workflow. Thus, the experiment measures the incremental contribution of PISE within an otherwise capable professional drafting agent.

Table R2. Overall PISE ablation results (1–10, higher is better).

EvaluationWith PISEWithout PISEGain95% CIp-value
QA16.135.92+0.21[-0.14, 0.55]0.207
QA26.606.01+0.59[-0.03, 1.21]0.059

Table R3. Improvement attributable to PISE by quality dimension.

Quality dimensionQA1 gainQA2 gain
Prosecution Resilience-0.08+0.36
Claim Strength+0.09+0.31
Disclosure Strength-0.06+0.23
Figure Quality+0.76+0.99
Filing Readiness+0.32+1.06
Overall+0.21+0.59

The independent QA2 evaluator shows a positive mean benefit from PISE across all five quality dimensions. The strongest improvements occur in Filing Readiness (+1.06) and Figure Quality (+0.99).

These results suggest that PISE particularly helps coordinate the different components of complete patent packages. The overall QA2 improvement favors PISE on 6/9 inventions and approaches conventional statistical significance (p=0.059). Given the limited sample size, we interpret this as promising component-level evidence rather than a conclusive population-level effect.

The no-PISE agent remains capable because it retains the professional skill library. Its relatively strong performance provides a meaningful comparison for evaluating the incremental role of PISE.

QA2 uses the same professional rubric as the patent attorney. We will incorporate the detailed ablation procedure, invention-level results, and uncertainty estimates into the revised manuscript.

AuthorsOpenReview ↗

Evaluation independence, generalization, and reproducibility

We thank the reviewer for the detailed questions. Here we clarify the remaining evaluation and reproducibility concerns.

Q2. Attorney blinding and evaluator independence. The patent attorney evaluated drafts blind to the generating configuration. QA1, QA2, and the attorney conducted their assessments independently, without exchanging scores, rankings, or evaluation feedback. QA2 just applied the identical professional rubric used by the attorney.

EvaluatorEvaluation procedure
QA1Its feedback is partly used for evaluation of the judge-guided patent revision process.
QA2Independently evaluates drafts without receiving QA1 feedback or attorney scores.
AttorneyEvaluates drafts blindly, without drafting-configuration labels or QA1/QA2 results.

The use of QA1 feedback during revision is part of the experimental intervention. The independent QA2 and attorney assessments provide external validation of the resulting drafts.

Q3. Inter-attorney reliability. We acknowledge that evaluation by one blinded attorney provides a single professional reference and does not establish inter-attorney reliability. We currently have no access to another attorney, and the limitation will be clearly mentioned in the paper, leaving inter-attorney evaluations as a future work.

Q4. Transfer across inventions. We reanalyzed the nine matching inventions, comparing our GPT-5.6 Medium meta-agent with the generalist Instant configuration.

AnalysisQA1QA2Blinded attorney
Mean meta-agent advantage+0.70+0.90+0.77
Inventions favoring meta-agent9/99/99/9
Robotics subject+0.55+0.73+0.55
Other inventions+0.78+0.98+0.88

The meta-agent receives higher ratings on all nine inventions under all three evaluators (exact two-sided sign test, p=0.0039 for each evaluator). The positive advantage is present in both subsets.

We additionally performed a leave-one-invention-out analysis. The mean advantage remains positive regardless of which invention is removed: +0.66 to +0.74 under QA1, +0.83 to +1.00 under QA2, and +0.66 to +0.82 under the attorney.

These findings indicate that the aggregate improvement over generalist Instant is consistent across the evaluated inventions and is not driven by a single patent.

We also find meaningful variation among stronger configurations. Compared with the Pro/Extra-High generalist condition, the meta-agent is preferred on 7/9 inventions by QA1, 7/9 by QA2, and 4/9 by the attorney. We will report these variations alongside the aggregate results.

The study contains nine inventions, so broader domain generalization remains an important direction for future evaluation.

Q5. Reproducibility. We will expand the experimental documentation with evaluator configurations, rubric mappings, scoring instructions, individual available scores, and representative patent artifacts that can be shared. We will distinguish releasable experimental materials from proprietary instructions requiring separate approval.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.