acceptodds
Under review as a conference paper at ICLR 2027

Reasonable-RLVR: Step-Verified Rewards for Reasoning over Knowledge Graphs

Abstract

Reinforcement learning with verifiable rewards (RLVR) trains reasoning models with programs that check their outputs. Such programs check answers in mathematics and code, and they check steps in formal theorem proving. Knowledge-intensive domains such as medicine have no such language, so a correct answer reached by unsound reasoning receives full credit. Knowledge graphs make entailment decidable, but no reward checks cited steps against their closure. We introduce Reasonable-RLVR, which verifies each cited proof step against the OWL 2 RL closure of a graph and checks whether the steps establish the answer. The verifier tests the supported chain against a relation contract, has configurable coverage, and needs no reference proof or learned judge. We define two versions of the reward, a two-band form that satisfies five ordering properties on anchored traces and a simpler weighted sum. Reasonable-RLVR obtains a higher correct-and-valid rate than outcome and reference-path rewards on four knowledge-graph benchmarks. Without a supervised warm-up on WebQSP, it reaches 0.597 ± 0.120 correct-and-valid proofs over three seeds, while the outcome reward reaches 0.000 at 94% answer accuracy. Verifier feedback at inference also improves frontier-model proofs. These results extend verifiable rewards to reasoning over knowledge graphs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reinforcement learning with verifiable rewards, process reward, process verification, knowledge graph, large language models

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_4ouPOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Reasonable-rlvr trains a model to answer questions with cited proof steps. The input includes a question and graph facts that contain the answer. The verifier checks claims against graph closure and uses a relation contract to check whether the supported chain reaches the answer. Only the verifier sees this contract. Training combines answer correctness with process rewards from these checks. The reported correct & valid rates are higher than with outcome rewards on 4 main benchmarks and smaller graphs.

Critical strengths

(1) The checker separates supported facts from a chain that establishes the answer. Table 12 explains the disagreements with the independent checker. These come from answer parsing, repeated ids and related specification choices. No closure disagreement was found in that audit. This helps show what is still unclear.

(2) It is a good experimental design choice to use a common verifier for the reward comparisons. The outcome+format control helps check whether gains come mainly from learning the output format. The 8 sample eval also gives useful support beyond the greedy comparison.

(3) On Webqsp 2 seeds reach the same reward but only 1 collapses. Some runs also recover after a period of producing only one claim. The ordering tests alone do not ensure stable training: Cou passes the tests yet learns less reliably than Cdu. The adaptive Webqsp runs reduce collapse.

Critical weaknesses

(1) The main benchmarks have no class declarations, transitivity or property chain rules. The richer ontology layer therefore has limited direct support in these results.

(2) The contract comes from a reference relation sequence. It is unclear how this covers CWQ constraints outside the chosen chain. The independent checker checks the same specification rather than whether the contract covers the full question. This leaves the broader reasoning claim less clear. It does not show that an invalid proof was accepted.

(3) The coverage result may depend on how a reversal is written. On Dynamic-kgqa, an explicit hidden reverse triple still hurts soundness under abstention. A forward fact used backwards keeps its support. In masked edge experiment, masked claims leave the soundness denominator and can count towards chain/progress. The protocols are described. It would be good to use trace counts to explain how much this difference affects recovery.

Questions for authors

please see weakness part.

4 comments

We thank the reviewer. All results are new and in the revised PDF.

(1) The ontology layer in the main benchmarks.

How we read the concern. The four benchmarks declare no classes, transitivity or property chains, so the main results give little direct evidence for the richer rule layer.

(a) Rules a proof must use. We built test sets on which a valid proof needs a declared rule (Table 55, three seeds). With the inverse fact removed from Dynamic-KGQA blocks, the proof-trained policies prove .51 to .58 of the 2,958 questions by citing the forward fact, which the checker accepts through the declared inverse (.006 without it). On 235 MINTQA questions answerable only through a transitive run of two located-in or part-of facts, CWQ-trained CDU policies, never trained on such questions, prove .39 to .49 at 1.5B to 7B and cite both facts in .63 to .82 of replies against .38 to .44 for outcome (0 with transitivity off). With a warm-up that writes the rule as a step, the policies write the inverse rule explicitly on .94 to .96 of the inverse questions and keep .47 to .51 with inverse reading off (App. M).

(b) Classes and types. With each source's own classes and domain and range axioms added to the checker, the type condition rejects 107 and 537 proofs whose answer is the wrong kind of entity and keeps every correct-and-valid verdict (App. M). Property chains are declared on UMLS, CoDEx-S and the synthetic graphs, where the checker verifies 4,513 proofs that depend on a transitivity or chain rule (Table 56).

(2) CWQ constraints outside the chain, and the independent checker.

How we read the concern. The contract follows the reference relation chain, so a comparative or superlative constraint may lie outside it, and agreement with the independent checker is agreement on the same specification.

(a) What is enforced. The correct-and-valid rate requires both a valid chain and a correct answer, so the constraint is enforced through the answer. On 1,000 mutated reference proofs per benchmark (Table 58), both checkers reject every swapped object, dropped step, fabricated identifier, restated premise and wrong symmetry rule on CWQ, Dynamic-KGQA and WebQSP, and the answer key rejects every proof that moves the answer to another entity on the chain. On the constraint questions, 182 of 192 CDU and 190 of 199 gated valid proofs that end at a wrong answer name an entity that loses the comparison or is not a compared candidate, and the answer key rejects every one (App. M).

(b) Checking the constraint itself. For 239 constraint questions we linked the compared date or count from the 2015 Freebase dump, added the 388 facts to the evidence block, and required the proof to cite the answer's attribute fact and the comparison to hold (Table 62, three seeds pooled, n = 717). Without any training on such facts, the proof-trained policies pass on .46 to .69 of questions at 1.5B, 3B and 7B, against .00 to .18 for outcome (p below 1e-40), and cite the answer's attribute fact in .62 to .89 of replies.

(c) Contracts from the question. Training with the question's endpoints and a hop bound instead of reference relations gives .729 (CDU) and .712 (gated) against .704 and .708 (App. H).

(3) Reversals, masked claims and the soundness denominator.

How we read the concern. Under abstention a masked claim leaves the soundness denominator in training, and a reversal can be written two ways. The reviewer asks for trace counts showing how much this contributes to the recovery. Evaluation always uses the full verifier, so a claim set aside in training counts only if it is true.

(a) Trace counts. We counted valid evaluation proofs that need a claim on a relation hidden in training (Table 59). At 0.5B, where the paper reports the recovery, abstaining training kept such proofs on the toy graph (10 of 18, 12 of 19 and 9 of 16 over three seeds) while strict training suppressed them (0 of 7, 3 of 12, 0 of 0), and every such claim is confirmed by the full verifier. The recovery is the policy continuing to write true facts the training verifier cannot see.

(b) Training credit. In the stored training rollouts of a three-seed 0.5B replay (8,000 per cell), a false masked claim earns establishment credit in at most 6 rollouts per cell and never earns progress credit, and in the table runs 12,684 of 12,688 masked claims are true (Table 61, App. M).

(c) Reversal form. On Dynamic-KGQA most replies write both forms, and the strict and abstaining policies distribute over them alike (Table 60). Hiding both directions of every masked relation, which removes the forward-fact route, keeps abstaining above strict at 0.5B (.279 against .197, p = .03, and .803 against .439, p = 2e-5) and at or above it at 1.5B (Table 63). At 3B over seven seeds, abstaining recovers the full-verifier rate (.819 against .805, n = 343). The recovery does not rest on how the reversal is written.

Thank you again for your careful review. We hope our reply and the revision address your points. In brief:

  • (1) Ontology layer. Test sets on which a valid proof must use a declared inverse or transitive rule, type conditions from each source's own axioms, and 4,513 verified proofs that depend on a transitivity or chain rule (App. M, Tables 55 and 56).
  • (2) CWQ constraints. A direct check of the compared date or count on 239 constraint questions, where the proof-trained policies pass on .46 to .69 against .00 to .18 for outcome (Table 62).
  • (3) Reversals and masked claims. Trace counts show that the recovery comes from true facts the training verifier cannot see, and abstaining stays at or above strict when both directions of a masked relation are hidden (Tables 59 to 63).

If these results resolve your concerns, we would be grateful if you would consider updating your assessment. We are glad to answer any further question before the discussion closes.

Thanks for the detailed response! which address all my concerns and I have raised my score to accept accordingly.

We are glad our responses addressed your concerns. Thank you for the thoughtful review and for your endorsement of our work.

Reviewer_UPwuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates process verification in knowledge graph question answering, aiming to simultaneously reward correct answers and verifiable supporting proofs through Reasonable-RLVR. The authors propose Reasonable-RLVR, which parses references in claim lines and checks whether these references support the corresponding claim within the OWL 2 RL closure of the knowledge graph. Relational contracts specific to the question are used to check whether the reasoning chain is sufficient to establish the given answer. A verifier provides three types of signals: soundness, progress, and answer establishment.

Critical strengths

This paper distinguishes between answer accuracy, the proportion of fully valid reasoning trajectories, and the proportion of correct answers with valid reasoning trajectories, and examines the supporting evidence for citations and claims, as well as whether the answers are established. The appendix also reports on independent verification of 2,000 reasoning trajectories and explains discrepancies in the verification results, which helps assess the verifier's reliability.

Critical weaknesses
  • Sections 2–3 and Appendix B indicate that the supporting subgraphs contain the answer. The relation contract also originates from the reference path; therefore, the "without proof labels" requirement still involves supervision derived from reference information.

  • Appendix B notes that none of the four benchmarks include transitivity or attribute chain declarations, nor are type conditions enabled. The rule derivation steps primarily involve retrieval, symmetric relations, and inverse relations, while most problems in Appendix C involve only one or two hops.

Questions for authors

For the experiments in Tables 3 and 5, please add two control groups: retries without verifier feedback and retries with only general error messages, keeping the number of generation attempts consistent. This will distinguish the value of specific feedback from the benefit of additional attempts.

6 comments

We thank the reviewer. All results are new and in the revised PDF.

W1. Does the method still need the reference path? We removed the reference relations from the contract and built the evidence from the question alone.

Contract (1.5B, CWQ, three seeds, Table 32):

Training contractCDUGated
Reference contract (paper).704.708
No reference relations (question entity, answer, hop bound).729.712
Only a hop bound predicted from the question.723.706

On a matched pool of retrieved blocks, the question-derived contract is level with or above the reference contract on every seed and arm (App. I). At evaluation, scoring without reference relations changes the rate by at most .004 on CWQ, Dynamic-KGQA and WebQSP (App. H).

Evidence retrieved from the question alone (1.5B, seed 0, Tables 35 and 36):

CWQDynamic-KGQAWebQSP
Blocks that keep a route to the answer.588.585.686
CDU on those blocks.579.569.617
Gated.592.586.566
Outcome.343.474.000

On blocks with no route every policy answers about .10 of CWQ, Dynamic-KGQA and WebQSP questions, so the policies read the block rather than recall the answer. With every entity name in a MINTQA block exchanged for another of the same kind, so that a recalled answer is wrong, the MINTQA-trained policies keep .84 to .88 against .88 to .91 with the original names (Table 38). Training on retrieved blocks keeps the original test rate (within .007, three seeds, App. I).

W2. Rules beyond retrieval, symmetry and inverse, and depth. We built test sets on which a valid proof must use a declared rule (three seeds, Table 55):

Test setRule neededProof-trained policiesWithout the rule in the checker
Dynamic-KGQA, inverse fact removed (n = 2,958)declared inverseCDU .550 to .558, gated .514 to .584at most .006
MINTQA, only a transitive run answers (n = 235), CWQ-trained CDUtransitive located-in, part-of1.5B .39 to .48, 3B .49, 7B .46 to .490

At 1.5B, CDU cites both facts of the run in .63 to .79 of replies against .38 to .44 for outcome. With each source's own classes and domain and range axioms added, the type condition rejects 107 and 537 proofs whose answer is the wrong kind of entity and keeps every correct-and-valid verdict (App. M). On UMLS, CoDEx-S and the synthetic graphs, which declare property chains, the checker verifies 4,513 proofs that depend on a chain or transitivity rule (Table 56).

Depth. MINTQA, 5,481 Wikidata questions of one to four hops that no policy was trained on (three seeds, Table 39):

3 hops4 hops
1.5B warm-up.078.015
1.5B CDU.591.270
7B CDU.721.420

The policies prove questions deeper than any they were trained on: most three-hop questions and up to .42 of four-hop questions, where their warm-up proves almost none.

Q. Plain retries and a general error message for Tables 3 and 5. We added both controls for every model of Table 3 and every policy of Table 5, with the same number of generations and the same first attempt: a plain retry with no feedback, and a general message saying only that the checker rejected the proof. Full results: Tables 43 and 47 (Table 3 models) and Tables 42 and 51 (Table 5 policies).

Correct-and-valid rate (frontier: two rounds, seed 0; others: three rounds, two repair seeds):

Model or policyBenchmarkPlain retryGeneral messageNamed step
Gemini 3.6 Flash (Table 3)Dynamic-KGQA.864.974.966
Claude Sonnet 5 (Table 3)CWQ.770.774.840
Claude Sonnet 5 (Table 3)WebQSP.856.904.916
Qwen2.5-7B, untrained (Table 3)MINTQA, 3 to 4 hops.192.202.210
1.5B CDU, multi-turn (Table 42)CWQ.782.805.805
1.5B gated, multi-turn (Table 42)CWQ.760.810.818
3B CDU, multi-turn (seed 0)CWQ.716.827.843
3B gated, multi-turn (seed 0)MINTQA, 3 to 4 hops.511.691.691
  • The verifier makes every extra attempt useful. Every retry follows a rejection by the verifier, so even a plain retry repairs proofs at every model size.
  • Specific feedback adds to it. For Gemini both messages beat plain retries in all six cells. For Sonnet the named step beats the general message on CWQ at both seeds (p below 1e-5), mostly by fixing replies with no usable proof. For the untrained 7B model the named step is ahead of a plain retry on all three benchmarks at the second attempt (Table 41) and leads both controls on CWQ and WebQSP after three rounds (Table 47).
  • Most of all after multi-turn training. With feedback in the loop during training, the named step beats a plain retry in 12 of 13 policy and benchmark cells (Table 51), and its gain is larger on two-hop than on one-hop questions in all nine 1.5B policy cells.
AuthorsOpenReview ↗

**Update: the 3B repair results on three seeds.**

Update: the 3B repair results on three seeds. The 3B multi-turn policies were one training seed when we posted. Seeds 1 and 2 are now complete (Table 51, App. K). After multi-turn training with feedback in the loop, the named step beats a plain retry on every seed, for both rewards, on CWQ and on the MINTQA questions of three or more hops (12 of 12 cells, every p at or below 4e-18). Means over the three seeds:

3B multi-turn policyBenchmarkPlain retryGeneral messageNamed step
CDUCWQ.796.851.857
gatedCWQ.805.856.860
CDUMINTQA, 3 to 4 hops.603.686.691
gatedMINTQA, 3 to 4 hops.465.621.627

With the 1.5B policies, the named step beats a plain retry in 20 of 21 multi-turn policy and seed cells.

Revised table for Q (correct-and-valid rate; frontier: two rounds, seed 0; others: three rounds, two repair seeds; 3B rows now means over three policy seeds):

Model or policyBenchmarkPlain retryGeneral messageNamed step
Gemini 3.6 Flash (Table 3)Dynamic-KGQA.864.974.966
Claude Sonnet 5 (Table 3)CWQ.770.774.840
Claude Sonnet 5 (Table 3)WebQSP.856.904.916
Qwen2.5-7B, untrained (Table 3)MINTQA, 3 to 4 hops.192.202.210
1.5B CDU, multi-turn (Table 42)CWQ.782.805.805
1.5B gated, multi-turn (Table 42)CWQ.760.810.818
3B CDU, multi-turn (3 seeds, Table 51)CWQ.796.851.857
3B gated, multi-turn (3 seeds, Table 51)MINTQA, 3 to 4 hops.465.621.627

In short, verifier feedback improves repair beyond a plain retry for frontier models without any training: both messages beat a plain retry in all six Gemini cells, and for Sonnet the named step leads both controls on CWQ and WebQSP. Once a policy is trained with feedback in the loop, the gain holds consistently across training seeds, rewards and model sizes, and on questions deeper than any seen in training, with the named step giving the highest rate in all four 3B rows.

We appreciate the additional experiments, which address many of our concerns. Could the authors clarify what reference information is still used in the question-derived contract and whether step-specific feedback provides consistent additional benefits over generic error messages?

Reviewer_issGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Reasonable-RLVR, which investigates how programmatic verification over knowledge graphs can provide process rewards for large language models. The setup simulates perfect retrieval by providing triples covering the gold path and its surrounding neighborhood, and requires the model to generate structured proofs with fact citations. The method checks whether individual steps are supported by the OWL 2 RL closure and uses a relation contract derived from the gold path to determine whether the generated reasoning chain establishes the stated answer. The authors propose CDU, a weighted process reward, and COU, which satisfies several reward-ordering properties. They conduct reinforcement learning experiments with Qwen2.5-1.5B-Instruct on four knowledge-graph question-answering benchmarks to compare different reward strategies. The results show that verifier-based rewards substantially increase the proportion of answers that are both correct and accompanied by fully valid proofs.

Critical strengths
  1. The method goes beyond checking whether the model cites supported facts by also assessing the supported-claim fraction, reasoning progress, contradictions, and answer establishment. This provides more targeted feedback than answer matching or reference-path overlap alone.
  2. In addition to reporting improvements in valid-proof rates, the paper analyzes failure modes of reward design, including single-claim collapse and padding with unused supported facts.
Critical weaknesses
  1. The improvements in the main table’s correct-and-valid metric may largely reflect alignment between the training rewards and the proof-validity evaluation criteria. The proposed rewards directly optimize signals closely related to proof validity, so some gains may come from preserving citation formats and completing evidence chains rather than improving answer-solving ability. Answer accuracy does not consistently improve and is lower than that of other reward strategies on some datasets.
  2. The process reward uses multiple manually specified weights, but the reported experiments lack systematic component-wise ablations and weight-sensitivity analyses.
  3. The main training experiments use only a 1.5B model, while larger models are primarily evaluated in inference-time experiments without task-specific training. Whether the reward-design findings and training stability extend to larger policies therefore remains unclear.
Questions for authors
  1. Could the authors disentangle the factors contributing to the improvements in correct-and-valid rates and the decreases in answer accuracy?
  2. Could the authors provide independent ablations of the key reward components and investigate how changes in their weights affect answer accuracy?
3 comments

We thank the reviewer. All results below are new and in the revised PDF (Section 6, App. G to M).

W1 and Q1. Reward and metric alignment, and answer accuracy.

How we read the concern. Whether the gain comes from the format or from completing chains rather than better answering, and (Q1) what drives the rise in the correct-and-valid rate and the fall in accuracy.

Our position. The proof rewards are designed to produce verified evidence chains, which is what the correct-and-valid rate measures, so the gain is meant to come from completed chains, not from higher accuracy. We show below that the chains are genuine and that answering is not harmed.

(a) Is it the format? Table 2 of the submission includes a control that pays for a correct answer only when the reply is written as a proof with cited claim lines, without checking the claims. It reaches .401 against .704 for CDU, so the gain comes from checking the proof, not from its format.

(b) Are the chains genuine? An independent checker built on a different OWL reasoner agrees with ours on .978 to .998 of traces (kappa .956 to .996, Table 12), and proofs with a swapped object, a dropped step or an invented identifier are rejected by both checkers in every case on CWQ, Dynamic-KGQA and WebQSP (Table 58). CDU's proofs also link the question to the answer along the graph's edge direction more often than gated in 16 of 19 runs (App. G).

(c) What drives the rise? The correct-and-valid rate is accuracy times the share of correct answers with a valid proof. Against outcome, CDU changes accuracy by -.05 to +.002, while that share rises from between .00 and .43 to between .37 and .97 (Table 29). The rise is correct answers gaining valid proofs.

(d) Why is raw accuracy slightly lower? Accuracy is level on Dynamic-KGQA and higher for outcome by .01 to .05 on CWQ, WebQSP and 2Wiki (Table 29). We classified every test question on which the two disagree (Table 30: four benchmarks, four models, CDU and gated, three seeds). Where only outcome is correct, the proof-trained policy's chain stops short of the answer (.54 to .92) or validly reaches another entity (.02 to .42) and rarely cites an unsupported fact (at most .08), while outcome's own reply has no valid proof in .65 to 1.00 of cases. Raw accuracy counts these unsupported answers, while the proof-trained policy answers through the chain it writes. In exchange its answers can be checked: a CDU answer with a valid proof is correct in .93 to .98 of cases, above outcome's overall accuracy for every benchmark, model and seed, and CDU gives such a proof on .75 to .92 of answers on CWQ and Dynamic-KGQA (Table 29). So the rise is correct answers gaining valid proofs, and the gap in raw accuracy is answers given without support.

(e) Is answering ability lost? No. With 64 samples per question on 300 CWQ questions, the RL policies reach a correct answer on the same share of questions as their warm-up, while the share with a valid proof rises by .12 to .19 under CDU and gated (App. G, Table 28). On MINTQA (5,481 unseen questions of one to four hops), 7B CDU is more accurate than 7B outcome on every seed (.808, .745, .755 against .561, .621, .723, App. G).

W2 and Q2. Components and weights (1.5B, CWQ, five seeds, paired exact tests against the default run of the same seed, Table 54).

(a) Which terms are needed? Removing soundness lowers the correct-and-valid rate on all five seeds and removing establishment on four, and every ablated run (.621 to .747) stays far above outcome (.374).

(b) Are the default weights tuned? Three variants move weight between two terms. Progress to establishment gives a five-seed mean of .723 (significantly above the default on two seeds), progress to soundness .717 (on one), and establishment to progress .732 (above on all five, significantly on three), against .711 for the default. No variant is significantly below the default on any seed, and accuracy stays within .021 above and .007 below it (Q2). The reward is robust to its weights, and the reported results do not depend on a tuned choice.

W3. Larger policies. We trained Qwen2.5-3B, Qwen2.5-7B (all weights) and Llama-3.2-3B as in the paper, three seeds each. CWQ correct-and-valid rate:

ModelOutcomeGatedCDU
Qwen2.5-1.5B (paper, three-seed mean).374.708.704
Qwen2.5-3B.540, .489, .000.805, .796, .795.802, .836, .806
Qwen2.5-7B.000, .034, .003.843, .834, .784.849, .822, .832
Llama-3.2-3B.000, .528, .502.808, .761, .814.875, .861, .877

The rate rises with size under the proof rewards, which stay above outcome at every size. On MINTQA and GTSQA, which none of them was trained on, the proof rewards reach .54 to .91 against .03 to .54 for outcome (seed means, Table 31). All 3B and Llama-3.2-3B runs trained stably, and App. G reports every 7B run. Full results: Table 8, App. G.

Thank you again for your review. We hope our reply and the revision address your points. In brief:

  • W1 and Q1. The gain comes from checking the proof, not from its format (.704 for CDU against .401 for the format control, Table 2). With 64 samples per question the RL policies answer as many questions as their warm-up, and more of those answers carry a valid proof (App. G, Table 28).
  • W2 and Q2. The five-seed ablation and weight sweep show that removing soundness lowers the rate on every seed and that the reward is robust to its weights (App. L, Table 54).
  • W3. Qwen2.5-3B, Qwen2.5-7B and Llama-3.2-3B, three seeds each, reach higher correct-and-valid rates under the proof rewards than the 1.5B model (App. G, Table 8). Since our reply, Qwen2.5-3B on Dynamic-KGQA has completed three seeds: CDU reaches .893, .915 and .921 against .802, .726 and .785 for outcome, at the same answer accuracy (.95 to .96, n = 3,765 per seed, App. G).

If these results resolve your concerns, we would be grateful if you would consider updating your assessment. We are glad to answer any further question before the discussion closes.

As the discussion closes soon, we wanted to thank you again for your time and feedback.

Comments on the paper

1 comment

Summary of the revision (to all reviewers)

We thank the three reviewers. Appendices G to M are new, the latest changes are marked in blue, and the main text keeps its structure: Section 6 gains short paragraphs on model size, contract source and retrieved evidence, depth and rules, and the repair controls, and a sentence on reward weights. The new appendices hold the full results:

AppendixContentReviewer points
G. Model size3B, 7B (all weights) and Llama-3.2-3B, three seeds; pass@64; where the accuracy difference comes from; transfer to MINTQA and GTSQAissG W1, W3
H. Contract sourcecontracts without reference relations, in training and at evaluationUPwu W1, 4ouP (2)
I. Retrieved evidenceblocks retrieved from the question alone, training on them, swapped and coded entity namesUPwu W1
J. Deeper questionsMINTQA questions of up to four hops, GTSQAUPwu W2
K. Repair controlsplain retries and a general message for every model of Table 3 and every policy of Table 5, eight rounds, deeper questions, 3BUPwu Q
L. Reward componentsfive-seed component ablation and weight sweepissG W2
M. Verifier coveragerule necessity, type conditions, constraint check, mutation test, masked claims, reversalsUPwu W2, 4ouP (1) to (3)

Our reply under each review answers its individual points and gives the table numbers of the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.