acceptodds
Under review as a conference paper at ICLR 2027

CREDO: Evidence-Conditioned Credit Assignment for Temporal Reasoning Agents

Abstract

Temporal Knowledge Graph Question Answering (TKGQA) requires language model agents to retrieve evidence, select relevant facts, and perform multi-step temporal reasoning. However, the trajectory-level outcome signal in GRPO does not explicitly distinguish the quality of intermediate decisions: downstream failures can penalize useful evidence acquisition, while identical rewards in all-failed groups eliminate the relative outcome signal despite differences in intermediate progress. We propose Credit Redistribution with Evidence-Driven Opportunities (CREDO), which uses intermediate evidence to recover local supervision and determine where outcome supervision acts. CREDO assigns positive stage-level credit through comparisons among rollouts of the same question and uses retained answer overlap as an opportunity proxy to shift part of the outcome-policy weight toward reasoning and answering. The framework preserves terminal rewards and trajectory-level advantages, while redistribution conserves total outcome-policy weight per trajectory. Following the public evaluation protocol of prior work, CREDO achieves 82.4% overall accuracy on MultiTQ, surpassing the strongest published baseline in our comparison by 4.4 percentage points overall and 10.6 percentage points on Multiple questions. On TimelineKGQA, it improves cross-KG accuracy over our reproduced baseline by 5.5 percentage points via direct transfer to ICEWS Actor under matched training and evaluation settings.

Primary area:
reinforcement learning
Keywords:
Temporal Knowledge Graph Question Answering, Reinforcement Learning, Fine-Grained Credit Assignment, LLM Agents, Temporal Reasoning, Retrieval-Augmented Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_p8xKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work investigates credit assignment difficulties encountered when training reinforcement learning agents for temporal knowledge graph question answering (TKGQA) tasks. The authors propose CREDO (Credit Redistribution with Evidence-Driven Opportunities), which uses intermediate evidence to recover local supervision signals even when all rollouts for a question fail. CREDO assigns positive stage-level credit through within-question rollout comparisons and uses evidence opportunity proxies to shift outcome policy weight toward reasoning and answering stages. Experiments on MultiTQ and TimelineKGQA datasets show significant improvements over baseline methods, particularly for multi-hop temporal reasoning tasks.

Critical strengths

Critical Strengths

1.The paper addresses an important credit assignment problem in reinforcement learning for language agents.

2.The idea of using intermediate evidence as a proxy for partial credit is conceptually appealing.

Critical weaknesses

Critical Weaknesses

1.Evaluation limitations: The paper only evaluates on two small temporal KG QA datasets, and the reported improvements are marginal (less than 5% absolute) compared to strong GRPO baselines with proper hyperparameter tuning. The improvements disappear when using larger backbone models, suggesting the approach has limited practical value.

2.Theoretical inconsistencies: The credit redistribution mechanism violates the policy gradient theorem by arbitrarily shifting weights between stages without proper theoretical justification, which can lead to unstable training dynamics not reported in the paper.

3.Lack of ablation analysis: The paper does not include sufficient ablation studies to isolate the contribution of the credit assignment mechanism from other confounding factors in the training pipeline.

4.Limited generalizability: The approach is highly specialized to TKGQA tasks with explicit intermediate evidence markers, and cannot be easily extended to general agent tasks where intermediate evidence is not clearly defined.

Questions for authors

None

3 comments
AuthorsOpenReview ↗

Response to Reviewer

We sincerely thank the reviewer for the careful reading of our manuscript and the constructive feedback. We have revised the manuscript and added supplementary experiments and analyses to address the concerns raised. Changes in the revised PDF are highlighted in blue. The weighted-objective clarification and coefficient derivation are in Section 3.4 and Appendix A.1; the main 8B results and paired comparisons are in Appendix A.3 (Tables 5 and 6). The additional 3B method comparison is reported here in this response.

Weakness1: Evaluation scope, improvement magnitude, and backbone size

Our main benchmark and component experiments use Meta-Llama-3.1-8B-Instruct. The reproduced outcome-only GRPO baseline shares CREDO's SFT initialization, training data, terminal reward, and base training and evaluation settings. We distinguish this matched comparison from the published Temp-R1 reference:

ComparatorMetricBaseline (%)CREDO (%)Gain (pp)
Matched GRPOOverall77.0082.40+5.40
Matched GRPOMultiple51.8865.63+13.75
Published Temp-R1 referenceOverall78.0082.40+4.40

We additionally report a 3B method comparison, selecting each method's highest-Overall checkpoint among the reported checkpoints. All category-level results come from the same selected checkpoint for each method. Accuracy values are percentages.

MethodOverallMultipleSingleEntityTimeCREDO Overall gain (pp)
CREDO73.2045.6286.1866.0493.80—
CWO72.0045.6284.4165.2391.47+1.20
GRPO70.4041.2584.1263.3490.70+2.80

The main gains are already obtained with the larger 8B backbone. Both reported model scales show positive Overall gains in their respective experimental settings, directly addressing the concern that the improvement disappears on the larger backbone.

Weakness2: Optimization objective, policy gradients, and training control

CREDO optimizes an explicitly defined evidence-conditioned weighted PPO surrogate:

J(θ)=Joutcomew(θ)+λlocalJlocal(θ).J(\theta)=J_{\mathrm{outcome}}^{w}(\theta)+\lambda_{\mathrm{local}}J_{\mathrm{local}}(\theta).

The outcome term retains the original terminal rewards and trajectory-level GRPO advantages, while its token weights specify where that supervision acts. The local term supplies the additional stage-level learning signal.

Redistribution changes the surrogate being optimized; it does not assert that stage reweighting leaves the original unweighted policy gradient unchanged. Weight conservation controls the allocation budget, while the weighted objective explicitly defines the optimization update.

QuantityDefinition or controlDefault setting or implication
Total trajectory policy weight∑twit=ni\sum_t w_{it}=n_iAllocation budget is conserved
Reasoning/answering weight sharef+ρ(1−f)f+\rho(1-f)With ρ=0.04\rho=0.04, a 5% token share receives 8.8% of the weight
Minimum eligible reasoning fractionf≥ηf\geq\etaη=0.05\eta=0.05
Trainable-token coefficient bounds1−ρ≤wit≤1−ρ+ρ/η1-\rho\leq w_{it}\leq1-\rho+\rho/\eta[0.96,1.76][0.96,1.76]

Here, ff is the reasoning/answering fraction of trainable tokens. These bounds prevent unbounded weight concentration on a short reasoning span. The implementation also retains PPO ratio clipping and KL regularization. Sections 3.3-3.4 and Appendix A.1 define the objective and allocation controls; Figure 5(a) and Table 3 provide training-stage evaluations and coefficient-sensitivity diagnostics. Table 3 concerns the existing sensitivity study, while the additional 3B method comparison is given above.

AuthorsOpenReview ↗

Response to Reviewer

We continue our response with the component ablations and applicability beyond TKGQA.

Weakness3: Component ablations and attribution of the observed gains

The main 8B experiments include component comparisons examining local credit, redistribution target positions, and evidence-conditioned eligibility:

VariantConfigurationOverall (%)CREDO gain (pp)
GRPOOutcome-only supervision77.0+5.4
CWOLocal credit without outcome redistribution80.2+2.2
RandomLocal credit with randomized redistribution targets79.0+3.4
No-gateLocal credit and redistribution without evidence-overlap eligibility78.4+4.0
CREDOFull evidence-conditioned configuration82.4—

These comparisons share base training settings, the checkpoint-selection criterion, and final evaluation settings. Random preserves redistribution eligibility, transferred weight, and target-token count while changing the target positions. No-gate removes the evidence-overlap requirement for redistribution while retaining structural and length eligibility; its local-credit mechanism remains unchanged.

The full configuration achieves the highest observed accuracy across these comparisons. Component results and the corresponding paired analyses are reported in Appendix A.3, including Table 6.

The original CREDO training trace separately demonstrates the availability of intermediate supervision within the same all-wrong question groups:

Learning signal in all-wrong groupsGroups with a nonzero signalProportion
Outcome-derived GRPO advantage0/1,4730.00%
CREDO stage-local credit564/1,47338.29%

This directly demonstrates that the intended intermediate learning signal remains available when outcome advantages are zero. The original trace and counts appear in Figure 3 and Appendix A.3 ("All-wrong groups").

Weakness4: Applicability beyond TKGQA

CREDO targets multi-stage evidence-seeking agents, with TKGQA as the experimental instantiation. Its local-credit and redistribution rules operate on stage quality, upstream evidence opportunity, and stage-token masks; these rules do not depend on a particular knowledge-graph schema or temporal operator.

A concrete extension is extractive document QA, where the required inputs can be instantiated as follows:

CREDO interfaceTKGQA instantiationExtractive document QA instantiation
Retrieved evidenceRetrieved KG factsRetrieved passages
Retained evidenceSelected factsSelected passages
Evidence-opportunity proxyReference-answer coverage in factsReference-answer coverage in passages
Answering-stage qualityGenerated answer compared with referencesGenerated answer compared with references
Stage-token masksRetrieval, selection, and answering spansCorresponding spans in the document-QA trace

The same reference-answer coverage definitions can be applied to passages, allowing the within-question comparisons and outcome-redistribution rules to be instantiated without changing their core formulation.

The required interface is observable intermediate evidence and identifiable stage boundaries. In our implementation, these boundaries come from the agent's interaction trace and do not require additional human annotations of intermediate-step correctness. Thus, applicability depends on the structure and observability of the agent workflow. CREDO is a credit-assignment framework for staged evidence-seeking agents, instantiated and evaluated here on TKGQA.

We hope these responses address the concerns raised and that the reviewer will reconsider the assessment.

Dear Reviewer p8xK,

We are writing to kindly follow up on our rebuttal submitted a few days ago. We would appreciate your feedback on whether our responses address your concerns.

Thank you very much for your time and continued effort in reviewing our work. We look forward to any further feedback you may have.

Best regards.

Reviewer_cfMwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CREDO, which augments outcome-based reinforcement learning with local evidence-conditioned credit and redistribution of outcome supervision across retrieval, selection, and answering stages. CREDO is evaluated on temporal question answering, with matched training comparisons, component ablations, and fixed-evidence distraction tests. The method uses differences among trajectories for the same question to guide which stages receive additional supervision.

Critical strengths
  1. The method addresses a relevant limitation of outcome-only training: intermediate stages can receive useful local supervision even when all final answers are wrong. This makes the objective responsive to partial progress in evidence acquisition instead of relying exclusively on terminal correctness.

  2. MultiTQ accuracy improves from 77.0% to 82.4% compared with GRPO. The paper also explicitly documents limitations of its evidence proxies.

Critical weaknesses
  1. Evidence quality is approximated by answer-token overlap, while evaluation accepts normalized answer substrings. An answer can appear in a response even when the supporting temporal relation is wrong, so semantic correctness remains an unresolved question.

  2. The added value of redistribution over local credit alone remains uncertain. CREDO exceeds CWO by 2.2 pp, but the paired interval is [−0.80, 5.20]. Thus, the overall improvement does not yet isolate this component's benefit.

Questions for authors
  1. Do the gains survive strict single-answer scoring?

  2. How often does the full evidence gate accept temporally or relationally incorrect support on sampled natural trajectories?

3 comments
AuthorsOpenReview ↗

Response to Reviewer

We sincerely thank the reviewer for the careful reading of our manuscript and the constructive feedback. We have revised the manuscript and added supplementary experiments and analyses to address the concerns raised. Changes in the revised PDF are highlighted in blue. The new strict-answer results and gate audit are reported in Appendix A.5 (Tables 9 and 10 and the accompanying text); the original paired component comparisons are reported in Appendix A.3 (Table 6).

Q1. Do the gains survive strict single-answer scoring?

Yes. The supplementary strict-reward checkpoints retain CREDO's accuracy advantage under strict single-answer scoring. We evaluate these checkpoints on the fixed 500-question MultiTQ set, separately from the original substring-trained models. Strict exact match requires the complete normalized extracted answer to equal an acceptable reference; additional candidates do not receive credit merely through answer containment. Both metrics evaluate the same generated responses for each checkpoint.

MethodStrict EM (%)Substring (%)Substring-only correctCREDO gain in strict EM (pp)
CREDO79.4079.400/500—
CWO77.8077.800/500+1.60
GRPO75.2076.004/500+4.20

CREDO retains its accuracy under both scoring rules, with no substring-only correct outputs. Its observed advantage in this supplementary evaluation therefore persists when scoring requires an exact normalized answer.

Q2. How often does the full evidence gate accept temporally or relationally incorrect support on sampled natural trajectories?

We measured this directly by auditing 1,072 natural trajectories from 134 explicit-date questions, using rule-based full-KG verification to check supporting facts. The audit evaluates the complete runtime gate, including its structural and token-length conditions.

Support among gate-accepted trajectoriesCountShare (%)Question-cluster 95% CI (%)
Valid support520/55194.37—
Temporal mismatch5/5510.91—
Relation/object-support failure26/5514.72—
Invalid support (subtotal of the two failure categories)31/5515.63[2.05, 10.12]

The table quantifies how often the actual gate accepts invalid support and identifies the corresponding error types. Within the audited explicit-date subset, gate-accepted trajectories predominantly contain valid supporting evidence, providing a measured basis for using the signal to guide stage-level credit allocation.

Additional clarification on the contribution of redistribution

The original component comparisons examine whether to redistribute outcome supervision, where to direct it, and when to activate it. Under matched base training and evaluation settings:

MethodConfigurationOverall (%)CREDO gain (pp)
CREDOFull configuration82.4—
CWOLocal credit without outcome redistribution80.2+2.2
RandomRandomized redistribution targets79.0+3.4
No-gateRedistribution without evidence-overlap eligibility78.4+4.0

Random changes the target positions while preserving the redistribution budget and target count. No-gate removes the evidence-overlap conditions for redistribution while retaining structural and length requirements. The full configuration achieves the highest observed accuracy across these comparisons, providing complementary evidence for combining reasoning/answering targets with evidence-conditioned eligibility. The corresponding paired comparisons remain available in Appendix A.3, Table 6.

We hope these responses address your questions and the concern about redistribution, and that you will reconsider the assessment.

Dear Reviewer cfMw,

We are writing to kindly follow up on our rebuttal submitted a few days ago. We would appreciate your feedback on whether our responses address your concerns.

Thank you very much for your time and continued effort in reviewing our work. We look forward to any further feedback you may have.

Best regards.

I thank the authors for their responses. I would like to maintain my score.

Reviewer_YvffOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

CREDO aims to enable agents to learn from unsuccessful responses. It compares multiple trajectories for the same query, rewards superior intermediate steps, and adjusts training weights across the retrieval, filtering, and response-generation stages based on the available evidence.

Critical strengths
  1. The rationale is sound: an incorrect final answer does not mean that the preceding steps lack value. Furthermore, this approach requires no additional step-by-step annotations or evaluation models.
  2. Overall performance has improved. Under identical experimental settings, accuracy on MultiTQ increased from 77.0% to 82.4%, with gains also observed in cross-knowledge-graph testing.
Critical weaknesses
  1. The novelty seems limited. The method looks like an incremental change rather than a clear methodological advance.
  2. The evidence score does not tell us whether the temporal information is correct. A passage may contain the right answer name but refer to the wrong time or relationship. This makes it hard to separate poor evidence retrieval from mistakes in reasoning, which is central to the paper’s motivation.
  3. The benefit of reallocation is unclear. Keeping the total weight unchanged does not show that the new allocation is better. The full method improves over local rewards alone by 2.2 percentage points, but the confidence interval includes zero. This leaves the added value of reallocation uncertain.
  4. The evaluation can reward answers that contain both correct and incorrect candidates. Since an output receives credit for containing the ground-truth answer, a higher score does not necessarily mean the model has selected the right answer. This weakens the evidence that the method improves temporal reasoning and answer reliability.
Questions for authors
  1. Compared to existing methods, what is the most significant new contribution of this paper?
  2. How does the method handle cases where the evidence contains the answer term, but the temporal aspect or the relationship is incorrect?
  3. Is the current scoring approach still applicable if the answer requires calculation and does not appear explicitly in the evidence?
  4. Why should a larger, fixed proportion of training weight be allocated to the reasoning and answering phases once certain conditions are met?
  5. In an evaluation setting that allows for additional candidate answers, what conclusions regarding temporal reasoning capabilities do the authors believe the results support?
6 comments
AuthorsOpenReview ↗

Response to Reviewer

We sincerely thank the reviewer for the careful reading of our manuscript and the constructive feedback. We have revised the manuscript and added supplementary experiments and analyses to address the concerns raised. Changes in the revised PDF are highlighted in blue. The original training-trace data are reported in Figure 3 and Appendix A.3; the newly added gate-audit results and error breakdown are reported in Appendix A.5, Table 10 and the following paragraph. Below, we respond to each question in detail.

Q1. Compared to existing methods, what is the most significant new contribution of this paper?

Existing approaches construct credit from different sources. Outcome-supervised GRPO derives trajectory-level advantages from terminal rewards. Finer-grained approaches introduce intermediate feedback: TIPS uses changes in a teacher model's correct-answer likelihood, StepSearch uses information gain and redundancy penalties for search supervision, and GiGPO constructs step-level comparison groups from repeated environment states across trajectories.

CREDO's distinguishing contribution is to explicitly separate stage-local quality from upstream evidence opportunity in the retrieval-selection-answering pipeline. These quantities answer different questions: how well a stage performed, and what answer-related evidence preceding stages made available. CREDO compares corresponding stages across trajectories for the same question and conditions downstream local credit on upstream evidence. This allows intermediate progress to remain learnable even when terminal rewards coincide, without requiring identical intermediate states or teacher-likelihood scoring.

The evidence-opportunity signal also determines eligibility for redistributing the original outcome supervision toward reasoning and answering, while preserving terminal rewards and trajectory-level GRPO advantages. The main contribution is therefore a framework in which intermediate evidence jointly determines which local behaviors to reinforce and where outcome supervision should act.

The original CREDO training trace illustrates the complementary learning signals available within the same all-wrong question groups:

Learning signal in all-wrong groupsGroups with a nonzero signalProportion
Outcome-derived GRPO advantage0/1,4730.00%
CREDO stage-local credit564/1,47338.29%

Thus, CREDO provides an intermediate learning signal in groups where outcome-derived advantages vanish, directly supporting the intended mechanism.

Q2. How does the method handle cases where the evidence contains the answer term, but the temporal aspect or the relationship is incorrect?

CREDO combines answer-overlap scores with stage eligibility and within-question comparisons to assign local credit, and uses the full runtime gate to determine whether outcome redistribution is activated. Temporally or relationally incorrect evidence can still receive positive overlap and pass these checks. We therefore directly evaluated the complete gate on natural trajectories, measuring both support validity and the remaining error types.

The audit covers 1,072 trajectories from 134 explicit-date questions, with supporting facts checked through rule-based full-KG verification. The results among gate-accepted trajectories are:

Rule-verified support among accepted trajectoriesCountShare of accepted trajectories
Valid support520/55194.37%
Temporal mismatch5/5510.91%
Relation/object-support failure26/5514.72%
Invalid support (subtotal of the two failure categories)31/5515.63%

The audit evaluates the actual selection made by the complete gate and identifies the specific semantic errors that remain. Within the audited explicit-date subset, accepted trajectories predominantly contain valid supporting evidence, providing an empirical basis for using the signal to guide stage-level credit allocation. CREDO uses this evidence-relevance signal to direct learning across stages; overlap itself is not a definitive diagnosis of whether an error originated in retrieval or reasoning.

AuthorsOpenReview ↗

Response to Reviewer

Q3. Is the current scoring approach still applicable if the answer requires calculation and does not appear explicitly in the evidence?

CREDO evaluates computed results against reference answers and assigns auxiliary credit at the retrieval, evidence-selection, and reasoning/answering stages. A correct computed result can receive terminal reward even when it does not appear in the supporting facts. The reasoning/answering stage is the credit unit, with its quality assessed through the final answer; individual arithmetic operations are not separately scored. When retrieved and retained evidence both have zero reference-answer token overlap, the auxiliary local advantages are zero and redistribution uses the original actor weights. This fallback preserves the original outcome-supervised objective for these trajectories, while additional evidence-conditioned credit is applied when its eligibility conditions are satisfied. The computed-answer clarification is included in Appendix A.5, following the Table 10 audit results.

Q4. Why should a larger, fixed proportion of training weight be allocated to the reasoning and answering phases once certain conditions are met?

Retained answer overlap signals an opportunity to use the acquired information for reasoning and answering. The coefficient ρ\rho controls redistribution strength, rather than fixing the final proportion assigned to this stage. If reasoning/answering accounts for a fraction ff of trainable tokens, its outcome-policy weight share becomes f+ρ(1−f)f+\rho(1-f). With the default ρ=0.04\rho=0.04, a 5% token share becomes an 8.8% weight share.

Weight conservation controls the redistribution budget, while component comparisons assess the allocation's practical value. Under matched base training and evaluation settings:

MethodConfigurationOverall (%)CREDO gain (pp)
CREDOFull configuration82.4—
CWOWithout outcome redistribution80.2+2.2
RandomRandomized redistribution targets79.0+3.4
No-gateWithout evidence-overlap eligibility78.4+4.0

Random preserves the redistribution budget and target count while changing target positions. No-gate retains structural and length requirements. These comparisons examine whether to redistribute, where to direct supervision, and when to activate redistribution. CREDO achieves the highest observed accuracy, supporting the combination of evidence-conditioned eligibility and redistribution toward reasoning and answering. Component results and paired comparisons are reported in Appendix A.3, including Table 6.

Q5. In an evaluation setting that allows for additional candidate answers, what conclusions regarding temporal reasoning capabilities do the authors believe the results support?

We examine answer selection and temporal evidence use through two complementary evaluations, reported in Appendix A.5 (Tables 9 and 11).

First, we evaluate strict-reward follow-up checkpoints on the fixed 500-question MultiTQ set, separately from the original substring-trained models. Strict exact match requires the complete normalized extracted answer to equal an acceptable reference, so additional candidates do not receive credit merely through answer containment. Both metrics score the same generated responses for each checkpoint.

MethodStrict EM (%)Substring (%)Substring-only correct
CREDO79.4079.400/500
CWO77.8077.800/500
GRPO75.2076.004/500

CREDO retains its accuracy and advantage under strict scoring, with no substring-only correct outputs.

Second, we test temporal evidence use through a date-swap diagnostic using separate diagnostic checkpoints. We exchange the dates of two facts while preserving the question, entities, relations, and fact order. The original and modified gold-answer sets are disjoint, and success requires both answers to be correct. The diagnostic covers 60 questions and two decoding seeds:

MethodBoth versions correctPaired accuracy (%)
CREDO70/12058.33
CWO53/12044.17

Counts refer to output pairs across the two decoding seeds. A fixed answer cannot satisfy both versions, so success requires correct answer adaptation to the changed temporal information.

Together, these evaluations provide evidence for accurate answer selection under strict scoring and correct final-answer adaptation to temporal changes.

We hope these address Q1-Q5 and that the reviewer will reconsider the assessment.

Thank you for your responses and additional experiments. While the new results help clarify several of my concerns, I still have reservations about the methodological novelty, the reliability of evidence-based credit assignment and also the independent contribution of outcome redistribution. Therefore, I will maintain my original rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Clarifications and Additional Evidence

We thank the reviewers for their careful feedback. We summarize the clarifications and evidence addressing recurring concerns about answer evaluation, evidence reliability, and credit assignment. Changes are highlighted in blue in the revised PDF. New answer/evidence diagnostics appear in Appendix A.5; the optimization clarification is in Section 3.4 and Appendix A.1.

What CREDO contributes. CREDO separates stage-local quality from upstream evidence opportunity. Intermediate evidence determines both which local behaviors receive additional credit and where outcome supervision acts. The original training trace demonstrates the complementary signals available within the same all-wrong groups:

Signal in the same all-wrong groupsGroups with active signal
Outcome-derived advantage0/1,473
CREDO local credit564/1,473 (38.29%)

The trace and counts are reported in Figure 3 and Appendix A.3.

The supplementary advantage persists under strict answer selection. Strict exact match requires the complete normalized extracted answer to equal an acceptable reference. These strict-reward checkpoints are separate from the original substring-trained models; both scoring rules evaluate the same responses for each checkpoint.

Supplementary evaluationCREDOCWOGRPO
Strict EM, 500 questions79.40%77.80%75.20%
Substring-only correct outputs0/5000/5004/500
Date-swap pairs both correct70/120 (58.33%)53/120 (44.17%)—

The date-swap diagnostic uses separate diagnostic checkpoints and 60 questions with two decoding seeds. Original and modified gold sets are disjoint, so repeating a fixed answer cannot pass both versions. These results support strict answer selection and correct final-answer adaptation to temporal changes (Appendix A.5, Tables 9 and 11).

The full gate's reliability is now directly measured. We audited the complete runtime gate on 1,072 natural trajectories from 134 explicit-date questions using rule-based full-KG verification.

Support among gate-accepted trajectoriesCountShare
Valid support520/55194.37%
Temporal mismatch5/5510.91%
Relation/object-support failure26/5514.72%
Invalid support (subtotal)31/5515.63%

The audit measures the gate's actual support validity and remaining error types in the audited subset. It evaluates the full eligibility mechanism used in training, beyond answer overlap alone (Appendix A.5, Table 10 and accompanying text).

The main gains already occur on the 8B backbone, with explicit component controls.

Setting / metricCREDOCWOMatched GRPORandomNo-gate
8B Overall82.40%80.20%77.00%79.00%78.40%
8B Multiple65.63%61.88%51.88%57.50%57.50%

CWO, Random, and No-gate respectively test whether to redistribute outcome supervision, where to direct it, and when to activate it. The full configuration achieves the highest observed accuracy. Appendix A.3 reports the original component results and paired estimates (Tables 5 and 6 and accompanying text). Published Temp-R1 scores are separately identified as literature references.

Additional model scaleCREDOCWOGRPO
3B Overall73.20%72.00%70.40%

This additional 3B comparison is reported in our reviewer response, with each method selected by its highest reported Overall accuracy. Both reported scales show positive Overall gains in their respective settings.

Redistribution specifies the optimization target and has bounded allocation. CREDO optimizes an evidence-conditioned weighted PPO surrogate while retaining terminal rewards and trajectory-level GRPO advantages. The fixed parameter ρ\rho controls mixing strength; the final reasoning/answering share is f+ρ(1−f)f+\rho(1-f) and depends on the trajectory's token allocation. Appendix A.1 derives weight conservation and bounded token coefficients. PPO clipping and KL regularization are retained. These statements concern the defined weighted objective, rather than an unchanged original unweighted policy gradient.

We hope these clarifications and additional results help the reviewers reassess the contribution and empirical evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.