acceptodds
Under review as a conference paper at ICLR 2027

Revision-Aware Independent Agent Graphs for Dynamic Reasoning

Abstract

Conventional reasoning protocols present a fixed, preselected task, so they cannot test whether an agent propagates relevant updates, preserves unaffected work, or reconstructs a historical task binding. We therefore study dynamic task routing, in which an event stream revises task bindings and a system must select the document version valid at each query time before solving it. To study this problem, we repurpose six widely used benchmarks: MMLU, MMLU-Pro, MedMCQA, MATH, GPQA, and HumanEval into 31,119 dynamic episodes comprising 373,428 temporally categorized queries. This setting exposes a central trade-off: recomputing after every event wastes work, whereas unguarded reuse returns stale conclusions. We introduce the Revision-Aware Independent Agent Graph (RIAG), a bounded multi-agent policy that separates deterministic temporal resolution from task reasoning. RIAG caches solutions by immutable document identity, starts each fresh task with two unexposed attempts, and conditionally invokes audit and repair, using at most four calls per document version. On this collection, homogeneous RIAG achieves 54.24% joint routing-and-answer accuracy at 0.62 calls/query, compared with 32.22% at 18.00 calls/query for the strongest comparison method; heterogeneous RIAG reaches 49.78% at 0.63 calls/query. The datasets and codebase for reproducibility are available in the supplementary materials.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Harness, Agent Graph, Multi-Agent LLM System, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qcaCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces dynamic task routing, an evaluation setting in which a system must select the task document valid at a requested time before answering it. The authors construct 31,119 episodes from six existing benchmarks, with 373,428 queries covering task changes, preservation, propagation, and historical queries. Task contents remain fixed; events change their temporal bindings. The proposed RIAG system combines a deterministic resolver, version-keyed caching, and two initially unexposed reasoning agents, followed by conditional audit and repair.

The evaluation compares RIAG with several multi-agent methods and reports 54.24% joint routing-and-answer accuracy at 0.62 calls/query for homogeneous RIAG, compared with 32.22% at 18 calls/query for Debate. Additional MATH experiments examine resolver transfer, inference settings, cache reuse, and response diversity.

Critical strengths

The evaluation targets a meaningful gap in task-based reasoning benchmarks. A system operating over an event history must distinguish updates that change its task from events that should leave prior work unaffected. The protocol makes this distinction explicit and also tests how retractions affect historical queries. Separately scoring document selection is important: the paper shows that different documents can share the same gold answer string, so answer-only evaluation can hide an incorrect task binding. I regard this evaluation design as the paper’s main contribution.

The experiments show that this distinction matters for the evaluated methods. In Table 1, the comparison methods select the correct document on only about 59–64% of queries, despite using multiple model calls. Their performance also falls substantially between baseline and historical queries. These results identify a concrete weakness of the tested multi-agent pipelines, rather than simply proposing another evaluation category. The fact that a deterministic resolver handles the routing successfully does not diminish this finding: it provides a useful reference and shows that more deliberation is not a substitute for appropriate state handling in these configurations.

Critical weaknesses

The benchmark would benefit from a simpler, matched reference baseline. RIAG’s reasoning agents receive an already resolved task, whereas conventional baselines receive the event history and must select the document themselves. RIAG also reuses cached answers. This is a legitimate whole-system comparison, but it leaves open how much of RIAG is needed to obtain the reported benefits. A resolver + cache + single LLM baseline is particularly important here, since each episode contains only three immutable documents and twelve queries. Its accuracy is unknown, but it would retain the basic routing and reuse mechanisms. The resolver-controlled GoA experiment helps, although it also changes the agent configuration and decoding. These omissions limit the conclusions about RIAG’s reasoning procedure; they do not, in my view, invalidate the evaluation contribution.

The recovery procedure makes the main results difficult to compare consistently. Appendix D.3 states that the homogeneous result includes 34,796 recovery calls replacing 20,739 parse-failed queries, while the heterogeneous result retains 4,453 failures from its pre-fix run. The extra calls are included in the reported cost, but the two configurations were still evaluated under different recovery procedures. It is also unclear how the combined original and recovery executions relate to the stated four-call-per-key limit, given that failed calls are supposed to consume budget and failed results are cached. This needs clarification before the homogeneous–heterogeneous difference and the production-run budget can be interpreted confidently.

Questions for authors
  1. Matched reference baseline. Are results available for a single LLM using the same resolver, cache, and output handling as RIAG? This comparison would help establish a useful reference for the benchmark and show whether the additional reasoning stages improve the accuracy–cost trade-off.

  2. Recovery protocol. What exactly was changed and re-executed in the homogeneous recovery run? Why was the heterogeneous configuration not evaluated with the same corrected implementation? Please clarify whether the four-call limit holds when original and recovery calls for the same key are counted together, or whether the guarantee applies only to a separate corrected execution.

  3. Metric definitions. Table 5 says that RIAG’s answer and joint accuracy rows are identical because its artifacts record one correctness predicate. Were the two metrics computed separately according to Section 5.1? Also, does a missing or malformed answer count as a selection failure even when the resolver selected the correct document? This affects the interpretation of the routing results.

3 comments
AuthorsOpenReview ↗

Response to Reviewer qcaC

Q1. Matched reference baseline with the same resolver, cache, and output handling

Matched control. We added a matched single-solver reference (Section 5.2, Appendix E.1). On 373,428 queries, Qwen2.5-7B-Instruct uses matched prompts, parsing, decoding, and scoring with a 2,048-token cap. All calls and tokens count toward cost.

ConfigurationJoint (%)Parse failuresCalls/queryTokens/query
B1: resolver + solver57.193071.000632.4
B2: B1 + cache57.172820.229145.0
B5: full RIAG58.9700.519355.1

Accuracy–cost decomposition. Caching saves 77.1% of calls and tokens (B2 vs. B1), with −0.02 accuracy points (95% CI [−0.36, +0.30]). Full RIAG adds 1.80 points over B2 (CI [+1.19, +2.38]) at 2.26× the calls and 2.45× the tokens. Dataset gains vary (+1.45 MMLU, +2.55 MMLU-Pro, −1.35 GPQA). The full historical-baseline gap cannot be attributed to multi-agent reasoning.

Routing control. On 600 diagnostic episodes with three seeds (21,600 queries per configuration), a history-based document selector followed by the same solver reaches 33.08% joint accuracy versus 52.96% for resolver + single solver without caching, a +19.88-point difference (CI [+17.05, +22.90]). The solver is matched.

Larger task pools. Qwen3.6-27B evaluates 3/6/12-task pools on matched cores with two seeds (Appendix E.4, Table 13). RIAG's core joint accuracy is 93.1/93.8/94.4%, versus 85.4/86.1/84.7% for the cached single solver. The RIAG core K=12 minus K=3 change is +1.39 points (CI [−2.08, +5.56]). Its full-schedule accuracy declines from 93.1% to 89.1%, while the cached solver declines from 85.4% to 83.2%. At K=12, RIAG uses 0.551 calls/query versus 0.250 for the cached solver. Fixed reuse exposure limits generalization.

Output-budget check. In a separate two-seed 27B cohort, increasing the cap to 4,096 reduces RIAG parse failures from 40 to 3 per 864 queries (Appendix E.3, Table 12). RIAG joint accuracy changes from 83.2% to 86.1%, versus 82.6% to 81.4% for the cached solver. RIAG's advantage is +0.58 and +4.75 points, but source-item CIs [−11.40, +12.77] and [−3.10, +21.47] cross zero. Output validity improves, but a robust accuracy gain is unestablished.

Q2. Recovery protocol and the four-call-per-key limit

Updated matched results. We reran both configurations independently on all 373,428 queries with matched inference, a 2,048-token cap, empty caches, no recovery overlay, and all calls charged. Results replace Table 1 and Tables 4–6, with costs in Table 3.

ConfigurationReturned selectionJoint accuracyCalls/query
Homogeneous RIAG100.0%58.97%0.519
Heterogeneous RIAG100.0%50.64%0.612

Comparison. Heterogeneous minus homogeneous joint accuracy is −8.33 points (95% CI [−9.27, −7.44]). The three-seed diagnostic difference is −6.15 (CI [−10.70, −1.74]). Homogeneous RIAG uses Qwen2.5-7B-Instruct for all roles. Heterogeneous RIAG uses Qwen2.5-7B-Instruct, Mathstral-7B, Mistral-7B-Instruct, and Qwen2.5-Coder-7B-Instruct for solver, checker, auditor, and repairer. Neither matched full run has parse failures or graphs exceeding four calls.

Historical recovery audit. Earlier homogeneous recovery changed parsing, canonicalization, output instructions, and multiple-choice constraints. It replaced 20,739 failed query outputs using 34,796 retained recovery calls. The heterogeneous row retained 4,453 pre-fix failures despite 8,557 attempted recovery calls. The reason for non-substitution is unrecorded, so this historical asymmetry cannot establish a model-pool effect (Appendix E.2).

Four-call scope. Each individual reasoning graph uses at most four calls, including failures. Combining original and retained recovery executions instead put 7,273 of 85,637 semantic keys above four cumulative calls, up to 80 per key. Historical spending totaled 232,616 homogeneous calls plus 39,087 other unretained recovery calls. The guarantee is per fresh execution, not across independent reruns. Cached failures cannot restart the budget.

Q3. Metric definitions, answer and selection failures

Independent metrics. We rescored saved predictions (Section 5.1, Appendix D.1, Table 5). Selection requires a schema-valid final response with the correct document. Answer accuracy checks the returned answer against the gold task regardless of document label. Joint accuracy requires both. Missing or malformed responses score zero on all three, even if the internal resolver route is correct.

Why answer and joint coincide. The original homogeneous run had 94.45% returned selection versus 100% internal routing because of malformed responses. Both matched RIAG runs have 100% returned selection and no parse failures. Answer and joint coincide because accepted responses carry the correct document, not from a shared scoring predicate.

We sincerely appreciate your time and thoughtful evaluation.

Your feedback is invaluable in improving our work. We would be grateful if you could share any further thoughts on our revisions, particularly whether there are any remaining concerns we could address during the discussion period.

Thank you again for your consideration.

Dear Reviewer qcaC,

We wonder if our new results resolve your concerns. If so, we hope you could consider adjusting score if you are comfortable.

Thank you very much.

The author team

Reviewer_RFg2OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2(+1)
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper introduces Dynamic Task Routing, a synthetic evaluation setting constructed from six existing reasoning benchmarks, where agents must select and solve the correct task under evolving temporal events. The authors also propose RIAG, which combines deterministic temporal resolution, version-based caching, and conditional multi-agent reasoning. Experiments show improvements in joint accuracy and inference efficiency over existing multi-agent baselines.

Critical strengths
  1. A new evaluation setting for temporal task management. The benchmark explicitly tests task switching, delayed updates, expiration, retraction, and historical queries, which are not captured by conventional static reasoning benchmarks.
  2. Improved accuracy and efficiency. RIAG achieves substantially higher joint accuracy with fewer model calls than the evaluated baselines.
Critical weaknesses

The benchmark relies on synthetic temporal events applied to three independent, immutable questions from existing non-agentic datasets. It primarily evaluates task routing rather than realistic dynamic reasoning, multitasking, or context switching. Existing benchmarks such as LongMemEval, taubench, Gaia2 already study agents operating in dynamic, asynchronous environments. The paper does not sufficiently establish the unique value or practical relevance of its proposed setting. The experiments primarily use older 7B models, such as Qwen2.5-7B-Instruct. The paper overall has some gap with the recent progression on LLM agents that need to be addressed.

Questions for authors

The paper seems trying to tackle a temporal problem similar to multi-tasking, context switching problems in LLM agents. How is this the newly proposed task compared with those LLM agentic datasets like Gaia2 (Froger et al., 2026), which already evaluates agents in asynchronous environments with evolving tasks and temporal constraints? Given that the proposed evaluation relies on synthetic events and static, independent questions, how well do the findings generalize to realistic interactive agent scenarios?

4 comments
AuthorsOpenReview ↗

Response to Reviewer RFg2

Q1. Comparison with Gaia2 and generalization to interactive scenarios

Unique contribution. Our benchmark scores document selection, answer correctness, and reuse cost under evolving event histories. Gaia2 tests asynchronous tools, τ-bench policy-compliant tool use, and LongMemEval temporal memory. Our diagnostic complements them (Section 2, Appendix E.6).

Why binding matters. Delayed, future-effective, expired, and retracted events alter valid documents. In episode mmlu-task-version-006271, the same historical query selects C before retraction and B afterward. Excluding HumanEval, a history-reading LLM obtains 34.4% answer accuracy but 26.4% joint accuracy, meaning 23.3% of its answer-only credit involves wrong-document selection. Resolver plus cache obtains 50.1% on both metrics.

Why three questions? Three immutable tasks permit competing bindings, switches, and revisits while controlling content and dependencies. This follows bAbI (Weston et al., 2015) and Test of Time (Fatemi et al., 2024), without claiming optimality. We extend the study to 6 and 12 tasks.

Larger task pools. Qwen3.6-27B is tested on 12 source-disjoint MMLU/GPQA/MATH blocks, two seeds, and three methods (216 trajectories, 6,048 queries). A 12-query matched core fixes target content. Full schedules retain four queries per document. Settings are matched (2,048-token cap, thinking disabled).

MethodCore joint, K=3/6/12 (%)Full joint, K=3/6/12 (%)Calls/q at K=12
History reader68.1/75.0/71.268.1/64.6/62.51.000
Cached solver85.4/86.1/84.785.4/83.3/83.20.250
RIAG93.1/93.8/94.493.1/92.7/89.10.551

RIAG's core K=12−K=3 change is +1.39 points (95% CI [−2.08, +5.56]), while full-schedule accuracy decreases to 89.1%. Fixed reuse frequency limits claims about lower-reuse workloads (Appendix E.4, Table 13).

Task dependencies. We test 16 constructed chain, diamond, fanout, and mixed workflows, two seeds, four conditions (128 trajectories, 3,840 queries), with fixed reasoning stages. Supplied dependency tracking without sharing reduces calls by 26.8% and tokens by 26.7% versus full-context invalidation. Adding answer sharing changes workflow-macro terminal accuracy by +0.26 points (CI [−2.08, +2.60]) and leaves overall accuracy unchanged at 903/960. Inferred graphs recover 31/32 exact structures, but one missed edge causes eight stale returned results among 960 queries. Inferred versus supplied sharing changes terminal accuracy by −2.34 points (CI [−9.38, +2.34]). This does not test open-domain semantic discovery (Appendix E.5, Table 14).

Prose-input sensitivity. On 100 MATH episodes in three prose styles, gold-free normalization changes routing-only replay accuracy from 68.3/81.1/0.8% to 77.8/98.8/29.3%. Not end-to-end scores.

Native Gaia2 evaluation. We test 18 held-out calendar/ride and calendar/messaging Adaptability scenarios, three seeds, and matched budgets. The local soft judge is 27B, not the default 70B. Writes are not cached.

MethodPass@1Pass@3Calls/trajectory
ReAct24.1%44.4%19.93
Goal-cache18.5%38.9%12.81
RIAG27.8%61.1%30.59

RIAG succeeds in 15/54 runs across 11/18 scenarios. Its exploratory, post-hoc +3.70-point advantage over ReAct has CI [−7.41, +14.81]. The prespecified variant did not beat both controls. These results show limited interactive execution, not superiority or broad transfer (Appendix E.6, Table 15).

Q2. Experiments relying on older 7B models

Newer-model evidence. We compare Qwen3.6-27B and Qwen2.5-7B-Instruct over 600 episodes, three seeds, and 21,600 queries per configuration/model (matched 2,048-token cap). Joint accuracy requires correct selection and answer.

Configuration7B joint (%)27B joint (%)27B selection (%)
History reader27.364.377.2
History selector + solver33.170.485.4
Resolver + cached solver (B2)52.979.396.2
Full RIAG (B5)54.582.498.2

At 27B, B2 exceeds the history selector by 8.91 joint points (95% CI [7.18, 10.65]) and B5 exceeds B2 by 3.13 ([0.90, 5.46]). However, parse-failure rates are 13.5/4.5/3.8/1.8% in table order. Output validity contributes to the gain. Checkpoints differ beyond size, precluding scaling claims (Appendix E.3, Table 11).

Longer output budgets. On a separate 36-episode, six-dataset, two-seed subset (864 queries per cell), raising the cap from 2,048 to 4,096 reduces RIAG parse failures from 40 to 3 and raises joint accuracy from 83.2% to 86.1%. Cached-solver accuracy changes from 82.6% to 81.4%. RIAG's advantage is +0.58 points (source-item CI [−11.40, +12.77]) and +4.75 ([−3.10, +21.47]) at the two caps. The +4.17-point gap change is also uncertain (CI [−8.84, +22.48]). Longer outputs improve validity without establishing a robust accuracy or cap-dependent advantage (Appendix E.3, Table 12).

Reviewer_RFg2OpenReview ↗

Response to authors

Thank you for the additional experiments. I find them helpful and am happy to adjust the rating.

Thank you for your positive feedback and willingness to reconsider the rating. We truly appreciate your time and thoughtful evaluation.

Your feedback is invaluable in improving our work. We would be grateful if you could let us know whether any concerns remain unclear or insufficiently addressed, so that we can provide further clarification.

AuthorsOpenReview ↗

Final Day to Adjust Rating

Dear Reviewer RFg2,

Thank you for finding our additional experiments helpful and for being willing to adjust your rating. Today is the last day of the rebuttal period. When you have a chance, could you kindly adjust your rating?

Thank you very much.

The author team

Reviewer_ASBsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RIAG is a multi-agent reasoning framework designed for dynamic task environments. The authors focus on the scenario where, as tasks evolve over time, agents should not re-reason from scratch on every occasion, nor should they directly reuse stale answers. Instead, an agent must first determine which task it actually needs to solve, then decide whether historical results can be reused, and finally complete the reasoning through limited multi-agent collaboration. The framework combines Temporal Task Routing, Version-Aware Caching, and Independent Multi-Agent Reasoning into a controllable reasoning system.

Critical strengths

Traditional multi-agent reasoning assumes the task is known in advance, whereas RIAG isolates "selecting the correct task" into an independently evaluable, executable module. This formulation of the problem can surface errors that conventional static benchmarks are unable to assess. Moreover, the combination of caching with temporal revision carries practical significance: an ordinary cache only needs to determine whether the input is identical, whereas a dynamic event system must additionally decide on the valid version of the task first. By parsing the task first and then performing a precise cache lookup, RIAG avoids the problem of directly reusing outdated answers after a task switch.

Critical weaknesses

My main concern is that RIAG lacks ablation experiments that independently validate the effectiveness of each subcomponent. The authors simultaneously employ the Deterministic Resolver, the Version-Keyed Cache, the Independent Solver + Checker, and the Conditional Audit + Repair, yet the paper does not systematically remove these mechanisms one by one to examine their individual contributions.

Questions for authors
  1. Could you supplement a more systematic ablation study—for example, progressively introducing the Version-Keyed Cache, the Independent Solver + Checker, the Auditor, and the Repairer, while keeping the same Resolver, base model, and inference configuration for comparison? This would help further validate the independent contribution of each RIAG module.
    1. Currently, RIAG relies mainly on Exact Version Matching for answer reuse and does not explicitly model semantic associations or dependency relationships between different tasks. For tasks within the same domain that are correlated or exhibit dependency relations, have the authors considered establishing connections between tasks through a Task-Level Graph, so as to support cross-task information sharing and more effective multi-agent reasoning? Could such an extension further improve RIAG's generalization capability in complex dynamic task environments?
3 comments
AuthorsOpenReview ↗

Response to Reviewer ASBs

Q1. Systematic ablation study of each RIAG module

Progressive ablation. We added the B1–B5 ablation (Appendix E.1, Tables 7–10), holding the Qwen2.5-7B model, resolver, prompts, parser, and decoding settings fixed (2,048-token cap). The cohort has 21,600 queries per configuration (600 episodes, three seeds). B3 uses the unexposed checker as fallback without replacing valid solver answers. B4/B5 audit disagreement or invalidity. All calls/tokens count, and malformed outputs fail.

ConfigurationJoint (%)Parse failuresCalls/qTokens/q
B1: resolver + solver52.962131.000667
B2: B1 + cache52.942060.227151
B3: B2 + checker52.72590.454304
B4: B3 + auditor54.6680.548433
B5: B4 + repairer54.4500.553443

Individual component effects. Source-aware 95% intervals for accuracy increments are B1→B2: −0.01 [−1.41, 1.35], B2→B3: −0.23 [−1.86, 1.53], B3→B4: +1.94 [−0.56, 4.85], and B4→B5: −0.20 [−2.15, 1.94] percentage points. None establishes an independent accuracy benefit.

Matched accuracy–cost controls. On the full collection (373,428 queries, one seed), B1, B2, and B5 achieve 57.19%, 57.17%, and 58.97% joint accuracy at 1.000, 0.229, and 0.519 calls/query, respectively. Caching saves 77.1% of calls and tokens (accuracy −0.02 points [−0.36, 0.30]). Adding the reasoning stages improves accuracy by 1.80 points [1.19, 2.38] over B2, at 2.26× calls and 2.45× tokens. Diagnostic uncached B6 achieves 55.07% at 2.438 calls/query versus B5 at 54.45% and 0.553, saving 77.3% of calls (accuracy −0.62 points [−2.34, 1.02]). Cohorts are not pooled.

Recovery and escalation. Retained outputs from 85,637 fresh B5 graphs assess recovery without resampling. Unanswered graphs fall from 63 with the solver alone to 13 after checker fallback, 2 after audit, and 0 after repair. The checker recovers 50/63 responses, including 22 correct. Among 21,625 flagged graphs, solver error is 70.8% versus 33.4% under agreement, but flags capture only 41.8% of solver errors. Audit produces 4,329 corrections and 2,826 regressions (net +1,503 fresh tasks). Repair yields 125 corrections and 133 regressions (net −8). Its final two previously unanswered recoveries are both incorrect. Repair runs in 847 graphs (0.99%), using 0.44% of calls and 0.78% of tokens. No fresh graph exceeds four calls.

Interpretation. Caching saves computation, checking enables fallback/escalation, and auditing yields net corrections in retained traces. Repair offers limited output recovery without established accuracy benefit. These descriptive results do not establish component necessity. The bound excludes reruns.

Q2. Task-Level Graph for cross-task information sharing

Dependent-workflow evaluation. We added a Task-Level Graph evaluation (Appendix E.5, Table 14) that separates dependency-aware invalidation, sharing intermediate answers, and inferring relationships. Qwen3.6-27B evaluates 16 constructed workflows across chain, diamond, fanout, and mixed structures, with two seeds and four conditions (128 trajectories, 3,840 queries). Each has two MATH roots, four dependent tasks, and five checkpoints. The reasoning procedure is fixed (2,048-token cap, temperature 0.7, thinking disabled). Only solvers see shared answers. Extraction cost is included.

ConditionTerminal correctOverall correctCallsTokensStale
Original RIAG, full invalidation196/224898/9602,1714,465,0130
Supplied graph, no sharing197/224903/9601,5903,272,4380
Supplied graph + sharing198/224903/9601,5993,384,6710
Inferred graph + sharing195/224893/9601,6183,365,9558

Dependency tracking versus answer sharing. Supplied dependency tracking without sharing saves 26.8% of calls and 26.7% of tokens relative to conservative invalidation. The workflow-macro terminal change is +0.91 points (95% CI [−0.52, 2.86]). Sharing raises pooled terminal correctness by just one query, leaves overall correctness unchanged, and increases cost. Its workflow-macro change is +0.26 points [−2.08, 2.60], so an accuracy benefit is not established.

Inferred dependencies and stale reuse. The inferred condition predicts relationships from public titles and paraphrased descriptions without supplied edges, answers, update schedules, or query order. Across 32 graphs, it finds 167 correct edges, zero spurious edges, and one missed edge (31/32 exact). That single omission causes eight stale returned results, versus zero with supplied graphs. Inferred sharing changes workflow-macro terminal accuracy by −2.34 points [−9.38, 2.34] relative to supplied sharing. All runs are retained. Intervals use 10,000 paired bootstrap draws. These constructed workflows support selective reuse under correct dependencies, not sharing gains, safe inferred reuse, or broad generalization.

We sincerely appreciate your time and thoughtful evaluation.

Your feedback is invaluable in improving our work. We would be grateful if you could share any further thoughts on our revisions, particularly whether there are any remaining concerns we could address during the discussion period.

Thank you again for your consideration.

Dear Reviewer ASBs,

We wonder if our new results resolve your concerns. If so, we hope you could consider adjusting score if you are comfortable.

Thank you very much.

Best, The author team

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response summary for Reviewers and ACs

Overview. We thank the reviewers for their constructive feedback, which has helped strengthen the manuscript. In response, we independently reran homogeneous and heterogeneous RIAG under a matched protocol and added a matched single-solver baseline, progressive component ablations, newer-model and output-budget evaluations, larger task pools, dependent workflows, and a limited native Gaia2 study. We also clarified evaluation metrics and historical recovery accounting.

Main finding. On 373,428 queries, version-keyed caching reduces model calls and tokens by 77.1%. Relative to the matched cached single solver, full RIAG improves joint accuracy by 1.80 percentage points (95% CI [1.19, 2.38]). The revised abstract, introduction, and conclusion accordingly emphasize controlled temporal task binding, version-safe reuse, and bounded inference rather than general dynamic reasoning.

Summary of manuscript revisions.

RevisionAdded evidence or clarificationLocation
Matched baselineSame resolver, cache, solver prompt, and output handling, isolating the reasoning stack's accuracy–cost trade-off§5.2, App. E.1
Component ablationB1–B5 progression with uncertainty, parsing, computation, checker escalation, auditing, and repair analysisApp. E.1, Tables 7–10
Recovery and call budgetMatched homogeneous/heterogeneous reruns, historical recovery audit, and the scope of the four-call boundTables 1, 3–6, Apps. D.3, E.2
Metric definitionsIndependent selection, answer, and joint scoring, including malformed-response handling and revised Table 5 caption§5.1, App. D.1, Table 5
Newer model and output budgetQwen3.6-27B comparisons and matched 2,048/4,096-token testsApp. E.3, Tables 11–12
Task-pool sensitivityMatched-core and full-schedule evaluation with 3, 6, and 12 tasksApp. E.4, Table 13
Dependent workflowsDependency tracking, intermediate-answer sharing, inferred edges, and stale reuse across four graph familiesApp. E.5, Table 14
Interactive scopeBenchmark positioning, prose-input analysis, and bounded native Gaia2 evidence§2, App. E.6, Table 15

Evaluation scope. The primary matched study uses Qwen2.5-7B-Instruct across MMLU, MMLU-Pro, GPQA, MATH, HumanEval, and MedMCQA, with 31,119 episodes and 373,428 queries (one seed). Diagnostic analyses use 600 episodes and three seeds. Accuracy differences are reported in percentage points, and uncertainty calculations account for clustering or source-question reuse as specified for each study.

Dear Reviewers,

We are now halfway through the discussion phase. The authors have provided their responses to your reviews. If their responses have not adequately addressed your concerns, please take this opportunity to raise any remaining questions or concerns.

I encourage you to engage with the authors as soon as possible to ensure a productive discussion.

Many thanks,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.