- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Summary
This paper proposes ReTrace, a two-timescale framework for web traversal agents that combines task-internal recovery with cross-task consolidation of recovery experience. The method aims to improve web navigation by enabling agents to backtrack, explore alternative branches, reflect on failed attempts, and reuse successful recovery strategies in subsequent tasks. The paper evaluates the framework on WebWalkerQA and V-GEMS and includes ablations of its fast adaptation and slow learning components.
Critical strengths
Strengths
-
A well-motivated framework that addresses both local recovery and cross-task learning. The paper identifies two complementary limitations of existing web agents: their tendency to persist along unproductive trajectories and their inability to systematically reuse lessons from previous failures. The two-timescale design provides a coherent framework for studying these issues, rather than treating reflection, backtracking, and memory as isolated heuristics.
-
Comprehensive empirical evaluation with systematic component analysis. The experiments compare the full framework with task-internal fast adaptation and slow-learned skills separately, examine different recovery budgets, and report both task success and computational costs. The reported results show that Full ReTrace improves WebWalkerQA pass@4 to 66.98%, compared with 47.17% for WebWalker and 65.09% for Fast alone. The additional transfer evaluation and failure analysis strengthen the empirical characterization of the method.
-
Useful analysis beyond aggregate success rates. The paper examines recovery behavior, the efficiency trade-off between additional reasoning and fewer environment actions, and the types of failures that remain after applying ReTrace. This analysis helps clarify when recovery is useful and why some tasks remain unsolved, making the work more informative than a method paper supported only by an overall benchmark score.
Critical weaknesses
Weaknesses
-
The contribution of slow cross-task learning is less convincing than that of fast recovery. On WebWalkerQA, the full method improves pass@4 by only 1.89 percentage points over Fast alone (66.98% versus 65.09%). Although the gain on V-GEMS is larger, the current results suggest that much of the improvement on the main benchmark may come from task-local recovery rather than the proposed cross-task meta-learning mechanism. A stronger analysis of when the slow-learned scaffold provides benefits beyond retaining or retrieving successful recovery episodes would make the central methodological contribution more convincing.
-
The evaluation provides limited evidence of generalization across agent configurations and task distributions. The experiments primarily use one model family and two web-traversal QA benchmarks. Moreover, the WebWalkerQA evaluation retains tasks that pass an annotated-path reachability audit. While this audit helps separate interface accessibility from reasoning ability, the resulting task distribution may not fully represent the broader range of real-world web navigation problems. Evaluation with additional backbones, more diverse task types, and less structured environments would better establish the generality of the approach.
-
The practical availability and cost of the slow-learning feedback signal require further clarification. The slow-learning process uses task-level ground-truth feedback to determine which behavioral changes should be consolidated. Such feedback is convenient in controlled benchmarks but may not be readily available in real deployments. The paper would be stronger with a clearer account of the supervision and evaluation costs required for scaffold updates, and with experiments establishing whether reliable cross-task improvement persists when task-level ground truth is replaced by weaker or delayed feedback.
Questions for authors
-
Clarification The paper compares ReTrace with a structured Experience Bank under the same executor, fast-recovery mechanism, model, budget, and evaluator. Could the authors clarify whether the two methods are also matched in terms of update frequency, context/token budget, and the amount of historical information available at each decision? This clarification is important for determining whether the reported gains come from the proposed slow-learning and consolidation mechanism rather than differences in information access or computational resources.
-
Assessment-critical evidence The WebWalkerQA evaluation retains tasks that pass an annotated-path reachability audit. How many tasks are excluded by this filtering, and how sensitive are the reported improvements to the audit criteria? In particular, does the relative advantage of ReTrace remain similar when evaluated on the full original task set, with interface-inaccessible tasks reported separately? This would help assess whether the measured gains reflect general recovery capability or are partly influenced by the selected task distribution.
4 comments
Questions
Q1. Experience Bank: update frequency, context budget, and historical information availability. Experience Bank and ReTrace use the same Executor, Fast Recovery mechanism, backbone, interaction budget, and Evaluator. Both use completed-task experience without future outcomes; they differ in how they organize it.
Experience Bank stores evaluator-validated historical episodes plus task-level and subtask-level Procedural Skills, retrieving relevant entries for each new task. It attempts cross-task memory consolidation after each evaluated task. ReTrace instead runs Evidence Analysis, Synthesis, and Review after each batch of completed WebWalkerQA tasks, extracting recurring decision principles into a versioned Scaffold. Only an approved Scaffold is published for later tasks. The Executor sees this version, not the analysis ledger or historical trajectories.
| Aspect | Experience Bank | ReTrace Slow |
|---|---|---|
| WebWalkerQA update | Attempts memory consolidation after each scored task | One evolution process after each batch of completed tasks; publish after approval |
| V-GEMS | Database frozen; retrieval only, no writes | Scaffold frozen; no evolution |
| Knowledge injected | Top-5 experience skills and related lessons, entry-limited | Only published Scaffold |
| Context limit | Per-entry limits | Fixed 1,115-token publication target; 1,239-token reference metadata |
| Actual Scaffold length | — | WebWalkerQA nonempty versions ~322–445 tokens; frozen V-GEMS 584 tokens |
| Executor limits | ≤30 steps; max_tokens=4096 per generation | ≤30 steps; max_tokens=4096 per generation |
The 1,115-token publication ceiling is fixed across WebWalkerQA evolution; 1,239 tokens is reference metadata. The Bank retrieves up to five skills and related lessons under entry-level limits; Slow publishes a compact consolidated policy.
Slow compares unsuccessful search and corrected behavior through Evidence Analysis, Synthesis, and Review. It translates reusable patterns into conditional strategies with observable triggers, specific actions, and stopping conditions, excluding task-specific URLs, answers, and examples. The Bank retains and retrieves relevant historical episodes and procedural skills. Results reflect this distinction: Full ReTrace's WebWalkerQA pass@4 of 66.98% versus Experience Bank's 61.32%, and frozen V-GEMS pass@1 of 55.00% versus 37.86%. Both use the shared runtime and freeze persistent experience during V-GEMS transfer.
Q2. WebWalkerQA reachability filtering and its impact. The reachability audit separates failures caused by agent search decisions from non-executable paths under the current WebWalker interface. Live sites change: pages move, links expire, access becomes restricted, button mappings change, and fetching may fail. Thus an annotated path may no longer be reproducible.
The criterion is whether an annotated path can be executed through the current interface, not whether an agent answers its question correctly. The audit does not run an answering agent, use model accuracy to select questions, or consult Baseline/ReTrace outcomes when deciding which tasks to retain. Every method is evaluated on the same retained task set. Retention is determined before model evaluation. The audit replays the annotated reference route, not an agent-generated trajectory. A failed path replay records an interface or access obstacle; it does not constitute an incorrect predicted answer. This separates environmental executability from the search policy being measured, while keeping the evaluated task subset identical across methods.
The audit checks starting-page availability, replays annotated navigation actions, and records missing links/buttons and fetch failures. WebWalker fetches pages, converts content to text, and exposes navigable links as buttons. Changing sites or interface-level mapping and access restrictions can therefore prevent execution of particular annotated steps. It checks execution, not branch quality or alternative solving routes.
We additionally conducted experiments beyond the original filtered WebWalkerQA task distribution. First, with the WebWalkerQA-learned Scaffold frozen on V-GEMS, we evaluated pass@1–pass@4: Full ReTrace achieves 55.00/56.43/59.29/60.71%, versus Fast's 47.14/49.28/52.86/57.14%. The +7.86-point pass@1 and +3.57-point pass@4 gains show transfer under distinct retry budgets. Second, replacing the backbone with DeepSeek-v4.1-flash gives 45.71% frozen V-GEMS pass@1 for Full versus 42.14% for Baseline, 42.85% for Fast, and 44.28% for Slow. Third, we reversed the transfer direction: evolving on V-GEMS and freezing the Scaffold for WebWalkerQA gives Full 49.06% pass@1 versus 44.34% (Baseline), 48.11% (Fast), and 46.23% (Slow). These complementary tests examine whether the learned prior remains useful across backbone choices, experience sources, target distributions, and retry budgets, beyond the original filtered tasks.
Thank you again for your constructive feedback. In our rebuttal, we added analyses on Slow’s independent contribution, comparisons with AWM, ReasoningBank, and Experience Bank, cross-backbone and reverse cross-dataset transfer, and further clarification of feedback, context budget, and evolution costs.
We would greatly appreciate your thoughts on whether these results adequately address your concerns and resolve the remaining uncertainty. If so, we hope the additional evidence can help strengthen your confidence in the soundness and generality of our approach.
Reviewer wHjh
We appreciate your positive assessment, particularly your recognition of our two-timescale design, the complementary roles of Fast and Slow, and our empirical analyses. Your questions have also helped us identify aspects warranting further investigation.
In response, we added experiments on Slow's independent contribution, cross-model and cross-dataset generalisation, and comparison fairness. We sincerely hope these results further clarify the distinct value of Slow beyond task-level recovery and address your remaining concerns. As the discussion period draws to a close, we would be most grateful for any further thoughts you might wish to share.
We thank the reviewer for recognizing our clear presentation, two-timescale design, and computational cost analysis.
Weaknesses
W1. Slow Learning's independent contribution and complementarity with Fast Recovery. Fast detects stagnation, rolls back, and reflects within tasks; Slow consolidates search and recovery into reusable guidance. With Fast disabled, Slow achieves 49.06% WebWalkerQA pass@1 versus Fast's 48.11%. On frozen V-GEMS, Slow reaches 49.29% versus Fast's 47.14%, demonstrating an independent search prior.
Slow also advances success and improves Fast's efficiency. On WebWalkerQA, Full ReTrace raises pass@1 from 48.11% (Fast) to 50.00%, while reducing total actions by ~8.3%, navigation actions by 9.76%, LLM calls by 8.3%, and total tokens by 14.0% relative to Fast. At pass@4, Full reaches 66.98% versus Fast's 65.09%, using ~7.9% fewer total tokens. Fast recovers; Slow guides branch selection, evidence use, and post-recovery search.
We added frozen V-GEMS pass@2–pass@4 to assess retry budgets (%, WW=WebWalkerQA; VG=V-GEMS):
On V-GEMS, Full exceeds Fast by 7.86 points at pass@1 (55.00% vs. 47.14%) and 3.57 at pass@4 (60.71% vs. 57.14%). The larger first-run gain reflects improved initial search; benefits vary with task distribution and retry budget.
Why learn a scaffold rather than retrieve successful traces? We added Agent Workflow Memory (AWM) and ReasoningBank comparisons alongside Experience Bank:
AWM and ReasoningBank retrieve past knowledge. Experience Bank shares the Executor, Fast Recovery, backbone, budget, and Evaluator, retrieving evaluated episodes and task/subtask procedural skills. ReTrace uses Evidence Analysis, Synthesis, and Review to compare failed and corrected searches and consolidate recurring lessons into persistent strategies with observable triggers, actions, and stopping conditions, excluding task-specific URLs, answers, and details. Thus, retrieval differs from consolidation. Full exceeds Experience Bank by 5.66 points on WebWalkerQA pass@4 (66.98% vs. 61.32%) and 17.14 on frozen V-GEMS pass@1 (55.00% vs. 37.86%). Fast supplies recovery traces; Slow guides later Fast adaptation.
W2. Generalization across backbones and task distributions. With DeepSeek-v4.1-flash, we evolved on WebWalkerQA and froze the scaffold for 140 V-GEMS tasks:
Full improves WebWalkerQA pass@1 by 4.72 points over Baseline and 3.77 over Fast; frozen V-GEMS pass@1 reaches 45.71% versus 42.14%, 42.85%, and 44.28% for Baseline, Fast, and Slow.
We also reversed transfer, evolving on V-GEMS and freezing the scaffold for WebWalkerQA:
Full reaches 58.57% V-GEMS pass@4 (Baseline 53.57%; Fast 57.14%) and 49.06% frozen WebWalkerQA pass@1 (Baseline 44.34%; Fast 48.11%; Slow 46.23%). These tests show transfer across backbones and both dataset directions.
W3. Feedback granularity, availability, and update cost. Slow uses delayed task-level outcomes, not step-level labels. The Evaluator provides no correct paths or branch, rollback, or scaffold-edit instructions. Fast judges stagnation and selects rollback/reflection during execution; Slow analyzes completed trajectories and outcomes afterward. Both benchmarks allow factual answers to be checked against references without expert step annotations. Ground truth never guides live navigation. WebWalkerQA updates affect only later tasks; V-GEMS transfer freezes the Scaffold. Costs include task evaluation and Evidence Analysis, Synthesis, and Review calls per window. A fixed Scaffold budget bounds injected context as history grows; costs appear in the paper.