- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training framework that enables LLM agents to proactively explore unfamiliar environments and construct reusable world knowledge without access to downstream tasks at inference time. The method employs an outcome-based reward measuring the downstream utility of generated knowledge, followed by supervised fine-tuning (SFT) on teacher-generated trajectories and iterative rejection-sampling fine-tuning (RFT). Experiments on WebWalker and WebVoyager demonstrate substantial improvements in task performance, with additional results suggesting that the generated knowledge can transfer across different backbone models.
Critical strengths
S1. Effective training framework for task-independent exploration.
The paper introduces an outcome-based reward that optimizes environment exploration according to the downstream utility of generated knowledge. Combining teacher-guided SFT with iterative RFT provides a practical approach to learning long-horizon exploration strategies. The separation between task-dependent training rewards and task-independent inference is particularly interesting.
S2. Strong empirical performance and cross-model transferability.
The method achieves substantial improvements on WebWalker, increasing Qwen3-30B-A3B accuracy from 20.71% to 45.34%. Experiments across different backbone models and knowledge-transfer settings further demonstrate its practical potential. These results suggest that the generated world knowledge captures useful information beyond model-specific representations.
Critical weaknesses
W1. Insufficient justification of the claimed self-evolution capability.
The proposed method primarily performs environment exploration and external knowledge construction, without modifying the agent's policy during inference. Moreover, it relies on manually designed website preprocessing and exploration prompts, which weakens the claim of workflow-free autonomy. The authors should better distinguish their approach from existing exploration and memory-based adaptation methods.
W2. Lack of sufficiently controlled comparisons and end-to-end efficiency analysis.
The main baseline does not receive the additional environment exploration and knowledge available to the proposed method, making it difficult to isolate the contribution of the learned exploration policy. Although prompt-only baselines are included, comparisons with conventional exploration strategies under matched computational and information budgets remain insufficient. Furthermore, the reported efficiency improvements do not fully account for the cost of generating world knowledge.
Questions for authors
Q1. What distinguishes the proposed method from conventional exploration and external-memory construction?
Since inference-time adaptation occurs through generated world knowledge rather than policy updates, could the authors clarify the operational definition of self-evolution? In particular, what capabilities distinguish the proposed method from existing knowledge-based agent adaptation approaches?
Q2. How much improvement remains under matched exploration budgets?
Could the authors compare their method against conventional exploration or summarization strategies with identical exploration steps, token budgets, and knowledge-length constraints? This would help isolate the contribution of the learned exploration policy.
Q3. How important is the graph-based website preprocessing?
Could the authors provide an ablation removing the graph-based preprocessing ? Since this component explicitly structures the exploration process, understanding its contribution is important for evaluating the claimed autonomy.
Q4. Does the method improve end-to-end computational efficiency?
The reported reduction in downstream execution steps does not account for the additional cost of generating world knowledge. Could the authors report total token consumption, browser interactions, and wall-clock time, including the exploration phase?