- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies "world internalization": converting an agent's exploration experience in an environment into weight updates without downstream tasks, rewards, or success feedback. The proposed U-WIN uses the frozen base model to generate two types of data: Type I notes that summarize what was learned during exploration, used to acquire environmental knowledge, and Type II solution trajectories generated with relevant notes in context and trained with the notes removed, used to learn how to apply that knowledge. Across legal-document, codebase, and mathematical environments, downstream and closed-book performance generally improve as internalization data scales from 1M to 100M tokens, and Type II data helps preserve the model's agentic behavior.
Critical strengths
- The setting is fully unsupervised (no downstream questions, no rewards, no trajectory filtering), and the paper clearly distinguishes it from test-time training, reinforcement learning, and synthetic continued pretraining.
- All training data is generated by the frozen base model without a stronger teacher, which rules out distillation from a stronger model as an explanation and makes the self-improvement claim more credible.
- The paper shows a scaling trend from 1M to 100M tokens and candidly acknowledges that it does not perform genuine continual learning.
Critical weaknesses
- The tasks emphasize memory reinforcement rather than generalization, and the method may induce hallucination. The closed-book and recall tests probe content the model has already seen in its notes, and the Equational Theories test items may be derivable from known implications; the gains in the Fig. 5 examples come from directly recalling specific matter IDs and file paths. In addition, removing the notes when training on Type II trajectories (In context distillation) may teach the model to "recall" facts without grounding and cause hallucination, yet the paper does not test.
- Limited scope and an overstated title. Only the Qwen3.5 family is used, forgetting of general capabilities outside the target environment is not measured, and no continual or test-time self-improvement experiments of the kind implied by the title are conducted.
- Missing comparisons. The paper compares neither against letting the base model retrieve the same notes at test time (e.g., RAG) nor against methods such as synthetic continued pretraining, so it cannot show that updating weights is better than keeping the experience in context.
- Writing and rigor issues:
- Typos and grammar: "whther" (Abstract); "their goal is distill" and "in a the environment" (Sec. 2); "faor C&H" (Sec. 5); "Equational Theorie" (Fig. 2 caption).
- Inconsistencies: the StudyBench model is "Qwen3.8-27B" in Sec. 5 but "Qwen3.5-27B" in Sec. 1 and Fig. 4; the Equational Theories training scales stop at 30M in Sec. 5, whereas Tables 2 and 4 and Figs. 2–3 include 100M; the DSPy codebase size is given as ~0.33M, ~650K, and 0.46M (Fig. 4) tokens.
- Sructure: The caption of Table 4 is incomplete; Sec. 4.2 refers to Type I as "Stage I"; Sec. 6 has a single subsection (6.1, titled for Type I data) that also contains the Type II analyses and the conclusion.
- Eleven leftover
:chatgpt-content-referenceindex="N"markers remain in Appendix A.1–A.3.
Questions for authors
See weakness.