- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies adaptation in resource-constrained language agents from an information-allocation perspective, treating prompts and model parameters as two knowledge-storage mechanisms with different capacities, update costs, and persistence. The authors propose an oracle–student–controller architecture. The student first learns the required output interface through offline distillation. During deployment, the controller preserves protocol information and compresses the semantic state into a prompt budget suitable for the student. Observable signals from the student and environment trigger oracle corrections, while a forgetting-risk gate determines whether accumulated corrections should be consolidated into the model parameters. Immediate corrections can affect the current task, whereas parameter updates take place between tasks.
I appreciate the paper's perspective. Its main value lies in bringing context management and parameter adaptation, which are often studied separately, into a unified problem formulation and translating this view into an executable system. The reported empirical improvements also support the usefulness of this perspective.
Critical strengths
-
The paper frames adaptation as a decision about whether task information should remain in the prompt or be consolidated into model parameters under constraints on capacity, latency, and write costs. This perspective reveals the connection between context management and parameter learning and offers a clear way to reason about transient state and persistent behavioral corrections. I consider this reframing conceptually valuable, with the potential to inform future research on agent memory and continual adaptation.
-
The authors turn this perspective into a complete workflow involving protocol preservation, semantic-state selection, selective oracle correction, and parameter updates. The system distinguishes immediate assistance within the current task from parameter learning that can benefit subsequent tasks. In particular, the trigger relies only on student and environment signals, allowing it to decide whether to request assistance without first obtaining an oracle response. This makes selective oracle use operationally feasible.
-
The paper uses prompt length and attention behavior to characterize the student's reliable operating range and uses this characterization to motivate a budget-constrained projection procedure. Although these proxy analyses have explicit assumptions and a limited scope, they provide a testable basis for setting context budgets. The authors also connect theory and experiments through attention measurements, prompt-length sweeps, and calibration across models, improving the framework's interpretability.
-
For Llama-3.1-8B, success increases from 11.6% with distillation alone to 27.3% on WebArena, and from 56.7% to 79.9% on the unseen ALFWorld split. Under a matched oracle budget, the method also outperforms PivotRL on WebArena, achieving 27.3% versus 23.6%. In addition, the authors separately examine projection, immediate corrections, parameter updates, different oracles, and constrained decoding. These experiments help explain the roles of individual mechanisms beyond demonstrating the full system's improvement over an unadapted model. (Tables 2, 15, 17, 21, and 38–41.)
Critical weaknesses
-
In Definition 4, If denotes the number of applied parameter updates, as described in Algorithm 1, then using the reported values , , , and yields a rejection probability of approximately at . This magnitude appears difficult to reconcile with the update suppression reported in Table 32. Please clarify the actual unit of the counter , whether additional scaling or normalization is applied, and provide a corresponding trace of gate values and update decisions. This is a request to clarify the consistency between the mathematical description and implementation, rather than a conclusion that the experimental results are incorrect. Nevertheless, the explanation would affect my assessment of the risk-gating mechanism.
-
The authors explicitly state that the attention theorem provides a prompt-side proxy constraint and does not explain every form of long-context failure. The calibration across models in Table 9 also provides useful initial evidence. I would like to better understand how much calibration cost the theoretical proxy saves relative to directly sweeping prompt length, or how much it improves budget selection for new models and input types. This would help establish how the analysis guides controller design in practice.
-
Table 27 indicates that protocol protection is an important source of improvement. I suggest comparing the proposed projection strategy with simple truncation, retrieval, or generic compression methods that preserve the same protocol information and operate under matched token and oracle budgets on WebArena and ALFWorld. The interval-summary comparison already provided for MFBO is helpful. Extending similar comparisons to interactive tasks would better distinguish the benefits of the unified framework from those of the specific information-selection algorithm.
-
In Table 21, the frozen-adapter control already approaches the performance of the full online system, while Table 37 shows that the full system has higher per-step latency than the distillation-only baseline. These results suggest that the value of continued updates may depend on deployment duration and the extent of environmental change. Learning curves across different task orders and comparisons under matched total compute budgets would help identify when continual adaptation is preferable to adaptation before deployment or less expensive context management. Learning across tasks is consistent with the paper's objective; the concern here is the stability and scope of its benefits.
Questions for authors
-
How are the update counter, scaling, and normalization in the gate defined in the actual implementation, and can these details explain the differences in update counts in Table 32?
-
What measurable benefits does the theoretically guided prompt budget provide over direct empirical calibration?
-
With identical protocol protection and resource budgets, how much additional benefit does the proposed projection strategy offer over simpler methods?
-
Which deployment durations, environmental changes, or task-stream settings most clearly demonstrate the value of continued parameter updates?
We thank the reviewer for a detailed and quantitative review. We answer each point below.
Q1 — Gate: formula, counter, reset, values
Implemented formula. The gate evaluates Definition 4 directly:
It is evaluated once per task, between tasks, and only when |B| ≥ B_min = 32.
Counter units and reset. T counts applied updates. It is incremented only when an update is applied. A rejected candidate leaves T unchanged and keeps its supervision in the buffer, where it is reconsidered at the next candidate. T is reset when accuracy on the fixed 200-prompt schema probe returns to its initial value. This happened twice in the 600-step runs (App. A.7.5).
The value of ε was misreported. The reviewer's arithmetic is correct: with ε = 0.10, as printed in Table 22 and Appendix B, E_T/ε never exceeds 0.05 and the gate would admit essentially every update. The deployed tolerance is ε = 6.53·10⁻⁷, and 0.10 is an error in the paper. The ε column of Table 33 was reported on the same incorrect scale. The correct values are given below.
How ε was set. E_T bounds a normalized forgetting whose normalizer, ½ν²d_Lλ_max(H₀), is not estimated (App. A.3). The tolerance therefore cannot be derived from the bound and has to be set empirically. A value of order 10⁻⁷ is the natural scale: E_T grows by about η²·(q_D/d_L) = 2·10⁻⁸ per applied update. We chose ε on a held-out task stream, separate from the evaluation tasks, as the best balance between schema retention and semantic learning. Table 33 then sweeps ε around this value on the evaluation stream and confirms the choice. The deployed value has probe 0.99, semantic probe 0.56 and the highest success, 27.3%. Smaller tolerances under-learn (semantic probe 0.46–0.52) and larger ones forget (probe 0.92–0.96).
With the correct values, Definition 4 reproduces every update count in Tables 32–34. In the tables below, "expected" is the mean ± s.d. of admitted updates out of the 17 candidates of the always-update run, computed exactly from Definition 4. Probe-triggered resets are not included.
Table 22 (deployed setting)
Table 33 (tolerance sweep, B_min = 32)
The B_min rows of Table 33 (8, 64, 128) use the deployed tolerance, 6.53·10⁻⁷. Their update counts differ because B_min changes how many candidates reach the gate.
Table 34 (misspecified q_D/d_L, ε = 6.53·10⁻⁷)
With the correct values and no per-row tuning, fourteen of the fifteen reported counts lie within 1.7 standard deviations of the value Definition 4 predicts. The remaining one (ε = 1.31·10⁻⁷: 4 against 5.3 ± 0.6) is at about two standard deviations.
Rejection probability for each candidate. The rejection probability depends on a candidate only through its counter value. The values below are therefore the exact probabilities applied to every candidate in the deployed configuration (Table 32, "Risk-aware gate"). T is the counter value the update would produce if admitted.
For any other row of Tables 33 and 34, multiply by (6.53·10⁻⁷/ε) · (q_D/d_L / 0.50). After each probe-triggered reset, the sequence restarts at T = 1.
What Table 32 establishes. App. A.3 states that E_T is a relative exposure score with an unestimated normalizer. Table 32 compares update controls at matched count (13 updates each). At that count, the gate keeps a schema-probe accuracy of 0.99, against 0.92 for the fixed schedule and 0.95 for random selection, while reaching the highest semantic-probe accuracy (0.56). The gate's benefit therefore comes from which updates it admits, not only from admitting fewer.