- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work studies the generalization of supervised fine-tuning for LLM tool-use agents.
The central hypothesis is that whether SFT transfers depends on two factors: the workflow length seen in training, and how the current state is represented to the agent. Under this hypothesis, a compact view that shows only what the next action needs helps execution, but training on it transfers asymmetrically back to full observations. Step-level accuracy and end-to-end completion can also disagree.
The claim is supported along several experimental directions:
- a controlled, procedurally generated table-workflow environment across model sizes and workflow lengths;
- horizon-transfer tests;
- test-time view switching and train-view × test-view crossovers;
- interface controls;
- an external check on ScienceWorld.
Notable aspects of the experiments:
- The paper reports step accuracy and task completion separately and shows that they dissociate in both directions.
- It identifies an asymmetric view-transfer effect.
- It is explicit about which statistics are exploratory.
Critical strengths
- A careful demonstration that per-step accuracy and end-to-end completion measure different things, with metric definitions that keep them apart.
- Common agent evaluations report either per-step agreement or task success. The paper instead separates oracle-prefix accuracy on restored gold states, clean oracle-path rate, and free-run success, and reports all three side by side (§3.3; App. A.6).
- This design reveals a dissociation in both directions that a single metric would hide:
- Untuned 27B under FULL at H16 has no clean oracle paths and a low legal-action rate, yet still completes most tasks within the call budget (§4.5, Table 4; App. D.3).
- On branching tasks, COMPACT raises canonical accuracy substantially while success barely moves, and an audit shows the remaining canonical mismatches are legal, progress-increasing actions. This checks that the metric means what it says (App. D.1, Tables 16–17).
- On ScienceWorld, mixed-horizon training has higher local agreement but fewer completions than short-only training on the same workflows, and high-agreement workflows do not succeed more often under mixed-horizon training (§4.6, Table 5; App. F, Table 21).
- This is a practically useful caution for anyone using step-level accuracy as a training signal or model-selection criterion (§4.5; §5).
- Transparent statistical reporting and honest scoping.
- Compared with the common practice of reporting single aggregate rates, the paper uses task-level pairing, paired task-bootstrap intervals, and exact sign-flip / binomial tests throughout, and labels post hoc contrasts as exploratory and unadjusted. The reported tests are consistent with the implied discordant counts, and the pilot contrasts match the pilot table (App. A.6; App. B, Table 10; §4.4, Table 3; App. E, Table 18).
- The paper repeatedly states what each analysis does not show: no training-seed uncertainty, no isolation of model size, no matched-failure recovery, and no pairing of untuned and tuned 27B. It also reports the failed repository pilots rather than omitting them. This candor makes the paper easy to audit, and I credit it (§3.3; §4.5; §5; App. A.6; App. D.1; App. G).
Critical weaknesses
- The main experimental manipulation does not cleanly match the paper's central claim about state representation.
The paper frames its contribution as a study of how state representation shapes long-horizon learning and transfer. However, the main FULL/COMPACT intervention changes several factors at once:
- In COMPACT, artifact contents are retained only for the inputs of the reference next action.
- During SFT this reference is the gold action, and during evaluation it is the evaluator-selected action (Eq. 1, l.161–168).
- The paper itself notes that the intervention both reduces displayed content and supplies relevance information (l.170–172).
As a result, the main crossover does not isolate a general property of state representation. FULL and COMPACT differ jointly in four respects: information content, relevance cues, formatting, and sequence length. The observed transfer asymmetry may therefore reflect specialization to a target-conditioned observation distribution, rather than a broader failure to transfer across state representations.
This matters because of how the strongest headline result is read. COMPACT-trained models perform well under COMPACT but poorly under FULL. At present, this is easier to interpret as dependence on the training-time relevance cue than as evidence about state representation in general. The paper is commendably explicit about this causal limitation: it notes that relevance-matched and matched-length controls remain untested (l.440–443).
The paper should therefore either:
- narrow its central claim and terminology to the specific observation-view intervention studied here; or
- give a clearer argument for why this intervention is representative of the broader notion of "state representation".
- The experiments establish a specific transfer phenomenon, but the evidence is not yet sufficient for the broader framing.
The paper does show a real and interesting asymmetry for the evaluated adapters: under compact training, compact-view accuracy improves, while full-view accuracy can decline substantially (Fig. 1). However, the strongest crossover result is based on one training seed per view (l.197–198). The paper explicitly notes that the reported intervals describe task variation conditional on those trained adapters, not training-run uncertainty (l.228–229; Fig. 1 caption).
The external ScienceWorld evaluation also shows a different pattern. Compact-trained models improve under both views, rather than reproducing the strong synthetic-workflow asymmetry (§4.6, Table 20). The paper appropriately presents this as a boundary on generality rather than as confirmation (l.079–081).
Accordingly, I find the following empirical claim supported: this particular target-conditioned compact view can induce asymmetric transfer. The broader claim, that state representation generally shapes long-horizon learning and transfer, is not yet established at the same level. The current experiments support a narrower proposition than the title and framing suggest.
Questions for authors
Main questions:
Q1. What exactly is the paper claiming by "state representation", and how does the FULL/COMPACT experiment identify that claim? The main intervention uses the reference next action to decide which artifact contents remain visible. COMPACT therefore supplies relevance information, in addition to changing the observation format and length (Eq. 1, l.161–172). Could you clarify:
- what formal or operational definition of "state representation" you intend;
- which part of the FULL/COMPACT difference is central to the claim;
- whether the intended conclusion is about state representation broadly, or more specifically about transfer across target-conditioned observation views? If the latter is the intended claim, I would encourage narrowing the title and main framing accordingly.
Q2. How much of the observed transfer asymmetry should be interpreted as specialization to the relevance cue rather than to the representation itself? The compact view exposes the contents of the reference action's inputs and hides the rest. The paper already acknowledges that this couples relevance information, formatting, and length (l.440–443). I do not think a large battery of new experiments is necessary for the current version, but I would like the authors to clarify what conclusion the existing evidence justifies. In particular, do you view the current result as evidence that:
- supervised policies can specialize to a particular observation distribution; or
- state representations, in a more general sense, fundamentally determine transfer? I currently find the first interpretation well supported and the second too broad.
Q3. How robust is the main crossover conclusion to training-run variation? The main H16 crossover uses one training seed per view, and the paper notes that task-level intervals do not capture training-seed uncertainty (l.197–198, l.228–229). The follow-up pilot shows noticeably different absolute full-view accuracy for compact-trained models (Table 11 vs. Table 3). How much confidence do you place in the magnitude of the asymmetry, as opposed to only its direction? I would be comfortable with a more limited claim if the authors explicitly distinguish "observed for these trained adapters" from a general property of the training procedure.
Minor questions:
- The ScienceWorld experiment uses a different observation construction (l.214–216) and shows compact-trained accuracy improving under both views (§4.6). Which part of the main claim is ScienceWorld intended to validate: representation dependence, horizon transfer, or only the separation between local agreement and task completion?
- The distinction between oracle-prefix accuracy and autonomous completion is one of the paper's strongest results. Would the authors consider foregrounding this contribution more clearly? It appears better supported across settings than the broader representation-transfer claim.
Response to Reviewer RBd1: Clarifications and Additional Experiments
Thank you for this distinction. We agree: supervised policies can specialize to an observation distribution. Our title is "Observation Views Shape Long-Horizon Agent Learning and Transfer."
W1 / Q1. What we claim. A view is the rendered state and supplied recipes. Full/Compact holds tasks and transitions fixed but changes relevance cues, content, format, and length together. We claim transfer effects across these combined views, without attributing the gap to content alone or claiming that representations generally determine transfer.
Q2. Relevance cues and the remaining gap. We cannot assign a fraction of the gap to relevance information. Listing the same reference input names in both views gives the original H16 adapter 1/20 completions under Full plus cue and 20/20 under Compact plus cue. For the H4-trained adapter, the cue raises Full from 0/20 to 7/20; cued Compact gives 19/20. Implicit visibility cues, content, and length still differ.
We now manipulate training correlations in a separate choice task where both recipes use the same public input and the mask never reads the correct recipe. Six new 1.7B adapters form three initialization-matched pairs. An irrelevant background cue is correlated with the target or independently balanced; relevant inputs and targets are fixed, with prompt lengths and active-input positions matched using whitespace padding. Correlated training gives 20/20 completions under an agreeing cue and 0/20 under a conflicting cue. Independent training gives 20/20 under both in every seed. Both conditions remain at 0/20 under Full. This identifies a removable training dependence in this task; it does not identify an internal neural rule or explain the entire original crossover. The length-matched Neutral control remains mixed.
W2 / Q3. Training-run variation. The main H16 endpoint now has three 1.7B training-seed pairs on 50 common held-out tasks. Each Compact-trained adapter completes 50/50 under Compact and 0/50 under Full. Full-trained outcomes vary: 35-49/50 under Full and 24-50/50 under Compact. This repeats the endpoint gap for these adapters, without estimating its expected magnitude for a new training run.
The learning curves do not show a uniform decline: from step 20 to 100, Full-view local agreement during Compact training changes by -29.4, -3.1, and +11.9 percentage points. We distinguish the original task set, replication curves, and pilot protocols. Task-level intervals condition on fixed adapters and do not measure training-run uncertainty.
Historical local training set Trainer seeds after LoRA creation; matched initial hashes were not recorded. The new cue-correlation and retrieval studies seed before LoRA creation and verify matching initial weights. Each uses three paired 1.7B seeds; historical pairs remain labeled separately.
Minor question 1. What ScienceWorld tests. It tests view sensitivity and local versus autonomous performance. The earlier 27B study uses different views and does not replicate the table crossover. In the history-window study, Full retains earlier exchanges and Compact2 keeps the last two, without reading the reference action. Three 1.7B pairs and one 8B pair favor matched views on DEV local decisions. On 20 frozen TEST tasks, the 8B Full-trained adapter completes 8/20 under Full and 0/20 under Compact2; the Compact2-trained adapter completes 1/20 under either view. This does not validate the table mask's mechanism or external H16 transfer. ALFWorld's four 8B pairs retain mixed effects; AppWorld remains a nonreplication.
Minor question 2. Foregrounding the diagnostics. Local agreement, call validity, and completion are central in the Introduction and Results. New controls make the distinction concrete.
Using the same three Compact-trained adapters on ten new H16 chains, we supply 12 correct steps before continuation. With four steps left, Full completion is 0/10, 0/10, and 4/10, versus 10/10 each under history-based Recent-1. Previous rollout mistakes are therefore unnecessary for the gap. Views share a state and budget at each prefix; across prefixes, both state and remaining work change. Some Full-trained adapters work well under both views.
On 18 fresh ALFWorld games, two existing Compact2-trained adapters issue identical first actions under both history views, yet complete 17/18 and 15/18 continuations under Compact2 versus 11/18 and 9/18 under Full. In a separate 24-game retrieval study, three retrieval-trained adapters gain 3, 1, and 2 expert agreements out of 144 under retrieval versus Recent-2, while completion changes by -1, +1, and 0 games. Both rules use observed history only. These tasks have short expert paths and supplied menus. Expert disagreement need not imply an invalid action; local gains need not improve completion.