- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper asks what a chain-of-thought intervention should be compared against. Instead of asking whether an edited trace changes the answer, the authors build a reference that says what the edit ought to change: an ACT-R-style driver model runs in a simulator, its internal state is written out as a templated trace, and re-running the same seed with one mechanism altered gives a ground-truth counterfactual. Small Qwen and Llama models (1.5B to 7B) are fine-tuned to produce these traces before predicting the crash-severity outcome.
The main finding is that the students track whatever trace they are given but do not work out what an edited state implies. Replacing the whole trace moves the answer in the right direction most of the time, but so does pasting in a trace from an unrelated run, so the replacement effect is not evidence about the mechanism. On the hands mechanism all three students fall short of a simple majority rule; on the peripheral-vision mechanism the 3B student matches it. Along the way the authors catch their own trace format leaking the run duration and rebuild it. They conclude that reading a trace and simulating from it are different things, and offer the protocol as a way to tell them apart.
Critical strengths
-
The executable reference process is a useful methodological contribution. Shared randomness supports paired mechanism interventions, while fresh-seed re-executions provide empirical predictive references. The paper carefully distinguishes these finite-sample references from performance bounds.
-
The donor controls demonstrate why a large replacement effect alone is insufficient evidence of mechanism-specific consequence prediction. OTHER-CELL DONOR reaches 0.821 / 0.830 compared with REPLACE at 0.735 / 0.784 in the two donor-tested models. Each condition is evaluated against its own outcome change, making the comparison informative about the specificity of effect magnitude.
-
The leakage and selection audits are valuable. The authors identify outcome information in termination metadata, retrain with a revised layout, and show that an apparent conditioning gain disappears under selection matching. The analysis chronology and limitations are reported candidly.
Critical weaknesses
-
The training support for the hands intervention is unclear, limiting the interpretation of its predictive shortfall. From Appendices C and E, hand position appears tied to glance type: a body lean removes steering authority, while the training campaign varies glance type and duration without listing hand position as an independent factor. If the counterfactual combination of body lean with hands on the wheel is absent from training, the hands assay also tests generalization to an unseen state combination. This does not invalidate the observed shortfall, but it leaves unclear how much it reflects that distribution shift rather than a more general limitation in predicting consequences from supplied states. The peripheral intervention may also introduce an unseen combination, so this possibility alone does not explain the asymmetry. Training-set support counts and a controlled intervention-exposure experiment would make the interpretation stronger.
-
The assay’s ability to distinguish consequence prediction from simpler strategies needs further validation. The paper appropriately states that observable prediction accuracy does not diagnose internal computation, and a positive control is not necessary to establish that these students fall below a specified benchmark. However, the title and broader framing suggest a distinction between reading and simulating that the current comparisons establish only indirectly. A predictor that explicitly propagates the supplied state under the reference dynamics, operating under the same information boundary and scoring procedure, would help validate the assay. Alternatively, the framing could more consistently emphasize predictive performance under intervention rather than simulation as a computational capability.
-
The abstract and conclusion could better reflect heterogeneity across mechanisms and reference rules. The introduction and main text do discuss peripheral, but the headline empirical contrast emphasizes hands. On peripheral, the 3B student is above B1* and is not significantly different from B2*, while the 7B student is below B2*. This is neither uniform failure nor demonstrated equivalence to the reference. Similarly, the 3B peripheral continuation has higher answer DA than full-trace input (0.875 versus 0.819), but substantially lower invariance (0.677 versus 0.938), so it should not be described as an overall improvement. A concise statement of this heterogeneity in the abstract and conclusion would better delimit the evidence.
-
The presentation makes the central comparisons difficult to track. The paper introduces several reference rules, two outcome endpoints, and both conditional and nominal denominators, with important qualifications distributed across the main text and appendices. A compact guide connecting each headline claim to its endpoint, evaluation population, reference rule, and robustness result would substantially improve readability.
Questions for authors
Q1. What is the training support for the intervention states?
Could you report the joint distribution of glance type and hand position in the training examples, including whether body lean with hands on the wheel ever occurs? Please provide the analogous support information for the peripheral intervention. If these combinations are absent, how do you distinguish intervention generalization failure from other sources of consequence-prediction error? An experiment varying hand position independently of glance type would be particularly informative.
Q2. Can you validate the assay using a known consequence predictor?
Could a predictor that explicitly propagates the supplied state under the reference dynamics be evaluated using the same information boundary and scoring procedure? Since the boundary omits some seed-dependent information, this need not imply perfect agreement. The useful check is whether a known consequence-prediction procedure behaves as expected relative to the empirical references. This would strengthen the interpretation of the assay beyond documenting the students’ benchmark shortfalls.
Q3. Can the abstract and conclusion state the mechanism-dependent results more explicitly?
The introduction already discusses the peripheral findings. Could the abstract and conclusion also acknowledge that peripheral 3B is above B1* without a statistically supported difference from B2*, whereas peripheral 7B is below B2*? For hands, could the headline statement also preserve the distinction between the all-model shortfall against B2* and the more reference-dependent comparisons against B1* and B∆?