- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies prior-induced literal override in competitive-programming prompts, where a model reads an authoritative decimal bound such as 105 as the power of ten 10^5 that the same digits would form after exponent loss. Using seven collision-control pairs across seven contexts on four open-weight models, it shows that the attractor share rises from near zero in neutral copying to 74.5% in full problem statements. On ten scale-sensitive problems with six frontier models, the 105 surface lowers executable accuracy relative to the 100 and 145 controls, with supporting tokenization, probability, and activation-patching analyses.
Critical strengths
-
The controlled design is careful. For example, the digit string is fixed, each collision has a same-length matched control, and the difference-in-differences analysis is validated with free generation.
-
The corpus audit showing exponent-loss corruption in 25.75% of CodeContests-O descriptions is a useful data-quality finding on its own.
Critical weaknesses
-
The paper studies only one specific case. Section 3.1 gives a general definition of notation collision, but the experiments instantiate a single transformation on seven power-of-ten strings in one domain, so the evidence does not support a general class of failure.
-
The paper has no clear contribution on how to avoid the misinterpretation. The only reliably effective intervention is explicit disambiguation, which requires knowing in advance which number is ambiguous and telling the model directly, and the other interventions are inconsistent across models.
Questions for authors
-
Does the effect hold for other instances of the Section 3.1 definition, such as subscript loss, other bases (2^10 written as 210), composite forms (10^9+7 written as 109+7), or domains outside competitive programming?
-
Is there a mitigation that does not require identifying the colliding literal in advance? The collision-agnostic instructions in Table 17 change attraction in opposite directions for different models.