- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether an explicitly supplied cumulative spatial coordinate remains useful when that coordinate is already computable from the model’s inputs. The setting is conditional nonspace-character prediction in proportional-font Shift_JIS text art. Paired Transformers receive the complete previous row, true local glyph advances, target history, target query, and unrestricted query access to source positions; they differ only in whether an additional scalar is a matched character index or the true cumulative rendered position. A source-side canonicalizer maps equivalent U+0020/U+3000 whitespace runs to a unique string, making both models exactly invariant to this specified pixel-preserving rewrite family. Under this controlled setup, cumulative position improves NLL by 0.0365 bits/target in the original normalized stage and by 0.0410 bits/target in three additional fixed-endpoint paired initializations. Additional controls compare against width-only, calibrated row-scale, pseudo-cumulative, dropout, larger normalized, and raw/augmented causal variants, with a separate frozen evaluation on 19 screened event artworks.
Critical strengths
- The central comparison is well controlled. W-index and W-scan share the architecture, parameter count, true local widths, target inputs, full source access, initialization, and data order within each seed pair. This makes the scalar-feature intervention cleaner than a standard feature-addition comparison.
- The whitespace canonicalization provides a precise intervention. The source-side U+0020/U+3000 canonicalization removes a well-defined encoding redundancy while preserving rendered pixels for the specified rewrite family under the fixed MS PGothic 16px metric.
- The authors report several useful controls. Width-only, Linear, Pseudo, dropout, larger-model, and frozen-event experiments help rule out some simple explanations and show that the effect is not confined to a single training configuration.
- The paper is transparent about limitations and reproducibility. The appendices document protocol history, reused data, endpoint selection, per-seed results, intervals that cross zero, and detailed replay information.
Critical weaknesses
- The significance and generality of the problem are limited. The experiments concern teacher-forced next-character prediction in Shift_JIS text art under one fixed MS PGothic rendering rule, using relatively small custom Transformers. The paper does not show relevance to modern language models, layout-aware systems, other fonts, free generation, or downstream applications. As a result, the work feels more like a carefully controlled niche case study than a broadly significant contribution.
- The comparison does not isolate the mechanism behind the gain. The Scan feature provides coordinates in the same metric as the target query, so the improvement may reflect easier source–target coordinate alignment rather than the value of prefix accumulation itself. The paper lacks learned cumulative-position, CoPE-like, or query-relative coordinate baselines that would distinguish these explanations.
- The empirical evidence remains statistically and materially narrow. Most experiments reuse the same ordinary additions and blocks, with only three paired fits per stage. Several descriptive fit intervals cross zero. The event evaluation adds useful evidence, but contains only 19 works from one event and does not establish independent-author generalization.
- The raw/augmented larger-model comparison is not fully matched. For the causal d128 setting, Index and Scan select different optimization recipes and endpoints. The resulting gap may therefore mix the representation effect with tuning differences.
- The practical effect is small. Across the ordinary-material prediction comparisons, the reported gains are approximately 0.0207–0.0410 bits/target, with accuracy gains of 0.200–0.412 percentage points. The frozen event evaluation shows a larger descriptive accuracy gain of 0.886 percentage points, but it uses a separate 19-work population. Without evidence for improved free generation, layout quality, or another downstream metric, the practical importance of the effect remains unclear.
Questions for authors
- How can the authors distinguish the benefit of explicit prefix accumulation from the benefit of providing source coordinates in the same metric as the target query?
- Why is there no learned cumulative-position, CoPE-like, or query-relative coordinate baseline? Is the claimed invariance strictly source-side? What happens when equivalent whitespace rewrites occur in the target history or current-row prefix?
- How should reviewers interpret the positive material bootstrap intervals together with several descriptive three-fit intervals that cross zero? Do the NLL improvements translate into better free generation, layout preservation, or another practically meaningful outcome?