- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Motivated by the concepts of distance decay and gravitational fields from linguistics, this paper provides a reinterpretation of power-law relative positional bias. It introduces AGF-S—which incorporates self-position weights, directional parameters, and shell boundaries—along with two multi-shell extensions, AGF-MS. The authors report modest BLEU score improvements across two machine translation tasks and interpret model behavior through the learned positional curves. Since the base AGF model is equivalent to log-KERPLE, the primary methodological contributions lie in the shell-based parameterization and the associated mixing mechanism.
Critical strengths
- The relationship between the proposed method and existing families of functions is clear. Section 3.3 explicitly defines the parameter mapping between AGF and log-KERPLE, making it easy to distinguish between reinterpretation and novel design.
- Positional bias is amenable to analysis. Components such as position, orientation, shell boundaries, and mixing weights are all explicitly parameterized; Figure 3 illustrates the curves for different attention heads, rendering the design empirically testable.
Critical weaknesses
- **Power law explanations mainly come from analogies and have not yet formed a verifiable theory. **PASL of §3 lacks explicit sampling, conditional events, and estimation procedures. It does not follow directly from Shannon's entropy, Zipf's law, or the reliability growth analogy that this distance distribution must be a power law. Although Figure 2 is marked "Actual Data", it does not explain the source of the data and the meaning of the coordinates, nor does it compare the fitting quality of other distributions.
- **The learned position prior is not sufficient to verify the syntactic interpretation. ** The objects in Figure 3 are position functions, not actual attention or syntactic dependency metrics after normalization. The concentration of shell radii at 1–2 does not prove that functional attention primarily captures syntactic relations at this distance range, nor does it rule out further dependencies. Ablation differences fixed at self-position weights of 0 or 1 do not exceed 0.1 BLEU, again insufficient to confirm the so-called zero-distance paradox (§§5.2–5.5).
- **The small experience gain lacks statistical support. ** The gain is only +0.2/+0.1 BLEU over sinusoidal encoding and +0.3/+0.2 over learnable ALiBi. Neither Table 1 nor text ablation has seed numbers, intervals, or significance tests, so stability is unclear. The +0.3–0.5 highlighted in the introduction does not cover all baseline comparisons.
- **Insufficient implementation details. ** Clarification is needed between the integer definition of radius and continuous learning; the handling of parameter initialization, constraints, bidirectional multi-shell implementation, and zero-weight logarithmization are not fully explained. The number of training steps, tokenization, decoding configuration and the complete sacreBLEU signature are also missing. §5.1 points to the experimental setup and §7 is actually the conclusion.
Questions for authors
See Weakness.
Hello, the new version of the paper seems very substantially different. It would be helpful if you could produce a point-by-point list of changes so that we can assess (1) whether and what concerns were addressed, what claims were withdrawn, etc and (2) whether this is a completely new paper