- Rating
- 2(+1)
- Confidence
- 2(−1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a synthetic classification task over the AES field GF(2^8). Each input is a sequence of byte pairs, and the label is whether every pair's ratio lies in a fixed subgroup . Single-byte frequencies are the same in both classes, so the label depends only on relations between bytes. The proposed model, GARN, uses attention to propose byte pairs, computes their ratios exactly with fixed multiplication and inverse tables, and learns which ratios belong to . With adjacent pairs, GARN reaches 99.5% accuracy from 256 examples, while Transformers, MLPs, and discrete relaxations stay near 50% up to 1,024 examples. With shuffled positions, GARN reaches 79.9%, compared with 67.0% for a fixed-adjacency probe.
Critical strengths
The experiments are carefully controlled. The matched-marginal design makes single-byte frequencies the same in both classes, so the label depends only on relations between bytes. Replacing the exact tables with random ones drops GARN to chance, and template-free masks show that the gain does not rely only on supplying the target subgroup.
Critical weaknesses
Limited insight. The main finding follows naturally from the task design. The label is whether lies in , and GARN is given exact tables that compute . Once the ratio is computed, the remaining problem is to learn which of 255 values belong to . This is far easier than learning GF() multiplication from ≤1024 sequences. The default template bank also contains the target subgroup itself, so HardProbe with templates already reaches 100% at . Given this setup, it is not surprising that access to the exact operator is what drives sample efficiency. The learned pairing is the only part of GARN that goes beyond the task definition, and the paper has little evidence that it helps. On shuffled data at , pooling exact memberships over all pairs gets 96.9%, while GARN gets 79.9%.
Questions for authors
In the current task, the exact tables directly compute the quantity that defines the label. Can the authors show a task where the tables help but do not directly give the answer, for example a label that depends on a multi-step relation, or a real task
Thank you for the insightful and constructive review, and for recognizing the rigor of our benchmark controls (matched marginals, adjacent/shuffled variants, density checks, and corrupted tables). Below, we directly address your two primary concerns, including new empirical results evaluating relation-aware attention baselines.
1. Response to W1 & Q1: Relation to Relative Position Bias (Shaw et al. / T5) and New Empirical Baselines
Following Reviewer Sy8N’s suggestion, we implemented and evaluated Transformers using exact-ratio attention biases under our standardized benchmark.
Experimental Setup
To ensure a strictly controlled, apple-to-apple comparison:
Empirical Results (5 seeds, mean ± std)
(Note: In the main text, GARN with order-17 templates reaches 99.5% at n=256; the template-free variant is compared here to strictly isolate architectural interfaces without subgroup priors).
Findings & Architectural Takeaways