- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
SAA learns coefficients over fixed blockwise signed-permutation operators, supporting residual-state and weight-space adaptation without an explicit low-rank constraint. Comparisons with LoRA cover mathematical SFT, E2E generation, and mathematical RL. Weight-space SAA improves some SFT results, E2E scores are comparable, and residual SAA improves all four reported RL evaluations.
Critical strengths
Table 4 reports consistent gains in answer accuracy over both LoRA and the frozen backbone across four RL evaluations. The paper also distinguishes improvements in output formatting from gains in mathematical correctness and explicitly reports negative transfer under SFT, making its empirical findings easier to interpret.
Critical weaknesses
W1. The method is insufficiently specified to reproduce or assess its central contribution. Section 3 presents a generic linear combination of fixed operators but omits the explicit XOR/Walsh construction, block size, basis-selection rule, coefficient-sharing scheme, and mapping to rectangular weight matrices. The exact RL adapter parameter budget and key SFT training settings are also missing. Section 4.3 refers to an appendix containing the full configuration sweep, but the supplied PDF ends with the references. These omissions prevent verification of the implemented method and the claimed parameter matching. W2. The evaluation does not isolate the contribution of the proposed algebraic structure. All trained baselines are LoRA variants; the experiments include neither FourierFT, which also learns coefficients in a fixed representation, nor a random-basis control under a matched parameter budget. FourierFT is already discussed in the paper, so the missing comparison represents an experimental gap. Moreover, Section 4.3 selects configurations based on the highest observed test BLEU, meaning that the reported comparison of 66.77 versus 66.60 does not provide an unbiased estimate of relative test performance. Each comparison also uses a single training seed (Section 4.1), and potential overlap between the RL training data and evaluation datasets remains unassessed (lines 419–420). Taken together, these limitations leave the reported gains insufficient to establish a reliable advantage of the proposed operator family.
Questions for authors
- Can you provide the explicit operator construction, rectangular mapping, complete training configurations, exact parameter counts, and the referenced appendix?
- How does SAA compare with FourierFT and a random-basis control under matched budgets and placements? Since the reported experiments use one seed, do the key gains persist across training seeds?
- Can you provide E2E results with configurations selected solely on validation data, and document Math10K provenance and overlap with the RL evaluation sets?
Addressing Methodological and Experimental Concerns Fixes
We thank the reviewer for the detailed and actionable feedback. The revised manuscript directly addresses the central methodological and experimental concerns in W1 and W2.
W1: Method specification and reproducibility.
Section 3 now distinguishes the full-rank elementary signed-permutation directions from their learned linear combinations, specifies square-block updates for rectangular projections, and describes both residual-state and weight-space insertion sites.
Appendix A provides the complete SFT configurations. Appendix C adds the previously missing Qwen RL configurations, including exact parameter counts, initialization, learning rates, and group sizes. For example, residual SAA and residual LoRA each use 1,048,576 trainable parameters, while the matched 131K SAA/FourierFT configurations each use 131,072 parameters ( Table 9). The referenced appendices are now included in the PDF.
The residual mask-bank selection algorithm is not yet fully specified, which we acknowledge as a remaining reproducibility limitation.
W2: Alternative bases and the contribution of algebraic structure.
We added FourierFT and WaveFT baselines at matched placements and budgets, as well as random signed-permutation, random sparse, random dense, and single-entry controls (Appendix E.1).
At 393K on E2E, random signed permutations achieve 66.41 BLEU versus 66.35 for SAA. We therefore do not claim that the XOR/Walsh structure itself produces a demonstrated performance advantage on this benchmark.
E2E selection bias and statistical reliability.
The original exploratory test-BLEU selection has been replaced by validation-loss-based selection. All reported principal SFT and E2E results now use three seeds.
At 48K, SAA obtains 65.52 +/- 0.89 BLEU compared with FourierFT's 64.34 +/- 0.82 under greedy decoding. The paired test-input bootstrap interval for the difference is [0.63, 1.69], although this interval does not measure training-seed variability and the advantage disappears under beam search.
RL evaluation and dataset overlap.
The revised RL experiments now use three seeds, independently tuned learning rates, multiple adaptation placements, and two completion-group sizes where applicable ( Tables 4 and 9-12).
The Math10K preparation removes duplicates and screens for exact matches and word 6-gram near-duplicates against GSM8K, SVAMP, and MATH-500. Of 8,731 numeric records, 6,021 remain; we reserve 500 validation problems and sample 1,200 training examples per seed from the remaining 5,521.
The expanded RL results are more informative than the original single-seed comparison. SAA is competitive, but the strongest baseline depends on the setting, and most test-set differences remain within seed variability.
We have revised our claims to reflect these results rather than asserting a universal or algebra-specific advantage.