- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MQ-ACM, a low-rank adaptation method in which each adapted attention block keeps a recurrent associative state (scalar-gated delta rule) and reads it through query-specific decoder blocks, widening the readout to while state storage stays fixed at the state. §3 develops an "allocation–expression" vocabulary (effective correction rank) and Proposition 1 shows tied decoders collapse the attainable span to rank ≤ r. Experiments fine-tune Llama-3.1-8B on math/code/instruction data and Llama-3-8B on commonsense: at r=8, K=4 improves rank-8 LoRA by 0.50 MT-Bench, 5.96 GSM8K, 6.30 HumanEval; inference is 3.04× slower than LoRA.
Critical strengths
- Unusually reproducible specification — executable pseudocode, pinned normalization/state-update/mixed-precision decisions, closed-form storage and FLOP accounting, and honest reporting of the unfavourable 3.04× latency.
- The paper reports its inconvenient results. The best K is 1, 8, 4 per benchmark; retraining ablations favour Neither/No-JI on two of three benchmarks; a depressive baseline run is retained and noted. The candour is worth crediting even where it damages the narrative.
- Real diagnostic effort with the right controls — the fixed-R factorization sweep (E.2) holds R = 32 while varying (K, r), and intervention protocols are specified down to token masks and verification that λ = 1 reproduces unmodified logits.
Critical weaknesses
- The mechanism the paper is named after is not the source of the reported gains. The best MT-Bench score is at K=1 (6.75 vs 6.58 at the advertised default K=4) — a configuration with no multi-query at all; on GSM8K the K=4-over-K=1 margin (0.50) is far inside seed std. Only HumanEval shows a multi-read effect, and it is non-monotonic. Two of three headline numbers are carried by the recurrent memory + direct path + joint initialization, not by the readout organization the title claims.
- The headline GSM8K/HumanEval gains rest on baselines the authors did not run — LoRA and LoRA-GA numbers are transposed from other papers (§5.1, C.3), exactly the two columns carrying the +5.96/+6.30 claims, with no significance testing anywhere.
- The closest structural precursor (δ-mem) is cited as such but never compared against, so the paper cannot partition credit between multiple reads, the recurrent-correction route, and joint initialization. The one baseline that would settle it is missing.
- The paper's own theory makes a sharp falsifiable prediction and the paper declines to test it. Proposition 1 predicts tied decoders forfeit the benefit; the clean tied-decoder retrain at matched parameters is deferred in E.5 — a substantive gap, not a scope note, since decoder sharing is the thesis.
Questions for authors
- The mechanism in the title is not the source of the gains. Please report a decomposition table at matched parameter count and identical data/recipe: (a) rank-8 LoRA → (b) the δ-mem-style recurrent correction alone (single read, no direct path, no joint init) → (c) + direct path → (d) + joint init → (e) + multiple reads (K=4). This is the experiment that determines whether the title is earned — §5.2's K-sweep cannot answer it, because K=1 removes the multi-read but retains everything else. Please also explain mechanistically why multiple reads help code synthesis but not math or conversation, and whether the non-monotonic HumanEval curve is a real interface effect or seed noise.
- The headline gains rest on baselines you did not run. Please re-run LoRA, LoRA-GA (and GoRA) in your own environment for GSM8K and HumanEval, with paired significance testing (paired bootstrap over seeds, or more than three seeds) for the +5.96/+6.30 claims. These two columns carry the central quantitative claim and are exactly the ones transposed from other papers; if the transposed numbers cannot be reproduced, please state the gap explicitly.
- The closest structural precursor is never compared. δ-mem is cited in your own words as "the closest structural precursor," yet appears in none of Tables 1, 2, 3. Please add the δ-mem baseline; without it, none of the gains can be attributed to multiple reads versus the δ-mem route versus joint initialization.
- Your own theory makes a falsifiable prediction you declined to test. Proposition 1 predicts that tying collapses the attainable span to with dimension ≤ r. Please run the tied-decoder retrain at K=4 with matched parameters and report it — a null or small result there would be highly informative. Deferring it in E.5 is a substantive gap, not a scope note, since decoder sharing is the paper's thesis.
Response to author
I thank the authors for the detailed and candid response, and for the substantial new ablations and significance analyses. While I acknowledge that MQ-ACM outperforms static LoRA at comparable parameter budgets and most published low-rank variants under the paper's own reported numbers, the requested in-house reruns of LoRA, LoRA-GA, and GoRA with paired statistics were not delivered, so the headline math and code advantages over external baselines — which rest on cross-recipe comparisons — remain unestablished (W2/Q2). The decomposition ladder still lacks the rank-8 LoRA reference row and the single-read MT-Bench entries, and the authors' own tied-decoder retrain indicates that the titled mechanism pays only on code, so I think the current evidence remains insufficient to support the abstract's broad claim and I maintain my rating。