- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes SpectraGain, a PEFT operator that learns a compact spectral gain and applies it to the Fourier representation of a frozen weight matrix. The resulting carrier-conditioned circular correction can have a substantially higher realized rank than the compact trainable generator.
The paper evaluates the method across language and vision tasks and compares it with low-rank, high-rank, Fourier-based, and circular adaptation methods. It also includes same-generator circular-versus-pointwise comparisons, tangent-space analyses, correction-rank interventions, and runtime and memory measurements.
Critical strengths
-
The paper provides a technically informative analysis of different adaptation geometries, especially the distinction between generator rank and realized correction rank.
-
The controlled same-generator experiments and systems analyses are careful and strengthen the scientific value of the work.
Critical weaknesses
-
The conceptual novelty is somewhat compositional. Closely related prior work already studies spectral weight adaptation, carrier-conditioned pointwise modulation, and additive circular adaptation separately. SpectraGain’s main novelty is the specific combination of a low-rank spectral gain with frozen-carrier conditioning and circular coupling, rather than introducing each of these ingredients independently.
-
The controlled experiments support a task-dependent effect of circular versus pointwise coupling rather than a consistent advantage of SpectraGain. Circular coupling is favorable on T5 but unfavorable on CLIP in the fixed-checkpoint comparison, while the independently retrained differences are small and statistically inconclusive.
-
The practical efficiency benefit is strongly sequence-length dependent. At 1K tokens, SpectraGain takes 2.064 seconds per step compared with 0.241 seconds for fused LoRA, while the relative overhead becomes much smaller at long sequence lengths. The efficiency case is therefore most compelling for long-context training rather than for PEFT workloads in general.
Questions for authors
Given that the circular-versus-pointwise effect changes sign between T5 and CLIP, what is the intended general conclusion: that circular coupling is expected to be preferable under identifiable task conditions, or that the main contribution is to characterize task-dependent coupling geometry rather than to establish a generally superior coupling operator?
Thanks to the authors for their response and additional experiments. These are helpful, though my concerns about the limited methodological novelty and the small gains over HiRA on T5 and CLIP remain. I think further refinements to the method and improvements to the experiments could make the paper more convincing. I will therefore keep my original score.