Nonlinearity-Aware LoRA: Structured Gate Adaptation under Low-Rank Constraints

Low-rank adaptation (LoRA) constrains weight updates, yet their matrix-space fit does not by itself determine which features a nonlinear feed-forward network (FFN) transmits. We identify gate-selection mismatch: target and adapter scores can induce different transmission coefficients even when their weight-space discrepancy is small. To explain this mechanism, we introduce a multi-criteria decision-making interpretation of FFN evaluation and selection, then develop a homogeneity-based response framework for low-rank adaptation. It links relative gate response, channel aggregation, and the coupled FFN output, motivating persistent channel prioritization and regime-conditioned coordination. We instantiate these principles in NA-LoRA, using a temporally smoothed response-sensitive channel mask and gradient modulation coordinated across the gate, value, and output projections. The mask generalizes across FFN architectures, while the current macro rule specializes to SwiGLU's characteristic geometry. Language and vision-language experiments show improvements over vanilla LoRA and competitive results against strong PEFT variants; component ablations and nonlinear-state diagnostics support the proposed framework and design. Both training-only controls preserve LoRA's rank, forward parameterization, and inference cost.