- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Q-NeuS, a method that adjusts FFN neuron activations according to each query while keeping the base language model frozen. A shared gate uses intermediate activations and layer embeddings to generate neuron-wise modulation coefficients. During training, subject-specific activation statistics identify candidate neurons, and sampled intervention configurations are evaluated through correctness, answer-margin, and intervention-strength rewards. Neu-GRPO then uses relative rollout feedback to optimize the gate. At inference, query activations guide modulation without subject labels. Experiments cover seven models from two families and three in-domain benchmarks, with improvements across all 21 combinations. Cross-benchmark experiments assess transfer from MMLU-Pro, while comparisons with fixed steering and LoRA-GRPO examine the benefits of the learned intervention.
Critical strengths
- The method improves all 21 in-domain model–benchmark combinations, covering two model families and sizes from 3B to 32B.
- The preliminary neuron analyses motivate the design, and the reward and sampling ablations provide useful evidence about its components.
Critical weaknesses
- Section 4.2 defines rollout rewards but leaves the Neu-GRPO update largely implicit. The policy probabilities, advantage calculation, clipping, and gradient path require a complete specification.
- The fixed-intervention baseline changes optimization and intervention coverage alongside query conditioning. A learned query-independent gate would isolate the contribution of adaptivity.
- Transfer varies by task. LiveBench shows four losses across seven models, and several advantages over LoRA-GRPO are small. Repeated runs would help establish the reliability of these differences.
Questions for authors
- What exact objective does Neu-GRPO optimize, and how are probabilities and gradients computed for the sampled intervention configurations?
- How much improvement remains with a learned query-independent gate under the same training budget?
5 comments
Response to Reviewer NM4L
W2&Q2 How to Validate the Independent Contribution of Query Adaptivity?
Thank you for your comments. We agree that comparisons with fixed intervention methods alone are insufficient to isolate the contribution of query adaptivity. Following your suggestion, we introduce a Query-Independent Trainable Gate with the same training budget on Qwen3-8B, along with two additional baselines: Averaged Coefficients and IA³ .
Query-Independent Trainable Gate. We replace the activation input to the GateNetwork with a fixed vector while retaining the layer embeddings, allowing it to learn layer-specific and neuron-wise modulation coefficients that remain unchanged across queries. The training configurationand intervention scope are kept identical to those of Q-NeuS.
Averaged Coefficients. We average the modulation coefficients generated by the trained GateNetwork over a calibration set containing queries from different subjects. The resulting fixed coefficients are then applied uniformly to all queries during inference.
IA³. IA³ is a parameter-efficient fine-tuning method that learns input-independent activation scaling parameters while keeping the base model frozen. We include it as an additional trainable static scaling baseline to compare with the dynamic modulation of Q-NeuS.
The detailed results are presented below.
In-Domain: MMLU-Pro. Cross-Benchmark: all remaining benchmarks.
| Method | MMLU-Pro | GPQA-M | GPQA-D | SuperGPQA | LiveBench | AGIEval | GSM8K | ARC-C |
|---|---|---|---|---|---|---|---|---|
| Base | 44.20 | 36.16 | 33.84 | 28.43 | 51.33 | 59.65 | 92.65 | 90.96 |
| Query-Independent Gate | 44.28 (+0.08) | 35.71 (-0.45) | 31.81 (-2.03) | 28.68 (+0.25) | 50.44 (-0.89) | 59.35 (-0.30) | 88.70 (-3.95) | 90.96 (0.00) |
| Averaged Coefficients | 46.30 (+2.10) | 33.04 (-3.12) | 34.85 (+1.01) | 28.52 (+0.09) | 25.81 (-25.52) | 52.81 (-6.84) | 46.32 (-46.33) | 90.78 (-0.18) |
| IA³ | 46.47 (+2.27) | 35.94 (-0.22) | 33.84 (0.00) | 28.50 (+0.07) | 50.21 (-1.12) | 59.86 (+0.21) | 90.98 (-1.67) | 91.30 (+0.34) |
| Q-NeuS | 48.53 (+4.33) | 36.38 (+0.22) | 36.87 (+3.03) | 29.54 (+1.11) | 51.96 (+0.63) | 59.28 (-0.37) | 93.25 (+0.60) | 91.64 (+0.68) |
The Query-Independent Trainable Gate yields only a marginal improvement on the in-domain benchmark and shows limited transfer performance. Averaged Coefficients improves accuracy on a few benchmarks but exhibits substantial degradation on others, while IA³ generally underperforms Q-NeuS. These results suggest that static modulation strategies, whether learned directly or derived from a trained GateNetwork, may have limitations in accommodating heterogeneous queries.
In comparison, Q-NeuS outperforms all three static baselines on nearly all evaluated benchmarks, suggesting that dynamic neuron modulation provides greater performance benefits on these tasks and offering further evidence for the value of query adaptivity.
Response to Reviewer NM4L
W3 Variability in Cross-Task Transfer Performance and Opportunities for Further Reliability Validation
Thank you for pointing out this concern. We agree that repeated runs can help improve the reliability of experimental evaluations. To address this, we repeated the experiments with another random seed while keeping the data splits and all other training settings unchanged. The complete results are presented below.
In-Domain: MMLU-Pro. Cross-Benchmark: all remaining benchmarks.
| Model | MMLU-Pro | GPQA-M | GPQA-D | SuperGPQA | LiveBench | AGIEval | GSM8K | ARC-C |
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-3B | 37.22 (+5.79) | 31.03 (+0.90) | 34.34 (+2.02) | 22.44 (+3.18) | 23.53 (-1.10) | 52.55 (+0.54) | 83.32 (-0.15) | 83.62 (+1.03) |
| Qwen3-4B | 46.80 (+1.94) | 37.72 (+1.11) | 40.40 (+3.03) | 27.74 (+1.32) | 55.06 (0.00) | 58.68 (-0.24) | 90.98 (+0.53) | 89.76 (-0.43) |
| Qwen3-8B | 48.86 (+4.66) | 37.95 (+1.79) | 38.89 (+5.05) | 29.61 (+1.18) | 52.08 (+0.75) | 60.55 (+0.90) | 93.48 (+0.83) | 91.30 (+0.34) |
| Qwen3-14B | 53.70 (+2.81) | 39.06 (+0.89) | 37.88 (-1.01) | 32.51 (+3.03) | 56.72 (-0.49) | 65.98 (+0.57) | 93.48 (-0.83) | 92.92 (+0.68) |
| Qwen3-32B | 59.31 (+4.75) | 44.87 (+1.34) | 45.45 (+2.02) | 32.96 (+3.64) | 59.09 (+0.40) | 69.37 (0.00) | 95.38 (+0.46) | 94.80 (+0.26) |
| Ministral3-8B | 48.86 (+4.04) | 39.06 (+1.78) | 39.90 (+5.05) | 28.17 (+3.24) | 39.38 (-0.74) | 54.67 (+0.15) | 91.21 (+0.84) | 90.44 (-1.03) |
| Ministral3-14B | 53.82 (+3.34) | 35.71 (-0.90) | 35.86 (+1.52) | 30.31 (+1.72) | 42.26 (+0.21) | 58.99 (+0.18) | 94.24 (+0.76) | 93.17 (+0.85) |
| Win / Tie / Loss | 7 / 0 / 0 | 6 / 0 / 1 | 6 / 0 / 1 | 7 / 0 / 0 | 3 / 1 / 3 | 5 / 1 / 1 | 5 / 0 / 2 | 5 / 0 / 2 |
These results show that the main performance gains are maintained under another training seed, providing additional evidence for the reliability of Q-NeuS.
Regarding the comparable performance with LoRA-GRPO, our primary goal is to explore query-adaptive neuron modulation while keeping the pretrained LLM frozen. Unlike LoRA-GRPO, which learns shared parameter updates, Q-NeuS dynamically adjusts neuron-wise modulation coefficients based on query-induced activations. Despite updating only the GateNetwork, Q-NeuS achieves performance comparable to LoRA-GRPO on in-domain benchmarks, demonstrating the potential of query-adaptive neuron steering.
Response to Reviewer NM4L
Thanks for your positive feedback and constructive comments. We would like to address your concerns in weaknesses and questions as follows.
W1&Q1 Complete Mathematical Formulation and Gradient Propagation Mechanism of Neu-GRPO
Thank you for pointing out that the original manuscript did not explicitly specify the Neu-GRPO update. For each query q, we sample K neuron masks M(k) from subject-guided candidate sets and hold them fixed during the parameter update. Under each mask, the GateNetwork generates neuron-wise modulation coefficients. We apply softmax to the valid single-token answer-option logits at the final prompt position to obtain πϕ(a∣q,M(k)). Thus, the objective uses answer probabilities conditioned on the sampled masks, rather than probabilities of the masks themselves. Each rollout samples an answer a(k) from this distribution and receives a reward R(k). Rewards for the same query are normalized within the group:
A(k)=σR+ϵaR(k)−μR.
Neu-GRPO then minimizes the following GateNetwork loss:
=-\frac{1}{K}\sum_{k=1}^{K} \operatorname{sg}(A^{(k)}) \log\pi_\phi(a^{(k)}\mid q,\mathbf{M}^{(k)}),$$ where $\mu_R$ and $\sigma_R$ are the mean and standard deviation of the $K$ rewards, $\epsilon_a=10^{-8}$ provides numerical stability, and $\operatorname{sg}$ denotes stop-gradient. During the update, the masks, sampled answers, and advantages are held fixed, while answer probabilities are recomputed under the same masks with gradient tracking. Gradients pass from these probabilities through the frozen LLM’s differentiable operations, the modulated FFN activations, and the continuous coefficients to the GateNetwork parameters; the backbone parameters remain unchanged. We accumulate gradients over the $K$ rollouts, clip their global norm, and take one Adam step. The revised Appendix B provides the full update procedure and gradient derivation.