- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MoRe, a prompt utility metric that measures KV-cache sensitivity to gate modulation without additional rollouts or backward passes. The authors establish a theoretical connection between MoRe and gradient feedback energy, supported by empirical and mechanistic analyses. Building on this metric, they develop a two-stage prompt sampling strategy that combines reward-based filtering with MoRe-guided selection. Experiments on mathematical reasoning tasks demonstrate improved sample efficiency and accuracy over existing baselines.
Critical strengths
-
The paper addresses sample-efficient prompt selection in RLVR by considering model-internal signals beyond reward history.
-
MoRe provides a mathematically grounded measure of prompt sensitivity through gate modulation, without requiring additional rollouts or backward passes.
-
The paper investigates the relationship between MoRe and gradient feedback through matched comparisons and attention interventions.
Critical weaknesses
-
MoRe measures Jacobian sensitivity and is empirically associated with gradient feedback energy. However, larger gradient energy does not necessarily imply more useful policy updates, as it may simply reflect higher gradient variance.
-
The theoretical connection between MoRe and gradient energy relies on isotropic reference feedback, whereas actual GRPO feedback can be highly anisotropic.
-
The experiments are limited to relatively small models (up to 4B parameters) and primarily focus on mathematical reasoning tasks. Moreover, the analysis connecting MoRe to gradient feedback is conducted on a single 0.5B model. Evaluations on larger models and more diverse RLVR tasks, such as code generation and agentic tasks, would help establish the generalizability of the proposed metric.
Questions for authors
MoRe only measures sensitivity along gate-modulation directions through the final-layer prompt KV-cache. While this makes the metric computationally tractable, it is unclear whether this restricted subspace is representative of the full-parameter gradients used in RLVR. Comparisons with other layers and parameter subspaces would strengthen the motivation for this design.