- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes AdaSparse-PRL, an efficient multimodal fine-tuning framework that combines: (1) implicit cross-modal decoupling of visual and textual representations into shared and modality-specific components, (2) input-adaptive sparse parameter updates based on gradient and representation statistics, and (3) progressive reinforcement learning involving reward-model training, scheduled PPO optimization, and sparsity-preserving alignment. The method is evaluated on MM-Vet, MMBench, and VQAv2 using several Qwen-VL model sizes. The paper reports improvements over PPO, RFT, LoRA, QLoRA, GRPO, and other fine-tuning methods while claiming substantial reductions in training time, FLOPs, and memory.
The problem is important, but the proposed algorithm is not described consistently enough to be reproducible. More seriously, the headline results, hyperparameters, masking procedures, and memory measurements contradict one another across the paper. These problems prevent confidence in either the method or the empirical conclusions.
Critical strengths
Reducing the cost of preference alignment for multimodal models is a timely and practically meaningful objective. A method that retains model quality while reducing PPO training cost would be valuable, particularly for groups with limited computational resources. The paper also attempts to evaluate multiple model sizes and report accuracy, training time, FLOPs, and memory rather than considering task performance alone.
The authors provide ablations for cross-modal decoupling, adaptive sparsity, and progressive RL across multiple datasets and model sizes. This is the appropriate general structure for evaluating a framework containing several distinct components. Unfortunately, the value of these ablations is substantially reduced by inconsistencies between the main results and appendix tables.
Critical weaknesses
The paper does not clearly distinguish between masking weights, gradients, optimizer updates, activations, or structured model components. Equation 6 applies the mask, which suggests that parameters are removed during the forward pass. Elsewhere, inactive parameters are described as “frozen,” which would preserve them during the forward pass and only suppress their updates. These are fundamentally different algorithms. If only gradients are masked, most forward and backward computation remains dense, and the claimed FLOP reductions do not follow. If weights are removed, the authors must explain how input-dependent, unstructured parameter sparsity is converted into efficient GPU execution.
The importance score also depends on the gradient of every parameter. Computing these scores appears to require a dense backward pass before selecting the sparse subset, potentially eliminating the claimed savings. The paper does not explain whether gradients are computed densely, estimated from a separate batch, or reused across mask-update intervals. Nor does it describe how optimizer states are managed when the active subset changes.
Several mathematical terms are additionally problematic. The norm of a fixed-size hard Top K mask is constant and therefore cannot optimize mask selection. The gradient-truncation penalty appears to require second-order derivatives unless its gradient is detached; if detached, it provides no training signal. The KL divergence between deterministic Bernoulli masks is either zero or infinite, while the momentum update of two binary masks produces a non-binary mask. The paper does not provide a continuous relaxation, straight-through estimator, or thresholding procedure that resolves these issues.
The mask definition also changes across sections. The main method selects parameters using gradient magnitude, private-representation variance, and activation norms, whereas the appendix uses weight magnitude. Mask-update intervals are reported as both 50 and 10 steps. Consequently, it is unclear which algorithm was actually implemented.
Headline results differ dramatically between the main paper and appendix. For example, the reported Qwen-2-VL-2B MM-Vet score is 47.8 in the main table but 58.3 in the appendix. For Qwen-2-VL-7B on MMBench, the corresponding values are 81.8 and 59.8. The 7B VQAv2 result is variously reported as 81.2, 79.4, and 81.5. Differences of this magnitude cannot reasonably be attributed to random seeds. Similar inconsistencies occur for the 32B experiments.
The reward-stability motivation is also contradicted by the manuscript’s own values: AdaSparse-PRL is reported to have reward variance 0.28, compared with 0.26–0.27 for DPO and PPO, despite the text claiming lower variance. Hyperparameters likewise conflict: the initial KL coefficient is given as both 0.5 and 1.0.
Memory reporting is similarly unclear. The paper reports approximately 14.2 GB for the 7B model in one section, 18.2–20.3 GB in another, and approximately 36 GB or 48.6–51.2 GB for the 32B configuration in different places. The manuscript does not provide enough information about batch size, sequence length, quantization, checkpointing, ZeRO configuration, or offloading to reconcile these numbers.
Baseline fairness is also uncertain. The proposed method uses 4-bit loading, gradient checkpointing, ZeRO-3, CPU offloading, and sparse optimizer states, but it is not clear whether the same systems optimizations are applied to PPO, LoRA, QLoRA, and other baselines. Thus, the reported efficiency improvements may reflect unequal infrastructure settings rather than the proposed algorithm.
Finally, benchmark evaluation is underspecified. MM-Vet requires a clearly identified evaluator and scoring pipeline, while MMBench requires the split, language version, circular-evaluation protocol, and answer-extraction method. The paper instead reports generic “accuracy” without sufficient details. It also provides no confidence intervals or per-seed results for the main accuracy claims.
The theoretical analysis does not resolve these concerns. It reduces the method to a generic stochastic optimization argument and does not analyze hard TopK selection, changing masks, PPO clipping, learned reward-model error, or policy-generated data. The claimed improvement in the convergence rate relies on an unsupported assumption concerning variance reduction under adaptive TopK selection.
Overall, the paper addresses a worthwhile problem, but the current manuscript does not define a coherent, reproducible algorithm. The extensive numerical contradictions further prevent the reported results from being treated as reliable evidence. I therefore recommend strong reject.
Questions for authors
Clarification: What exactly is masked: forward-pass weights, gradients, optimizer updates, activations, or structured units? Please provide executable pseudocode for one complete training step, including mask construction and optimizer-state handling.
Clarification: How are per-parameter gradient importance scores computed without first performing a dense backward pass? Is the cost of scoring and global TopK selection included in all FLOP, memory, and wall-clock measurements?
Which reported results are correct? In particular, please reconcile the main and appendix results for the 2B, 7B, and 32B models, including the three different VQAv2 values reported for the 7B configuration.
Which mask rule was actually used: gradient/variance/activation-based selection or weight-magnitude selection? Was the mask updated every 10 or 50 steps?
Were all baselines evaluated with identical quantization, gradient checkpointing, ZeRO stage, CPU offloading, sequence length, batch size, rollout budget, and number of optimization steps?
What exact evaluation pipelines were used for MM-Vet, MMBench, and VQAv2, including evaluator models, dataset splits, prompt templates, decoding settings, and answer normalization?