- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RILO targets industrial anomaly question answering on MMAD. After a structured four-field SFT, it trains a policy and an implicit process-reward branch (both LoRA adapters) against a frozen SFT reference. The reward branch learns from a lexical, rule-based process score, and its token-level log-ratio serves as a dense PRIME-style reward for PPO. The distinctive component is visual modulation: every 64 generated tokens, the prefix is scored, the scores are turned into a global magnitude and 20 group weights, a scalar residual is added to contiguous groups of query-image embeddings, and the context is re-prefilled . RILO-4B reports 81.26 (one-shot) and 79.52 (zero-shot) on MMAD.
Critical strengths
-
Reusing one learned process signal for both credit assignment and visual adaptation is intuitive and clearly presented.
-
The ablations are careful for the scale of the study: three seeds, a 2×2 decomposition of training-time modulation versus inference-time control, and an explicit cost analysis.
Critical weaknesses
1, Weak motivation. The premise that "static" visual representations cannot follow changing evidence needs overlooks that the decoder already re-reads the image at every step through attention; only the input embeddings are fixed. No pilot study or analysis shows that fixed embeddings are an actual bottleneck.
2., The modulation is a small, coarse perturbation rather than evidence-guided re-inspection. Each of 20 contiguous token groups receives one scalar on all dimensions (α = 0.01), with weights interpolated from answer-token order onto image-token order, and the authors acknowledge that the profile is not tied to image regions . The loop also rarely fires: responses are capped at 160–192 tokens, and the 4B model's mean response length falls to 41 tokens, below the first 64-token boundary.
-
The gains are not attributed to the claimed mechanism. Most of the improvement comes from the structured verifier plus PRIME-style implicit rewards (68.00 → 73.32 on 3B) and from applying modulation during training (+1.24 on 4B), while inference-time control adds only +0.16 on 4B at 1.88× evaluation cost. All specificity controls are inference-time. Without a training-time control that replaces the reward-guided profile with a magnitude-matched random perturbation, the training-time gain may simply be a regularization or input-noise effect.
-
Small margins and reporting issues. The margins over the strongest baselines are small (+0.07 over Gemini-2.5-Pro and +0.43 over DCR2-AD (+DK)-7B, one-shot), baseline numbers come from mixed sources, Table 1 duplicates two rows, and β_RM and β_vis are not reported.
Questions for authors
1, What happens if training-time modulation uses a magnitude-matched random or uniform perturbation instead of the reward-guided profile?
2, Can you provide evidence that fixed input embeddings, as opposed to attention, are a bottleneck for anomaly reasoning?
5 comments
Rebuttal to Reviewer HySr (2/4)
Q2. On the Spatial Relevance and Actual Activation Frequency of Visual Modulation
A2: We thank the reviewer for the concern about the visual-modulation strategy and its practical effect. We agree that RILO uses lightweight group-level visual-feature modulation and does not establish an explicit correspondence between language tokens and image spatial positions. We do not claim that fixed visual embeddings prevent the decoder from dynamically attending to image evidence. Rather, RILO investigates whether process-reward feedback can provide an additional visual-state update within the current rollout during RL training. Nevertheless, the reviewer's question about whether this interpolated modulation has actual spatial relevance deserves further examination.
To this end, we added a spatial analysis based on ground-truth defect masks. Specifically, we first map the ground-truth defect masks to the image regions represented by the visual tokens after Qwen3-VL's actual image preprocessing. We then apply RILO's original 20-group index mapping to associate each visual token with its corresponding modulation weight and calculate the normalized proportion of modulation weight assigned to defect-region tokens. The masks are used only for offline spatial analysis and are not involved in training, inference, or modulation-weight generation.
Table 2. Differences in normalized modulation-weight allocation to ground-truth defect regions between Learned and the two controls (percentage points).
| Analysis Set | Query Images | Learned − Position-only (pp) | Learned − Cross-sample (pp) |
|---|---|---|---|
| Four defect-related QA | 550 | 0.196 | 0.199 |
| Defect Localization | 119 | 0.059 | 0.068 |
Across 550 query images, Learned achieves higher normalized weight allocations to actual defect regions than Position-only and Cross-sample, with differences of +0.196 and +0.199 percentage points, respectively. The Defect Localization subset shows positive differences in the same direction (+0.059 and +0.068 pp). The Defect Localization subset shows results in the same direction. Learned therefore has a positive advantage in weight allocation to actual defect regions over both controls, further supporting the observation of a spatial association between reward-conditioned modulation and actual defect regions.
Regarding the VM activation frequency, we further examined the actual training logs from the first 188 steps of RILO-4B RL training. The average response length of approximately 41 tokens reported in Appendix L of the original paper corresponds to the later stage of training, rather than the entire training process.
Table 3. VM activation across different training stages.
| Training Phase | Activated / Total | Rate |
|---|---|---|
| Pass 1–4 | 14,367 / 16,800 | 85.52% |
| Pass 5–8 | 890 / 16,800 | 5.30% |
| Pass 9–13 | 21 / 19,040 | 0.11% |
| Overall | 15,278 / 52,640 | 29.02% |
The activation statistics cover the first 188 RL training steps actually recorded, totaling 52,640 rollouts. Pass 13 includes only the 8 steps that had been completed. Effective Activation means that at least one actual non-zero visual-state update was performed in the corresponding rollout.
The statistics show that the effective activation rate is 85.52% (14,367/16,800) in the first four training passes, decreases to 5.30% in Passes 5–8, and further decreases to 0.11% in Passes 9–13. In total, 15,278 of the 52,640 rollouts have effective VM activation, corresponding to an overall activation rate of 29.02%. Therefore, we agree with the reviewer that VM is triggered less frequently in the later stages of training. However, a large number of effective visual updates occur during early training, so the use of modulation throughout the training process cannot be judged solely from the response length at the later stage.
In addition, the newly added two-seed matched training ablation shows that the two-seed mean MMAD Avg. of Learned VM is 1.09 and 1.52 percentage points higher than Reward-Random and Norm-matched Uniform, respectively. Learned VM also outperforms both controls under each independent seed. This further supports the training gains of process-reward-conditioned visual modulation.
In summary, the new mask analysis provides supplementary evidence of a positive spatial association between reward-conditioned modulation weights and actual defect regions; the training logs clarify the actual use of VM at different training stages; and the training ablation further supports the learning gains from process-reward-driven visual-state updates. Together, these results address the reviewer's questions about spatial relevance, practical effects, and activation frequency, while clarifying that RILO does not rely on an explicit spatial-correspondence assumption.
Rebuttal to Reviewer HySr (3/4)
Q3. On the Attribution of Gains to Training-Time Visual Modulation
A3: We thank the reviewer for the in-depth analysis of the sources of RILO's performance gains. We would like to further clarify that RILO's core contribution is to reuse implicit process rewards not only for policy optimization but also to guide visual-state updates within an RL rollout, thereby improving the final Actor. Table 3 of the original paper already distinguishes this contribution through a stage-wise ablation: even with the inference-time Controller disabled, training-time VM increases the MMAD Avg. of RILO-4B from 79.86% to 81.10% (+1.24 pp). To address the reviewer's key question of whether this gain comes from effective process-reward guidance or merely from the regularization effect of ordinary visual perturbations, we added direct training-time controls.
Specifically, under a unified training and evaluation protocol, we conducted a matched two-seed RL training ablation for RILO-4B, using two independent training seeds to compare Learned VM, Reward-Random VM, and Norm-matched Uniform VM. Reward-Random samples modulation strengths and visual-group weights from a fixed calibration distribution derived from Learned VM. It does not depend on the current sample's PRM feedback, and its perturbation magnitude is calibrated according to the actual number of visual tokens. Norm-matched Uniform retains the current sample's global PRM modulation signal and matches the perturbation strength using the visual-token-weighted RMS, but sets the weights of all visual groups to 1. These two controls separately examine whether ordinary random perturbations can produce the same gains and whether the non-uniform weight allocation formed by process rewards has an additional effect.
As shown in Table 1 in our response to Q1, Learned VM achieves the highest accuracy under both independent seeds, with a two-seed mean MMAD Avg. of 80.97%. This is 1.09 and 1.52 percentage points higher than Random and Uniform, respectively. Furthermore, the per-seed gains of Learned over Random are +1.16 and +1.02 pp, while those over Uniform are +1.45 and +1.59 pp, showing consistent performance advantages. These results provide direct evidence for attributing RILO's training gains: under matched training conditions, non-uniform visual modulation conditioned on process rewards is more beneficial to final policy learning than ordinary random perturbations and uniform offsets of the same strength. This further supports RILO's core design of extending implicit process rewards from policy-optimization signals to visual feedback within a rollout, allowing process rewards not only to participate in training credit assignment but also to improve Actor learning through visual-state updates.
Regarding the inference cost raised by the reviewer, we agree that the online Controller increases computational cost, which has been quantified in Appendix K. The main performance gains of RILO are already retained in the trained Actor, and the inference-time Controller is an optional mechanism for further improving accuracy rather than a requirement for deploying RILO. According to Table 3 of the original paper, the 4B model retains approximately 89% of its overall performance gain with the inference-time Controller disabled. Therefore, for latency-sensitive deployment, the trained Controller-Off Actor can be used directly, retaining the main performance gains while avoiding additional online control overhead.
In summary, the newly added training-time controls show that the gains from RILO are not reproduced by the tested generic visual perturbations; rather, they depend on how process rewards guide visual-state updates. This clarifies the core contribution we aim to emphasize: using closed-loop process feedback during training to improve the Actor's visual reasoning capability, without requiring continuous online visual modulation at inference time.
Rebuttal to Reviewer HySr (1/4)
Q1. On the Motivation for RILO
A1: We thank the reviewer for pointing out that our presentation of the research motivation was not sufficiently precise. We agree that although visual-token representations remain fixed during decoding, the decoder can still dynamically adjust its use of visual information according to the generation context through attention. At the same time, we would like to clarify that RILO does not claim that fixed visual embeddings prevent the model from dynamically attending to the image, and our main objective is not to improve performance through inference-time visual modulation.
RILO focuses on how process-evaluation feedback can be used to adjust the visual state within the current rollout during reinforcement-learning training, thereby improving the final policy learning. Conventional implicit process rewards are mainly used for policy optimization after trajectory generation. RILO further uses this feedback to update the visual features on which subsequent generation within the current rollout depends, allowing the Actor to learn under process-reward-conditioned dynamic visual states. This does not conflict with the decoder dynamically allocating attention according to the text context; rather, it adds a process-evaluation-driven visual feedback path on top of that capability.
This motivation is supported by the experimental results. Table 3 of the original paper shows that, even with the Controller completely disabled at inference time, enabling VM during training increases the 4B model's MMAD Avg. from 79.86% to 81.10% (+1.24 pp). This indicates that the main gains are reflected in the Actor obtained after training, rather than depending on continuously modifying visual features during inference.
To directly examine whether the training-time visual-modulation gains arise merely from general feature perturbations, we added a two-seed RL training ablation for RILO-4B, using two independent training seeds. All models in this training ablation are evaluated with the inference-time Controller disabled, so that the effects of the training-time visual-modulation strategies can be compared independently.
Table 1. Two-seed reproduction of the training-time VM ablation (MMAD Avg., %).
Reward-Random uses modulation weights and strengths sampled from a fixed Learned calibration distribution, independently of the current sample's PRM feedback, with perturbation magnitude calibrated to the actual number of visual tokens. Norm-matched Uniform retains the current sample's global PRM modulation signal and matches the perturbation magnitude using the visual-token-weighted RMS, while setting all visual-group weights to one.
As shown in Table 1, Learned VM achieves the highest accuracy under both independent seeds, with a two-seed mean MMAD Avg. of 80.97%. This is 1.09 and 1.52 percentage points higher than Random (79.88%) and Uniform (79.45%), respectively. Specifically, Learned improves over Random by 1.16 and 1.02 percentage points under the two seeds, and over Uniform by 1.45 and 1.59 percentage points, respectively, showing consistent performance advantages. These results indicate that process-reward-conditioned visual-state updates are more beneficial to final policy learning than the tested random perturbation and uniform offset, rather than merely increasing visual-feature perturbations.
We will further clarify this distinction in the revised manuscript: RILO does not treat fixed visual representations as an inherent limitation on dynamic attention. Instead, it studies how implicit process evaluation can be converted into visual-state feedback within an RL rollout, thereby improving the final learned policy.