- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VISTA, a training framework for active multimodal agents that shares visual observations across same-input rollouts. CVED constructs decision-aligned privileged teacher contexts and distills teacher distributions along student-generated response prefixes. HAPI reinforces successful trajectories and applies distillation to unsuccessful ones. The trained agent retains its visual tools and uses only its own interaction history at inference.
Critical strengths
The paper is generally clear and offers a useful perspective on reusing visual discoveries beyond scalar outcome rewards. Controlled comparisons show consistent improvements across perception and reasoning benchmarks and several Qwen backbones. The structured-context, self-only hindsight, and crossed-routing ablations provide informative evidence for the value of collective experience and differentiated supervision.
Critical weaknesses
- Important experience-construction details are delayed to the appendix. A concise main-text description and a concrete example would substantially improve accessibility. Please show the source trajectories, selected crops and relations, and the instantiated teacher history for one student decision, including useful evidence contributed by an unsuccessful trajectory.
- The mechanism behind HAPI's advantage remains only partly understood. Uniform Hybrid differs from HAPI in both reinforcement and distillation routing, not merely in whether successful trajectories receive distillation. Table 11 usefully separates these masks, but changing the distillation subset also changes its token denominator and effective weighting. Controls matching loss scales, together with gradient-agreement or teacher-disagreement analysis, would clarify why joint routing helps and whether distillation interferes with successful behavior.
- Experience selection and teacher reliability deserve further validation. It would be useful to measure teacher quality when all rollouts fail or provide redundant evidence, and whether privileged hindsight encourages answering before the student has acquired sufficient evidence. These diagnostics would help characterize when collective experience supplies reliable corrective guidance.
- The practical cost of the gains are insufficiently quantified. Training schedules are matched, but teacher inference and experience construction introduce additional computation. Please report GPU-hours, peak memory, and performance under matched total compute. Although results are averaged over three seeds, standard deviations or confidence intervals would help assess modest differences between routing and teacher-update variants.
Questions for authors
See weakness.
3 comments
Response to tGQm (2/2): Teacher guidance, premature answers, and cost
3. Teacher guidance from unsuccessful or redundant experience. An unsuccessful trajectory can still contain relevant evidence, even if the student fails to use it to produce a correct final answer. CVED preserves useful observations even from unsuccessful sources, and source/action relations allow the teacher to interpret them jointly. With binary rewards, an all-failure group has zero group-relative task-reward advantage, whereas complementary observations can still provide a distillation signal. The successful-source-only versus full-group comparison in Appendix H.3 empirically supports retaining failed-source evidence in mixed-success groups.
The key condition is whether the shared observations supply evidence missing from the student's history. Redundant or insufficient observations may leave that gap unresolved. Categorical feedback and acquisition context help the teacher interpret the available evidence, but do not guarantee correct guidance. Appendix H.3 clarifies this condition: collective experience can reuse discoveries that outcome-only supervision leaves underused, with its value depending on the information those discoveries provide.
4. Premature answering under CVED and Oracle ROI. We compare the two teacher-context variants from Table 3 on the same 100-question subset requiring additional evidence. HAPI routing and the student interface are fixed, and necessary evidence is annotated independently of the methods. A response is premature if the student answers before acquiring that evidence.
| Teacher context | Premature answers | Premature and incorrect |
|---|---|---|
| Oracle ROI | 18/100 | 15/100 |
| CVED | 10/100 | 7/100 |
On this subset, CVED reduces both rates by eight percentage points. The teacher receives collective observations with acquisition context and decision-relative alignment; oracle crops remain training-only in the Oracle ROI control. This comparison tests the complete teacher-context design, while Table 3's identical-observation CVED-versus-Unstructured comparison tests evidence organization. Appendix H.3 (Table 17) reports the diagnostic and its definitions.
5. Variability and teacher updates. Appendix H.5 provides task-level S1-S3 results and sample SD for the key ablations. HAPI exceeds Uniform in all three original runs (83.83 ± 0.22 versus 82.87 ± 0.12). For teacher updates, the six-task perception average is 73.81 ± 0.19 for EMA and 73.53 ± 0.59 for the frozen teacher. EMA's modest average advantage is task-dependent: it improves V*Bench/ZoomBench, while the frozen teacher scores higher on HR-8K/MME-RW-CN. Both teachers receive newly collected experience, so this comparison measures the additional effect of updating teacher weights.
6. Cost and common-budget performance. On eight H100 GPUs, Qwen3-VL-4B training uses 150/165/180 GPU-hours for GRPO/OPSD/VISTA, with similar peak memory of approximately 80 GB per GPU. Final three-visual-task averages are 80.57/81.61/83.83. At a common 150-GPU-hour cap they are 80.57/81.53/83.75, so VISTA retains gains of 3.18/2.22 points over GRPO/OPSD. At 100 GPU-hours, VISTA already reaches 82.09. These are three-run means using the last checkpoint within each cap, with the same initialization, data, and tool interface. Section 4.9 and Appendix H.2 (Table 13 and Figure 6) report the full comparison.
The new results and analyses are highlighted in blue in the revised manuscript.
We hope these responses and new results have addressed your concerns. Thank you again for your valuable suggestions and for helping us improve the manuscript. We look forward to your further feedback!
Follow-up on our response to Reviewer tGQm
Dear Reviewer tGQm,
We appreciate your constructive feedback on our submission. We have uploaded our responses and additional analyses addressing each of your points. As the discussion phase is drawing to a close, we would welcome your thoughts on whether our reply resolves your concerns, and we remain happy to provide any further clarification.
Thank you again for your time,
The authors
Response to tGQm (1/2): Concrete construction and routing evidence
We sincerely thank you for your thoughtful feedback and helpful suggestions on explaining and evaluating collective experience. In response, we have clarified its construction with a worked example and added a common-denominator routing control, along with analyses of teacher guidance, observation behavior, and training cost. We have incorporated these additions into the revised manuscript and provide detailed responses below.
1. A concise construction description and worked example. To make the construction easier to follow, Section 3.2 now describes each step: deduplicate crop images while retaining source occurrences; select up to B=4 artifacts including required crop ancestors; and serialize producing actions, artifact relations, decision-relative visibility, and categorical outcomes. Generated final answers and reference answers are excluded from the added teacher context.
The Fullerton example illustrates the missing evidence. A crop from an unsuccessful student trajectory contains "THE FULLERTON" but omits "HOTEL," the word distinguishing the answer options. A peer first localizes the building and then captures the full inscription. At the same student-sampled prefix, the teacher sees the original student history plus the peer views and their acquisition context; the student still sees only its original history. JSD transfers the resulting conditional distribution without replacing the sampled trajectory.
Appendix H.4 provides the source observations, the actions that produced them, their ancestry relationships, and a readable schematic of the teacher context for this example. The source-pool experiment provides complementary evidence for the value of observations from failed attempts: using full-group observations increases average accuracy from 83.29 to 83.83 compared with successful-source-only observations under the same B=4 cap. In a 200-group selector diagnostic with 765 artifacts selected per condition, 115 full-group artifacts come exclusively from unsuccessful trajectories. This complements the worked example with quantitative evidence for failed-source contributions (Appendix H.3).
2. Joint routing remains beneficial when the denominator is controlled. To address the normalization concern, we fix lambda = 0.10, the RL denominator, training data, optimization budget, and teacher updates, and normalize every distillation term by the full-group response-token count N.
Joint routing exceeds Uniform by 0.80 points and also outperforms both single-mask controls. The original denominator adds 0.16 points. The revised main-text analysis and Appendix H.1 (Table 12) report this control alongside the original factorial ablation.
The routing design assigns complementary roles to the two learning signals: verified successful behavior receives outcome-based reinforcement, while unsuccessful attempts receive token-level guidance informed by the group's additional observations. Since teacher and student condition on different information, they can prefer different continuations even along a successful trajectory. HAPI directs this corrective signal to unsuccessful attempts while preserving reinforcement of verified success. The common-denominator experiment supports this joint allocation without requiring the assumption that every teacher disagreement is harmful.