- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper constructs approximately 240K training trajectories through hierarchical audio-visual evidence modeling, multi-segment interaction, and teacher-guided error recovery, and further introduces SEPO based on evidence gain. The resulting 7B and 30B agents consistently outperform their respective base models across multiple audio-visual and video understanding benchmarks.
Critical strengths
- The data pipeline focuses on cross-span evidence comparison and error recovery, which are both useful capabilities for audio-visual agents.
- The experiments not only demonstrate the effectiveness of the proposed method but also provide insights into its performance across different data scales.
Critical weaknesses
-
The evidence scorer only matches intervals and modalities without verifying evidence understanding or including the initial overview. Evidence gains should be validated to ensure they reflect newly acquired information and actual answer improvement.
-
Reflection data includes both teacher corrections and successful follow-up trajectories, so the overall QA improvement cannot solely be attributed to the reflection ability itself, as it also involves an increase in data scale.
Questions for authors
- The paper mentions using Gemini-3.6 as the teacher. How is the COT in this trajectory generated? Additionally, the same teacher is used for reflection, so how does it manage to correct itself when mistakes occur in the steps?
- The effectiveness of SEPO compared to GSPO is not very evident. Could you report results from multiple checkpoints to ensure the findings are not cherry-picked, or provide additional evidence to demonstrate consistent effectiveness?
6 comments
Response to Reviewer RhjB (2/3)
W2: About Reflection Supervision and Data Scale
Unlike simply adding more Stage-2 trajectories, reflection trajectories branch directly from the student's own failed rollout prefixes (similar to DAgger [ref1]; Section 2.3), teaching recovery on error states the student actually visits. Notably, 9.7K of our 17.4K reflection trajectories (spanning 3,926 queries) recover hard cases where all 8 student rollouts failed or standard Stage-2 teacher sampling yielded no accepted trajectory.
Table C: Controlled ablation on a 25% OmniCoach subset (62.0K trajectories; component counts rounded independently; Qwen3-Omni-30B-A3B) comparing reflection against equal successful trajectories.
| 25% Data Composition (62.0K Traj.) | Multi-Span + Refl. | Sup. Tokens / Ep. | Video-MME | Video-MME-v2 | OmniVideoBench | LVOmniBench | Avg. |
|---|---|---|---|---|---|---|---|
| Replace Refl. w/ Equal Success | 57.7K + 4.4K Succ. | 21.15M | 72.0 | 37.4 | 41.4 | 41.3 | 48.0 |
| Refl. (1 turn after mistake) | 57.7K + 4.4K Refl. | 20.94M | 72.2 | 37.9 | 42.1 | 41.7 | 48.5 |
| Refl. (2 turns after mistake) | 57.7K + 4.4K Refl. | 21.56M | 72.6 | 38.5 | 42.5 | 42.1 | 48.9 |
| Refl. (3 turns after mistake) | 57.7K + 4.4K Refl. | 21.96M | 72.9 | 38.7 | 42.4 | 42.5 | 49.1 |
| 25% OmniCoach (Full Refl.) | 57.7K + 4.4K Refl. | 22.82M | 72.8 | 38.9 | 42.7 | 42.4 | 49.2 |
1. Outperforms equal successful-trajectory volume. With the total number of trajectories fixed, reflection supervision outperforms additional successful trajectories, especially on multi-step reasoning benchmarks.
2. Driven by corrective turns rather than follow-up trajectory length. With pre-mistake student turns masked (loss = 0), the first corrective turn alone already outperforms the equal-success baseline while using fewer tokens. Two or three corrective turns recover most of the full reflection gain; longer follow-up trajectories add little. We have added this ablation to Section 4.3.
[ref1] A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, AISTATS 2011.
Response to Reviewer RhjB (3/3)
Q1: Details About Teacher CoT and Error Correction
In Stage-2, we do not replay Gemini's original CoT or expose reference answers or evidence annotations. Given only the question, the initial media overview, and the tool interface, gemini-3.6-flash generates its observation and think fields on the fly at each step to decide whether to call a tool or answer, and we keep only the rollouts whose final answers pass verification.
In Stage-3, the errors being corrected come from failed student rollouts rather than gemini-3.6-flash itself. An independent verifier first checks the student trajectory against the media to locate the first unsupported or contradicted step. Given the prior history, that unexecuted failed turn, and the verifier's note, gemini-3.6-flash writes a media-grounded correction and revised action on the fly, then continues the rollout until reaching a verified answer (the verifier's note is stripped before training). We have clarified this workflow in Section 2.3.
Q2: About Multi-Checkpoint and Multi-Seed Results
Table F: Evaluation across RL training checkpoints from the SFT checkpoint (GSPO → SEPO, Accuracy %).
| RL Progress | Video-MME | Video-MME-v2 | OmniVideoBench | LVOmniBench | 4-Bench Avg. |
|---|---|---|---|---|---|
| 50% of schedule | 75.1 → 75.5 | 43.0 → 43.8 | 46.6 → 47.5 | 47.8 → 47.7 | 53.1 → 53.6 |
| 75% of schedule | 75.4 → 76.0 | 43.3 → 44.1 | 46.9 → 47.6 | 47.7 → 48.1 | 53.3 → 54.0 |
| 100% of schedule | 75.6 → 75.9 | 43.0 → 44.4 | 46.6 → 47.9 | 47.8 → 48.0 | 53.3 → 54.1 |
Table D: Comparison of SEPO vs. GSPO across 3 random seeds (Qwen3-Omni-30B-A3B, Accuracy %, mean ± std).
| Method | Video-MME | Video-MME-v2 | OmniVideoBench | LVOmniBench | Avg. |
|---|---|---|---|---|---|
| GSPO | 75.3 ± 0.22 | 43.2 ± 0.37 | 46.5 ± 0.40 | 47.7 ± 0.48 | 53.2 ± 0.27 |
| SEPO | 75.9 ± 0.19 | 44.4 ± 0.30 | 47.9 ± 0.45 | 48.1 ± 0.35 | 54.1 ± 0.22 |
1. Gains across checkpoints and seeds. On the 4-benchmark average, SEPO leads GSPO at the 50%, 75%, and 100% checkpoints (+0.5, +0.6, +0.8). The advantage therefore does not depend on choosing a favorable checkpoint. 2. Process-level gains on multi-step reasoning. We audited all 1,000 evaluation trajectories on OmniVideoBench (3,577 human-annotated step-level evidence facts) using objective tool-log metrics and a blind observation audit (full protocol under Reviewer 82hi W2). Focused Hit ≤30s is the recall of annotated timestamps within ±2s by spans ≤30s; Error-to-Correct (+ Ans. Right) measures how often an early WRONG observation is revised to CORRECT in later turns (and answered right).
Table E: Process-level evidence acquisition and error recovery on OmniVideoBench (1,000 queries).
| Method | Avg. Calls | Invalid / No-Prog. Calls (%) ↓ | Avg. Span (s) ↓ | Focused Hit ≤30s (%) ↑ | Error-to-Correct (+ Ans. Right) (%) ↑ |
|---|---|---|---|---|---|
| SFT | 3.9 | 8.4% | 43.3s | 58.2% | 9.0% (5.5%) |
| GSPO | 5.4 | 15.1% | 46.0s | 61.3% | 12.0% (5.0%) |
| SEPO | 5.5 | 13.2% | 42.1s | 62.9% | 12.6% (7.1%) |
At comparable search depth to GSPO, SEPO retrieves tighter, more relevant evidence, makes fewer invalid or unproductive calls than GSPO , and more often recovers from early perceptual errors.
Response to Reviewer RhjB (1/3)
Thank you for recognizing the importance of cross-span evidence comparison and error recovery for audio-visual agents. We also appreciate your positive assessment of our empirical results and data-scaling analyses. Below, we provide detailed responses to the issues you raise.
W1: Details About the Evidence Scorer
1. Role of the initial overview. All 8 rollouts of a query receive the same initial overview (0.5 FPS frames and full audio; Appendix D.1), while tool calls retrieve higher-resolution frames or synchronized clips for local details missed by this overview. Because the initial overview is shared across the rollout group, SEPO scores only the incremental local evidence retrieved by tool calls (Appendix E.2). When the overview alone suffices for a one-turn answer, the turn weight stays wi,1=1 without penalty (Appendix E.3, Eq. (17)).
2. Outcome gating. The sign of the trajectory advantage Ai is set by the final answer reward, and the evidence score only rescales turn weights within [0.9,1.1] without changing the sign of Ai (Eq. (6); Appendix E.3, Eq. (17)). Retrieving a relevant interval while misinterpreting its content and answering incorrectly therefore retains a non-positive advantage.