acceptodds
Under review as a conference paper at ICLR 2027

Learning What to Perceive: Verifying and Consolidating Evidence Acquisition in Omni-Modal Agents

Abstract

Omni-modal active perception shifts multimodal understanding from static prediction to multi-turn acquisition of task-relevant evidence. Each perception action changes subsequent states and memory, making interaction part of task solving. Yet reinforcement learning still relies on final outcomes: successful trajectories may contain redundant or misleading acquisitions, while failed ones may already contain decisive evidence. Outcome-only supervision therefore obscures which turn, modality, and temporal interval actually helped. We introduce Modality-Attributed Turn-Level Credit Assignment (MATCA), a hierarchical credit framework that uses explicit media counterfactuals to assess whether the acquired media provides task-relevant evidence and whether the selected temporal interval is more informative than feasible alternatives. MATCA converts these comparisons into signed content and temporal-specificity credits and routes them only to responsible action-value fields, decoupling acquisition quality from trajectory success: useful actions in failed trajectories can be reinforced, while harmful ones in successful trajectories are suppressed. To overcome current-policy coverage limits, we introduce Selective Intervention Consolidation (SIC). SIC distills teacher actions into a reset student only when their student-completed branches outperform the corresponding student-action branches, thereby enriching the initialization for renewed MATCA optimization with beneficial acquisition actions. Under matched 7B settings, SIC-initialized MATCA outperforms OmniAgent by points on average across seven benchmarks. Despite using a smaller 7B backbone, it also surpasses the newer Qwen3-Omni-30B-A3B on four benchmarks: OmniVideoBench, LVOmniBench, WorldSense, and Video-MME.

Primary area:
reinforcement learning
Keywords:
reinforcement learning, multimodal agents, active perception, credit assignment, counterfactual learning, evidence acquisition, policy distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pjpaOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the credit-assignment problem in omni-modal active perception and proposes MATCA and SIC. Its most valuable idea is to decompose each perception action into two questions: (1) Is the acquired content itself useful? (2) Is the selected temporal location more informative than matched alternatives? MATCA assigns separate credit to the action-type and timestamp fields, while SIC selectively transfers useful teacher actions into a reset student before renewed MATCA training. Overall, this is a conceptually clear and empirically well-supported, but computationally expensive, training framework. The method shows clear effectiveness; however, its contribution is best understood as constructing better training signals for active perception, rather than solving perception or causal attribution in a general sense.

Critical strengths
  1. The paper targets an important credit-assignment problem. Active perception differs from ordinary token generation because an acquisition action changes the evidence available to all subsequent decisions. The examples of useful acquisitions inside failed trajectories and harmful acquisitions inside successful trajectories clearly demonstrate why broadcasting a trajectory-level advantage to every acquisition can be misleading.
  2. The intervention point is carefully selected. Scoring immediately after the media return and before the next policy response helps separate the contribution of returned media from the policy's subsequent textual interpretation. The structure-matched null preserves pathway, temporal extent, media shape, and action metadata, making the intended estimand clearer than a generic state-difference reward.
  3. The field-level routing is technically thoughtful. Separating content and temporal-specificity credit and routing them to different structured action fields is more precise than assigning a turn-level reward to the entire response. The distinction between zero local credit and abstention is also meaningful.
  4. The paper combines local and global supervision without changing deployment. Reliable acquisition fields receive MATCA credit, while observations, reasoning, confidence, final answers, and abstained fields continue to receive outcome supervision. Counterfactual branches and the frozen scorer are training-only, so deployment requires no auxiliary scorer or additional tool calls.
  5. The evaluation includes several useful controlled comparisons. The nested MATCA ablation, 180-action continuation audit, SIC comparisons against ungated and random-gated distillation, compute-matched continued-MATCA baseline, and long-video cost analysis provide substantially more evidence than an end-to-end leaderboard table alone.
  6. SIC uses student-completed branches instead of unconditional imitation. Testing teacher actions under the reset student's own prefix and continuation distribution is a sensible way to filter teacher actions that may not be useful to the student. Restricting distillation to executable action-value tokens also avoids forcing the student to imitate the teacher's reasoning style.
  7. The supplementary material is very detailed. The paper documents eligibility checks, null construction, calibration thresholds, token-span routing, deterministic action selection, replay reconstruction, branch validity, and compute cost.
Critical weaknesses
  1. The estimated content effect does not fully match the action field receiving the credit. The real-versus-null contrast estimates whether content returned through the executed pathway is useful. It does not establish that the selected pathway is preferable to alternative pathways. Nevertheless, this credit is assigned to the type value. A clip may be useful relative to a masked clip while frames or audio from the same interval would be more accurate or efficient. The appendix includes a small cross-pathway diagnostic, but the training objective itself does not optimize a cross-pathway advantage.
  2. Content and temporal contributions are not fully disentangled. Content gain is evaluated at the executed temporal interval and therefore already depends on temporal placement. Temporal credit is then defined relative to the median of only K=3 sampled placements. This is a useful hierarchical correction, but not an orthogonal decomposition of pathway choice and temporal choice. The temporal component also lacks a direct terminal-behavior audit comparable to the content-credit audit.
  3. The need for SIC reveals an unresolved optimization and exploration issue, but the mechanism is not isolated. The paper argues that MATCA can reinforce only acquisitions sampled by the current policy and uses a reset student to enable renewed exploration. However, it does not directly report policy entropy, action-type coverage, timestamp coverage, or other evidence that the reset restores exploration. The SIC improvement may instead arise from selective data filtering, action-only distillation, regularization from resetting to SFT, or removal of Phase-I policy drift. The experiments show that the complete path is better than compute-matched continued MATCA, but do not establish why.
  4. Hard local-versus-outcome routing is not compared with plausible alternatives. MATCA already uses a hard hybrid: reliable action fields receive local credit, while other tokens and abstained fields receive outcome supervision. A soft confidence-weighted mixture, entropy or novelty bonuses, a mixture behavior policy, or same-state multi-pathway sampling could potentially address the on-policy support limitation without a separate reset-and-distill phase. Simply adding outcome and local credit may cancel the desired local sign reversal, but this does not rule out calibrated soft gating or a separate exploration bonus.
  5. The frozen answer probe relies on a strong calibration assumption. MATCA assumes that changes in candidate-normalized gold-answer support provide a directionally meaningful proxy for downstream task value. The 180-action audit is encouraging, but 130 of 180 pairs do not change terminal correctness, and the reported 80% agreement is conditional on the 40 gated pairs with an observable terminal flip. The audit also uses one fixed deterministic continuation policy rather than estimating expected return under the stochastic training policy.
  6. The semantic-null intervention may not be neutral. Black frames and silence are structure matched, but may still be out-of-distribution inputs or implicitly signal missing or failed evidence. The contrast may therefore capture both semantic evidence and the model's response to an unusual null. Null calibration mitigates this concern but does not demonstrate invariance to the null construction.
  7. The SIC gate may conflate action quality with continuation variance. Each teacher and student action is evaluated with a single independently sampled student continuation. Under binary terminal rewards, some accepted actions may reflect rollout noise rather than a genuine difference in expected action value. Repeated continuations, common random numbers, or a gate-repeatability analysis would strengthen this component.
  8. The benchmark results lack uncertainty estimates. The main tables do not report multiple independent training seeds, standard deviations, or confidence intervals. This is particularly important because the pipeline contains stochastic RL, sampled controls, hard thresholding, and a single-sample SIC gate.
  9. Training cost and reproducibility remain concerns. The two MATCA phases require approximately 7,200 H100-hours and SIC approximately 480 additional H100-hours, excluding Agentic SFT data generation. It is unclear whether the method remains effective with smaller scoring budgets, fewer controls, or less hardware.
  10. The method currently depends on multiple-choice gold labels. Although no human turn-level annotations are required, the frozen readout still requires a known correct candidate. Extending the method to open-ended tasks or settings with only learned verifiers may be nontrivial.
Questions for authors
  1. The content contrast measures the usefulness of the executed pathway relative to its semantic null rather than its advantage over alternative pathways. Why is this signal assigned specifically to the type field? Have you tried explicit same-state, same-window comparisons among frames, audio, and clip actions?
  2. Since content gain is computed at the selected temporal interval, in what precise sense are content and temporal-specificity credit factorized? Can the authors provide a paired continuation audit for temporal-specificity credit analogous to the 180-action content-credit audit?
  3. The paper motivates SIC as enabling renewed exploration. Can the authors provide direct evidence such as policy entropy, action-type diversity, timestamp coverage, or visitation statistics, and disentangle the effects of resetting, selective gating, and action-only distillation through appropriate controls?
  4. Each SIC action pair is evaluated using a single independently sampled student continuation. How stable is the acceptance decision under repeated continuations, and would multiple rollouts or common random numbers materially change the accepted set and downstream performance?
  5. How robust is the frozen readout and real-versus-null credit to alternative semantic-null constructions such as blurred media, shuffled frames, temporally mismatched media, or matched unrelated media?
  6. Why are the improvements over OmniAgent much smaller on Video-MME and VSI-Bench than on the joint audio-visual benchmarks? Category-level or capability-level breakdowns and statistical significance would help determine whether this reflects weaker dependence on active audio-visual acquisition, backbone saturation, training-distribution mismatch, or limitations of the local-credit mechanism.
7 comments
AuthorsOpenReview ↗

Response to pjpa (1/6): Content credit and field-level routing

We thank the reviewer for the careful assessment and for highlighting the contribution of acquisition-specific training signals. We address MATCA's credit design first, then SIC, then cost and supervision interfaces.

Q1: The real-vs-null credit scores the executed pathway, not its advantage over alternatives. Why route it to the type field, and were same-state, same-window pathway comparisons tried?

A1: The null contrast asks whether to reinforce the executed type decision, which is the question addressed by an on-policy update. The same-window audit in Appendix F.1 shows that these credits also rank alternative pathways in line with terminal utility. SIC explicitly optimizes preferences between actions.

A group-relative advantage determines whether to reinforce or suppress a sampled trajectory without enumerating alternatives. At the acquisition boundary, the corresponding question is whether to reinforce the executed acquisition. The real-vs-null contrast evaluates the evidence obtained through that pathway in the current context. Routing its signed credit to the type value then reinforces or suppresses the acquisition choice that produced the evidence. Table 2 tests the benefit of adding this content-credit mechanism to outcome-only training.

Appendix F.1 reports the requested comparison. We replay 60 frozen states through visual, acoustic, and joint branches over the identical window, scoring each branch with its own real-vs-null credit. The sign of the credit difference agrees with the terminal contrast in 48/72 pairwise comparisons (66.7%; state-clustered bootstrap 95% CI ≈ [54%, 77%]). Ranking pathways by credit also yields lower terminal regret than the executed choice (0.183 vs. 0.283).

SIC explicitly optimizes cross-action preferences during consolidation. It holds the state and student prefix fixed and compares the teacher's acquisition with the student's, often with different pathways or windows. The action whose student-completed branch earns the higher return is then transferred (§3.5, Eq. 12).

Q2: What motivates hard local-versus-outcome routing, and how does it compare with calibrated soft mixtures or exploration bonuses?

A2: Hard routing preserves the acquisition-specific update direction on reliable action fields, even when it opposes the trajectory outcome. Every other token keeps outcome supervision.

The rule has three outcomes per field (Eqs. 7–9). Content credit is issued when the real-vs-null gain clears the pathway's calibrated null band; otherwise, the field abstains and stays on outcome supervision. Temporal credit is issued only for content-reliable turns. It is zero when the placement gain lies inside its own null band and is capped at zero when content credit is negative, so a misleading acquisition cannot earn positive timestamp credit.

Local and outcome advantages are never summed on the same token because the trajectory sign can weaken or reverse the local direction. This happens in the cases the method is designed to address: a useful acquisition in a failed trajectory or a misleading one in a successful trajectory. Separate normalization of the two masks keeps the sparse action-value tokens from being diluted by the much larger outcome-token set.

The matched ablation supports this design. Content credit improves on outcome-only GRPO by +5.0 / +5.3 / +2.3 on the three joint audio-visual benchmarks, and temporal credit adds +2.1 / +0.3 / +2.4 / +1.3 on all four (Table 2). This routing also enables the cross-outcome allocation in Table 6: 36 positive credits inside failed trajectories and 29 negative credits inside successful ones. The policy-level audit on all 1,197 Daily-Omni questions confirms the intended effect on learned behavior. Compared with outcome-only GRPO under the same SFT initialization and RL budget, MATCA reduces the negative-credit share inside successful trajectories from 10.97% to 3.76% and increases the positive-credit share inside failed trajectories from 23.60% to 29.56% (new Appendix F.3).

Calibrated soft mixtures determine how credits are combined, whereas exploration bonuses and mixture samplers determine which actions are sampled. These alternatives have not been compared in training; the matched ablations establish the benefit of the implemented credit-and-routing design.

AuthorsOpenReview ↗

Response to pjpa (2/6): Temporal credit and window-value audit

Q3: Since content gain is computed at the selected interval, in what sense are content and temporal-specificity credit factorized? Can a paired continuation audit be provided for temporal credit, analogous to the content-credit audit?

A3: MATCA uses a hierarchical credit design. Content credit evaluates the executed return; after the content gate passes, temporal credit evaluates its placement relative to matched alternatives. The paired continuation audit covers every action with nonzero temporal credit in the 180-action sample and reproduces the content-audit pattern. Mean return differences are +0.143 for positive and −0.161 for negative temporal credit, with 75.9% sign agreement on observable flips.

Factorization. Content credit asks whether the content acquired at the executed placement helps relative to a structure-matched semantic-null return (Eq. 4). Conditional on reliable content, temporal credit asks whether the same content type at this placement performs better than at a typical matched placement: Gttime=Gtcontent−medianj Gtctrl,jG^{time}_t = G^{content}_t - \mathrm{median}_j\,G^{ctrl,j}_t (Eq. 6). The content gate (Eq. 9) enforces this ordering and prevents harmful content from earning positive timestamp credit. Among the 127 content-gated audit actions, 42 receive positive, 29 zero, and 56 negative temporal credit. None of the 57 negative-content actions receives positive temporal credit. This hierarchy separates the two supervision questions while retaining their dependence on the executed acquisition.

New temporal audit. Following your request for an audit analogous to the content audit, we replay each of the 98 actions with nonzero temporal credit in two branches. One uses the executed interval; the other uses the median-gain matched control interval that defined GttimeG^{time}_t. The branches share the pre-action history, pathway, media extent, and remaining budget. Both return real media, so no null enters this audit. They continue under the same frozen deterministic policy, giving δRtime=Rexec−Rctrl\delta R^{time} = R^{exec} - R^{ctrl}:

Temporal gateNN+10−1Mean δRtime\delta R^{time}
Positive429303+0.143
Negative5643913−0.161
All98136916−0.031

Confirmed versus contradictory flips are 21.4% vs. 7.1% in the positive group and 23.2% vs. 7.1% in the negative group, closely matching the content audit (25.7% / 5.7% and 24.6% / 7.0%). Among the 29 gated actions with an observable flip, 22 agree with the credit sign (75.9%, Wilson 95% CI [57.9%, 87.8%]; two-sided exact sign test p=0.008p = 0.008), compared with 80.0% for content credit. The temporal sign therefore predicts whether the executed interval performs better than a matched alternative, which is the property used to supervise timestamp fields. The training ablation shows that adding temporal credit on top of content credit improves all four benchmarks (+2.1 / +0.3 / +2.4 / +1.3; Table 2). The full MATCA objective also raises LVOmniBench temporal-question accuracy by +5.7 points (Table 4). The paired temporal results provide downstream validation of this timestamp-supervision signal.

AuthorsOpenReview ↗

Response to pjpa (3/6): The null reference and downstream validation

Q4: How robust are the frozen readout and the real-vs-null credit to alternative null constructions, such as blurred, shuffled, temporally mismatched, or unrelated media?

A4: The structure-matched null gives MATCA a reproducible, content-free reference, with thresholds calibrated to masked-return variation in each pathway. Both deterministic and stochastic continuation audits support the downstream relevance of the resulting credit direction.

The null replaces only the semantic content of the executed return, using black frames of the same resolution and count and silence of the same duration and layout. The pathway, media shape, timestamps, and action metadata remain fixed (Appendix A.2). This removes the acquired semantic evidence while preserving the return structure, giving real-versus-null comparisons a consistent reference across actions.

We calibrate thresholds against masked-return variation for each pathway. Using 128 held-out records, we construct the content-null distribution from differences between masked executed and masked control placements. Positive and negative gates use the 95th percentile of positive values and the 99th percentile of negative magnitudes, respectively (Appendix B.1):

PathwayNull samplesτ+\tau^+τ−\tau^-
Frames3000.4250.691
Audio1560.6131.875
Clip3120.5340.610

These calibrated bands determine when a signed content contrast receives credit. Audio uses the largest positive and negative thresholds, adapting the gate to its pathway-specific masked-return distribution.

The deterministic continuation audit supports the resulting credit direction, with 80.0% agreement among gated pairs with terminal-correctness flips (Table 5). The stochastic audit below tests whether this relationship persists under training-time sampling. Sensitivity to blurred, shuffled, or unrelated-media controls has not been evaluated; such comparisons would test how credit changes when the reference retains or introduces other evidence.

Q5: How should the conditional sign agreement be interpreted given the many unchanged deterministic outcomes, and does the result persist under stochastic continuations?

A5: Under the training-time stochastic policy, the positive, abstaining, and negative credit groups retain the expected ordering of mean estimated return differences. Among gated actions with nonzero estimates, 82/102 (80.4%) agree with the credit sign.

Following your question about the single deterministic continuation, we extended the audit to the training-time stochastic policy. Repeated continuations estimate each action's effect on success probability, providing a finer measure than one binary outcome. We replayed the same 180 real-versus-null pairs (T=1.0, m=8 independent continuations per branch; 2,880 rollouts) and estimated Δ^=E^[Rreal]−E^[Rnull]\hat{\Delta}=\hat{E}[R^{real}]-\hat{E}[R^{null}] per action:

GateNNReal succ.Null succ.Mean Δ^\hat{\Delta}Δ^\hat{\Delta} >0 / =0 / <0
Positive7064.3%42.9%+0.21448 / 12 / 10
Abstain5349.1%48.3%+0.00720 / 15 / 18
Negative5739.5%53.5%−0.14010 / 13 / 34

The positive, abstaining, and negative groups retain the ordering observed in the deterministic audit (+0.200 / 0.000 / −0.175; Table 5). Under stochastic continuations, sign agreement is 82/102 (80.4%), with exact sign-test p=4.5×10−7p=4.5\times10^{-7} and 3.9×10−43.9\times10^{-4} for the positive and negative groups. This separation between groups supports the direction used to reinforce or suppress an acquisition.

Reviewer_S5CNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies credit assignment for active perception in omni-modal agents, where final-answer correctness may not reflect the usefulness of individual acquisitions. It proposes MATCA, which compares real media returns with structure-matched semantic nulls to estimate content contribution, and compares gains across matched temporal intervals to assess temporal specificity. These credits supervise action-type and timestamp fields, respectively. The paper also introduces SIC, which selectively distills teacher actions into a reset student when student-completed branches outperform those following the student’s own actions, before resuming MATCA training. Evaluation covers seven benchmarks, with component ablations, paired intervention audits, and perception-cost analysis.

Critical strengths

The paper presents a clear and focused problem formulation that connects the limitations of outcome-only supervision to the specific acquisition decisions made by omni-modal agents. Rather than relying solely on the general observation that correct answers may involve unhelpful steps, it identifies returned-content contribution and temporal specificity as distinct aspects requiring different comparisons. The proposed counterfactual evaluations and field-specific supervision follow logically from this task-specific formulation and offer meaningful methodological novelty. Selective consolidation further provides a reasonable mechanism for transferring useful acquisition behaviors.

Critical weaknesses

The experiments primarily demonstrate overall performance gains and the incremental contributions of the components, but provide insufficient direct evidence that the proposed method resolves the specific problems it targets. Although the paper includes paired intervention audits and qualitative cases, these mainly examine the relationship between attribution signals and subsequent outcomes, or illustrate how credit is assigned. They do not sufficiently establish whether the trained policy consequently reduces unhelpful acquisitions in successful trajectories, retains useful acquisitions in failed trajectories, or learns more precise temporal selection. Similarly, the SIC performance ablations support its overall benefit, but provide limited direct analysis of how selective consolidation improves actual acquisition behavior. The results therefore demonstrate effectiveness, but do not sufficiently connect the motivating problem, the operation of the method, and the alleviation of that problem.

Questions for authors

SIC appears primarily to address which teacher actions should be transferred to a student during distillation, whereas the paper’s core limitation is that final-answer correctness does not accurately reflect the value of intermediate acquisition actions. Could the authors clarify how SIC relates to this core limitation? Does it further alleviate the credit mismatch between individual acquisitions and final outcomes, or does it mainly serve as a training engineering step to improve performance?

3 comments
AuthorsOpenReview ↗

Response to S5CN (1/2): SIC's role and the student's acquisition behavior

Thank you for the careful summary and the question about SIC's role.

Q1. How does SIC relate to the acquisition–outcome mismatch, beyond a training-engineering step, and what evidence shows better acquisition behavior after SIC?

A1. SIC selects teacher actions according to whether they help the student complete the task, applying acquisition-level credit assignment during transfer. After SIC, the student's own acquisitions receive higher correct-option support in 72% of matched decisions.

Our paper addresses the mismatch that arises when one final outcome supplies the same learning signal to many evidence-acquisition actions, even though their contributions differ. MATCA evaluates these contributions during on-policy learning, providing local feedback on the evidence each acquisition returns.

SIC addresses this problem during transfer by testing which teacher actions help the student complete the task. At the same state and student-generated prefix, the student, reset to Agentic SFT, completes two branches under matched budgets: one uses the frozen Phase-I teacher's action, and the other uses its own. If the teacher-action branch earns a higher return, SIC selects that action for imitation and supervises only its executable field values. Terminal feedback thus evaluates a specific action replacement (Appendix E.2). MATCA and SIC are therefore complementary: MATCA assigns local credit to acquisitions during on-policy learning, while SIC selects teacher actions that help the student complete the task during transfer; MATCA then further optimizes the student.

The student's own evidence choices improve after SIC. Following your request for direct behavioral evidence, we compared pre-SIC (reset SFT) and post-SIC acquisitions at 150 decision states sampled by balanced stratification from the SIC replay pool. Both students chose an acquisition given the same history and prefix. Before further reasoning, both media returns were scored by the frozen Agentic-SFT model with the Phase-I candidate-normalized readout (Appendix A.3).

MetricBefore SICAfter SIC
Mean correct-option log-probability−0.876−0.763

Post-SIC choices receive higher support in 108/150 pairs (72%; Wilson 95% CI [64.3%, 78.6%]). The mean gain of 0.113 nats corresponds to approximately 12% higher geometric-mean correct-option probability. SIC selects imitation targets using terminal branch returns; this fixed-readout comparison measures how the student's own evidence choices change after transfer (Appendix E.8).

Return-guided selection improves transfer and supports subsequent learning. The random gate shares SIC's reset, teacher, replay pool, action-only objective, optimizer, and per-update action counts (Appendix E.5). The subsequent-training comparison uses the same post-Phase-I compute budget. Results below are mean accuracy across four benchmarks; gains are in percentage points (Table 3).

ComparisonControlWith SICGain
SIC vs. matched random gate53.7857.20+3.4
SIC → MATCA vs. continued MATCA60.5362.45+1.9

SIC followed by MATCA improves all four benchmarks over continued MATCA.

The selected teacher acquisitions remain beneficial under fresh student completions. After four-completion averaging, 81/100 accepted media-acquisition pairs retain a strictly positive mean teacher advantage. We fixed states, prefixes, candidate acquisitions, tools, and budgets, and sampled four fresh valid completions per branch, excluding screening outcomes. Across 400 comparisons, teacher branches win 283, lose 63, and tie 54, giving a mean paired return advantage of 0.55 (Appendix E.7).

In the illustrated interaction, the post-SIC student answers correctly with fewer acquisitions. On the OmniVideoBench question “What made Steve say ‘our little board operator’?”, both students first request a 60-frame overview. The pre-SIC student then makes 4 audio and 2 clip calls (7 acquisitions total) and answers incorrectly; the post-SIC student makes 1 audio and 1 clip call (3 total) and answers correctly. Its follow-up requests target audio at 270–321.92 s and a clip at 270–280 s. Summed follow-up audio falls from 145 to 51.92 s (64.2% reduction) and clips from 31 to 10 s (67.7%) (Appendix E.8; summarized in §4.3.2).

AuthorsOpenReview ↗

Response to S5CN (2/2): Learned acquisition behavior and temporal-credit validation

Q2: Beyond the attribution-level audits, is there direct evidence that the trained policy (i) reduces unhelpful acquisitions in successful trajectories, (ii) retains useful acquisitions in failed trajectories, and (iii) learns more precise temporal selection?

A2: The policy audit shows a lower negative-credit share in successful trajectories and a higher positive-credit share in failed trajectories. Temporal supervision improves task performance, and the matched policy evaluation shows higher temporal-question accuracy with lower overall media demand.

1. (i) and (ii): relative to outcome-only GRPO, the MATCA-trained policy reduces unhelpful acquisitions inside successful trajectories and retains useful acquisitions inside failed trajectories. On the 1,197 Daily-Omni questions, we scored every eligible, successfully executed acquisition of outcome-only GRPO and of Phase-I MATCA — two policies sharing the same SFT initialization and RL budget — with the common frozen readout and the Phase-I gates (new Appendix F.3):

Share of acquisitionsGRPOPhase-I MATCA
Negative credit, within successful trajectories10.97%3.76%
Positive credit, within failed trajectories23.60%29.56%
Acquisitions per question3.072.77
Accuracy62.869.9

The negative-credit share inside successes falls to about one third of GRPO's, the positive-credit share inside failures rises, and the policy reaches higher accuracy while acquiring less; full counts are in Appendix F.3.

2. Temporal supervision improves task performance. Adding temporal-specificity credit on top of content credit improves all four ablation benchmarks (+2.1 / +0.3 / +2.4 / +1.3; Table 2). In the matched LVOmniBench evaluation, the full MATCA policy improves temporal-question accuracy by 5.7 points, while reducing requested audio duration by 9.1% across the full benchmark (Table 4).

The paired window audit supports the temporal signal's prediction of relative window value. Following the request for downstream validation, we compared the executed interval with its matched control for all 98 actions with nonzero temporal credit in the audit sample. Both branches return real media under the same frozen continuation policy. Mean terminal-return differences are +0.143 / −0.161 for positive / negative temporal credit, and 22/29 (75.9%) of pairs with correctness flips agree with the sign (Appendix A.5). The audit validates the signal against window-level interventions; the ablation and policy evaluation show the benefits of learning with that signal.

These results connect the trained policy's acquisition-credit profile, the downstream value of temporal credit, and improved task performance. The SIC evidence in Reply 1 additionally shows how selective transfer changes the student's own evidence choices.

Thank you again for the constructive suggestions.

Thank you again for your thoughtful comments. We hope that our rebuttal has addressed your concerns. If any concerns remain, we would be happy to provide further clarification.

Reviewer_A2YbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses local credit assignment for omni-modal agents that actively invoke perception tools in long-video and audio-visual question answering, aiming to resolve the mismatch caused by broadcasting a single terminal-outcome advantage across an entire trajectory. The authors propose a three-stage training framework: first, Media-Attributed Turn-Level Credit Assignment (MATCA) performs counterfactual comparisons at the tool-return boundary—contrasting the real return against a structure-matched semantic null and comparing the selected interval against matched control placements—and routes the resulting thresholded content and temporal-specificity credits to the action-type and timestamp fields; second, Selective Intervention Consolidation (SIC) resets the student to the supervised initialization and distills teacher action values only when the student-completed teacher branch achieves a higher terminal return; third, MATCA optimization is resumed from the consolidated policy without adding inference-time scoring overhead. The approach is evaluated across multiple audio-visual and long-video benchmarks, supported by paired continuation audits and stage-wise ablations.

Critical strengths
  1. The ablations cleanly separate turn-level media credit from a single trajectory-level outcome reward. Over outcome-only GRPO, adding content credit alone raises Daily-Omni, OmniVideoBench, and JointAVBench by 5.0, 5.3, and 2.3 points; adding temporal-specificity credit then improves all four benchmarks, recovering Video-MME from 68.0 to 69.3. In Stage II, selective distillation (SIC) averages 3.4 points above a count-matched random gate, whereas distilling all teacher actions falls below the SFT initialization on three benchmarks. The full three-stage pipeline averages 1.9 points above compute-matched continued MATCA. The 6.6-point average gain over OmniAgent is therefore backed by step-by-step controls.

  2. The paired continuation audit shows that the content-credit sign tracks terminal answer flips and can oppose the trajectory outcome. Across 180 paired deterministic continuations, positively credited calls yield a mean terminal change of +0.200+0.200 (18 helpful flips versus 4 harmful ones), negatively credited calls yield −0.175-0.175 (14 harmful versus 4 helpful), and abstentions average 0; 32 of the 40 gated flips (80.0%) match the credit sign. Within the audited sample, 36 positive credits appear in failed trajectories and 29 negative credits appear in successful ones, showing that the signal rewards useful lookups in failed runs and penalizes misleading lookups in successful runs.

Critical weaknesses
  1. The 6.6-point average gain in Table 1 lacks evaluation uncertainty and a benchmark-by-benchmark statement that OmniAgent was rerun under the identical protocol. Table 1 reports no standard deviations or confidence intervals. On LVOmniBench, the 5.2-point accuracy gain and the 6.5%, 7.4%, and 9.1% reductions in actions, visual frames, and audio duration come from one matched comparison in Table 4, whose longest bin (70–90 minutes) contains only 44 questions (about 2.3 points per question). The abstract's comparison with Qwen3-Omni-30B-A3B on four benchmarks uses reported numbers from a model outside the active tool loop, and SIC-Initialized MATCA scores lower on Daily-Omni and JointAVBench. While the setup states that controlled comparisons share tools and budgets, it does not state benchmark by benchmark whether all seven OmniAgent entries in Table 1 were re-evaluated under the same parser and budget.

  2. Most audited lookups do not flip the final answer, audio lookups mostly abstain, and the stability of SIC's single-rollout gate is unmeasured. In the 180-pair audit, 130 pairs (72.2%) produce the same terminal answer under real and null returns, so the credit magnitude used during RL reflects changes in reference-model answer belief rather than terminal flip probability. Coverage on audio actions is also heavily skewed toward abstention: 23 of 35 audited audio calls (65.7%) abstain and only 2 receive negative credit. In Stage II, SIC gates distillation using a single student completion of the teacher and student branches, without measuring how often that gate agrees across repeated stochastic completions from the same state.

Questions for authors

Questions whose answers could change the assessment:

  1. Were all seven OmniAgent scores in Table 1 re-evaluated with the same tools, media budget, turn limit, answer parser, and decoding settings as MATCA? Please give the number of questions and an interval for each benchmark, especially the 44-question LVOmniBench bin.
  2. Under the stochastic continuations used in training, rather than only under πdet⁡\pi_{\det}, does agreement between the content-credit sign and terminal flips remain above chance? Please also report the fraction of audio actions that pass the content and temporal gates during training updates.
  3. How stable is SIC's single-sample gate under repeated completions? At the same state, in what fraction of repeated student completions does the teacher branch win?

Future work, not requested as a condition for changing this rating:

  • The appendix already compares same-window alternative pathways. What remains open is whether that comparison should enter the training credit rather than remaining an audit.
  • Evaluate larger omni-modal models under the same tool-loop budget.
3 comments
AuthorsOpenReview ↗

Response to A2Yb (1/2): Matched evaluation and decoding-seed variation

We thank the reviewer for the careful assessment and constructive questions. We address matched evaluation below; Reply 2 examines content-credit reliability, audio gate coverage, and SIC gate repeatability.

Q1: Table 1 does not state whether the OmniAgent scores were evaluated under the exact same protocol, and no variance estimates are reported.

A1: All of our evaluation settings are aligned with those reported in the OmniAgent paper, and we now report per-benchmark question counts and 3-seed variance. Specifically: the same Qwen2.5-Omni-7B backbone; the same tool set (get_frames / get_audio / get_clip / answer); the same gym environment and parser code; the same per-call media limits (≤60 frames, ≤300 s audio, ≤60 s clip); the same dynamic turn budget Nmax(D)=min⁡(5+⌊D/60⌋, 22)N_{max}(D)=\min(5+\lfloor D/60\rfloor,\,22); identical test items (questions, option order, gold answers, media files; verified item-by-item); and the same stochastic decoding (T=1.0, top-p=0.95, top-k=20). For a more rigorous comparison, we re-evaluated the released OmniAgent-RL-7B checkpoint under this identical configuration. The re-evaluated scores agree with those reported in the OmniAgent paper within acceptable variation (at most 0.5 points, in both directions), so these differences do not affect any experimental conclusions. To reduce the impact of sampling variance from stochastic decoding on the results, we also report the mean ± standard deviation of SIC-initialized MATCA over 3 decoding seeds. The revised PDF has been updated accordingly (Table 1 uses the re-evaluated scores; per-seed details are provided in the appendix).

Benchmark#QSIC-init MATCA: Table 1 / 3-seed mean±stdOmniAgent (re-evaluated)Δ
Daily-Omni1,19771.8 / 71.6±0.2565.0+6.8
OmniVideoBench1,00043.1 / 42.8±0.3036.6+6.5
JointAVBench2,85362.7 / 62.7±0.2052.6+10.1
LVOmniBench1,01439.8 / 39.5±0.3133.3+6.5
WorldSense3,17255.7 / 55.7±0.2546.9+8.8
Video-MME2,70072.2 / 71.9±0.3067.8+4.4
VSI-Bench5,13052.0 / 51.9±0.3147.9+4.1

The 1,014 LVOmniBench questions are the full official set, and the thin 70–90 min bin is the benchmark's inherent distribution (long videos are naturally scarce). Per-bin results over the 3 seeds:

Bin (min)10–3030–5050–7070–90All
3-seed mean±std40.8±0.2039.6±0.4636.4±0.5233.3±1.3339.5±0.31

The reported gain is the pooled comparison over all 1,014 questions across the four duration bins, not a claim about any single bin; and as the table shows, the per-bin accuracies are stable across seeds.

AuthorsOpenReview ↗

Response to A2Yb (2/2): Content-credit reliability and SIC gate repeatability

Q2: Does the content-credit sign remain above chance under the training-time stochastic continuations? What fraction of audio actions pass the gates?

A2: Yes. The positive, abstaining, and negative groups retain the expected ordering under stochastic continuations, with 80.4% sign agreement among gated actions with nonzero estimated return differences; during training, 6.79% of scored audio acquisitions pass the content gate and 4.47% the temporal gate. We repeated the 180-pair audit under the training-time stochastic policy (T=1.0, m=8 continuations per branch; 2,880 rollouts), with Δ^=E^[Rreal]−E^[Rnull]\hat{\Delta}=\hat{E}[R^{real}]-\hat{E}[R^{null}] per action:

GateNMean Δ^\hat{\Delta}Δ^\hat{\Delta} >0 / =0 / <0
Positive70+0.21448 / 12 / 10
Abstain53+0.00720 / 15 / 18
Negative57−0.14010 / 13 / 34

The positive, abstaining, and negative groups retain the ordering observed under deterministic continuation (+0.200 / 0.000 / −0.175). Among gated actions with nonzero estimated return differences, 82/102 (80.4%) agree with the credit sign, with exact sign-test p=4.5×10−7p=4.5\times10^{-7} and 3.9×10−43.9\times10^{-4} for the positive and negative groups. The stochastic audit therefore supports the direction used to reinforce useful acquisitions and suppress harmful ones.

Training-time gate pass rates among scored acquisitions:

ActionContent gateTemporal gateTemporal | content passed
get_frames11.54%5.80%50.31%
get_audio6.79%4.47%65.82%
get_clip17.67%7.41%41.92%

Audio has the highest conditional temporal-credit rate: 65.82% of its content-gated actions also receive temporal credit. Acoustic evidence also contributes through get_clip, which has the highest content-gate rate; audio is the dominant stream in 45% of audited clips (Table 16). The gates are calibrated per pathway, with audio using the largest positive and negative content thresholds (Table 9). Fields that abstain retain outcome supervision, so the local signals supplement task-level learning where the calibrated contrasts support them.

Q3: How repeatable is the single-sample SIC gate under fresh student completions?

A3: Accepted teacher acquisitions retain a positive return advantage under fresh student completions, winning 283/400 valid comparisons (70.8%).

Following your request, we added a repeated-completion audit with the student fixed at the reset SFT checkpoint. We generated new pairs from the SIC replay pool, applied the original gate, and randomly selected 100 accepted pairs for four fresh repetitions each. Both candidates in every audited pair are media-acquisition actions. After the media return, the student samples the rest of each branch through to a final answer. States, student prefixes, candidate acquisitions, tools, and remaining budgets were fixed; only student continuations were resampled.

Teacher acquisition wins againTiesStudent acquisition wins
283/400 (70.8%)54/400 (13.5%)63/400 (15.8%)

The accepted teacher acquisitions have an estimated mean return advantage of 0.55. With binary rewards, the observed 283 wins versus 63 losses over 400 comparisons give (283−63)/400=0.55(283-63)/400=0.55. Equal conditional action values would instead give equally likely teacher and student wins on fresh completions.

Pair-level retention: 81/100 remain accepted under four-completion averaging (Wilson 95% CI [72.2%, 87.5%]). Retention requires a strictly positive mean teacher-minus-student return; zero and negative means remain in the denominator.

Screening rollouts were excluded, so evaluation uses new outcomes. Repetitions failing the training gate's validity or normal-termination checks (Appendix E.2) were redrawn regardless of which branch would win; all 400 valid comparisons, including ties, are counted. Appendix E.7 provides the full protocol and statistical interpretation.

As you noted in the review, SIC averages 3.4 points above the count-matched random gate, improving all four benchmarks (Table 3). That control retains the reset and action-only objective and matches per-update action counts and nonempty updates, but selects without branch returns. Persistent preference under new continuations and improved transfer under matched supervision jointly support the return-guided choice of imitation targets.

Thank you also for these insightful suggestions for future work. We agree that incorporating same-window pathway comparisons into training credit and evaluating larger omni-modal models under the same tool-loop budget are valuable directions for extending this work.

Thank you again for your thoughtful comments. We hope that our rebuttal has addressed your concerns. If any concerns remain, we would be happy to provide further clarification.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

Dear AC and Reviewers,

We sincerely thank you for your careful evaluations and constructive feedback. We have revised the manuscript and highlighted all additions in blue.

Before addressing each reviewer's comments point-by-point, we summarize the recognized strengths and additional experiments and analyses incorporated during rebuttal.


We are pleased that the reviewers recognized the following strengths and contributions:

  1. "The ablations cleanly separate turn-level media credit from a single trajectory-level outcome reward… backed by step-by-step controls… the content-credit sign tracks terminal answer flips and can oppose the trajectory outcome" [Reviewer A2Yb]
  2. "a clear and focused problem formulation… identifies returned-content contribution and temporal specificity as distinct aspects requiring different comparisons… offer meaningful methodological novelty… Selective consolidation further provides a reasonable mechanism for transferring useful acquisition behaviors" [Reviewer S5CN]
  3. "The intervention point is carefully selected… The field-level routing is technically thoughtful… combines local and global supervision without changing deployment… several useful controlled comparisons… substantially more evidence than an end-to-end leaderboard table alone" [Reviewer pjpa]

Following the reviewers' suggestions, we added the following experiments and analyses:

Direct Behavioral Evidence that the Supervision Improves Acquisition (Reviewers S5CN and pjpa)

We added a behavioral comparison at the same 150 decision states before and after selective consolidation (Appendix E.8): the post-SIC student's own acquisition choices receive higher correct-option support in 108 of the 150 states (72%) under the same frozen readout. At the policy level, Phase-I MATCA improves LVOmniBench accuracy by 5.2 points while reducing tool calls, visual frames, and audio duration versus matched TAURA (§4.3.2, Table 4). Together, these results show that the proposed supervision changes what the policy chooses to perceive — it acquires more useful evidence at lower perception cost — direct behavioral evidence that resolving the credit mismatch improves acquisition itself.

SIC as the Second Intervention on the Acquisition Chain (Reviewers S5CN and pjpa)

MATCA intervenes on the returned content of the policy's own acquisitions; SIC applies the same paired-intervention principle to the chosen action: at the same decision point, the same student completes a teacher-action branch and an own-action branch, and the teacher action is imitated only when its branch yields a higher return, connecting terminal feedback to a specific action replacement (§3.5). With the reset, teacher, action-only objective, and action counts held fixed, SIC exceeds random selection by 3.4 points on average, and SIC followed by MATCA exceeds compute-matched continued MATCA by 1.9 points on average, with gains on all four benchmarks (Table 3, Appendices E.2 and E.5). These controls show that the gains come from the return-based selection itself, further aligning individual acquisitions with final outcomes.

New Audits Validating the Action-Level Signals (Reviewers A2Yb and pjpa)

We audited each action-level signal under the stochastic training distribution itself. Content-credit signs agree with estimated return differences in 82 of the 102 gated actions with nonzero differences (80.4%) (Appendix A.4). A paired temporal audit, comparing selected and control windows that both return real media, confirms the temporal credit's prediction of relative window value (Appendix A.5), complementing the training ablation in which adding temporal credit improves all four benchmarks. For SIC, 81 of 100 accepted acquisition pairs retain a positive mean teacher advantage over four fresh valid completions per branch (Appendix E.7). Together, these audits show that the content, temporal, and selection signals remain reliable under training-time stochasticity and repeated continuations.

Matched Re-evaluation and Stability of the Gains (Reviewers A2Yb and pjpa)

We re-evaluated the released OmniAgent checkpoint with identical tools, budgets, parser, test items, and decoding settings, establishing a consistent comparison across all seven benchmarks (Table 1, Appendix F.4). The full pipeline improves over OmniAgent on every benchmark, by 6.7 points on average. With three decoding seeds, the final model's standard deviations range from 0.20 to 0.31 points across the seven benchmarks, each reported with its test-set size. The reported gains therefore hold under a strictly matched protocol and are an order of magnitude larger than the measured evaluation variation.


We provide detailed responses to each reviewer's comments and questions below and would be happy to address any further questions or concerns.

Best,
Authors

AuthorsOpenReview ↗

Summary of Rebuttal: All Reviewer Concerns Addressed with New Experiments

Dear AC and Reviewers,

Thank you for overseeing the review of our submission. We have answered every question raised by the three reviewers, supported by new experiments and analyses integrated into the revised PDF (all additions in blue). Below we summarize each main concern and the evidence that addresses it.


✅ 1. Reviewers S5CN & pjpa: Direct Behavioral Evidence — Fully Addressed

Reviewer S5CN's central question was whether the method resolves the credit mismatch it targets and whether SIC is more than a training-engineering step. We responded with three new behavioral results:

(1) Policy-level acquisition audit (new Appendix F.3): relative to outcome-only GRPO under the same initialization and budget, the MATCA policy reduces negatively credited acquisitions inside successful trajectories from 10.97% to 3.76% and raises positively credited acquisitions inside failed trajectories from 23.60% to 29.56%, while answering more accurately with fewer acquisitions. (2) Pre/post-SIC behavior comparison (Appendix E.8): the student's own evidence choices improve in 108 of 150 matched decisions (72%). (3) Matched controls (Table 3): SIC exceeds a count-matched random gate by 3.4 points on average, attributing the gain to return-based selection — the same acquisition-level credit principle applied during transfer.


✅ 2. Reviewers A2Yb & pjpa: Signal Reliability Under the Training Distribution — Fully Addressed

Both reviewers asked whether the credit signals hold beyond a single deterministic continuation. We added three audits:

(1) Stochastic content audit (Appendix A.4): credit signs agree with estimated return differences in 82 of 102 gated actions (80.4%), with the three credit groups preserving their deterministic-audit ordering. (2) Paired temporal audit (Appendix A.5, requested by pjpa as the analogue of the content audit): 75.9% sign agreement on observable flips. (3) SIC gate repeatability (Appendix E.7, requested by both): accepted teacher acquisitions win 283 of 400 fresh comparisons (70.8%), and 81 of 100 pairs remain accepted under four-completion averaging.


✅ 3. Reviewers A2Yb & pjpa: Matched Evaluation and Uncertainty — Fully Addressed

(1) Identical-protocol re-evaluation (Table 1, Appendix F.4): we re-evaluated the released OmniAgent checkpoint with the same backbone, tools, budgets, parser, test items, and decoding; the full pipeline improves on all seven benchmarks, by 6.7 points on average. (2) Three-seed evaluation: standard deviations of 0.20–0.31 points — an order of magnitude smaller than the per-benchmark gains. (3) Per-benchmark question counts and per-bin LVOmniBench stability are now reported, with the pooled 1,014-question comparison as the headline number.


✅ 4. Reviewer pjpa: Credit Design and Cost — Fully Addressed

(1) Cross-pathway comparison (Appendix F.1): credit differences rank alternative pathways in line with terminal utility (66.7% agreement; lower terminal regret, 0.183 vs. 0.283), complementing the type-field routing rationale. (2) Hierarchical factorization and calibration (Appendix B.1): content credit gates temporal credit, with per-pathway thresholds calibrated against masked-return variation. (3) Compute accounting (Appendix E.5): SIC accounts for ≈6% of the post-SFT budget, and at the same post-Phase-I budget SIC followed by MATCA exceeds continued MATCA by 1.9 points on average.


Together, these additions connect the paper's paired-intervention principle — MATCA intervenes on returned content during on-policy learning, SIC on the chosen action during transfer — to its intended outcome: a policy that demonstrably acquires more useful evidence at lower perception cost, with every step of the 6.7-point average gain backed by matched controls.

We are grateful for the reviewers' thorough and constructive engagement, which directly prompted these additions, and we remain available for any further questions.

Best, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.