- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Dynamic Reasoning Routing (DRR), a training-free framework for online reasoning video object segmentation. It uses motion-compensated residuals to adaptively trigger semantic reassessment and confidence-weighted mask encoding to reduce error accumulation. Experiments on five benchmarks demonstrate an improved trade-off between segmentation accuracy and computational efficiency.
Critical strengths
-
Experiments on five benchmarks demonstrate competitive performance against state-of-the-art methods, with substantial efficiency gains.
-
The paper is well written and clearly organized.
Critical weaknesses
-
Motion-compensated residuals may respond to illumination changes, deformation, or occlusion that do not alter the queried target, while missing subtle interactions or relevant events outside the monitored neighborhood. Since the monitored regions depend on the predicted mask, incorrect tracking may also compromise subsequent invocation decisions. The aggregate results are encouraging, but the reliability and failure conditions of this proxy need further analysis.
-
The contribution of the scheduling policy is not fully separated from changes to memory management. Section 3.3 states that the baseline resets the propagation state when accepting new grounding, whereas DRR retains earlier references. However, Table 3 does not explicitly isolate this change from residual-guided scheduling and confidence-weighted encoding. Consequently, it remains unclear how much of the accuracy improvement comes from better invocation timing versus retaining historical information.
-
The confidence-weighted memory mechanism relies on an insufficiently justified confidence proxy. The selector’s Yes/No probability measures its confidence that a frame is suitable for grounding, which does not necessarily indicate the accuracy of the mask produced by the grounding module. Its additional gain in Table 3 is modest, from 56.7 to 57.1, and fixed weights achieve similar results in Table 4. Moreover, the target-absent analysis still shows substantial degradation after reassessment, suggesting that the mechanism provides limited protection against unreliable updates.
Questions for authors
-
How does residual-guided scheduling behave under irrelevant appearance changes, subtle semantic events, and incorrect target tracking? Could the authors report missed reassessments and the delay between relevant evidence appearing and successful target correction?
-
Could the authors compare fixed, random, and adaptive scheduling under identical memory-management settings, and separately ablate resetting versus retaining historical memory?
-
At comparable invocation budgets, how does the proposed residual signal compare with simpler alternatives, such as direct frame differences, residuals without motion compensation, and global residual aggregation?
-
Given the substantial negative reassessment benefit when the target is absent, how does DRR handle target disappearance and reappearance? Could the authors report false-positive predictions during absence and recovery delays after reappearance?
-
Please include the component-wise time cost and memory usage to support the contribution of each submodule.
4 comments
Re: Simpler visual cues (Q3). We thank the reviewer for suggesting these comparisons. DRR achieves 56.7 J&F on ReasonVOS at the same 5.2% selector-call rate, outperforming uncompensated frame differences over the same target and context regions by 1.5 J&F, whole-frame motion-compensated residuals by 2.2 J&F, and direct whole-frame differences by 2.4 J&F. These comparisons show benefits from both motion compensation and regional aggregation; see Re: Frame differences (Q2), Reviewer KAx1.
| Signal | Region | J&F | J&F, subset |
|---|---|---|---|
| Random | — | 54.6 | 52.9 |
| Frame difference | Whole frame | 54.3 | 52.4 |
| Motion-compensated residual | Whole frame | 54.5 | 53.1 |
| Frame difference | Target + context | 55.2 | 54.0 |
| DRR | Target + context | 56.7 | 55.7 |
All variants share anchor, memory, bootstrap and fallback policies and omit confidence weighting. To examine clips most affected by alignment, we select the third with the largest reduction in pixel mismatch with the grounding anchor as the subset. DRR's gains are larger on these clips, at 3.3 J&F over direct whole-frame differences and 1.7 J&F over uncompensated target–context differences. We have expanded App. G.4 of the revised manuscript to include these comparisons.
Re: Disappearance and reappearance (Q4). DRR reduces false positives during absence and recovers faster after reappearance. The reasons are three fold. First, the target-absent loss in Fig. 7 measures local update damage with reassessment blocked for 16 frames. Allowing online reassessment reduces weighted loss from 12.7 to 4.6 IoU over the same horizon; see Re: Confidence weighting (W3) above. Second, EgoMask distinguishes target-present accuracy from performance including absence. On its medium and long subsets, DRR-LLaVA-1.5-7B has lower IoU_gold and T_recall but higher IoU_all than CoT-RVS, consistent with fewer spurious foreground predictions during absence. App. G.3 and Fig. 9 report these results. Third, the direct ReasonVOS comparison below uses retained references and comparable invocation budgets.
| Schedule | Absence false positives % | Median recovery frames | Recovered within 16 frames % |
|---|---|---|---|
| Fixed period | 74.6 | 19 | 33.8 |
| DRR | 51.8 | 8 | 59.4 |
As shown, DRR lowers false positives from 74.6% to 51.8%, reduces median recovery delay from 19 to 8 frames, and raises recovery within 16 frames from 33.8% to 59.4%. Mechanistically, foreground or context changes can request reassessment, while an empty mask requests fallback evaluation subject to the minimum gap; these mechanisms provide further correction opportunities. App. H.1 reports this analysis.
Re: Time and memory per component (Q5). Residual computation and confidence-weighted encoding together require only 2.6 ms and approximately 0.43 GFLOPs per frame, with no additional model weights. The selector and reasoning segmenter run on 5.2% and 4.3% of frames, yet account for 47% of total component runtime and 63% of FLOPs. We measure the following costs on ReasonVOS using one H200, with GPU synchronization.
| Component | ms per frame | GFLOPs per frame | Weights MiB | Peak activations MiB |
|---|---|---|---|---|
| Residual cues | 2.1 | 0.003 | 0 | 0 |
| Selector | 19.8 | 510 | 13,472 | 1,120 |
| Reasoning segmenter | 49.5 | 925 | 16,672 | 2,350 |
| Weighted encoding | 0.5 | 0.43 | 0 | 20 |
| Propagation | 76.0 | 850 | 856 | 1,640 |
Component costs total 147.9 ms, plus 1.1 ms framework overhead, giving 149 ms per frame and 6.7 FPS. Encoding includes the full mask-encoding stage for an accepted grounding. Peak GPU memory is 33,400 MiB, comprising 31,000 MiB of weights and approximately 2,400 MiB of activations and other runtime memory; component activation peaks are not additive. The two reasoning models dominate parameter memory, and reducing their invocation frequency provides the throughput gain. App. G.5 reports the full breakdown, with a summary in Sec. 4.4.
reply to authors
I thank for the authors' efforts to provide the detailed explanation and additional experiments. Most of my concerns are well addressed. This paper provided a well-engineered pipeline with convincing results. However, this also raises another concerns about the technical contributions, since the proposed framework is composed with existing tools. I hope the authors can provide more explanations.
Thank you for your response. We respectfully note that building on established components does not preclude a methodological contribution, and prior works [R1, R2] likewise build on the same components. Deciding when to invoke costly reasoning models is an active research problem in streaming video understanding, with recent works [R3, R4] using learned decision mechanisms.
Our main contribution is to establish motion-compensated residuals, referenced to semantic grounding, as a training-free signal for allocating costly reasoning. The substantive finding is that these residuals identify opportunities for useful reassessment beyond what can be inferred from visual change alone. At the same invocation rate, motion compensation improves ReasonVOS J&F by 1.5 over uncompensated differences in the same target and context regions. On a common pool of saved states, DRR-selected reassessments yield a mean downstream IoU gain of 1.6, compared with 0.6 for frame-difference selection. Grounding refreshes the monitoring reference, closing this loop without a learned scheduling model. The policy’s transfer to language-specified tracking further supports its value as a reusable computation-allocation mechanism; see Re: Methodological contribution (W1), Reviewer KAx1. We welcome clarification on any remaining concerns.
[R1] “Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation,” ICLR 2026.
[R2] “Towards Streaming Referring Video Segmentation via Large Language Model,” CVPR 2026.
[R3] “Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction,” CVPR 2025.
[R4] “StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant,” NeurIPS 2025.
Thank you for your review. Our responses follow below.
Re: Residual reliability and recovery (W1, Q1). We agree that residual cues may respond to irrelevant appearance changes or miss relevant events. DRR includes safeguards for these failure modes through selector verification and fallback evaluation. First, appearance changes can trigger evaluation, but a No verdict leaves the propagated mask and memory unchanged, costing one 380 ms selector call. Normalization and target–context contrast suppress shared changes; the selector declines 13.8% of residual-triggered evaluations. Second, events outside the monitored neighborhood may not trigger evaluation, but the 32-frame maximum gap forces one and supplies the first evaluation in 19% of DRR’s failure episodes. Third, tracking errors can shift the monitored regions. An empty mask requests evaluation at the next frame permitted by the minimum gap; a wrong non-empty track remains subject to the maximum gap.
On ReasonVOS, DRR recovers 78.5% of tracking-failure episodes versus 71.6% for fixed-period scheduling. Mean recovery delay falls from 7.6 to 5.1 frames among recovered episodes, and the share receiving no evaluation falls from 45.9% to 31.2%.
An episode starts when target-present IoU falls below 0.5 after reaching that threshold and ends at recovery or disappearance. Each schedule generates its own episodes. These results measure tracking recovery, not delay from an independently annotated semantic event; the 19% figure measures fallback evaluation coverage, not successful correction. App. H.1 reports the analysis.
Re: Scheduling and memory (W2, Q2). With shared memory retention and comparable selector-call rates, DRR improves J&F by 5.5 over fixed-period scheduling and 2.1 over random scheduling, without confidence weighting. All results below use LLaVA-1.5-7B on ReasonVOS.
Keeping each fixed-period schedule unchanged, memory retention adds 0.8 J&F at ξ=4 and 1.1 at ξ=20. With retention shared and comparable call budgets, DRR still reaches 56.7 J&F versus 51.2 for fixed-period scheduling. Memory retention alone therefore does not explain DRR’s advantage. The original Table 3 retains memory for random and adaptive variants and preserves CoT-RVS’s published reset configuration. We have added fixed-period retention baselines to the revised Table 3 and clarified each variant’s memory policy in Sec. 4.4.
Re: Confidence weighting (W3). The reviewer highlights an important distinction between frame suitability and mask accuracy. Selector confidence measures frame suitability, and our results show that it is also informative about the accuracy of the resulting mask. Across confidence bins on ReasonVOS, mean grounding IoU rises from 0.41 to 0.78, while the share of target-absent updates falls from 24.4% to 3.3%. These trends support using selector confidence to control a grounding’s influence on memory.
Further, confidence weighting improves segmentation by selectively attenuating low-confidence updates. It achieves 57.1 J&F, compared with 56.7 without weighting and 56.4 with a fixed weight of 0.5. The stratified replay shows why weighting by confidence helps. Low-confidence target-absent updates are more damaging before weighting, with a mean IoU loss of 15.8 versus 12.1 for high-confidence updates. Weighting attenuates the low-confidence group and reduces its loss to 13.3 IoU, a 16% relative reduction.
Importantly, the reported negative reassessment in Fig.7 reflects a local update effect measured by paired replay. It measures mean 16-frame IoU loss relative to skipping the update from the same saved state, with further reassessment blocked. We also report an online comparison with both branches continuing normal reassessment. The weighted loss is 4.6 IoU online versus 12.7 in replay; online loss without weighting is 7.3. Together, selective memory weighting and subsequent reassessment mitigate the downstream impact of incorrect groundings; see Re: Harmful updates on target-absent frames (W2, Q3), Reviewer oFGS. Sec. 3.3 in the revision clarifies confidence-weighted encoding, and App. E.1 reports these analyses.