- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies coherent scene reconstruction for video object removal. TIE represents the source video, object mask, and noisy target as separate token streams. Its directed attention keeps the noisy target from updating the condition streams. During training, Evidence-Guided Supervision uses paired edited targets and frozen cross-frame matching to estimate where the source provides valid reconstruction evidence, then allocates point-wise and region-wise auxiliary supervision. At inference, the method uses only the source video, mask, and text. The paper evaluates TIE on PROVE-M and VORD-L, component ablations, and a 40-participant user study.
Critical strengths
-
The problem and the information-flow design are well motivated. Unmasked source regions can still contain an object's shadows, reflections, or trails, so treating all visible source content as reliable background can harm reconstruction. Separating condition and target streams is an intuitive way to address this issue, and the EGS branch is used only during training.
-
The complete system gives strong results on important measures. On PROVE-M, TIE reaches 28.9711 PSNR and 0.2490 RC-T, improving over the best listed comparison on both measures. The user study includes 40 participants and 30 clips; the pooled preferences favor TIE for removal completeness (79.00%) and visual consistency (77.69%). These results support a practical benefit on the tested samples.
Critical weaknesses
-
The current ablations do not establish a fair quality-cost comparison for the three-stream design. In Table 3, changing from channel concatenation to token concatenation gives the largest PSNR increase, about 4.78 dB, while adding directed type isolation gives a much smaller change with mixed movement in other metrics. The paper does not fully specify the channel baseline's mask encoding, input projection, initialization or freezing, and shared training settings. The three streams also increase the token sequence from N to 3N, but matched FLOPs, peak memory, training time, and end-to-end inference time are not reported. The main gain is therefore hard to separate from the representation change and its extra cost.
-
The added value of evidence-guided allocation is not fully isolated. The paper compares no EGS, geometric support, raw matching, and uniform point-wise supervision, but it does not keep both point-wise and region-wise losses and total auxiliary strength fixed while changing only the allocation rule. A matched constant or shuffled allocation is needed. Table 5 also shows that uniform point-wise supervision already matches the full model's RC-T of 0.249, while several reported margins are small and have no paired uncertainty. Separate shadow/reflection results and tests with moderate mask-boundary errors would make the EGS claim much stronger.
Questions for authors
-
How was the channel baseline's input projection initialized and trained, and what training settings were used for each concatenation design? Does the advantage persist with comparable training compute? Please report the corresponding quality and cost measurements, including complete inference timing and any condition caching.
-
On shadow and reflection cases, how much does full EGS improve over both the existing no-EGS baseline and a fixed mixture of the same auxiliary losses? Is the improvement stable across runs and when object masks have moderate boundary errors?