- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PACE, an efficient video inpainting framework that exploits the asymmetric roles of masked and non-mask regions. Instead of propagating the full non-mask token sequence through the DiT, PACE compresses the known non-mask trajectory into a small set of position-aware context slots and combines them with masked-region features through adaptive attention fusion. The method is motivated by an analysis showing that masked queries concentrate attention on a relatively small subset of non-mask tokens. Experiments on DAVIS, YouTube-VOS, and the proposed AdErase benchmark show improved quality over efficient baselines while achieving faster inference.
Critical strengths
- The method directly exploits the asymmetry between masked-region synthesis and non-mask context processing.
- PACE improves over EditCtrl and YOSE while remaining competitive with full-video methods, with a reported 1.7×–2.1× average speedup.
- Evaluated on AdErase, DAVIS, and YouTube-VOS, covering both advertising removal and public video inpainting benchmarks, and matches or exceeds the efficient baselines across the reported metrics.
Critical weaknesses
- Limited evaluation of the synthesized regions. PSNR, SSIM, and LPIPS are computed only outside the removal mask, while the remaining quality metrics are evaluated on the full video. This is relevant because many masks are small, with 55.9% of analyzed frames below an 8% mask ratio. A controlled benchmark with clean masked-region targets would provide more direct evidence of completion quality.
- The ablations do not fully isolate the proposed design choices. The context ablation removes the entire branch, including positional information, and the calibration ablation evaluates the count correction and learned residual jointly. The results therefore support the complete modules, but do not show how much the spatial anchors or individual calibration terms contribute. More targeted ablations would make the attribution clearer.
Questions for authors
- In Fig. 5, the labels "w/o PACE" and "w/o adaptive fusion" are used to denote the two ablation studies, which is inconsistent with the table on the left. Although the authors provide an explanation in the main text, the notation is confusing at first glance, why not use consistent terminology?
- After increasing the slot budget from K=91 to K=390, some metrics such as LPIPS actually worsen slightly, can you explain why?
2 comments
Thank you for your response and the additional experiments. I have no further concerns. I will increase my rating.
One small suggestion: If possible, it would be better to include a few visual examples from the new clean-target evaluation, with the ground truth for comparison.
Good luck!
Thank you for your thoughtful and constructive feedback. We are encouraged that you appreciate our method design, favorable efficiency–quality trade-off, and comprehensive experimental evaluation. Below are our responses to your concerns.
(1) AdErase covers a broad range of mask-area ratios. We sampled videos evenly across four mask-area bins: 0–10%, 10–20%, 20–30%, and 30–40% of the frame area. The cited 55.9% statistic describes the online source footage, not our test set. We will clarify this sampling strategy in the dataset description.
(2) The supplemented clean-target evaluation supports PACE's reconstruction quality. We construct 500 input–target pairs from clean online videos by synthetically masking regions and retaining the original videos as clean targets. In addition to the original metrics, we report PSNR, SSIM, and LPIPS within the masked regions. PACE outperforms efficient baselines EditCtrl and YOSE on all metrics. PACE also achieves higher masked PSNR and lower masked LPIPS than the full-video methods MiniMax-RM, ROSE, and VACE. Although ProPainter achieves higher masked PSNR and SSIM, its substantially worse masked LPIPS (0.247 vs. 0.196) is consistent with its blurrier textures compared with PACE’s sharper details in Fig. 4.
(1) Spatial anchoring improves the reported scores while retaining the context branch. The original Table 3 reports
w/o context branch. In this rebuttal, we addw/o spatial anchors, which removes the positional encoding. Removing spatial anchors lowers Mask-Image Quality from 0.468 to 0.461, whole-video Imaging Quality from 0.570 to 0.558, and non-mask PSNR from 33.63 to 33.30 dB. These results support the contribution of query anchoring to local image quality and background preservation in this setting.(2) The combined count and residual correction achieves the best or tied results on all seven metrics. The total correction is
β = β_cnt + δ. The count term β_cnt = log(N_nm/K) compensates for token-count imbalance, while the learned residual δ provides a learned bias and a content-dependent adjustment. The original Table 3 reportsw/o calibration(β=0). In this rebuttal, we addw/o count term(β_cnt = 0) andw/o learned residual(δ = 0). Removing either term lowers the performance. The combined configuration performs best overall in this comparison, supporting the use of both terms together.We will use “w/o context” and “w/o calibration” consistently in Fig. 5 and Table 3 for clarity.
The perceptual declines are small: non-mask LPIPS rises by 0.001, and Aesthetic Quality and Imaging Quality fall by 0.002 and 0.003, respectively. A plausible explanation is that the extra slots encode finer details of the same background, where masked queries already focus on a small set of relevant cues (Fig. 2). Those details may reduce pixel error while slightly altering local textures near the mask boundary during decoding, which can degrad LPIPS even as PSNR improves.