- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes DecodeAlign, a post-decoding module for cfm. They use a frame-wise NAFNet to enhance the frames. It is trained on x265/DCVC-UF-compressed Vimeo-90k frames with a Charbonnier pixel loss and a cosine feature loss on the frozen Faster R-CNN (ResNeXt-101) backbone features. The proposed method splits the feature error of decoded frames into an irreducible term ("information availability") and a mismatch term ("representation availability"). This paper demonstrates that post-processing can reduce only the mismatch term. Experiments show up to a 45.6% BD-rate reduction.
Critical strengths
- The proposed method is a simple plug-in with consistent gains at low bitrates and can improve all kinds of codecs
Critical weaknesses
- The contributions overlap substantially with prior work that is not cited or is misdescribed.
- Representation-space insight (Sec. 1, 3.1–3.2). This is the premise of [H]: Eq. 2 here is the intermediate-layer distortion of [H] (Eq. 10), and [H] proves that coding optimized for input fidelity is no better, and under mild conditions strictly worse, than coding optimized for task distortion (Thms. 3–4). Beyond [H], Sec. 3 only adds Prop. 1. That is the textbook MSE decomposition around , and it gives no guarantee for a frame-domain .
- Post-decoding framework. Post-filters that adapt a human-centric codec to machine tasks already exist. [A] post-filters VVC-decoded images and reports ~49%/45% BD-rate gains for detection/segmentation. [B] trains post-filters after a VTM decoder with a feature-domain proxy loss. The paper cites [B] but lists it among methods that change "the coding process", and then claims that all existing approaches "adapt the coding process" (L048–050). Both statements are inaccurate as far as I understand.
- Training objective. Training a restoration network with a loss on a frozen task network's backbone features, without labels, and transferring it to other task networks, is TTAC [C]. [H] likewise uses multi-level feature-MSE distillation on frozen Faster R-CNN/YOLOv3 features, and task-driven restoration is also well studied [D, E, F]. Fig. 5 shows generic DINOv2 features work about as well (+2.00 vs. +2.28 mAP on Faster R-CNN; +0.95 vs. +0.92 on YOLOv8m). The gain therefore looks like perceptual-loss [G] artifact removal rather than alignment specific to the target model.
- The experiments do not support the practicality and generality claims.
- The only baseline is the unprocessed host output. There is no comparison with [A, B], TTAC [C], a strong compressed-video enhancement network, or even simply fine-tuning the detector on decoded frames.
- Inference complexity is not provided. Needs to report runtime, MACs, or peak memory.
Reference:
[A] Ahonen et al., "Learned Enhancement Filters for Image Coding for Machines," IEEE ISM 2021.
[B] Zhang et al., "Competitive Learning for Achieving Content-specific Filters in Video Coding for Machines," ICIP 2024.
[C] Ehrlich et al., "Analyzing and Mitigating JPEG Compression Defects in Deep Learning," ICCV Workshops 2021.
[D] Liu et al., "When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach," IJCAI 2018.
[E] Son et al., "URIE: Universal Image Enhancement for Visual Recognition in the Wild," ECCV 2020.
[F] Haris et al., "Task-Driven Super Resolution: Object Detection in Low-resolution Images," arXiv 2018.
[G] Johnson et al., "Perceptual Losses for Real-Time Style Transfer and Super-Resolution," ECCV 2016.
[H] Harell et al., "Rate-Distortion Theory in Coding for Machines and Its Applications," IEEE TPAMI, vol. 47, no. 7, pp. 5501–5519, 2025.
Questions for authors
Please see the weakness.