- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes BiNar for bitstream-corrupted image restoration. It extracts semantic information directly from corrupted bitstreams, combines it with available visual information, and predicts corrupted regions to guide diffusion-based restoration. Experiments on ADE20K show improvements over existing methods on most evaluation metrics.
Critical strengths
- The paper addresses an interesting problem by extracting semantic information directly from corrupted bitstreams instead of relying only on decoded images.
- The experimental results show improvements over existing methods on most restoration metrics.
Critical weaknesses
-
The proposed framework mainly combines existing components, including ByteFormer, semantic modeling, corruption localization, and diffusion-based restoration. The technical novelty of the overall framework is relatively limited.
-
The experiments are only conducted on ADE20K with low-resolution images. It is unclear whether the method can generalize to other datasets and higher-resolution images (4K).
-
The framework involves multiple encoders, a language decoder, and a diffusion model, but no inference speed, GPU memory usage, or computational cost is reported. This makes it difficult to evaluate its practical efficiency.
Questions for authors
- What are the main technical differences between BiNar and existing bitstream semantic understanding methods, such as Cibic?
- Can the authors provide results on other datasets and higher-resolution images to demonstrate generalization?
4 comments
3. Reply to W2 & Q2: Evaluation on additional datasets and 4K resolution images
Thank you for your suggestion. We additionally evaluate on Flickr8k, whose focus on actions and events complements the scene-centric ADE20K dataset. In particular, we evaluate on 4K images from Aesthetic-Train. To handle ultra-long bitstreams from 4K images, we incorporate global positional encoding into our bitstream semantic modeling. Lower encoder layers process byte segments separately, while higher layers integrate their features through cross-segment attention.
| Dataset | Method | PSNR↑ | SSIM↑ | MAE↓ | VFID↓ | DreamSim↓ | LPIPS↓ | CLIP-I↑ | DINO-I↑ |
|---|---|---|---|---|---|---|---|---|---|
| Flickr8k | B2SCVR (ACM MM 2025) | 12.8858 | 0.4203 | 50.2292 | 23.7166 | 0.5837 | 0.6958 | 0.4811 | 0.3654 |
| M-GDM (CVPR 2025) | 11.2924 | 0.3715 | 55.0095 | 29.4191 | 0.6417 | 0.7168 | 0.4267 | 0.2835 | |
| LucidFlux (ICLR 2026) | 8.6541 | 0.1823 | 81.0130 | 30.3244 | 0.6657 | 0.7873 | 0.4158 | 0.2926 | |
| BiNar (Ours) | 16.1084 | 0.6136 | 26.4015 | 7.5046 | 0.2143 | 0.3154 | 0.8124 | 0.7682 | |
| Aesthetic-Train (4K resolution) | B2SCVR (ACM MM 2025) | 14.8073 | 0.6625 | 34.5261 | 28.4937 | 0.4306 | 0.5418 | 0.6517 | 0.6014 |
| M-GDM (CVPR 2025) | 13.8146 | 0.6217 | 39.0468 | 30.5172 | 0.4518 | 0.5716 | 0.6314 | 0.5719 | |
| LucidFlux (ICLR 2026) | 8.9135 | 0.3528 | 72.0146 | 40.2175 | 0.5527 | 0.7018 | 0.5124 | 0.4513 | |
| BiNar (Ours) | 15.3322 | 0.6776 | 23.5265 | 22.2466 | 0.3008 | 0.3721 | 0.7484 | 0.6678 |
BiNar outperforms all compared methods on all eight restoration metrics on both additional datasets, demonstrating its generalization across different image distributions and the effectiveness of our extension to higher-resolution (4K) image restoration.
4. Reply to W3: Reporting inference speed and GPU memory usage
We report inference speed, GPU memory usage, and restoration performance for BiNar and competing methods as follows. BiNar achieves superior restoration performance at a moderate computational cost. Although B2SCVR is computationally lighter, BiNar outperforms it on all eight restoration metrics. BiNar also outperforms M-GDM and LucidFlux while being 6.52× and 2.85× faster, respectively, demonstrating its high practical efficiency.
| Method | Inference speed (s/image)↓ | GPU memory usage (GiB)↓ | PSNR↑ | LPIPS↓ | CLIP-I↑ | DINO-I↑ |
|---|---|---|---|---|---|---|
| B2SCVR (ACM MM 2025) | 5.3348 | 3.1748 | 12.8858 | 0.6958 | 0.4811 | 0.3654 |
| M-GDM (CVPR 2025) | 72.5217 | 13.3213 | 11.2924 | 0.7168 | 0.4267 | 0.2835 |
| LucidFlux (ICLR 2026) | 31.7578 | 25.5969 | 8.6541 | 0.7873 | 0.4158 | 0.2926 |
| BiNar (Ours) | 11.1253 | 23.3744 | 16.1084 | 0.3154 | 0.8124 | 0.7682 |
We will add these additional experiments and clarifications into the camera-ready version.
I really appreciate the additional experiments. However, I still have some doubts about the novelty of this method. Although the authors have clarified several technical differences, it is unclear whether these modifications go beyond the integration of existing components. I would appreciate it if the authors could explain more clearly the core innovation and motivation.
Secondly, I have some questions regarding the comparison with video restoration methods. Given the significant differences between image bitstreams and video bitstreams, could the authors elaborate on the implementation details of how the image bitstream is adapted for video restoration methods, and discuss whether this adaptation is fair?
1. Reply: Clarifying the Core Innovation and Motivation
We thank the reviewer for raising this important question, which gives us an opportunity to clarify the core innovation and motivation of our method.
Bitstream-corrupted image restoration is a challenging task in real-world multimedia communications. In this paper, we propose a specialized solution comprising multiple novel techniques to address each challenge.
Challenge 1: Native semantics in the bitstream modality are not fully and reliably exploited
Existing bitstream models such as Cibic can only generate brief and unreliable captions, which are not verified by the residual visual information in corrupted images.
Our Specialized Innovation 1:
To address this challenge, we propose BSeM and CMSA network components in our framework.
Specialized Innovation 1.1 Our BSeM performs bitstream-text alignment through contrastive learning (ignored by ByteFormer, Cibic, and other bitstream models), enabling fully aligned semantic modeling and generating coarse and fine descriptions to fill the modality gap between bitstreams and language.
Specialized Innovation 1.2 Our CMSA further performs reliable multimodal semantic aggregation. We make full use of the reliable visual features remaining in corrupted images to refine the bitstream semantics. To the best of our knowledge, this is the first attempt to adaptively enhance the reliability of bitstream semantics through multimodal aggregation. Our additional experiments in the response to Reviewer APLt demonstrate the contribution of the proposed CMSA..
Challenge 2: Accurate and labor-free corruption localization is an urgent need in practical bitstream-corrupted image and video restoration
Existing methods such as DAC (MM '26) and BSCVR (NeurIPS '23) rely on labor-intensive human annotation of error masks in the initial frames of a sample.
Our Specialized Innovation 2:
We propose BCoL, a labor-free corruption localization module in our framework. Faced with diverse bitstream corruption types and severities, and the resulting unpredictable error patterns in decoded images, BCoL learns the distributions of varying error patterns without relying on human-provided initial labels or prompts. Please also note that the generated error masks are well coordinated with the reliable bitstream semantics previously aligned and aggregated in Our Specialized Innovations 1.1 and 1.2. These Specialized Innovations provide complementary guidance as conditions for subsequent diffusion models.
Our framework design is strongly motivated by practical challenges. All the proposed modules in our framework are not simply integrations of existing components. They are also not separate modules that work independently. Instead, they collaborate systematically to achieve strong image restoration capabilities, as demonstrated by our experimental results. Therefore, we believe the core innovation and motivation of our framework go far beyond component integration.
2. Reply: Clarifying the Adaptation of Video-Method Comparisons
Thank you for your question. We replicate each corrupted test image 16 times and encode the frames into a video, which is then processed using each video restoration method's original inference pipeline. This adaptation allows video restoration methods to operate on the image restoration task and is included to broaden the scope of our comparisons. Our primary comparisons remain focused on image restoration methods.
We thank you for these thoughtful questions and constructive feedback. We hope our clarifications address your concerns.
1. Reply to W1: Clarifying technical novelty beyond existing components
Thank you for your valuable comment. Our proposed BiNar has fundamental technical novelty and methodological contributions to practically solve real-world bitstream-corrupted image restoration.
Our proposed BSeM introduces explicit semantic alignment objectives and two-phase adaptation, going beyond caption-generation training in existing bitstream semantic modeling methods. We first adapt bitstream representations with the language decoder frozen, then adapt language generation with the bitstream encoder frozen.
Our proposed CMSA adaptively uses available visual information to improve the quality of bitstream-derived semantics, addressing the limitation of existing methods that extract semantics from either bitstreams or images alone.
Our proposed BCoL directly predicts corrupted regions from the damaged image without point, box, or mask prompts, eliminating the need for an initial corruption mask required by prior methods.
Our restoration framework jointly uses bitstream semantics, visual cues, and corruption localization to guide diffusion-based restoration. This design maintains robust restoration performance under decoding failure, where existing restoration frameworks relying on visual input become ineffective.
2. Reply to Q1: Clarifying technical differences from existing bitstream semantic understanding methods
The main technical differences between BiNar and Cibic are semantic alignment and the use of complementary visual information. Compared with Cibic, we additionally introduce token alignment and contrastive objectives to explicitly align bitstream and language representations, and adaptively use available visual information to improve the quality of bitstream-derived semantics.
To compare the effectiveness of the extracted semantics in restoration, we conduct a controlled comparison by replacing only our bitstream semantic model with Cibic. The experimental results are shown in the following table (we additionally evaluate on the Flickr8k dataset). Our pipeline outperforms the Cibic pipeline on all eight restoration metrics on both datasets, demonstrating the advantages of our bitstream semantic modeling in improving restoration fidelity and semantic consistency.