acceptodds
Under review as a conference paper at ICLR 2027

BiNar: Bitstream Native Restorer for Bitstream-corrupted Image Restoration

Abstract

Bitstream-corrupted image restoration (BCIR) aims to reconstruct faithful images from corrupted compressed bitstreams. However, corruption occurs in bitstream-domain may lead to unpredictable error propagation and distortion amplification in pixel-domain, and even result in complete undecodability by standard decoders, posing significant challenge to existing image restoration paradigms. To address this challenge, we propose Bitstream Native Restorer (BiNar), a generative restoration framework that leverages fine-grained bitstream-native semantics to guide bitstream-corrupted image restoration. In BiNar, Bitstream Semantic Modeling (BSeM) extracts robust informative semantics and models sequential bytes directly from corrupted bitstream, enabling to generate fine-grained bitstream descriptions without relying on pixel decoding. Based on the descriptions, the Corruption-adaptive Multimodal Semantic Aggregation (CMSA) further extracts adaptive visual modal cues in decoded corrupted images and aggregates to bitstream descriptions. The multimodal aggregated fine-grained descriptions can serve as coherent restoration guidance. Finally, Blind Corruption Localization (BCoL) exploits the segmentation prior of visual foundation models to identify and locate error regions that requires recovery in corrupted images. The located error regions and the fine-grained descriptions jointly serve as visual prompt and language prompt of diffusion models image restoration, respectively. Extensive experiments under diverse bitstream corruption conditions show that BiNar outperforms all State-of-the-arts image restoration models regardless of perceptual quality or objective quality. Ablation studies demonstrate the effectiveness of fine-grained bitstream-native semantics in BCIR task.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image restoration, Bitstream corruption, Byte-level modeling, Generative models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jLyROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BiNar for bitstream-corrupted image restoration. It extracts semantic information directly from corrupted bitstreams, combines it with available visual information, and predicts corrupted regions to guide diffusion-based restoration. Experiments on ADE20K show improvements over existing methods on most evaluation metrics.

Critical strengths
  1. The paper addresses an interesting problem by extracting semantic information directly from corrupted bitstreams instead of relying only on decoded images.
  2. The experimental results show improvements over existing methods on most restoration metrics.
Critical weaknesses
  1. The proposed framework mainly combines existing components, including ByteFormer, semantic modeling, corruption localization, and diffusion-based restoration. The technical novelty of the overall framework is relatively limited.

  2. The experiments are only conducted on ADE20K with low-resolution images. It is unclear whether the method can generalize to other datasets and higher-resolution images (4K).

  3. The framework involves multiple encoders, a language decoder, and a diffusion model, but no inference speed, GPU memory usage, or computational cost is reported. This makes it difficult to evaluate its practical efficiency.

Questions for authors
  1. What are the main technical differences between BiNar and existing bitstream semantic understanding methods, such as Cibic?
  2. Can the authors provide results on other datasets and higher-resolution images to demonstrate generalization?
4 comments

1. Reply to W1: Clarifying technical novelty beyond existing components

Thank you for your valuable comment. Our proposed BiNar has fundamental technical novelty and methodological contributions to practically solve real-world bitstream-corrupted image restoration.

Our proposed BSeM introduces explicit semantic alignment objectives and two-phase adaptation, going beyond caption-generation training in existing bitstream semantic modeling methods. We first adapt bitstream representations with the language decoder frozen, then adapt language generation with the bitstream encoder frozen.

Our proposed CMSA adaptively uses available visual information to improve the quality of bitstream-derived semantics, addressing the limitation of existing methods that extract semantics from either bitstreams or images alone.

Our proposed BCoL directly predicts corrupted regions from the damaged image without point, box, or mask prompts, eliminating the need for an initial corruption mask required by prior methods.

Our restoration framework jointly uses bitstream semantics, visual cues, and corruption localization to guide diffusion-based restoration. This design maintains robust restoration performance under decoding failure, where existing restoration frameworks relying on visual input become ineffective.


2. Reply to Q1: Clarifying technical differences from existing bitstream semantic understanding methods

The main technical differences between BiNar and Cibic are semantic alignment and the use of complementary visual information. Compared with Cibic, we additionally introduce token alignment and contrastive objectives to explicitly align bitstream and language representations, and adaptively use available visual information to improve the quality of bitstream-derived semantics.

To compare the effectiveness of the extracted semantics in restoration, we conduct a controlled comparison by replacing only our bitstream semantic model with Cibic. The experimental results are shown in the following table (we additionally evaluate on the Flickr8k dataset). Our pipeline outperforms the Cibic pipeline on all eight restoration metrics on both datasets, demonstrating the advantages of our bitstream semantic modeling in improving restoration fidelity and semantic consistency.

DatasetMethodPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
ADE20KCibic pipeline14.30660.456736.525917.04170.39350.44740.67900.6470
Our pipeline15.86290.519831.597514.89100.36800.42310.72100.7103
Flickr8kCibic pipeline14.29370.576330.94559.09200.25160.34760.78080.7319
Our pipeline16.10840.613626.40157.50460.21430.31540.81240.7682

3. Reply to W2 & Q2: Evaluation on additional datasets and 4K resolution images

Thank you for your suggestion. We additionally evaluate on Flickr8k, whose focus on actions and events complements the scene-centric ADE20K dataset. In particular, we evaluate on 4K images from Aesthetic-Train. To handle ultra-long bitstreams from 4K images, we incorporate global positional encoding into our bitstream semantic modeling. Lower encoder layers process byte segments separately, while higher layers integrate their features through cross-segment attention.

DatasetMethodPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
Flickr8kB2SCVR (ACM MM 2025)12.88580.420350.229223.71660.58370.69580.48110.3654
M-GDM (CVPR 2025)11.29240.371555.009529.41910.64170.71680.42670.2835
LucidFlux (ICLR 2026)8.65410.182381.013030.32440.66570.78730.41580.2926
BiNar (Ours)16.10840.613626.40157.50460.21430.31540.81240.7682
Aesthetic-Train (4K resolution)B2SCVR (ACM MM 2025)14.80730.662534.526128.49370.43060.54180.65170.6014
M-GDM (CVPR 2025)13.81460.621739.046830.51720.45180.57160.63140.5719
LucidFlux (ICLR 2026)8.91350.352872.014640.21750.55270.70180.51240.4513
BiNar (Ours)15.33220.677623.526522.24660.30080.37210.74840.6678

BiNar outperforms all compared methods on all eight restoration metrics on both additional datasets, demonstrating its generalization across different image distributions and the effectiveness of our extension to higher-resolution (4K) image restoration.


4. Reply to W3: Reporting inference speed and GPU memory usage

We report inference speed, GPU memory usage, and restoration performance for BiNar and competing methods as follows. BiNar achieves superior restoration performance at a moderate computational cost. Although B2SCVR is computationally lighter, BiNar outperforms it on all eight restoration metrics. BiNar also outperforms M-GDM and LucidFlux while being 6.52× and 2.85× faster, respectively, demonstrating its high practical efficiency.

MethodInference speed (s/image)↓GPU memory usage (GiB)↓PSNR↑LPIPS↓CLIP-I↑DINO-I↑
B2SCVR (ACM MM 2025)5.33483.174812.88580.69580.48110.3654
M-GDM (CVPR 2025)72.521713.321311.29240.71680.42670.2835
LucidFlux (ICLR 2026)31.757825.59698.65410.78730.41580.2926
BiNar (Ours)11.125323.374416.10840.31540.81240.7682

We will add these additional experiments and clarifications into the camera-ready version.

I really appreciate the additional experiments. However, I still have some doubts about the novelty of this method. Although the authors have clarified several technical differences, it is unclear whether these modifications go beyond the integration of existing components. I would appreciate it if the authors could explain more clearly the core innovation and motivation.

Secondly, I have some questions regarding the comparison with video restoration methods. Given the significant differences between image bitstreams and video bitstreams, could the authors elaborate on the implementation details of how the image bitstream is adapted for video restoration methods, and discuss whether this adaptation is fair?

1. Reply: Clarifying the Core Innovation and Motivation

We thank the reviewer for raising this important question, which gives us an opportunity to clarify the core innovation and motivation of our method.

Bitstream-corrupted image restoration is a challenging task in real-world multimedia communications. In this paper, we propose a specialized solution comprising multiple novel techniques to address each challenge.

Challenge 1: Native semantics in the bitstream modality are not fully and reliably exploited

Existing bitstream models such as Cibic can only generate brief and unreliable captions, which are not verified by the residual visual information in corrupted images.

Our Specialized Innovation 1:

To address this challenge, we propose BSeM and CMSA network components in our framework.

Specialized Innovation 1.1 Our BSeM performs bitstream-text alignment through contrastive learning (ignored by ByteFormer, Cibic, and other bitstream models), enabling fully aligned semantic modeling and generating coarse and fine descriptions to fill the modality gap between bitstreams and language.

Specialized Innovation 1.2 Our CMSA further performs reliable multimodal semantic aggregation. We make full use of the reliable visual features remaining in corrupted images to refine the bitstream semantics. To the best of our knowledge, this is the first attempt to adaptively enhance the reliability of bitstream semantics through multimodal aggregation. Our additional experiments in the response to Reviewer APLt demonstrate the contribution of the proposed CMSA..

Challenge 2: Accurate and labor-free corruption localization is an urgent need in practical bitstream-corrupted image and video restoration

Existing methods such as DAC (MM '26) and BSCVR (NeurIPS '23) rely on labor-intensive human annotation of error masks in the initial frames of a sample.

Our Specialized Innovation 2:

We propose BCoL, a labor-free corruption localization module in our framework. Faced with diverse bitstream corruption types and severities, and the resulting unpredictable error patterns in decoded images, BCoL learns the distributions of varying error patterns without relying on human-provided initial labels or prompts. Please also note that the generated error masks are well coordinated with the reliable bitstream semantics previously aligned and aggregated in Our Specialized Innovations 1.1 and 1.2. These Specialized Innovations provide complementary guidance as conditions for subsequent diffusion models.

Our framework design is strongly motivated by practical challenges. All the proposed modules in our framework are not simply integrations of existing components. They are also not separate modules that work independently. Instead, they collaborate systematically to achieve strong image restoration capabilities, as demonstrated by our experimental results. Therefore, we believe the core innovation and motivation of our framework go far beyond component integration.


2. Reply: Clarifying the Adaptation of Video-Method Comparisons

Thank you for your question. We replicate each corrupted test image 16 times and encode the frames into a video, which is then processed using each video restoration method's original inference pipeline. This adaptation allows video restoration methods to operate on the image restoration task and is included to broaden the scope of our comparisons. Our primary comparisons remain focused on image restoration methods.

We thank you for these thoughtful questions and constructive feedback. We hope our clarifications address your concerns.

Reviewer_CBmCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bitstream-corrupted image restoration (BCIR), which is to reconstruct faithful images from corrupted compressed bitstreams. The framework consists of three components: Bitstream Semantic Modeling (BSeM), which extracts Plot and Scene semantics directly from corrupted bitstreams; Corruption-adaptive Multimodal Semantic Aggregation (CMSA), which combines bitstream-derived semantics with available visual information; Blind Corruption Localization (BCoL), which identifies damaged regions. The resulting semantic descriptions and spatial masks guide a pretrained diffusion model for image restoration. The experimental results in ADE20K images are better than several existing restoration methods.

Critical strengths
  1. The paper addresses an important limitation of conventional image restoration methods: corrupted compressed bitstreams can produce severe visual distortions or decoding failures, making pixel-domain information unreliable.

2.The framework combines semantic information extracted from bitstreams, visual cues from decoded images, and spatial corruption localization. This design offers a potentially useful approach to restoration under corruption.

Critical weaknesses
  1. This paper appears primarily to integrate existing techniques into an engineering driven framework.
    (1) BSeM builds upon ByteFormer [1] and resembles Cibic [2], which already extracts semantic descriptions directly from corrupted bitstreams using a bitstream encoder and language decoder.
    (2) CMSA follows the direction of BLMSP [3], which leverages bitstream-native semantic priors for downstream visual tasks.
    (3) BCoL resembles Detect Any Corruption module [4], which utilizes visual foundation models for blind corruption localization.
    (4) semantic and visual prompting for diffusion-based image restoration has already been investigated in DPIR [5].

  2. This paper combines multiple ByteFormer encoders, language models, visual encoders, SAM, and a diffusion model into a complicated multi-stage pipeline. However, Table 2 lacks of other SOTA methods comparisons using Cibic or BLMSP derived semantics with the same diffusion backbone. Furthermore, the ablation studies lacks evaluation on final restoration performance, making it difficult to isolate the actual contributions of the proposed modules. The paper also lacks computational efficiency measurements, such as latency and computation cost.

  3. This paper only test on one dataset ADE20K, using synthetic bitstream corruption. Although multiple corruption configurations are considered, they do not establish generalization across different image distributions.Table 2 reports only averaged restoration results across corruption settings, obscuring performance under different corruption severities and decoding failures. Furthermore, the comparison in Table 1 is unfair, as BiNar is specifically trained for the target task, whereas the competing VLMs are evaluated in a zero-shot setting.

References
[1] Horton et al. (2023). Bytes Are All You Need: Transformers Operating Directly on File Bytes. arXiv:2306.00238.
[2] Liang et al. (2026). Cibic: Pixel-free Foundation Model for Robust Corrupted Image Bitstream Captioning. Pattern Recognition.
[3] Huang et al. (2026). Towards Bitstream-corrupted Harsh Visual Understanding: Through Bitstream Language Modeling as Robust Semantic Priors. arXiv:2608.21837.
[4] Liu et al. (2025). Towards Blind Bitstream-corrupted Video Recovery: A Visual Foundation Model-driven Framework. ACM Multimedia.
[5] Kong et al. (2025). Dual Prompting Image Restoration with Diffusion Transformers. CVPR.

Questions for authors
  1. Given the similarities to Cibic [2], BLMSP [3], and DAC [4], what are the fundamental methodological contributions of BiNar beyond integrating existing techniques?

  2. Could the authors compare BiNar with using Cibic or BLMSP derived semantics with the same diffusion backbone? Could the authors provide ablation studies evaluating each proposed module's contribution to final image restoration performance?

  3. What are the inference latency, computational cost, and GPU memory requirements of BiNar? How do these compare with simpler restoration pipelines, and are the performance improvements sufficient to justify the additional complexity?

  4. Could the authors evaluate BiNar on additional datasets and report restoration results separately for different corruption severities and decoding failure conditions?

  5. How do the authors justify the fairness of comparison in table 1? Could they include task adapted baselines or existing bitstream-native semantic models for a more meaningful evaluation?

4 comments

1. Reply to W1 & Q1: Clarifying methodological contributions beyond existing techniques

Thank you for your valuable comment. BiNar addresses practical restoration from corrupted bitstreams through semantic modeling, adaptive fusion, and corruption localization.

Cibic lacks explicit semantic alignment objectives between bitstream and language representations. We introduce semantic alignment objectives and two-stage training, first emphasizing semantic extraction and alignment, then adapting language generation.

BLMSP focuses on bitstream semantics when constructing its semantic prior, overlooking complementary visual information in the damaged image. We design a multimodal semantic aggregation module that adaptively uses available visual information to improve the quality of bitstream-derived semantics.

DAC relies on an initial corruption mask to initialize corruption localization. We design a corruption localization module that directly predicts damaged regions from the corrupted image without point, box, or mask prompts, providing spatial guidance for restoration.

DPIR explores semantic and visual prompting for restoration, but relies on decoded visual information that becomes unavailable when decoding fails. Our framework jointly uses bitstream semantics, visual cues, and corruption localization to guide diffusion-based restoration, maintaining robust restoration performance even under decoding failure.


2. Reply to W2 & Q2: Comparison with other SOTA bitstream methods and module contribution ablation studies

We integrate Cibic into our BiNar pipeline by replacing only our bitstream semantic model. Results on ADE20K and the additional Flickr8k dataset are shown below. Our pipeline outperforms the Cibic pipeline on all eight metrics on both datasets, improving both restoration fidelity and semantic consistency.

DatasetMethodPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
ADE20KCibic pipeline14.30660.456736.525917.04170.39350.44740.67900.6470
Our pipeline15.86290.519831.597514.89100.36800.42310.72100.7103
Flickr8kCibic pipeline14.29370.576330.94559.09200.25160.34760.78080.7319
Our pipeline16.10840.613626.40157.50460.21430.31540.81240.7682

We remove each module individually from BiNar to evaluate its contribution to restoration, as shown below. BiNar achieves the best results on all eight metrics on both datasets, demonstrating each module’s contribution.

DatasetSettingPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
ADE20Kw/o BSeM13.27430.387350.695625.49090.49070.51650.52580.4729
w/o CMSA13.42260.383147.096015.91010.42430.47690.61790.5933
w/o BCoL8.90990.138376.827221.17760.64290.70580.45070.3992
BiNar15.86290.519831.597514.89100.36800.42310.72100.7103
Flickr8kw/o BSeM12.79450.413249.446415.93690.48940.55110.54310.4705
w/o CMSA12.93860.416747.13948.09700.42420.54140.63330.5525
w/o BCoL9.52120.195371.020510.91090.62120.79740.46680.3453
BiNar16.10840.613626.40157.50460.21430.31540.81240.7682

3. Reply to W2 & Q3: Trade-off between computational cost and restoration performance

We report inference latency, GPU memory requirements, and restoration performance below. BiNar achieves significant performance advantages at a moderate computational cost. Although B2SCVR is computationally lighter, BiNar outperforms it on all restoration metrics. BiNar also outperforms M-GDM and LucidFlux on all eight metrics while being 6.52× and 2.85× faster, respectively. These results demonstrate that BiNar’s substantial restoration gains justify its additional computational cost.

MethodInference latency (s)↓GPU memory requirements (GiB)↓PSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
B2SCVR (ACM MM 2025)5.33483.174812.88580.420350.229223.71660.58370.69580.48110.3654
M-GDM (CVPR 2025)72.521713.321311.29240.371555.009529.41910.64170.71680.42670.2835
LucidFlux (ICLR 2026)31.757825.59698.65410.182381.013030.32440.66570.78730.41580.2926
Ours11.125323.374416.10840.613626.40157.50460.21430.31540.81240.7682

4. Reply to W3 & Q4: Evaluation on an additional dataset and under different corruption conditions

Thank you for your suggestion. We supplement our evaluation with an additional dataset (Flickr8k) and separately report restoration results for different corruption severities and decoding failure conditions.

We additionally evaluate on Flickr8k against all compared methods, with results for the strongest competing method shown below. BiNar outperforms all compared methods on all restoration metrics, demonstrating its generalization across different image distributions.

MethodPSNR↑LPIPS↓CLIP-I↑DINO-I↑
B2SCVR (ACM MM 2025)12.88580.69580.48110.3654
Ours16.1084 (+3.2226)0.3154 (−0.3804)0.8124 (+0.3313)0.7682 (+0.4028)

We separately report restoration performance under different corruption severities on both datasets, as shown in the following table.

DatasetCorruption severityPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
ADE20K25%20.75280.787015.206413.60170.14490.19500.88980.9250
50%16.94860.569526.112820.95140.25960.32600.80260.8015
75%13.59900.377439.582527.29880.46580.50780.67430.6375
100%12.15120.345345.488338.98330.60170.66370.51730.4772
Flickr8k25%21.84730.88277.42637.84410.03870.10370.95390.9637
50%16.90230.671218.753117.96490.14730.24360.85270.8426
75%13.69450.514330.614726.39130.27960.38310.73480.6573
100%11.98950.386248.811941.33430.39160.53120.70820.6092

We separately report restoration results under decoding failure conditions on both datasets, with comparisons against strong competing methods shown below. We also include our overall results for reference. BiNar outperforms all compared methods on all eight restoration metrics, demonstrating its significant performance advantage under decoding failure conditions.

DatasetMethodEvaluation samplesPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
ADE20KOursAll samples15.86290.519831.597514.89100.36800.42310.72100.7103
OursDecoding failure samples10.29030.136163.380222.58760.61860.71860.45680.3894
LD-RPS (ICCV 2025)Decoding failure samples7.36250.023791.611343.50950.84970.82480.27090.0588
Flickr8kOursAll samples16.10840.613626.40157.50460.21430.31540.81240.7682
OursDecoding failure samples9.51060.388152.486315.37280.41760.61240.66870.5732
B2SCVR (ACM MM 2025)Decoding failure samples8.00840.083465.9253125.33970.92290.92720.20640.0607

5. Reply to W3 & Q5: Justifying the fairness of semantic comparisons

Thank you for your suggestion. To ensure fairness, we also specifically train competing VLMs and a bitstream-native semantic model (GLM-4.6V, InternVL3, Qwen3-VL, and Cibic) for the target task. The results are reported below. Our proposed BiNar still outperforms these trained models on all five metrics, indicating higher quality of extracted semantics.

MethodBLEU-1↑BLEU-4↑ROUGE-L↑CIDEr↑Token-F1↑
GLM-4.6V0.18270.05540.19690.28140.2868
InternVL30.10990.02650.17380.10070.2453
Qwen3-VL0.03020.00870.15220.02710.2149
Cibic0.34060.14550.33450.17930.4062
BiNar (Ours)0.45410.22480.43670.29340.5028

We will add these additional experiments and clarifications into the camera-ready version.

Thank you for the detailed rebuttal and additional experiments. The new comparisons with Cibic, restoration-level ablations, computational efficiency measurements, and evaluations on Flickr8k and 4K images address several of my original concerns. These results strengthen the empirical evaluation of the paper.

However, my main concern regarding methodological novelty remains. The clarified contributions, including semantic alignment and two-stage training in BSeM, adaptive visual weighting in CMSA, and prompt-free localization in BCoL, still appear to be incremental modifications of established techniques rather than fundamentally new mechanisms. While the new experiments demonstrate the effectiveness of these components, they do not fully establish their methodological novelty.

Regarding architectural complexity, the reported efficiency results are helpful. Nevertheless, BiNar requires 11.13 seconds and 23.37 GiB of GPU memory per image, compared with 5.33 seconds and 3.17 GiB for B2SCVR. The additional computational cost remains substantial, although the restoration improvements are acknowledged.

Overall, given the remaining limitations in fundamental methodological novelty and architectural efficiency, I maintain my rating of 2 (Weak Rejection).

Reply: Clarifying the Methodological Novelty and Computational Trade-offs

Thank you for your follow-up comments and recognition of our additional experiments. We respectfully disagree that our contributions are merely incremental and further clarify our methodological novelty, coordinated framework design, and computational trade-offs below.

1. Our framework introduces problem-oriented mechanisms to address real-world bitstream corruption challenges

Building upon established techniques does not necessarily diminish methodological novelty. Recent works in top conferences and journals, such as FreeAdapt (ICLR’26), Detective SAM (ICLR’26), and SLURPP (TPAMI’26), demonstrate the value of problem-oriented innovations based on established techniques. Bitstream-corrupted image restoration poses unique challenges in real-world multimedia communication. To address these challenges, we propose a problem-oriented framework comprising multiple specialized mechanisms, rather than merely introducing incremental modifications to established methods.

Challenge 1: Native semantics in the bitstream modality are not fully exploited

Existing bitstream models such as Cibic generate only brief captions, failing to capture rich and fine-grained semantics from bitstreams.

Specialized Mechanism 1: Bitstream–language semantic alignment.

Our BSeM performs bitstream–text alignment through contrastive learning, overlooked by ByteFormer, Cibic, and other bitstream models. It further generates coarse and fine descriptions to bridge the modality gap between bitstreams and language.

Challenge 2: Residual visual information remains unexplored for bitstream semantic verification

Existing methods such as BLMSP rely solely on bitstream semantics, without leveraging residual visual information for semantic verification.

Specialized Mechanism 2: Reliable multimodal semantic aggregation.

To the best of our knowledge, CMSA is the first approach to adaptively enhance the reliability of bitstream semantics through multimodal aggregation. Specifically, CMSA models visual-token reliability to adaptively regulate multimodal semantic aggregation, enabling residual visual information to verify and refine bitstream semantics.

Challenge 3: Practical bitstream-corrupted image restoration demands accurate and labor-free corruption localization

Existing methods such as DAC require labor-intensive manual annotation of initial error masks, hindering fully automatic restoration.

Specialized Mechanism 3: Automatic corruption localization.

We introduce BCoL, a labor-free corruption localization module that learns diverse error-pattern distributions arising from different bitstream corruption types and severities. This mechanism enables automatic localization of irregular corruption regions without requiring human-annotated initial masks or prompts.

2. The coordinated framework contributes beyond its individual components

Our unified framework coordinates the reliable bitstream semantics established by BSeM and CMSA with the corruption masks generated by BCoL, providing complementary semantic and spatial conditions for diffusion-based restoration. Rather than operating as independent components, these specialized mechanisms collaborate systematically to address the interconnected challenges of bitstream-corrupted image restoration. Our ablation experiments further demonstrate their complementary contributions and the necessity of each mechanism for effective restoration. Therefore, we believe that the methodological contribution of our unified framework extends well beyond its individual components.

3. Pursuing extremely low computational cost is not a universal requirement across different application scenarios

We believe that computational cost should not be the primary criterion for evaluating the performance of a restoration system. For instance, large-scale restoration models such as SUPIR (CVPR’24) and LucidFlux (ICLR’26) leverage substantial computational resources to achieve superior restoration performance. This demonstrates that evaluation priorities should be determined by specific application scenarios and requirements.

We introduce a specialized framework to tackle the extreme challenges of bitstream-corrupted image restoration in real-world multimedia communication. Compared with B2SCVR, BiNar reduces LPIPS by 54.7% and increases CLIP-I and DINO-I by 68.9% and 110.2%, respectively. These substantial gains demonstrate the significance of our methodological contributions, which should not be overshadowed by additional computational complexity.

We respectfully encourage the reviewer to reconsider the assessment in light of our detailed responses and additional evidence, ensuring that the final evaluation fairly reflects the problem-oriented novelty, technical contributions, and overall merits of our work.

Reviewer_APLtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes BiNar, which extracts subject and scene semantics directly from corrupted image bitstreams and combines them with decoded visual cues and corruption localization to guide diffusion-based restoration. The framework comprises Bitstream Semantic Modeling (BSeM), Corruption-adaptive Multimodal Semantic Aggregation (CMSA), and Blind Corruption Localization (BCoL), and is evaluated on corrupted data constructed from ADE20K.

Critical strengths

1.The motivation is clear: error propagation during decoding does not imply an equivalent loss of information in the bitstream. Surviving bytes therefore provide a meaningful complementary source for restoration.

2.Figure 4 isolates the semantic source while keeping localization masks, the diffusion model, and sampling settings fixed, providing targeted evidence for bitstream-native semantics. The full model also achieves the best results on seven of eight restoration metrics among the evaluated methods.

Critical weaknesses
  1. Restoration fidelity under complete decoding failure is insufficiently validated. Section 3.4 states that, in this case, the entire image is restored using only semantic descriptions extracted from the bitstream. However, the paper does not separately report the number or results of such samples.

  2. The corruption-adaptive mechanism in CMSA is unclear. Section 3.3 mainly presents CMSA as an aggregation function without explaining how visual reliability is estimated or how modality contributions change with corruption severity.

  3. The scope of baseline comparisons and the added contribution need clarification. Thumbnail-based methods receive downsampled corrupted images, while video methods receive repeated copies of a single frame, altering the auxiliary information these methods rely on. The results should therefore be interpreted within this restricted-input setting. Moreover, although the paper discusses existing bitstream captioning methods such as Cibic, it does not compare against integrating them into the same restoration pipeline. The gains over existing bitstream semantic modeling thus remain unclear.

Questions for authors
  1. What proportion of samples completely fail to decode, and are all such samples included in the final metrics? How is the diffusion model’s input initialized in this case, and do separate evaluations demonstrate fidelity to the original layout and details?
  2. How does CMSA estimate visual reliability and adjust modality contributions? Is there a comparison with fixed fusion using the same inputs and aggregation model to isolate the effect of adaptation?
  3. What are the actual bit-flip or byte-deletion rates within each corruption range, and are file headers protected? How are BCoL’s pixel-level ground-truth corruption masks constructed?
2 comments

1. Reply to W1 & Q1: Reporting restoration results under complete decoding failure

Thank you for your valuable comment. The proportion of test samples that completely fail to decode is 35.13% (562/1,600), and all such samples are included in the final metrics in our experimental results. In this case, the diffusion model’s input is initialized using an all-black image, with bitstream-derived semantics guiding the restoration. We separately report restoration results for these decoding failure samples as follows. The results show that our proposed model outperforms the competing methods on all eight metrics, demonstrating better restoration fidelity under complete decoding failure.

MethodEvaluation samplesPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
OursAll samples15.86290.519831.597514.89100.36800.42310.72100.7103
OursDecoding failure samples10.29030.136163.380222.58760.61860.71860.45680.3894
LD-RPS (ICCV 2025)Decoding failure samples7.36250.023791.611343.50950.84970.82480.27090.0588

2. Reply to W2 & Q2: Clarifying CMSA’s corruption-adaptive mechanism

We introduce a visual reliability estimation module. For each visual token feature vᵢ, a lightweight network fθ takes its concatenation with the image’s mean visual feature v̄ and predicts reliability in [0,1]. This prediction determines the token’s weight:

wi=0.1+0.9fθ([vi;vˉ]).w_i = 0.1 + 0.9 f_\theta\left(\left[\mathbf{v}_i;\bar{\mathbf{v}}\right]\right).

The weighted visual tokens are fused with text tokens to generate the semantic description for restoration. Lower weights reduce the contribution of unreliable visual information during fusion. Training jointly optimizes reliability prediction, supervised by clean–corrupted image differences, and description generation.

We compare our corruption-adaptive fusion and fixed fusion using the same inputs and aggregation model, as shown below. The results demonstrate the advantage of our fusion mechanism over fixed fusion in semantic fusion. We additionally evaluate the fusion comparison on the Flickr8k dataset, which also shows the same advantage of our corruption-adaptive fusion mechanism.

DatasetMethodBLEU-1↑BLEU-4↑ROUGE-L↑CIDEr↑Token-F1↑Micro-content-F1↑BERTScore-F1↑
ADE20KFixed fusion0.40850.18590.41590.20620.49220.33010.9766
Ours0.45410.22480.43670.29340.50280.34350.9776
Flickr8kFixed fusion0.55650.37230.54940.74100.57360.45700.9370
Ours0.58830.39660.58280.80670.60780.48740.9865

3. Reply to W3: Clarifying the baseline comparison scope and the contribution of bitstream semantics

Most evaluated methods require neither thumbnails nor reference frames, and BiNar also outperforms these methods, showing that its advantage is not limited to comparisons involving modified auxiliary inputs. We additionally include thumbnail-based and video-based methods to broaden the comparison. Following your suggestion, we will clarify the input conditions for these two categories in the manuscript and interpret their results accordingly.

We additionally evaluate on Flickr8k and conduct the pipeline comparison on both datasets, integrating Cibic into the same restoration pipeline by replacing only our bitstream semantic model. The results are shown in the following table. Our pipeline outperforms the Cibic pipeline on all eight metrics on both datasets, demonstrating improvements in both restoration fidelity and semantic consistency.

DatasetMethodPSNR↑SSIM↑MAE↓VFID↓DreamSim↓LPIPS↓CLIP-I↑DINO-I↑
ADE20KCibic pipeline14.30660.456736.525917.04170.39350.44740.67900.6470
Our pipeline15.86290.519831.597514.89100.36800.42310.72100.7103
Flickr8kCibic pipeline14.29370.576330.94559.09200.25160.34760.78080.7319
Our pipeline16.10840.613626.40157.50460.21430.31540.81240.7682

4. Reply to Q3: Clarifying corruption rates, header protection, and ground-truth masks

For each corruption range, we use a 0.01% bit-flip rate and a 0.5% byte-deletion rate. We do not protect file headers.

We construct BCoL’s pixel-level ground-truth corruption masks by comparing the decoded corrupted and clean images, marking pixels whose maximum absolute RGB difference exceeds 12 on the 0–255 scale.

We will add these additional experiments and clarifications into the camera-ready version.

We sincerely thank you for your valuable comments. We have conducted additional experiments and provided further clarifications to address your concerns. We would be pleased to address any remaining questions or concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank the reviewers for their time, effort, and valuable feedback. In response to the concerns of the reviewers, we conducted extensive additional experiments and provided detailed technical clarifications, which enhanced the quality, rigor, and reliability of our work. If further clarification is needed, please feel free to let us know and we are happy to provide more information.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.