acceptodds
Under review as a conference paper at ICLR 2027

Self-Healing IMages: Unifying Manipulation Detection and Recovery via Latent Self-Embedding

Abstract

Generative image editing raises two questions: where was an image manipulated, and what content was overwritten? Passive localization addresses the first but cannot recover source content that is no longer present. Existing self-recovery methods preserve this content through embedded representations, often coupling payload construction and spatial dispersion at image resolution. We introduce SHIM (Self-Healing IMages), which embeds a compact, spatially shuffled source latent to support both localization and recovery. The receiver decodes the extracted latent in two orders: direct decoding retains manipulation-related inconsistencies, while inverse-shuffle decoding restores source alignment. A locator combines both views with the received image, and a mask-guided refiner corrects the aligned estimate within the predicted repair support. On valAGE-Set, SHIM maintains high container fidelity while achieving accurate manipulation localization and effective source-content recovery after channel degradation. A separate evaluation on COCO, ImageNet, and CelebA-HQ shows stronger dataset-averaged recovery than Imuge+ and ReImage across diverse channel conditions. Controlled analyses show that the direct view complements the aligned view and that learned refinement improves recovery over raw composition. These results support compact latent self-embedding as a common representation for proactive localization and source-conditioned recovery.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image self-recovery, Image manipulation localization, Generative image editing, Proactive image forensics, Watermark

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_qV4HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SHIM embeds a compact source representation into an image before distribution to support subsequent tamper localization and recovery. A frozen DC-AE encodes the source, and an INN embeds its spatially shuffled latent. At the receiver, direct and inverse-shuffled decoding provide complementary inputs to a locator. Recovery combines source correction, global restoration, and mask-constrained fusion. Experiments cover five manipulation types on valAGE-Set and additional cross-dataset evaluations. The reported results show strong container fidelity, localization accuracy, and full-image recovery quality.

Critical strengths
  1. SHIM reports strong end-to-end results on the complete 1,000-image valAGE-Set split. It achieves .910 localization F1 and 28.70 dB recovery PSNR under degradation, while maintaining 39.13 dB container PSNR. Evaluating actual UINT8 images makes these results more practically relevant than evaluations restricted to floating-point images.

  2. The receiver reuses one extracted source representation for both localization and recovery. Direct decoding preserves manipulation-related inconsistencies, while inverse-shuffled decoding supplies source-aligned content. This is a coherent design with a clear purpose for each view.

Critical weaknesses
  1. The contribution beyond the closest prior work is not sufficiently isolated. ReImage already combines shuffled self-embedding, INN-based extraction, contextual enhancement, and mask-based recovery [A]. RecoverMark also compresses protected content into a latent representation before embedding it for localization and recovery [B]. SHIM’s clearest distinction is therefore dual-order decoding within a compact latent framework. However, Table 4(a) removes inputs from a fixed trained locator rather than comparing independently trained single-view and dual-view models. The clean F1 gain from adding the direct view is .951 to .955, and the degraded gain is .897 to .910. These results show that the trained locator uses the additional input, but do not establish its advantage over a properly trained aligned-only alternative. Table 4(c) also changes representation, payload size, and training together, leaving the benefit of compact latent self-embedding unresolved.

  2. The evaluation leaves important uncertainty about both visual fidelity and recovery quality. Figure 5 shows pronounced, spatially regular patterns in SHIM’s amplified embedding residuals, with faint texture also visible in smooth regions of the protected images. Appendix A.2 explicitly mentions minimizing grid artifacts during checkpoint selection, but does not explain their source or establish whether the remaining patterns are perceptible under normal viewing conditions. High average PSNR does not resolve this concern because it does not account for the spatial organization of errors.

The recovery comparisons also contain confounding factors. Tables 1–2 combine different stochastic generation batches for several baselines. Table 3 retains different channel orders, JPEG implementations, and quantization paths; notably, ReImage receives degradation before tampering, whereas SHIM receives it afterward. Finally, SHIM restores pixels outside the predicted repair region, so its full-image PSNR improvements can include general restoration rather than recovery of overwritten content. Separate measurements inside and outside the manipulated region are needed to distinguish these effects.

Reference

[A] Kim, M., & Seo, P. H. (2026). Robust image self-recovery against tampering using watermark generation with pixel shuffling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Findings (pp. 8877–8886).

[B] An, H., Ye, X., Hua, G., Tao, Y., Cao, H., Yu, X., & Fang, Y. (2026). RecoverMark: Robust watermarking for localization and recovery of manipulated faces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8587–8597).

Questions for authors

Does dual-order decoding outperform an independently trained aligned-only locator with comparable capacity and training budget? Can a controlled representation comparison separate the effect of the compact latent from changes in architecture and training?

What causes the regular patterns in Figure 5, and are they perceptible under normal viewing conditions? Please provide native-resolution crops, particularly for smooth regions, to support the claimed visual fidelity.

Do the recovery gains persist under a common evaluation pipeline with matched channel order, degradation assignments, JPEG implementation, and delivery quantization? Please also clarify whether the earlier threshold/post-processing sweep used the reported valAGE-Set test split.

How much of the recovery improvement occurs inside the annotated manipulated region? Separate inside/outside errors and a global-restoration-only comparison would clarify the contribution of embedded source information.

Reviewer_x52KOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SHIM embeds a shuffled latent representation of the source image in a protected image. At the receiver, two decoding orders provide localization cues and a source-aligned estimate; a learned recovery module combines that estimate with the received image. The evaluation covers several manipulations and channel degradations on valAGE-Set, plus cross-dataset experiments.

Critical strengths

The motivation is compelling: after an edit overwrites a region, the received pixels cannot in general reveal the original content. SHIM gives the receiver a source-derived signal to work with, and the reported recovery improvements are substantial. I also appreciate that the authors state where their comparisons are not directly matched.

Critical weaknesses

The ablations do not yet show which parts of the system are needed. In Table 4(a), the authors remove a decoded view while keeping the locator weights fixed; that is not the same as training a locator that only sees the aligned view. Table 4(b) compares raw fusion with the full recovery module, so local correction, global restoration, and fusion all change at once. The current results support the deployed SHIM system, but give less evidence for the specific architectural explanation. The main comparisons use different stochastic edit batches across methods (Table 1 and Appendix D.1). That matters for generative edits, where two nominally identical attacks need not produce equally difficult images. The recovery metric is also full-image PSNR, which mixes the edited area with localization errors and untouched pixels. I therefore find the headline ranking promising, but not yet decisive about recovery of the overwritten content.

Questions for authors
  1. For Table 4(a), could you train an aligned-view-only locator from the same initialization and training data, then compare it with the dual-view locator on both Clean and Degradation? This would tell us whether the direct view adds useful information after the single-view model has been allowed to adapt.
  2. In Table 4(b), which part accounts for the improvement over raw fusion? Please separate the local correction, global restoration, and mask-guided fusion contributions; otherwise it is difficult to know what the learned recovery module is doing.
  3. Could the strongest recovery baselines be compared under matched attack and channel conditions? Since each method produces a different protected image, the corrupted inputs will differ. The comparison should use the same source images and edit masks, attack types and settings, and channel operations and parameters. Per-image paired results and a metric focused on the edited regions would make the recovery comparison more informative.
Reviewer_E8zWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SHIM embeds a compact, spatially permuted autoencoder representation into an image before distribution. After manipulation, two decoded views are used to locate the modified region, and a local/global refinement module recovers the image. The paper evaluates protection quality, localization, and recovery on valAGE-Set and additional datasets, and reports improvements over existing self-embedding methods.

Critical strengths

[s1] The method connects localization and recovery through the same embedded representation. This is a sensible design for recovering manipulated images without requiring an external copy of the original at the receiver, and gives the work a clear practical purpose.

[s2] The reported improvements cover protection quality as well as localization and recovery. The attention to UINT8 storage, including adapting a baseline to this setting, makes the evaluation more relevant to how protected images would actually be stored and distributed.

Critical weaknesses

[w1] The experiments leave uncertainty about the source of the advantage. The dual-view ablation removes inputs from an already trained locator rather than comparing separately trained alternatives. The cross-dataset comparison also differs across methods in the order of tampering and channel degradation and in the JPEG implementation. These controls matter because they determine whether the gains can be attributed to the proposed design under comparable conditions.

[w2] The training description is not yet sufficient to reproduce the main model. The reported continuation stages do not give the complete training history, and the main-model losses, their weights, and initialization sources need to be specified together. The separate full-RGB baseline description does not resolve these omissions. This limits confidence that the central results can be independently reproduced.

Questions for authors
  1. Clarification: Could you provide the complete training procedure for the main INN, locator, and refiner, including their losses, initialization sources, and stages preceding the reported continuation?

  2. Evidence needed for assessment: Does the dual-view locator outperform an aligned-only locator when both are trained separately under the same budget?

  3. Clarification and evidence needed for assessment: Were the degradation pipelines and threshold-selection procedures matched across methods? If not, is there a comparison using the same channel order and JPEG implementation, with thresholds selected on separate validation data?

Reviewer_euqJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SHIM, which embeds a compact, spatially shuffled latent representation of the original image before distribution and uses dual-order decoding to support tamper localization and source-content recovery. The method combines local correction, global restoration, and mask-constrained fusion, achieving strong localization and full-image recovery performance under the evaluated manipulation and degradation conditions.

Critical strengths

1.Meaningful research objective. The paper addresses both tamper localization and recovery of overwritten content, offering clear value for proactive image protection.

2.Coherent design. Compact latent embedding, dual-order decoding, and mask-constrained fusion serve clearly defined purposes, using the same embedded information to support both localization and recovery.

Critical weaknesses

1.Recovery evaluation could be more specific. Although I am not an expert in this field, I wonder whether recovery quality should be evaluated separately inside and outside the tampered regions. The reported full-image PSNR, SSIM, and LPIPS may partly reflect denoising of untampered regions, rather than reconstruction of overwritten content.

Questions for authors

Could the authors clarify whether separate evaluation inside and outside the tampered regions is needed to distinguish source-content reconstruction from global restoration? If available, could they provide these results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.