acceptodds
Under review as a conference paper at ICLR 2027

RoGRE: Image Manipulation Localization as Controlled Hypothesis Revision

Abstract

Image manipulation localization (IML) requires combining heterogeneous evidence with different localization capabilities and failure modes. Existing methods typically fuse such representations as equal-status features, obscuring how each source should modify the current prediction. We instead formulate IML as controlled hypothesis revision: localization begins from an initial estimate, and complementary evidence progressively corrects its remaining errors. Based on this view, we introduce RoGRE, which maintains a shared logit-valued localization state and revises it through role-specific stages. DINOv3 first establishes an initial hypothesis, SAM2 features correct its spatial organization, and forensic residual features refine manipulation evidence that remains weakly represented. Each refinement proposes a bounded signed correction whose magnitude is regulated by a gate conditioned on both the incoming state and stage-specific evidence, allowing informative revisions while limiting unreliable updates. The evolved state is finally decoded by SAM2 into the manipulation mask. RoGRE achieves mean pixel-level F1 scores of 0.888 and 0.813 under the CAT-Net and MVSS-Net protocols, respectively. Ablations further validate the correction order and state-dependent gating, supporting controlled hypothesis revision as an effective approach to heterogeneous evidence integration for IML.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Image Manipulation Localization

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_vs2AOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors offer their own perspective on solving the Image Manipulation Localization (IML) task. Instead of combining heterogeneous features as equals, the authors propose a sequential refinement of the manipulation mask hypothesis. RoGRE, which maintains a shared localization state through initial proposal via DINOv3, spatial correction via SAM2, and forensic refinement via Constrained Bayar and Zero-DC SRM, using gated mechanism to generate mask manipulation.The authors evaluated RoGRE under two standard protocols: CAT-Net (four-source) and MVSS-Net, using pixel-level F1 as the evaluation metric. The method achieves mean F1 scores of 0.888 and 0.813 under the CAT-Net and MVSS-Net protocols, respectively.

Critical strengths

The problem formulation considered in the paper is relevant due to its applicability in various security and image verification systems. RoGRE demonstrates improved F1 scores on both major IML evaluation protocols compared with many recent state-of-the-art methods.

Critical weaknesses
  • The Initial Localization Proposal is instantiated only with DINOv3, although several other foundation models are conceptually well suited for this role. Since no alternative backbone is evaluated, it remains unclear whether the reported gains stem from the proposed controlled-revision framework or from the specific choice of DINOv3.
  • The choice of SAM2 is not justified. Both the Spatial Hypothesis Correction stage and the final decoder rely on SAM2, whereas SAM 3 (Carion et al., 2025) was publicly available well before the submission and also supports mask prompts. The paper neither motivates the choice of SAM2 nor evaluates SAM 3 in this role.
  • Outdated forensic components. LFFE combines Bayar constrained convolutions (2016) with SRM residuals (2012). Noiseprint++ (Guillaro et al., CVPR 2023) serves essentially the same purpose, extracting a forensic noise residual, and originates from TruFor, which the authors already use as a baseline. The paper does not discuss why this hand-designed combination is preferred over a stronger learned residual extractor.
  • (minor) The third contribution (the gated residual mechanism) is effectively a component of RoGRE, which is already covered by the second contribution.
Questions for authors
  • Have the authors tried other models for the Initial Localization Proposal, e.g., CLIP, SigLIP 2, or C-RADIOv4? The latter is distilled from SigLIP 2, DINOv3, and SAM 3, so it should provide an informative general-purpose representation. It could also test whether separate semantic and structural backbones are needed at all.
  • How does RoGRE perform when SAM2 is replaced with SAM 3 in the Spatial Hypothesis Correction stage (and in the final decoder)?
  • How does performance change if Bayar+SRM in LFFE is replaced with, or complemented by, Noiseprint++?
6 comments

Response to Reviewer vs2A

We appreciate the reviewer’s thoughtful feedback and suggestions regarding alternative representations. Below we report the results for the evaluated configurations and clarify our contribution statement.

Table R4-1. Comparison of alternative representations under the four-source CAT-Net setting. All configurations follow the same training protocol. Each run is evaluated using its final-epoch checkpoint. Mean F1 reports mean ± standard deviation across five seeds, with CASIAv1, Columbia, NIST16, and Coverage weighted equally within each run.

ConfigurationMean F1 ↑Authentic pixel FPR ↓
RoGRE (DINOv3 + SAM2 + LFFE)0.888 ± 0.0030.6433%
RoGRE with C-RADIOv4 replacing DINOv30.859 ± 0.0050.6538%
C-RADIOv4 + SAM2 decoder0.802 ± 0.0120.7923%
RoGRE with SAM3 replacing SAM20.890 ± 0.0030.6075%
RoGRE with Noiseprint++ added to LFFE0.883 ± 0.0060.9558%
RoGRE with Noiseprint++ replacing LFFE0.871 ± 0.0080.9312%

1. Alternative Initial Backbone and Separate Spatial Encoder

We evaluate C-RADIOv4 in two configurations. The first replaces DINOv3 as the initial-proposal backbone while retaining the SAM2 image encoder, spatial correction, forensic refinement, and final decoding. The second removes the SAM2 image encoder and combines C-RADIOv4 with SAM2 decoding.

Replacing DINOv3 with C-RADIOv4 yields 0.859 mean F1, compared with 0.888 for the reference configuration. Within the C-RADIOv4 configurations, removing the separate SAM2 image encoder lowers mean F1 from 0.859 to 0.802, a 5.7-point decrease, and increases authentic-image FPR from 0.6538% to 0.7923%. This difference compares the complete representation configurations with their respective decoder interfaces. The tested configuration retaining the separate SAM2 encoder performs better.

We would like to clarify that the contribution of the correction mechanism is evaluated separately through the same-backbone ablations in Table 2. With the same DINOv3 and SAM2 backbones, moving from A1 to Full improves F1 from 0.854 to 0.888. The representation replacement experiments evaluate backbone choices, whereas the controlled ablations support the contribution of the correction design.

2. SAM2 Versus SAM3

We replace SAM2 with SAM3 in both spatial correction and final decoding. The resulting mean F1 is 0.890 ± 0.003, compared with 0.888 ± 0.003 for SAM2. The observed difference is small relative to the reported seed variation, so these results alone do not establish a clear advantage for SAM3.

For context, the published full-model sizes are approximately 848M parameters for SAM3 and 80.8M for SAM2 Hiera-B+. These figures describe the foundation models rather than measurements of the complete RoGRE pipelines.

These results support SAM2 as a suitable choice for the evaluated setting and show that the correction framework can also be instantiated with SAM3.

3. Learnable Forensic Features and Noiseprint++

We would like to clarify that LFFE is a trainable forensic encoder. Its Bayar filters are optimized under prediction-error constraints, and its SRM-initialized filters remain learnable under a zero-DC constraint. These residual priors guide feature extraction, while subsequent learned layers form the final forensic representation.

Following your suggestion, we evaluate Noiseprint++ both as an additional input to LFFE and as a replacement for LFFE. Noiseprint++ participates in task-specific training in both variants. Adding Noiseprint++ yields 0.883 F1, while replacing LFFE yields 0.871, compared with 0.888 for LFFE alone. Both alternatives also increase authentic-image FPR from 0.6433% to 0.9558% and 0.9312%, respectively.

Neither alternative outperforms LFFE in the tested configurations. Combined with the Bayar and SRM ablations in Table 2, these results support retaining the learned LFFE representation for forensic correction.

Reviewer_igNnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an image manipulation localization method that cascades by combining DINOv3, SAM2, and a lightweight forensic fingerprint encoder (LFFE).

Critical strengths

The reported localization results seem good. The writing is clear.

Critical weaknesses

1 . The experiments on SID-Set are not fair. RoGRE was trained on SID-Set, while the comparison methods have not received this training. The tampered-image F1 is 0.420 for DINOv3-IML, 0.532 for zero-shot RoGRE, and 0.964 for SID-Set-trained RoGRE. So the large improvement mainly appears after introducing SID-Set training.

  1. I am not fully convinced by the claimed division into Initial Localization Proposal, Spatial Hypothesis Correction, and Forensic Residual Refinement. The paper does have role-specific losses and ablations. However, showing that each component improves F1 is still different from showing that it performs the claimed type of reasoning or correction.

  2. Table 5 also raises some concerns. The spatial and forensic updates mainly recover false negatives, by 22.85% and 30.39%, while their false-positive suppression is only 2.39% and 1.98%. Most false-positive filtering comes from the final SAM2 decoder. These observations support complementary foreground expansion followed by decoder filtering, but the distinction between spatial correction and forensic refinement remains less clear. What specific errors are handled by one stage but not the other?

  3. The novelty looks limited. The main operation is two gated residual updates to a localization state, followed by SAM2 decoding. Prediction refinement has already been studied in works such as CascadePSP and SegRefiner. These methods use different mechanisms, but the general idea of refining an evolving prediction is established. I understand that RoGRE combines different evidence sources. Still, what new principle do we learn from this particular cascade, beyond obtaining better results with strong pretrained features? The ordering ablation identifies a useful configuration, but the additional insight is not clear enough to me.

  4. The gate's reliability interpretation is not directly validated. A learned sigmoid gate and a candidate update bounded to [-2, 2] can regulate update magnitude, but do the gates actually suppress harmful corrections?

Questions for authors

See Weaknesses.

5 comments

Response to Reviewer igNn

We appreciate the reviewer’s careful reading and constructive feedback. Below we address the questions regarding comparison fairness, stage roles, and the contribution relative to existing refinement methods.

1. SID-Set Comparison Settings

We would like to clarify that the cross-method comparison in Table 7 is restricted to zero-shot results, with no SID-Set training applied to RoGRE or the compared methods. The SID-Set-trained row evaluates target-domain adaptation separately. We have revised Table 7 to make this distinction explicit and restricted the best-result highlights to the zero-shot group.

The larger performance gains after SID-Set training reflect adaptation with target-domain supervision. Common Response C4 reports both settings and discusses the remaining limitations in zero-shot full-synthetic image recognition.

2. Evidence for Complementary Stage Roles

To examine the stage roles beyond aggregate F1 gains, we provide regional error correction statistics using masks decoded from successive states via the shared decoder. Common Response C1, Table C1-1, shows that spatial correction primarily improves boundaries, whereas forensic refinement improves foreground interiors and background regions.

Fig. 7 provides qualitative examples of these effects. Spatial correction clarifies the fence gaps in (a), while forensic refinement removes the isolated ring-shaped false positive in (c) and recovers the thin annotated extension in (d). These observations complement the regional statistics.

The intended roles are complementary rather than mutually exclusive. Both stages can recover false negatives because a missed region may require spatial completion, forensic evidence, or both. Their shared capacity to address this error category is consistent with their complementary regional effects.

3. Role of Final Decoding

Table 5 illustrates foreground recovery during state updates accompanied by substantial false-positive suppression during final decoding, where the decoder suppresses 69.04% of the false positives present in the incoming evolved state.

We would like to clarify that Table 5 evaluates directly thresholded intermediate states, whereas Fig. 5 evaluates successive states through the shared fixed decoder. The latter shows that corrected states improve the decoded region and boundary metrics. The regional analysis in Common Response C1 and the visualizations in Fig. 7 also operate within this shared-decoder setting.

Together, these results support a complementary division of labor: the correction stages refine the localization evidence supplied as a dense prompt, while the image-conditioned decoder performs substantial false-positive filtering.

Reviewer_2NaKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

RoGRE proposes a role-structured model for image manipulation localization. A DINOv3 branch forms an initial localization map; SAM2 features apply a gated spatial correction; and a lightweight forensic fingerprint encoder proposes a second gated residual update before SAM2 decodes the final mask. The authors evaluate pixel-level F1 under four-source CAT-Net and MVSS-Net training protocols across standard manipulation datasets, as well as degradation tests and a supplementary SID-Set evaluation. RoGRE reports mean F1 of 0.888 versus 0.847 for DINOv3-IML under the four-source protocol, and 0.813 versus 0.774 under MVSS-Net. Ablations support the sequential correction stages and gated update mechanism. The zero-shot SID-Set results are much weaker for fully synthetic images than for localized tampering.

Critical strengths
  1. RoGRE improves over a strong DINOv3-IML baseline under both reported training protocols, and its component ablations show that the spatial and forensic updates contribute complementary gains. This makes the architectural proposal more convincing than a fusion design supported only by aggregate performance.
  2. The paper reports degradation tests, inference cost, and an additional evaluation across localized AIGC edits, fully synthetic images, and real images. The supplementary analysis also makes a notable limitation explicit: zero-shot full-synthetic recognition remains weak.
Critical weaknesses
  1. The method depends on large pretrained DINOv3 and SAM2 encoders; the reported full model has about 385M parameters. The added correction stages are relatively inexpensive compared with the base, but the complete system is not lightweight, which limits claims about practical efficiency.
  2. The main evidence comes from established manipulation benchmarks and fixed evaluation protocols. The supplementary SID-Set evaluation shows a large gap between localized tampering and fully synthetic images in the zero-shot setting, leaving robustness to broader image origins and real-world processing chains unresolved.
Questions for authors

Were all baseline scores in both protocols re-evaluated with the same data splits, preprocessing, and fixed 0.5 threshold as RoGRE, or were some taken from published reports? This affects how directly the cross-method gains can be compared.

3 comments

Response to Reviewer 2NaK

We appreciate the reviewer’s thoughtful feedback and recognition of our correction stages and diagnostic analyses. Below we address the questions regarding baseline comparisons, computational cost, and broader robustness.

1. Baseline Comparability and Evaluation Protocol

We would like to clarify that the external baseline scores in Table 1 are taken directly from published studies rather than re-evaluated under identical settings, thus retaining their original training and evaluation protocols. Our internal ablations use a shared training setup, except for the components or objectives explicitly varied, to assess individual design choices.

For RoGRE, evaluation is performed at a fixed threshold of 0.5, using the final-epoch checkpoint without validation- or test-based selection. Common Response C3 provides details on the training datasets, split source, and metric aggregation.

2. Full-Model Cost and Correction Overhead

We agree that the computational cost should be considered at the full-system level. Our use of “lightweight” refers specifically to the additional correction modules. Table 6 reports 385.02M total parameters and 12.76M trainable parameters for the complete model. While the pretrained backbones are frozen except for LoRA adaptation, their forward passes still contribute to overall inference cost.

Relative to A1, which uses the same pretrained encoders and decoder, these modules add 1.09M trainable parameters and 7.29 ms of inference latency (a 12.4% increase) under identical profiling conditions. The revised efficiency text explicitly distinguishes this incremental overhead from the total system footprint.

3. Generalization and Robustness Beyond Standard Benchmarks

Our evaluation covers standard manipulation benchmarks, SID-Set, and individual image degradations. Broader generalization across image origins and combined processing chains remains to be evaluated. Common Response C4 clarifies the distinction between zero-shot transfer on SID-Set and adaptation following domain-specific training. While the zero-shot model transfers to localized AIGC edits, its full-synthetic F1 of 0.013 highlights a limitation in zero-shot generalization to full-image synthesis. The gains observed after SID-Set training reflect target-domain adaptation rather than zero-shot capabilities.

To further assess robustness, Fig. 4 evaluates performance under individual degradation types, including Gaussian blur, Gaussian noise, and JPEG compression across multiple severity levels. Investigating combined acquisition and processing chains remains an important direction for future work.

Thank you again for your constructive feedback and positive assessment. We hope these clarifications address your concerns and make the scope of the reported results clearer.

Reviewer_ppFMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RoGRE treats image manipulation localization as sequential revision of a shared logit map. A LoRA-adapted DINOv3 ViT-L/16 produces an initial map z(0)z^{(0)}. SAM2 Hiera-B+ features and a Bayar/SRM forensic encoder then add gated, bounded corrections (gt⊙αttanh⁡(htΔ(Ct))g_t \odot \alpha_t \tanh(h^{\Delta}_t(C_t))), producing z(1)z^{(1)} and z(2)z^{(2)}, and the SAM2 decoder decodes z(2)z^{(2)} as a dense prompt. Stage-specific auxiliary losses supervise z(0)z^{(0)}, z(1)z^{(1)} (boundary) and the forensic update (residual-error supports). RoGRE reports mean pixel-F1 of 0.888 (CAT-Net protocol) and 0.813 (MVSS-Net protocol), compared with 0.847 and 0.774 for DINOv3-IML, supported by component, ordering, gating and loss ablations and by state-decoding diagnostics.

Critical strengths
  1. Strong empirical results. RoGRE has the best average under both protocols, is best on all five MVSS-Net test sets, and improves NIST16 from 0.636 to 0.759 under CAT-Net. The internal A1 baseline (DINOv3 proposal plus SAM2 decoder) controls for the second foundation encoder, and the correction stages raise it from 0.854 to 0.888 at modest overhead (+1.09M parameters, +7.3 ms). These results show that the architecture is effective, and they are the main reason my rating is not lower.

  2. Thorough, candid diagnostics. Fixed-decoder state decoding (Fig. 5), per-transition error conversion (Table 5), authentic-image FPR, threshold sensitivity, mask-area analysis and AIGC transfer make the method's actual behavior inspectable. This transparency is valuable in its own right, and it is also what allows the mechanism claims to be checked.

Critical weaknesses
  1. The central claim, that controlled hypothesis revision explains the gains, is not established by the ablations. The paper's contribution is framed as a mechanism (role order and state-dependent gating), not as an architecture, so this is the claim that needs support.

    • Small margins without variability. Over A1 the revision design adds +3.4 pp, but the comparisons that actually test the mechanism are much smaller: +1.3 over parallel updates (P1) and +1.6 over state-agnostic gates (P3). These are five-seed means with no reported variability, on test sets as small as 100 (Coverage) and 180 (Columbia) tampered images. P2 (0.868) also falls below both single-stage variants (A2 0.870, A3 0.872), which the ordering account does not explain.
    • Supervision and capacity are not controlled. The ordering comparison does not clearly control for supervision: Lfor\mathcal{L}_{\text{for}} is defined on error supports derived from p(1)p^{(1)}, and the paper does not say how the role-specific losses are reassigned in P1 and P2. These auxiliaries alone contribute 1.8 pp (Table 4), the same scale as the ordering gaps. Likewise, the +1.6 pp from adding spatial correction to A3 is not separated from the added capacity and boundary supervision, and the full model has more trainable parameters than A1 (12.76M for Full vs. 11.67M for A1, Table 6).
    • The key fusion baseline is underspecified. P0's implementation and capacity are not described in enough detail to judge it, although it is the direct test of the "equal-status fusion" that Fig. 1(a) positions the paper against.
    • The gating claim is not verified. The abstract states that the gates limit unreliable updates. Table 5 instead shows that the updates mainly recover foreground (FP suppression of about 2%) while authentic-image FPR rises from 0.60% to 3.69%, and that the decoder removes 69.04% of false positives. No error-level comparison with ungated corrections (P4) is provided to support the claim.
    • Novelty relative to refinement methods is unclear. The paper does not explain what its update rule adds beyond gated residual refinement and prior iterative-refinement methods (e.g., iterative error feedback, CascadePSP), particularly in state conditioning and stage-specific supervision.
  2. The external comparison and evaluation coverage are insufficient to judge the reported margins.

    • The paper does not state which external baselines were retrained with matching data, splice augmentation and checkpoint selection, how F1 is aggregated, or what validation data were used.
    • On the traditional benchmarks, the main metric excludes the 1,673 authentic test images, and no comparative authentic-image FPR or pixel AUROC/AP is given. It is therefore unclear whether the localization gains come with reliable behavior on unmanipulated inputs; the SID-Set comparison (Table 7) covers a different setting.
    • As a minor point, Fig. 4 omits DINOv3-IML, which limits the robustness comparison against the strongest baseline.

Overall assessment. The results are strong, and the A1 comparison shows that the correction stages add something beyond the second encoder. But the paper's contribution is the claim that ordered, state-gated revision is why the method works, and the current evidence does not show that. The decisive comparisons are small, have no reported variance, and do not control for the stage-specific losses, and the gating claim is not tested at the level of errors. I lean toward rejection in the current form. Seed variance, matched supervision in P0–P2, and a clearly specified fusion baseline could change my assessment.

Questions for authors

The following questions bear directly on my recommendation. Questions 1–3 address whether the central mechanism claim holds; my assessment depends mainly on them.

  1. Supervision in the ordering ablations. In P0–P2, how are the boundary loss, the forensic loss and the residual-error supports defined, given that Lfor\mathcal{L}_{\text{for}} depends on p(1)p^{(1)}? Does the S$\to$F order still outperform P1 and P2 when the role-specific auxiliary losses are removed or matched across variants? If the ordering advantage does not survive matched supervision, the paper's main claim is not supported.
  2. The fusion baseline. What exactly is concatenated in P0, how is it mapped to the dense prompt, and what is its trainable capacity relative to the full model? A capacity-matched feature-level fusion of the three sources, decoded by the same SAM2 decoder, is the comparison needed to show that revision outperforms equal-status fusion. Without it, the contrast drawn in Fig. 1(a) remains untested.
  3. Seed variability. Please report standard deviations over the five seeds for Tables 1, 2 and 4, in particular for Full versus P1–P3. Are the 1.3–1.6 pp gaps larger than seed variation?
  4. The gating claim. Can you compare error-level statistics (false positives introduced, especially on authentic images) between the gated model and the ungated variant P4? This is the evidence required for the claim that the gates limit unreliable updates.
  5. Baseline protocol. Were the external baselines reproduced under your exact protocol (no CompRAISE, 512$\times$512, splice augmentation, threshold 0.5)? Is F1 computed per image or pooled per dataset, and what validation data and checkpoint-selection rule were used?
  6. Authentic images. On the traditional benchmarks, please report authentic-image FPR and pixel AUC/AP (including authentic images) for RoGRE, A1 and DINOv3-IML.
  7. Novelty. What does the proposed update add beyond existing gated or iterative refinement methods, specifically in its update rule, state conditioning and supervision?
8 comments

Response to Reviewer ppFM

We thank the reviewer for their valuable feedback and constructive suggestions, which have helped us improve the manuscript. We address each point below, with shared analyses provided in the Common Response.

1. Ordering Effects and Auxiliary Supervision

The S→F ordering advantage persists when all auxiliary losses are removed. We provide this comparison below and clarify how supervision is assigned in the original ordering ablations.

Ordering comparison without auxiliary supervision.

We set λD=λS=λF=0\lambda_D=\lambda_S=\lambda_F=0 and train all three variants using only the final decoder mask objective. S→F corresponds to A7 in Table 4.

Table R1-1. Ordering comparison without auxiliary supervision. Results report mean ± standard deviation across five seeds.

VariantMean F1 ↑Mean IoU ↑Mean Acc. ↑
S→F (A7)0.866 ± 0.0080.772 ± 0.0090.970 ± 0.004
P1: Parallel0.847 ± 0.0070.761 ± 0.0050.964 ± 0.007
P2: F→S0.832 ± 0.0080.746 ± 0.0050.963 ± 0.006

S→F outperforms parallel and reversed updates by 1.9 and 3.4 F1 percentage points, respectively. Thus, the ordering advantage persists without role-specific auxiliary supervision.

Supervision in the original ablations.

To clarify how supervision changes with the update order, Appendix A now specifies the following:

  • P1: Parallel updates. Both branches start from z(0)z^{(0)}. Boundary supervision is applied to the spatially corrected state, while forensic supervision is applied to the independently forensic-corrected state. Forensic supports are constructed from detached z(0)z^{(0)}.
  • P2: F→S. Forensic supports are constructed from detached z(0)z^{(0)}. Forensic supervision follows the first update, and boundary supervision follows the subsequent spatial update.
  • Full and P3. Forensic supports are constructed from detached z(1)z^{(1)} after spatial correction. P3 retains Full’s supervision and removes only the probability and uncertainty inputs from the gates.

Initial-mask supervision and auxiliary weights are shared. P1 and P2 gates use the incoming states associated with their respective update orders.

P2 reaches 0.862 mean F1, below A2 and A3 at 0.870 and 0.873, respectively. Reversing the order also changes the states on which the gates operate. The subsequent spatial update may alter regions already supported by forensic evidence, potentially explaining why P2 underperforms the single-stage variants.

2. Implementation and Capacity of the Fusion Baseline

We would like to clarify that P0 uses the same encoders, LoRA configuration, LFFE, prompt encoder, and decoder training setup as Full. Their total trainable parameter counts are comparable at 12.57M and 12.76M, respectively.

P0 spatially aligns DINOv3, SAM2, and LFFE features and concatenates their 1024, 256, and 64 channels. A two-layer 3×33\times3 convolutional network with channels 1344→64→11344\rightarrow64\rightarrow1 maps these features to a single-channel SAM2 dense prompt. This fusion network replaces the sequential correction path and contains 0.775M trainable parameters, including biases.

P0 retains auxiliary mask supervision on the DINOv3 initial logits. Boundary and forensic supervision are applied to single-channel heads from the SAM2 and LFFE branch features, respectively. The forensic loss supervises the LFFE branch logit using supports derived from detached z(0)z^{(0)}. Auxiliary weights match Full, and the final mask objective supervises the decoded fusion prompt. These supervision targets are now specified in Appendix A.

P0 achieves 0.861±0.0060.861\pm0.006 mean F1, compared with 0.888±0.0030.888\pm0.003 for Full. This 2.7-point gain supports the proposed correction process over the tested feature-concatenation baseline with comparable capacity.

3. Variability Across Random Seeds

We now report mean ± standard deviation across five seeds for RoGRE’s F1 results in Tables 1, 2, and 4. In Table 2, Full exceeds P1, P2, and P3 by 1.5, 2.6, and 2.1 F1 percentage points, respectively, compared with individual configuration SDs of 0.3–0.5 points. These gains exceed the reported seed-variation scale.

We also corrected several reported means to align them with the dataset-level scores. In Table 2, the corrections are A3: 0.872 to 0.873, P1: 0.875 to 0.873, P2: 0.868 to 0.862, P3: 0.872 to 0.867, and P4: 0.860 to 0.854. Full remains 0.888. In Table 4, the mean F1 for λS=0\lambda_S=0 is corrected from 0.879 to 0.877. External baselines retain their published statistics.

Comments on the paper

8 comments

Common Response

We thank all reviewers for their careful review and constructive suggestions. We address the shared questions below and provide responses to each reviewer separately.

C1. Contribution and Stage Roles

Relation to prior refinement methods.

Compared with existing prediction-refinement methods, RoGRE offers a substantially different perspective on manipulation localization through role-aware integration of heterogeneous evidence. Semantic and spatial cues can support plausible but incorrect regions, while forensic residuals can respond to benign texture or post-processing. RoGRE assigns spatial and forensic evidence complementary correction roles through Role-Specific Evidence, uses the incoming probability and entropy to regulate bounded signed updates through State-Conditioned Gating, and aligns mask, boundary, and residual-error objectives with the intended stage behavior through Role-Aligned Supervision.

We have expanded Section 2 to discuss feedback-based correction [1], global/local refinement [2], uncertainty-driven point refinement [3], diffusion [4], and coarse-mask prompting [5]. In contrast, RoGRE introduces evidence roles as the organizing principle of refinement, coordinating spatial and forensic corrections of a shared hypothesis while accounting for their complementary strengths and failure modes. Fusion, ordering, and gating ablations support this organization, with the ordering advantage persisting even without auxiliary supervision. The regional analysis below further examines the complementary effects of the two correction stages.

Evidence for complementary stage roles.

We compare the decoded transitions y0→y1y_0 \rightarrow y_1 and y1→y2y_1 \rightarrow y_2 to evaluate spatial and forensic correction, respectively. All binary masks are obtained using the same decoder and a threshold of 0.5, following the setup in Fig. 7. The ground-truth mask defines three disjoint regions: a boundary band obtained via a 7×77\times7 morphological gradient, the foreground interior outside the band, and the background outside the band. As summarized in Table C1-1, spatial correction primarily improves localization boundaries, whereas forensic refinement improves foreground interiors and background regions. This empirical pattern supports the complementary roles of the two stages, consistent with the qualitative visualizations in Fig. 7.

Table C1-1. Regional net error correction in decoded masks. Net correction is the number of incorrect-to-correct transitions minus correct-to-incorrect transitions, divided by the total pixels in each region and expressed as a percentage. Counts are pooled across all test images.

RegionSpatial net correction rate ↑Forensic net correction rate ↑
Boundary band+0.9066%−0.0024%
Foreground interior−0.0388%+1.0521%
Background−0.0158%+0.0621%

The small negative values indicate local trade-offs rather than uniform improvement across all regions.

C2. Effectiveness of State-Conditioned Gating

We evaluate gating through two complementary comparisons: an inference-time intervention measuring its effect within the trained Full pipeline, and a separately trained ungated model (P4) assessing the benefit of gating when each configuration is trained accordingly.

Inference-time gate intervention.

Using the same Full checkpoint, we hold each stage’s incoming state fixed and compare the gated update gt⊙Δ~tg_t\odot\widetilde{\Delta}_t with the ungated candidate Δ~t\widetilde{\Delta}_t, without retraining. We decode the incoming state and both updated states using the same decoder and image features. Transition rates are measured relative to the decoded incoming mask across the four CAT-Net test datasets at threshold 0.5.

Table C2-1. Effect of gating on harmful updates and error recovery. New error rate is the number of correct-to-incorrect transitions divided by the total previously correct pixels. Error recovery rate is the number of incorrect-to-correct transitions divided by the total previously incorrect pixels. Both are expressed as percentages, with counts pooled across all test images. Authentic pixel FPR is the fraction of pixels predicted as foreground across the 1,673 authentic images.

StageGateNew error rate ↓Error recovery rate ↑Authentic pixel FPR ↓
SpatialEnabled0.1043%7.8183%0.6492%
SpatialDisabled0.1277%5.7225%0.7178%
ForensicEnabled0.0037%7.1626%0.6433%
ForensicDisabled0.0296%0.4019%0.6931%

In the trained Full pipeline, retaining the learned gates reduces the new-error rate by 18.3% for spatial correction and 87.5% for forensic refinement, while increasing error recovery and lowering authentic-image FPR at both stages. This intervention measures the trained pipeline's response to applying gated versus full candidate updates. Because the decoder is nonlinear, a larger prompt update need not produce greater error recovery after decoding.

Comparison with the ungated model.

To complement the fixed-checkpoint intervention, we compare Full with P4, which is trained separately under the same training setup with both gates removed.

Table C2-2. Final-output error comparison with P4. Authentic-image pixel FPR is the fraction of pixels predicted as foreground across 1,673 authentic images. FP and FN fractions are normalized by all test pixels.

Final-output measureGatedUngated (P4)
Authentic-image pixel FPR ↓0.6433%0.7663%
FP fraction of all pixels ↓0.6264%0.7653%
FN fraction of all pixels ↓0.7176%0.7540%

Full reduces authentic-image FPR by 16.1% relative to P4 and lowers both FP and FN fractions. It also improves mean F1 from 0.854±0.0060.854\pm0.006 to 0.888±0.0030.888\pm0.003.

The inference-time intervention demonstrates the benefit of retaining the learned gates within Full; the separately trained P4 comparison establishes the final-performance benefit of gated correction in the tested configurations. Table 2 further supports the value of state conditioning: removing probability and uncertainty inputs from the gates (P3) lowers mean F1 from 0.888 to 0.867.

C3. Evaluation Protocol and Checkpoint Selection

The external baseline scores in Table 1 are taken from their respective published papers rather than obtained by retraining all methods under a unified configuration. These results are grouped by the reported CAT-Net and MVSS-Net protocols, while retaining each baseline’s original training and evaluation settings. They provide context for overall performance. Our internal ablations use a shared training setup, except for the components or objectives explicitly varied, to assess the proposed design choices.

For RoGRE, the four-source CAT-Net setting uses CASIA-v2, FantasticReality, IMD2020, and TampCOCO, excluding CompRAISE. The MVSS-Net setting uses CASIA-v2 alone. We follow the released IMDLBenCo splits, with input resolution, augmentation, and optimization settings detailed in Appendix A. Evaluation uses a fixed threshold of 0.5. Within each test dataset, we pool pixel-level true positives, false positives, and false negatives across tampered images to compute F1. The cross-dataset mean weights these dataset-level scores equally.

For each run, we use the final-epoch checkpoint, without selecting an epoch based on validation or test performance. RoGRE’s F1 results in Tables 1, 2, and 4 report mean ± standard deviation across five random seeds. External baselines retain their published statistics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.