acceptodds
Under review as a conference paper at ICLR 2027

VizAnchor: Decoding Manipulation Intent from Tampering Visualizations via Dual-Anchor Reasoning

Abstract

Data visualizations are widely used for communicating information, but they are also vulnerable to intentional manipulations that induce misleading interpretations. Existing methods focus on locating tampered regions or recovering hidden information, without explaining how the visualization has been manipulated or why the resulting changes may mislead viewers. We propose **VizAnchor**, a framework for visualization manipulation understanding through dual-anchor evidence construction and VLM-based reasoning. In the first stage, VizAnchor constructs a semantic anchor to recover authentic chart information and a spatial anchor to localize tampered regions. In the second stage, three task-specific VLM reasoning steps progressively ground manipulation attributes, reconstruct original and tampered chart narratives, and infer misleading intent by integrating visual evidence with intermediate reasoning results. We further construct a dataset for tampering localization and a dataset for misleading intent inferring. Evaluation shows that VizAnchor accurately localizes manipulations and produces faithful explanations of their manipulation, misleaders, and misleading intents.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Misleading Visualization, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CWEmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents VizAnchor, a framework designed to detect, localize, and explain post-creation semantic manipulations in data visualizations. The core idea is to couple tamper localization with an invertible watermarking module that embeds a metadata payload (a URL referencing the authentic chart). A Vision-Language Model (VLM) sequentially infers the tamper type, reconstructs contrasting chart narratives, and decodes the misleading intent based on the authentic chart, tampered chart, tamper map, and localized comparison. The framework is evaluated quantitatively on evidence construction and manipulation understanding benchmarks, complemented by a user study assessing human-perceived explanation quality.

Critical strengths
  • Leveraging an authentic reference chart retrieved via watermarking provides a viable strategy for semantic manipulation analysis.
  • The multi-stage workflow is systematically structured and promising.
Critical weaknesses
  • Limited Technical Novelty and Confounded Performance Gains. The core technical components are invertible watermarking tamper localization, which are all well-established techniques. This paper thus reads more like a combination of VisDefender and the access to the authentic chart. It is expected that downstream VLM understanding and explanation metrics will improve substantially with the reference. Consequently, it is unclear whether the observed gains reflect algorithmic/reasoning innovations or simply asymmetric information access over reference-blind baselines.

  • Ablations Reveal Minimal Impact of the Proposed Spatial and Reasoning Components. As shown in Appendix Table 10, the vast majority of performance improvements are achieved simply by introducing the authentic reference chart. Adding the spatial tamper map and localized comparisons yields negligible marginal gains, and in fact leads to slight degradation on some metrics. This directly undermines the claim that the spatial anchor and fine-grained visual comparison components are essential drivers of the framework's reasoning capabilities.

  • Lack of a Direct Pixel-Differencing Baseline for Localization. Once the authentic reference chart is retrieved, a direct pixel-wise difference would naturally highlight modified regions. The paper lacks a comparison against such a simple differencing baseline. Without this benchmark, it is difficult to justify the complexity and overhead of training a U-Net localizer, especially when a minimal pipeline that solely embeds the URL payload and applies direct visual differencing might suffice in practice.

Questions for authors
  • Does the U-Net provide statistically meaningful localization gains over direct pixel-differencing?
  • Does combining VizDefender with authentic reference chart produces a comparable result?
2 comments

I appreciate the authors' efforts in addressing my concerns. The additional experiments have strengthened the evaluation. However, I remain concerned about the technical novelty, particularly the necessity of the proposed crop-aware localization mechanism when the authentic reference chart is available. It is unclear whether conventional image registration or alignment techniques could achieve comparable results without the proposed position-map-based design. Therefore, I remain inclined to maintain my original rating.

Thank you again for your careful review and constructive discussion! Your feedback has helped us improve the clarity and rigor of our work. We sincerely appreciate the time and attention you have devoted to our paper.

Best regards,

The Authors

Reviewer_7NT5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VizAnchor, a framework for chart-tampering localization and misleading-intent analysis. It uses proactive watermarking to recover reference information and retrieve the original chart, combining crop recovery and local-edit localization to construct semantic and spatial anchors. A three-step VLM pipeline then identifies manipulation attributes, reconstructs chart narratives, and infers the likely misleading intent. Evaluation covers reference retrieval, localization, manipulation understanding, ablations, and a user study.

Critical strengths

1.The problem is practically relevant, and the central design is well motivated. Retrieving an authentic chart and combining it with localized modification evidence reduces the ambiguity of analyzing a tampered chart alone. This enables explanations grounded in specific changes to values, categories, and comparisons, providing a useful approach to verifiable chart-tampering analysis.

2.The framework is comprehensive and shows promising empirical performance. The method integrates reference recovery, localization, and semantic explanation, with evaluations and ablations covering multiple stages. The localization results and qualitative examples demonstrate useful fine-grained manipulation analysis, while the supplementary implementation and evaluation details improve transparency

Critical weaknesses

The evaluation protocols limit confidence in the reported advantages and their generalizability. Appendix F.3 states that some participants were selected for the reported user-study cohort using observed between-method score differences, so this study cannot serve as independent confirmatory evidence. Additionally, the localization training and validation sets are not separated by source chart, and source-chart content may recur across collections (A.3). Missing or unparsable predictions are excluded from metric denominators without comprehensive reporting of valid sample counts for each method (C.6). These choices make generalization to unseen charts and performance over the full evaluation set difficult to assess, motivating reanalysis and targeted validation.

Questions for authors

1.How many responses were initially collected for the user study, and what exact selection procedure produced the 30-participant cohort? Do the conclusions hold when all responses satisfying the original inclusion criteria are analyzed?

2.What is the extent of source-chart overlap among the localization training, validation, and evaluation sets? Can the authors provide source-disjoint results to assess generalization to unseen charts?

3.How many predictions are missing or unparsable for each method and ablation setting? Are the conclusions stable on a common valid-sample subset and under a full-sample evaluation that accounts for failures?

The answers to these questions could materially affect my assessment. I would be inclined to raise my rating if the authors satisfactorily address these concerns through clarification, reanalysis, or additional evidence.

1 comment

Thank you again for your careful review and constructive discussion! Your feedback has helped us improve the clarity and rigor of our work. We sincerely appreciate the time and attention you have devoted to our paper.

Best regards,

The Authors

Reviewer_QdtxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VizAnchor is a pipeline in which the authors combines recoverable chart watermarks, a reference-based tamper localization technique, together with a structured VLM reasoning module to explain chart modifications and their potentially misleading effects.

Critical strengths

I like the separation between reference recovery, localization, and explanation, as the proposed pipeline combines several sections, such a setting can make the contribution of each part countable. Reference retrieval remains strong under substantial corruption and cropping, and local-edit IoU improves from 0.6067 to 0.7418. What adds on to this work is the blind human study, which includes a useful dual-anchor vanilla-VLM control to show the effect of this work.

Critical weaknesses

However, I have several concerns towards this work. First, the localization contribution needs an equivalent-reference/baseline comparison. Based on my understanding, the proposed module receives the original and modified images together with a thresholded difference map. It is unclear whether the reported baselines receive comparable information. I believe a paired-image change detector using the same reference and training data would better establish the contribution of the localization design.

I think the intent evaluation needs clearer ground truth and calibration. The paper should explain how reference intents were annotated and how alternative evidence-consistent interpretations are scored. The human study provides independent support, but covers only ten examples, which could be very biased. Independent adjudication would strengthen conclusions currently relying heavily on automated similarity and judge scores.

Also I am curiously about the comparison between the proposed methods with the froniter close-sourced models as I kinda feel that the latest models are much better on tasks like intent prediction.

Some typos need to be double checked in the main document, e.g., in section 4.1 about the dataset, "Detailed construction protocols, sampling strategies, and quality-control procedures are provided in Appendix ??." which leave the important reference in appendix empty.

Questions for authors

Pls refer to the weakness.

1 comment

Thank you again for your careful review and constructive discussion! Your feedback has helped us improve the clarity and rigor of our work. We sincerely appreciate the time and attention you have devoted to our paper.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary Response

Dear Area Chair and Reviewers,

We sincerely appreciate the reviewers' thoughtful feedback and suggestions. In response, we have revised the manuscript to address concerns regarding our technical contributions, experimental comparisons, and evaluation methodology. Below, we summarize the major revisions and their corresponding reviewer concerns. Detailed responses, additional results, and evaluation protocols are provided in the individual reviewer responses. All manuscript changes are highlighted in blue.

Main Revisions

1. Clarified crop-aware module and technical contributions (Qdtx Q1; CWEm Q1, Q5). We clarified that the primary contribution of the Crop-Aware Module is position-map-based spatial recovery rather than crop localization alone. The embedded position map restores the original spatial layout before metadata decoding, enabling provenance recovery under cropping. The resulting crop mask provides complementary spatial evidence for subsequent manipulation analysis.

2. Stronger localization comparisons and robustness evaluation (Qdtx Q1; CWEm Q1, Q3–Q5; 7NT5 Q2). We added paired-image baselines (ChangeStar, Change3D, and FC-Siam-conc) and direct pixel-differencing baselines. We also introduced a four-branch localization ensemble combining raw and compensated U-Nets, an improved Siamese U-Net, and an improved Change3D branch. Additional experiments evaluate robustness to brightness, hue, contrast, Gaussian noise, and JPEG compression.

3. Source-disjoint data splits and updated evaluations (7NT5 Q2). We established a revised source-disjoint split containing 227 training source charts (908 training cases), 64 validation source charts, and 200 external test edits, with no detected source-group overlap. All affected experiments were rerun, and Tables 3 and 5 were updated. The main conclusions remain consistent.

4. Stronger frontier-model and reference-aware comparisons (Qdtx Q3). We added a comparison with GPT-6 Sol using the two-image input setting, with reasoning enabled for GPT-6 Sol and disabled for VizAnchor. We also added a VizDefender (+Orig.) baseline to distinguish the contribution of authentic-reference access from the proposed components. Tables 6 and 7 were updated.

5. Re-evaluated component contributions under controlled reasoning settings (CWEm Q2). We reran all visual-evidence ablations (A1–A5) and reasoning-stage ablations (B1–B4) with internal reasoning disabled to better isolate individual component contributions. The revised results are reported in the main paper, while the original reasoning-enabled results are retained in the appendix. The new experiments demonstrate complementary contributions from the Spatial Anchor, Localized Comparison, and task-specific reasoning stages.

6. Improved intent evaluation and human-based validation (Qdtx Q2). We clarified the expert annotation protocol and distinguished semantic agreement from evidence consistency. We additionally conducted an agreement analysis between AI-FA and human judgments, using 1,980 individual judgments (33 participants × 10 examples × 3 methods × 2 tasks). Both process and intent evaluations show positive Spearman correlations, with cluster-bootstrap confidence intervals.

7. Full-sample user study and transparent participant selection (7NT5 Q3). We updated the main paper to report all 33 collected responses, moving the original 30-participant analysis and selection procedure to the appendix. We report descriptive statistics, Friedman tests, paired Wilcoxon tests with Holm correction, and effect sizes. Anonymized participant records are also provided in the supplementary materials. Figure 4 was updated accordingly.

8. Improved methodological clarity and reproducibility. We revised the method descriptions, clarified the position-map recovery and metadata-decoding sequence, corrected incomplete appendix references, and expanded the supplementary implementation and evaluation details.

We acknowledge that visualization manipulation forensics and misleading-intent understanding remain emerging research areas, and we view VizAnchor as a step toward more comprehensive and verifiable chart analysis. We sincerely appreciate the reviewers' constructive critiques, which have helped us strengthen the technical contributions, experimental rigor, and clarity of our work. We hope that the revised manuscript provides a clearer account of the potential of this research direction, as well as the contributions and limitations of our current approach.

Thank you once again for your time, careful evaluation, and valuable suggestions.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.