acceptodds
Under review as a conference paper at ICLR 2027

Infographics as a Visual Language: Learning through Context and Reconstruction

Abstract

Advances in multimodal models have improved chart understanding and image generation, yet it remains unclear how well these models capture the relationships that make an infographic meaningful. Infographics communicate through a visual language in which numerical values, text, and graphical elements constrain and complement one another. We introduce **CoReL (Contextual Reconstruction Learning)**, a data construction and pretraining recipe for learning infographic visual language through contextual reconstruction. CoReL organizes learning around three complementary tasks—**Numerical Reasoning**, **Numerical Rendering**, and **Contextual Synthesis**—that connect numerical meaning, visual expression, and contextual compatibility. We build **InfoContext**, a training corpus of **1.7M images** with **27M numerical-text bounding boxes** and **22M graphical bounding boxes**. We also introduce **InfoContext-Bench**, a human-annotated benchmark spanning the three tasks. We apply CoReL to an existing unified understanding-and-generation model through element-masked reconstruction. Experiments reveal substantial limitations in the evaluated public models, even when additional information about the missing content is provided, while our model achieves stronger performance across all three tasks. Numerical-reasoning pretraining with CoReL further improves downstream chart question answering on both **BAGEL and Qwen**, with consistent gains across benchmarks. These results establish contextual reconstruction as an effective framework for learning infographic visual language, with benefits extending from element completion to reasoning across chart elements.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Infographic understanding, visual reasoning, multimodal pretraining, contextual reconstruction, image generation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_2vrnOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CoReL, a BAGEL-based pretraining approach that reconstructs masked infographic elements through three tasks: Numerical Reasoning, Numerical Rendering, and Contextual Synthesis. It also presents InfoContext, a large corpus with automatically detected or transcribed elements, and InfoContext-Bench, a benchmark for numerical and graphical reconstruction. The reported results show gains over evaluated open models on reconstruction and downstream chart QA.

Critical strengths
  1. The paper targets meaningful relationships among chart values, visual marks, labels, and infographic context, which are not directly tested by many whole-image QA tasks.
  2. InfoContext could provide useful element-level training data at scale, and InfoContext-Bench evaluates both numerical inference and visual completion.
  3. The paper includes downstream QA experiments across two backbones and an out-of-domain benchmark, providing some evidence that numerical reconstruction pretraining can transfer to chart understanding.
Critical weaknesses
  1. Some of the evaluation results need to be enhanced. Most reported image metrics measure similarity to a single reference, although graphical elements can have multiple valid completions. The paper acknowledges this ambiguity but provides no human evaluation or direct measure of whether generated content is appropriate for the chart’s numerical and semantic context.

  2. The contribution of the three-task formulation is not isolated. The paper does not compare joint training of the three tasks against the individual tasks. The downstream transfer experiments use Numerical Reasoning pretraining alone, so they do not establish that the three-task framework—or the image-generation tasks in particular—improves chart understanding. It is therefore difficult to distinguish the value of the proposed formulation from that of additional task-specific training.

Questions for authors

See the weaknesses above

5 comments

We thank the reviewer for recognizing that CoReL targets meaningful relationships among chart values, visual marks, labels, and context, that InfoContext and InfoContext-Bench are useful at scale, and that the two-backbone, out-of-domain QA experiments support transfer.

Q1: Evaluation results need to be enhanced: metrics measure similarity to a single reference despite multiple valid completions, with no direct measure of contextual appropriateness.

R: Our evaluation covers both points: set-level metrics (FID, CLIP-FID) account for multiple valid completions, and the newly added ChartEval directly measures contextual appropriateness.

  1. Set-level metrics. Numerical Reasoning and Numerical Rendering have deterministic answers; only Contextual Synthesis admits multiple valid completions. For Semantic Targets, Tables 4, 16, and 17 already report FID and CLIP-FID, computed over image sets rather than against a single reference. We additionally report them on Structural Targets (target crop), where CoReL is again the best:
ModelFID ↓CLIP-FID ↓
Qwen-Image-Edit-2511107.1612.61
FireRed-Image-Edit-1.1151.6820.47
LongCat-Image-Edit146.6318.98
CoReL (ours)90.5511.28
  1. Contextual appropriateness. We add ChartEval (IJCNLP-AACL 2025), covering visual similarity, semantic alignment, and data fidelity; CoReL is the best across all metrics. Please see Q4 in our response to Reviewer twar.

Q2: The contribution of the three-task formulation is not isolated.

R: Our current evidence already supports that joint training is better than individual tasks, as detailed below. A matched individual-task comparison would make this more complete, and we will include it in the revision; each such run takes roughly 64 H20 GPUs for a week on BAGEL, which exceeds the rebuttal period.

  1. Fig. S4 in the newly uploaded supplementary material: Numerical Rendering alone generates a plausible appearance but large numerical errors; adding Numerical Reasoning significantly reduces them.

  2. The context ablation (Figure 5; more examples in Fig. S1 of the newly uploaded supplementary material) shows that the jointly trained model learns contextual relationships within the image, which single tasks cannot provide.

As Reviewer XuFw notes, the same masked inputs supervise both text prediction and image generation, making the joint formulation a natural fit for unified models.

We would be very happy to answer any further questions.

Thank you for your response.

Q1. How did you construct the set of valid completions used to compute the set-level metrics? Does this set include all possible valid completions?

Q2. I remain unconvinced by the indirect evidence, though I recognize that conducting an additional experiment would require more time than the rebuttal period allows.

Thank you for the follow-up.

Q1 (Follow-up): Reference-free evaluation for multiple valid completions

The reference set consists of the checked images provided by InfoDet (ICLR 2026) [1], which we treat as valid completions. We agree that it does not cover all valid completions; FID and CLIP-FID measure alignment with this empirical reference distribution.

To address this, we additionally adopt the human-aligned MLLM-as-a-Judge framework of [2], which was validated against human judgments for fine-grained image editing, and apply it to our completion task. It evaluates whether a completion satisfies the task and its context without reference images, which suits settings with multiple valid completions.

We randomly sample 50 semantic examples and evaluate Ours, Qwen, FireRed, and LongCat with three independent judges: GPT-5.5, Claude Opus 5, and Gemini 3.8 Flash (200 assessments per judge, 600 in total). Each judge receives the chart context with the target icon fully masked, the generated chart, and an enlarged crop of the generated target region, but no reference image.

We adapt the framework to six dimensions: context relevance (CR), information expression (IE), style consistency (SC), layout fit (LF), rendering quality (RQ), and outside-region preservation (OP). Each is scored from 1 to 7 (higher is better) and averaged over the 50 examples. Filling the region with background, chart marks, or text does not count as generating an icon; the rubric scores missing icons 1 on CR, IE, and RQ.

JudgeMethodCRIESCLFRQOP
GPT-5.5Ours6.185.825.526.325.926.92
Qwen1.001.001.021.021.044.56
FireRed1.001.001.121.241.082.62
LongCat1.001.001.041.001.005.66
Claude Opus 5Ours5.264.584.205.604.066.86
Qwen1.021.001.501.281.043.46
FireRed1.041.001.221.321.001.84
LongCat1.021.001.781.181.044.14
Gemini 3.8 FlashOurs5.144.784.605.904.946.60
Qwen1.341.341.541.501.444.44
FireRed1.281.281.341.401.322.68
LongCat1.281.281.441.421.385.34

Baseline editors mostly fill the masked region with background or surrounding marks instead of generating an icon, which explains their near-minimum CR, IE, and RQ scores (see also the third row of Figure 4 and Figure 8). Ours scores highest on all six dimensions under all three judges. Higher CR and IE indicate more contextually appropriate and meaningful icons. Higher SC, LF, and RQ indicate better visual integration and rendering, and OP of 6.60–6.92 indicates strong preservation of the surrounding chart. The advantage therefore holds when completions are judged on their own merits rather than by resemblance to a single reference.

Together with the ChartEval results (Q4 in our response to Reviewer twar) and Tables 4 and 15–17, these complementary evaluations provide substantial evidence of our method's effectiveness while directly addressing the non-uniqueness of valid completions. We welcome any further questions.

[1] InfoDet: A Dataset for Infographic Element Detection. ICLR 2026.

[2] Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis. arXiv:2602.13028, February 2026.

Reviewer_XuFwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces CoReL (Contextual Reconstruction Learning), a pretraining recipe that masks individual infographic elements and trains a unified understanding-and-generation model to reconstruct them from the surrounding context. It uses three tasks. Numerical Reasoning predicts a masked numeric label as text, with a cross-entropy loss. Numerical Rendering paints the inferred value back into the image, with a flow-matching loss. Contextual Synthesis regenerates masked graphical elements (bars, lines, areas, icons), also with flow matching. The authors also introduce InfoContext, a training corpus of 1.7M images with 49M annotated regions, and InfoContext-Bench, a human-annotated benchmark spanning the three tasks; both are to be publicly released. Experiments show that CoReL outperforms open VLMs on Numerical Reasoning and image-editing models on the two generation tasks. Numerical Reasoning pretraining also improves downstream chart question answering on both BAGEL and Qwen3.5-4B.

Critical strengths

The paper is well-motivated — recovering a masked element forces the model to link values to their visual encodings, which whole-image QA often does not require. The observation that many of ChartQA questions can be answered from text alone motivates this well. The supervision is also cheap, since the original elements serve as reconstruction targets. In addition, the same masked inputs supervise both text prediction and image generation, which makes the approach a natural fit for unified models. Another strength of the paper is the dataset. If released, the 49M-region training corpus and the human-annotated benchmark would be valuable to the community.

Critical weaknesses
  1. Limited novelty. Learning numerical, visual and semantic relationships through three reconstruction tasks is an interesting framing, but each component is incremental. Masked reconstruction is well established (MAE). The three tasks are standard objectives (masked value prediction and region inpainting) without new modules. The main contributions are therefore the data and the application to a unified model, not a new learning principle.

  2. Narrow scope. The method is demonstrated only on infographics, and its training images come from a single synthetic source (ChartGalaxy). The underlying idea, reconstructing elements from their context, should apply equally to documents, scientific figures, diagrams, slides and UI screenshots, where text and graphics also constrain each other. Without evidence in at least one other visual domain, it is unclear whether CoReL is a general recipe or one tuned to a specific dataset.

  3. Unclear benefits:

3.A The value of the data itself is not isolated. The transfer experiment compares CoReL pretraining followed by QA SFT against QA SFT alone. It does not show that this specific data or objective beats an equally sized pretraining stage on other chart or infographic data, such as chart-to-table derendering, additional QA on ChartQA, PlotQA or ChartGalaxy, or plain captioning. Without a control matched for data size and compute, the small gains could come from extra training rather than from contextual reconstruction.

3.B Training and evaluation sets are from the same distribution — for Numerical Reasoning and Rendering, as well as Contextual Synthesis. The large reconstruction margins may therefore reflect fit to the distribution rather than general contextual reasoning, especially since the baselines runs zero-shot.

3.C Generalization to standard benchmarks is untested. Downstream evaluation covers only ChartGalaxy QA, which is in-domain and uses QA data the authors generated, and EncQA. It is unclear whether the approach would generalize to other datasets such as CharXiv (real scientific charts) and InfographicVQA (real infographics, directly in scope) .

Questions for authors

Regarding downstream QA, does CoReL pretraining beat an equally sized pretraining stage on other chart data? Please address the weaknesses above.

4 comments

We thank the reviewer for recognizing that the paper is well motivated, that the supervision is cheap and a natural fit for unified models, and that InfoContext and InfoContext-Bench would be valuable to the community.

Q1: Limited novelty.

R: We agree that masked reconstruction and the underlying losses are established. CoReL's contribution lies in the prediction problem: what is hidden, what contextual evidence supports its recovery, and in which representation it is recovered. As the reviewer and Reviewers twar and 2vrn note, this links values to their visual encodings, which whole-image QA often does not require.

Novelty of the task formulation. Our design builds on two key properties of infographics: (1) numerical text has both token and visual representations, and (2) textual and graphical elements that are connected through quantitative and semantic constraints. Accordingly, numerical Reasoning and Numerical Rendering recover hidden numerical content as tokens and rendered labels, respectively; Contextual Synthesis reconstructs graphical elements from the remaining context. The tasks are complementary: inferring values from marks, and reconstructing marks from values and context.

Unlike MAE, which reconstructs randomly masked pixel patches, CoReL masks annotated numerical or graphical elements, so recovery must rely on relationships with other elements. Useful supervision thus requires that the remaining context offer evidence about the missing target, not merely co-occurring text and graphics. InfoContext provides such localized reconstruction targets at scale, making data construction integral to the training design.

Following the suggestion from Reviewer twar, we conducted an additional controlled experiment comparing CoReL against the chart-to-table derendering pretraining objective of ChartAst [6]. This experiment also provides further empirical evidence supporting the novelty and effectiveness of our proposed learning formulation. Under matched training settings, CoReL consistently outperforms ChartAst on both in-domain ChartGalaxy (+1.38) and out-of-domain EncQA (+4.74), demonstrating the effectiveness of our context-dependent numerical reconstruction objective and its stronger cross-domain generalization beyond conventional chart pretraining.

ChartGalaxyEncQA
TrainingOverallCalc.Comp.CountIdent.TrendOverallCVCVRFVFE
Base33.6724.9037.3318.3643.2146.5574.9664.3366.0065.5694.44
QA SFT74.6566.9389.1938.8985.8191.3878.6767.6774.0068.8997.33
ChartAst pretraining + QA SFT75.4466.8089.9845.6584.6793.9778.1570.0071.3366.8997.11
CoReL pretraining + QA SFT (ours)76.8267.8389.5949.0386.4594.8382.8975.6776.6774.4498.22

Q2: Narrow scope.

R: We agree that broader cross-domain validation is valuable. Our results support transfer beyond the training distribution (real ChartGalaxy infographics unseen in training, out-of-domain EncQA, and the newly added InfoChartQA; please see Q4 and Q5).

Other domains require more than co-occurring text and graphics. Fig. S2 in the newly uploaded supplementary material shows examples from ChartNet [1], FigureQA [2], ArXivQA [3], and SlideVQA [4]: removed bars may lack independent value information, the displayed pie chart provides no numerical-label targets, and scientific trajectories or slide illustrations may be weakly constrained by the remaining image. Such data may require distinct task formulations and supervision strategies, which we view as a complementary direction largely orthogonal to the infographic-specific formulation studied here.

To our knowledge, no existing ready-to-use dataset supports all three CoReL tasks at comparable scale with the required element annotations and contextual information; this gap motivates InfoContext.

Q3 (Weakness 3.A): The value of the data and objective is not isolated; a control matched for data size and compute is missing.

R: We add such a control with chart-to-table derendering pretraining (ChartAst [6]), and CoReL outperforms it. Please see Q1 in our response to Reviewer twar.

Q4 (Weakness 3.B): Training and evaluation sets are from the same distribution.

R: Most numerical evaluation targets come from a source absent from training, and three forms of evidence support generalization beyond the training distribution.

  1. Real infographics unseen during training. The numerical benchmark deliberately includes real ChartGalaxy infographics, while training uses only its synthetic collection. Real images contribute 3,321 of 5,353 numerical targets (62%; 321 images), exceeding the 2,032 synthetic targets (Table 1).

  2. Out-of-domain downstream transfer. With identical QA SFT, Numerical Reasoning pretraining improves EncQA accuracy by 9.48 and 4.22 percentage points on BAGEL and Qwen3.5-4B, respectively (Table 5); neither pretraining nor SFT uses EncQA-domain data.

  3. Additional out-of-domain examples. Fig. S3 in the newly uploaded supplementary material shows successful contextual reconstruction on style-transferred images and Vietnamese infographics from ViInfographicVQA [5], a dataset never used in CoReL training.

Q5 (Weakness 3.C): Generalization to standard benchmarks is untested.

R: We additionally evaluate on InfoChartQA [7], which covers both infographic (Info) and plain-chart (Plain) formats. CoReL improves over both the original checkpoints and QA SFT on both subsets and both backbones.

BackboneTrainingInfoPlain
BAGEL-7BBase58.82%66.89%
QA SFT57.51%64.15%
CoReL pretraining + QA SFT59.18%67.27%
Qwen3.5-4BBase55.28%67.25%
QA SFT59.91%70.05%
ChartAst pretraining + QA SFT58.42%67.80%
CoReL pretraining + QA SFT61.14%71.86%

Relative to QA SFT, the gains are 1.67/3.12 percentage points for BAGEL and 1.23/1.81 for Qwen. In contrast, ChartAst pretraining underperforms QA SFT alone on both subsets, again showing that additional pretraining does not necessarily help and supporting CoReL's pretraining design.

We would be very happy to answer any further questions.

[1] ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding. CVPR 2026.

[2] FigureQA: An Annotated Figure Dataset for Visual Reasoning. ICLR Workshop 2018.

[3] Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models. ACL 2024.

[4] SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images. AAAI 2023.

[5] ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographics. AI4Research Workshop at AAAI 2026.

[6] ChartAssistant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning. Findings of ACL 2024.

[7] InfoChartQA: A Benchmark for Multimodal Question Answering on Infographic Charts. NeurIPS 2025.

Dear Reviewer,

Thank you for your valuable feedback. We have posted our responses to your comments and would greatly appreciate it if you could take a look when you have a chance.

We would be happy to further discuss any remaining questions or concerns and provide additional clarifications if needed. We look forward to hearing your thoughts!

Best regards, The Authors

Supplementary Response to Q4 (Weakness 3.B): Real vs. Synthetic Breakdown

To further address Q4, we break down the Numerical Reasoning results in Table 2 by real and synthetic partitions, comparing CoReL with the two strongest public baselines:

ModelOverall (5,353)Real (3,321)Synthetic (2,032)
Qwen3.5-9B64.5059.0473.43
Qwen3.6-27B82.0577.4089.65
CoReL (ours)85.1981.2791.60

Although CoReL's training data contain no real ChartGalaxy images, it achieves 81.27% on the real partition, outperforming the larger Qwen3.6-27B by 3.87 pp. Its margin over Qwen3.6-27B is larger on real data than on synthetic data (3.87 vs. 1.95 pp), and its real–synthetic gap is the smallest of the three models (10.33 vs. 12.25 and 14.39 pp). These results indicate that CoReL's gains come from contextual reasoning that transfers to real infographics, not from overfitting to the synthetic training distribution.

Reviewer_bvk1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CoReL, which trains BAGEL to recover masked numerical labels as text, render missing numbers, and reconstruct graphical elements. It contributes InfoContext, containing 1.7M images and 49M region annotations, and InfoContext-Bench. Experiments report improvements over released models on reconstruction, while numerical-reasoning pretraining improves downstream chart QA on BAGEL and Qwen.

Critical strengths
  • A useful combination of data and tasks. The region annotations support both textual and visual reconstruction within one framework. Evaluating numerical inference, rendering, and graphical completion separately provides a broader assessment than chart QA alone.

  • Promising results beyond the training task. Numerical-reasoning pretraining improves ChartGalaxy and EncQA accuracy on both backbones. The fixed-target evaluation in Table 12 is also valuable: it shows that the multi-mask advantage persists when the scored targets remain unchanged.

Critical weaknesses
  • The comparisons do not isolate the proposed training recipe. Reconstruction results compare extensively adapted CoReL with released models, without reporting unmodified BAGEL on the same tasks or an alternative pretraining objective under matched data and compute. Table 5 controls QA fine-tuning but adds pretraining only to CoReL. Appendix H acknowledges this limitation. Consequently, the benefit of the particular reconstruction formulation, beyond additional domain training, remains unclear.

  • The metrics only partly establish faithful reconstruction. Structural reconstruction is evaluated exclusively through whole-image similarity, which combines target recovery with preservation of surrounding content. For semantic targets, similarity to one reference does not establish contextual suitability. Numerical scoring also ignores magnitude suffixes, so “57” and “57M” can receive the same parsed-value score. These choices weaken the connection between the reported results and preservation of infographic meaning.

  • Important validation and reproduction details are missing. Appendix A describes annotation filtering and merging without specifying the rules or reporting an annotation-quality audit. Appendices A and G also omit the duration and optimization settings of the separate numerical-reasoning pretraining used in Table 5. No uncertainty estimates are provided for the downstream gains, making the smaller improvements harder to assess.

Questions for authors
  • Clarification: What were the exact numerical-only pretraining settings for Table 5, and how were the automatically generated region annotations checked and filtered?

  • Evidence needed for reassessment: How does CoReL compare with unmodified BAGEL and with an alternative chart-pretraining objective using the same backbone, data, and training budget?

  • Evidence needed for reassessment: Do the conclusions hold with magnitude-aware numerical scoring and direct evaluation of reconstructed chart geometry and contextual suitability, assessed separately from background preservation?

3 comments

We thank the reviewer for recognizing the useful combination of data and tasks, the promising downstream results on both backbones, and the value of the fixed-target evaluation (Table 12).

Q1: The comparisons do not isolate the proposed training recipe.

R: We add a control to Table 5 that uses an alternative chart-pretraining objective (ChartAst) with the same backbone, pretraining images, and SFT settings. CoReL outperforms it, isolating the benefit of our training recipe. Please see Q1 in our response to Reviewer twar.

Q2: Unmodified BAGEL is not reported on the same tasks.

R: We now report unmodified BAGEL on all three InfoContext-Bench tasks; CoReL outperforms it by a large margin on every metric.

Numerical Reasoning (single mask)

ModelOverallEasyMediumHard
Unmodified BAGEL52.8538.1262.7437.14
CoReL (ours)85.1992.7184.1056.22

Numerical Rendering (context-only)

ModelScore ↑Exact (%) ↑PSNR ↑SSIM ↑LPIPS ↓
LongCat-Image-Edit3.650.4121.060.90510.0875
Unmodified BAGEL3.410.4722.180.86700.1857
CoReL (ours)75.5155.6736.420.98600.0048

Contextual Synthesis

ModelStructural Targets (target crop)Semantic Targets
MAE ↓LPIPS ↓CLIP-I ↑DINO ↑Full FID ↓Crop FID ↓Crop DISTS ↓
LongCat-Image-Edit65.680.62240.76140.419526.31235.330.5382
Unmodified BAGEL68.280.67580.74630.341228.66251.470.5643
CoReL (ours)20.260.23900.86240.658821.00118.700.3638

Q3: The metrics only partly establish faithful reconstruction.

R: We address the three points in turn.

  1. Whole-image similarity for structural reconstruction. The Structural Targets results in Table 4 are in fact target-crop metrics (mislabeled as whole-image; we apologize for the confusion), so they measure target recovery apart from surrounding content. We also add whole-image and unmasked-region metrics; please see Q3 in our response to Reviewer twar.

  2. Contextual suitability. We add ChartEval (IJCNLP-AACL 2025), on which CoReL is the best across all metrics; please see Q4 in our response to Reviewer twar.

  3. Magnitude suffixes. Suffixes are omitted only during numerical scoring, not from the prediction task: training targets retain units and magnitude suffixes, and the evaluation prompt explicitly requests them, including "k" and "M" (Appendix I.1). The deterministic parser gives a uniform, inexpensive comparison across 12 public VLM baselines and one commercial reference under five masking settings, accommodating different output conventions without model-based judgment. Related normalization choices appear in ChartAgent [1] and ChartM³ [2]. In the revision, we will additionally report magnitude-aware scores.

Q4: Important validation and reproduction details are missing.

R: 1. Annotation checking and filtering.

  • Training annotations. Before large-scale annotation, we refined the prompts and model settings and manually inspected 100 randomly sampled images per model; detection accuracy exceeded 95% for DeepSeek-OCR-2 and 85% for Qwen3.6-27B. Rule-based and regular-expression filters remove empty or non-numerical boxes from DeepSeek-OCR-2 outputs. For Qwen3.6-27B, we discard boxes covering less than 1% or more than 45% of the image area; the upper threshold addresses potential over-detection.
  • Benchmark annotations. All benchmark data were manually verified.
  1. Numerical Reasoning pretraining settings in Table 5. We train on all InfoContext data for 25k steps (about 3.2M samples seen) on 32 H20 GPUs.

We would be very happy to answer any further questions.

[1] ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering. ACL 2026.

[2] ChartM³: A Multi-Stage Code-Driven Pipeline for Constructing Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension. Findings of EMNLP 2025.

Thank you for adding the unmodified BAGEL results. After CoReL training, single-mask numerical reasoning improves from 52.85 to 85.19, and context-only rendering exact match rises from 0.47% to 55.67% under the current scoring rule. This resolves my concern about the missing baseline. The 25,000-step training schedule and the description of the annotation checks also clarify the setup.

  • The ChartAst comparison is the result that matters most to my recommendation. Please include the CoReL and ChartAst scores for each backbone on ChartGalaxy and EncQA in Table 5, alongside the pretraining budget for each method. Were the training tokens or compute comparable? For small margins, a paired confidence interval would help establish how much the choice of objective contributes.

  • I accept the clarification that Table 4 reports target-crop scores. Please correct the same description in Section 5.3, Table 15, and Appendices B.4 and F.1. For the semantic targets, did ChartEval assess the restored object itself? In particular, how would it score a different icon that still fits the chart?

  • The magnitude issue remains open. Under the current parser, “57k” and “57M” receive the same numerical score. How many reference labels contain a magnitude suffix, and how does interpreting that suffix change exact-match accuracy for CoReL and the strongest baseline on those labels?

Thank you for the follow-up suggestions. We address each point below.

Q5: ChartAst comparison on each backbone, and the pretraining budget of each method.

R: We additionally run ChartAst pretraining on BAGEL with the same 3.2M seen image samples. CoReL still achieves better results, especially on the out-of-domain EncQA, although ChartAst actually uses more training compute.

BAGELChartGalaxyEncQA
TrainingOverallCalc.Comp.CountIdent.TrendOverallCVCVRFVFE
QA SFT70.1463.1684.0932.3780.8595.6967.0461.6756.0044.4496.89
ChartAst pretraining + QA SFT71.0358.3784.2839.8684.2092.2470.3763.0057.3352.4497.56
CoReL pretraining + QA SFT (ours)72.1463.8184.0941.7982.5891.3876.5274.6760.0060.6799.11

Pretraining budget. The input training tokens are the same: CoReL takes the masked image (1 to 6 randomly masked boxes per image, 3.5 in expectation), and ChartAst takes the unmasked image. The loss tokens differ: CoReL predicts only the values of the masked boxes, whereas ChartAst predicts the whole table, e.g., |Project|TimeSpent|BudgetAllocation|\n|---|---:|---:|\n|Renaissance Portrait|125|8.5|\n|Byzantine Mosaic|210|14|\n.... InfoContext contains 30.78 numbers per image on average, so ChartAst is supervised on roughly 9× more numbers per image than CoReL, in addition to all headers and labels.

We attribute CoReL's advantage to its objective. ChartAst is a chart-to-table translation task, whereas CoReL forces the model to recover masked values from their context on the chart image, which improves its understanding of the context underlying the numbers.

Q6: For the semantic targets, did ChartEval assess the restored object itself?

R: 1. We will correct the description in Table 4, Section 5.3, Table 15, and Appendices B.4 and F.1.

  1. Thank you for raising this. ChartEval mainly scores data-related chart components (e.g., title, chart type, x/y axes, data points, and bars); purely visual icons that do not encode data are not counted. We therefore further apply a human-aligned MLLM-as-a-Judge evaluation [3], which requires no reference images and thus suits settings with multiple valid completions. With GPT-5.5, Claude Opus 5, and Gemini 3.8 Flash as independent judges, CoReL scores highest on all six dimensions under all three judges. Please see Q1 (Follow-up) in our response to Reviewer 2vrn.

Q7: The magnitude issue remains open.

R: Following your suggestion, we report the distribution of unit types among the labels and re-compute the metrics with unit-aware scoring. The effect on our conclusions is small: CoReL remains better than the strongest baseline overall (84.01 vs. 80.63).

Distribution of unit types in InfoContext-Bench and the InfoContext training set:

CategoryDescriptionBenchmarkTraining set
No unitPlain numbers60.3%64.8%
Common unitsNumbers with % or $25.9%15.1%
Other unitsAll remaining units, including magnitude suffixes such as "k" and "M"13.7%20.1%

Unit-aware scoring. If the unit of a prediction does not match that of the label, the prediction is treated as incorrect and scores 0; otherwise we follow the metric in the paper. We re-compute the single-mask results of CoReL and the two strongest baselines, Qwen3.6-27B and Qwen3.5-9B:

CategoryCountCoReL (original)CoReL (unit-aware)Qwen3.6-27B (original)Qwen3.6-27B (unit-aware)Qwen3.5-9B (original)Qwen3.5-9B (unit-aware)
No unit3,230 (60.3%)88.0387.8282.0381.3860.2659.41
Common units1,388 (25.9%)89.0788.8181.1480.7370.0768.78
Other units735 (13.7%)65.3558.1883.8377.1572.6263.64
Overall5,35385.1984.0182.0580.6364.5062.42

Unit-aware scoring lowers all three models by a similar margin overall (1.18/1.42/2.08 points), mostly on Other units. On that subset, CoReL trails Qwen3.6-27B under both scorings (58.18 vs. 77.15 with unit-aware scoring). We will add these results to the revision.

[3] Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis. arXiv 2026.

Reviewer_twarOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoReL, a context reconstruction learning framework that masks local elements in infographics and tasks the model with three types of tasks—numerical inference, numerical rendering, and graphic completion—to learn the relationships between numbers, text, and graphics. Its main contribution is unifying infographic understanding and generation into an element reconstruction task, and constructing the InfoContext dataset—containing 1.7 million images and 49 million region annotations—as well as the manually annotated evaluation set, InfoContext-Bench. The authors conducted joint training for the three tasks based on BAGEL, achieving overall better performance than the compared open-source models in numerical restoration and image reconstruction evaluations. Furthermore, after pre-training exclusively on numerical reasoning, both BAGEL and Qwen achieved higher overall question-answering accuracy on ChartGalaxy and the out-of-domain EncQA than controls that were only fine-tuned for question-answering, supporting the transfer value of element reconstruction training for downstream chart understanding.

Critical strengths

The paper organizes numerical inference, numerical rendering, and graphic completion into a unified context reconstruction task, establishing a direct link between the numerical meaning and visual representation in infographics, and enabling large-scale training supported by 1.7 million images and 49 million region annotations. This contribution, which combines task design with data resources, provides a valuable foundation for research on infographic understanding and generation. Second, the experiments not only validated the reconstruction performance but also demonstrated, under controlled conditions regarding subsequent question-answering fine-tuning data and the number of training steps, the benefits of numerical reasoning pre-training for BAGEL and Qwen. Both models achieved improved overall accuracy on ChartGalaxy and the out-of-domain EncQA dataset. These cross-model and cross-task results provide concrete support for the method’s transferability and serve as important evidence supporting its acceptance.

Critical weaknesses

First, the attribution of method-based gains remains insufficient: the reconstruction experiments primarily compare CoReL—which has undergone task-specific training—with publicly available models that have not undergone such training, while the downstream experiments additionally incorporate a pre-training phase. There is still a lack of alternative control groups with matching training resources, as well as ablation studies capable of isolating the contributions of the three task categories. Therefore, while the existing results support the effectiveness of this training scheme, the evidence regarding the independent value of the context reconstruction objective and the joint design of the three tasks remains limited. Second, the evaluation of graphic completion falls short of the core objective of “learning relationships between elements”: for structural primitives, whole-image similarity is primarily used, and scores may be skewed by fidelity gains from unoccluded regions; while semantic objects provide local metrics, they also primarily measure similarity to the reference image. Existing evaluations have not systematically verified whether the generated graphics satisfy quantitative, geometric, and semantic constraints, and context ablation studies primarily rely on single examples, which limits the ability to determine whether the model has learned relevant relationships.

Questions for authors

First, in the downstream transfer experiments, how much data, how many training steps, and how much computational resources were specifically used for the pre-training of numerical reasoning? Are there any comparison results using different training objectives under the same backbone, similar data, and comparable computational budgets, or records comparing single-task and joint training? Regarding graphic completion, have there been any correctness verification or error analysis conducted on the generated regions to determine whether structural elements satisfy constraints such as quantity and proportion, and whether semantic objects align with the context?

2 comments

We sincerely thank the reviewer for recognizing the contributions of our unified task formulation, large-scale dataset, and extensive experimental validation, particularly the effectiveness and transferability of our numerical reasoning pre-training across different models, tasks, and datasets.

Q1: Lack of alternative control groups with matching training resources.

R: To address the reviewer's concern regarding the attribution of performance gains, we conduct an additional controlled experiment with a matched pretraining baseline. Specifically, we adopt the chart-to-table derendering objective from ChartAst [1] and train it using the same pretraining images, training budget, and downstream QA SFT settings as CoReL (Table 5). We perform this comparison on both backbones, Qwen3.5-4B and BAGEL. The experimental results are as below:

Qwen3.5-4BChartGalaxyEncQA
TrainingOverallCalc.Comp.CountIdent.TrendOverallCVCVRFVFE
Base33.6724.9037.3318.3643.2146.5574.9664.3366.0065.5694.44
QA SFT74.6566.9389.1938.8985.8191.3878.6767.6774.0068.8997.33
ChartAst pretraining + QA SFT75.4466.8089.9845.6584.6793.9778.1570.0071.3366.8997.11
CoReL pretraining + QA SFT (ours)76.8267.8389.5949.0386.4594.8382.8975.6776.6774.4498.22
BAGELChartGalaxyEncQA
TrainingOverallCalc.Comp.CountIdent.TrendOverallCVCVRFVFE
Base62.1045.2785.4631.6472.6587.0773.3370.0058.6756.2297.56
QA SFT70.1463.1684.0932.3780.8595.6967.0461.6756.0044.4496.89
ChartAst pretraining + QA SFT71.0358.3784.2839.8684.2092.2470.3763.0057.3352.4497.56
CoReL pretraining + QA SFT (ours)72.1463.8184.0941.7982.5891.3876.5274.6760.0060.6799.11

Under matched settings, CoReL outperforms ChartAst on both backbones: +1.38 on ChartGalaxy and +4.74 on EncQA with Qwen3.5-4B, and +1.11 and +6.15 with BAGEL. While ChartAst pretraining yields smaller and less consistent gains over QA SFT (+0.79/−0.52 on Qwen3.5-4B; +0.89/+3.33 on BAGEL), CoReL consistently improves both benchmarks (+2.17/+4.22 and +2.00/+9.48). These results confirm that the improvements stem from the proposed context reconstruction objective rather than merely additional pretraining, demonstrating its effectiveness.

Q2: In the downstream transfer experiments, how much data, how many training steps, and how much compute were used for Numerical Reasoning pretraining?

R: We train on all InfoContext data for 25k steps (about 3.2M samples seen) on 32 H20 GPUs.

Q3: For structural primitives, whole-image similarity is primarily used, and scores may be skewed by fidelity gains from unoccluded regions.

R: The Structural Targets results in Table 4 (and Table 15) are in fact target-crop metrics and thus already exclude unoccluded regions; they were mislabeled as whole-image, and we apologize for the confusion. For a comprehensive evaluation, we additionally report whole-image and unmasked-region metrics. CoReL is consistently the best in all three regions.

RegionMethodMAE ↓LPIPS ↓CLIP-I ↑DINO ↑
Target crop (reported in Table 4)Qwen-Image-Edit-251143.460.41990.83180.5876
FireRed-Image-Edit-1.164.730.63640.77460.4251
LongCat-Image-Edit65.680.62240.76140.4195
CoReL (ours)20.260.23900.86240.6588
Whole imageQwen-Image-Edit-251110.460.11940.98370.8893
FireRed-Image-Edit-1.117.460.19050.97270.8384
LongCat-Image-Edit24.270.18550.96690.8478
CoReL (ours)5.670.05120.98980.9270
Unmasked region (outside target boxes)Qwen-Image-Edit-25114.630.02950.99150.9482
FireRed-Image-Edit-1.19.220.07890.98640.9174
LongCat-Image-Edit17.370.07550.98230.9222
CoReL (ours)2.230.00110.99460.9675

As expected, scores rank unmasked region > whole image > target crop: the unmasked region should ideally remain unchanged, while the target crop isolates the reconstructed content and is therefore the strictest.

Q4: Existing evaluations have not systematically verified whether the generated graphics satisfy quantitative, geometric, and semantic constraints.

R: 1. Scene-graph evaluation (ChartEval). We supplement Table 4 with ChartEval [2], on which CoReL is the best across all metrics. ChartEval parses chart images into scene graphs and applies graph-based similarity measures covering data fidelity (quantitative), structure (geometric), and semantic alignment. Its four metrics are GraphBERT Score (semantic similarity), Hallucination Rate (spurious or incorrect information), Omission Rate (missing critical elements), and Graph Edit Distance (GED; structural differences). For detailed definitions, please refer to [2]. We use the GPT-5.5 API.

MethodGraphBERT Precision ↑GraphBERT Recall ↑GraphBERT F1 ↑Hallucination (%) ↓Omission (%) ↓Normalized Approximate GED ↓
Qwen-Image-Edit-251193.7393.9693.7532.2234.530.6375
FireRed-Image-Edit-1.191.5391.7691.5236.7941.030.7607
LongCat-Image-Edit94.6394.7594.6129.4932.720.6021
CoReL (ours)96.5496.8396.6423.5723.840.4449

2. Reference-free evaluation (MLLM-as-a-Judge). We further apply a human-aligned MLLM-as-a-Judge evaluation [3], which requires no reference images, using GPT-5.5, Claude Opus 5, and Gemini 3.8 Flash as independent judges; CoReL scores highest on all six dimensions under all three judges (see Q1 (Follow-up) in our response to Reviewer 2vrn).

Together with Table 4 and Q3, these two complementary evaluations comprehensively confirm CoReL's advantage.

Q5: Context ablation studies primarily rely on single examples, which limits the ability to determine whether the model has learned relevant relationships.

R: We add more examples in Fig. S1 of the newly uploaded supplementary material. Consistent with Figure 5, CoReL learns the contextual relationships between numbers and graphical elements, whereas Qwen-Image-Edit-2511 does not. The three controlled conditions are:

  • Full context: normal generation.
  • Target value erased: the number associated with the target graphical elements is erased and filled with background pixels. Without it, the model struggles to generate the correct number of elements.
  • Context pairs erased: the remaining reference value–mark pairs are erased. The model still uses the retained context to generate reasonable elements with the correct number, which differ from the ground truth but remain valid.

Given the limited rebuttal period, we will add quantitative results in the revision.

Q6: Ablation studies isolating the contributions of the three tasks (single-task versus joint training).

R: We have now trained the three single-task models with the same per-task training samples as joint training. Joint training substantially improves Numerical Rendering (Score 64.61 → 78.64) and Contextual Synthesis (Structural MAE 28.50 → 21.24), and is on par on Numerical Reasoning (89.05 vs. 89.89): generation benefits from being trained together with numerical understanding, while reasoning accuracy is preserved. Please see Q2 (Follow-up) in our response to Reviewer 2vrn for the full results.

We would be very happy to answer any further questions.

[1] ChartAssistant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning. Findings of ACL 2024.

[2] ChartEval: LLM-Driven Chart Generation Evaluation Using Scene Graph Parsing. IJCNLP-AACL 2025. https://github.com/chartEval/ChartEval.

[3] Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis. arXiv:2602.13028, February 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.