acceptodds
Under review as a conference paper at ICLR 2027

ChartCraft: Towards Aesthetic and Context-Aware Chart Generation

Abstract

Structured data visualizations such as tables and charts are widely used to communicate information, yet existing chart generation datasets often suffer from limited aesthetics and insufficient contextual cues, hindering interpretability. We present ChartCraft, a framework and curated dataset designed to advance aesthetic and context-aware chart generation. Our approach focuses on transforming raw or minimally structured inputs into visually refined and context-enriched tables and charts that improve readability, semantic clarity, and visual coherence. To support this goal, we construct a diverse dataset spanning four visualization types (tables, bar, line, and pie charts), and define six generation tasks with annotations for layout, styling, and contextual augmentation. Building on this resource, we train a model termed ChartCrafter using SFT and RL, jointly modeling visual design and contextual reasoning for chart generation. We further introduce a new evaluation benchmark, CraftEval, to assess both aesthetic quality and contextual awareness. Experimental results demonstrate that models trained with ChartCraft produce charts that are more visually consistent and contextually informative, surpassing the baseline by +26.81 and +24.01 points, respectively on T2I and editing tasks of CraftEval. Our work highlights the importance of integrating design-aware and context-aware principles into aesthetic chart generation.

Primary area:
generative models
Keywords:
Chart generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_m2onOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ChartCraft, a synthetic dataset of approximately 1.92 million samples covering six chart-image generation and editing tasks, together with adapted ChartCrafter models and the CraftEval benchmark. Evaluation shows substantial gains over the original Qwen backbones, supplemented by human preferences, an external judge, extraction audits, and split-integrity checks. The task-organized supervision is useful, but the evidence establishes reference-conditioned adaptation more clearly than general design quality or reliable preservation of chart semantics. Overall, I lean towards rejection because the contribution reduces largely to distilling a proprietary teacher under teacher-aligned evaluation, and the data fidelity of the generated charts remains low.

Critical strengths
  • The dataset clearly organizes six generation and editing tasks, with explicit source–target pairings and detailed prompt-construction procedures.
  • ChartCrafter achieves substantial improvements over its original backbones, and the paper appropriately distinguishes these adaptation gains from a controlled data-only comparison.
Critical weaknesses
  1. Teacher distillation under teacher-aligned evaluation. My main concern is that the approach essentially distills a proprietary teacher, while the evaluation shares the teacher's model family. Training targets are produced by Nano Banana Pro, instructions and consistency filtering by Gemini-3-Pro, and CraftEval is judged by Gemini-3-Pro, which also supplies the RL reward. Sec. 2.3 acknowledges that the results demonstrate task adaptation rather than an advantage over the teacher, and Nano Banana Pro remains the strongest system. Moreover, AQA explicitly frames evaluation as design replication, while CSA rewards the reference's exact thematic appearance, so an output that follows the instruction through a different, equally valid design may receive a lower score. The improvements therefore primarily establish reference-conditioned alignment with the teacher's outputs, which limits the contribution beyond distillation into an open backbone.

  2. Low data fidelity. The absolute data fidelity of the generated charts remains low. ChartCrafter reaches DSI scores of only 54.9 on T2I and 51.2 on Edit, with Text Preservation at 37.8 and 30.1. For charts, incorrect values or labels compromise the visualization regardless of its aesthetic quality. This is difficult to reconcile with Table 12, which reports numerical or label corruption in only 15 of 200 T2I samples without identifying the evaluated model. The training targets compound this concern: automated filtering checks consistency among generated artifacts, and the independent audit (Appendix A.6) checks extracted code against source images, but neither quantifies the fidelity of the final training targets. Some conversions are also semantically inappropriate. For example, Fig. 16 requests stacking Min, Q1, Median, Q3, and Max, which are not additive components. Prior pictorial-chart generation work, such as ChArtist (Xiao et al., CVPR 2026) and Semantic-Structural Alignment for Generative Pictorial Charts (2026), explicitly conditions generation on chart structure to preserve data encoding and evaluates data accuracy directly, whereas ChartCraft relies on unconstrained pixel-space generation and neither discusses nor compares with these approaches.

Questions for authors
  1. Which model's outputs are summarized in Table 12, and how should its 15 of 200 T2I corruption cases be reconciled with the Text Preservation score of 37.8 in Table 3?
  2. The SFT stage uses about 50,000 examples from a resource of 1.92 million. How was this subset selected, and is there any evidence that training on larger portions of ChartCraft improves performance?
  3. How many human evaluators took part in Appendices A.3–A.5, what was their expertise, and how many judgments were collected per example? How were tied scores handled, and how were the sub-item ordered-pair counts in Table 6 constructed?
  4. Did the authors measure numerical accuracy directly, for example by extracting values from generated charts and comparing them with the source data, independently of the reference-based judge?
Reviewer_hpJfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ChartCraft provides paired supervision for pixel-space chart generation and editing under aesthetic and contextual instructions. The paper fine-tunes image generation/editing backbones and applies reinforcement learning to the editing model. CraftEval measures data/structure integrity, aesthetic quality, and contextual/style alignment. The appendix includes human preferences, an external judge, and data-quality audits.

Critical strengths
  1. Paired aesthetic and contextual supervision is a potentially useful resource for image-based chart generation and editing.

  2. Improvements over the corresponding backbones are substantial. Human preferences and an external judge provide useful corroboration beyond the primary evaluation protocol.

Critical weaknesses
  1. Curation contribution. Backbone adaptation gains establish useful supervision, but do not isolate the aesthetic/contextual construction stages. A matched chart-domain training control would clarify what those choices add, without requiring a new learning algorithm.

  2. Communicative claims. Style execution and visual appeal do not establish better quantitative comprehension or usability. These broader claims should be distinguished from the demonstrated generation and editing improvements.

  3. Fidelity interpretation. Text Preservation is 37.8/100 for T2I FT and 30.1/100 for Edit RL, while audits report corruption in 15/200 and 31/400 outputs. These measures need not conflict, but aligned checkpoints and criteria are needed to determine complete value-and-label correctness.

  4. Independent RL evidence. Human and external-judge checks are valuable, but baseline preferences do not isolate FT versus RL. Since aesthetic/contextual scores exclude data errors, overall gains also need to be read alongside fidelity outcomes.

Questions for authors
  1. What does aesthetic/contextual curation add over matched chart-domain supervision?
  2. Which evidence supports improved comprehension, or should those claims be narrowed?
  3. How do the audit counts relate to Text Preservation on the same outputs and checkpoints?
  4. What independent evidence distinguishes FT from RL while accounting for fidelity?
1 comment

We thank the reviewer for recognizing the value of ChartCraft’s paired supervision and the improvements over the corresponding backbones. We provide additional human evaluation results and clarify the scope of our evidence below.

  1. Contribution of aesthetic and contextual curation

Human evaluation provides additional evidence about the quality of the curated training targets. Conventional charts received an average aesthetic rating of 2.4/5, compared with 3.9/5 for ChartCraft’s synthetic training charts. The ChartCraft assessment involved 10 human evaluators and over 500 training samples.

These results support the aesthetic quality of our curated supervision. We agree, however, that this comparison does not replace a matched chart-domain training control. Our comparisons between the original backbones and the fine-tuned models establish the combined benefit of ChartCraft supervision, but do not separately quantify the contributions of domain adaptation and aesthetic/contextual curation. We will clarify this distinction and avoid attributing the entire improvement to individual construction stages.

  1. Evidence supporting improved information extraction

We conducted a user study involving more than 20 participants, measuring information-extraction accuracy and efficiency. Compared with conventional charts, participants exhibited an 11% reduction in information-extraction error rate and a 13% reduction in task completion time.

These behavioral measurements provide evidence beyond aesthetic preference: they support improved information-extraction accuracy and efficiency in the evaluated tasks. We will include the study protocol, comparison baseline, and results, and limit broader claims about comprehension and usability to the tasks and conditions actually evaluated.

  1. Relationship between Text Preservation and the failure audit

We checked the audit criteria and confirmed a difference in evaluation scope. The numerical/label-corruption category in Table 12 counts incorrect numerical values or labels, but does not include missing text in this category. In contrast, Text Preservation evaluates both the completeness and correctness of titles, axis labels, and numerical values. It uses per-output scores of 0/1/2, subsequently aggregated and rescaled to 0–100.

An output can therefore receive a low Text Preservation score because required textual elements are missing, while not being counted as numerical/label corruption. Accordingly, the counts of 15/200 and 31/400 describe this specific failure category; they do not establish complete value-and-label correctness for the remaining outputs. Likewise, the Text Preservation scores of 37.8 and 30.1 are graded scores, rather than percentages of fully correct charts.

We agree that omissions are also fidelity failures. We will make the distinction between incorrect and missing content explicit, identify the checkpoints and output sets associated with each analysis, and clarify that Table 12’s corruption counts should not be interpreted as an overall fidelity success rate. The low Text Preservation scores remain an important limitation, even when explicit numerical/label corruption is less frequent.

  1. Independent evidence for RL over SFT, alongside fidelity

We conducted a direct human comparison between SFT and RL on 150 editing examples, with 10 evaluators per example. Model identities were hidden, and left/right placement was randomized. After aggregating judgments within each example, 46% favored RL, 23% favored SFT, and 31% were rated “both good.” This directly supports an incremental preference benefit from RL over SFT.

Separately, the independent external judge in Table 7 corroborates this ordering: Qwen3.5-27B scores increase from 55.4 for SFT to 62.1 for RL. This judge was not used for reward optimization.

We also agree that preference and aesthetic improvements must be considered alongside fidelity. Table 4 reports increases from 47.5 to 51.2 in DSI and from 25.4 to 30.1 in Text Preservation after RL. Thus, independent evidence of increased preference is accompanied by improvements in CraftEval’s fidelity dimensions. These scores do not establish complete numerical correctness. We will present the preference and fidelity findings together and explicitly acknowledge the remaining text-preservation limitations.

Reviewer_2QvFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ChartCraft, a large-scale dataset for aesthetic and context-aware chart generation, covering four chart types and six generation and editing tasks. The authors fine-tune Qwen-Image models using SoRA and RL, and introduce CraftEval to evaluate data integrity, aesthetic quality, and style alignment. Experiments show substantial improvements over the base models, supported by additional human and independent-judge evaluations.

Critical strengths
  1. A valuable dataset with diverse visual styles and contextual information. ChartCraft addresses the limited aesthetic and contextual diversity of existing chart datasets. The examples in Figure 1 illustrate the richer visual styles and contextual details provided by the dataset. I see this as the main contribution of the paper.
  2. Clear improvements in chart generation and editing. The fine-tuned models achieve substantial gains over their base models across multiple tasks. The authors also provide human evaluations and additional validation experiments, which strengthen the experimental results.
Critical weaknesses
  1. The quality and diversity of the synthetic data need stronger validation. The dataset is expanded from approximately 96K source images using synthetic generation, including 23 aesthetic styles and five user personas. However, the motivation for selecting these categories and their distribution in the final dataset are unclear. While the authors check data consistency, there is limited independent validation of the aesthetic quality of the generated training samples. It is also unclear whether the trained models generalize to unseen styles or mainly learn the visual preferences of the generating model.
  2. Concerns about the reliability of CraftEval. CraftEval relies on Gemini-3-Pro for scoring, which is also used as the reward model during RL training. Although the authors provide additional human evaluations, Table 6 reports only 69.9% and 68.6% pairwise ordering consistency for AQA and CSA, respectively. The stability of the scores across repeated evaluations is also unclear. Furthermore, the independent human preference study in Table 7 compares the trained models against the original baseline, but does not directly establish whether the improvement from RL over SFT is preferred by humans.
  3. A minor issue: Figure 4 contains two unlabeled boxes in the data construction pipeline, which should be corrected.
Questions for authors
  1. How were the 23 aesthetic styles and five user personas selected? Was the aesthetic quality of the synthetic training data independently validated?
  2. Have the authors evaluated generalization to unseen aesthetic styles?
  3. How consistent are CraftEval scores across repeated runs? Does a direct human comparison between the SFT and RL models support the reported RL improvements?
2 comments

We thank the reviewer for recognizing the value of ChartCraft’s dataset and its improvements in chart generation and editing. We clarify the design choices and provide additional validation below.

  1. Selection of styles and personas, and independent validation of training-data quality

The 23 aesthetic styles and five user personas were manually curated by the authors in consultation with professional graphic designers. The styles span five groups—Modern & Tech, Retro & Nostalgia, Artistic & Organic, Clean & Professional, and Fun & Bold—covering diverse palettes, typography, textures, and layouts. The five personas—Executive, Artist, Analyst, Gamer/Geek, and Minimalist—introduce different communication styles and design priorities into user instructions. These choices aim to broaden the coverage of visual treatments and user requests. We will clarify this rationale and the sampling procedure, and report the empirical style/persona distributions after filtering in the revision.

We also conducted an independent human assessment of the synthetic training data. Ten human raters evaluated more than 500 training samples on a five-point aesthetic-quality scale, yielding a mean score of 3.9/5. This directly evaluates the aesthetics of the synthetic training samples and complements the data-consistency checks described in the paper.

  1. Generalization to unseen aesthetic styles

We evaluated generalization to two held-out styles, Hand-Drawn Sketch and Memphis Design, using human assessment. The generated outputs received an overall mean human rating of 3.8/5 across the two styles. This provides evidence of transfer to styles held out from task-specific training. We will describe the held-out-style evaluation explicitly in the revision and limit the generalization claim to the tested styles.

  1. CraftEval agreement with humans and repeated-run stability

We agree that the originally reported 69.9% AQA and 68.6% CSA ordering consistency provides only partial evidence of alignment with human judgments. Following the pairwise-ordering consistency protocol in Table 6, we expanded the human evaluation panel from 3 to 10 raters. The updated overall human–Gemini ordering consistency reached 87%. This result concerns overall-score ordering; dimension-specific agreement should be interpreted separately.

We additionally assessed repeated-run stability. CraftEval scores exhibited approximately 1.5% variation relative to their mean, and RL consistently outperformed SFT across all evaluation runs. These observations support the stability of the measured RL improvement under repeated scoring. We complement this evidence with the direct human comparison and independent-judge evaluation described below.

  1. Direct human comparison between SFT and RL

The reviewer correctly observes that the human comparisons in Table 7 were conducted against the original backbone. We now provide a direct comparison between ChartCrafter-Edit-FT and ChartCrafter-Edit-RL on 150 editing examples, with 10 human ratings per example, totaling 1,500 judgments. Model identities were hidden, and the left/right presentation order was randomized.

After aggregating ratings within each example, the preference distribution was 46% for RL, 23% for SFT, and 31% for “both good” (rounded percentages). This provides direct human evidence supporting the incremental benefit of RL over SFT.

The independent Qwen3.5-27B evaluation in Table 7 also supports this improvement: its overall score increases from 55.4 for SFT to 62.1 for RL, corresponding to a +6.7 improvement. This judge was not used for reward optimization. Together, these evaluations support the improvement beyond the Gemini-based reward and evaluation pipeline.

  1. Figure 4

Thank you for identifying the missing labels. We will add labels to both boxes and verify their consistency with the pipeline description.

Reviewer_2QvFOpenReview ↗

Comment

Thanks for the comprehensive response. While the authors have clarified my concerns, I still find the core contribution lacks sufficient novelty. Therefore, I will keep my score unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.