- Rating
- 2
- Confidence
- 4
- Informativeness
- 1(−1)
- Presentation
- 2
- Soundness
- 1(−1)
Read review
Paper summary
This paper presents FocusTikZ, a reinforcement learning framework for text-to-TikZ code generation. The method introduces a quality-aware reward that combines compilation success, code similarity, and three learned assessment models for logic, layout, and style. It further proposes Difference-and-Entropy Focused GRPO (DE-GRPO), which selectively optimizes tokens that differ from the reference code and exhibit high generation uncertainty. Experiments on SKeTikZ and three additional benchmarks demonstrate improvements in code quality, compilation success, and visual fidelity over general-purpose and task-specific baselines.
Critical strengths
1: Practical and well-motivated task. The paper addresses the challenge of generating executable and visually faithful TikZ diagrams, where syntactic correctness alone does not guarantee semantic or structural accuracy.
2: Consistent empirical improvements. FocusTikZ achieves competitive results across multiple benchmarks, outperforming strong task-specific baselines such as TikZilla on several visual quality metrics. The improvements are further supported by human evaluation.
3: Reasonably comprehensive experimental analysis. The paper includes ablations on training stages, reward components, and token selection strategies, as well as additional evaluations of the assessment models and repeated-run results.
Critical weaknesses
1: Limited methodological novelty. The central idea of selectively optimizing informative tokens in GRPO has been explored in prior works, including high-entropy token optimization and critical-token selection for visual generation. DE-GRPO primarily combines reference-based token differences with an existing entropy-based selection strategy. The paper does not sufficiently establish what fundamental methodological insight this combination introduces beyond existing approaches.
2: Insufficient justification for the token selection strategy. Reference differences do not necessarily indicate incorrect or visually important tokens, as different TikZ implementations can produce equivalent diagrams. Similarly, high entropy does not necessarily imply greater impact on visual quality. The intersection of these two sets is therefore largely heuristic. Although the ablations demonstrate its effectiveness, they do not establish that the selected tokens are genuinely more important to diagram quality.
3: Limited visual grounding of the quality reward. Despite emphasizing rendered diagram quality, the three assessment models evaluate textual descriptions and generated code rather than the rendered images themselves. Stronger comparisons with directly image-based reward models would be valuable.
-
Token Reward Design. The proposed token reward is essentially a code-level similarity measure based on the Jaccard similarity between token sets. However, this approach ignores token order, frequency, and structural relationships. What motivates this design, and what advantages does it offer over n-gram-based metrics or structure-aware alternatives such as AST similarity? A comparative experiment would help justify this choice.
-
Limited qualitative examples and diagram diversity. More qualitative comparisons would help demonstrate the superiority of the proposed method over existing baselines. Additionally, the evaluated diagram types are relatively limited, and broader coverage would strengthen the evaluation.
Questions for authors
See weakness above.
6 comments
Response to Reviewer K18a (W3, W4 and W5)
[Weakness 3] Visual validation and comparison with VLM rewards
We collected 250 generated candidates on 50 prompts from five models. Fifteen annotators rated logic, layout, and style from descriptions and rendered diagrams; AMs scored description–code pairs. Both human and AM scores were normalized.
Within-prompt agreement from revised Table 6. PA measures pairwise agreement excluding human-tied pairs; pooled tau-b retains tie correction.
| Dimension | PA (%) ↑ | Pooled tau-b ↑ |
|---|---|---|
| Logic | 95.12 | 0.8952 |
| Layout | 95.77 | 0.9117 |
| Style | 97.34 | 0.9364 |
| Overall quality | 96.40 | 0.9280 |
AM rankings closely track human visual judgments. Within-model correlations in Table 7 also remain strong across every generator.
We replace only the quality reward with an LLM judge (Qwen2.5-Coder-7B) or VLM judge (Qwen3-VL-8B), keeping other settings unchanged. The VLM receives the description and rendered image, scores each dimension from 0 to 4, and sums the scores after dividing each by 4. The LLM baseline was already included in Figure 4(b) and analyzed in Appendix E.4.
Quality-reward comparison from revised Table 8.
| Quality reward | LPIPS ↓ | PSNR ↑ | MS-SSIM ↑ | CSR ↑ |
|---|---|---|---|---|
| LLM-as-a-Judge | 0.210 | 25.547 | 0.806 | 0.960 |
| VLM-as-a-Judge | 0.242 | 25.139 | 0.809 | 0.941 |
| Three AMs | 0.189 | 26.499 | 0.828 | 0.963 |
We further compare the AM- and VLM-reward generators on 200 shared prompts using DeepSeek-V4.1-Flash as an independent description–image judge.
Description-based error rates from revised Table 5 (%).
| Reward | Node ↓ | Edge ↓ | Relation/layout ↓ | Label ↓ | Style ↓ | Fully compliant ↑ |
|---|---|---|---|---|---|---|
| VLM | 34.0 | 31.0 | 39.0 | 11.5 | 8.5 | 38.5 |
| AMs | 29.5 | 22.0 | 35.0 | 7.5 | 4.0 | 55.5 |
Error categories may overlap; fully compliant means no identified error. AM rewards yield better generation metrics and fewer description violations than the tested VLM reward, supporting effective visual-quality supervision.
Revision: Tables 5–8; Appendix G.4, lines 1842–1877; Section 5.5, lines 519–528; and Appendix G.2, lines 1784–1809, report these analyses.
[Weakness 4] Jaccard reward motivation and AST comparison
Different TikZ command orders can render the same diagram. Unlike order-sensitive n-gram matching, Jaccard supplies an order-insensitive overlap signal. The initially lower AM rewards in Figure 24 further motivate complementary code-level guidance early in RL.
We replace only Jaccard with normalized AST similarity in [0,1], keeping other settings unchanged.
Auxiliary reward ablation from revised Table 26.
| Auxiliary reward | LPIPS ↓ | PSNR ↑ | MS-SSIM ↑ | CSR ↑ |
|---|---|---|---|---|
| Jaccard | 0.189 | 26.499 | 0.828 | 0.963 |
| Normalized AST | 0.195 | 26.357 | 0.817 | 0.952 |
| None | 0.204 | 26.126 | 0.814 | 0.932 |
Both auxiliary rewards improve over removing the term, while Jaccard performs best in this comparison, supporting its complementary role alongside quality assessment.
Revision: Table 26 and Appendix G.8, lines 2029–2053, explain the design and comparison; Figure 24, lines 2089–2090, shows the reward trajectories.
[Weakness 5] Qualitative coverage and diagram diversity
Figure 25 adds comparisons with DeepSeek-V3, Qwen3-Max, and GPT-5.4-Thinking across graphs, geometry, grid paths, scientific plots, bar charts, and function plots. Appendix E.5 also evaluates external-benchmark diagram types not represented in the SKETIKZ test set.
Cross-type results from revised Tables 16 and 18.
| Dataset | Method | LPIPS ↓ | PSNR ↑ | MS-SSIM ↑ |
|---|---|---|---|---|
| DaTikZ-V1 | Llama-3.1-Instruct | 0.188 | 26.059 | 0.849 |
| DaTikZ-V1 | DiagramCodeAgent | 0.294 | 22.016 | 0.736 |
| DaTikZ-V1 | FocusTikZ | 0.163 | 26.970 | 0.866 |
| DiagramGen | Llama-3.1-Instruct | 0.320 | 21.832 | 0.718 |
| DiagramGen | DiagramCodeAgent | 0.370 | 19.169 | 0.633 |
| DiagramGen | FocusTikZ | 0.307 | 22.227 | 0.725 |
FocusTikZ leads on LPIPS, PSNR, and MS-SSIM in both cross-type subsets, supporting broader diagram coverage beyond the main benchmark.
Revision: Figure 25 and Appendix G.10, lines 2142–2155, expand qualitative coverage. Appendix E.5, lines 1486–1514, explains the type-based evaluation; Tables 16 and 18, lines 1583–1598 and 1638–1653, report cross-type results.
Thank you for the response. However, it still does not fully address my major concerns.
Weaknesses 4 and 5 have been mostly addressed.
Regarding Weaknesses 2 and 3, I appreciate the additional experiments. However, I still do not see sufficient rationale or diagram-specific adaptation behind the token selection strategy. It still appears to be a relatively straightforward application of an established approach. Moreover, although the authors have added experiments on image-based quality rewards, I believe these should have been included in the original submission, as the paper emphasizes rendered diagram quality while relying on rewards that are not directly related to it. In my view, these two critical weaknesses concern the original motivation and method design, rather than simply a lack of experiments.
My most significant concern remains the novelty and positioning of the paper. After carefully reviewing the paper again, I still find limited methodological novelty in the overall design. This concern cannot be resolved merely by adding further experiments. I also notice that the paper has undergone huge revisions, which raises concerns about the completeness of the initial submission.
Overall, while I appreciate the authors' efforts and additional experiments, I cannot justify a rating higher than 2 based on my assessment. I therefore maintain a negative recommendation.
Response to Reviewer K18a (W1 and W2)
Dear Reviewer K18a,
Thank you for your valuable feedback. We are pleased that you recognized the practical motivation, consistent empirical improvements, and comprehensive experimental analysis. We address your concerns point by point below.
Beyond the 80/20 Rule selects high-entropy tokens for reasoning updates, while Group Critical-token Policy Optimization for Autoregressive Image Generation (GCPO) selects critical image tokens and adjusts token-level advantages.
TikZ code combines predictable syntax with diagram-specific coordinates, connections, and styles. Short fragments in the latter group can affect multiple rendered elements. Reference alignment filters out code fragments matching the reference, while entropy prioritizes uncertain decisions among the remaining tokens. Our method builds on these characteristics to select key token positions for more efficient policy updates.
Table 8 shows gains over entropy-only selection and controls with equal token budgets. Table 23 supports retaining entropy filtering, while Tables 24–25 show larger rendering changes from resampling selected positions. Together, these results demonstrate the effectiveness of our selection mechanism.
Revision: Based on this discussion, we strengthened the motivation and comparison with prior methods in the Introduction, lines 79–96. Table 8 and Section 5.5 present the supporting ablations, with further sensitivity and resampling results in Tables 23–25. Section 3.3 and Appendix A.3 explain the selection mechanism and its differences from prior methods.
We train controls by sampling the same number of tokens as DE-GRPO from the union of reference-difference and high-entropy positions, all eligible positions, or positions outside its mask, with the same loss normalization and other settings.
Matched-budget controls from revised Table 8.
DE-GRPO outperforms all controls, showing that the selected positions matter beyond update sparsity or count-dependent normalization.
We additionally mask DE-selected, random, or unselected positions in SFT-generated code under equal masking budgets and resample them with Qwen2.5-Coder-7B, preserving the remaining code.
Comparison with reference code and images from revised Table 24.
Comparison with original SFT code and images from revised Table 25.
Under equal masking budgets, resampling DE-selected positions produces larger rendering changes and greater losses in reference-image fidelity. Together with the training controls, this supports their visual relevance. Figures 22–23 illustrate the affected code fragments and renderings.
Revision: Table 8 and Appendix G.5, lines 1881–1893, report matched-budget controls; Tables 24–25 and Appendix G.7, lines 1921–1964, report resampling. Figures 22–23, lines 1992–1995 and 2022–2024, provide qualitative evidence.