- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an action-specific credit assignment method for the image-to-TikZ task. The policy first generates code, then revises or repairs it based on the rendered output or execution failure feedback. Both outputs are scored by their rendered similarity to the target image, with failed executions receiving a score of zero. The shared policy is then updated using a GRPO-like algorithm. Unlike trajectory-level GRPO that assigns the same final-outcome credit to both actions, this method assigns the final score to the initial generation and the score improvement to the corrective action. Group-relative advantages are computed separately for the two stages.
Critical strengths
- The method assigns action-specific credit using execution scores from different stages, without introducing additional trainable components or requiring human-annotated corrections.
- The evaluation distinguishes improvements in initial generation from gains due to correction and examines when refinement helps or harms an existing output.
Critical weaknesses
- The paper lacks a two-stage GRPO baseline with fixed rollout-policy probabilities. Although variant C assigns the final trajectory score to both stages, it uses the paper’s modified probability-ratio computation and therefore does not provide this comparison. Moreover, the continued single-stage GRPO baseline does not match the proposed method in generation cost or optimizer update count, making it difficult to isolate the benefits of the proposed credit assignment.
- The evaluation is limited to an initial generation followed by a single corrective action. It remains unclear whether the proposed credit assignment retains its advantage over trajectory-level credit assignment across multiple correction rounds. Experiments with longer interaction horizons would help assess its generality and the accumulation of beneficial or harmful revisions.
- Some panels in Figure 5 appear blurry, making fine visual details difficult to inspect. Higher-resolution images or enlarged crops would make the qualitative comparisons easier to assess.
- The quantitative ablation results are dispersed across the appendix. A consolidated table in the main text summarizing correction quality, generation performance, and training stability across all credit-assignment variants would make the comparisons easier to assess.
- The collapse of variant C in ablation may be confounded by the probability-ratio implementation. Recomputing the denominator from the current policy before each update keeps the ratio at unity under identical forward passes, rendering clipping inactive. Consequently, repeated updates on the collected trajectories lack the clipping-based safeguard against excessive deviation from the rollout policy. This may contribute to the observed instability, so the collapse cannot be attributed solely to shared final-outcome credit. Repeating the ablation with fixed rollout-policy probabilities would help distinguish the effects of credit assignment from those of the optimization implementation.
Questions for authors
-
Although variant C uses shared final-outcome credit, it retains the modified probability-ratio implementation. Could the authors provide a two-stage GRPO baseline with fixed rollout-policy probabilities and a matched training budget? This would help isolate the benefits of the proposed credit assignment from differences in interaction cost and optimizer update count.
-
The experiments consider only one corrective action. Does the proposed credit assignment retain its advantage over shared trajectory-level credit across multiple correction rounds? An evaluation with longer interaction horizons would help establish the scope of the findings.
-
Appendix B.1 explains that denominator log probabilities are recomputed from the current policy and reports that clipping remains inactive. What motivates this choice over keeping rollout-policy probabilities fixed? Could this implementation choice contribute to the collapse of variant C, and does the collapse persist when the ablation is repeated with fixed rollout-policy probabilities?