- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TRAJSP, a structured prompt enhancement method based on visual feedback from the generation trajectory, aiming to address the issue that traditional prompt enhancement methods cannot perceive intermediate states during generation. Built on unified multimodal models, the method uses an early-stage Polisher to convert visual information from noisy images into structured generation conditions, and a late-stage Validator to check generation results, revising conditions and regenerating when necessary. In addition, the authors construct the GSP-30K dataset and adopt a two-stage training strategy to enhance the model's noisy visual understanding and structured condition generation capabilities. Experiments show that the method achieves performance improvements over base models on multiple benchmarks such as GenEval, WISE, and GenAI-Bench.
Critical strengths
1.The research problem is practically meaningful, and the framework design is reasonable. This paper addresses the issue that traditional prompt enhancement methods cannot leverage intermediate visual states during generation, which is of research value. By reusing the unified multimodal model's own visual understanding capability, TRAJSP introduces structured condition updates based on the generation trajectory into the denoising process, without requiring an additional independent prompt enhancement model at inference time. This provides a practical way to coordinate understanding and generation capabilities within unified multimodal models.
2.Performance improvements are achieved on multiple benchmarks, with corresponding ablation validation. TRAJSP outperforms the SenseNova-U1 baseline on multiple evaluation benchmarks, including GenEval, WISE, and GenAI-Bench. Ablation experiments further show that, compared with ordinary fine-tuning, the Polisher brings clear performance gains, while the Validator provides additional improvements on some metrics. These results provide experimental evidence for the effectiveness of the proposed framework.
Critical weaknesses
-
Verification of the independent contribution of Trajectory-Grounded Feedback remains insufficient. The paper shows that the Polisher can significantly improve performance over ordinary fine-tuning, but current experiments still cannot distinguish whether the gains come from "structured prompting itself" or from "visual feedback in the current generation trajectory." In particular, given that VD-SP has already demonstrated that structured prompting alone is highly effective, the paper lacks a controlled experiment directly comparing text-only structured prompting with trajectory-grounded structured prompting under the same model, schema format, and training settings. Therefore, the paper's core Trajectory Grounding contribution has not yet been adequately isolated and validated.
-
The Polisher's ability to correct erroneous intermediate states lacks sufficient analysis. The schemas in GSP-30K are constructed as descriptions of actual image content, so even if the generated image does not fully satisfy the original prompt, the corrected schema still remains consistent with the current image. Consequently, the Polisher's supervision more directly encourages the model to understand and describe the current generation state, rather than explicitly modeling "the deviation between the current state and the target prompt." When the intermediate image already contains errors in count, attributes, or relations, whether such image-grounded schemas can help the model correct errors, or mainly reinforce the existing composition, has not been systematically verified.
Questions for authors
-
The paper's core contribution is to use intermediate generation states to achieve trajectory-grounded structured prompting. Table 2 already shows that the Polisher brings clear improvements over ordinary fine-tuning, but this result still does not fully disentangle the effects of structured prompting itself and trajectory-grounded visual feedback. Can the authors compare, under the same model, schema format, training data, and inference settings, a method that generates schemas based only on the original prompt with one that generates schemas based on the original prompt plus intermediate states? This comparison would more directly validate the independent contribution of intermediate visual trajectory information.
-
The schemas in GSP-30K are corrected to be consistent with the actual image content, so the Polisher more directly learns to understand and describe the current generation state. Can the authors further analyze the Polisher's behavior when the intermediate state has already deviated from the original prompt, for example, when incorrect object counts, attributes, or spatial relations appear? Specifically, can these image-grounded schemas help subsequent denoising recover from errors, or are they mainly used to refine and consolidate content that has already been formed? Reporting recovery effects separately on "semantically correct" and "semantically incorrect" intermediate states would help clarify the functional boundary between the Polisher and the Validator.
-
Can the authors report the Validator's automatic trigger rate, error detection accuracy, and the proportion of cases that actually improve after regeneration is triggered? Figure 8 currently uses forced regeneration, so it cannot directly reflect the true effectiveness of the automatic Validator.
If these concerns can be convincingly addressed through additional controlled experiments or analyses in the rebuttal, I would be open to revising my evaluation upward.