- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduced SIBYE, a dataset of about 194,000 image-editing examples that pair input and edited images with explicit operations and parameters, covering cropping, global adjustments, and local masked edits. Fine-tuning Qwen3-VL on this dataset improved operation recognition and parameter prediction, with additional gains on an external photo-retouching benchmark.
Critical strengths
-
By showing the effects of individual operations, this work makes it easier to study whether the models actually understand what different editing controls do. Generating the results with real editing tools also gives the examples clear, concrete supervision.
-
Fine-tuning raises ten-class operation recognition accuracy from 28.92% to 89.97% and cropping IoU from 0.4133 to 0.9069. The external benchmark evaluation also provides some evidence that the benefits extend beyond the dataset itself.
Critical weaknesses
-
Most tasks ask the model to identify operations or recover parameters from an input–target image pair. This shows that the model can better interpret an existing edit, but it does not establish that it can predict an operation’s visual effect before seeing the result.
-
In the experiments, the combined training setting adds 5,000 SIBYE examples to the MMart data, so it is hard to separate the value of SIBYE from the benefit of more training data. The aesthetic score gain is also small, from 7.665 to 7.738, with no uncertainty estimates reported.
Questions for authors
-
Could you please add a direct anticipation test, for example, giving the model an input image and editing parameters, then asking it to select the correct result from several candidates? Closely spaced parameter values would be particularly useful here.
-
Would the performance gains in experiments hold with matched training set sizes and update counts? Could you also report variation across runs or confidence intervals?
-
In Exp. F, the valid output counts differ considerably across models: JarvisArt produces only one valid output, while your model produces 168. Is MSE calculated only over valid outputs, and how are failed outputs reflected in the overall comparison?
-
What explains the lower accuracy on the full 40-operation task, where your model achieves 52.83% compared with Gemini’s 56%? A per-operation breakdown or confusion matrix would help show which edits remain difficult to distinguish.