acceptodds
Withdrawn from ICLR 2027

See It Before You Edit: A Large-Scale Dataset for Operation-Aware Image Editing Anticipation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

With the development of multi-modal large language models, many fields have begun to introduce related agents, among which photographer agents are one example. Existing agents are typically trained on holistic parameter data, outputting Lightroom editing parameters for images, yet lacking the ability to anticipate the results of their operations. This paper introduces SIBYE, a tripartite dataset comprising about 190,000 image pairs and operation parameters, corresponding to cropping, global editing, and masked local editing in photographic post-processing. The cropping subset provides about 47,000 compositional variations, generated using normalized cropping ratios along the top, bottom, left, and right edges. The editing subset includes about 100,000 global adjustments based on 10 basic operations, 24 HSL color grading operations, and 6 color calibration operations in Lightroom. The masking subset contains about 43,000 linear and radial gradient masks, with 10 basic operations applied per mask type to achieve local editing. Each data entry is a quadruple (original image, result image, operation type, operation parameters), enabling the model to learn the forward mapping from operations to visual outcomes and the reverse inference from results to operations. We design tasks of varying difficulty, including single-operation classification, operation parameter prediction, multi-operation parameter prediction, among others—and establish benchmark experiments to validate the dataset’s effectiveness and challenges. This dataset will provide critical support for research areas such as interpretable image editing, operation-aware generative models, and intelligent photography assistants. The dataset can be downloaded at https://doi.org/10.5281/zenodo.19369384.

Primary area:
datasets and benchmarks
Keywords:
photo retouching, image editing dataset, operation parameter prediction, multimodal large language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_MDZhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduced SIBYE, a dataset of about 194,000 image-editing examples that pair input and edited images with explicit operations and parameters, covering cropping, global adjustments, and local masked edits. Fine-tuning Qwen3-VL on this dataset improved operation recognition and parameter prediction, with additional gains on an external photo-retouching benchmark.

Critical strengths
  1. By showing the effects of individual operations, this work makes it easier to study whether the models actually understand what different editing controls do. Generating the results with real editing tools also gives the examples clear, concrete supervision.

  2. Fine-tuning raises ten-class operation recognition accuracy from 28.92% to 89.97% and cropping IoU from 0.4133 to 0.9069. The external benchmark evaluation also provides some evidence that the benefits extend beyond the dataset itself.

Critical weaknesses
  1. Most tasks ask the model to identify operations or recover parameters from an input–target image pair. This shows that the model can better interpret an existing edit, but it does not establish that it can predict an operation’s visual effect before seeing the result.

  2. In the experiments, the combined training setting adds 5,000 SIBYE examples to the MMart data, so it is hard to separate the value of SIBYE from the benefit of more training data. The aesthetic score gain is also small, from 7.665 to 7.738, with no uncertainty estimates reported.

Questions for authors
  1. Could you please add a direct anticipation test, for example, giving the model an input image and editing parameters, then asking it to select the correct result from several candidates? Closely spaced parameter values would be particularly useful here.

  2. Would the performance gains in experiments hold with matched training set sizes and update counts? Could you also report variation across runs or confidence intervals?

  3. In Exp. F, the valid output counts differ considerably across models: JarvisArt produces only one valid output, while your model produces 168. Is MSE calculated only over valid outputs, and how are failed outputs reflected in the overall comparison?

  4. What explains the lower accuracy on the full 40-operation task, where your model achieves 52.83% compared with Gemini’s 56%? A per-operation breakdown or confusion matrix would help show which edits remain difficult to distinguish.

Reviewer_w8jGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces SIBYE, a dataset for operation-aware image editing anticipation. Built upon the MIT-Adobe FiveK dataset, SIBYE contains approximately 194K samples generated through cropping, global color adjustments, and mask-based local editing using Photoshop/Lightroom operations. Each sample provides the original image, edited result, operation type, and corresponding parameters. The authors also define several difficulty levels and evaluate the dataset on seven tasks involving operation recognition, parameter prediction, cropping localization, and editing reconstruction.

Critical strengths
  1. The paper focuses on a potentially useful problem: enabling multimodal models to recognize editing operations and understand their visual effects rather than directly predicting complete editing sequences.

  2. The authors provide benchmark experiments and publicly release the dataset and generation pipeline, which may facilitate reproducibility.

Critical weaknesses
  1. Limited novelty and technical contributions. The primary contribution is a dataset constructed by applying predefined Photoshop/Lightroom operations to existing images from MIT-Adobe FiveK. The overall construction process consists of randomly sampling operation types and parameter values, applying the corresponding edits, and filtering samples based on simple geometric constraints or pixel-level differences.

  2. Incremental methods. The dataset-construction procedures are straightforward to implement using existing image-editing software and scripts. The paper does not introduce a novel data collection strategy, annotation methodology, learning framework, or technically challenging synthesis process. While dataset contributions do not necessarily require methodological novelty, they should establish substantial value through unique data, challenging annotations, or previously unavailable evaluation capabilities. The current work does not sufficiently demonstrate such contributions.

  3. The experimental improvements do not sufficiently establish the dataset's practical value. Most experiments (Exp. A–F) evaluate models on synthetic samples generated using the same operation definitions and sampling procedures as the training data. Under this setting, strong improvements after fine-tuning are not particularly surprising, since the model is explicitly trained to recognize the types of transformations used to construct the evaluation samples.

Questions for authors
  1. What specific capabilities does SIBYE provide that cannot be obtained by applying standard editing augmentations to existing photo-editing datasets? Can the authors demonstrate a concrete advantage over simpler synthetic augmentation baselines?

  2. Can the authors demonstrate improvements in practical agentic editing tasks, such as iterative editing, operation planning, or forward prediction of edited image appearances, rather than mainly evaluating inverse operation recognition?

Reviewer_BwEBOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces SIBYE, a dataset of around 190K image editing examples covering cropping, global adjustments, and local masked edits. Experiments on SIBYE improves operation recognition and parameter prediction, with some gains on a downstream photo retouching benchmark.

Critical strengths

The dataset covers a useful range of editing operations, including cropping and local masks, with parameter labels tied to actual outputs from the editing software. The gains over the same base model are also clear, suggesting that the data is useful for learning to recognize operations and predict their parameters.

Critical weaknesses
  1. The title emphasizes anticipation, but the experiments mostly test operation recognition, parameter recovery, and the error after applying predicted edits. These results do not directly show whether the model can predict what an image will look like given an input image and editing parameters. Figure 1 illustrates this task, but there is no clear experiment testing it.
  2. Some sample counts do not add up. Table 1 reports 193,953 samples, but the training and validation sets total 187,869. Also, Exp. B expands from 10 to 40 operations under the same threshold, yet the training set drops from 14,931 to 5,221 samples without an explanation.
  3. The transfer gains may partly come from using more data and taking more training steps. The mixed setting adds 5,000 SIBYE samples to the 2,309 MMart samples, so training both settings for one epoch does not control for this. The aesthetic score improvement from 7.665 to 7.738 is also small, and it is unclear how much it varies across runs.
Questions for authors

Do you directly evaluate whether the model can anticipate the result of a given edit, as illustrated in Figure 1?

Could you clarify the missing 6,084 samples in Table 1 and explain why Exp. B uses fewer training samples than Exp. A?

Does the transfer improvement hold when both settings use the same number of training steps? How consistent is the aesthetic score gain across repeated runs or evaluations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.