acceptodds
Withdrawn from ICLR 2027

EditJev: System One Routing for Image Editing Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Instruction-based image editing models exhibit pronounced sample-specific performance variation: different models can produce substantially different results for the same source image and editing instruction, and the best-performing model often varies across requests. These models also differ substantially in inference latency, making fixed-model deployment inefficient. We therefore study image editing model routing, which dynamically selects an editor from a candidate pool given an input image and editing instruction. We introduce EditJev, a System One routing model that directly predicts sample-level preference scores over candidate image editing models in a single forward pass. A lightweight preference head on a vision-language backbone produces a fixed-dimensional score vector without autoregressive model-name generation or response parsing. To train EditJev, we construct pairwise preferences from offline candidate outputs and optimize a pairwise ranking objective to learn relative model suitability. At inference time, EditJev combines the predicted preference scores with model-level speed scores to balance editing quality and generation latency. Experiments on ImgEdit and GEdit-En show that EditJev improves aggregate quality over the strongest fixed editor, from 4.540 to 4.592 and from 8.194 to 8.255, respectively, while reducing estimated end-to-end latency by 60.8% and 31.7%. These results demonstrate the effectiveness of direct, structured model routing for efficient image editing.

Primary area:
generative models
Keywords:
Image Editing, Model Routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hhzNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper addresses model routing for instruction-based image editing. Since editing models differ in which requests they handle well and in latency, the paper proposes to pick a model per request from the input image and instruction alone. The proposed method, EditJev, adds a small preference head on a VLM backbone that scores every candidate model in one forward pass, and combines these scores with a weighted speed score at inference. The authors claim that on two benchmarks it slightly outperforms the strongest fixed model while greatly reducing latency.

Critical strengths

S1. The problem setting is practical and clearly stated. Candidate editing models differ greatly in latency, so choosing a model per request is a meaningful problem for deployment. The authors formulate it cleanly as an objective over both quality and latency. The evaluation protocol, which precomputes benchmark results for all candidates and evaluates routing by lookup, is reproducible and can be reused directly by follow-up work.

S2. The method is explained clearly. The text and figures (especially Figure 2) make the idea easy to follow, and the method is evaluated on two benchmarks with several ablations.

Critical weaknesses

W1. The presented evidence cannot separate the benefit of sample-level routing from a simple category rule. If each model is clearly better at certain editing types, a rule built from that pattern would likely perform similarly without training a head. An MLLM given the per-model characteristics as meta information might do the same. The paper does not analyze this pattern or check whether the actual routing results follow it. It also has no comparison with rule-based routing, random routing or a per-request oracle. Without these I cannot see why a trained head is needed. Moreover, Figures 1 and 3 and the text describe the method as if each model has its own area of strength and EditJev sends each request to the model that edits it best. The quantitative results show nothing beyond choosing the faster model when the preferences are similar. Latency does drop substantially. For quality alone, however, using Qwen-Image-Edit by itself is practically sufficient. Either the claim should be narrowed or the paper needs analysis showing that routing does more than trade off latency.

W2. The differences in the results are marginal but interpreted as large, and the comparison is not fair. Quality is almost the same as the single model, and only the latency reduction can be credited. Claims that the quantitative results do not support are backed by qualitative examples instead. The ablation variants show almost no difference compared with the score gaps in Table 1, yet the authors draw conclusions from them. In addition, the MLLM routing baselines choose without knowing anything about the individual models, and I expect them to be close to random. Comparing them with the trained EditJev does not show whether a trained head is necessary.

Questions for authors
  1. Can you summarize which editing types each model is good at and show whether the actual routing results follow that pattern? What is the performance when routing by rule using this pattern, or when the per-model characteristics are given to an MLLM as meta information? Please also report a per-request oracle and random routing.
  2. Can you show that the score differences in the main results and ablations are meaningful? Variance or confidence intervals over seeds or repeated scoring are needed.
  3. Is the VIEScore-based preference score reliable enough? In my experience an MLLM judge gives different scores even for the same sample. If each candidate output was scored separately, there is no guarantee that the scores are comparable across candidates.
  4. Why do the hyperparameters differ so much between benchmarks, and how were they chosen? Pairs with a score difference below 0.5 are discarded in training. Does this not make small score differences hard for the model to predict?
  5. The min-max normalized speed score depends heavily on the candidate set. A single very fast but low-quality model can compress the speed scores of the remaining models. An analysis of this would be helpful.
  6. Please show failure cases of the routing, and discuss the deployment cost of keeping all candidates loaded on the server as a limitation.
Reviewer_PaoXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper formulates sample-level model routing for instruction-based image editing, aiming to select the best editing model from a fixed pool of K=5K=5 open-source editors (flux2_klein_4b, flux2_klein_9b, qwen_image_edit, kontext, and omnigen2) for each input image and instruction prior to image generation. The proposed framework, EditJev, attaches a 5-output linear preference head onto a LoRA-tuned Qwen3-VL-4B-Instruct backbone ("System One" single forward pass without autoregressive token generation). To construct offline training supervision on ~53k image-instruction pairs, all 5 editors are executed and scored by an MLLM judge (Qwen3-VL-32B with VIEScore prompts); candidate pairs whose score gap exceeds δ=0.5\delta=0.5 are used to train the router via a pairwise softplus ranking loss. At inference time, EditJev combines the predicted preference score u^i\hat{u}_i with a pre-measured static normalized model speed score sis_i via i^=arg⁡max⁡i(u^i+βsi)\hat{i} = \arg\max_i (\hat{u}_i + \beta s_i). Experiments on ImgEdit and GEdit-En show that EditJev outperforms individual models and zero-shot MLLM routers while reducing average latency relative to the slowest editor (omnigen2).

Critical strengths
  1. Practical problem formulation with negligible routing overhead: Routing image-editing requests across heterogeneous editors of varying speed and specialization is a practically relevant deployment problem. Formulating the router as a single-pass classification/preference head on top of a 4B VLM adds only ~0.04s of overhead—negligible compared to diffusion generation times (4s–38s)—while allowing inference-time latency control via β\beta.
  2. Consistent empirical gains over zero-shot MLLM routers: Zero-shot prompting of general-purpose LLMs/MLLMs (e.g., GPT-5.5, Gemini-3.7-Flash) performs near random guessing when selecting among specialized image editors from text/image inputs alone, whereas supervised pairwise preference learning effectively captures editor-specific strengths across editing categories.
Critical weaknesses
  1. Closed-loop evaluator circularity (training and testing on the exact same MLLM judge): In Appendix A, the authors state that the 53k training pairs are scored using VIEScore with Qwen3-VL-32B as the judge. In Section 4.1, the main benchmark evaluations on ImgEdit and GEdit-En also appear to use the exact same automatic evaluator (VIEScore / Qwen3-VL-32B) to report the main quality tables. Training a Qwen3-VL-4B student to predict which candidate Qwen3-VL-32B prefers, and then evaluating the resulting router using Qwen3-VL-32B's own scores, creates a closed-loop circularity: the router may simply learn the idiosyncratic biases of the Qwen3-VL judge rather than true editing quality. Moreover, the marginal overall score gain over simply always picking omnigen2 (4.5916 vs. 4.5446 on ImgEdit, a +0.047 difference on a 5-point scale) or qwen_image_edit (4.4995 vs. 4.4426 on GEdit-En) is very small and within typical MLLM judge variance.
  2. Rigid architecture tied to a fixed model pool and oversimplified static latency formulation: EditJev hardcodes the candidate model pool into a fixed KK-dimensional linear output head (Gϕ:Rd→RKG_\phi: \mathbb{R}^d \to \mathbb{R}^K) and relies on a fixed prompt string of model names. Adding a new editing model or removing an old one requires generating outputs for all 53k training samples and retraining the router from scratch, as the router learns no generalizable representation of candidate models. Additionally, the speed score sis_i (Eq. 4) is a single static constant per model (measured average runtime across dataset), ignoring sample-level resolution or step variations, and the main latency headline ("1.73x faster than slowest editor") simply reflects not calling omnigen2 (38.58s) on every sample—compared to qwen_image_edit (16.41s, score 4.4832), EditJev is actually 36% slower (22.33s) for a +0.108 score difference.
Questions for authors
  1. Independent evaluation judge and train-test split integrity (Clarification — Critical): Was the exact same judge (Qwen3-VL-32B with VIEScore) used both to generate the 53k training labels and to compute the test benchmark scores in Tables 1 and 2? If so, does EditJev still outperform single-editor baselines (omnigen2, qwen_image_edit) when evaluated by an independent judge family (e.g., GPT-4o / Gemini-2.5-Pro or standard ImgEdit/GEdit official evaluators) and human raters on a larger sample than 20 cases? Additionally, since the 53k training set draws source images and instructions from ImgEdit (Appendix A), how did the authors ensure strict zero overlap (both images and instruction templates) with the ImgEdit benchmark test set?
  2. Generalization to unseen or dynamically changing candidate pools (Clarification): Because GϕG_\phi is a fixed 5-way linear classifier head, how can EditJev handle adding a 6th editing model without running inference on all 53,000 training images and retraining the entire router?
  3. Stronger non-MLLM baselines (Clarification): How does EditJev compare against simple task-category routing (e.g., classifying the instruction into one of the 9–14 benchmark task categories and picking the single best model per category based on validation statistics)? Looking at Table 1, omnigen2 or qwen_image_edit dominates almost every individual category, suggesting a simple category lookup table might achieve nearly the same score without training a 4B VLM router.
Reviewer_7j7oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies model routing for instruction-based image editing: selecting an editor from a candidate pool before generation, based on the source image and editing instruction, to balance quality and latency. The proposed method trains a multimodal backbone with a linear preference head using pairwise preferences derived from offline candidate outputs, then combines predicted preference scores with speed scores at inference time. The problem is practically relevant, but the technical novelty and experimental support require further clarification.

Critical strengths
  1. The problem has practical value. Different editors exhibit complementary capabilities, and routing before generation can reduce unnecessary inference costs.
  2. The method is simple and clearly described. The paper evaluates both quality and estimated end-to-end latency and includes ablations on the training objective, visual input, and backbone size.
  3. On two benchmarks, the method achieves slightly higher average quality than the strongest fixed editor while substantially reducing estimated generation latency, suggesting potential deployment benefits.
Critical weaknesses
  1. The technical contribution beyond existing routing methods is insufficiently established. The method combines input encoding, candidate-model scoring, pairwise ranking, and cost-aware selection. These components closely relate to existing work on preference-based routing, learning to rank, and quality-cost optimization. RouteLLM already studies routing learned from preference data, while RouterBench considers selection based on predicted performance and weighted cost. The clearest contribution therefore appears to be adapting these techniques to instruction-based image editing and constructing the corresponding supervision. The authors should explain which challenges are specific to image editing, why existing routers cannot adequately address them, and which proposed components are necessary to overcome these challenges. The “System One” terminology does not, by itself, establish a distinct technical mechanism.

  2. The main baselines differ substantially in training and access to candidate capability information. The general-purpose MLLM baselines receive the source image, instruction, candidate editor names, and average generation times. They are not provided with capability descriptions, demonstrations, or empirical performance records, and no task-specific training is reported. In contrast, EditJev learns candidate suitability from output evaluations associated with approximately 53,000 requests.These comparisons primarily demonstrate that a task-trained router outperforms simple prompted model selection. They do not adequately separate the benefits of training data from those of the routing architecture. Jev additionally lacks image input, and its training background and knowledge of the candidate editors are unclear. The evaluation would be stronger with supervised baselines trained on the same data, including a multiclass router, a lightweight router using frozen multimodal features, and adaptations of existing preference-based routing methods.

  3. EditJev uses the hidden state of the last non-padding token as the sole representation for predicting candidate-model preferences. However, the paper does not justify this choice or evaluate its impact. Although this position can attend to the preceding image and text tokens, this does not establish that it provides the most suitable representation for routing. Furthermore, because the input includes a chat template and generation-prompt suffix, it is unclear which token is actually used. The authors should clarify the exact token position and provide a controlled comparison with alternatives, such as mean pooling over valid tokens or a dedicated routing token, to assess the sensitivity of the results to this design choice.

  4. The two benchmarks also use substantially different speed weights, β=0.7 and β=0.02, without a sufficiently clear validation procedure. The ranking objective does not explicitly calibrate the preference-score scale, whereas speed scores are normalized to [0,1]. Their combination therefore depends on the learned score scale. The authors should explain howβand the final checkpoint were selected and whether test results informed either choice.

Questions for authors
  1. What challenges specific to instruction-based image editing require methodological changes beyond existing preference-based and cost-aware routing methods? Which component of EditJev addresses these challenges, and what evidence supports its contribution?

  2. How does EditJev compare with supervised routers trained on the same preference data and using the same multimodal backbone?

  3. Which exact token supplies the hidden state after applying the chat template and generation-prompt suffix? How sensitive is routing performance to this representation choice compared with alternatives such as mean pooling or a dedicated routing token?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.