acceptodds
Under review as a conference paper at ICLR 2027

Visual Abstention in Unified Multimodal Models

24 views

Abstract

Unified multimodal models (UMMs) integrate understanding and generation, yet their generative behavior is rarely governed by what they understand about the task. We formalize visual abstention: when a requested visual transformation is impossible under the task's rules, the model should recognize that no valid solution exists, state this, and decline to generate. We introduce Draw-or-Decline (DoD), a benchmark of 1,050 feasible–infeasible request pairs across 7 task categories that jointly measures editing success and the refusal of infeasible requests. Evaluating 8 UMMs, we find that editing ability and abstention are distinct capabilities: even the strongest editor, at 68.4% editing accuracy, refuses only 0.4% of infeasible requests under ordinary instructions. Their reasoning shows why: the models rarely notice the conflict, and instead plan the edit as if the request were possible, often describing objects that are not in the image, or quietly change the request into one they can complete. Explicitly prompting these UMMs to report infeasibility increases textual refusals but reduces editing accuracy. We propose VisTA (Visual Transformation and Abstention), a training method that pairs feasible and infeasible examples so that a model judges feasibility before deciding whether to generate. We train VisTA-BAGEL to perform feasible edits and decline infeasible requests. Without any reminder, it refuses 93.0% of infeasible requests, up from 0.4% for the strongest editor, while falsely refusing only 0.8% of feasible ones. Unlike a reminder, this does not cost editing accuracy: VisTA-BAGEL completes 74.3% of feasible edits, more than any of the 8 evaluated UMMs.

Primary area:
datasets and benchmarks
Keywords:
Visual Abstention, Unified Multimodal Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_oQJSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies visual abstention in unified multimodal models: recognizing when an image-editing request has no valid solution under its constraints and declining without generating an image. The authors introduce Draw-or-Decline (DoD), a benchmark containing 1,050 feasible–infeasible pairs across seven categories. Evaluations of eight existing models show that editing competence does not guarantee spontaneous refusal, while a simple reminder often increases both correct and false refusals. The proposed VisTA approach fine-tunes UniREdit-BAGEL using paired supervision for successful editing and explained abstention. Without an inference-time reminder, refusal success increases from 0.4% to 93.0%, with a false refusal rate of 0.8%, while editing success increases from 68.4% to 74.3%.

Critical strengths
  1. The paper identifies a meaningful limitation of generative models. Producing a plausible image is insufficient when the original request cannot be satisfied. Evaluating editing success, refusal success, and false refusals together provides a useful framework for assessing whether a model can decide when generation is appropriate. Tables 1–3 clearly illustrate why either editing accuracy or refusal frequency alone would be inadequate.

  2. The benchmark includes carefully controlled examples and verification. The six scene-based categories pair different instructions with the same image, while maze pairs retain the same instruction and change a single corridor cell. The latter design is particularly valuable because feasibility is verified by breadth-first search and cannot be inferred from instruction wording alone. Human review and the reported agreement between automated and human judgments further strengthen the evaluation.

  3. The main improvement in refusal behavior is substantial. VisTA achieves a large increase in refusal success while keeping false refusals low. Its refusal success is at least 88.0% in every category without a reminder, suggesting that the improvement is not confined to a single task. The results provide convincing evidence that targeted training can substantially improve abstention on DoD.

Critical weaknesses
  1. My main concern is that the aggregate editing improvement is primarily focused on maze-solving. Maze editing success increases from 51.3% to 88.0%, contributing approximately 5.24 percentage points to the overall improvement of about 5.9 points. Calculated from the reported category scores, the mean over the other six categories increases only from approximately 71.2% to 72.0%, with decreases in three categories. The improvement over the feasible-only control, 74.3% versus 72.0%, also comes from single training runs.

  2. Generalization beyond the benchmark’s construction process remains uncertain. The six “real-world” categories use synthetic images, and training and evaluation employ related categories and instruction-construction procedures. The deduplication checks and two category-holdout experiments are valuable, but they do not establish robustness to natural photos.

  3. The primary refusal metric only partially captures the formal definition of visual abstention. The formal definition requires both an explicit refusal and no generated image. However, Section 4.1 credits textual refusals even when an image follows, and the refusal verdict can receive credit despite an incorrect explanation.

Questions for authors
  1. Could you provide a consolidated comparison of textual refusal, refusal without image generation, explanation correctness, and their joint success rate across the evaluated methods?

  2. Could you evaluate natural photographs and independently authored instructions containing unseen types of infeasibility, together with closely matched feasible requests? For example, can the model reliably distinguish requests to modify an existing object from requests that creating it.

  3. You have convincingly demonstrated that the model has learned to refuse. I would also like to verify whether the improvement in editing scores stems primarily from the specialized maze training. After accounting for differences in training setup and random fluctuations, does training on infeasible samples also help the model better execute edits that are actually feasible? For instance, by conducting evaluations on other types of image editing tasks.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer oQJS (Part 1/2)

Thank you for the detailed review and for finding the refusal improvement convincing. We address each point below.

W1: My main concern is that the aggregate editing improvement is primarily focused on maze-solving. ... The improvement over the feasible-only control, 74.3% versus 72.0%, also comes from single training runs.

Q3: ... After accounting for differences in training setup and random fluctuations, does training on infeasible samples also help the model better execute edits that are actually feasible? For instance, by conducting evaluations on other types of image editing tasks.

R1: Thank you for this careful analysis. We agree that the editing gain of VisTA-BAGEL on DoD comes mainly from maze. As shown in Appx. C.7, this gain comes from the maze training pairs: when maze is held out of training, maze editing stays close to the initialization. We therefore see the main benefit of VisTA as adding refusal while preserving editing ability, rather than improving editing itself.

To account for random fluctuation, we trained VisTA-BAGEL and the feasible-only control (Appx. C.5) with two additional seeds. The table reports the mean and standard deviation over three runs, without a reminder.

ModelMaze EOther six EOverall ED
UniREdit-BAGEL51.3%71.2%68.4%0.4%
Feasible-only86.2 ± 4.5%70.4 ± 1.7%72.7 ± 1.7%0.3 ± 0.2%
VisTA-BAGEL88.0 ± 5.3%71.8 ± 1.8%74.1 ± 2.3%92.6 ± 1.0%

VisTA-BAGEL edits slightly better than the feasible-only control, but the gap (1.4% on both the six scene categories and overall) is within one standard deviation. Across three runs, we do not find clear evidence that infeasible-example training improves feasible editing. The main observed benefit is substantially stronger refusal.

We also evaluated on PhyEditBench [1], whose inputs are frames of real-world videos and whose feasible requests were written by its authors, independently of DoD. Of its 238 real-world instances, we use the 224 for which human annotators could write a valid infeasible counterpart on the same input image, giving 224 feasible-infeasible pairs.

ModelEDDf
BAGEL67.4%0.0%0.0%
UniREdit-BAGEL66.1%0.0%0.0%
VisTA-BAGEL64.3%90.2%4.5%

On this out-of-domain benchmark, editing likewise shows no gain. We therefore see the main benefit of VisTA as adding refusal while preserving editing ability, rather than improving editing itself.

[1] PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing

W2: Generalization beyond the benchmark's construction process remains uncertain. ... they do not establish robustness to natural photos.

Q2: Could you evaluate natural photographs and independently authored instructions containing unseen types of infeasibility, together with closely matched feasible requests? For example, can the model reliably distinguish requests to modify an existing object from requests that creating it.

R2: On PhyEditBench (see above), refusal carries over to real photographs and to infeasible requests written by human annotators: VisTA-BAGEL refuses most infeasible requests, while the baselines almost never refuse, and false refusals stay low.

To check whether the model separates creating an object from modifying an absent one, we classified each feasible DoD request in the six scene categories by whether it introduces an object not yet in the image. VisTA-BAGEL falsely refuses very few requests of either kind (0.5% and 0.4%), so on DoD it does not treat a newly requested object as a missing referent. On PhyEditBench, some of its false refusals come from requests that bring a new object into the scene. For “Drop a film strip from the top right so that it falls past the white pillow and lands on the bed surface”, the model reasons that “there is no film strip at the top right or elsewhere that could be set in motion”. Such creation requests are phrased as acting on the new object, and the model sometimes reads that object as an existing referent.

AuthorsOpenReview ↗

Reply to Reviewer oQJS (Part 2/2)

W3: The primary refusal metric only partially captures the formal definition of visual abstention. ... the refusal verdict can receive credit despite an incorrect explanation.

Q1: Could you provide a consolidated comparison of textual refusal, refusal without image generation, explanation correctness, and their joint success rate across the evaluated methods?

R3: We agree that the textual rule is weaker than the formal definition, and we will report all four measures in the revision. We adopted the textual rule because most UMMs always generate an image after their reasoning, even when the reasoning refuses (Sec. 4.1, Appx. B.2). The tables below cover the 1,050 infeasible DoD requests. Textual refusal is the D in the paper. Refusal without image additionally requires that no image is generated, as in Eq. (4). Correct explanation means the stated reason matches the verified cause. Joint success requires both refusal without image and a correct explanation.

Without a reminder:

MethodTextual refusalRefusal without imageCorrect explanationJoint success
BAGEL0.0%0.0%0.0%0.0%
Uni-CoT0.0%0.0%0.0%0.0%
BAGEL-Canvas0.0%0.0%0.0%0.0%
ThinkMorph1.7%0.1%1.5%0.0%
UniREdit-BAGEL0.4%0.0%0.4%0.0%
UniReason0.1%0.0%0.1%0.0%
InternVL-U0.2%0.0%0.2%0.0%
SenseNova-U1.00.5%0.0%0.5%0.0%
VisTA-BAGEL93.0%93.0%91.8%91.8%

With the reminder:

MethodTextual refusalRefusal without imageCorrect explanationJoint success
BAGEL55.0%0.0%32.9%0.0%
Uni-CoT40.4%0.0%21.8%0.0%
BAGEL-Canvas16.0%0.0%1.2%0.0%
ThinkMorph57.5%0.4%45.4%0.3%
UniREdit-BAGEL5.2%0.0%4.9%0.0%
UniReason7.0%0.0%6.2%0.0%
InternVL-U28.1%0.0%15.1%0.0%
SenseNova-U1.016.9%0.0%13.6%0.0%
VisTA-BAGEL93.4%93.4%93.0%93.0%

VisTA-BAGEL never generates an image after refusing, so its scores are the same under the textual and strict rules, and almost all of its refusals give the correct reason. The 8 UMMs behave differently. The reminder makes them state infeasibility more often, but they then generate an image anyway, and many of these refusals are not grounded in the image: for BAGEL, only about 60% give the correct reason.

We hope these responses address your concerns. Thank you again for your constructive feedback, and we would be happy to answer any further questions.

The authors have largely addressed my concerns. I will maintain my score and lean toward acceptance.

Thank you for your response and for leaning toward acceptance.

Reviewer_V4JLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies visual abstention: recognizing that an image-editing request has no valid solution under its constraints and declining to generate an image. It introduces Draw-or-Decline (DoD), with 1,050 feasible-infeasible pairs across seven categories, and evaluates eight unified multimodal models and four decoupled pipelines. It also proposes VisTA, paired supervised training on feasible edits and explained refusals. VisTA-BAGEL achieves 74.3% editing success, 93.0% refusal success, and a 0.8% false refusal rate without a reminder. The evaluation includes prompting controls, a feasible-only training control, a reasoning ablation, and two held-out-category experiments. Baseline refusal comparisons use a lenient textual-refusal criterion to accommodate interfaces that always generate an image.

Critical strengths
  1. The paper identifies a useful evaluation problem beyond successful image editing: a system should also recognize requests that cannot be completed under the stated constraints. Evaluating feasible edits, infeasible requests, and false refusals together makes it difficult to appear successful merely by refusing everything. The paired benchmark, human verification of scene requests, and programmatic verification of maze feasibility provide a concrete basis for studying this behavior.

  2. The reported training results are promising. VisTA-BAGEL substantially improves refusal success over its initialization while retaining editing performance and a low false refusal rate. The feasible-only control and reasoning ablation help assess the contribution of refusal supervision, and the held-out-category experiments provide initial evidence of transfer. These results make the benchmark and training recipe useful despite the remaining evaluation questions.

Critical weaknesses
  1. The prompting comparison does not yet establish how much fine-tuning adds over explicitly eliciting feasibility judgments from the base model. The current reminder is appended to an editing request, while the supplied BAGEL inference system prompt still asks the model to plan and generate an image. The decoupled baselines are also framed around rewriting editing instructions. A dedicated feasibility-first baseline is needed to distinguish limitations of the elicitation protocol from limitations of the underlying model.

  2. The evaluation could better substantiate the distinction between recognizing that no solution exists and failing to find a solution. The maze setting uses a limited range of sizes and a particular generation procedure; false refusal on more difficult but certified-feasible problems is not directly characterized. In addition, although the benchmark is motivated by minimal pairs, separate aggregate editing and refusal rates do not show how often the model handles both members of the same pair correctly. These are addressable evaluation gaps rather than reasons to dismiss the reported gains.

Questions for authors
  1. Could you add a training-free feasibility-first baseline using the same UMM checkpoint? First explicitly ask whether any valid edit exists for the input image under all task constraints, requiring a FEASIBLE/INFEASIBLE decision with brief supporting evidence. If feasible, run the ordinary editor on the original image and instruction; otherwise, stop. The current reminder and prompt-rewriting pipelines do not directly test this setup. Please report editing success E, refusal success D, false refusal rate Df, and additional inference cost, and compare this baseline with VisTA.

  2. Can the model distinguish a genuinely infeasible request from a feasible request that is simply difficult to solve? Please evaluate harder, programmatically certified-feasible tasks, such as larger mazes or mazes with cycles and multiple valid paths, and report false refusal rates as difficulty increases. This would test whether abstention reflects infeasibility recognition rather than an inability to find a solution.

  3. Could you report the joint success rate over the benchmark's minimal pairs: the proportion for which the feasible member is edited correctly and the infeasible member is correctly refused? Please include a per-category breakdown and analyze cases where the small change between paired requests does not produce the appropriate change in behavior. This would directly assess the paired discrimination that motivates DoD.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer V4JL (Part 1/3)

Thank you for your valuable feedback and for finding our problem useful and our results promising. We address your concerns below.

W1: ... The current reminder is appended to an editing request, while the supplied BAGEL inference system prompt still asks the model to plan and generate an image. ... A dedicated feasibility-first baseline is needed to distinguish limitations of the elicitation protocol from limitations of the underlying model.

Q1: Could you add a training-free feasibility-first baseline using the same UMM checkpoint? ... Please report editing success E, refusal success D, false refusal rate Df, and additional inference cost, and compare this baseline with VisTA.

R1: Thank you for the suggestion. We implemented this baseline as described and ran it on all 8 UMMs. Given the input image and the request, the model first decides whether a valid edit exists and answers FEASIBLE or INFEASIBLE with brief evidence. Only if it answers FEASIBLE do we run the ordinary editor on the original image and instruction. Results on DoD:

ModelSettingEDDfExtra time / request
UniREdit-BAGELOrdinary editor68.4%0.4%0.0%—
UniREdit-BAGELFeasibility-first60.3%40.1%12.7%+8.3 s
BAGELOrdinary editor36.8%0.0%0.0%—
BAGELFeasibility-first29.4%61.0%19.6%+20.9 s
InternVL-UOrdinary editor32.0%0.2%0.0%—
InternVL-UFeasibility-first21.0%29.6%35.7%+1.6 s
SenseNova-U1.0Ordinary editor64.0%0.5%0.0%—
SenseNova-U1.0Feasibility-first58.9%73.3%9.7%+59.4 s
UniReasonOrdinary editor52.8%0.1%0.0%—
UniReasonFeasibility-first47.1%39.3%11.0%+8.2 s
Uni-CoTOrdinary editor42.1%0.0%0.0%—
Uni-CoTFeasibility-first10.6%91.0%68.1%+50.7 s
BAGEL-CanvasOrdinary editor9.8%0.0%0.0%—
BAGEL-CanvasFeasibility-first8.1%35.2%19.6%+13.0 s
ThinkMorphOrdinary editor8.0%1.7%0.2%—
ThinkMorphFeasibility-first3.5%67.4%42.9%+8.7 s
VisTA-BAGELSingle pass74.3%93.0%0.8%0

The feasibility check raises refusal for every model, from at most 1.7% to between 29.6% and 91.0%. Explicit elicitation therefore recovers part of the base models' feasibility judgment, but this judgment is not reliable. Every model also refuses 9.7% to 68.1% of feasible requests, so editing success falls for all 8. The strongest training-free result, SenseNova-U1.0, is still worse than VisTA-BAGEL on all three measures (E 58.9% vs. 74.3%, D 73.3% vs. 93.0%, Df 9.7% vs. 0.8%), and the check adds 59.4 s per request, while VisTA-BAGEL decides in a single pass.

To test whether the editing system prompt holds the models back, we also added one sentence permitting refusal to the system prompt of the ordinary editor and kept everything else unchanged.

ModelSystem promptEDDf
UniREdit-BAGELOriginal68.4%0.4%0.0%
UniREdit-BAGELRefusal allowed68.2%1.1%0.0%
BAGELOriginal36.8%0.0%0.0%
BAGELRefusal allowed39.6%0.0%0.0%
InternVL-UOriginal32.0%0.2%0.0%
InternVL-URefusal allowed30.9%4.9%0.4%
SenseNova-U1.0Original64.0%0.5%0.0%
SenseNova-U1.0Refusal allowed61.3%0.6%0.0%
UniReasonOriginal52.8%0.1%0.0%
UniReasonRefusal allowed53.2%2.5%0.0%
Uni-CoTOriginal42.1%0.0%0.0%
Uni-CoTRefusal allowed43.8%0.2%0.0%
BAGEL-CanvasOriginal9.8%0.0%0.0%
BAGEL-CanvasRefusal allowed11.2%1.1%0.2%
ThinkMorphOriginal8.0%1.7%0.2%
ThinkMorphRefusal allowed7.8%2.5%1.9%

This barely changes behavior. Neither prompting approach makes the base models refuse the right requests reliably, so the gap to VisTA-BAGEL reflects the base models’ feasibility judgment.

AuthorsOpenReview ↗

Reply to Reviewer V4JL (Part 2/3)

W2: ... false refusal on more difficult but certified-feasible problems is not directly characterized.

Q2: Can the model distinguish a genuinely infeasible request from a feasible request that is simply difficult to solve? Please evaluate harder, programmatically certified-feasible tasks, such as larger mazes or mazes with cycles and multiple valid paths, and report false refusal rates as difficulty increases.

R2: Thank you for this insightful question. We understand the concern that a refusal might reflect a failure to find a path rather than the recognition that no path exists. To test harder feasible mazes, we built 400 additional maze pairs, all certified by BFS. They include mazes without cycles at sizes 13, 15 and 21, beyond the training sizes, and mazes with cycles at sizes 9, 11, 13, 15 and 21. In each pair, the infeasible maze differs from the feasible one in a single cell, which becomes a wall and cuts every path. The new mazes were evaluated with VisTA-BAGEL. As you suggested, the tables report the false refusal rate Df (%) on these certified-feasible mazes as difficulty increases.

Mazes without cycles (sizes 7 to 11 from DoD):

Setting7911131521
No reminder2.51.73.82.08.04.0
Reminder5.03.43.82.06.06.0

Mazes with cycles:

Setting911131521
No reminder2.00.00.04.02.0
Reminder2.00.00.04.02.0

Df shows no clear rise with difficulty: 2.7% at DoD sizes, 4.7% at larger unseen sizes, and 1.6% on mazes with cycles. Hard but solvable mazes are rarely refused, so refusals are not triggered by difficulty.

To check whether refusals come from recognizing infeasibility, we compared the reasoning of VisTA-BAGEL on the DoD mazes against the BFS certificate and judge whether the cited barrier is correct.

No reminderReminder
Infeasible mazes refused (D)144/150146/150
Refusals citing the correct barrier143/144144/146
Feasible mazes falsely refused (Df)4/1506/150

Almost every refusal names the wall that actually separates the start from the goal. In summary, harder feasible mazes do not lead to more false refusals, and refusals rest on the correct barrier, which suggests that they come from recognizing infeasibility rather than from failing to find a path.

AuthorsOpenReview ↗

Reply to Reviewer V4JL (Part 3/3)

W2: ... separate aggregate editing and refusal rates do not show how often the model handles both members of the same pair correctly.

Q3: Could you report the joint success rate over the benchmark's minimal pairs ... Please include a per-category breakdown and analyze cases where the small change between paired requests does not produce the appropriate change in behavior.

R3: Thank you for this suggestion. For each of the 1,050 DoD pairs, we count a joint success when the feasible request is edited correctly and the infeasible one is refused. Joint success (%) per category:

ModelSettingMazeMat.Med.Mot.PoseSpat.Temp.Overall
UniREdit-BAGELWith the reminder0.04.74.72.06.07.33.34.0
SenseNova-U1.0With the reminder4.024.03.32.710.717.38.710.1
UniReasonWith the reminder0.03.30.71.33.39.32.02.9
Uni-CoTWith the reminder2.712.75.38.022.05.38.09.1
BAGELWith the reminder0.028.714.710.026.711.312.014.8
InternVL-UWith the reminder0.72.72.06.014.04.74.75.0
BAGEL-CanvasWith the reminder0.70.00.00.00.70.70.00.3
ThinkMorphWith the reminder0.08.72.00.01.30.01.31.9
VisTA-BAGELWithout a reminder84.771.360.759.367.370.071.369.2
VisTA-BAGELWith the reminder86.072.060.760.770.770.073.370.5

Without a reminder, every other model stays at or below 2.0% in every category and 0.4% overall, so we omit them from the table. VisTA-BAGEL reaches at least 59.3% in every category in both settings, while the best other model reaches 28.7%.

Most failures come from editing quality rather than paired discrimination. Without a reminder, VisTA-BAGEL decides both requests correctly in 92.2% of pairs, and 23.0% fail only because the edit is not good enough. Only 82 pairs (7.8%) get a wrong decision: the model edits both requests in 74, refuses both in 8, and never swaps the two.

We analyzed the 82 pairs with a wrong decision. In the 74 pairs where both requests are edited, the infeasible responses are exactly the non-refusal cases of VisTA-BAGEL reported in Sec. 5.3, and they fall into the 3 failure groups defined in Sec. 4.3. In 57 pairs, the response falls into Premise Acceptance: the reasoning keeps the instruction as written and plans the edit as if the premise held. In 15, it falls into Premise Rewriting: the reasoning quietly turns the instruction into a feasible one without saying so. In the remaining 2, it falls into Acknowledge then Comply: the reasoning notices the problem but still plans an edit. Figs. 5 and 6 (Appx. B.5) show an example of each group. In the 8 pairs refused on both sides, the feasible request is falsely refused, and 4 of these are mazes where the model misreads the grid and concludes that no path exists.

We hope these responses address your concerns. Thank you again for your constructive feedback, and we would be happy to answer any further questions.

Reviewer_WaaTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates under what circumstances an image editing model should reject a user’s request rather than generate an image that does not meet the requirements. The authors propose the DoD benchmark dataset, which contains 1,050 pairs of feasible and infeasible requests across seven task categories. Additionally, the authors introduce VisTA, which trains the model to evaluate the feasibility of a request before performing the edit or rejecting the request. Experimental results show that VisTA-BAGEL achieves a rejection rate of 93.0% for infeasible requests and a false rejection rate of only 0.8% for feasible requests, while achieving an editing success rate of 74.3%.

Critical strengths
  1. The problem is clearly defined and has practical significance. The paper distinguishes between two scenarios: cases where the request itself has no valid solution, and cases where the model simply fails to complete the task. The paper demonstrates that strong image editing capabilities do not imply that a model can reliably determine when to reject a request.

  2. The experimental results fully support the paper’s core argument. The experiments compared eight models and included refusal reminders, a processing workflow that separates understanding from generation, and a control experiment trained exclusively on feasible requests. The results show that models can learn to reject infeasible requests through training while largely preserving their image editing capabilities.

  3. The validation process is rigorous. For the maze task, the paper programmatically verifies the feasibility of requests; for other requests in the benchmark, manual review is employed. Furthermore, the automated evaluation results show a high degree of consistency with human judgments.

Critical weaknesses
  1. Limited evidence for real-world use. The six everyday-scene categories use synthetic images, and training and evaluation follow similar data-construction procedures.
  2. The scoring rule is weaker than the stated definition. Visual abstention requires both an explicit refusal and no generated image, but baseline evaluations credit textual refusals even when an image follows.
Questions for authors
  1. Does the model maintain high refusal success and low false-refusal rates on real photographs and new requests written by humans?
  2. In held‑out category experiment, maze editing performance drops heavily without seeing maze training data, while refusal capability still transfers well. Could you elaborate why refusal generalizes better than editing capability?
  3. Would the VisTA training pipeline transfer to other UMM backbones besides BAGEL‑family models?
3 comments
AuthorsOpenReview ↗

Reply to Reviewer WaaT (Part 1/2)

Thank you for your valuable feedback. We appreciate that you highlight the motivation, idea and the clarity of our work. We provide detailed responses to address your specific concerns as outlined below.

W1: Limited evidence for real-world use. The six everyday-scene categories use synthetic images, and training and evaluation follow similar data-construction procedures.

Q1: Does the model maintain high refusal success and low false-refusal rates on real photographs and new requests written by humans?

R1: We evaluated on PhyEditBench [1], whose inputs are frames of real-world videos and whose feasible requests were written by its authors, independently of DoD. Of its 238 real-world instances, we use the 224 for which human annotators could write a valid infeasible counterpart on the same input image, giving 224 feasible-infeasible pairs.

ModelEDDf
BAGEL67.4%0.0%0.0%
UniREdit-BAGEL66.1%0.0%0.0%
UniReason68.8%0.4%0.0%
InternVL-U61.2%0.0%0.0%
VisTA-BAGEL64.3%90.2%4.5%

The refusal behavior learned on DoD carries over to real photographs and new requests: VisTA-BAGEL refuses 90.2% of infeasible requests, while the baselines almost never refuse.

[1] PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing

W2: The scoring rule is weaker than the stated definition. Visual abstention requires both an explicit refusal and no generated image, but baseline evaluations credit textual refusals even when an image follows.

R2: We agree that the textual rule is weaker than the formal definition, and we will report both rules in the revision. As detailed in Sec. 4.1 and Appx. B.2, we adopted the textual rule because most UMMs always generate an image after their reasoning, even when the reasoning refuses. Under the strict rule, these models score near zero regardless of their judgment, so the textual rule is needed to measure whether they recognize infeasibility at all. We applied the same rule to every model, including VisTA-BAGEL. The table reports both rules on DoD.

ModelTextual (no reminder)Strict (no reminder)Textual (reminder)Strict (reminder)
BAGEL0.0%0.0%55.0%0.0%
Uni-CoT0.0%0.0%40.4%0.0%
BAGEL-Canvas0.0%0.0%16.0%0.0%
ThinkMorph1.7%0.1%57.5%0.4%
UniREdit-BAGEL0.4%0.0%5.2%0.0%
UniReason0.1%0.0%7.0%0.0%
InternVL-U0.2%0.0%28.1%0.0%
SenseNova-U1.00.5%0.0%16.9%0.0%
VisTA-BAGEL93.0%93.0%93.4%93.4%

The textual rule does not inflate VisTA-BAGEL: it never generates an image after refusing, so its scores are identical under both rules. The reminder makes the 8 UMMs state infeasibility more often.

AuthorsOpenReview ↗

Reply to Reviewer WaaT (Part 2/2)

Q2: In held‑out category experiment, maze editing performance drops heavily without seeing maze training data, while refusal capability still transfers well. Could you elaborate why refusal generalizes better than editing capability?

R3: Thank you for this insightful question. We believe refusal and editing rely on different capabilities. Refusal is a decision made in the reasoning trace, where the model checks whether the premises of the instruction actually hold in the input image before deciding whether to proceed. Editing, on the other hand, requires category-specific execution, such as drawing a continuous path through a maze, which the scene categories in our training data do not teach. This is reflected in the held-out experiment (Appx. C.7). With maze held out of training, the model still refuses 94.0% of infeasible mazes, close to the 96.0% of VisTA-BAGEL, while its maze editing stays near the initialization (48.0% vs. 51.3% for UniREdit-BAGEL). The no-reasoning ablation (Appx. C.6) further supports this view: although that model is trained on mazes, it receives only generic refusal text in place of the reasoning and refuses just 41.3% of infeasible mazes, which suggests that refusal comes from the reasoning rather than from recognizing the task type.

Q3: Would the VisTA training pipeline transfer to other UMM backbones besides BAGEL‑family models?

R4: Yes. We applied the same VisTA recipe to two UMMs outside the BAGEL family, SenseNova-U1.0 and InternVL-U, keeping the training data, pair schedule and number of steps unchanged. Results on DoD without a reminder:

BackboneModelEDDf
BAGELUniREdit-BAGEL68.4%0.4%0.0%
BAGEL+ VisTA74.3%93.0%0.8%
SenseNova-U1.0Official64.0%0.5%0.0%
SenseNova-U1.0+ VisTA66.1%94.0%0.2%
InternVL-UOfficial32.0%0.2%0.0%
InternVL-U+ VisTA66.4%80.2%3.1%

On both backbones, VisTA raises refusal from near zero to a high level while keeping false refusals low and editing success no lower than before. The large editing gain on InternVL-U mostly comes from the feasible editing data in our training set. The recipe therefore does not depend on the BAGEL architecture.

We hope these responses address your concerns. Thank you again for your constructive feedback, and we would be happy to answer any further questions.

Through supplementary experiments and detailed explanations, the author has effectively addressed the concerns I raised. I will uphold my rate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.