acceptodds
Under review as a conference paper at ICLR 2027

ViCoB: Perceptual-to-Symbolic Binding with Selective Rebinding

Abstract

Multimodal editing systems must translate perceptual intent into exact symbolic references that downstream editors can execute. Even when a model identifies the intended object, it may emit an invalid or valid-but-wrong reference. Correcting such errors requires not only recovering a plausible alternative, but also deciding **whether the existing binding should be overwritten**. We introduce **ViCoB**, a selective-rebinding interface that separates recovery from overwrite authorization. ViCoB retrieves an operation-compatible alternative from executable source, but replaces the incumbent only when inference-visible evidence supports intervention. Under fixed retrieval, where all policies receive the same recovered alternative, Always-rerank repairs more errors than ViCoB (230 vs. 188) but causes substantially more regressions (85 vs. 13), yielding lower exact binding (63.8% vs. 66.7%). Across 1,750 cases and three frozen multimodal backbones, ViCoB improves exact binding from 49.1% to 64.4%. Downstream and transfer experiments further show that corrected bindings can improve executable editing and that selective rebinding extends across SVG and DOM. These results support treating overwrite authorization as a distinct, measurable decision between multimodal perception and symbolic execution.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Multimodal Editing, Visual Grounding, Executable Binding, Selective Prediction, Vision-Language Models, Structured Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LRgTOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ViCoB, a selective rebinding framework for multimodal editing that separates target recovery from overwrite authorization. By combining executable-source evidence with heuristic scoring and rule-based intervention, the method aims to improve binding accuracy while reducing unnecessary corrections. Experiments across multiple multimodal backbones and executable representations report improvements in binding accuracy.

Critical strengths
  1. The paper addresses a practical problem in multimodal editing by distinguishing target recovery from overwrite authorization.
  2. The experiments cover multiple multimodal backbones and executable representations, providing useful evidence of the method's applicability.
Critical weaknesses
  1. The methodological novelty of ViCoB appears rather limited, as the proposed framework essentially combines manually weighted candidate scoring with heuristic rules for deciding whether to overwrite an existing prediction. The distinction between target recovery and overwrite authorization is conceptually reasonable, but the paper does not convincingly demonstrate a substantial technical advance beyond existing reranking and selective prediction methods.
  2. I am concerned that the baseline comparisons are insufficient to establish the effectiveness of the proposed authorization mechanism. In particular, the learned authorization baselines operate at substantially lower intervention coverage than ViCoB, making their final accuracy less directly comparable. Without stronger coverage-matched comparisons, it remains unclear whether ViCoB's improvements stem from its proposed decision mechanism or simply from selecting a more favorable operating point.
  3. The ablation results raise questions about the necessity and justification of certain components in the proposed framework. For example, removing unary geometry produces exactly the same overall accuracy (64.4%) as the full ViCoB model, suggesting that this component contributes little to the reported performance. Such results weaken the justification for the proposed combination of evidence signals.
  4. The paper suffers from noticeable presentation and formatting issues that undermine its overall quality. In particular, experimental Tables 4 and 5 appear before the reference list has ended, resulting in an improperly organized manuscript. Furthermore, these tables receive insufficient discussion and analysis in the main text, making their placement particularly difficult to justify.
Questions for authors

Please refer to the weaknesses.

5 comments

Thank you for noting the value of separating recovery from authorization across backbones and representations, and for pinpointing four concrete concerns resolved below.

1. Technical Advance Beyond Reranking and Selective Prediction

Reranking replaces whenever an alternative scores higher (S(r)>S(i)S(r) > S(i)), ignoring regressions, while selective prediction thresholds single-prediction confidence p(r)p(r) to abstain. Overwrite authorization governs a counterfactual transition i→ri \to r between two executable commitments: equal confidence p(r)p(r) carries different risk depending on whether incumbent ii may already be correct, giving expected utility p(r)−λp(i)p(r) - \lambda p(i) (Eq. 11). Both our fixed rule and learned authorizer instantiate this interface.

To verify that gains stem from modeling incumbent correctness—not extra training or coverage—we hold one frozen posterior pθp_\theta, identical (i,r)(i,r) pairs, and operation budgets fixed across 7,274 Mind2Web actions, varying only the score:

SplitActions / overwritesConfidence pθ(r)p_\theta(r): repairs / regressionsPaired value pθ(r)−pθ(i)p_\theta(r) - p_\theta(i): repairs / regressions
Cross-Domain5,901 / 1,430483 / 101453 / 60
Cross-Website1,373 / 512134 / 43123 / 29

Conditioning on pθ(i)p_\theta(i) cuts Cross-Domain regressions from 101 to 60 (−0.69 pp, website-clustered 95% CI [−1.02, −0.39], with 30 fewer repairs; exact binding 22.00% vs. 22.18%, task CI [−0.22, 0.60] pp) and Cross-Website regressions from 43 to 29.

Manuscript: Section 3.4, Eq. (11); Section 4.7, Table 5.

2. Coverage-Matched Baseline Comparisons

We compare nine policies on 1,041 held-out SVG observations / 152 case groups (excluding six candidate-mismatched rows uniformly). Seven selective policies share alternatives and execute exactly 337 overwrites (32.37%) without test labels, with budget curves for score-based policies (Figure 3):

PolicyRepairsRegressionsExact binding
Uniform random, expectation151.957.659.29%
Runner-up margin2085864.65%
Validity + runner-up margin1812065.71%
Validity + incumbent margin1642064.07%
LR1791665.90%
ViCoB rule1821366.47%
LR + incumbent state194968.01%

Endpoints are Retain (50.24%) and Always-rerank (497 overwrites, 224/85, 63.59%). At K=337K = 337, ViCoB rule (66.47%) beats confidence/validity margins and base LR (65.90%), while adding incumbent type, presence, and challenge-state features to the same LR and split reaches 68.01% (194/9) (+2.11 pp, 95% CI [1.05, 3.35]).

The same ordering holds under dev calibration (matched dev coverage: 67.92% vs. 65.99% at 333/347 overwrites; cap 31: 170/5 vs. 155/7, Table 17) and on Mind2Web (20.35% rule, 21.66% augmented LR, 22.18% learned vs. 19.61% Always-rerank, Table 5; cost utilities in Table 40).

Manuscript: Section 4.4, Table 2; Appendix C.3, Table 17, Figures 3–4; Appendix H.1, Table 40.

3. Geometry Ablation

We audited the historical preset and isolated both geometry terms across all 5,250 SVG observations:

ComparisonChanged bindingsNet correct
Historical geometry/retention preset vs. full rule5260
Remove unary geometry alone vs. replay center00
Remove endpoint-pair geometry alone vs. replay center32+5

The historical "no geometry" preset also altered retention, changing 526 bindings (157 unique wins per arm, 64.38%). In isolated replays (64.25% center), removing unary geometry (wgeom=0w_{\text{geom}} = 0) changes 0 bindings and pair geometry 32 (+5 net correct).

We relabeled the preset, added isolated controls in Table 4, drop any geometry necessity claim, and attribute symbolic gains to text (64.38% → 36.4%) and type (64.38% → 62.0%), keeping the original recipe frozen.

Manuscript: Section 4.6, Table 4; Appendix C.2, Table 16.

4. Table Placement and Discussion

\FloatBarrier places Downstream (Table 3) and Evidence (Table 4) beside their analyses on page 7, before REFERENCES (page 9), verified on 960 matched cases/editor:

EditorRawUnguardedGuardedGuarded repairs / regressions
Qwen2.5-VL-3B58.8%52.9%64.9%111 / 52
Qwen2.5-VL-7B79.6%84.8%84.8%65 / 15
Qwen3-VL-4B85.1%86.4%86.5%24 / 11

Section 4.5 explains why unguarded 3B drops (52.9%: fewer wrong bindings coexist with more schema-invalid patches, Table 31) and how the gold-free validity guard restores 64.9%; Section 4.6 discusses the symbolic ablations and geometry controls.

Manuscript: Sections 4.5–4.6; Tables 3–4 on page 7 (References start on page 9).

Thank you for the response. The Mind2Web evaluation and perturbation grid directly address both of my concerns—the authorization interface holds outside the ExBind ecosystem via the learned authorizer, and the SVG-specific qualification of the +15.3 headline is the right call. Rating raised to 3 (Weak Accept).

Thank you again for your detailed feedback. We have added controlled comparisons with fixed candidates and matched intervention coverage, conducted isolated geometry ablations, and revised the claims about geometry’s contribution. We have also corrected table placement and expanded the discussion in the main text.

We would appreciate your assessment of whether these revisions address your concerns, particularly whether the controlled comparisons better support the contribution of overwrite authorization. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Reviewer_8CzrOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the correction of perceptual-to-symbolic bindings in multimodal editing: a frozen VLM maps an instruction and a rendering to an executable reference (an SVG element ID or an ordered endpoint pair), which may be invalid or valid-but-wrong, and the downstream editor executes whatever reference it receives. The authors decompose correction into two decisions—target recovery (which operation-compatible candidate is best supported by evidence parsed from the executable source) and overwrite authorization (whether that candidate should replace the incumbent)—and argue the second is a counterfactual, asymmetric retain-or-replace decision that candidate ranking alone does not resolve. ViCoB is a deterministic, training-free instantiation: a fixed linear evidence score over label/text/geometry/VLM-preference/type/raw-prior features recovers a top alternative, and a three-state pre-decision classification of the incumbent (HARD-INVALID / CHALLENGED / SUPPORTED, defined by executable validity, a generic-identifier indicator, the incumbent type score, and the score margin) feeds a fixed correction set that decides replacement. The key experimental device is a case-disjoint fixed-retrieval replay in which all policies receive the same candidate inventory, scores, VLM preference, and retrieved alternative and differ only in the retain-or-replace action: Always-rerank repairs more errors than ViCoB (230 vs. 188) but regresses far more (85 vs. 13), yielding lower exact binding (63.8% vs. 66.7% on the held-out split). Across 1,750 cases and three frozen backbones (5,250 model–case observations), ViCoB improves exact binding from 49.1% to 64.4%, beating both a Qwen3-VL-8B executable-input verifier (58.9%, at an extra 0.73s per observation) and learned overwrite gates. Downstream experiments show improved bindings translate into better patches only under an execution-validity guard (the unguarded 3B editor drops below Raw), and transfer experiments cover GLM-4.6V, a source-disjoint SVGEditBench V2 conversion, an adapted Novel-SVG replay, DOM (74.6% to 99.2%), and an exploratory WebMMU real-web check. The evaluation is built on the ExBind setting and benchmark inventory (Wang et al., 2026).

Critical strengths

(I) A genuinely novel and clean problem formulation with an isolating experiment to match. Casting binding correction as the composition of recovery and overwrite authorization, where authorization is a counterfactual retain-or-replace decision under asymmetric repair–regression risk, is a real conceptual contribution that I have not seen articulated this way in the grounding, constrained-decoding, or selective-prediction literatures the paper positions against. Crucially, the claim is not left verbal: the fixed-retrieval replay isolates the authorization action while holding candidates, scores, VLM preference, and the retrieved alternative identical across policies, and the valid-but-wrong challenge then explains exactly where selectivity pays off (protecting incumbents of unknown correctness, not ranking). The demonstration that Always-rerank repairs more yet binds worse (230/85 vs. 188/13) is the kind of crisp, counterintuitive result that changes how one thinks about correction layers. This is consequential to my assessment because it is the paper's central claim, and it is both well-formulated and well-isolated. (II) The paper separates binding quality from downstream consumption (Stage-B with an execution-validity guard, and the honest finding that the unguarded 3B editor drops below Raw despite better bindings), uses the correct statistical unit throughout (case-clustered bootstrap; never treating the 5,250 model–case observations as independent), documents a configuration freeze ledger and post-freeze sensitivity replays rather than claiming tuning-free virtue without evidence, and includes an explicit claim–evidence status table that labels which conclusions are direct, supported, or hypothetical. The training-free rule beating an 8B verifier with the full executable contract (64.4% vs. 58.9% at zero extra inference cost) is a strong, well-controlled datapoint. This discipline is what makes the central claims credible despite the self-constructed evaluation ecosystem.

Critical weaknesses

The headline numbers are established almost entirely within a single inherited evaluation ecosystem. The +15.3-point primary result is measured on 1,750 cases of which 1,650 come from the legacy ExBind-Bench suites (EVG-Bench, EVG-Perturbed, Novel-SVG, SEB), where SEB is itself the ExBind-side conversion of SVGEditBench V2 into the EVG-compatible format; the ExBind setting, the candidate inventory format, and the evidence features ViCoB scores over (identifier regime, type, text, geometry) were all developed within this same ecosystem. The strongest external evidence in the paper is a 100-case source-disjoint conversion (+4.0 from 16.0%), a 95-case exploratory WebMMU root-mapping (+6.3, with the authors themselves declining to call it a portability claim), and the near-saturated DOM template result (which the authors correctly decline to generalize). Within the ecosystem the gains are largest precisely where evidence is sparsest and the gate's generic-ID/validity rules bite hardest (SEB suites, +20.0/+28.7/+20.6), which is consistent with the mechanism but also with an interface validated on the very distribution it was developed against. For a paper whose contribution is the interface and its evaluation protocol, the absence of any non-ExBind benchmark of comparable scale is the single fact that most limits how far I am willing to extrapolate the headline claim.

Questions for authors

(I) Of the 1,750 primary cases, 1,650 come from the legacy ExBind-Bench suites within which the interface was developed (Table 8), and the largest gains occur there (SEB +20.0/+28.7/+20.6). Can you evaluate on at least one non-ExBind executable-binding benchmark of comparable scale (e.g., an independent SVG- or DOM-editing corpus with gold element references), even if the gold format requires an adapter? If the gate's advantage shrinks substantially outside the ecosystem, the headline claim needs qualification. (II) The sensitivity evidence perturbs the raw prior, a joint symbolic scale (Table 14), the type weight (0.5×/1.5×), and four joint (τg,τt) settings. Please additionally perturb each of the remaining score weights (s_label, s_text, s_geom, s_vlm, s_raw) and each gate constant (τg, τt, τr) independently, and report the resulting repair/regression/exact-binding surface. This is the minimal evidence needed to know whether the reported operating point is robust or coincidental.

2 comments

Thank you for your thoughtful synthesis of our recovery–authorization formulation, fixed-retrieval isolation, and statistical protocol, and for requesting a comparable-scale external benchmark and individual parameter perturbations.

I. Comparable-scale Non-ExBind Benchmark (Mind2Web DOM)

To evaluate ViCoB outside the ExBind ecosystem at scale, we tested 7,274 actions (>4× our 1,750-case SVG suite) across two official Mind2Web splits using original human-annotated backend DOM targets. All policies share the released top-50 DOM inventory, top-ranked incumbent, and fixed recovered alternative, with the operation and preceding history supplied; candidate retrieval misses (40.1% Cross-Domain, 45.1% Cross-Website) remain errors in the denominator.

SplitActionsTasks / websitesRetainAlways-rerankViCoB ruleAugmented LRLearned authorization
Cross-Domain5,901912 / 5415.52%19.61%20.35%21.66%22.18%
Cross-Website1,373177 / 1012.24%16.53%17.04%18.14%19.08%

To isolate authorization from coverage, ViCoB rule, Augmented LR, and Learned authorization (pθ(r)−pθ(i)p_\theta(r) - p_\theta(i)) execute the exact same 1,430 / 512 overwrites, matched within CLICK, TYPE, and SELECT, whereas Always-rerank overwrites all 3,548 / 886 eligible incumbents (590/349 and 143/84 repairs/regressions). Fitting (6,240 actions) and calibration (1,522 actions) use public Train data with zero website overlap with either test split (10 unresolved Cross-Domain annotations are excluded by an annotation-only rule; prior evaluation subsets and the frozen checkpoint are disclosed in Appendix H).

Frozen rule vs. authorization interface. Consistent with your hypothesis, the frozen rule's gain over Always-rerank narrows to +0.75 / +0.51 pp (unrounded; 398/113 and 106/40 repairs/regressions, or +4.83 / +4.81 pp over Retain) because Mind2Web incumbents start at 12–16% accuracy (fewer correct bindings at risk) and use heterogeneous HTML attributes rather than ExBind's generic-ID convention.

However, instantiating the same overwrite-authorization interface with our learned paired-value authorizer reaches 22.18% / 19.08%, outperforming the frozen rule by +1.83 pp (task-clustered 95% CI [1.29, 2.37]), Augmented LR by +0.53 pp ([0.10, 0.96]), and Always-rerank by +2.57 / +2.55 pp while cutting Cross-Domain regressions from 349 (Always-rerank) and 101 (confidence pθ(r)p_\theta(r)) to 60 (453/60).

We qualify the manuscript accordingly: the +15.3 pp frozen-rule gain reflects structured SVG editing, while Mind2Web confirms that selective overwrite authorization generalizes at scale with representation-appropriate scoring.

Manuscript: Sections 4.1 and 4.7; Table 5; Appendix H (Tables 40–41).

II. Independent Perturbations of All Score Weights and Gate Constants

We replayed all 5,250 SVG model–case observations across 32 configurations, covering all 16 requested one-factor perturbations (0.5× / 1.5× around defaults; 0.10 / 0.25 for τg=0\tau_g=0), the 3×33 \times 3 (τg,τt)(\tau_g, \tau_t) grid, and zero-component controls.

Cells report setting · repairs/regressions · exact binding around the verified replay center (895/100 · 64.25%; historical emissions remain 902/100 · 64.38%, differing on 87 rows, and the original recipe stays frozen).

Parameter (frozen default)Lower perturbationHigher perturbation
Label weight wlabelw_{\text{label}} (0.50)0.25 · 887/99 · 64.11%0.75 · 895/97 · 64.30%
Text weight wtextw_{\text{text}} (1.00)0.50 · 855/100 · 63.49%1.50 · 919/100 · 64.70%
Unary geometry wgeomw_{\text{geom}} (0.75)0.375 · 895/100 · 64.25%1.125 · 895/100 · 64.25%
VLM preference wvlmw_{\text{vlm}} (1.00)0.50 · 835/80 · 63.49%1.50 · 873/100 · 63.83%
Incumbent prior wraww_{\text{raw}} (0.15)0.075 · 910/100 · 64.53%0.225 · 860/100 · 63.58%
Generic-ID margin τg\tau_g (0.00)0.10 · 850/100 · 63.39%0.25 · 834/96 · 63.16%
Min. incumbent type τt\tau_t (0.50)0.25 · 895/100 · 64.25%0.75 · 895/100 · 64.25%
Rerank margin τr\tau_r (0.25)0.125 · 895/100 · 64.25%0.375 · 895/100 · 64.25%

Across the 3×33 \times 3 (τg,τt)(\tau_g, \tau_t) grid, exact binding is 64.25%, 63.39%, and 63.16% at τg∈{0,0.10,0.25}\tau_g \in \{0, 0.10, 0.25\} for all three τt∈{0.25,0.50,0.75}\tau_t \in \{0.25, 0.50, 0.75\} (binary type scores separate cleanly around 0.50).

Across all 16 single-factor perturbations, exact binding stays within 63.16–64.70% (+14.1 to +15.6 pp over Raw 49.1%), showing a broad plateau rather than a narrow optimum.

Manuscript: Section 4.6; Appendix C.2, Table 16.

Thank you for reviewing our response and confirming that the Mind2Web evaluation and independent parameter perturbations address your concerns. Your suggestions helped us strengthen the evidence beyond ExBind and more clearly distinguish the scope of the SVG results from the broader applicability of the overwrite-authorization interface.

We appreciate your positive assessment of these revisions and would be happy to provide any further clarification.

Thank you again for your careful review and constructive suggestions.

Reviewer_DgMuOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ViCoB, a post-processing method for correcting object references produced by multimodal models in visual editing tasks. It ranks candidates using executable-source information and applies hand-designed rules to decide whether to replace the model’s original selection. Experiments examine binding accuracy, the tradeoff between repairs and regressions, downstream editing, and transfer across SVG and DOM settings.

Critical strengths
  1. The paper addresses a practical problem: an executable reference can be valid but still point to the wrong object.
  2. Holding candidate retrieval fixed while comparing replacement policies is a useful experimental design.
Critical weaknesses
  1. The novelty is not clear to me. The method mainly scores candidates with a weighted sum, then decides whether to replace the original answer based on ID validity, type compatibility, and score margins. It makes sense to account for corrections that might turn a right answer into a wrong one, but I am not sure what the method adds beyond existing confidence-based filtering and selective correction.

  2. The comparison with learned methods does not seem fair. The learned baselines choose thresholds that allow no regressions on the calibration set, while ViCoB has no equivalent restriction. In Table 23, once thresholds are chosen based on repair benefits and regression costs, logistic regression performs better than ViCoB under all five cost settings. This makes the advantage of the rule-based method less convincing.

  3. Simply letting the model choose from a candidate list already works better. In Table 16, this approach reaches 70.5% accuracy, compared with 64.4% for the original predictions followed by ViCoB. Adding ViCoB on top of candidate selection gives only about another 2.3 percentage points.

Questions for authors

see weakness above

4 comments

Thank you for recognizing the valid-but-wrong reference problem and fixed-retrieval design, and for pressing us to separate overwrite authorization from confidence filtering, matched learned operating points, and candidate selection.

1. Contribution Beyond Confidence Filtering

Confidence filtering and selective prediction threshold single-candidate confidence p(r)p(r) (or runner-up margin) to act or abstain. Overwrite authorization acts on an existing executable binding ii: equal confidence p(r)p(r) warrants replacement when ii is invalid or contradicted (p(i)≈0p(i)\approx 0), yet risks a regression when ii is plausible (p(i)>0p(i)>0). Under asymmetric repair–regression loss, expected overwrite utility for r≠ir \neq i is p(r)−λp(i)p(r) - \lambda p(i) (Eq. 11) rather than p(r)p(r).

We test this on 7,274 Mind2Web DOM actions, holding one frozen posterior pθp_\theta, identical (i,r)(i,r) pairs, and identical per-operation budgets fixed while changing only the decision score:

SplitActions / overwritesConfidence pθ(r)p_\theta(r): repairs / regressionsPaired value pθ(r)−pθ(i)p_\theta(r) - p_\theta(i): repairs / regressions
Cross-Domain5,901 / 1,430483 / 101453 / 60
Cross-Website1,373 / 512134 / 43123 / 29

Conditioning on pθ(i)p_\theta(i) eliminates 41 regressions (−40.6%) on Cross-Domain (−0.69 pp, website-clustered 95% CI [−1.02, −0.39]; exact binding 22.00% vs. 22.18%, task-clustered CI [−0.22, 0.60]) and 14 regressions (−32.6%) on Cross-Website. On SVG at equal coverage (K=337K = 337, Table 2), incumbent-state evidence similarly beats Runner-up margin (64.65%, 58 regressions) and base LR (65.90% → 68.01%, +2.11 pp, 95% CI [1.05, 3.35]).

Manuscript: Section 3.4, Eq. (11); Section 4.4, Table 2; Section 4.7, Table 5.

2. Fair Calibration and Matched Operating Points

We agree: restricting learned gates to zero calibration regressions was unmatched, and cost-calibrated LR beats the frozen rule at all five costs (Table 24 [orig. Table 23]). Our claim is not that a fixed rule beats calibrated learned models, but that incumbent-aware authorization improves both training-free and learned policies at matched operating points (keeping the original rule frozen).

Equal test coverage (1,041 held-out SVG observations / 152 case groups, K=337K = 337, no test labels):

PolicyRepairsRegressionsExact binding
Runner-up margin2085864.65%
Validity + runner-up margin1812065.71%
LR1791665.90%
ViCoB rule1821366.47%
LR + incumbent state194968.01%

Under the same split and solver, adding incumbent type, presence, and challenge-state features to LR adds +2.11 pp (95% CI [1.05, 3.35]), while ViCoB rule (66.47%) edges out base LR (65.90%). Six candidate-mismatched rows are excluded uniformly; emission auditing corrects projected 188 repairs to 182.

Development-only calibration: On 5,901 Cross-Domain actions, thresholds chosen on dev data to maximize R−λGR - \lambda G favor incumbent-aware learned authorization at all five costs (overwrites in Table 40):

Policyλ = 0.5λ = 1λ = 2λ = 5λ = 10
ViCoB rule341.5285172−167−732
LR358.5344318255146
Augmented LR399359321267179
Learned authorization447365359339334

On SVG (Table 17), matched dev coverage gives 65.99% (LR, K=347K=347) vs. 67.92% (LR + state, K=333K=333), and dev regression cap 31 gives 155/7 vs. 170/5 (283/287 overwrites), with thresholds frozen before testing.

Manuscript: Section 4.4; Appendices C.3, D.2, H.1; Tables 2, 17, 24, 40.

3. Incremental Value Over Candidate Selection

Free-form correction (49.1% → 64.4%, Table 6 [orig. Table 1]) and candidate prompting (70.51%, Table 1 [orig. Table 16]) serve two upstream regimes: unconstrained VLM outputs vs. prompting with our pre-extracted inventory. Crucially, they compose on the exact same 5,250 frozen candidate-only outputs:

PolicyExact bindingRepairsRegressionsExtra calls / case
Candidate-only70.51%000
+ ViCoB72.76%11800
+ Always-rerank76.74%377500

Valid-but-wrong group/endpoint errors persist after candidate selection: ViCoB adds 118 repairs with 0 regressions (+2.25 pp, 95% CI [1.25, 3.37], fixing 7.6% of remaining errors) at zero extra calls, whereas Always-rerank (76.74%) incurs 50 regressions. A 240-action DOM prompt control gives 6.25% for direct same-pair choice vs. 7.50% for KEEP/REPLACE, CI [−2.08, 4.74], claiming no prompt-only gain.

Manuscript: Section 4.2, Table 1; Appendix H.3.

Thank you again for your thoughtful feedback. We have added coverage-matched comparisons, development-set calibration results, and controlled experiments examining the role of incumbent-state information. We have also clarified ViCoB’s incremental value after candidate selection and its repair–regression tradeoff.

We would appreciate your assessment of whether these additions address your concerns, particularly regarding the contribution of overwrite authorization and its distinction from existing approaches. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Thank authors for the response and additional experiments. They address my concerns about the comparison with learned methods and the incremental value beyond candidate selection. However, the novelty relative to existing confidence-based filtering and selective correction remains unclear to me. I will therefore maintain my original score.

Thank you for confirming that the additional evidence addresses the comparison and candidate-selection concerns. We would like to clarify the intended contribution: separating recovery of an executable reference from the decision to overwrite an existing binding. Our fixed-recovery protocol tests this distinction by holding the posterior, candidate pairs, and overwrite budgets constant and measuring repairs and regressions. The fixed rule and standard expected-utility expression instantiate this interface; we view the formulation and its isolating evaluation as the contribution. If there is a specific prior formulation you consider closest, we would appreciate the reference to make that comparison more precise.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

The reviewers’ follow-ups acknowledge that the additional experiments resolve their empirical concerns, including comparison fairness, independent validation, and incremental value beyond candidate selection. We have also documented the data splits, calibration procedures, configurations, and evaluation artifacts for reproducibility. The remaining discussion centers on novelty. Our contribution is to model executable binding as an explicit intermediate decision layer between perception and action: it represents the incumbent reference and separates recovering an alternative from authorizing its replacement. Controlled comparisons isolate this decision from candidate quality and intervention coverage. A lightweight implementation yields 118 additional correct bindings after candidate selection, with no observed regressions or additional model calls, providing direct evidence of the value of explicitly modeling this layer. The decomposition also offers a basis for investigating intermediate credit assignment in long-horizon computer-use RL by distinguishing binding errors, overwrite consequences, and downstream execution failures. We hope this modeling contribution and its empirical validation can inform the final assessment. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.