- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ViCoB, a selective rebinding framework for multimodal editing that separates target recovery from overwrite authorization. By combining executable-source evidence with heuristic scoring and rule-based intervention, the method aims to improve binding accuracy while reducing unnecessary corrections. Experiments across multiple multimodal backbones and executable representations report improvements in binding accuracy.
Critical strengths
- The paper addresses a practical problem in multimodal editing by distinguishing target recovery from overwrite authorization.
- The experiments cover multiple multimodal backbones and executable representations, providing useful evidence of the method's applicability.
Critical weaknesses
- The methodological novelty of ViCoB appears rather limited, as the proposed framework essentially combines manually weighted candidate scoring with heuristic rules for deciding whether to overwrite an existing prediction. The distinction between target recovery and overwrite authorization is conceptually reasonable, but the paper does not convincingly demonstrate a substantial technical advance beyond existing reranking and selective prediction methods.
- I am concerned that the baseline comparisons are insufficient to establish the effectiveness of the proposed authorization mechanism. In particular, the learned authorization baselines operate at substantially lower intervention coverage than ViCoB, making their final accuracy less directly comparable. Without stronger coverage-matched comparisons, it remains unclear whether ViCoB's improvements stem from its proposed decision mechanism or simply from selecting a more favorable operating point.
- The ablation results raise questions about the necessity and justification of certain components in the proposed framework. For example, removing unary geometry produces exactly the same overall accuracy (64.4%) as the full ViCoB model, suggesting that this component contributes little to the reported performance. Such results weaken the justification for the proposed combination of evidence signals.
- The paper suffers from noticeable presentation and formatting issues that undermine its overall quality. In particular, experimental Tables 4 and 5 appear before the reference list has ended, resulting in an improperly organized manuscript. Furthermore, these tables receive insufficient discussion and analysis in the main text, making their placement particularly difficult to justify.
Questions for authors
Please refer to the weaknesses.
5 comments
Thank you for the response. The Mind2Web evaluation and perturbation grid directly address both of my concerns—the authorization interface holds outside the ExBind ecosystem via the learned authorizer, and the SVG-specific qualification of the +15.3 headline is the right call. Rating raised to 3 (Weak Accept).
Thank you again for your detailed feedback. We have added controlled comparisons with fixed candidates and matched intervention coverage, conducted isolated geometry ablations, and revised the claims about geometry’s contribution. We have also corrected table placement and expanded the discussion in the main text.
We would appreciate your assessment of whether these revisions address your concerns, particularly whether the controlled comparisons better support the contribution of overwrite authorization. We would be happy to clarify any remaining questions.
Thank you for your time and consideration.
Thank you for noting the value of separating recovery from authorization across backbones and representations, and for pinpointing four concrete concerns resolved below.
1. Technical Advance Beyond Reranking and Selective Prediction
Reranking replaces whenever an alternative scores higher (S(r)>S(i)), ignoring regressions, while selective prediction thresholds single-prediction confidence p(r) to abstain. Overwrite authorization governs a counterfactual transition i→r between two executable commitments: equal confidence p(r) carries different risk depending on whether incumbent i may already be correct, giving expected utility p(r)−λp(i) (Eq. 11). Both our fixed rule and learned authorizer instantiate this interface.
To verify that gains stem from modeling incumbent correctness—not extra training or coverage—we hold one frozen posterior pθ, identical (i,r) pairs, and operation budgets fixed across 7,274 Mind2Web actions, varying only the score:
Conditioning on pθ(i) cuts Cross-Domain regressions from 101 to 60 (−0.69 pp, website-clustered 95% CI [−1.02, −0.39], with 30 fewer repairs; exact binding 22.00% vs. 22.18%, task CI [−0.22, 0.60] pp) and Cross-Website regressions from 43 to 29.
Manuscript: Section 3.4, Eq. (11); Section 4.7, Table 5.
2. Coverage-Matched Baseline Comparisons
We compare nine policies on 1,041 held-out SVG observations / 152 case groups (excluding six candidate-mismatched rows uniformly). Seven selective policies share alternatives and execute exactly 337 overwrites (32.37%) without test labels, with budget curves for score-based policies (Figure 3):
Endpoints are Retain (50.24%) and Always-rerank (497 overwrites, 224/85, 63.59%). At K=337, ViCoB rule (66.47%) beats confidence/validity margins and base LR (65.90%), while adding incumbent type, presence, and challenge-state features to the same LR and split reaches 68.01% (194/9) (+2.11 pp, 95% CI [1.05, 3.35]).
The same ordering holds under dev calibration (matched dev coverage: 67.92% vs. 65.99% at 333/347 overwrites; cap 31: 170/5 vs. 155/7, Table 17) and on Mind2Web (20.35% rule, 21.66% augmented LR, 22.18% learned vs. 19.61% Always-rerank, Table 5; cost utilities in Table 40).
Manuscript: Section 4.4, Table 2; Appendix C.3, Table 17, Figures 3–4; Appendix H.1, Table 40.
3. Geometry Ablation
We audited the historical preset and isolated both geometry terms across all 5,250 SVG observations:
The historical "no geometry" preset also altered retention, changing 526 bindings (157 unique wins per arm, 64.38%). In isolated replays (64.25% center), removing unary geometry (wgeom=0) changes 0 bindings and pair geometry 32 (+5 net correct).
We relabeled the preset, added isolated controls in Table 4, drop any geometry necessity claim, and attribute symbolic gains to text (64.38% → 36.4%) and type (64.38% → 62.0%), keeping the original recipe frozen.
Manuscript: Section 4.6, Table 4; Appendix C.2, Table 16.
4. Table Placement and Discussion
\FloatBarrierplaces Downstream (Table 3) and Evidence (Table 4) beside their analyses on page 7, before REFERENCES (page 9), verified on 960 matched cases/editor:Section 4.5 explains why unguarded 3B drops (52.9%: fewer wrong bindings coexist with more schema-invalid patches, Table 31) and how the gold-free validity guard restores 64.9%; Section 4.6 discusses the symbolic ablations and geometry controls.
Manuscript: Sections 4.5–4.6; Tables 3–4 on page 7 (References start on page 9).