acceptodds
Withdrawn from ICLR 2027

CueInsert: Compiling Relational Intent into Verifiable Plans for Video Object Insertion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video object insertion is inherently a planning-and-generation problem: a system must infer where and how an object should move from high-level intent, and then faithfully realize that plan in the generated video. Existing methods largely focus on visual synthesis or require manually specified low-level controls, leaving intent-driven insertion planning underexplored. We introduce CueInsert, a training-free framework for intent-driven video object insertion via neuro-symbolic planning and plan-conditioned rendering. Given a background video, a reference object, and an insertion intent, CueInsert analyzes scene structure and anchor motion to generate candidate plans over object scale, placement, orientation, and trajectory. Explicit temporal relation predicates then diagnose whether the proposed motion satisfies the intended interaction, while an iterative repair mechanism refines violated trajectories under proposal-preservation and smoothness constraints. The refined candidates are globally reranked and converted into coarse motion and mask conditions for training-free rendering without additional model training or inversion. We further construct DAVIS-Ins, an intent-conditioned benchmark derived from semantically and viewpoint-compatible DAVIS videos for systematic evaluation of planning and rendering, and introduce Insertion Action Coverage (IAC), which measures the fraction of temporal transitions in which the inserted object is present and exhibits residual motion beyond global translation. Experiments demonstrate improved intent consistency, trajectory control, object fidelity, background preservation, and temporal quality over existing video insertion baselines.

Primary area:
generative models
Keywords:
Video object insertion, Neuro-symbolic planning, diffusion model, video editing

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_zxRcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes CueInsert, a training-free neuro-symbolic framework for intent-driven video object insertion that explicitly separates planning from rendering. Given a background video, a reference object, and a high-level relational intent (e.g., follow, approach, avoid, or parallel), the system generates candidate insertion plans over scale, placement, orientation, and trajectory with an MLLM, verifies and repairs them using symbolic spatiotemporal predicates, and renders the selected plan through coarse-composite and mask-conditioned diffusion sampling. The authors also introduce DAVIS-Ins, a curated benchmark containing 576 intent-conditioned insertion instances derived from nine DAVIS videos and 72 ordered background–object pairs, and Insertion Action Coverage (IAC), a residual-motion metric quantifying the temporal coverage of inserted-object actions beyond global translation. CueInsert achieves the highest average score and leads on several metrics among the evaluated baselines. The paper additionally evaluates pre-rendering relation satisfaction, reports a before/after symbolic-repair ablation, and presents a user study.

Critical strengths
  1. The roles of the MLLM (semantic proposals and scale-plausibility judgments) and symbolic components (verification and repair) are explicitly delineated.

  2. DAVIS-Ins and IAC provide a structured evaluation setting for intent-driven insertion and could benefit the community beyond this work.

  3. The framework supports different relational behaviors for the same object and background without additional model training or inversion, providing a practical interface for controllable video insertion.

Critical weaknesses
  1. Reliance on external components and insufficiently characterized failure propagation. The framework heavily relies on MLLM planning and judging, object detection/tracking, and monocular depth. While symbolic repair addresses violations in proposed trajectories, the main paper does not sufficiently characterize robustness when the underlying anchor observations, depth estimates, or semantic judgments are themselves incorrect. Errors in these components may propagate through both planning and rendering, and improved predicate satisfaction with respect to an erroneous anchor track would not necessarily imply correct interaction in the scene.

  2. Limited benchmark diversity. DAVIS-Ins contains 576 intent-conditioned insertion instances, but these are derived from only nine side-view DAVIS videos and 72 ordered background–object pairs (Sec. 3, pp. 6, lines 302–308). The number of instances therefore does not translate into comparable diversity of underlying scenes and viewpoints. Although the benchmark is carefully curated, the main paper provides limited evidence of generalization to broader scene geometries, substantial camera motion, changing viewpoints, and complex occlusions.

  3. Insufficiently established methodological novelty and incomplete controlled comparisons. Place-it-R1 [1] already describes MLLM scene and interaction reasoning, automatic insertion trajectories, and reasoning-guided diffusion for video object insertion, yet it is neither cited nor compared in the provided manuscript. VOID (ECCV 2026) [2] also uses VLM-based causal reasoning to identify interaction-affected regions and translate them into mask guidance for video diffusion. Although VOID addresses object removal rather than insertion, it further demonstrates that reasoning about object interactions to guide video editing is not new. Thus, neither MLLM planning before generation nor interaction-aware reasoning alone establishes a novel paradigm. CueInsert’s explicit relational verification and trajectory repair are potential distinctions, but their incremental contribution remains insufficiently demonstrated through controlled comparisons. I am therefore unconvinced that the paper establishes substantial methodological novelty beyond combining existing components, rather than merely concerned about missing citations.

  4. Limited sensitivity and efficiency analysis. The main paper does not report systematic sensitivity experiments for anchor-tracking noise, depth errors, or MLLM mispredictions. It also does not report runtime/cost profiling as candidate count, repair iterations, and MLLM calls vary. These analyses would help establish both the robustness and practical overhead of the proposed pipeline.

  5. Inconsistent aggregate planning statistics. Sec. 4.3 (pp. 8, lines 422–423) reports Feasible-1, Feasible-2, and average valid plans per case as 83.97%, 68.22%, and 2.38, respectively. However, the Overall row of Table 3 (pp. 9) reports 82.72%, 68.38%, and 2.45. The manuscript should reconcile these values and clarify the evaluation population and aggregation procedure. This inconsistency does not by itself establish that the experiments are invalid, but it makes the aggregate planning results difficult to interpret confidently.

  6. Unclear verification criteria. The title emphasizes “verifiable plans,” but the main paper does not clearly explain how a plan is actually judged to pass or fail verification. Sec. 2.2 introduces relation predicates based on relative positions and motion, but does not give the concrete checks and thresholds for each relation. For example, what exactly must a trajectory satisfy to count as “follow” or “approach”? Without these details, it is difficult to reproduce the verification results.

[1] Place-it-R1: Unlocking Environment-aware Reasoning Potential of MLLM for Video Object Insertion. arXiv:2603.06140. [2] VOID: Video Object and Interaction Deletion. ECCV 2026. arXiv:2604.02296.

Questions for authors

1.How robust is the pipeline to anchor-tracking noise, depth errors, and MLLM mispredictions? Can you provide controlled sensitivity experiments and planning runtime/cost measurements as candidate count, repair iterations, and MLLM calls vary? In particular, how are incorrect anchor observations distinguished from violations in an otherwise correctly observed candidate trajectory?

2.Can you provide additional evaluation under substantial camera motion, changing viewpoints, and complex occlusions? Table 3 evaluates relation satisfaction before rendering, while Table 4 reports subjective intent-consistency ratings, including 4.41/5 for CueInsert. Since IAC does not directly measure relational correctness, can you additionally report per-intent relation satisfaction on the final rendered videos, including how often predicate-certified plans remain relation-consistent after rendering?

3.How does CueInsert differ methodologically from Place-it-R1’s MLLM reasoning and automatic insertion planning? Could comparisons with MLLM-only and simple anchor-relative rule-based planners under the same renderer isolate the contribution of symbolic verification and repair? A direct Place-it-R1 comparison, if feasible, would also be informative, with differences in backbone and training requirements clearly documented.

4.Can you reconcile the aggregate statistics in Sec. 4.3 with Table 3? Which values are correct, and do the text and table use the same set of evaluated cases and the same aggregation procedure?

5.How exactly do you verify a plan? Please state the checks and thresholds used to decide whether it passes or fails each requested relation, and provide an example of a passing and a failing trajectory. If these details are in the supplement, please indicate where.

Reviewer_R4W9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposed CueInsert, a training-free framework for video object insertion with relational intention. By incorporating (1) intention-driven candidate planning and (2) neuro-symbolic feed back, the framework is able to analyze the background video and properly insert the reference object based on the given intention. The paper further construct a benchmark from DAVIS. Experiments show the proposed method achive good performance.

Critical strengths
  1. The paper is well-motivated.
  2. The framework is sound.
  3. Qualitative results is impressive.
Critical weaknesses
  1. Many details is missing, for example, the definition of the RpR_p in Eq. 4 is missing, and the soft violation Lrel\mathcal{L}_{\rm{rel}} in Line. 214 is not given. Such missing details make the paper hard to follow.
  2. Evaluation is insufficient: In table 2, the paper only include general video quality metrics. However, despite that "relational intention" is the main focus of this paper, there is no evaluation metric that measure if the generated video really follow the given intention. The paper provided user study results on "intent consistency" in Table 4. However, it does not include all baselines, and the details of the user study is missing.
Questions for authors

To my understanding, Section 2.1 and 2.2 is not part of Sec. 2 (Related Works). Are these sections placed under related work by mistake?

Reviewer_1hAzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces CueInsert, a training-free framework for intent-driven video object insertion. An MLLM proposes object placement, scale, orientation, and trajectories; anchor-relative temporal predicates diagnose relation violations; and a repair-and-reranking stage selects a plan for rendering. The selected plan is realized through coarse composites and mask-guided sampling using a UniVideo or HunyuanVideo backend. This work also introduces DAVIS-Ins and Insertion Action Coverage (IAC). However, the evaluation does not establish how reliably verified plans translate into correct rendered interactions, and the current comparisons do not sufficiently isolate the proposed planning contribution.

Critical strengths
  1. Translating high-level relational intent into executable insertion controls addresses a practical limitation of interfaces that require users to specify trajectories manually. Separating planning from rendering is a sensible design and makes intermediate decisions inspectable.

  2. Combining semantic proposals with explicit anchor-relative constraints is well motivated. Candidate repair, proposal-preservation constraints, and certification-aware selection provide a plausible way to improve imperfect MLLM proposals. This pipeline requires no additional training of the rendering model.

  3. This work compares against seven insertion or editing methods and reports leading CLIP-I, DINO-I, Subject Consistency, and IAC scores on DAVIS-Ins. Symbolic repair increases candidate-level RSR from 24.28% to 49.30%.

Critical weaknesses
  1. Verification is demonstrated on plans, with limited evidence of transfer to rendered interactions. RSR evaluates candidate trajectories using the predicates that also guide repair. It does not measure relation satisfaction after diffusion rendering, particularly after the planned region is released from reference anchoring. The method can also render an uncertified fallback when no certified admissible plan exists. The qualitative examples and user ratings are helpful, but the paper does not report the selected-plan certification rate, trajectory execution error, or final-video relation satisfaction. Consequently, the scope and practical value of the verification step remain unclear.

  2. IAC has important ambiguities and does not directly measure intent satisfaction.** Eqs. (11)–(12) subtract prescribed center displacement from observed optical flow. A rigid object translating differently from the plan can therefore have nonzero residual flow even without internal action. Rotation, tracking errors, or visual deformation may also contribute. Appendix C.2 illustrates differences from generic motion metrics, but does not establish robustness to these confounds or agreement with human action judgments. The threshold, object localization procedure, and source of prescribed trajectories for competing methods are insufficiently documented.

  3. The comparisons do not isolate the planning and rendering contributions. CueInsert combines MLLM proposals, scale feedback, symbolic repair, candidate selection, and adapted UniVideo/Time-to-Move sampling. The repair ablation measures pre-render predicate satisfaction, while the same-backbone rendering ablation is qualitative. Quantitative comparisons using the same renderer with simple anchor-relative trajectories and unrepaired MLLM proposals are needed to determine the added value of the proposed planner. Without such controls, the contribution beyond integrating existing planning and rendering ingredients is difficult to assess.

Questions for authors
  1. Could the authors provide the exact relation predicates, thresholds, temporal aggregation rules, relation losses, and repair operators? Please also specify the proposal MLLM and prompts, detector/tracker, depth estimator, and rendering start/release settings needed to reproduce the pipeline.

  2. What fraction of selected plans is certified, how often is the uncertified fallback used, and what fraction of final rendered videos satisfies the requested relation? Please quantify plan-to-render trajectory error and compare final interaction success before and after symbolic repair, ideally with an independent human assessment.

  3. How are object masks, presence decisions, the residual-motion threshold, and prescribed center trajectories obtained for IAC across all methods? Could controlled examples distinguish correct translation, incorrect translation, genuine articulated motion, and visual corruption, and test whether IAC agrees with human judgments?

  4. Could the authors provide quantitative comparisons under the same renderer and candidate budget using simple anchor-relative planning, MLLM proposals without repair, and the complete planner? Separating the effects of scale feedback, reranking, and sampling guidance would further clarify which components account for the gains.

Reviewer_LJV9OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents CueInsert, a framework for relation-intent-driven video object insertion. It uses a multimodal language model to propose insertion plans, checks and adjusts them through explicit relational predicates and trajectory repair, and renders the resulting plans using a video generation model. The paper also introduces the DAVIS-Ins benchmark and the Insertion Action Coverage (IAC) metric.

Critical strengths
  1. The task is practically relevant. Translating high-level relational intent into executable controls such as trajectory, scale, and orientation addresses an important challenge in video object insertion.

  2. The separation between planning and rendering is clear, and explicit relational checks provide a useful mechanism for diagnosing planning errors.

  3. The automatic evaluation and human study show promising results.

Critical weaknesses
  1. The necessity and incremental value of the proposed framework. The overall design is natural, and the system demonstrates useful results. However, the current experiments do not clearly establish what the framework contributes beyond the trajectory planning and feedback-based refinement capabilities of strong multimodal models. The framework is primarily validated using a single understanding model with the UniVideo/HunyuanVideo rendering backend, without controlled substitutions of different combos. It therefore remains unclear whether the gains address limitations of a particular model combination or persist across stronger models, for example strong LLM with strong agentic behaviour + different backends (e.g. more capable R2V models). This concern is not to require the model to outperform the strong alternative, but is to see more informative conclusions: when symbolic repair is useful, whether stronger models still benefit from it, and whether it offers more reliable relational control at a lower cost. Such findings would better establish the methodological contribution beyond the demonstrated system.

  2. The benefit of explicit symbolic repair over ordinary MLLM feedback-based refinement remains unclear. The current experiments do not sufficiently demonstrate the distinctive advantage of this mechanism over ordinary feedback-based refinement by a strong MLLM. A clean comparison is needed. Rather than emphasizing the neuro-symbolic terminology, the paper would be stronger if it established the specific benefits of explicit predicates and repair through such controlled comparisons.

  3. I recognize the value of IAC as a complementary measure of sustained object motion. However, it measures residual motion after subtracting the prescribed translation, rather than directly assessing whether relations such as follow, approach, or avoid are correctly realized. An object moving in the wrong direction could still receive a high IAC score.

Questions for authors
  1. Can you provide visualizations showing the full traj (the initial plan, detected constraint violations, to the repaired video, and the iterations) in the main text or appendix, including both successful and unsuccessful cases?

  2. With the same input information, renderer, and comparable inference budgets, how's symbolic repair compared with ordinary feedback-based refinement by a common/stronger MLLM?

  3. Can you evaluate representative combinations of understanding and generation models to establish how the framework’s benefits change with base-model capability and under which conditions it is most useful?

  4. Do improvements in internal predicate satisfaction translate into independently measured improvements in relational correctness in the final generated videos? Can you provide quantitative evidence of this connection?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.