acceptodds
Under review as a conference paper at ICLR 2027

What Happened vs. What Can Be Known: Ontic–Epistemic Video Revisit Reasoning

Abstract

Video-language models are increasingly capable of recognizing temporal state changes, yet they often conflate what happened in the world with what can be concluded from the visual evidence they actually observed. We study this distinction through ontic–epistemic reasoning across video revisits, where a model must compare entity states across temporally separated visits, recognize a physical transition, and respond appropriately when visit-specific evidence is incomplete. We construct structured Revisit Packs from paired observations of the same environment. Each pack contains controlled variants that remove or perturb visit-specific evidence while preserving the underlying physical case. We first extend Behavior Pack Optimization (BPO), originally designed around counterfactual views of a single video-question instance, to these paired revisits so that cross-visit transition behavior can be optimized jointly. Although BPO provides anchor-relative updates across the pack, it does not explicitly prioritize the intervention relations that remain unsatisfied. We therefore introduce Intervention-Structured Bottleneck Behavior Pack Optimization (BBPO), which uses detached pack-local relation-success scores to redistribute BPO advantage toward poorly satisfied revisit relations while preserving the mean optimization scale of the pack. Experiments show that BBPO raises mean all-view pack correctness from 63.6% to 66.3%, including improvements from 54.0% to 57.5% on Revisit Packs and from 68.4% to 70.7% on Visit Packs. Extensive results suggest that reliable reasoning across video revisits benefits not only from jointly optimizing evidence interventions, but also from allocating post-training credit according to the behavioral relation that most limits all-view revisit correctness.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Ontic, Epistemic, Video Revisit Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_qH5jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates ontic–epistemic reasoning across video revisits, distinguishing physical state transitions from conclusions justified by incomplete visual evidence. The authors construct structured VisitPacks and RevisitPacks containing evidence-complete, evidence-withheld, and nuisance-modified observations. Building on the existing Behavior Pack Optimization (BPO) framework, they propose Bottleneck Behavior Pack Optimization (BBPO), which reallocates anchor-relative advantages toward unsatisfied behavioral relations.

Critical strengths
  • The distinction between what physically happened and what can be inferred from available visual evidence is meaningful and offers an interesting perspective on persistent video reasoning. -The proposed VisitPacks and RevisitPacks provide a systematic way to examine transition recognition, evidence sufficiency, and invariance to irrelevant visual changes.
  • The paper compares BPO and BBPO using matched training settings and reports results across multiple seeds.
Critical weaknesses
  • The central optimization framework is directly inherited from Behavior Pack Optimization for Video MLLM Post-Training (NeurIPS 2026), including behavior packs, counterfactual interventions, behavioral contracts, and anchor-relative advantages. The proposed BBPO primarily introduces normalized failure-aware weighting of existing BPO advantages. This modification closely resembles established hard-example mining and difficulty-aware loss reweighting strategies. Although the ontic–epistemic revisit formulation is interesting, the paper does not sufficiently establish that its optimization mechanism constitutes a substantial methodological advance over BPO or standard reweighting approaches.
  • The evaluation focuses on seven predefined binary object-state properties and structured multiple-choice questions. This restricted setting may allow models to exploit recurring question templates, object-state priors, or intervention-specific cues instead of performing genuine ontic–epistemic reasoning.
  • The experiments use only 270 simulated scenes, with 216/27/27 scenes allocated to training, validation, and testing.
  • The experiments use only Qwen3.5-2B and Qwen3.5-4B, which belong to the same model family and are relatively small backbones.
  • The main comparisons focus on SFT and standard BPO, without evaluating whether the proposed gain can be reproduced using simpler failure-aware weighting or established post-training objectives.
  • The propositions primarily establish straightforward properties of normalized weighting, including mean preservation and greater weights for failed relations. They do not provide guarantees concerning optimization convergence, generalization, or improved ontic–epistemic reasoning.
Questions for authors
  • Given that BPO already introduces behavior packs, intervention-based rewards, and anchor-relative optimization, what is the fundamental algorithmic novelty of BBPO beyond failure-aware advantage reweighting?
  • Why are established post-training alternatives such as GRPO, difficulty-weighted policy optimization, and standard supervised objectives not included under matched training conditions?
  • Could the authors extend the evaluation to larger and architecturally different VLMs, such as 7B/8B or larger backbones, and independent real-world video datasets?
2 comments
AuthorsOpenReview ↗

Response 1 to Reviewer qH5j

Thank you for emphasizing the scientific distinction between physical state changes and what visual evidence can justify.

1. What is new beyond BPO and generic hard-example mining?

We explicitly build on BPO's single-video behavior packs, behavioral contracts, and anchor-relative advantage. We jointly introduce (1) an ontic–epistemic paired-revisit task that holds the world's physical transition fixed while varying the availability of premises across Visit 1 and Visit 2, (2) five-view Visit and eight-view Revisit Packs built from natural temporal withholding and nuisance controls, with a query-conditioned target-state representation for two visits (Appendix B), and (3) intervention-relation-defined, pack-local bottleneck weighting on top of adapted BPO. Unlike an ordinary answer-level difficulty indicator, BBPO defines its success signal using the anchor, the intervened answer, and the required cross-view relation.

2. Matched alternatives, more models

We introduce three additional comparisons. First, we implement difficulty-weighted BPO, which scales anchor-relative advantages according to individual answer correctness and normalizes the weights over the active global batch: A~iv=1+α(1−civ)1∣Bopen∣∑(j,u)∈Bopen[1+α(1−cju)]Aiv,α=1,\displaystyle \tilde A_{iv}=\frac{1+\alpha(1-c_{iv})}{\frac{1}{|\mathcal B_{\mathrm{open}}|}\sum_{(j,u)\in\mathcal B_{\mathrm{open}}}[1+\alpha(1-c_{ju})]}A_{iv},\quad \alpha=1, where civ∈{0,1}c_{iv}\in\{0,1\} is answer correctness and Bopen\mathcal B_{\mathrm{open}} contains all active, anchor-gated view-rollouts in the distributed global batch. This preserves mean weight one globally. Second, we use direct supervised post-training using answer-token cross-entropy (CE) on the ground-truth response for every intervention view. Third, we compare GRPO and BBPO with four rollouts per view, using matched training packs, initialization, and optimizer steps.

AlternativesOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
BPO63.6±1.154.0±4.368.4±0.889.7±2.882.2±0.885.1±3.586.8±2.291.4±2.5
Difficulty-weighted BPO63.2±0.950.6±1.669.5±0.887.4±4.382.2±0.883.9±2.985.6±4.588.8±0.7
Direct CE on all views65.9±0.557.5±1.670.1±0.887.4±3.381.6±0.883.9±2.287.9±2.893.1±1.4
BBPO66.3±0.557.5±3.370.7±1.489.7±2.882.2±0.885.6±1.688.5±0.891.4±1.9
More RolloutsOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
GRPO k=447.9±8.242.5±3.350.6±10.887.4±1.683.3±0.885.6±0.884.5±2.883.3±2.9
BBPO k=458.6±1.948.3±4.963.8±1.489.7±2.882.2±0.885.6±3.585.1±1.688.2±3.9

The additional baselines demonstrate that BBPO outperforms the tested answer-error-weighted BPO objective and achieves higher Overall AV accuracy than GRPO under the matched four-rollout setting. Direct CE supervision is also highly competitive. The principal difference lies in the optimization formulation: direct CE explicitly trains the model toward the ground-truth answer for each intervention view, whereas BBPO updates sampled responses using verifier-derived rewards and pack-local weighting. Nevertheless, BBPO provides a reward-based mechanism for prioritizing poorly satisfied behaviors without directly teacher-forcing the corresponding answer tokens, which could be feasible when correct intervention labels are unavailable but a verifier can score behavior.

Furthermore, we evaluate with larger models as below. Please see more details in Response 1 to Reviewer wi5Q (part 1) and Response 1 to Reviewer VBVy (part 1).

Larger ModelOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
Qwen3.8-27B (zero-shot)2.30.03.431.079.331.039.744.8
GPT-6.1 Terra (API, zero-shot)73.679.370.782.889.782.887.999.1
BPO, Qwen3.5-9B-v169.7±0.567.8±3.370.7±1.492.0±1.690.2±2.986.2±0.096.6±0.097.7±0.4
BBPO, Qwen3.5-9B-v172.4±0.971.3±1.673.0±2.292.0±1.690.8±0.888.5±1.696.0±0.897.4±0.0
BPO, Qwen3.5-9B-v269.7±0.566.7±5.971.3±3.390.8±1.687.4±1.685.6±0.894.8±2.496.8±1.5
BBPO, Qwen3.5-9B-v271.6±1.470.1±5.972.4±1.490.8±1.688.5±1.687.9±1.494.8±1.496.8±0.8
AuthorsOpenReview ↗

Response 2 to Reviewer qH5j

3. Seven binary states and shortcuts

Our method supports seven binary property types, not seven object identities. The M3R-5 experiments actually span four property types, 13 object categories, and 152 persistent scene-object instances. We agree this is bounded, structured multiple-choice reasoning, not arbitrary open-world state understanding. However, the proposed shortcut explanation is testable: all intervention views within a pack share exactly the same question and option order but require different correct answers. Under deterministic inference a text-only/template-only constant answer cannot complete such a pack.

To investigate whether the model can solve the task using question templates or answer priors alone, we introduce a blank-video control that preserves the question, answer-option order, and number of input frames while replacing all visual frames with zeros. Under this intervention, BBPO's Overall AV decreases from 66.3% to 0%, indicating that the evaluated model cannot satisfy the complete behavioral contracts using the retained nonvisual information alone. Together with the natural temporal-withholding ablation (Table 7), the experiment addresses two important shortcut mechanisms.

SettingsOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
Normal BBPO66.3±0.557.5±3.370.7±1.489.7±2.882.2±0.885.6±1.688.5±0.891.4±1.9
Masking cue (Table 7)41.4±2.512.6±3.355.7±3.571.3±1.682.2±0.875.9±2.471.3±3.355.7±5.9
Blank video0.0±0.00.0±0.00.0±0.00.0±0.03.4±2.40.0±0.00.0±0.050.0±0.0
Normal − blank+66.3 pp+57.5 pp+70.7 pp+89.7 pp+78.7 pp+85.6 pp+88.5 pp+41.4 pp

4. Scope of propositions, failure-density analysis

We agree that mean preservation and relative weighting do not prove convergence or generalization. Instead, these propositions characterize how BBPO redistributes optimization credit among intervention relations within a behavior pack. In particular, Proposition A.2 describes how the relative emphasis on failed relations depends on their prevalence within the pack, yielding a testable prediction about the relationship between failure density and bottleneck-aware optimization.

A relation fails unless the anchor answer, intervention answer, and required change/invariance relation are all correct. We report: (i) All exposures: mean number of failed non-anchor relations over every behavior pack encountered during training, including packs whose identity/anchor answer was wrong. (ii) Anchor-gated: the same mean, but only for training exposures where the anchor answer was correct and the BPO/BBPO policy loss was therefore active.

Online Training table summarizes failed non-anchor relations in online training exposures.

MethodAll exposures ↓Anchor-gated ↓
BPO, 2B1.596±0.0090.970±0.026
BBPO, 2B1.597±0.0390.945±0.059
BPO, 9B-v21.469±0.0871.153±0.085
BBPO, 9B-v21.447±0.0741.117±0.075

The failure-density analyses below provide additional insight into where BBPO improves Overall AV correctness on the 87 evaluation packs. We report: (i) Pack-seed pairs: one behavior pack evaluated by one trained seed. With 87 packs and 3 seeds, there are 87×3=26187\times3=261 pack-seed outcomes. The same physical pack appears once per seed. (ii) BPO AV: the percentage of those outcomes where BPO answers every view in the pack correctly. For a row with one or more BPO failed relations, this is necessarily 0%.

For the 2B backbone, BBPO repairs 12/42 one-failure outcomes and 3/53 outcomes with two or more failures; it loses 8 previously complete outcomes.

BPO failed relationsPack-seed pairsBPO AVBBPO AVAV deltaMean failed-relation reduction
0166100.0%95.2%-4.8 pp-0.102
1420.0%28.6%+28.6 pp+0.262
290.0%11.1%+11.1 pp+0.222
3+440.0%4.5%+4.5 pp+0.318

For the 9B backbone, BBPO repairs 4/36 one-failure outcomes and 2/43 outcomes with two or more failures; it loses 1 previously complete outcome. Again, results using 9B backbone are not tuned by the validation set. For the group containing three or more initial failures, BBPO exhibits an average reduction of 0.5 failed relations per pack-seed outcome.

BPO failed relationsPack-seed pairsBPO AVBBPO AVAV deltaMean failed-relation reduction (per pack)
0182100.0%99.5%-0.5 pp-0.005
1360.0%11.1%+11.1 pp-0.194
2130.0%7.7%+7.7 pp+0.231
3+300.0%3.3%+3.3 pp+0.500
Reviewer_VvbxOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce Revisit Packs as an extension of Behavior Pack Optimization’s (BPO) paired observations that emphasize counterfactual views. Revisit Packs remove or perturb visit-specific evidence of the same environment. The paper also introduces Intervention-Structured Bottleneck Behavior Pack Optimization (BBPO) where it differentiates from BPO by how it weights the training signal from each intervention within a pack.

Critical strengths
  • The paper extends BPO’s paired observations to be Revisit Packs where physical state transition, missing premise, irrelevant change are evaluated, while formulating the evaluation as ontic-epistemic video revisit reasoning.
  • The authors propose BBPO which retrains BPO’s objective while reweighting the training signals to emphasize poorly satisfied revisit relations.
Critical weaknesses
  • BBPO introduces limited methodological novelty over pre-existing BPO by reweighting of BPO's advantages.
  • BBPO shows limited improvement over BPO results in Table 1, with 2.7% increase in overall all-view accuracy. BBPO provides no improvement in Revisit Pack accuracy under the more challenging M3R-3 setting, despite Revisit Pack being one of the central contributions of the paper.
Questions for authors
  • If improvements are limited, would extending the training data to both real-world and simulated data and both indoor and outdoor scenes provide better results?
2 comments
AuthorsOpenReview ↗

Response Summary to Reviewer Vvbx

We appreciate the reviewer's recognition of two substantive strengths: the extension from single-video BPO to paired visits involving physical transitions and missing premises, and BBPO's explicit emphasis on failed revisit relations. However, we respectfully disagree with evaluating the paper's methodological contribution almost entirely as a small change to BPO's advantage weights. That characterization leaves out the new reasoning formulation, target-state architecture, and evidence-controlled revisit protocol—all necessary to make ontic and epistemic conclusions jointly testable, which are not present in the original single-video BPO framework. Finally, we would appreciate clarification regarding the soundness rating of 1. We would welcome identification of any specific technical or experimental validity concerns that warrant this rating, so that we can address them directly.

AuthorsOpenReview ↗

Response 1 to Reviewer Vvbx

1. Contribution clarification

Original BPO scores counterfactual perturbations of one video. Our setting takes two temporally separated visits to the same scene and asks two distinct, coupled questions: (i) what physical state transition occurred (ontic), and (ii) whether the evidence retained from each visit warrants that conclusion (epistemic). Holding the physical transition fixed while withholding one visit's relevant evidence can change the justified answer from a definite transition to insufficient evidence. This is not reducible to a single-video temporal perturbation or to generic hard-example mining. We operationalize the distinction in five-view Visit Packs and eight-view Revisit Packs, including irrelevant-change controls and natural frame withholding that avoids spatial-mask shortcuts. Crucially, Appendix B of the original submission already specified a Query-Conditioned Target-State Module: question-dependent object queries attend to each visit's visual evidence, produce visit-state representations and transition supervision, and inject a bounded residual into the visual-token stream. This is a distinct representation/architecture contribution, not BPO inherited unchanged. The newly added standard-Qwen+language-LoRA control reaches 2.3% Overall AV and 0% Revisit AV, versus 62.1%/66.7% Overall AV and 48.3%/55.2% Revisit AV (BPO/BBPO) for the full pipeline in the matched run. BBPO is then a complementary, deliberately parsimonious optimization extension for conjunctive pack correctness. Our framework is therefore a broader methodological contribution than the weighting rule alone.

2. Experiment improvement

On the primary M3R-5 study, SFT-B is 14.2% Overall AV; adapting BPO to the paired-visit architecture and behavioral contract reaches 63.6%, and BBPO further reaches 66.3% (+2.7 pp). These are different comparisons and answer different scientific questions. Importantly, all-view accuracy is a conjunctive measure: a pack counts as correct only if every required observation/withholding relation is handled appropriately; +2.7 pp is not simply a 2.7% improvement in isolated multiple-choice accuracy. The objective also benefits Visit Packs, which are an equal part of our ontic–epistemic evaluation.

Importantly, the larger-backbone experiments provide complementary evidence under M3R-5, where BBPO improves Revisit AV 67.8% →\rightarrow 71.3% with Qwen3.5-9B. It demonstrates that improvements in Revisit-Pack correctness are attainable with the proposed optimization under a different backbone-capacity setting. Please see more details in Response 1 to Reviewer wi5Q (part 1) and Response 1 to Reviewer VBVy (part 1).

We acknowledge that BPO and BBPO have both 47.62% Revisit AV accuracy under M3R-3. Nevertheless, BBPO improves Overall AV from 54.92% to 56.83% and Visit AV from 58.57% to 61.43%, with smaller variance on Revisit AV.

M3R-3Overall AVRevisit AVVisit AV
BPO54.9±3.847.6±7.158.6±3.5
BBPO56.8±2.047.6±3.661.4±1.2

3. Additional experiments.

We added Habitat/HSSD, a distinct indoor simulation stack with different rendering, assets, and scene organization. Crucially, the complete 148-pack HSSD evaluation (37 scenes, 20 evaluated object categories) is an external test set for the present comparison: the post-training hyperparameters were transferred from M3R without selection on HSSD's nominal validation portion. We train the two methods on the same 420 HSSD training packs and compare their results over three runs. Please see more details in Response 1 to Reviewer wi5Q (part 1) and Response 1 to Reviewer VBVy (part 1).

BBPO achieves improvements in missing-evidence handling (+3.2 pp), withholding accuracy (approximately +2.5 pp), Visit-1 withholding (+5.5 pp), and nuisance invariance (+1.4 pp). The HSSD experiments provide an independent test of the framework under a different indoor simulation environment. Both methods are trained on the same 420 HSSD training packs, while their hyperparameters are transferred from M3R without selection on HSSD's evaluation partition.

Our scientific target is indoor revisits, directly motivated by embodied and long-horizon scene-memory settings; outdoor scenes are not an unaddressed prerequisite for showing the problem exists. Real-world M3-Verse-Real provides only original-answer labels, not certified two-visit state/evidence interventions, so identity-only real-video QA cannot test the paper's core ontic–epistemic contracts. We will state that clearly in the revision.

Reviewer_VBVyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Using paired before/after visits from the simulated part of M3-Verse, the paper builds Visit Packs (5 views) and Revisit Packs (8 views) that withhold target evidence by dropping frames rather than masking pixels. The model must detect state transitions, abstain when a visit's evidence is missing, and stay stable under nuisance changes. It adapts Behavior Pack Optimization (BPO) to these packs and proposes BBPO, which reweights each view's advantage toward failed relations via a softmax mixed with a uniform term. On a Qwen3.5-2B model with an added target-state module, all-view accuracy rises from 63.6% to 66.3%.

Critical strengths
  • The question is well posed: separate what changed in the world from what the evidence supports.
  • Replacing spatial masks with natural temporal withholding is a useful finding (all-view 41.4 → 66.3) and removes an obvious shortcut.
  • BBPO is simple and mean-preserving. Results use three seeds, and the limitations are stated honestly.
Critical weaknesses
  • The evaluation is far too small. Validation and test are merged into 87 packs (54 + 33). Hyperparameters, including λ and the reference type, are chosen on the validation half of that same set. The main gain of +2.7 pp is about two packs, and Revisit-AV std is 3–4 pp, so the improvement is not statistically convincing.
  • The numbers do not line up. λ = 0 should reduce to BPO, but Table 6 gives 64.37 while Table 1 gives 63.6 for adaptive BPO. λ = 0, 0.25 and 0.5 give identical results (64.37), suggesting very coarse resolution. The main model uses λ = 0.7, but the 4B and M3R-3 runs use 0.35, without explanation.
  • The method contribution is thin. With one rollout per view, the importance ratio fixed to 1 and clipping inactive, BBPO is REINFORCE with a hard-example weight on the advantage. The propositions are short algebra. The work also depends heavily on BPO, which is itself unpublished.
  • Scope is narrow: one simulator, seven binary states, multiple-choice answers, 2B/4B models. There is no comparison with other abstention or consistency training methods, or with off-the-shelf MLLMs, and the custom target-state module confounds the comparison.
Questions for authors
  1. Can you report results on the test split alone, with hyperparameters chosen only on validation, and give confidence intervals?
  2. Why does λ = 0 not match BPO?
  3. How does BBPO compare with simply using more rollouts per view or with group-relative advantages?
2 comments
AuthorsOpenReview ↗

Response 1 to Reviewer VBVy

1. Held-out test with confidence intervals, additional simulator and backbone, stronger backbone (9B)

We report paired 95% confidence intervals (CIs were constructed using paired bootstrap) for held-out M3R-5 test set as below. BBPO improves Overall AV by 2.0 pp and Visit AV by approximately 3.0 pp, while Revisit AV remains unchanged. However, the confidence intervals include zero, indicating that the limited held-out sample does not establish a statistically reliable improvement. To provide additional evidence beyond this small test set, we evaluate both methods on an independently new simulator Habitat/HSSD using hyperparameters transferred from M3R without tuning on HSSD's validation data.

M3R-5 test-setBPO (95% CI)BBPO (95% CI)Paired delta (95% CI)
Overall AV52.5% [34.3, 71.7]54.5% [34.3, 73.7]+2.0 pp [-6.1, +11.1]
Revisit AV48.5% [21.2, 75.8]48.5% [21.2, 75.8]+0.0 pp [-15.2, +15.2]
Visit AV54.5% [31.8, 75.8]57.6% [34.8, 78.8]+3.0 pp [-7.6, +12.1]
HSSDBPO (95% CI)BBPO (95% CI)Paired delta (95% CI)
Overall AV40.8% [32.4, 48.6]40.5% [32.7, 48.0]-0.2 pp [-2.9, +2.5]
Revisit AV23.4% [14.9, 32.4]24.3% [16.2, 33.3]+0.9 pp [-3.2, +5.4]
Visit AV58.1% [47.3, 68.9]56.8% [46.4, 67.1]-1.4 pp [-5.9, +2.7]
Transition69.6% [61.9, 77.0]68.7% [61.0, 76.1]-0.9 pp [-4.5, +2.7]
Nuisance63.1% [54.5, 71.6]64.4% [55.4, 73.9]+1.4 pp [-2.7, +5.4]
Missing71.7% [61.0, 81.2]74.9% [62.8, 84.7]+3.2 pp [-0.7, +7.5]
Withholding accuracy75.9% [67.0, 83.6]78.3% [69.0, 86.0]+2.5 pp [-0.2, +5.6]
Visit-1 withholding68.0% [55.4, 79.3]73.5% [60.7, 84.7]+5.5 pp [+1.4, +10.0]

Furthermore, we evaluate Qwen3.5-9B to examine whether the observed improvements persist with increased backbone capacity. Since we lost the per-example prediction records required for paired confidence-interval calculation in the initial 9B (v1) experiment (please refer to Response 1 to Reviewer wi5Q for details). We therefore repeated the experiments with the same training configuration and saved the prediction records for statistical analysis; this rerun is denoted 9B-v2.

No 9B-specific hyperparameter selection was performed using the validation results. We report both the original 33-pack held-out test results and the val+test 87-pack evaluation for completeness. On the val+test evaluation, BBPO-v2 improves Overall AV by 1.9 pp and Revisit AV by 3.4 pp. On the 33-pack held-out test set, the corresponding improvements are 4.0 and 6.1 pp, respectively. Both experiments support the same positive direction, although the confidence intervals remain broad because of the limited sample size.

Val+testBPO, 9B-v2 (95% CI)BBPO, 9B-v2 (95% CI)Delta (95% CI)
Overall AV69.7% [60.5, 79.2]71.6% [62.8, 80.9]+1.9 pp [-1.1, +6.5]
Revisit AV66.7% [50.5, 84.0]70.1% [53.5, 87.2]+3.4 pp [+0.0, +11.1]
Visit AV71.3% [60.0, 82.1]72.4% [61.7, 82.7]+1.1 pp [-2.4, +5.6]
Test-onlyBPO, 9B-v2 (95% CI)BBPO, 9B-v2 (95% CI)Delta (95% CI)
Overall AV61.6% [45.5, 75.8]65.7% [51.5, 78.8]+4.0 pp [-4.0, +15.2]
Revisit AV54.5% [27.3, 81.8]60.6% [30.3, 87.9]+6.1 pp [+0.0, +24.2]
Visit AV65.2% [42.4, 84.8]68.2% [48.5, 86.4]+3.0 pp [-6.1, +13.6]

2. More rollouts

We implemented a four-rollout-per-view controlled comparison using the same 377 training packs, seed-matched SFT-B initialization, target-state architecture, adaptive reference, 96 optimizer steps and per-view rollout count. GRPO computes group-normalized advantages from four responses to each intervention view, whereas BBPO retains anchor-relative advantages and applies pack-local bottleneck weighting. Under this setting, BBPO achieves higher Overall AV accuracy than GRPO on both the val+test and held-out test-only evaluations. These results demonstrate an advantage of the BBPO training recipe over same-view GRPO at the matched rollout count.

val+test / test-onlyOverall AVRevisit AVVisit AV/Overall AVRevisit AVVisit AV
GRPO k=447.9±8.242.5±3.350.6±10.8/36.4±7.430.3±4.339.4±11.3
BBPO k=458.6±1.948.3±4.963.8±1.4/49.5±5.736.4±12.956.1±2.1
Delta+10.7 pp+5.7 pp+13.2 pp/+13.1 pp+6.1 pp+16.7 pp

Although we do not tune λ\lambda in this experiment, we still report validation+test and test-only results to demonstrate the consistency.

AuthorsOpenReview ↗

Response 2 to Reviewer VBVy

3. Clarification of λ=0\lambda=0 comparison

By construction, BBPO reduces analytically to the BPO objective at λ=0\lambda=0, since all bottleneck weighting coefficients become one. Table 6's 64.37% BBPO (λ=0\lambda=0) result and Table 1's 63.60% BPO result were obtained from independent stochastic training runs with different sampled responses, rather than a controlled same-rollout gradient comparison. Their 0.77 pp empirical difference is within the main-table BPO seed variation (SD 1.1 pp). Thus, the observed difference is consistent with stochastic variability.

We also clarify that λ=0.7\lambda=0.7 is used for the main 2B setting, while λ=0.35\lambda=0.35 is used for the reported 4B and M3R-3 settings. This parameter determines the strength of bottleneck emphasis and is selected empirically based on validation performance. Proposition A.2 characterizes how the resulting weighting depends on the distribution of failed relations, but does not prescribe a universally optimal λ\lambda.

4. What is novel beyond weighted REINFORCE

Thank your for the comments. The on-policy estimator is a detached, weighted policy-gradient update, not active-clipping PPO. This is not our principal novelty claim. There are three distinct contributions: (i) a paired-video ontic–epistemic task in which the same underlying physical transition is tested under different availability of visit-specific evidence; (ii) structured Visit/Revisit Packs, natural temporal withholding, and a query-conditioned target-state representation that operates across two visits (architecture specified in Appendix B of the submission); and (iii) the pack-local, relation-defined bottleneck reallocation that extends BPO's anchor-relative training to conjunctive revisit correctness. We explicitly credit the original single-video BPO framework [Kang et al., NeurIPS 2026]. The normalized weighting is mathematically simple; its scientific value is in the behavioral unit being weighted, its failure-density effect, and controlled experiment.

5. Off-the-shelf MLLM zero-shot comparison

We evaluate two off-the-shelf multimodal models in a zero-shot setting. Qwen3.8-27B is evaluated locally using its native multimodal interface, while GPT-6.1 Terra receives temporally ordered frames through its API. With the larger Qwen3.5-9B backbone, our fine-tuned BBPO model achieves 72.4% Overall AV, approaching the 73.6% achieved by GPT-6.1 Terra, despite using a substantially smaller backbone. These results demonstrate the competitiveness of the trained framework on the structured revisit benchmark.

Val+testOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
Qwen3.8-27B (zero-shot)2.30.03.431.079.331.039.744.8
GPT-6.1 Terra (API, zero-shot)73.679.370.782.889.782.887.999.1
BBPO, Qwen3.5-9B-v172.4±0.971.3±1.673.0±2.292.0±1.690.8±0.888.5±1.696.0±0.897.4±0.0
BBPO, Qwen3.5-9B-v271.6±1.470.1±5.972.4±1.490.8±1.688.5±1.687.9±1.494.8±1.496.8±0.8
Test-onlyOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
Qwen3.8-27B (zero-shot)0.00.00.036.477.340.945.540.9
GPT-6.1 Terra (API, zero-shot)72.772.772.781.895.581.886.497.7
BBPO, Qwen3.5-9B-v265.7±2.960.6±11.368.2±3.784.8±4.389.4±2.181.8±3.798.5±2.193.9±2.1
Reviewer_wi5QOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies “ontic-epistemic" reasoning in video-language models across separate visits. The major research question is whether a model tell the difference between what actually changed in a room (ontic) and what can be proven from incomplete clips (epistemic). To test this, this paper extends Behavior Pack Optimization (BPO) to paired video revisits. It builds 5-view Visit Packs and 8-view Revisit Packs. Instead of using artificial spatial masks, the paper drops frames naturally. A pack only counts as correct if every single view succeeds, so one failed relation ruins the whole score. To fix this bottleneck, the authors propose BBPO. It dynamically shifts weight toward failed relations inside each pack while keeping the average pack weight at 1. Experiments on Qwen3.5-2B and 4B over simulated M3M^3-Verse scenes show consistent gains. Overall all-view accuracy rises from 63.6% to 66.3%.

Critical strengths
  • The problem setup hits a real pain point. Video models often guess when visual evidence is missing. Separating physical state changes from epistemic justification is a great way to test visual memory.

  • Natural frame dropping works much better than spatial masking. Dropping target frames or picking target-free subclips avoids obvious pixel artifacts. Table 7 proves this clearly. Replacing spatial masks with temporal withholding boosts Revisit-Pack all-view accuracy by +44.9 percentage points.

  • BBPO is simple and mathematically clean. Conjunctive metrics are harsh. Uniform weighting wastes effort on relations the model already knows. BBPO reallocates credit using detached binary success scores (Propositions 3.1 & A.1), targeting the pack bottleneck without adding extra loss terms.

  • The ablations are well controlled. The authors compare frozen and adaptive references. They also track within-pack advantage variance (Fig. 3), scale the backbone from 2B to 4B, and test smaller target sizes (M3R-3).

Critical weaknesses
  • The evaluation set is very small, and val/test splits are merged. All data comes from 270 simulated scenes in M3M^3-Verse. More importantly, Appendix C states that 54 validation packs and 33 test packs are combined into 87 packs for final reporting. Yet the same section notes that hyperparameters like λ\lambda were tuned on the validation set. Reporting final metrics on a set that is 62% validation data raises overfitting concerns.

  • The standalone gain from BBPO is modest. Post-training as a whole gives a huge jump over SFT-B (14.2% to >62%). However, going from adaptive BPO to adaptive BBPO adds +2.7% overall. On Revisit AV (54.0±4.354.0 \pm 4.3 vs. 57.5±3.357.5 \pm 3.3), the standard deviations overlap across the three seeds.

  • The custom architecture module is never ablated. Appendix B adds a Query-Conditioned Target-State Module with spatial cross-attention and auxiliary heads. It uses a relatively high learning rate (5×10−55\times 10^{-5}), but we never see how vanilla Qwen3.5+LoRA performs without it.

Questions for authors
  • Can you report results on the 33 test packs alone? Since λ\lambda and checkpoints were tuned on the 54 validation packs, we need separate test numbers to see if the +2.7% gain truly generalizes.

  • How much does the Query-Conditioned Target-State Module matter? If you remove this custom module and run SFT-B + BBPO with standard LoRA on Qwen3.5, do the main trends hold?

  • Why keep the clipped PPO equations in Section 3? Since clipping is inactive in your on-policy setup, why not write the exact gradient estimator used in practice?

2 comments
AuthorsOpenReview ↗

Response 1 to Reviewer wi5Q

1. Test-only results

We seperate the test-only results as below. The 33-pack M3R-5 held-out test set shows a modest Overall AV improvement for BBPO (+2.0 pp), while Revisit AV remains unchanged (although smaller variance). Since all available paired videos of M3-Verse are already included in the experiments, we cannot enlarge the size of test set.

M3R-5 test-setOverall AVRevisit AVVisit AV
BPO52.5±3.848.5±8.654.5±3.7
BBPO54.5±2.548.5±4.357.6±2.1

To complement the limited M3R-5 held-out test set, we introduce an independent evaluation based on Habitat/HSSD. The generated HSSD dataset contains 142 scenes, with 30 object categories used in Visit/Revisit behavior packs and another 62 categories used only during SFT-A. Its evaluation partition contains 148 packs from 37 scenes covering 20 object categories, providing a larger behavioral evaluation set and broader object-category coverage than M3R-5. We train BPO and BBPO on the same HSSD training packs using λ=0.35\lambda=0.35, transferred from the M3R-3 setting without further hyperparameter selection on HSSD.

HSSDOverall AVRevisit AVVisit AVTransitionNuisanceMissingWithholding accuracyVisit-1 withholding
BPO40.8±1.823.4±2.358.1±3.369.6±0.663.1±1.771.7±5.975.9±4.668.0±5.5
BBPO40.5±1.524.3±2.256.8±3.368.7±1.664.4±3.574.9±7.078.3±5.073.5±5.7

The results suggest BBPO favors epistemic evidence-handling, whose gains are concentrated in missing-evidence and nuisance-invariance behavior, especially withholding. Both M3R-5 test-set and HSSD show a competitive performance between BPO and BBPO. These results suggest that the benefits of bottleneck weighting may be limited by the backbone capacity. Hence, we further evaluate a Qwen3.5-9B (v1) backbone. There is no validation set for 9B-specific hyperparameter tuning. BBPO improves Overall AV by 2.7 pp and Revisit AV by 3.4pp. For more detailed 9B-backbone and difference between v1 and v2 results, please refer to Response 1 to Reviewer VBVy.

MethodOverall AVRevisit AVVisit AV
BPO, Qwen3.5-9B-v169.7±0.567.8±3.370.7±1.4
BBPO, Qwen3.5-9B-v172.4±0.971.3±1.673.0±2.2
Delta+2.7 pp+3.4 pp+2.3 pp

2. Architecture ablation

Thank you for identifying this missing ablation. We added a strict standard-Qwen3.5+LoRA baseline and retrained the whole SFT-A → SFT-B → BPO/BBPO pipeline from scratch. This baseline removes the Query-Conditioned Target-State Module and other designed representation pathways, leaving ordinary language LoRA adaptation (LoRA-only). LoRA-only models shows identical results. The baseline only removing Query-Conditioned Target-State Module is also reported (TS-disabled).

SettingOverall AVRevisit AVVisit AVRevisit IDVisit IDTransitionNuisanceMissing
BPO (LoRA-only)2.30.03.40.069.00.00.040.5
BBPO (LoRA-only)2.30.03.40.069.00.00.040.5
BPO (TS-disabled)51.3±2.024.1±5.664.9±2.282.8±0.082.2±0.876.4±1.684.5±0.073.6±2.5
BBPO (TS-disabled)51.0±2.027.6±2.862.6±2.982.8±2.881.6±0.876.4±2.283.3±4.374.4±2.5
Delta-0.4 pp+3.4 pp-2.3 pp+0.0 pp-0.6 pp+0.0 pp-1.1 pp+0.9 pp

The ablation demonstrates that the proposed architecture is critical to revisit reasoning. Removing only the Target-State Module reduces Overall AV from 63.6% to 51.3% for BPO and from 66.3% to 51.0% for BBPO. Removing the complete representation design further reduces both methods to 2.3%. These results establish the importance of the target-state representation and its associated architectural pathways for effective revisit reasoning.

We will move the architecture detail from Appendix to the main paper with these results. Meanwhile, we will clarify the second contribution in Introduction to: We extend behavior-pack post-training from single-video counterfactual QA to paired video revisits with explicit architectural designs for formulating target-state representation that is essential to post-training.

The Target-State Module uses a higher learning rate (5×10⁻⁵) because it is newly initialized, whereas pretrained language LoRA and vision parameters use 3×10⁻⁶ and 1×10⁻⁶.

AuthorsOpenReview ↗

Response 2 to Reviewer wi5Q

3. Why keep the clipped PPO equations

Thank you for the constructive comment.

In our on-policy implementation the reference rollout log-probability is stop-gradient: ρi(θ)=exp⁡ ⁣[log⁡πθ(yi∣vi,q)−sg⁡(log⁡πθ(yi∣vi,q))].\rho_i(\theta)=\exp\!\left[\log\pi_\theta(y_i\mid v_i,q)-\operatorname{sg}\bigl(\log\pi_\theta(y_i\mid v_i,q)\bigr)\right]. Therefore ρi=1\rho_i=1 in the forward pass, clipping is inactive, but ∇θρi=∇θlog⁡πθ(yi∣vi,q)\nabla_\theta\rho_i=\nabla_\theta\log\pi_\theta(y_i\mid v_i,q). With detached advantages/weights, the effective policy-gradient estimator (up to the actual token normalization) is

∇θLpolicy=−1N∑i≠0sg⁡ ⁣[wi(Ri−R0)]∇θlog⁡πθ(yi∣vi,q),\nabla_\theta\mathcal L_{\rm policy}=-\frac{1}{N}\sum_{i\ne 0}\operatorname{sg}\!\left[w_i(R_i-R_0)\right]\nabla_\theta\log\pi_\theta(y_i\mid v_i,q),

plus the separately implemented reference regularizer. We will replace the clipped-surrogate presentation in the revision so that the paper states what is actually optimized.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.