acceptodds
Under review as a conference paper at ICLR 2027

Cognitive Liquid World Models: Continuous-Time Causal Verification over Decoupled Latent Manifolds

Abstract

Monolithic video world models compress multimodal observations into uniform representations updated over rigid discrete intervals (Δt=1.0). For embodied visual reasoning, this introduces four structural bottlenecks: categorical semantics dominate subtle spatial mechanics, rigid integer clocks miss irregular multi-scale transitions, static visual shortcuts bypass physical causal directionality, and competitive softmax normalization forces overconfident false positives under out-of-distribution inputs. We propose the Cognitive Liquid World Model (CLWM), reformulating physical world modeling as continuous-time causal hypothesis verification across a two-stage developmental firewall (an autonomous physics engine coupled to a frozen epistemic probe). CLWM resolves these limitations through four coordinated mechanisms: (1) asymmetric manifold bifurcation, decoupling an expressive semantic stream from an algebraically constricted geometry bottleneck stabilized by non-gradient continuous plastic codebooks; (2) top-down elastic ordinary differential equations (ODEs), where effective integration timescales warp dynamically across interaction boundaries to anchor persistent state transitions; (3) temporal group permutation optimization (TGPO), regularizing chronologically scrambled sequences to maximum Shannon entropy to enforce physical non-commutativity (A -> B = B -> A) and induce an emergent [0.40, 0.60] epistemic abstention attractor; and (4) an unconstrained continuous-time epistemic margin (CTEM) protocol that replaces closed-simplex probability competition with calibrated hypothesis verification. On Something-Something-v2, CLWM attains 0.9331 ROC-AUC (90.58% specificity) across all 174 classes and 0.9592 ROC-AUC on asymmetric causal transitions; targeted knock-outs collapse discriminability to chance baselines (0.4984, 0.5941, and 0.5546). Under a 50% frame drop (Δt=2.0), CLWM retains 99.35% of baseline discriminability without retraining, while prefix evaluations demonstrate sharp verification commitment at the physical contact boundary. Across 174 concurrent candidates, the unconstrained CTEM protocol yields a +0.2898 belief separation without competitive softmax normalization.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Causal Representation Learning, Continuous-Time Neural ODEs, Epistemic Calibration, Liquid Neural Networks, Spatio-Temporal Video Understanding, World Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zMaeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents CLWM, a cognitive liquid world model that reformulates physical world modeling from passive video as continuous-time causal hypothesis testing over decoupled latent representations. On top of frozen visual and text encoders, the authors train a lightweight video-text verification model combining a dual-stream liquid network, codebook memory, and query-conditioned verification. Stage one builds asymmetric semantic and motion streams from pretrained features and predicts the next feature state via self-supervision. Stage two uses LLM-generated questions, relevance-based modulation, and TGPO to produce three-way verification decisions. Evaluation is on Something-Something-v2.

Critical strengths

The paper tackles a reasonable weakness in video models—their reliance on static appearance at the expense of temporal dynamics. The proposed dual-stream design with an adjustable temporal update mechanism is a sensible direction with clear value.

By reusing frozen features from pretrained encoders, the method keeps downstream learning focused on dynamics and verification. The authors report that two-stage training runs on a single RTX 5090, which shows the setup is practically feasible.

Critical weaknesses

Feature bypass and implementation inconsistencies undermine the core method: Equation (5) queries the codebook directly with the original 768-dimensional features, letting information bypass the 32-dimensional hidden state—so the bottleneck alone can't justify the claim that appearance information is eliminated. Meanwhile, Algorithm 3 fixes the clock multiplier at γ = 4 and computes the visual trajectory only once per video, which directly conflicts with the query-dependent clock modulation described in the main text and Table E.1. This leaves the actual implementation of a central mechanism ambiguous.

No external baselines for evaluation: While the introduction draws conceptual comparisons to RL world models, generative models, V-JEPA, and standard liquid networks like LTC/CfC, the paper provides no quantitative results against any external baseline. Since CLWM depends on three frozen pretrained backbones, the lack of even simple baselines using the same features makes it impossible to tell whether performance gains come from the proposed liquid dynamics or from the upstream representations.

Questions for authors

Equation (5) concatenates the original 768-dimensional features with the hidden state. Why wouldn't this path retain appearance information? What if you queried the codebook using only the 32-dimensional hidden state?

Algorithm 3 fixes γ = 4 and computes the visual trajectory once per video, yet Table E.1 credits the gains to query-dependent clock modulation. Which version was actually used in these experiments?

Please add baselines—e.g., GRU or MLP models using the same frozen features—alongside comparisons to other relevant methods to shore up the empirical evidence.

In Stage 2, what information goes into the video descriptions fed to the text-only Qwen model for generating questions about visual properties like speed and contact boundaries, and which annotations supply that information? Please show concrete examples of videos, input descriptions, generated questions, and explain how you verify factual consistency with the video, answerability, and specificity to the individual clip.

Why do the same full model, evaluated on the same asymmetric test set at the same threshold, yield different results across Tables 2/E.2, 3, D.1, and E.3? The ROC-AUC values are 0.9592 / 0.9597 / 0.9784 / 0.9669, and TNR values are 87.00% / 86.71% / 96.30% / 91.62%.

1 comment

Thank you for your thorough technical audit. We resolved all five points directly in our revision:

Response to Issue 1 & Question 1

In Stage 1, Eq. (5) queries concatenated inputs and states solely to reconstruct full-rank 768d targets (ŝ(t+1), k̂(t+1)), sculpting ODE attractor basins. In Stage 2, we detached the CPC readout completely (0d raw bypass, Eq. 10; Figure 1; Table 2) so the verifier receives only the 128d ventral and 32d dorsal ODE trajectories hv(1:T) and hd(1:T). Removing raw 768d VideoMAE features purges static clutter, lifting canonical asymmetric performance from 0.9592 to 0.9724 ROC-AUC (90.78% Acc, 88.24% TPR, 93.27% TNR at θ = 0.60 vs. 88.81% / 86.54% / 91.08% for Eq. 5; 0.9888 pooled / 0.9946 macro ROC-AUC across the 20-trigger taxonomy in Table D.2, including 0.9627 on surface adhesion/attachment). Zeroing the 32d dorsal (0.5349 AUC) or ventral (0.5304 AUC) stream collapses discriminability to chance (Table 2), confirming zero 768d leakage.

Response to Issue 1 & Question 2

All reported benchmarks (Section 4.3 CTEM, Table E.1) used the fully conditioned dynamic loop, expanding each video across candidate queries ([174, T, D]) so each query modulates τ(eff) = τ · clamp(4.0 - 3.5 · R(t), 0.5, 4.0). Original Algorithm 3 inadvertently transcribed our fast-inference cache (fixed γ = 4.0); we updated Algorithm 3 (Appendix C.4) to formalize the batched dynamic pass and noted single-pass caching in the remark.

Response to Issue 2 & Question 3

Action-conditioned models (DreamerV3, IRIS) require motor actions absent in passive SSv2 video, while pixel generators (Sora, Genie) optimize RGB synthesis rather than causal verification (Table A.2). We trained five parameter-matched baselines (Bi-RNN, Bi-GRU, Bi-LSTM, Temporal MLP, Transformer) for 25 epochs (~98k steps) on identical 1280d frozen features, splits, and TGPO loss (Section 4.2, Table 3; Table E.4). Unified baselines reach 0.9620–0.9801 clean AUC, yet Transformer and Bi-GRU affirm 47.23% and 27.90% of scrambled clips, and all five retain 0.7280–0.8226 V-KO AUC via static texture leakage in 768d motion tokens—whereas CLWM achieves 90.78% clean accuracy, 0.11% scrambled TPR, and 0.5304 V-KO AUC.

Response to Question 4

Qwen2.5-7B-Instruct never infers kinematics from pixels; it reformulates human-annotated, object-grounded SSv2 captions (e.g., "Pushing [something] from left to right" + "ceramic mug" → "Pushing a ceramic mug from left to right") strictly as a syntactic formatter (Appendix D.2, Table D.1):

  • Ex. 1 ("Pushing a ceramic mug from left to right across a wooden desk"): Sem.: "What specific objects and surfaces are involved in the scene?" | Kin.: "Along what horizontal direction and contact plane does the displacement occur?" | Bin.: "Is the ceramic mug being pushed from left to right across the wooden desk?"
  • Ex. 2 ("Dropping a metal spoon into an empty glass"): Sem.: "What items are present in the interaction, and what is the state of the glass?" | Kin.: "Does the trajectory exhibit downward vertical acceleration into the container?" | Bin.: "Is the metal spoon being dropped into the empty glass?"

Triplets pass automated checks for JSON validity, verbatim retention of annotated SSv2 object nouns in semantic/binary queries (specificity and answerability), and non-degeneracy; negatives pair queries with time-reversed clips (Flip) or same-noun distractors (pushing vs. pulling the same mug).

Response to Question 5

We re-evaluated all tables on the Strict 32d ODE checkpoint and annotated jitter, threshold θ, and negative sampling in every caption:

Table (Orig./Rev.)JitterθOrig. (Eq. 5)Rev. (Strict 32d)Negative Sampling
Table 2 / E.2[-1, 1]0.40/0.600.9592 / 87.00%0.9724 / 93.27%†Reversals + distractors
Table 3 → 4Fixed (0)0.400.9597 / 86.71%0.9716 / 90.35%Deterministic stride (T=16)
Table D.1 → D.2Fixed (0)0.400.9784 / 96.30%0.9888 / 96.28%Pure time reversals (Flip)
Table E.3 (Sweep)Fixed (0)0.600.9669 / 91.62%0.9669 / 91.62%Loss sensitivity grid

†Table 2 uses θ = 0.60 (91.08% vs. 93.27% TNR); Table E.2 uses θ = 0.40 (87.00% vs. 90.53% TNR, 0.9719 AUC on an independent jitter draw) and reports θ in {0.40, 0.50, 0.60}.

Reviewer_WosJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CLWM, which separates semantic and motion representations and uses continuous-time ODEs for physical reasoning. It also introduces temporal permutation training and a hypothesis verification module. The results on Something-Something-v2 show strong classification performance and robustness to frame dropping.

Critical strengths
  • Table 3 shows that dropping half the frames only reduces AUC from 0.9597 to 0.9535 without retraining. That's a useful result for irregular temporal sampling.

  • The stream ablations in Table 2 cause large performance drops, suggesting that the model uses information from both feature streams.

Critical weaknesses
  • Table 1 reports 0.7454 velocity alignment, but this is measured using differences in VideoMAE features, not actual object velocities. I don't think this is enough to claim the model has learned Newtonian motion.

  • The Table 2 ablations simply zero out entire feature streams. This changes the input distribution, so the performance collapse doesn't really show that the proposed architecture is better than a simpler single-stream model.

Questions for authors

In Table 3, what happens if you use the same 8 frames but keep Δt = 1 instead of 2? I'd like to see how much of the robustness actually comes from the continuous-time formulation.

1 comment

Thank you for your constructive feedback, your positive assessment of our irregular temporal sampling results, and for pointing out the exact controls needed to isolate our continuous-time formulation from unified single-stream models and subsampling artifacts. We have revised the manuscript to address all three points directly.

Response to Issue 1: Latent Feature Velocity vs. Physical Newtonian Motion

We agree with this distinction. Measuring ρ(velocity) = 0.7454 between first-order temporal differences of reconstructed and ground-truth VideoMAE embeddings (Δẑ(t) vs. Δk(t)) evaluates directional transition alignment in latent feature space rather than 3D physical object velocities. We revised the Abstract, Sections 1, 3.2, 4.1, and Appendix E.1 to state explicitly that ρ(velocity) = 0.7454 captures first-order directional momentum in latent feature space, confirming that the Liquid ODE tracks temporal transition gradients rather than interpolating static backgrounds. Adherence to physical arrow-of-time constraints (A → B vs. B → A) is evaluated separately across our 20-trigger Asymmetric Physical Taxonomy in Appendix D.5 (Table D.2), achieving 98.30% macro-averaged specificity (0.9946 macro ROC-AUC; 0.9888 pooled ROC-AUC) against time-reversed physical violations.

Response to Issue 2: Coupled Stream Architecture vs. Unified Single-Stream Models

That is a fair observation—Table 2 alone does not compare against a simpler unified model, and our two streams are not passively concatenated channels that can be interpreted like conventional feature dropping. In CLWM, the ventral (128d) and dorsal (32d) streams interact continuously in a closed loop: they exchange context at abrupt transitions via prediction-error cross-gating (Eq. 2) and jointly modulate ODE integration speed via cross-modal query relevance (Eqs. 8–9). Within this coupled loop, Ventral KO zeroes the appearance input (s_t → 0) to verify that static object identity does not leak into the 32d dorsal ODE (0.5304 AUC), while Dorsal KO ablates the post-ODE kinematic trajectory (h_{d,t} → 0) before Stage 2 fusion to show that the appearance stream alone cannot solve the task (0.5349 AUC). Crucially, rather than failing erratically under distribution shift, ablating the dorsal trajectory or scrambling time draws 99.68%–99.74% of prediction mass into the calibrated [0.40, 0.60] uncertainty zone (Appendix Table E.2), holding false affirmations at θ = 0.60 to 0.08%–0.11% TPR (99.96% TNR).

To test directly whether a simpler single-stream architecture suffices on intact inputs without distribution shift, we also trained five parameter-matched baselines (Bi-RNN, Bi-GRU, Bi-LSTM, Temporal MLP, and Transformer) for 25 epochs (~98k steps) on the concatenated 1280d features (512d ViT + 768d VideoMAE) under identical splits and TGPO regularization (Section 4.2, Table 3). Without two-stream decoupling, unified baselines either ignore frame order (27.90% false-positive TPR on scrambled clips for Bi-GRU; 47.23% TPR for Transformer) or retain 0.7280–0.8226 ROC-AUC when appearance is zeroed across all five models because static textures leak into their un-constricted 768d motion channels. CLWM achieves the highest clean accuracy (90.78%) while holding scrambled affirmations to 0.11% TPR and collapsing appearance-ablated performance to chance (0.5304 AUC).

Response to Question 1: Controlled Ablation (Same 8 Frames with Δt = 1 vs. Δt = 2)

We evaluated the exact same deterministically subsampled clips (zero random jitter) on our primary Strict 32d ODE model (N = 8,453, contradiction threshold θ = 0.40) without retraining, reported in Section 4.4 (Table 4):

Frame BudgetTemporal Step RegimeROC-AUCAcc (@0.40)TNR (Spec.)AUC Retention
T = 16 (Full Horizon)Base Continuous (Δt = 1.00)0.971691.26%90.35%100.0%
T = 8 (50% Drop)CLWM Analytical (Δt = 2.00)0.954288.34%90.17%98.20%
T = 8 (50% Drop)Rigid Integer Clock (Δt = 1.00)0.818664.41%97.11%84.25%
T = 4 (75% Drop)CLWM Analytical (Δt = 4.00)0.809670.40%85.07%83.32%
T = 4 (75% Drop)Rigid Integer Clock (Δt = 1.00)0.525250.60%99.76%54.06%

In the Zero-Order Hold update (Eq. 4), clamping Δt = 1.00 on the same 8 frames under-integrates incoming motion potentials by 2x, dropping accuracy by 23.93% (88.34% → 64.41%) and collapsing T = 4 to chance (50.60%). In Appendix E.9 (Table E.4), we also evaluate all five converged monolithic baselines on the exact same subsampled frames, showing that position-indexed discrete baselines degrade sharply at T = 8 (0.7432–0.9070 AUC, 57.91%–76.42% accuracy), whereas order-agnostic Bi-GRU and Transformer poolers remain flat (99.49%–99.71% retention) solely via static bag-of-frames shortcuts (0.8047–0.8226 V-KO AUC).

Reviewer_TiCbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Cognitive Liquid World Models (CLWM), a framework for video-based physical interaction understanding and causal verification. The method separates semantic and kinematic representations into two latent streams, uses continuous-time neural dynamics to model temporal evolution, and subsequently trains a language-conditioned verification module to evaluate whether observed video sequences support particular action hypotheses. The paper also introduces mechanisms for temporal-order sensitivity and uncertainty-aware verification. Experiments on Something-Something-v2 report promising results on action verification, temporal robustness, and several ablation settings.

Critical strengths

The paper explores an interesting direction: moving beyond static visual recognition toward understanding temporal transitions and verifying physical interactions. The idea of separating semantic information from motion-related representations, while explicitly accounting for temporal order and uncertainty, seems conceptually appealing. Although I am not an expert in this specific research area, I can appreciate the potential value of this approach, particularly for embodied reasoning.

The authors also appear to have invested considerable effort in the experimental evaluation. The paper includes multiple ablation studies, temporal robustness experiments, and analyses of model behavior under different conditions. These are useful attempts to understand the proposed system beyond its aggregate performance.

Critical weaknesses

My primary concern is the clarity and accessibility of the presentation. I found this paper exceptionally difficult to read and follow. Despite spending 4+ hours and going through the manuscript multiple times, I still struggled to develop a clear understanding of the overall modeling approach, the motivation behind individual components, and how they fit together. This made it difficult for me to assess the actual technical contributions.

The paper introduces a large number of specialized terms, architectural components, mathematical formulations, and auxiliary objectives in rapid succession, often without first establishing sufficient intuition. Terms such as "autonomous physics engine," "asymmetric manifold bifurcation," "developmental firewall," and "epistemic attractor" are difficult to interpret without substantial effort. In several places, I found myself trying to understand the terminology rather than the underlying method. More straightforward naming and clearer explanations of what each component actually computes would substantially improve the paper.

Relatedly, the presentation feels unnecessarily dense and mathematically overloaded. While mathematical rigor is important, many technical details appear before the reader has developed a basic understanding of the method. The resulting information density is overwhelming, and the extensive equations and terminology often distract from, rather than clarify, the main ideas. I would strongly encourage the authors to simplify the main narrative, explain the essential modeling choices through concrete examples, and move more implementation-specific derivations to the appendix.

I also found the experimental evidence somewhat difficult to interpret in relation to the paper's central claims. In particular, it is not entirely clear to me how much of the reported performance is attributable specifically to the proposed continuous-time dynamics, as opposed to more conventional temporal modeling mechanisms. The existing ablations are useful, but comparisons against simpler alternatives, such as standard recurrent networks or learned temporal gating, would help establish the necessity of the additional architectural complexity.

Overall, I am not necessarily questioning the potential value of the proposed ideas. Rather, I find that the current presentation makes it unusually difficult to identify, understand, and evaluate those ideas. This is a significant limitation for a research paper, particularly one introducing a relatively complex architecture.

Questions for authors
  1. Could the authors provide a simpler, end-to-end explanation of the proposed method? In particular, a concrete example showing what is predicted during Stage 1, how the learned representations are used in Stage 2, and what the final verification module actually outputs would be extremely helpful. I believe such an explanation should appear early in the main paper, before introducing the detailed mathematical formulation.

  2. Could the authors clarify the precise role and necessity of the continuous-time ODE component? From my reading, the underlying update resembles a form of adaptive temporal memory. What are the specific advantages over simpler recurrent or learned-gating alternatives, and are there experiments that isolate these advantages?

1 comment

Thank you for your careful reading and constructive feedback. We apologize for introducing specialized terminology and equations before establishing a concrete mental model, and restructured the paper around intuitive mechanics first.

Response to Issues 1–3: Terminology Pacing and Narrative Structure

We replaced opaque labels (e.g., "autonomous physics engine" → self-supervised dynamics encoder) and rewrote Sections 1 and 3 to lead with standard ML descriptions, keeping conceptual terms as secondary parenthetical anchors. We also added a Terminology Glossary (Appendix A.1, Table A.1) and consolidated all ODE, Lyapunov, and TGPO/deadzone proofs into Appendix B.

Response to Question 1: End-to-End Walkthrough on a Concrete Example

Figure 1 and a walkthrough at the start of Section 3 (before any equations) trace "a person pushing a ceramic mug from left to right across a wooden desk":

  • Stage 1 (Dynamics): Frozen backbones extract appearance s(t) in R^512 (ViT/DINOv2, mug/desk) and motion k(t) in R^768 (VideoMAE, horizontal sliding), projected to a 128d ventral stream and 32d dorsal bottleneck. Coupled ODEs integrate both streams—exchanging context via prediction-error gating at hand-to-mug contact—and extrapolate next-frame features (ŝ(t+1), k̂(t+1)) via 2048-entry prototype codebooks (CPC) without STE noise.
  • Stage 2 (Verification): Stage 1 recurrent weights and codebooks remain frozen (zero gradient) with the 768d CPC readout detached (0d raw bypass) to block static shortcuts. Given "Is the ceramic mug being pushed from left to right across the wooden desk?", query relevance accelerates ODE integration during sliding while dilating to damp ungrounded distractors ("pulling", "lifting"), fused via a causal GRU and MHA pooler.
  • Model Output: The verifier outputs calibrated belief p in [0, 1]: affirming the forward push (p = 0.88 > 0.60), rejecting time-reversed motion (p = 0.12 < 0.40), and abstaining inside [0.40, 0.60] on shuffled frames (p = 0.49). Across 174 slot-grounded CTEM candidates (Section 4.3), P(GT) = 0.7876 vs. P(Dist) = 0.4978 (+0.2898 margin).

Response to Issue 4 & Question 2: Role and Necessity of Continuous-Time ODEs

Under fixed unit spacing (Δt = 1.0), Eq. (4) algebraically resembles an adaptive gated cell (stated explicitly in Section 3.2). Parameterizing that update via continuous time constants τ(eff) = τ · γ(t) and elapsed time Δt yields four capabilities standard discrete gates lack (within a 4.34M-parameter core footprint; Table 3 & Table F.2):

  • Explicit physical time (Δt) & frame-drop elasticity (Section 4.4, Table 4 & Table E.4): Because Δt enters our closed-form exponential step (exp(-Δt / τ_eff)) analytically, CLWM adapts to irregular frame drops without retraining: at 50% frame loss (T = 8, θ = 0.40), Δt = 2.00 preserves 98.20% of baseline AUC (0.9542, 88.34% acc), outperforming a rigid integer clock (Δt = 1.00: 64.41% acc, −23.93%) and position-indexed discrete baselines (57.91%–76.42% acc). At 75% frame drop (T = 4), Δt = 4.00 restores +19.80% accuracy over a rigid clock (70.40% vs. 50.60%).
  • Emergent multi-scale dynamic time & peri-contact dilation (Fig. 2a & Fig. E.1): Without timescale supervision, appearance locks onto a fast memory (τ_v ≈ 4.18 frames) while motion spans an elastic spectrum (τ_d in [4, 80] frames; median 16.0). In event-locked telemetry (Fig. E.1a), dorsal memory dilates across contact (18.34 → 23.76 → 28.52 frames) as prediction error spikes (S_d = 7.73), anchoring irreversible transitions.
  • Query-modulated low-pass shock filtering (Table E.1, Eq. E.5 & Fig. E.2): τ(eff) sets the low-pass cutoff (ω_c = 1 / τ_eff) and shock gain (κ = 1 - exp(-Δt / τ_eff)). Grounded queries accelerate integration (γ_t → 0.5) to commit sharply at contact (+20.9% TPR surge at t = 8/16, Fig. 2b), whereas ungrounded queries dilate the clock (γ_t → 4.0), cutting shock gain by >70% (0.393 → 0.117) and dissipating impact spikes into the [0.40, 0.60] abstention zone (p = 0.4304 vs. false-positive latching at p = 0.9165 when clamped at γ_t = 1.0, which shrinks CTEM margin by 26.9%).
  • Provable contractive stability (Theorem 1, Appendix B.2): As an affine convex combination of bounded tanh potentials, Eq. (4) guarantees ||h(t)||_inf <= 1.0 for any Δt ≥ 0 and arbitrary horizons.

Comments on the paper

1 comment

The reviewers' constructive feedback directly helped us clarify the exposition, calibrate our physical claims, and strengthen our experimental controls. We have uploaded a revised manuscript in which all primary benchmarks are unified under our Strict 32d ODE configuration (0d raw feature bypass into Stage 2), addressing four core areas:

1. Intuitive Walkthrough, Terminology Mapping, and Appendix Reorganization (Reviewers 1 & 3)

  • Concern: Specialized dynamical-systems terms and dense equations appeared before establishing an intuitive mental model of how data flows through the two stages.
  • Revision: We placed Figure 1 and a plain-language, step-by-step walkthrough of a concrete physical interaction ("a person pushing a ceramic mug from left to right across a wooden desk") at the very start of Section 3 ahead of any equations. We also streamlined Sections 1 and 3 to lead with standard sequence-modeling descriptions, added a unified Terminology and Notation Legend in Appendix A.1 (Table A.1), and consolidated formal ODE, Lyapunov, and TGPO derivations into Appendix B.

2. Strict 32d Bottleneck (0d Raw Bypass), Algorithm 3 Alignment, and Query Provenance (Reviewer 3)

  • Concern: Querying the native-space codebook in Eq. (5) risked letting 768d features bypass the 32d motion bottleneck; Algorithm 3 showed a fixed clock multiplier rather than query-modulated integration; and Qwen query generation needed concrete examples and verification details.
  • Revision: We detached the native-space codebook readout from Stage 2 completely (Section 3.3, Eq. 10; Figure 1; Section 4.2, Table 2), feeding the verifier exclusively through the sculpted 32d ODE trajectory. Purging raw texture clutter improved canonical asymmetric discriminability from 0.9592 to 0.9724 ROC-AUC (Table 2) and 0.9888 pooled ROC-AUC across the 20-trigger physical taxonomy (Appendix Table D.2), while collapsing appearance-ablated performance to chance (0.5304 AUC). We also updated Algorithm 3 (Appendix C.4) to reflect the active batched query-modulation loop and documented Qwen2.5-7B-Instruct prompt provenance, concrete examples, and automated consistency checks in Appendix D.2 (Table D.1).

3. Feature-Controlled Monolithic Baselines and Continuous-Time (Δt) Isolation (Reviewers 1, 2 & 3)

  • Concern: Comparisons against simpler sequence architectures on the exact same frozen features were needed to isolate our bottleneck and continuous-time ODE from upstream representations, alongside testing subsampled frames with a fixed clock (Δt = 1.00) vs. adaptive Δt.
  • Revision: We trained five parameter-matched sequence baselines (Standard Bi-RNN, Bi-GRU, Bi-LSTM, Temporal MLP, and Transformer) for 25 epochs on the identical unified 1280d frozen features, splits, and TGPO regularization (Section 4.2, Table 3; Appendix E.10, Table E.4). Every unified baseline either affirms randomly scrambled videos (27.90%–47.23% false-positive TPR) or leaks static appearance when the appearance stream is zeroed (0.7280–0.8226 Ventral KO AUC), whereas CLWM achieves 90.78% clean accuracy while holding scrambled TPR to 0.11% and Ventral KO to chance (0.5304 AUC). On identical 50%-subsampled frames (T = 8), analytical stepping (Δt = 2.00) preserves 98.20% of baseline AUC (88.34% accuracy), outperforming a rigid integer clock (Δt = 1.00) by +23.93% accuracy (Section 4.4, Table 4).

4. Claim Calibration and Cross-Table Protocol Harmonization (Reviewers 2 & 3)

  • Concern: Velocity alignment measured in VideoMAE feature space (ρ = 0.7454) should not be equated with 3D Newtonian physics, and metric variations across tables needed explicit protocol annotations.
  • Revision: We reframed ρ(velocity) = 0.7454 throughout the manuscript as first-order directional momentum in latent feature space (Sections 1, 3.2, 4.1; Appendix E.1). We also re-evaluated all tables on the unified Strict 32d ODE checkpoint and explicitly annotated every caption with exact sample counts (N), frame jitter settings, negative sampling rules, and decision thresholds (θ = 0.60 vs. θ = 0.40, with side-by-side threshold sweeps in Appendix E.8, Table E.2).

Detailed point-by-point responses and full numerical tables are posted under each individual review. We appreciate the reviewers' and Area Chair's time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.