acceptodds
Under review as a conference paper at ICLR 2027

Probing Kinematic Chain Reasoning in Video Diffusion Transformers: A Study of 2D Gear System

Abstract

Video diffusion transformers (DiTs) are increasingly viewed as promising models for learning physical dynamics directly from video. However, off-the-shelf models struggle to generate physically plausible gear motions, even in simple scenarios involving only two gears. Simulating gear mechanisms is conceptually simple yet challenging, as the motion of a single gear strictly dictates the kinematics of the entire system. In particular, determining the rotation direction of each gear requires computing its rotational parity by traversing the underlying kinematic chain. To investigate whether video DiTs can learn such kinematic chain reasoning, we utilize 2D involute gear trains as a testbed to train and analyze video DiTs. Our analysis reveals that models can indeed learn kinematic chain reasoning, but they acquire two distinct types of reasoning mechanisms depending on the kinematic heights encountered during training: when trained on mechanisms with short kinematic heights, the model acquires a parallel BFS-like reasoning, using transformer layers as breadth-first search steps to incrementally determine parity across the kinematic tree. Conversely, when exposed to large kinematic heights during training, the model adopts a divide-and-conquer-like strategy—first resolving parity within local neighborhoods and subsequently merging them to achieve global consistency. Overall, this work demonstrates that video DiTs are capable of learning algorithmic reasoning over kinematic chains, while also uncovering their generalization limits and highlighting the critical role of training data complexities. Anonymous Project Page: https://Cog-DiT.github.io

Primary area:
generative models
Keywords:
Video diffusion transformers, Interpretability, Visual reasoning, Kinematic reasoning, Algorithmic reasoning, Gear mechanisms

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_rLFZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether video diffusion models can learn to reason about physical interactions of 2D gear systems. E.g., once a gear starts turning, the rotation direction of every other connected gears follow its chain of interactions. By fine-tuning Wan2.1 and probing its internal features, the authors discovered two main reasoning strategies: a breadth-first-search-like mechanism layer-by-layer for short chains, and a divide-and-conquer-like mechanism for longer chains where the model figures out local gear rotations first and then merges them for global consistency. Ultimately, the results show that while video models can learn structured physical reasoning, their ability to generalize is strongly tied to the complexity of their training data.

Critical strengths
  1. Very interesting topic. Instead of just evaluating the visual quality, the paper investigates capability to learn the structured physical reasoning itself, which gives a useful perspective on the capabilities and limitations of generative models.

  2. Well-scope formulation. Gear systems provide explicit yet complex physical constraints. The authors can systematically vary gear count, height, etc., making it easier to study under ablation setting.

  3. Interesting generalization findings. The observation that a model trained on systems with up to 10 gears can handle systems containing 100 gears when the kinematic height remains within the training distribution is notable.

  4. I also appreciate the PCA and MLP probing analysis which gives deeper insight on how kinematic information is represented across transformer layers.

Critical weaknesses
  1. Evidences from PCA and MLP may not be conclusive because even if the kinematic information is presented in intermediate features, it is not necessarily that the model actually uses the proposed algorithms to generate the final outputs.

  2. The scope of 2D gear system is narrow. There is big gap from understanding gears to generalize to real-world video understanding tasks. I understand that one paper could not do everything, but perhaps this should be in a discussion section.

  3. The study relies on a single model configuration. The experiments primarily investigate Wan2.1 with 1.3B parameters. Perhaps many findings are not true on different models trained with different technics/data.

Questions for authors
  1. How much of the change in reasoning behavior is attributable to kinematic height, as opposed to differences in gear count and topology?

  2. On temporal variation (sec. E3), the paper notes that individual frames do not encode identical parity information and that temporal pooling hurts performance. I think this is counter-intuitive (although I believe in this paper's empirical results), so could the authors clarify how the temporal dimension interacts with the spatial reasoning steps that leads to this worse result from temporal pooling?

  3. Could you test more on different generation models?

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer rLFZ (1/2)

We thank the reviewer for their thoughtful feedback and constructive comments. We are glad the reviewer found the topic interesting and well-scoped, and that our generalization findings and feature probing analyses provide useful insights. Below, we address each of the points in detail.

W1: Evidences from PCA and MLP may not be conclusive

We agree with the reviewer that feature probing does not indicate the model is executing exact algorithms as formal code. To reflect this, we intentionally use the suffix "-like" (i.e., parallel-BFS-like and divide-and-conquer-like) to describe high-level computational patterns.

For example, our interpretation of parallel-BFS-like reasoning is grounded in Figure 3, which shows that rotational parity is resolved depth-by-depth across transformer layers, starting from the driving gear. This order closely resembles a parallel BFS traversal. While the transformer may not run the exact algorithm, the evolution of its internal representations mirrors how these classical algorithms solve the task, hence we refer to this as parallel-BFS-like reasoning .

Furthermore, we have strong indications that these discovered reasoning mechanisms contribute to the final output. For instance, the model's generalization to unseen gear counts when kinematic height remains in-distribution (Table 1) is explained by the hypothesis that the model executes a depth-bounded parallel-BFS-like mechanism internally, which is consistent with our feature probing analysis visualized in Figures 3c and 4.

W2: The scope of 2D gear system is narrow

Our goal is to probe the upper-bound capabilities of video transformers to perform kinematic-chain reasoning with minimal confounding factors: if a model struggles with kinematic reasoning in this simplified setting, it will likely struggle even more in complex physical environments. Furthermore, narrowing the domain allows us to investigate internal reasoning mechanisms in depth rather than superficially covering broader settings. Specifically, the gear setting is particularly well-suited for our initial analysis for two reasons: (1) the gears' spatial positions do not change, allowing us to focus on the model's ability to infer rotational parity while easily monitoring its internal representations, and (2) all gears move simultaneously, enabling the study of long-chain kinematic reasoning without requiring long frame sequences.

Nevertheless, we agree that extending our analysis to other physical phenomena is a valuable next step, and we have revised the Discussion and Conclusion section of the paper to discuss this direction.

W3: Analysis with a different video backbone

To verify that our findings are not specific to Wan2.1-1.3B (a 30-layer DiT with cross-attention), we conducted additional experiments on CogVideoX-5B (a 42-layer MMDiT). Due to time and compute constraints during the rebuttal period, we partially fine-tuned CogVideoX-5B on systems with gear counts and kinematic heights up to 20. Despite limited training, MLP probing (visualized at https://cog-dit.github.io/#rebuttal-only) already reveals the same linear trend observed in Wan2.1-1.3B (Figures 3.d and 4 in the paper): parity prediction accuracy increases approximately linearly across transformer layers, which is characteristic of parallel BFS-like reasoning. Furthermore, as shown in the evaluation table on the rebuttal webpage (https://cog-dit.github.io/#rebuttal-only), when evaluated on unseen gear counts (e.g., 70, 90, 100 gears), the model generalizes to in-distribution kinematic heights (e.g., depths 8, 9, 10), while exhibiting low performance at out-of-distribution depths (e.g., depths 30, 40, etc.).

Please note that because training has not yet fully converged under the limited rebuttal budget, we currently focus our analysis on kinematic heights up to approximately 10, where the model performs reliably. We will continue updating the supplemental webpage with further results as training progresses. Nevertheless, these preliminary results already confirm that our core findings are not specific to Wan2.1-1.3B.

Thank you, the rebuttal is reasonable, so I kept my weak accept rating.

AuthorsOpenReview ↗

Author Response to Reviewer rLFZ (2/2)

We thank the reviewer for their response. For completeness, we address the remaining questions raised in the original review below.

Q1: How much of the change in reasoning behavior is attributable to kinematic height, as opposed to differences in gear count and topology?

We believe that the kinematic height in the training distribution is the primary factor influencing the model's reasoning behavior, rather than gear count or graph topology:

  • Topology (Branching Factor): In Table 2, we train our model solely on linear graphs yet demonstrate that it still learns parallel-BFS-like reasoning and generalizes to non-linear graphs (arbitrary tree topologies). This indicates that graph topology (specifically, the branching factor) is not the primary driver of the reasoning strategy.
  • Gear Count vs. Kinematic Height: Table 1 shows that when the model is trained up to a specific kinematic height and gear count, it exhibits reasonable generalization to unseen gear counts within the in-distribution kinematic height, whereas performance drops sharply at out-of-distribution kinematic heights. This indicates that the model's reasoning capability is governed primarily by kinematic height rather than total gear count.

Q2: Temporal variation

Thank you for this insightful question. We believe this temporal variation arises because not all frames are equally responsible for inferring rotational parity:

  1. The first frame does not require inferring rotational parity, as the initial state of the system is already provided as input.
  2. Even in subsequent frames, the full-attention mechanism allows the model to synchronize rotational parity information across frames at any time rather than independently computing it at every frame.

Because each frame is not enforced to execute the same reasoning operation, concatenation of all the feature vectors outperforms average temporal pooling in our MLP probing experiments.

Q3: testing on different generation models

Please refer to our original response to W3. Additionally, we have updated our results on CogVideoX-5B (https://cog-dit.github.io/#rebuttal-only). When trained on systems with up to 20 gears, CogVideoX-5B exhibits both parallel-BFS-like reasoning and a divide-and-conquer-like strategy, with parallel-BFS-like reasoning occurring up to a kinematic height of approximately 10. As shown in the evaluation table (https://cog-dit.github.io/#rebuttal-only), this leads to strong generalization to unseen gear counts up to a kinematic height of 10, and weaker generalization beyond that depth. This behavior is consistent with our experiments on Wan2.1-1.3B (Table 15, Figure 18, and Figure 25 in Appendix), further suggesting that our findings are not specific to Wan2.1-1.3B.

Reviewer_5epFOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper utilizes 2D involute gear trains as a testbed to train and analyze Wan2.1-1.3B, and the analysis reveals that models can learn kinematic chain reasoning but acquire two distinct types of reasoning mechanisms. For training on short kinematic heights, the model acquires a parallel BFS-like reasoning, using transformer layers as breadth-first search steps to incrementally determine parity across the kinematic tree. For training on large kinematic heights, the model adopts a divide-and-conquer-like strategy—first resolving parity within local neighborhoods and subsequently merging them to achieve global consistency.

Critical strengths
  1. The paper uses a controlled and quantitatively evaluable testbed by using 2D gear systems to provide explicit geometric and kinematic constraints, enabling objective evaluation rather than relying solely on visual inspection.
  2. The authors go beyond output-level evaluation by applying PCA and MLP probing to investigate how motion-related dependencies are represented in the model's internal activations.
  3. The experiments examine how performance and internal representations change with kinematic-chain complexity, offering useful insights into the model's capabilities and limitations.
Critical weaknesses
  1. Limited scope of the testbed. The paper uses gear systems as a testbed to evaluate video diffusion transformers’ ability to simulate interacting physical objects with kinematic dependencies. However, gear systems represent only one type of such system. Other settings, such as pulley systems and domino chains, could also be considered to evaluate whether the findings generalize to a broader range of physical interactions.

  2. Evaluation on only one video DiT architecture. The paper proposes to utilize 2D involute gear trains as a testbed to train and analyze video DiTs, but the experiments focus exclusively on Wan2.1-1.3B. Evaluating additional video DiT architectures, such as Open-Sora, CogVideoX, and HunyuanVideo, would help determine whether the observed behaviors and reasoning mechanisms are specific to Wan2.1-1.3B or generalize to other video diffusion models.

Questions for authors
  1. Lack of ablation studies on the RGB encoding/decoding scheme. The paper introduces a specialized RGB encoding/decoding scheme but does not provide an ablation study to investigate its contribution to the results. It would be helpful to evaluate the model without this scheme or with the original encoder and decoder to determine how much the proposed scheme contributes to the model’s performance and observed reasoning behaviors.

  2. Insufficient details on the commercial models used for comparison. The paper is motivated in part by failure cases of commercial video generation models, but it does not clarify whether these models use a similar RGB encoding/decoding scheme to represent the gear systems and their motion. Without this information, it is difficult to determine how directly comparable these failure cases are to the results obtained with the proposed setup.

  3. Unclear extent to which the task evaluates physical reasoning. The paper aims to investigate video DiTs’ ability to simulate interacting objects with kinematic dependencies. However, the model is provided with structured information about the gear system, including its spatial layout and the motion of the driving gear, rather than only a first frame and a text description, as in a more conventional video generation setting. Given this structured input, the gear motions can be determined by explicit kinematic constraints. Therefore, it remains unclear to what extent the experiments demonstrate the model’s ability to learn and reason about physical interactions, rather than to reproduce motion patterns from structured input representations.

7 comments
AuthorsOpenReview ↗

Author Response to Reviewer 5epF (1/2)

We thank the reviewer for their constructive feedback. We are glad that the reviewer appreciates our thorough analysis with PCA and MLP probing, and finds our insights into the model's reasoning capabilities useful. Below, we address each of the weaknesses raised in the review.

W1: Limited scope of the testbed (Domino Chains and Pulley Systems)

We thank the reviewer for highlighting domino chains and pulley systems as alternative settings beyond the gear domain.

Our goal is to probe the upper-bound reasoning capabilities of video transformers under minimal confounding factors: if a model struggles with kinematic reasonings in this idealized setting, it will likely struggle even more in complex physical environments. Furthermore, narrowing the domain allows us to investigate internal reasoning mechanisms in depth rather than superficially covering broader settings. We deliberately chose the gear setting as it is particularly well-suited for our analysis: (1) gears' spatial positions do not change, allowing us to focus on the model's ability to infer rotational parity while easily monitoring internal representations, and (2) all gears move simultaneously, enabling the study of long kinematic-chain reasoning without having to generate long frame sequences. In contrast, domino chains do not satisfy these properties, as dominoes fall sequentially and undergo translational motion. Meanwhile, pulley systems have limited branching factors (at most 2), whereas gear systems allow us to investigate more diverse graph structures.

Nevertheless, we agree that extending our analysis to other physical phenomena is a valuable next step, and we have revised the Conclusion section of the paper to discuss this.


W2: Analysis with a different video backbone

To verify that our findings are not specific to Wan2.1-1.3B (a 30-layer DiT with cross-attention), we conducted additional experiments on CogVideoX-5B (a 42-layer MMDiT). Due to time and compute constraints during the rebuttal period, we partially fine-tuned CogVideoX-5B on systems with up to 20 gear counts and kinematic heights. Despite limited training, MLP probing (visualized at https://cog-dit.github.io/#rebuttal-only) reveals the same linear trend observed in Wan2.1-1.3B (Figures 3.d and 4 in the paper): kinematic depth prediction accuracy increases approximately linearly across transformer layers, characteristic of parallel BFS-like reasoning. Furthermore, as demonstrated in the evaluation table on the rebuttal webpage (https://cog-dit.github.io/#rebuttal-only), when evaluated on unseen gear counts (e.g., 70, 90, 100 gears), the partially trained model generalizes to in-distribution kinematic heights (e.g., depths 8, 9, 10), while exhibiting significant performance drop at out-of-distribution depths (e.g., depths 30, 40, etc.).

Please note that because training has not yet fully converged under the limited rebuttal budget, we currently focus on kinematic heights up to approximately 10, where the model performs reliably. We will continue updating the supplemental webpage with further results as training progresses. Nevertheless, these preliminary results already strongly confirm that our core findings are not specific to Wan2.1-1.3B.

AuthorsOpenReview ↗

Author Response to Reviewer 5epF (2/2)

Q1: RGB encoding/decoding scheme

Our goal is to characterize the upper-bound kinematic-chain reasoning capabilities of video diffusion transformers. We therefore adopt our encoding scheme that minimizes unnecessary challenges, allowing us to focus solely on the kinematic-chain reasoning task itself. Specifically, this design serves two purposes:

  1. Explicitly encoding the driving gear's rotation direction through color eliminates the need for the model to infer absolute rotation direction at each frame by tracking visual textures.
  2. The RGB representation allows us to measure rotational correctness directly and deterministically, without relying on noisy point-tracking-based evaluation.

To clarify, we use the original VAE encoder and decoder without modifying them.


Q2: Commercial models used for comparison

In this paper we do not compare commercial models specifically, nor it is our goal to do so. While we do include the results of some commercial models in Figure 1 as a motivation, this is to show that even these strong models struggle with mechanical and kinematic reasoning (for clarity, those examples were generated by using off-the-shelf commercial models without any of the fine-tuning experiments we carry out in the paper, which are impossible to do for these closed-weights models).

The rest of the paper is dedicated to understanding the successes and gaps of video generators have in kinematic reasoning, as well as the mechanisms that they might use to do so. Because the latter requires probing models, we must use open-weight ones. However, these open-weights models are arguably weaker than commercial ones. When we show that open-weight models can perform kinematic reasoning, at least subject to fine-tuning, we would expect similar conclusions to transfer to stronger closed-weight models too.


Q3: Unclear extent to which the task evaluates physical reasoning

Physical reasoning involves many skills. Our paper focuses specifically on kinematic-chain reasoning, which is one of the important skills in physical reasoning. We deliberately simplify the setup (for example, by giving the gear layout and driving motion as input) so we can solely test whether the model can infer rotational parity across a kinematic chain. If a model cannot solve this task, it cannot solve harder general physical tasks. It would be an interesting direction to investigate other physical reasoning capabilities, which we added in the revised Discussion and Conclusion section.

Thank you for the response. It addresses part of my concerns.

W1, Q1, and Q2 are resolved.

W2 is still my main concern. To clarify my original point: the paper states that the model switches to divide-and-conquer “when parallel BFS requires more layers than are available in the DiT”. If this is true, a deeper model should switch at a larger height than Wan2.1. However, the current experiment uses heights up to 20, so neither model needs to switch. It cannot test this claim. I understand that more experiments may not be possible in the remaining time. I suggest presenting the CogVideoX-5B results as preliminary, not as strong confirmation. The link between model depth and the switch to divide-and-conquer should be stated as a hypothesis and discussed as a limitation.

For Q3, the authors agree that the task is simplified. The Conclusion has changed to “conducted in a simplified setting”. But other parts do not. For example, the abstract says “video DiTs are capable of learning algorithmic reasoning over kinematic chains”. Sec. 3 and App. D also describe the results in general terms. These claims should be limited to fine-tuned models in this simplified setting.

Based on the current evidence, I will keep my rating for now. If the revised paper makes its claims match its experiments and evidence, I will raise my rating to 3. For example, it should present the CogVideoX-5B results as preliminary, state the depth explanation as a hypothesis, and limit the claims to this simplified setting.

Reviewer_aStjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies kinematic chain reasoning in video diffusion transformers using 2D gear systems. The authors fine-tune Wan2.1-1.3B to generate gear motions conditioned on an initial layout and a driving gear’s motion. Feature probing reveals patterns of BFS like propagation after short-chain training and divide and conquer like computation after long-chain training. The experiments show that generalization depends strongly on dependency depth and training complexity, rather than object count alone.

Critical strengths
  1. The paper poses a clear, well-defined research question and studies a narrow setting in depth. The gear-system testbed offers precise kinematic constraints and measurable outcomes, enabling controlled evaluation and analysis.

  2. The evaluation is deterministic, measuring both rotation direction and speed through per-gear motion errors, while also reporting system-level success that requires every gear to move correctly.

  3. The videos on the anonymous project page are very clear, and the paper is easy to follow. Figures 3 and 6 effectively illustrate the BFS-like and divide-and-conquer-like patterns, respectively.

Critical weaknesses
  1. The analysis is limited to Wan2.1-1.3B. Evaluating other open-source video models, such as HunyuanVideo and CogVideoX, as well as larger model variants, would help determine whether the observed reasoning patterns persist or change with model architecture and scale.

  2. A promising future direction is to test whether these patterns generalize across other physical tasks with long dependencies, such as branching domino cascades, pulley systems. This would strengthen the broader significance of the findings, though the current paper is already self-contained.

Questions for authors

see weaknesses

1 comment
AuthorsOpenReview ↗

Author Response to Reviewer aStj

We thank the reviewer for their insightful feedback. We are glad that the reviewer finds our research question well-defined, our analysis thorough, and our results clearly presented and self-contained. In the following, we address each of the weaknesses raised in the review.

W1: Analysis with a different video backbone

To verify that our findings are not specific to Wan2.1-1.3B (a 30-layer DiT with cross-attention), we conducted additional experiments on CogVideoX-5B (a 42-layer MMDiT). Due to time and compute constraints during the rebuttal period, we partially fine-tuned CogVideoX-5B on systems with gear counts and kinematic heights up to 20. Despite limited training, MLP probing (visualized at https://cog-dit.github.io/#rebuttal-only) already reveals the same linear trend observed in Wan2.1-1.3B (Figures 3.d and 4 in the paper): parity prediction accuracy increases approximately linearly across transformer layers, which is characteristic of parallel BFS-like reasoning. Furthermore, as shown in the evaluation table on the rebuttal webpage (https://cog-dit.github.io/#rebuttal-only), when evaluated on unseen gear counts (e.g., 70, 90, 100 gears), the model generalizes to in-distribution kinematic heights (e.g., depths 8, 9, 10), while exhibiting low performance at out-of-distribution depths (e.g., depths 30, 40, etc.).

Please note that because training has not yet fully converged under the limited rebuttal budget, we currently focus our analysis on kinematic heights up to approximately 10, where the model performs reliably. We will continue updating the supplemental webpage with further results as training progresses. Nevertheless, these preliminary results already confirm that our core findings are not specific to Wan2.1-1.3B.

W2: Testing on Domino Cascades and Pulley Systems

We thank the reviewer for recognizing that our current study is already self-contained and for highlighting analysis on domino cascades and pulley systems as promising future extensions.

The gear setting is particularly well-suited for our initial analysis for two reasons: (1) the gears' spatial positions do not change, allowing us to focus on rotational parity, and (2) all gears move simultaneously, enabling the study of long kinematic-chain reasoning without requiring long frame sequences. In contrast, domino systems do not satisfy these properties, as dominoes fall sequentially (hence requiring long frame generation) and undergo translational motion (hence complicating the analysis). Furthermore, pulley systems have limited branching factors (at most 2, as they can not have multi-way branching), whereas gear systems allow us to investigate more diverse graph structures.

Nevertheless, we agree that extending our analysis to other physical phenomena is a valuable next step, and we have revised the Discussion and Conclusion section of the paper to discuss this direction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.