- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper studies how many state-to-state connections are needed when several action-conditioned dynamics must use the same coordinates. It compares the minimum block connectivity under separate and shared orthogonal bases. For generic symmetric operators, the paper gives a linear-to-quadratic separation and studies whether this gap persists under approximate compression. It also shows that a common sparse interaction structure can keep the shared representation sparse. The experiments cover synthetic matrix families and local Jacobians from world models trained on an NRI spring system.
Critical strengths
The paper gives a concrete formulation of the difference between individually simple dynamics and a simple shared representation. The finite-budget analysis is an important part of this formulation: it asks whether the difference remains after allowing weak connections to be discarded, rather than relying only on exact sparsity.
The learned-model experiments make a useful effort to isolate the effect of shared interaction support. Reporting prediction error and separate-basis compression error alongside the shared-coordinate penalty helps distinguish the effect from differences in model fit or individual-action compressibility. The slot-model comparison also provides a relevant control.
Critical weaknesses
- The connection between the main scaling result and learned world models remains limited. The linear-to-quadratic separation holds for generic symmetric operators. For general real operators, both costs are already quadratic, and the learned Jacobians in Section 6.4 are mostly nonsymmetric. The paper acknowledges this distinction and includes a potential-model control in Appendix E.4. However, I still find the link between the scaling results and the measured compression penalty in the learned models insufficiently developed. The experiments show that support sharing matters, but do not establish how the theoretical connectivity counts explain the size of the observed penalty.
- It is unclear what the measured penalty tells us about a world model beyond its local compressibility. The main learned-model result is a difference in , computed from local residual Jacobians in a small spring-system setting. This supports the proposed structural interpretation, but the paper does not show whether the difference matters when sparse dynamics are actually used for prediction or rollout, or whether it translates into a computational benefit. A new architecture or a SOTA comparison is not necessary here. My concern is that the paper stops short of explaining what a world-model researcher should conclude or do differently after observing a large . This limits the significance of the contribution for me.
Questions for authors
-
How should the general-real connectivity result in Eq. (10) be connected to , which measures approximation error under a per-action budget? Does the theory give a prediction about the dependence of this measured penalty on the budget or state dimension?
-
Is there evidence in the existing experiments or analysis that the difference in matters when the compressed dynamics are used for prediction or rollout? More generally, what conclusion should a model designer draw from a large penalty when the original models have similar prediction errors?
-
The paper distinguishes its objective from joint diagonalization and joint block diagonalization. Could the authors make the comparison more precise for Theorems 1 and 2, identifying the closest existing results and what additional result is needed to obtain the connectivity count and finite-budget bound?
3 comments
Practical Significance of F_b for Prediction, Rollout, and Efficiency
Interpreting F_b first requires distinguishing the compressibility of an individual action from the representational compatibility of multiple actions. A world model must maintain a persistent internal state, so it cannot freely redefine its state coordinates whenever the action changes. Even if each action admits a simple local representation on its own, all actions need not remain simple in one common set of state coordinates. F_b measures the additional structural cost imposed by this cross-action sharing constraint, rather than the predictive quality of the original model itself.
To test whether this structural difference affects prediction in practice, we added three real-robot tasks: Lift, Can, and Tool Hang. We use the same frozen dynamics model and compress only its residual update, thereby isolating the effect of the shared-coordinate constraint. The improvements in real-trajectory prediction MSE achieved by Separate over Shared are as follows:
- Lift: 2.67% for one-step prediction and 12.23% for ten-step prediction.
- Can: 4.22% for one-step prediction and 13.87% for ten-step prediction.
- Tool Hang: 3.00% for one-step prediction and 10.99% for ten-step prediction.
These results indicate that the structural cost of shared coordinates is not reflected only in the number of nonzero connections in local Jacobians. This incompatibility can also affect practical prediction when the compressed dynamics are actually used to update robot states.
Shared coordinates retain complexity in every dynamics update. By contrast, action-specific coordinates allow each action to maintain a simpler local structure, but require coordinate conversion when the action changes. Appendix D quantifies this cost and gives the average connectivity cost associated with action switches. Accordingly, action-specific coordinates may offer computational advantages when action switches are infrequent, whereas coordinate conversion itself may become a major overhead when switches are frequent.
For ten-step prediction, both sparse schemes proposed in the paper achieve approximately 1.32–1.33x speedup over the full dense model. This demonstrates computational savings at the level of the overall task.
These findings point to more than the fact that Separate outperforms Shared in some experiments. They suggest a broader model-design principle. When F_b is large, the difficulty may not lie in compressing any individual action, but in making different actions share the same local state representation economically. Model designers can respond by increasing the shared connection budget, exploiting a common sparse interaction structure, revisiting the state partition, or using action-conditioned coordinates when the cost of switching is acceptable. When the sharing penalty is small, a unified persistent state representation may be the more economical choice.
Relation to Joint Diagonalization and Joint Orthogonal Sparsification
Thank you for recommending relevant work on matrix decompositions. We will expand the comparison with these studies and clarify the contribution of our work. Joint diagonalization, finest common block-diagonal decomposition, and the identification of common block structures under congruence transformations mainly address the existence and identification of shared diagonal structures in a family of matrices. However, the absence of nontrivial common invariant subspaces does not imply that several matrices cannot remain sparse in the same basis. For example, a diagonal matrix with distinct diagonal entries and an irreducible symmetric tridiagonal matrix have no nontrivial common invariant coordinate blocks, yet both have only a linear number of nonzero entries in their original coordinates. Therefore, the finest common block-diagonal decomposition alone cannot determine the number of connections under arbitrary sparse supports.
The work by Ba et al. on joint orthogonal Hessian sparsification is also closely related to ours. It studies how multiple Hessian matrices can be sparsified through a common orthogonal transformation and analyzes the corresponding optimization problem and its properties. We further investigate two quantitative questions underlying this problem:
-
For a generic operator family, what are the exact minimum connection counts under action-specific coordinates and shared coordinates? In other words, what is the additional cost of enforcing a common orthogonal basis?
-
Does this sharing cost persist when weak connections are allowed to be discarded?
We therefore do not propose a new joint-diagonalization or joint-sparsification algorithm. Instead, starting from the geometric compatibility of common representations, we quantify the structural cost that multiple actions must incur when they share the same state coordinates, and characterize the conditions under which this cost persists under finite-budget approximation. We will discuss the closest related studies more precisely and clarify the additional theoretical guarantees provided by our two theorems.
General Real Operators and the Finite-Budget Penalty F_b
We agree that the linear-to-quadratic separation in the original manuscript was established for generic symmetric operators and should not be directly applied to learned Jacobians, which are typically nonsymmetric. However, the central question of this paper is more general. Different actions may each have simple local dynamics, but this does not mean that they can share a single orthogonal coordinate frame without additional cost. From the perspective of operator geometry, the issue is not merely how many matrix entries are nonzero. It is whether the local subspace decompositions induced by different actions are compatible with one global coordinate frame.
For general real operators, we have further derived the exact connection counts. For even block width,
C_{\mathrm{sep}}=k\frac{m(m+1)}{2}, \qquad C_{\mathrm{sh}}=(k-1)m^2+\frac{m(m+1)}{2}.
Thus, although both schemes have quadratic complexity for general real operators, shared coordinates still incur a quadratic absolute overhead in the number of connections:
The ratio of the two complexities approaches 2-1/k. This shows that nonsymmetry changes the relative growth of the sharing cost but does not remove the structural cost of requiring different actions to use the same coordinate frame. We have also derived corresponding upper and lower bounds for odd block widths.
We have also established a theoretical connection between the exact connection counts and the finite-budget penalty F_b. Let E_B^{\mathrm{total}} denote the minimum error when all actions share one orthogonal basis under a total connection budget of B. Then
E_{kb}^{\mathrm{total}}\leq E_b^{\mathrm{sh}},
which gives
This relation directly connects the theoretical approximation obstruction under a total budget to the shared penalty defined per action in our experiments. Furthermore, for general real Gaussian operator families, within an explicitly specified quadratic budget range, separate coordinates permit exact representation, whereas shared coordinates retain, with high probability, an approximation-error lower bound that does not vanish with dimension. In 15 numerical instances with k=4,r=2, the shared scheme still incurs feasible relative errors of approximately 5.42%–6.82%, even after more extensive optimization checks.
These results characterize the structural obstruction to shared representations from two perspectives. First, the geometric degrees of freedom in a common coordinate frame are insufficient to eliminate all cross-subspace couplings for a generic operator family. Second, this obstruction is not limited to exact sparsity and may persist under finite-budget approximation. We also distinguish theoretical guarantees from experiments on learned models. The Gaussian asymptotic result does not directly predict the specific value of F_b for each learned model. Rather, it provides a further theoretical characterization of the obstruction to shared representations for general nonsymmetric operator families. We will clarify the applicability conditions of these results and their relationship to the experiments with learned Jacobians.