- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the difference between fusion before compression and separate compression followed by fusion under limited representation capacity, emphasizing that their relative advantages depend on the prediction target. The theoretical analysis provides a dimensional lower bound for recovering cross-system relations under separate compression and a target-relative information decomposition. Experiments on nonlinear oscillators, Lorenz systems, stochastic double-well interactions, and planar contact systems examine how system partition, prediction target, physical coupling, and representation width affect the performance of different fusion strategies.
Critical strengths
-
The paper provides a clear theoretical argument with explicitly stated assumptions and limitations. It presents a useful perspective on how fusion strategies depend on the interplay among system partition, prediction target, and representation capacity.
-
The paper conducts extensive controlled experiments across several dynamical systems, providing empirical evidence to support its main observations.
Critical weaknesses
-
My main concern is that the contributions appear relatively incremental, while the empirical validation is largely limited to low-dimensional toy models. The distinction between early and late fusion has been extensively studied in multimodal learning, and the discussion of target-relevant information and representation capacity is closely related to established ideas in information bottleneck and distributed representation learning. For example, relevant works such as Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations (Mai et al., 2023) and Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework (Liang et al., 2023) are not discussed. Furthermore, the predominantly low-dimensional experimental settings make it difficult to assess how well the findings generalize to practical applications such as realistic embodied representation learning.
-
There appears to be an inconsistency in Appendix Figure 4. At , the separate architecture still has an MSE close to the mean-prediction baseline, whereas the figure caption suggests that it approaches the performance of the joint architecture once each branch has sufficient width. This also seems inconsistent with the discussion in Appendix B.2.
Questions for authors
-
How can the findings of this paper guide representation design in realistic embodied learning systems? Could the authors further clarify the practical implications beyond the controlled low-dimensional settings?
-
Could the authors clarify the discrepancy between Appendix Figure 4 and its description? Why does the separate architecture still perform close to the mean-prediction baseline at , despite having sufficient representation width in principle?