- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes SAPO to accelerate embodied large models incorporating continuous-depth networks through the co-design of structured prompts and low-rank computational modules. It is observed that explicitly partitioning text, sensor data, and queries enhances activation correlation across adjacent layers and reduces effective rank. Based on this, SAPO employs the Koopman–Mori–Zwanzig framework to replace high-dimensional LTC/Neural ODE layers with low-dimensional GRUs, matrix exponentials, and output projections. Experiments report training speedups of up to 3.55x and testing speedups of approximately 2.9x.
Critical strengths
-
The study offers a novel perspective by linking prompt structure to the model's internal computational efficiency.
-
Experiments were conducted across various datasets and architectures, reporting metrics such as wall-clock time, energy consumption, and effective rank.
-
The finding that structured prompts significantly enhance activation locality and reduce effective rank offers valuable insights.
Critical weaknesses
-
The mechanism by which structured prompts accelerate fixed-dimension dense GRU and matrix operations remains unclear.
-
The theoretical results primarily consist of an existence proof regarding the expressive power of GRUs, relying on strong steady-state assumptions. They fail to provide sufficient guarantees for the actual approximation error of finite-dimensional SAPOs.
-
There is an internal inconsistency in the experimental setting. For instance, the main text states that the primary experiment used P=16, yet the main results align with the P=4 values reported in the appendix.
-
The evaluation focuses primarily on specific models incorporating LTC/Neural ODEs and is insufficient to support claims of broad applicability to general LLMs or Transformers.
Questions for authors
Please refer to the Critical Weaknesses part.
Thank you reviewer 3TGN08 for giving some comments we can understand!
Response to Reviewer
Q1. The mechanism by which structured prompts accelerate fixed-dimension dense GRU and matrix operations remains unclear.
A1. Section 2 explains how structured and unstructured prompts differ, and shows the mechanism by which the structured prompt leads to accelerated fixed-dimension dense GRU and matrix operations.
Q2. The theoretical results primarily consist of an existence proof regarding the expressive power of GRUs, relying on strong steady-state assumptions. They fail to provide sufficient guarantees for the actual approximation error of finite-dimensional SAPOs.
A2. Section 3 and the Appendix give the full proof of the approximation error when the GRU replaces the LTC, so the result goes beyond an existence statement. The ablation study in Table 3 also measures this empirically: it shows how timing and loss differ under the replacement.
Q3. There is an internal inconsistency in the experimental setting. For instance, the main text states that the primary experiment used P=16, yet the main results align with the P=4 values reported in the appendix.
A3. We found that P=16 gives the best trade-off between accuracy and timing, so we use it for the main experiments. The P=4 setting appears in the Appendix as an extended experiment showing how performance and error change. With P=4 there is less computation, but the error is higher because the forward layer's weights store less information. Please see Table 24 for details.
Q4. The evaluation focuses primarily on specific models incorporating LTC/Neural ODEs and is insufficient to support claims of broad applicability to general LLMs or Transformers.
A4. Our paper focuses on continuous-time neural networks (CTNNs), and our theory is built on the CTNN formulation. Table 4 also includes several different RNN architectures to show that the approach applies beyond a single model.