- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SS-DT, using intra-head shared–private overlapping subspaces, spike-guided soft allocation and latent prediction feedback for offline decision making. Across 21 D4RL MuJoCo and Adroit tasks, it reports higher mean normalized return than PSSA on 19 tasks and a roughly 3.7-point overall average improvement. It includes progressive component ablations, an added-head comparison, seed-wise plots for selected tasks, and qualitative/entropy-based communication analyses. The method is plausible, but the evidence does not yet isolate the contribution and cost of the central inductive bias
Critical strengths
- Broad task coverage and several meaningful gains. In Table 1, the proposed method improves the reported means on 19 of 21 tasks relative to PSSA, notably pen-cloned (80.05 vs 44.8). The MuJoCo/Adroit split demonstrates that the design is not evaluated solely on locomotion.
- The work attempts to connect a communication-structure hypothesis to learned behavior rather than reporting only performance. Figure 1 motivates recurrent and variable token-pair supports; Figures 4–5 and supplementary Appendix C provide subspace allocations, Jaccard similarities and communication-entropy statistics. These are useful descriptive results, albeit not proof of causal necessity.
Critical weaknesses
- The central shared–private design is not independently justified by the current ablations. Equation (4) is an algebraic decomposition of concatenated projections, not evidence that common and private information are disentangled. Table 2 does contain an ablation, but its 'PSSA + Shared–Private' row improves only three of five listed tasks and decreases two relative to PSSA. The full method adds spike allocation, predictive feedback, a two-stage process and additional capacity; the PSSA+Heads comparison does not match those components and operation budgets. Moreover, no corresponding controlled ablation is reported for Adroit, where the largest benefits occur. Correlation regularization is mentioned in Eq. (19) but lacks a sufficiently explicit evaluated contribution. This weakens attribution to the proposed overlapping subspace inductive bias.
- Robustness and spiking-efficiency claims remain underdocumented. Figure 3 does not identify the comparison baseline sufficiently clearly, and its positive hopper-medium-replay seed-wise gains are difficult to reconcile with the Table 1 mean being 0.89 points below PSSA without further protocol/baseline details. Figure 3 also shows considerable seed variation on hopper-medium. Table 2 provides no uncertainty for incremental gains. In addition, the learned shared–private projections follow binary LIF Q/K/V, and the two-stage predictor plus mean-normalized sigmoid gating introduce dense real-valued operations; no end-to-end parameter/operation, latency or energy accounting is supplied. This does not make a spiking implementation impossible, but it leaves the return-versus-cost tradeoff unsupported.
- Figures 4–5 and supplementary analyses illustrate context-dependent subspace allocation, but these observations do not establish that such adaptation improves decision-making. Although Table 2 evaluates spike allocation, it does not distinguish the benefit of context-dependent gating from that of learned but fixed communication weights. This leaves the functional necessity of adaptive subspace routing insufficiently demonstrated
Questions for authors
- Please provide a resource-matched comparison of shared–private OSSA against fully independent and/or fully shared subspaces with the same gating, predictive objective and two-stage protocol. Can you distinguish the prediction loss from feedback modulation and add uncertainty on representative MuJoCo and Adroit tasks?
- Please identify Figure 3's exact baseline, checkpoint selection, training/evaluation seeds and episode counts, and explain the hopper-medium-replay discrepancy relative to Table 1. Which improvements are reliably larger than training/evaluation variance?
- After Eq. (3), are projected Q/K/V subspace features re-spiked or real-valued? How does supplementary Attn_spk map to main Eq. (5), and what are the complete inference costs (both stages, all subspaces, prediction and gating) relative to PSSA?
- Can the authors compare the proposed adaptive gating with fixed or context-shuffled gating while keeping the subspace projections and model capacity unchanged? This would help determine whether context-dependent communication itself contributes to improved decision-making performance.
5 comments
Q4: We compare learned token-independent gates, context-shuffled gates, and adaptive gates while retaining the same OSSA projections, predictor, and two-pass protocol. Context shuffling preserves the marginal gate-value distribution while changing which trajectory receives each gate vector; the causal/local attention mask remains unchanged. Table below reports the resulting returns.
Table: Effect of context-dependent communication allocation.
| Task | Learned fixed gate | Context-shuffled gate | Adaptive gate (SS-DT) |
|---|---|---|---|
| Hopper-medium-replay | |||
| Walker2d-medium-replay | |||
| Pen-cloned | |||
| Pen-human |
Adaptive gates exceed fixed gates by 2.51, 3.03, 4.85, and 3.57 points across the four tasks, respectively. Their gains over context-shuffled gates are 3.71, 3.83, 6.15, and 5.47 points. The shuffled control tests whether learned gate values suffice without being assigned to the corresponding decision context; the consistent reduction in returns supports the functional contribution of context-aligned allocation.
W1: The answer to Q1 provides the missing resource-matched structural comparison on both MuJoCo and Adroit: OSSA outperforms both fully shared and fully independent channels while retaining the other model components. The accompanying ablation separately evaluates predictive supervision, feedback modulation, and the explicitly defined private-projection correlation penalty. These controlled comparisons provide evidence that the shared--private design contributes to the observed performance gains.
W2: The answer to Q2 identifies the matched PSSA rerun behind the Hopper-medium-replay panel of Fig. 3 and reconciles it with the separately published PSSA aggregate in Table 1. For the distinct Hopper-medium task, we extended the paired evaluation from four to seven training seeds (123–129). SS-DT achieves higher returns on five of the seven seeds, with a mean paired improvement of 2.00 points. Although the gains vary across seeds, the overall mean remains positive. The answer to Q1 also reports seed-level uncertainty for the controlled component comparisons. Q3 clarifies that binary LIF outputs are followed by real-valued operations and accounts for both Transformer passes in the parameter and dense-equivalent FLOP estimates.
Table: Paired-seed comparison between PSSA and SS-DT on Hopper-medium.
| Training seed | PSSA Score | SS-DT Score | Paired gain |
|---|---|---|---|
| 123 | 79.32 | 76.98 | −2.34 |
| 124 | 78.84 | 77.87 | −0.97 |
| 125 | 72.24 | 73.30 | +1.06 |
| 126 | 71.89 | 80.99 | +9.10 |
| 127 | 76.30 | 77.05 | +0.75 |
| 128 | 75.69 | 78.45 | +2.76 |
| 129 | 73.29 | 76.91 | +3.62 |
| Mean ± sample SD | 75.37 ± 3.02 | 77.36 ± 2.29 | +2.00 ± 3.74 |
W3: Figures 4-5 visualize context-dependent subspace allocation, while the additional experiment in Q4 evaluates its contribution to performance. With subspace projections and model capacity unchanged, adaptive gating outperforms both learned fixed gating and context-shuffled gating across all reported tasks. In particular, the shuffled control retains the learned gate-value distribution but removes its alignment with the input trajectory, providing direct evidence that context-dependent communication contributes to improved decision-making.
Thanks for the constructive suggestions. Given the limited time, we are currently incorporating all relevant clarifications into our revised manuscript.
Q1: We thank the reviewer for identifying two attribution questions: whether shared-private communication is beneficial under matched resources, and whether predictive feedback contributes beyond auxiliary prediction supervision. We address them with controlled comparisons using the same outer attention-head count, training and evaluation protocol, return targets, checkpoint-selection rule, and paired training seeds. All reported returns are means ± standard deviations over matched training seeds, using the same seed set across compared variants.
Shared-private subspace construction. We compare three ways of constructing four communication channels within each attention head. Fully shared channels use the same query, key, and value projections; fully independent channels use separate projections; OSSA combines projections shared across channels with channel-specific private projections. The adaptive gate, latent prediction objective, feedback modulation, and two Transformer passes are held fixed. Channel widths are adjusted to match parameter and operation budgets as closely as possible. Table below reports the resulting costs and normalized returns. Under closely matched parameter counts, FLOPs, and inference latency, OSSA consistently outperforms both controls across all three representative MuJoCo and Adroit tasks.
Table: Resource-matched subspace constructions in the complete model.
| Variant | Params (M) | FLOPs (G) | Latency (ms) | Hopper-m-r | Walker2d-m-r | Pen-cloned |
|---|---|---|---|---|---|---|
| Fully shared | 6.433 | 1.276 | 36.90 | |||
| Fully independent | 6.438 | 1.273 | 37.70 | |||
| Shared–private OSSA (SS-DT) | 6.437 | 1.274 | 37.38 |
Auxiliary prediction versus feedback modulation. We hold OSSA fixed and compare three configurations: no latent prediction, auxiliary prediction without error-based gate modulation, and the full model with both. Each retains the two-pass protocol. We also remove only the private-projection correlation penalty from the full model to evaluate its separate contribution. Auxiliary prediction consistently improves performance over OSSA across all three tasks. Using the prediction error to further modulate the communication gate brings additional gains, reaching 3.85 points on Pen-cloned.
Removing only also leads to performance drops across all three tasks, confirming its additional contribution. These results demonstrate the complementary benefits of predictive supervision, error-based feedback modulation, and private-projection correlation regularization. We define it explicitly in the Method: it penalizes similarity among private projections, while the shared projection remains available to every channel.
Table: Attribution of predictive supervision, feedback modulation, and private-projection correlation regularization.
| Variant | Error modulates gate | Hopper-m-r | Walker2d-m-r | Pen-cloned | ||
|---|---|---|---|---|---|---|
| OSSA | ||||||
| OSSA + Aux-only | ||||||
| Full model w/o | ||||||
| Full model |
Q2: We apologize for not distinguishing the two sources of PSSA results in the manuscript. Table 1 cites the result published in the original PSSA paper, whereas Fig. 3 compares SS-DT with our rerun of the released PSSA code using matched training seeds 123–126. For each run, we select the highest-scoring evaluated checkpoint–RTG pair. Each target is evaluated over five environment-seed groups of ten episodes. The table below lists the selected iteration, target RTG, and score for each Hopper-medium-replay seed. The published PSSA Hopper-medium-replay score in Table 1 is 96.3±1.3, whereas our paired PSSA rerun achieves 92.38±2.45, compared with 95.41±2.58 for SS-DT. The four paired improvements are +2.17, +3.54, +2.77, and +3.61 points, respectively, yielding a mean paired improvement of 3.02 points with a sample standard deviation of 0.68. Thus, the Table 1 comparison and Fig. 3 paired comparison use different PSSA reporting sources. We identify both sources, the checkpoint-selection rule, and the evaluation episode count in the revised Table 1 note and Fig. 3 caption. All four training seeds yield positive paired improvements, ranging from 2.17 to 3.61 points, indicating consistent gains across the evaluated seeds. The sample standard deviation of the paired gains is 0.68 points, suggesting relatively low variation across training seeds.
Table: Provenance of the Hopper-medium-replay comparison in Fig. 3. Both models use d=256, four Transformer blocks, one attention head, and training seeds 123-126. Each evaluated target uses five environment-seed groups of ten episodes.
Q3: As in PSSA, Linear-Norm-LIF projections produce binary Q/K/V spikes. OSSA applies learned shared and private projections to these spikes; the resulting subspace features and their attention products are real-valued. After the subspace messages are combined, the attention output passes through an LIF neuron before continuing through the spiking Transformer. We replace the supplementary Attnspk shorthand with the explicit masked and positionally modulated QK⊤V computation corresponding to the main-text attention equation.
We profile the complete inference path, including both Transformer passes, all subspaces, dense projections, attention matrix products, latent prediction, communication gating, and the action head. The following table compares PSSA, its widened variant (PSSA-MoreHeads), and SS-DT under the same batch size and context length.
Table: Computational cost and performance comparison (B=1, context length 20).
Although SS-DT introduces additional parameters and inference overhead compared with the original PSSA, it achieves better performance and stability. And, as demonstrated in our response to Q1, resource-matched comparisons against fully shared and fully independent subspace constructions show that SS-DT consistently achieves better performance under closely matched parameter counts, FLOPs, and latency. This provides further evidence that the shared-private subspace design contributes to the performance gains beyond simply increasing model capacity.