- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes Delay-SSD, a modification of the SSD token mixer. The paper adds fixed-lag inputs (e.g., the row above in a flattened image) to the Mamba-2 state update, sharing one state and decay so the chunked scan is preserved. Experiments on classification, detection, segmentation and traffic forecasting demonstrates its effectiveness.
Critical strengths
-
The mechanism is simple and well-motivated.
-
The paper is easy-to-follow.
Critical weaknesses
- The paper misses a series of important and influential baselines. Spatial-Mamba is cited but not included in Table 1, although it is the closest functional competitor: it aggregates states over a spatial neighborhood; Backbones with explicit 2-D spatial decay or linear attention, such as RMT and MLLA, address the same scan-distance bias and are neither compared nor discussed; Since DSSD uses global attention in stages 3–4, it is a hybrid model instead of pure mamba model. It should be compared more systematically against hybrid backbones such as TransNeXt, not only MambaVision; It also includes no comparison against recent mixers such as Mamba-3 or Gated DeltaNet, even though the fused kernel is adapted from Mamba-3.
Questions for authors
-
How does Delay-SSD compare to an input-side stride- token shift followed by a standard SSD, at matched parameters and cost, on the ImageNet subset?
-
How does it compare to VMamba2/DSSD-without-lags with 5×5 or 7×7 depthwise convolutions at matched cost on the ImageNet subset?
-
Can you report ImageNet results for Spatial-Mamba, RMT and MLLA alongside yours, and add FLOPs- and throughput-matched versions of Table 1?
3 comments
Part 2/2
W4. Continued In two-stream recall, 46% of answers lie in the current row, which a scan reaches without a lag; this accounts for SSD's 49%. At state size 64 or 128, Mamba-3 gains at most 8 points on any task. These results go into Appendices D and E. The mechanisms are complementary: a lag can be added to Mamba-3's update directly, though we leave this as future work.
Q1 and Q2 share the protocol of Table G.1, to which these rows are added: DSSD-N on the 100-class ImageNet subset, 100 epochs, parameters held equal through the FFN width, mean standard deviation over 3 seeds.
| Variant | Top-1 |
|---|---|
| No lags, 3×3 conv | |
| No lags, input-side shift | |
| No lags, 5×5 conv | |
| No lags, 7×7 conv | |
| Lags, 3×3 conv |
W4. Mamba-3 and Gated DeltaNet. Our theorems and claims concern scalar-decay SSD and cover neither update, so we measured both. Mamba-3's two-term update combines the current and the previous token, a lag of 1; Delay-SSD writes a token steps back, and with , the row length, the pixel above enters the state undecayed. Setup: single layers on the synthetic tasks of Section 3, state size 16, public implementations at library defaults (mamba-ssm; fla, without Gated DeltaNet's short convolution), a learning rate chosen per mixer, 3 seeds.
Delay task (Section 3.3): output the input delayed by steps; relative RMSE, where 1 means predicting zero. Delay-SSD with stays at or below 0.014 for every from 8 to 1024. Mamba-3 reaches 0.23 at , 0.78 at 16 and 0.91 or above from 32. Gated DeltaNet and SSD are at 0.83 or above for every .
Routing tasks (Section 3.2): a 16 × 64 token grid scanned row by row (), without convolution or position encoding; best eval accuracy. Copy-above outputs the token directly above. Row-recall returns the value paired with a key in the row above. Two-stream recall returns the value of the most recent occurrence of a key in the current row or the row above.
| Task | Delay-SSD | Mamba-3 | Gated DeltaNet | SSD | Chance |
|---|---|---|---|---|---|
| copy-above | 100% | 35% | 19% | 13% | 13% |
| row-recall | 100% | 69% | 43% | 25% | 25% |
| two-stream recall | 98% | 57% | 51% | 49% | 6% |
In two-stream recall, 46% of answers lie in the current row, which a scan reaches without a lag; this accounts for SSD's 49%. At state size 64 or 128, Mamba-3 gains at most 8 points on any task. These results go into Appendices D and E. The mechanisms are complementary: a lag can be added to Mamba-3's update directly.
Q1 and Q2 share the protocol of Table G.1, to which these rows are added: DSSD-N on the 100-class ImageNet subset, 100 epochs, parameters held equal through the FFN width, mean ± standard deviation over 3 seeds.
| Variant | Top-1 |
|---|---|
| No lags, 3×3 conv | |
| No lags, input-side shift | |
| No lags, 5×5 conv | |
| No lags, 7×7 conv | |
| Lags, 3×3 conv |
Q1. Input-side stride- shift before a standard SSD. The shift is indistinguishable from the no-lag model, and the lags lead it in every seed, by 0.58 on average (paired one-sided -test, ).
Q2. Wider convolutions without lags. We widen the 3×3 depthwise convolution of every stage 1–2 block. The 5×5 model is within the seed spread of the no-lag model, and the lags lead it in all three seeds (by 0.02 to 0.67). The 7×7 model loses about a point in every seed at the same configuration.
Both alternatives train at the no-lag model's speed (about 1,470–1,480 images/s) and reach its accuracy, so the equal-time comparison is the one already in Table G.1: the no-lag model with the lags' time spent on extra stage 1–2 blocks reaches , a gap to the lags that is within its seed spread.
Q3. FLOPs- and throughput-matched views. The table in part 1 gives parameters and FLOPs for the requested models. We claim no FLOPs or throughput advantage at the model level. Table F.5 and Figure F.3 already report accuracy against inference throughput for the comparators that run on our stack, and DSSD is the slowest at each size: DSSD-T runs at 2,890 images/s against 4,077 for VMamba-T, and DSSD-S at 1,063 against 1,850. We have not timed Spatial-Mamba, RMT or MILA and claim no advantage over them. Our cost claim is at the mixer level: two lags take 1.36–1.48× the time of the four-scan SSD mixer, against about 2× for a second SSD layer (Section 3.4).
Thanks, I think this resolves my concerns. I would like to raise the score to 3.
(1) Our vision claims concern the lag mechanism and are made relative to Mamba2D, not state of the art across architectures. Spatial-Mamba, RMT, MILA and TransNeXt added to tables, additions reproduced below. (2) On our synthetic tasks, Mamba-3 and Gated DeltaNet share the limitation we address. (3) At equal parameters on the ImageNet subset, the lags beat an input-side shift in every seed, and a wider convolution stays within the seed spread of the no-lag model.
Part 1/2
We thank the reviewer for the thorough review and the baseline reccomendations.
Published results: ImageNet-1k top-1 at 224 px; ADE20K single-scale mIoU with UperNet (160k iterations), FLOPs at 512×2048. Rows are grouped by ImageNet FLOPs, so DSSD-S sits with the -B models. RMT is without token labeling. MILA and Spatial-Mamba train with MESA, which DSSD does not use; without it MILA reports 83.3% (T) and 85.0% (B).
W1. Spatial-Mamba. DSSD leads on ADE20K at both sizes (50.7 against 48.6, 52.2 against 51.8), at 11–12% higher UperNet FLOPs in part because its stage 3–4 global attention is quadratic in the token count (Table F.1). On ImageNet-1k, DSSD-T leads by 0.4 points, and DSSD-S matches Spatial-Mamba-B with about half the parameters. The mechanisms also differ. Spatial-Mamba runs the scan first and then mixes each token's state with its neighbours' states through dilated depthwise filters before the read-out, so later scan steps never see the fused states. Delay-SSD writes the lagged token into the recurrent state during the scan, so every later token may access it through the state.
W2. RMT and MLLA. Our vision claims are made relative to Mamba2D, whose macro-architecture and recipe DSSD shares: parity on ImageNet-1k and COCO, and an ADE20K gain that the lag-on/lag-off pair attributes to the lags. We do not claim state of the art across architectures. RMT and MILA (the current name of MLLA) address the mismatch between scan distance and image distance without a scan, RMT with a 2-D Manhattan-distance decay inside softmax attention and MILA with linear attention in every stage. We add both to Related Work, to Table I.2, which already lists attention and convolutional backbones, and to the new appendix table. Table 1 keeps its scope, SSM-based and SSM-hybrid backbones with Swin as the customary reference, and its caption clarifies this. We are open to adding these to Table 1 space permitting. On ImageNet-1k, RMT-S leads DSSD-T by 0.2 points and DSSD-S leads RMT-B by 0.3 at 45% more FLOPs. On ADE20K, DSSD leads RMT by 0.9 (T) and 0.2 (B) mIoU and MILA-B by 0.3, at higher UperNet FLOPs.
W3. Hybrid backbones such as TransNeXt. We agree that DSSD is a hybrid, and the abstract and Section 4 now clarify that selective-scan mixers in stages 1–2 and global attention in stages 3–4, as in Mamba2D. Table 1 already compares it with the two closest Mamba hybrids, Mamba2D and MambaVision. TransNeXt uses attention in every stage and no SSM, so it is categorized with RMT and MILA as linear attention architectures. It leads DSSD-T by 0.4 mIoU and 0.1 points and ties DSSD-S on ADE20K. Section 4.2's "the best backbone in every size group" on ADE20K is updated to "the best SSM-based backbone", and the abstract's T-size margin over Mamba2D uses the run under Mamba2D's released schedule (1.55 mIoU; see response to Reviewer iuXR, W4).
W4. Mamba-3 and Gated DeltaNet. Our theorems and claims concern scalar-decay SSD and cover neither update, so we measured both. Mamba-3's two-term update combines the current and the previous token, a lag of 1; Delay-SSD writes a token p steps back, and with p=W, the row length, the pixel above enters the state undecayed. Setup: single layers on the synthetic tasks of Section 3, state size 16, public implementations at library defaults (mamba-ssm; fla, without Gated DeltaNet's short convolution), a learning rate chosen per mixer, 3 seeds.
Delay task (Section 3.3): output the input delayed by τ steps; relative RMSE, where 1 means predicting zero. Delay-SSD with p=τ stays at or below 0.014 for every τ from 8 to 1024. Mamba-3 reaches 0.23 at τ=8, 0.78 at 16 and 0.91 or above from 32. Gated DeltaNet and SSD are at 0.83 or above for every τ.
Routing tasks (Section 3.2): a 16 × 64 token grid scanned row by row (W=64), without convolution or position encoding; best eval accuracy. Copy-above outputs the token directly above. Row-recall returns the value paired with a key in the row above. Two-stream recall returns the value of the most recent occurrence of a key in the current row or the row above.