acceptodds
Under review as a conference paper at ICLR 2027

Delay-SSD: Reading Past the Decay with Lagged State Updates

Abstract

Structured state space models such as Mamba-2 forget under one scalar decay per head, so a token's influence is fixed by its distance in the scan. A dependency at a fixed lag , such as the pixel above in a flattened image or the period of a seasonal series, must therefore be carried through decay steps and competes for state with every token in between. We introduce , which writes the input from steps earlier directly into the SSD state update, so that the lagged token is read at decay distance zero. All lags share one state and one decay, so the state size is unchanged and Mamba-2's chunked scan still applies, because the sequence transformation stays semiseparable of order outside a band of width . We show that neither more state nor another scan order buys this reach. With input-independent parameters, scalar-decay heads cannot isolate a lag beyond at any state size, input-dependent heads reading independent tokens are bound the same way on average, and no input-independent linear recurrence with fewer than states approximates a -step delay, whereas a single lag does both. On synthetic routing tasks, a scan order changes which offsets one SSD layer reaches but not how many, while one layer also reaches the rows above and below. With our TileLang kernels, two lags cost – the device time of the four-scan SSD mixer, compared with about for a second layer or a larger state. As a vision backbone, matches Mamba2D on ImageNet-1k ( top-1 at 50M parameters) and COCO, and leads it on ADE20K segmentation at every size (by , and mIoU). In forecasting, with only the mixer changed, lags lower the error on each of five traffic datasets, by relative to a no-lag control and by relative to Mamba-2.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
state space models, structured state space duality, Mamba, lagged connections, efficient sequence modeling, long-range dependencies, time series forecasting, vision backbones

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ENdTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Delay-SSD, a modification of the SSD token mixer. The paper adds fixed-lag inputs (e.g., the row above in a flattened image) to the Mamba-2 state update, sharing one state and decay so the chunked scan is preserved. Experiments on classification, detection, segmentation and traffic forecasting demonstrates its effectiveness.

Critical strengths
  • The mechanism is simple and well-motivated.

  • The paper is easy-to-follow.

Critical weaknesses
  • The paper misses a series of important and influential baselines. Spatial-Mamba is cited but not included in Table 1, although it is the closest functional competitor: it aggregates states over a spatial neighborhood; Backbones with explicit 2-D spatial decay or linear attention, such as RMT and MLLA, address the same scan-distance bias and are neither compared nor discussed; Since DSSD uses global attention in stages 3–4, it is a hybrid model instead of pure mamba model. It should be compared more systematically against hybrid backbones such as TransNeXt, not only MambaVision; It also includes no comparison against recent mixers such as Mamba-3 or Gated DeltaNet, even though the fused kernel is adapted from Mamba-3.
Questions for authors
  • How does Delay-SSD compare to an input-side stride-WW token shift followed by a standard SSD, at matched parameters and cost, on the ImageNet subset?

  • How does it compare to VMamba2/DSSD-without-lags with 5×5 or 7×7 depthwise convolutions at matched cost on the ImageNet subset?

  • Can you report ImageNet results for Spatial-Mamba, RMT and MLLA alongside yours, and add FLOPs- and throughput-matched versions of Table 1?

3 comments
AuthorsOpenReview ↗

(1) Our vision claims concern the lag mechanism and are made relative to Mamba2D, not state of the art across architectures. Spatial-Mamba, RMT, MILA and TransNeXt added to tables, additions reproduced below. (2) On our synthetic tasks, Mamba-3 and Gated DeltaNet share the limitation we address. (3) At equal parameters on the ImageNet subset, the lags beat an input-side shift in every seed, and a wider convolution stays within the seed spread of the no-lag model.

Part 1/2

We thank the reviewer for the thorough review and the baseline reccomendations.

Published results: ImageNet-1k top-1 at 224 px; ADE20K single-scale mIoU with UperNet (160k iterations), FLOPs at 512×2048. Rows are grouped by ImageNet FLOPs, so DSSD-S sits with the -B models. RMT is without token labeling. MILA and Spatial-Mamba train with MESA, which DSSD does not use; without it MILA reports 83.3% (T) and 85.0% (B).

ModelParamsFLOPsIN-1kADE20KUperNet FLOPs
DSSD-T27M5.0G83.950.71042G
Spatial-Mamba-T27M4.5G83.548.6936G
MILA-T25M4.2G83.5––
RMT-S27M4.5G84.149.8937G
TransNeXt-T28M5.7G84.051.1–
DSSD-S50M14.1G85.352.21315G
Spatial-Mamba-B96M15.8G85.351.81176G
MILA-B96M16.2G85.351.91183G
RMT-B54M9.7G85.052.01051G
TransNeXt-S50M10.3G84.752.2–

W1. Spatial-Mamba. DSSD leads on ADE20K at both sizes (50.7 against 48.6, 52.2 against 51.8), at 11–12% higher UperNet FLOPs in part because its stage 3–4 global attention is quadratic in the token count (Table F.1). On ImageNet-1k, DSSD-T leads by 0.4 points, and DSSD-S matches Spatial-Mamba-B with about half the parameters. The mechanisms also differ. Spatial-Mamba runs the scan first and then mixes each token's state with its neighbours' states through dilated depthwise filters before the read-out, so later scan steps never see the fused states. Delay-SSD writes the lagged token into the recurrent state during the scan, so every later token may access it through the state.

W2. RMT and MLLA. Our vision claims are made relative to Mamba2D, whose macro-architecture and recipe DSSD shares: parity on ImageNet-1k and COCO, and an ADE20K gain that the lag-on/lag-off pair attributes to the lags. We do not claim state of the art across architectures. RMT and MILA (the current name of MLLA) address the mismatch between scan distance and image distance without a scan, RMT with a 2-D Manhattan-distance decay inside softmax attention and MILA with linear attention in every stage. We add both to Related Work, to Table I.2, which already lists attention and convolutional backbones, and to the new appendix table. Table 1 keeps its scope, SSM-based and SSM-hybrid backbones with Swin as the customary reference, and its caption clarifies this. We are open to adding these to Table 1 space permitting. On ImageNet-1k, RMT-S leads DSSD-T by 0.2 points and DSSD-S leads RMT-B by 0.3 at 45% more FLOPs. On ADE20K, DSSD leads RMT by 0.9 (T) and 0.2 (B) mIoU and MILA-B by 0.3, at higher UperNet FLOPs.

W3. Hybrid backbones such as TransNeXt. We agree that DSSD is a hybrid, and the abstract and Section 4 now clarify that selective-scan mixers in stages 1–2 and global attention in stages 3–4, as in Mamba2D. Table 1 already compares it with the two closest Mamba hybrids, Mamba2D and MambaVision. TransNeXt uses attention in every stage and no SSM, so it is categorized with RMT and MILA as linear attention architectures. It leads DSSD-T by 0.4 mIoU and 0.1 points and ties DSSD-S on ADE20K. Section 4.2's "the best backbone in every size group" on ADE20K is updated to "the best SSM-based backbone", and the abstract's T-size margin over Mamba2D uses the run under Mamba2D's released schedule (1.55 mIoU; see response to Reviewer iuXR, W4).

W4. Mamba-3 and Gated DeltaNet. Our theorems and claims concern scalar-decay SSD and cover neither update, so we measured both. Mamba-3's two-term update combines the current and the previous token, a lag of 1; Delay-SSD writes a token pp steps back, and with p=Wp = W, the row length, the pixel above enters the state undecayed. Setup: single layers on the synthetic tasks of Section 3, state size 16, public implementations at library defaults (mamba-ssm; fla, without Gated DeltaNet's short convolution), a learning rate chosen per mixer, 3 seeds.

Delay task (Section 3.3): output the input delayed by τ\tau steps; relative RMSE, where 1 means predicting zero. Delay-SSD with p=τp = \tau stays at or below 0.014 for every τ\tau from 8 to 1024. Mamba-3 reaches 0.23 at τ=8\tau = 8, 0.78 at 16 and 0.91 or above from 32. Gated DeltaNet and SSD are at 0.83 or above for every τ\tau.

Routing tasks (Section 3.2): a 16 × 64 token grid scanned row by row (W=64W = 64), without convolution or position encoding; best eval accuracy. Copy-above outputs the token directly above. Row-recall returns the value paired with a key in the row above. Two-stream recall returns the value of the most recent occurrence of a key in the current row or the row above.

TaskDelay-SSDMamba-3Gated DeltaNetSSDChance
copy-above100%35%19%13%13%
row-recall100%69%43%25%25%
two-stream recall98%57%51%49%6%

Part 2/2

W4. Continued In two-stream recall, 46% of answers lie in the current row, which a scan reaches without a lag; this accounts for SSD's 49%. At state size 64 or 128, Mamba-3 gains at most 8 points on any task. These results go into Appendices D and E. The mechanisms are complementary: a lag can be added to Mamba-3's update directly, though we leave this as future work.

Q1 and Q2 share the protocol of Table G.1, to which these rows are added: DSSD-N on the 100-class ImageNet subset, 100 epochs, parameters held equal through the FFN width, mean ±\pm standard deviation over 3 seeds.

VariantTop-1
No lags, 3×3 conv74.10±0.0774.10 \pm 0.07
No lags, input-side shift74.04±0.4074.04 \pm 0.40
No lags, 5×5 conv74.26±0.3774.26 \pm 0.37
No lags, 7×7 conv73.01±0.2473.01 \pm 0.24
Lags, 3×3 conv74.62±0.0774.62 \pm 0.07

W4. Mamba-3 and Gated DeltaNet. Our theorems and claims concern scalar-decay SSD and cover neither update, so we measured both. Mamba-3's two-term update combines the current and the previous token, a lag of 1; Delay-SSD writes a token pp steps back, and with p=Wp = W, the row length, the pixel above enters the state undecayed. Setup: single layers on the synthetic tasks of Section 3, state size 16, public implementations at library defaults (mamba-ssm; fla, without Gated DeltaNet's short convolution), a learning rate chosen per mixer, 3 seeds.

Delay task (Section 3.3): output the input delayed by τ\tau steps; relative RMSE, where 1 means predicting zero. Delay-SSD with p=τp = \tau stays at or below 0.014 for every τ\tau from 8 to 1024. Mamba-3 reaches 0.23 at τ=8\tau = 8, 0.78 at 16 and 0.91 or above from 32. Gated DeltaNet and SSD are at 0.83 or above for every τ\tau.

Routing tasks (Section 3.2): a 16 × 64 token grid scanned row by row (W=64W = 64), without convolution or position encoding; best eval accuracy. Copy-above outputs the token directly above. Row-recall returns the value paired with a key in the row above. Two-stream recall returns the value of the most recent occurrence of a key in the current row or the row above.

TaskDelay-SSDMamba-3Gated DeltaNetSSDChance
copy-above100%35%19%13%13%
row-recall100%69%43%25%25%
two-stream recall98%57%51%49%6%

In two-stream recall, 46% of answers lie in the current row, which a scan reaches without a lag; this accounts for SSD's 49%. At state size 64 or 128, Mamba-3 gains at most 8 points on any task. These results go into Appendices D and E. The mechanisms are complementary: a lag can be added to Mamba-3's update directly.

Q1 and Q2 share the protocol of Table G.1, to which these rows are added: DSSD-N on the 100-class ImageNet subset, 100 epochs, parameters held equal through the FFN width, mean ± standard deviation over 3 seeds.

VariantTop-1
No lags, 3×3 conv74.10±0.0774.10 \pm 0.07
No lags, input-side shift74.04±0.4074.04 \pm 0.40
No lags, 5×5 conv74.26±0.3774.26 \pm 0.37
No lags, 7×7 conv73.01±0.2473.01 \pm 0.24
Lags, 3×3 conv74.62±0.0774.62 \pm 0.07

Q1. Input-side stride-WW shift before a standard SSD. The shift is indistinguishable from the no-lag model, and the lags lead it in every seed, by 0.58 on average (paired one-sided tt-test, p=0.04p = 0.04).

Q2. Wider convolutions without lags. We widen the 3×3 depthwise convolution of every stage 1–2 block. The 5×5 model is within the seed spread of the no-lag model, and the lags lead it in all three seeds (by 0.02 to 0.67). The 7×7 model loses about a point in every seed at the same configuration.

Both alternatives train at the no-lag model's speed (about 1,470–1,480 images/s) and reach its accuracy, so the equal-time comparison is the one already in Table G.1: the no-lag model with the lags' time spent on extra stage 1–2 blocks reaches 74.15±0.5574.15 \pm 0.55, a gap to the lags that is within its seed spread.

Q3. FLOPs- and throughput-matched views. The table in part 1 gives parameters and FLOPs for the requested models. We claim no FLOPs or throughput advantage at the model level. Table F.5 and Figure F.3 already report accuracy against inference throughput for the comparators that run on our stack, and DSSD is the slowest at each size: DSSD-T runs at 2,890 images/s against 4,077 for VMamba-T, and DSSD-S at 1,063 against 1,850. We have not timed Spatial-Mamba, RMT or MILA and claim no advantage over them. Our cost claim is at the mixer level: two lags take 1.36–1.48× the time of the four-scan SSD mixer, against about 2× for a second SSD layer (Section 3.4).

Thanks, I think this resolves my concerns. I would like to raise the score to 3.

Reviewer_AuAcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces Delay-SSD, a modification of the Mamba-2 (SSD) token mixer that writes inputs from a fixed set of lags directly into the state update. The key design choice is that all lags share one state and one decay, keeping the state size unchanged and preserving Mamba-2's chunked scan structure. The authors provide theoretical arguments that lags buy reach that neither more state nor more scan orders can, and evaluate on ImageNet-1k, COCO, ADE20K, and long-term forecasting.

Critical strengths
  • Extensive ablation studies: The paper conducts ablation studies from various perspectives, going beyond the proposed components alone, making the results informative.

  • Clear and intuitive idea: Using delays to construct inputs that explicitly avoid decay is a clear and intuitive idea. The results on the toy task in Fig. 3 provide a proof-of-principle validation of this mechanism.

Critical weaknesses
  • Unclear effectiveness on real-world tasks: The proposed method does not demonstrate a clear advantage over existing methods on ImageNet-1K. I am also concerned that the performance improvements reported in the ablation studies fall within the standard error. Furthermore, on COCO and ADE20K, the proposed method performs inconsistently compared with VSSD and M2D, which are used as baselines, sometimes outperforming them and sometimes underperforming them.

  • Novelty: Introducing delays into Mamba-2 appears to be a novel contribution. However, SSM-based methods that incorporate delays, such as H3 [1] and Hyena [2], already exist. I believe the paper needs to discuss its novelty more thoroughly in relation to these prior works.

[1] Hungry hungry hippos: Towards language modeling with state space models, ICLR2023.
[2] Hyena Hierarchy: Towards Larger Convolutional Language Models, ICML2023.

Questions for authors
  • For ADE20K, can the authors report lag-on vs lag-off results under exactly the same DSSD macro-architecture with FLOPs information? The existing cross-model comparison is difficult to attribute specifically to the lag mechanism because the backbone uses global attention in stages 3–4 and has substantially different FLOPs characteristics.
  • The synthetic routing advantage is dramatic, whereas the gain in real world problem is small. Can the authors quantify how much of the proposed positional-routing limitation remains after the 3×3 depthwise convolution and multiple stacked blocks used in the actual vision architecture?
2 comments
AuthorsOpenReview ↗

(1) We claim parity with Mamba2D on ImageNet-1k, not an advantage. (2) The ablation gain is points as a paired difference over five seeds, positive in every seed. (3) A full-scale pair that differs only in the lags shows they add 1.25 mIoU on ADE20K (Q1).

We thank the reviewer for the careful review and precise questions.

W1. Effectiveness on real-world tasks.

ImageNet-1k. DSSD-N/T/S reach 79.9 / 83.9 / 85.3% against 79.6 / 84.0 / 85.3% for Mamba2D, which we report as parity. Ablation and standard error. Over five seeds the means are 74.77±0.3274.77 \pm 0.32 with lags and 74.29±0.3474.29 \pm 0.34 without. Most of each spread is seed variation shared by both arms, so the relevant statistic is the paired difference, 0.48±0.120.48 \pm 0.12 points, positive in all five seeds (one-sided tt-test, p=0.0004p = 0.0004). The throughput-matched gap (0.46 points, three seeds) is within its spread; the revision says so and adds the five-seed rows to Table G.1.

COCO and ADE20K. On COCO box AP (1×1\times / 3×3\times), DSSD-T is within 0.2 AP of Mamba2D-T and 1.5–1.7 above VSSD-T. DSSD-N is 1.1 above Mamba2D-N at 1×1\times and 0.2 below at 3×3\times, and below VSSD-M, a backbone with twice its parameters. On ADE20K, DSSD leads VSSD at both sizes (46.1 against 45.6, 50.7 against 47.9) and Mamba2D at every size. Trained under Mamba2D's released ADE20K configuration, DSSD-T reaches 50.45 mIoU against 48.9 for Mamba2D-T (Reviewer iuXR, W4). Cross-model gaps mix several differences, so the direct evidence that the lags help is the pair in Q1.

W2. Novelty relative to H3 and Hyena.

We agree, and the revision adds this discussion to Related Work. Both address a related problem, reaching distant tokens below quadratic cost, in architectures designed for language, so we treat them as related work, not baselines.

  • H3. Its shift SSM holds the last mm inputs and filters them with input-independent weights; reaching lag τ\tau takes m>τm > \tau states per channel.
  • Hyena. Its implicit filters are as long as the input, so they reach any lag, but keep no fixed-size recurrent state.
  • Delay-SSD. The lagged token is written into the selective SSD state with its own input-dependent key and step size, under one shared decay (Eqs. 2–4), with the state size and Mamba-2's chunked scan unchanged (Proposition 1).

Delayed inputs to recurrences are an old idea, our contribution is their formulation inside the selective SSD update.

Q1. ADE20K lag-on against lag-off, same macro-architecture, with FLOPs.

DSSD-T (26.55M params)mIoU SSmIoU MSBackbone FLOPsTotal FLOPs
with lags50.751.2199G1042G
without lags49.4550.16189G1032G

Depths, widths, attention stages and recipe are identical, and a slightly larger FFN in the no-lag to keep the parameter count eqal. The lags add 1.25 mIoU single-scale and 1.04 multi-scale at 5% more backbone FLOPs (single runs, 512×2048512\times2048). The FLOPs gap to Mamba2D is a counting convention, since its published totals omit the decode head (Reviewer iuXR, W4).

Q2. How much of the routing limitation survives the convolution and depth? Q1 measures it directly, with the convolution and every block in place: 1.25 mIoU on ADE20K, and 0.18 points on ImageNet-1k, which is parity. The routing tasks agree (Appendix D.2, Table D.4). With the convolution, one SSD layer reaches 93.3±3.5%93.3 \pm 3.5\% on sources two rows away against 100% with lags; a 5×55\times5 convolution or a second layer also closes that gap. The lags still shift the trained stage-1 receptive field toward the adjacent rows (row ratio 0.88 to 1.07, Figure C.3).

Reviewer_AuAcOpenReview ↗

Reply to Author

Thank you for the additional experiments and evaluations. The ablation study on downstream tasks provides convincing evidence of the effectiveness of the proposed components. However, I remain somewhat concerned that the proposed method does not demonstrate a clear performance improvement over the baseline on ImageNet-1K. Given this concern, I will maintain my current score.

Reviewer_iuXROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Delay-SSD, which is a method based on Mamba-2 + a small set of fixed lags P, resulting in a hybrid vision backbone. The input from p steps back is written directly into the shared state, under one shared decay. The paper also argues that a lag provides positional reach that more state, more heads, or other scan orders cannot have. The proposed method is a hybrid vision backbone (SSD in stages 1–2, global attention in stages 3–4); it matches Mamba2D on ImageNet and COCO and has better scores on ADE20K. In forecasting, it has a lower MSE by 1.4–2.2% than Mamba-2.

Critical strengths
  1. The main idea of the paper is simple and well motivated in the domain of state-space models; it injects the token at a fixed lag at decay distance zero.

  2. The cost analysis in H.4 for COST OF THE TIME-TOKEN MODELS is interesting. The authors go beyond analytic FLOPs and measure what each design choice costs in practice.

  3. The synthetic routing tasks are cleanly designed, as they show one Delay-SSD layer solving tasks that one SSD layer cannot, at matched or lower FLOPs.

Critical weaknesses
  1. The major issue is that the improvements in vision tasks are very minor, and in the real vision model, the lags add little. The theory and routing tasks deliberately exclude convolutions and positional embeddings. The vision blocks, however, include a 3×3 depthwise convolution that already gives every token direct access to the row above and below.

  2. The forecasting gains are measured inside a weak backbone. All comparisons use a time-token variant built for this paper, and the published S-Mamba is much better on the same settings (Table H.7). For example, on Traffic its MSE is 0.385 versus 0.598 for T-DSSD, and on PEMS03 it is 0.065 versus 0.084. In Table H.7, I don't understand why the bolding marks the best time-token model only, and S-Mamba is excluded from ranking.

  3. Important baselines are missing. The closest prior mechanisms for reading a token at a fixed offset should be compared: the shift SSM in H3 [1]. So should the simplest alternative, a dilated short convolution with a tap at W before the SSD layer.

  4. The main results for vision tasks are single runs compared against numbers copied official paper numbers, each trained with its own recipe; I feel it's not apple-to-apple comparion at all. The ADE20K gains over Mamba2D are the paper's clearest vision result. However, the DSSD backbones use far more FLOPs at 512×2048 (1042G vs. 485G for the T size), because stages 3–4 use global attention.

  5. The paper is extremely dense, at 51 pages. Important caveats appear only in the appendix: single-seed vision runs, the hybrid attention stages, the gap to S-Mamba, and the drop on ETT and Weather. I would recommend moving this stuff to the main paper.

[1] D. Y. Fu, T. Dao, K. K. Saab, A. W. Thomas, A. Rudra, and C. Ré. Hungry Hungry Hippos: Towards Language Modeling with State Space Models. In ICLR, 2023.

Questions for authors
  1. Can we have a no-lag DSSD baseline on ImageNet and ADE20K under the same recipe, and ideally Mamba2D re-run under that recipe? This is the apple-to-apple comparison between the baseline, lag, and the proposed method.
3 comments
AuthorsOpenReview ↗

(1) A full-scale pair that differs only in the lags: removing them from DSSD-T costs 1.25 mIoU on ADE20K, with ImageNet-1k at parity (Q1). (2) Over five seeds of the controlled ablation, the lags add points, positive in every seed. (3) Trained under Mamba2D's released ADE20K configuration, DSSD-T still leads Mamba2D-T by 1.55 mIoU (W4). (4) Mamba2D's published ADE20K FLOPs omit the decode head; counted the same way, DSSD costs 1–2% more.

Part 1/2.

We thank the reviewer for the detailed review.

W1. Small vision gains; the 3×33\times3 convolution already reaches the adjacent rows. The convolution does reach them, and on ImageNet-1k we claim parity with Mamba2D, not a gain: the full-scale lag-on/lag-off difference there (0.18 points, single runs) is parity too. What the lags add on top of the convolution shows in two places. In the seeded ablation, with the convolution in both arms, they add 0.48±0.120.48 \pm 0.12 points over five seeds (paired difference, positive in every seed; one-sided tt-test, p=0.0004p = 0.0004). At full scale on ADE20K they add 1.25 mIoU (Q1). Mamba2D's own mixer ablation shows the same pattern: removing either path of its mixer changes ImageNet-1k top-1 by at most 0.2 points but ADE20K by up to 1.3 mIoU (Mamba2D, Table 7).

W2. Weak forecasting backbone; Table H.7 bolding. The reviewer's numbers are correct, and Section 4.3 is a controlled test of the mixer, not a state-of-the-art claim. A lag must act along time, and S-Mamba's tokens are variates, so the time-token model makes each time step a token and keeps S-Mamba's encoder; only the mixer changes. Delay-SSD lowers MSE by 2.2% against the no-lag control (18 of 20 settings, p<0.001p < 0.001) and by 1.4% against T-Mamba-2 (18 of 20, p=0.004p = 0.004), also at equal size and in a causal backbone (Table H.6). Table H.7 now ranks S-Mamba with all models.

W3. H3's shift SSM and a dilated convolution with a tap at WW. Dilated convolution. It is already in every model in the form of the 3×33\times3 depthwise convolution. Since it acts on the 2-D grid, its taps include ±W\pm W, and the no-lag rows of Table G.1 are this baseline. H3. H3's shift SSM is one half of a gated pair (shift SSM and diagonal SSM) with its own macro-architecture, so inserting it would not isolate the lag. Its closest analogue in our backbone is a fixed input-side shift by ±W\pm W, which we trained at equal parameters (three seeds): 74.04±0.4074.04 \pm 0.40, against 74.62±0.0774.62 \pm 0.07 with lags and 74.10±0.0774.10 \pm 0.07 without. It is indistinguishable from the no-lag model and below the lags in every seed. Both mechanisms hold the lagged input; they differ in its use. A shift SSM filters its last m≥W+1m \ge W + 1 inputs per channel with input-independent weights, whereas Delay-SSD writes it into the selective state with its own input-dependent key and step size (Eqs. 2–4). The revision adds the shift row to Table G.1, and H3 and Hyena to Related Work.

W4. Recipes, single runs and FLOPs.

Single runs. Limitations states that the full-scale results are single runs; ImageNet-1k pretraining of DSSD-T alone takes about 260 GPU-hours. A rerun on other hardware reached 83.96% against the reported 83.93%. Our claim about the lags rests on the lag-on/lag-off pair (Q1) and the five-seed ablation, not on cross-paper margins.

Recipe. On ImageNet-1k and COCO our models use Mamba2D's released code and configurations with the stage 1–2 mixer replaced (DSSD-N and DSSD-S add gradient clipping). On ADE20K the concern applies. Both use the 160k-iteration UperNet schedule, but Mamba2D's released configuration reaches batch 16 through 2-step accumulation, which halves the images seen relative to our DSSD-T run; our DSSD-N run already matches that schedule. Without its training logs we cannot tell whether its published numbers used that configuration, so we trained DSSD-T under it: it reaches 50.45 / 50.90 mIoU (single / multi-scale), against 48.9 / 49.3 for Mamba2D-T, a lead of 1.55 / 1.6 mIoU against the 1.8 / 1.9 in the paper; the revision reports this run.

FLOPs. Mamba2D also uses global attention in stages 3–4, so attention does not explain the gap; our Table I.2 caption makes that point about VMamba and Swin, and we clarify it. We use Mamba2D's published T-size total FLOPs (485G) which is below the cost of the UperNet decode head alone (843G at 512×2048512\times2048), so it omits the head. When counted the same way, Mamba2D-T costs 1029G and DSSD-T 1042G; across sizes DSSD costs 1–2% more in total and 6–11% more in the backbone. The same count reproduces VMamba-T's published 948.8G. Table I.2 now reports backbone and total FLOPs for DSSD and Mamba2D.

W5. Density and caveats. Agreed. Section 4 now states, where the results are reported, that the full-scale results are single runs, that the backbone is a hybrid, that the time-token models trail S-Mamba, and that on ETT and Weather the lagged model is 4.1% worse than the no-lag control over nine settings. We also add an appendix roadmap from each claim to its evidence.

Q1. No-lag DSSD on ImageNet-1k and ADE20K.

DSSD-T (26.55M params)ImageNet-1kADE20K mIoU (SS / MS)Backbone FLOPs
with lags83.93%50.7 / 51.2199G
without lags83.75%49.45 / 50.16189G

Only the lags differ with a slightly larger FFN in the no-lag arm to keep equal parameter count and both are single runs. The lags add 1.25 mIoU at 5% more backbone FLOPs. W4 reports the reverse of the requested Mamba2D rerun: DSSD-T under Mamba2D's ADE20K configuration.

Reviewer_iuXROpenReview ↗

Reply to authors rebuttal

Thank you for the rebuttal and answering my question about the Apple-to-Apple comparison, this partially solved some of the concerns I raised.

I still believe that the improvements are very incremental and the lags add little. The authors provided SD so it’s not in the noise range, but it’s a very minor improvement.

Please update the revised version of the paper with these changes and let me know and I will check it again.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment and Revision Formatting

We thank all three reviewers for their careful reviews.

Revision Formatting

New and changed text, rows and captions are in blue. To address the length, we removed appendix material that no claim relies on and condensed the forecasting and cost tables, so the Appendix loses 6 pages. Of the remaining Appendix length, ~6/30 pages come from white space from large tables and figures. Further advice on sections reviewers found unhelpful or excessive would be appreciated. Each removed section, table or figure remains as a gray, struck-through heading or caption marked "Removed in the revision", so every section, table and figure keeps its original number and the references in our responses and the reviews match both versions. New tables and subsections are numbered after the existing ones in their appendix (Tables D.5, E.2, I.4 and I.5; Appendices D.3 and I.6), and a roadmap before Appendix A maps each claim to its evidence.

Scope of our claims

Delay-SSD changes the SSD state update. Our vision claims are made relative to Mamba2D, whose macro-architecture and training code we share: parity on ImageNet-1k and COCO, and a gain on ADE20K which is the exact case we expect to see a gain - high resolution images. In these dense tasks, We do not claim state of the art across architectures. The revision states this scope in Section 4 and limits Section 4.2's ADE20K statement to SSM-based backbones.

New evidence

  • Full-scale lag-on/lag-off pair (iuXR Q1, AuAc Q1). DSSD-T retrained without lags under the same recipe, at the same parameter count (26.55M): ImageNet-1k 83.75% against 83.93%, which is parity, and ADE20K 49.45 / 50.16 against 50.7 / 51.2 mIoU (single / multi-scale), at 189G against 199G backbone FLOPs.
  • Five-seed ablation (AuAc W1, iuXR W1). On the 100-class ImageNet subset, the lags add 0.48±0.120.48 \pm 0.12 points as a paired difference, positive in all five seeds (one-sided tt-test, p=0.0004p = 0.0004). Controls at equal parameters (ENdT Q1/Q2, iuXR W3). An input-side shift by ±W\pm W reaches 74.04±0.4074.04 \pm 0.40 and a 5×55\times5 convolution 74.26±0.3774.26 \pm 0.37, against 74.10±0.0774.10 \pm 0.07 without lags and 74.62±0.0774.62 \pm 0.07 with lags; a 7×77\times7 convolution reaches 73.01±0.2473.01 \pm 0.24.
  • Equal ADE20K schedule (iuXR W4, AuAc W1). Under Mamba2D's presumed ADE20K configuration (release code does not include logs or full configuration files), DSSD-T reaches 50.45 / 50.90 mIoU against 48.9 / 49.3 for Mamba2D-T.
  • Mamba-3 and Gated DeltaNet (ENdT W4). As single layers on the synthetic tasks of Section 3, neither recovers a delay beyond a few steps or copies the row above when rows hold 64 tokens (35% and 19%, against 100% with one lag). FLOPs under one convention (iuXR W4). Counted as ours, Mamba2D-T costs 1029G against 1042G for DSSD-T; its published 485G omits the UperNet decode head.

Revisions

Baselines/References: Spatial-Mamba in Table 1 and Table I.2; RMT, MILA and TransNeXt in Table I.2 and a new appendix table of non-SSM backbones; H3, Hyena, RMT, MILA and Spatial-Mamba in Related Work.

Tables: Table G.1 with the five-seed and control rows; Table I.2 with the equal-schedule run and backbone FLOPs; Table H.7 ranks S-Mamba with all models.

Caveats in the main text: the full-scale results are single runs, the backbone is a hybrid with global attention in stages 3–4, the time-token models trail S-Mamba, and the lags hurt on ETT and Weather.

Appendices D and E add the Mamba-3 and Gated DeltaNet results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.