acceptodds
Withdrawn from ICLR 2027

CASSM: Content-Aware State Space Modeling for Ultra-High-Definition Low-Light Image Enhancement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Mamba-based architectures have shown promising results in low-light image enhancement by modeling visual tokens through selective, linear-complexity scanning. However, most existing approaches rely on predefined spatial scanning strategies that determine token order mainly based on spatial locations, ignoring content similarity among visual tokens. Consequently, content-similar yet spatially distant tokens may be placed far apart in the scanning sequence, limiting effective long-range dependency modeling, particularly for ultra-high-definition (UHD) images with millions of pixels. To address this limitation, we propose a Content-Aware State Space Model (CASSM) for UHD low-light image enhancement, which constructs content-aware visual sequences with linear complexity. Specifically, CASSM leverages a learned low-frequency codebook and a lightweight feature alignment adapter to efficiently generate a content-aware assignment map. Guided by these assignments, the Content-Aware Mamba Block performs token-level content grouping to gather content-similar tokens, followed by group-level reordering using 1D t-SNE projections of the codebook entries. This hierarchical organization brings spatially distant yet visually similar tokens closer in the resulting sequence, enabling more effective state-space modeling. Extensive experiments demonstrate that CASSM achieves an average PSNR improvement of 0.75 dB over the state-of-the-art methods across four benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Low-level vision, Low-light image enhancement, Ultra-High-Definition image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_QLHaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CASSM for UHD low-light image enhancement. The main idea is to replace the usual spatial scanning of visual state-space models with a content-aware sequence. A codebook is learned from low-frequency components of clean images. Restoration features are then assigned to codebook entries, tokens with the same assignment are grouped together, and the groups are ordered using a 1D t-SNE embedding before being processed by an SSM. The model is built on the C²SSM encoder-decoder architecture. The paper reports strong results on UHD low-light benchmarks and also evaluates generalization to standard-resolution low-light datasets. Ablations are provided for the proposed components and sequence construction.

Critical strengths

The main strength is the empirical performance. CASSM performs well on the reported UHD low-light benchmarks and also shows reasonable cross-dataset generalization. The idea of keeping all tokens while changing their serialization is also simple and potentially useful. I also appreciate that the paper includes C²SSM as a strong and closely related baseline and uses its architecture as the backbone. This makes the empirical comparison more meaningful than comparisons across unrelated architectures.

Critical weaknesses

My main concern is the motivation of the method. The paper argues that spatial scanning is suboptimal because visually similar tokens can be far apart in the sequence. I can understand this as a general argument about visual SSMs, but I do not see why it is specifically important for low-light enhancement. What property of low-light degradation makes codebook-based content grouping the appropriate solution?

In particular, low-light restoration depends strongly on local structures, illumination transitions, edges, noise, and spatially varying degradation. CASSM deliberately brings spatially distant tokens together based on feature similarity. This may improve content-level interaction, but it also breaks spatial continuity. The paper does not really study this trade-off. Why should content similarity be more important than spatial locality for this task, and under what conditions does this hold?

I have a related concern about the codebook. It is learned from the low-frequency components of clean images and is then used to define which restoration tokens should be close in the Mamba sequence. Why does similarity to this clean-image codebook correspond to the dependencies that are useful for correcting low-light degradation? At present this connection seems assumed rather than demonstrated. A comparison with codebooks learned from the task data in different ways, or an analysis of what the resulting groups actually represent, would help.

The novelty relative to C²SSM also needs to be stated more carefully. C²SSM already argues that pixel-wise spatial scanning is not ideal for UHD restoration and that content/cluster structure should be used for SSM modeling. CASSM is technically different: it keeps all tokens rather than compressing them into centroids and further introduces group ordering. I think this is a valid distinction, but it makes CASSM look more like a fine-grained extension of cluster-centric SSM than a new content-aware SSM paradigm. Could the authors clarify what new conclusion CASSM establishes beyond the main observation already made by C²SSM?

This also leads to an experimental issue. The comparison with C²SSM changes centroid compression, token grouping, the codebook, feature adaptation, and group ordering at the same time. A controlled experiment such as spatial all-token SSM → centroid-level SSM → grouped all-token SSM → grouped all-token SSM with the proposed ordering would be much more informative.

I am particularly unconvinced by the use of 1D t-SNE to order the groups. Why should a one-dimensional t-SNE embedding define a good state propagation order for Mamba? The current comparison does not rule out much simpler alternatives. I would like to see comparisons with, for example, PCA-based ordering, nearest-neighbour traversal in the codebook space, or a learned ordering. More directly, the authors could measure whether the proposed serialization actually reduces the sequence distance between semantically related tokens and whether this reduction correlates with restoration quality.

The current ablation is also somewhat revealing: content grouping alone gives only a small gain, while group ordering contributes more. This makes the ordering mechanism quite central to the claimed contribution, but it is currently one of the least justified parts of the method. I think this deserves substantially more analysis.

Besides, the codebook is pretrained using clean images from several UHD restoration datasets rather than only the training set of the target low-light benchmark. This is not data leakage, but it gives CASSM an additional clean-image prior that the compared methods may not have. How much of the reported generalization, especially the cross-dataset gain, remains if the codebook is learned only from the corresponding training dataset?

Questions for authors

See weaknesses

Reviewer_gEnkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CASSM introduces a "content-dependent sequence construction" mechanism within the Layer 3 decoder of the C2SSM backbone network. This mechanism leverages a fixed VQ-VAE codebook—learned from clear LL (low-low) wavelet subbands—to provide 1,024 anchors, each with 128 dimensions. Features are generated via a Convolutional Feature Alignment Adapter (FAA) and assigned to these anchors based on cosine similarity. During the grouping process, the raster-scan order within each group is preserved; these groups are subsequently concatenated according to a precomputed 1D t-SNE ordering. Following this, the sequence undergoes selective state-space processing, after which the spatial order is restored and residual fusion is performed (pp. 4–6, Equations 6–14; Appendix A.2, pp. 13–15). The experimental results presented in the paper cover two Ultra-High-Definition (UHD) benchmarks, the LOL-v1 dataset, and a transfer task from LOL-v1 to LOL-v2-Real (p. 7, Tables 1–3, and Section 4.1).

Critical strengths
  • The sequence-construction mechanism is concrete and understandable. Assignment costs O(NK) for fixed codebook size K, and the t-SNE permutation is computed offline; Appendix A.1 specifies normalization, perplexity, iterations, and seed (pp. 4, 6, 13). This is a plausible way to connect spatially distant features without all-pairs token matching.
  • The reported PSNR results are competitive: improvements over the strongest listed PSNR baseline are 1.32, 0.08, 0.24, and 1.34 dB on UHD-LOL4K, UHD-LL, LOL-v1, and LOL-v2-Real, respectively (p. 7, Tables 1–3). On UHD-LL, the small PSNR margin over Wave-Mamba accompanies an SSIM increase from 0.913 to 0.931.
  • There is useful controlled evidence at the stated 75K budget. Table 4 improves from 38.51 to 38.99 dB; Table 5 compares spatial ordering, index-ordered groups, and t-SNE-ordered groups at 38.68, 38.76, and 38.99 dB. Table 6 also compares alternative Mamba modules at the same insertion point (pp. 8–9, §4.3). These experiments support the design, even though some attribution questions remain.
Critical weaknesses
  1. The VQ-VAE pretraining configuration needs clarification. Section 4.1 states that both training stages use AdamW with an initial learning rate of 5×10⁻⁴, whereas Appendix A.2 specifies Adam with a constant learning rate of 1×10⁻⁴ for the VQ-VAE encoder and decoder (p. 7, lines 363–364; p. 15, lines 776–778). As Section 4.1 identifies codebook pretraining and LLIE training as the two stages, these descriptions appear inconsistent for the pretraining stage. The optimization settings concern the encoder and decoder, while the codebook entries are updated through EMA. The reported training duration also differs: 400K iterations in Section 4.1 versus 700 epochs, approximately 483K steps, in Appendix A.2. Please clarify the configuration used for the reported results and whether the two durations refer to the total training budget or the validation-selected checkpoint.

  2. The contribution of LL codebook pretraining is not isolated. Table 4 and Figure 5 demonstrate the effectiveness of FAA; Table 5 shows that employing t-SNE-based ordering yields a gain of 0.23 dB compared to index-based ordering (page 9). However, these control experiments do not explicitly clarify the extent to which the performance improvement is attributable to the anchors learned from the LL subbands. Although FAA allows for the backpropagation of reconstruction gradients to the CAMB via continuous feature representations (page 6, Equations 10–14), this training path alone is insufficient to substantiate the value of pre-training the anchors separately. A comparison against "frozen random anchors"—while keeping the codebook size, embedding dimension, normalization method, assignment and ordering process, network architecture, and LLIE training configuration strictly consistent—would help assess the contribution of LL codebook pretraining under this configuration. Furthermore, for the variants in Table 4 that exclude FAA, additional clarification is required regarding the feature dimensions employed and whether any form of projection operation is retained.

  3. The main-table comparisons require clearer reporting of result sources and protocols. The 75K ablations provide useful controlled evidence, while the main CASSM results use a 150K training budget (pp. 7–9). For Tables 1–3, the manuscript does not clearly distinguish results taken from prior work, evaluations of public checkpoints, and results obtained by retraining, or specify the corresponding training and evaluation settings. This information is important for interpreting the 1.32 dB gain over C2SSM and the Table 3 transfer comparison. In particular, each baseline's training data and checkpoint source should be identified for the LOL-v1-to-LOL-v2-Real evaluation. Given the parameter increase from 2.71M to 3.95M, the manuscript should also specify the components retained and parameter counts for the Table 5 variants to clarify the extent of capacity control.

  4. Practical UHD efficiency and variability across training runs need further reporting. Tables 3 and 8 give FLOPs without specifying the input dimensions, and full-resolution latency and peak memory are not reported (pp. 7, 16). The change from 0.407 to 0.449 GFLOPs therefore provides limited information about end-to-end 4K inference, including code assignment and token reordering. Reporting the L3 feature dimensions and complete-pipeline measurements under specified hardware, batch size, and precision would make the efficiency claim assessable. The manuscript also does not state the number of independent training runs or variability for the key ablations. This information would help assess the reproducibility of the 0.08 and 0.23 dB increments in Table 5 (p. 9).

  5. The contribution relative to MambaIRv2's SGN needs clearer positioning. Section 2.2 describes MambaIRv2 in terms of semantic prompts and routing (p. 3), but its Semantic Guided Neighboring mechanism also groups tokens by prompt category before state-space processing and restores spatial order afterward (MambaIRv2, §4.2). CASSM's frozen LL anchors and similarity-based inter-group ordering are identifiable differences. A more explicit comparison should explain the roles of these differences and identify which existing results support their incremental benefit. This would make the scope of the methodological contribution easier to assess.

Questions for authors
  1. Could you clarify the optimizer and learning-rate schedule used for the VQ-VAE encoder and decoder during codebook pretraining, and reconcile Section 4.1 with Appendix A.2? How do the reported 400K iterations relate to the approximately 483K steps and the validation-selected checkpoint?
  2. For the Table 4 variant without FAA, what feature dimensions and projections are used for code assignment? Could you assess the contribution of LL pretraining using frozen random anchors while keeping K, D, normalization, assignment and ordering procedures, architecture, and LLIE training settings unchanged?
  3. For Tables 1–3, which baseline results were taken from prior work, evaluated using public checkpoints, or obtained by retraining? Please specify the corresponding training and evaluation protocols, particularly each Table 3 baseline's training data and checkpoint source for the LOL-v1-to-LOL-v2-Real evaluation.
  4. For 3840×2160 input, could you report the L3 feature dimensions, token count, full-pipeline inference time, and peak memory under specified measurement settings? Please also provide parameter counts for the Table 5 variants and the number of independent training runs for the key ablations, with variability across repeated runs of the same configuration where available.
  5. Could you clarify CASSM's contribution relative to MambaIRv2's SGN, particularly the roles of frozen LL anchors and similarity-based inter-group ordering, and explain which existing results support the claimed advantages?
Reviewer_RCiYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CASSM introduces content-dependent state-space scanning for UHD low-light image enhancement. A VQ-VAE pretrained on low-frequency wavelet components provides a frozen codebook with 1,024 entries of dimension 128. FAA maps restoration features into this space, and nearest-codeword assignments group tokens by content. Tokens retain raster order within each group; groups follow a fixed ordering obtained through offline one-dimensional t-SNE. The reordered features pass through a state-space module before being restored to their spatial positions. Experiments cover UHD-LOL4K, UHD-LL, LOL-v1, and transfer from LOL-v1 to LOL-v2-Real, with ablations on grouping, ordering, and module placement.

Critical strengths
  1. Some of the reported gains are substantial. Table 1 improves UHD-LOL4K PSNR from C2SSM's 39.61 to 40.93 dB, while Table 3 improves LOL-v2-Real from 31.30 to 32.64 dB. The former is strong evidence for the method's practical value; interpreting the latter requires clarification of the transfer protocol.

  2. The sequence construction is tested directly. Table 5 improves PSNR from 38.68 dB with spatial scanning to 38.76 dB with grouping and 38.99 dB with inter-group reordering. Tables 4 and 6 examine FAA and alternative modules at the same insertion point. These controls support a contribution from the scanning design rather than only from changing the backbone.

Critical weaknesses
  1. The cross-dataset comparison is not specified well enough to support the generalization claim. Section 4.1 trains CASSM on LOL-v1 and evaluates it on LOL-v2-Real, but Table 3 does not establish whether each baseline follows that protocol or where its results come from. Appendix A.2 and Table 7 also use additional clean UHD images for codebook pretraining, so the complete training data differ from the restoration training set alone. The training descriptions conflict: Section 4.1 gives AdamW, a learning rate of 5e-4, and 400K iterations, whereas Appendix A.2 gives Adam, 1e-4, and 700 epochs, approximately 483K steps. Without resolving these details, the largest transfer gain cannot be attributed confidently to the proposed method.

  2. The claimed modest overhead at UHD resolution is not demonstrated. The 0.449G FLOPs in Table 3 and Appendix Table 8 lack an input resolution and an accounting scope, and no native-UHD latency or peak memory is reported. Direct codebook matching in Eq. (8) scales with N × 1,024 × 128, while Eqs. (9)–(14) also require grouping, permutation, and restoration of token order. Offline t-SNE is already explained in Appendix A.1; the unresolved cost is the per-image execution of these operations. This omission matters because the paper's application depends on the UHD quality–efficiency trade-off.

Questions for authors
  1. For each baseline in Table 3, what training data and checkpoints or published results were used? Does every LOL-v2-Real result follow LOL-v1-to-LOL-v2-Real transfer? Please distinguish codebook pretraining from restoration training and specify whether evaluation uses ground-truth-assisted brightness correction.

  2. Which optimizer, learning rate, training duration, and codebook selection rule produced the main results and ablations: those in Section 4.1 or Appendix A.2?

  3. What input size does 0.449G refer to, and does the count include codebook matching and SSM operations? Using the existing CASSM and C2SSM checkpoints, could you report native-4K latency and peak memory on the same device, at the same precision and batch size? Matched evaluation protocols and favorable UHD measurements would be the main grounds for changing my recommendation to acceptance.

Reviewer_Z7nzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CASSM, a Content-Aware State Space Model for ultra-high-definition low-light image enhancement. The main idea is to address the limitations of predefined spatial scanning in Mamba-based architectures by reorganizing visual tokens according to their content similarity. Specifically, the method learns a low-frequency codebook from clean image wavelet representations and introduces a Feature Alignment Adapter (FAA) to associate intermediate restoration features with codebook entries. The Content-Aware Mamba Block (CAMB) then constructs content-aware sequences through token-level grouping and group-level reordering based on one-dimensional t-SNE projections. The resulting sequences are processed by a selective state-space model for image enhancement. Experiments on UHD-LL, UHD-LOL4K, LOL-v1, and LOL-v2-Real demonstrate competitive reconstruction performance, with an average PSNR improvement of approximately 0.75 dB over the best previously reported results.

Critical strengths

1Well-motivated content-aware sequence modeling. The paper identifies a meaningful limitation of conventional spatial scanning strategies: visually related tokens may be separated by long distances in the resulting sequence. The proposed hierarchical sequence organization provides an intuitive approach to incorporating content relationships into state-space modeling. Using a compact codebook to avoid expensive pairwise token comparisons is also a practical design choice for high-resolution image restoration. 2. Competitive experimental performance across multiple benchmarks. The proposed method achieves the best reported PSNR on all four evaluated datasets. In particular, the improvements of 1.32 dB on UHD-LOL4K and 1.34 dB on LOL-v2-Real are encouraging. The component ablations and comparisons with alternative Mamba blocks provide additional evidence that the proposed framework can improve reconstruction performance.

Critical weaknesses
  1. Limited novelty of the proposed sequence organization. The core method combines codebook-based token grouping with t-SNE-based reordering. Although effective, the technical novelty beyond existing content-aware scanning and clustering-based Mamba methods is not sufficiently established.
  2. Insufficient justification for t-SNE-based ordering. The paper does not compare t-SNE with simpler alternatives, such as PCA-based, nearest-neighbor, or random ordering. The stability of the resulting sequence organization under different t-SNE initializations also remains unexplored.
  3. Unclear effectiveness of feature alignment. The FAA assigns features to a frozen codebook through discrete nearest-entry matching, but the paper does not clearly explain how reliable alignment is learned without an explicit alignment objective. The qualitative visualization alone provides limited evidence of meaningful content correspondence.
  4. Incomplete computational efficiency evaluation. Despite emphasizing linear complexity and UHD applicability, the paper lacks actual inference latency and peak memory comparisons at native UHD resolutions. The computational overhead of dynamic token grouping and sequence restoration is also insufficiently analyzed.
  5. Fairness and reproducibility concerns. The codebook is pretrained using clean images from five restoration datasets, making it difficult to isolate the benefits of content-aware modeling from additional training data. Furthermore, inconsistent codebook training configurations between Section 4.1 and Appendix A.2 raise reproducibility concerns.
Questions for authors

1.Content-aware ordering: Can the authors compare the proposed t-SNE-based group ordering against simpler alternatives, such as random ordering, PCA-based ordering, and nearest-neighbor ordering? It would also be useful to evaluate whether the results are stable across different t-SNE random seeds. 2. Feature alignment and codebook pretraining: How does the FAA learn meaningful correspondence with the frozen codebook when token assignments are computed through a discrete argmax operation? Can the authors clarify the training signal responsible for alignment and evaluate the effect of training the codebook only on the target low-light dataset, rather than using five restoration datasets? 3.Computational efficiency: What is the actual end-to-end complexity of token grouping, sequence construction, and index restoration? Can the authors provide inference latency and peak GPU memory consumption at native 4K resolution, including a direct comparison with C2 SSM under identical hardware and evaluation settings? Please also specify the input resolution used for the reported FLOPs. 4.Experimental reproducibility: Which codebook training configuration was used to obtain the reported results: the 400K-step AdamW configuration described in Section 4.1 or the approximately 483K-step Adam configuration in Appendix A.2? Please also clarify the value of the FFT loss weight (\lambda) in Equation (1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.