acceptodds
Withdrawn from ICLR 2027

Reliable but Compressed: Longer Timescales Lower Dimensionality with Readout-Dependent Task Costs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When a neural representation becomes more reliable and lower-dimensional, does it necessarily become less useful downstream? We test this question in a fixed auditory spiking network by increasing membrane and synaptic timescales while matching mean firing rates. Longer timescales increase reliability to input perturbations but reduce effective dimensionality, producing a reliable-but-compressed representation. Crucially, the apparent downstream cost of this compression is not stable across readout procedures: on the same frozen features, an expanded regularized readout substantially attenuates the long-timescale penalty while improving absolute predictive performance. The geometric change therefore persists, whereas its inferred task cost changes with the evaluation procedure. Mechanistic controls show that compression is much weaker in a separately activity-matched continuous implementation, while slower local state recovery closely follows passive membrane decay. Within this controlled system, these results separate three effects that are easily conflated—representation geometry, perturbation response, and fitted task accessibility—and show that lower dimensionality alone is insufficient to infer an intrinsic loss of downstream utility.

Primary area:
applications to neuroscience & cognitive science
Keywords:
spiking neural networks, intrinsic timescales, recurrent dynamics, representation geometry, perturbation reliability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pYNpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks how does the properties of the reservoir representation change with synaptic and membrane time constants of LIF neurons in a liquid-state reservoir model. The properties of the reservoir that are of interest are the reliability and the dimensionality of representations. Results suggest that longer timescales might increase the reliability while they decrease the dimensionality. The effect on the decoder performance depends on the readout model and the fitting procedure. The intervention on time constants is constrained about the firing rates, enforcing the same firing rates when time constants are varied.

Critical strengths

The question that is addressed is of interest, in particular to neuroscience, even though the question is rather narrow and is not of great significance. Some of the methodology appears appropriate to address the question.

Critical weaknesses
  1. The text is cryptic and very machine-like. As far as I can tell, the question of the paper is relatively simple, but the presentation makes it sound very complicated. Instead of simply explaining what they are doing in a comprehensive way, the text is full of overly complicated expressions that makes the understanding of the text unnecessarily challenging. \

From the beginning, the main question and the motivation are not very clear. Can the authors motivate the question addressed more clearly from the beginning? I suggest to form one or two sentences that make it very clear what is the question addressed, at the cost of some simplification, if necessary, as details can be added later. Further, I suggest to use some space to explain why is it relevant.

The paragraph starting with line 110 reads as a list of techniques that are relevant to the methods of the paper. I instead suggest to have something less list-like, where sentences connect to each other logically and have a coherent flow of meaning.

A variable Ap is described in line 152, but this variable appears only later, in Eq. 4.

  1. Authors perform a number of analyses, but the motivation for each analysis is poorly explained and the link between these analyses is essentially missing. The paper seems to be a set of results without drawing much connections between them. Again, this reads machine-like.

  2. Synaptic and membrane time constants of LIF neurons change together. These time constants control difference aspects of temporal processing of the network. The synaptic time constant determines the temporal profile of the signal that is than integrated by the membrane time constant. Therefore, clearly, the effect of changing the synaptic and membrane time constants are not independent, and changing both at the same time confounds their effect.

Further, from the biological plausibility point of view, the time constants used in this study are longer than typical time constants in biological neurons. Already for Fast neurons, 8 ms is out of range for AMPA synapses (which are within 2-5 ms). All time constants of slow neurons are outside of biologically plausible ranges.

Questions for authors

(1) What are variables p and n in Eq. 2? Why do we have a division with a minimum? (2) What is the subscript tr in line 161 about? (3) Eq. 4: is H(Y) the Entropy? What is CE? (4) How is spike count Wasserstein distance defined? (5) Why are synaptic and membrane time constants of LIF neurons varied together? (6) Why did authors choose the time constants ranges which are mostly outside of biologically plausible ranges? (7) Can authors comment on a hypothesis about what is causing the main observed effects, e.g., the dimensionality reduction and reliability increase with longer time scales? Reliability might increase almost trivially as longer time constants imply more integration of the signal and more discounting of the noise by temporal averaging. (8) Why are firing rates matched? I am not sure that this is actually useful or helpful in this setting.

Reviewer_nBn5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper varies membrane and synaptic timescales in a fixed auditory spiking network while matching population firing rates. Longer timescales increase input-perturbation reliability and reduce effective dimensionality. On identical frozen features, an expanded regularized linear readout substantially reduces the apparent long-timescale performance penalty and improves absolute prediction.

Critical strengths

The paired intervention holds sounds, connectivity, and model seeds fixed while matching population activity. This provides a useful controlled setting for studying how timescales affect representation geometry and reliability.

The central comparison changes the readout procedure on identical frozen features and tests the interaction directly. The supplied seed-level summaries support the reported effect. Absolute performance improves at both endpoints, so the reduced gap does not result from degrading both predictions.

Critical weaknesses

The relationship between compression and task performance remains insufficiently explained. Although Section 4.1 and Table 3 report reduced within-class variation and increased between-class variation at lower dimensionality, the analyses do not relate the contracting covariance directions to class discrimination or to the weights used by the two readouts. The 72% off-diagonal decomposition likewise pools correlations within and across the eight time bins, so temporal redundancy and correlations across neurons are not separated. The readout-dependence result is therefore supported, but what is compressed and why the revised readout removes the penalty remain unresolved, which limits the explanatory contribution.

The closing discussion in Appendix I.4 also adopts a defensive framing, repeating numerous exclusions and qualifications that make the positive takeaway harder to identify and add avoidable length. Because the requested analyses use the existing representations, addressing the gap above would not require additional benchmarks and could justify a higher rating.

Questions for authors

Do the existing analyses identify whether contracting directions carry class-discriminative variation, and whether the baseline and expanded readouts use those directions differently? Connecting these quantities would materially affect my assessment.

Can the covariance decomposition distinguish correlations within time bins from those across bins, clarifying how much of the compression reflects temporal redundancy?

Reviewer_h1kUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

For this summary, I will entirely quote from the text, for the reasons cited below. The abstract poses "When a neural representation becomes more reliable and lower-dimensional, does it necessarily become less useful downstream?" The authors state that they "test this question in a fixed auditory spiking network by increasing membrane and synaptic timescales while matching mean firing rates." They state that "longer timescales increase reliability to input perturbations but reduce effective dimensionality, producing a reliable-but-compressed representation," and that "the geometric change therefore persists, whereas its inferred task cost changes with the evaluation procedure."

Critical strengths

The relationship between embedding geometry and information decoding is interesting.

Critical weaknesses
  • The manuscript’s clarity is way below the minimum standard necessary for acceptance. This lack of clarity does not appear to be due to a sincere but failed attempt at clarity, but rather on a structural incoherence in the manuscript itself. Across sections — abstract, introduction, related work, methods, results — the text does not connect what is being presented back to the paper's main contribution. The reader has to assemble that connection themselves. Examples:
    • The Abstract does not setup a general reader to understand the problem, the approach, and the result.
    • The Introduction does not introduce the problem but rather is a short summary of the findings.
    • The background section does not provide enough detail to understand how related works relate to this work. The paragraph on RNNs seems genuinely unrelated.
    • Section 3.1 is too sparse to reconstruct the task, data, or model. There are no equations specifying the model architecture.
    • Figures 2 and 3 are never referenced in the text, a clear demonstrating of the structural lack of clarity and coherence.
    • Section 4 opens with specific numbers rather than a motivation and closes with a disclaimer rather than an interpretation. There is no displayed attempt to motivate the forthcoming analysis and how that analysis relates to the central thesis presented in the abstract. There is no presentation of text to help a reader understand the motivation for the section and then an interpretation of the presented results.
    • Both sections 4 and 5 read like a stream of text without any structure relating the results to the stated goals of the work.
  • The study design does not add up to a coherent scientific inquiry. The three central metrics (D, R, A_P) are introduced without rationale, the covariance-energy decomposition (eqs. 5–6) appears without stated purpose, and the appendix is full of audits (reproduction tolerances, counterfactual assays, minimal-mechanism follow-ups, TinySOL transfer, NAT4 recordings) whose relevance to the central claim is unclear and not stated. These are individually plausible analyses but do not read as a coherent piece of work. Correcting this to evaluate the value of this work would require a substantial rewrite.
Questions for authors
  • Figures 2 and 3 do not appear in the body text. Could the authors reference them and walk the reader through each figure as evidence for a specific argument?
  • Can the authors explain why these three central metrics (D, R, A_P) were selected, and how is each required for the central claim?
  • Could the authors state the purpose of Sections 4 and 5 and interpret the presented results, and explicitly connect the covariance-energy decomposition (eqs. 5–6) back to the central claim? As written, the decomposition appears without purpose.
  • Could the authors provide an expanded methods section so that the model and approach can be understood completely.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.