acceptodds
Under review as a conference paper at ICLR 2027

Rank Frontiers for Shared Linear Encoders: Paired Task Filters

Abstract

For paired task filters, we derive the exact information-loss frontier of a rank-limited shared linear encoder. At equal overlap, the global optimizer changes exactly from pairing rank units within output coordinates to spreading them across coordinates at . This allocation law is fragile to unequal overlap: an explicit three-output projector beats every block-aligned allocation, while the complete two-output unequal-overlap frontier remains solvable at all five ranks. A held-out learned-subspace study recovers the predicted transition within nats. These are results for a paired-filter linear–Gaussian model, not for nonlinear encoders or practical architecture selection. Common whitening covers one shared nonspherical covariance; distinct covariances receive only the stated residual results. Capacity-comparison signs are accounting-dependent, and all public-data selectors abstain, so neither supplies an architecture recommendation. Finite temporal histories are one covariance-structured application of the static Gaussian-vector result; we make no new claim about temporal dynamics.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
shared representations, information loss, linear encoders, rank constraints, multimodal fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_st6qOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents a study on rank-limited linear encoders for paired task filters. As part of this multiple propositions and a theorem are presented, developed from a mutual information perspective. The overall text is highly unclear, with large sections having been directly generated by AI and not written in the form of an academic paper. This makes the motivation, core contribution, and positioning of the paper difficult to evaluate and the manuscript difficult to parse.

Critical strengths

While the paper is not coherent in its current form, it presents multiple Propositions and one Theorem. Presented differently with fuller motivation and clarity, these results may have value to the community. Unfortunately it is difficult to understand the positioning, motivation, contribution, or importance of these results based on the submitted manuscript.

Critical weaknesses

The critical weakness for the paper is clarity. Large portions of the paper have been generated directly by AI, and the manuscript has not fully been written into the form of an academic paper. To give an example of this, L307-309 contains references to the experiment folder and agent harness: "The stable anonymous entry point artifact/v22_learned_frontier_recovery.py regenerates artifact/v22_learned_frontier_recovery_results.json, including all 60 seed gaps; artifact/README.md gives the command and environment."

As a result much of the text is incoherently presented. Specific issues include undefined jargon, unclear mathematical notation, unclear methodological experiments, focus on extraneous experimental details (e.g. L310 "Seeds are 11, 23, 37, 53, 71"), unclear motivation, and related works. The core contribution for the paper is difficult to identify.

Questions for authors

I do not believe the cumulative issues would be possible to address in a rebuttal.

I encourage the authors to rewrite this paper, focusing on providing clarity to a conference audience.

Reviewer_nUQ5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper focus on information loss when two streams of input try to predict a single outcome with a shared linear encoding unit. It is an important problem to tackle in modern learnable systems where multi-modality inputs are usually merged deep in the models.

Critical strengths

The authors build their proofs and insights on known results, such as log-det frontiers, fantope relaxation and prinicple-angle transition. The author provided a closed form solution for the bound and validated it on empirical tests that it holds up-to small error. IMHO this is a new result.

Critical weaknesses

The analysis and empirical tests are relevant for linear-gaussian models which is highly limited. So this result is interesting as basic science but not relevant for analyzing modern models.

Questions for authors

How would expand the bound for multiple sources? How would you shift from linear-gaussian to a broader setup?

Reviewer_aN39OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the information loss incurred when the same rank-limited linear encoder processes two independent Gaussian inputs to predict a common additive target. For paired task filters, it derives an exact variational characterization and a closed-form equal-overlap frontier, including a transition between concentrating rank within target coordinates and distributing it across coordinates. An explicit unequal-overlap counterexample shows that this allocation rule does not generalize, while the two-output unequal-overlap case is solved at every rank. Validation consists of proofs, numerical checks, and a synthetic learned-subspace experiment across six ranks, two overlap settings, and five seeds, reporting a maximum mean discrepancy of 0.0233 nats from the exact frontier.

Critical strengths
  1. An exact allocation result with an interpretable transition. Proposition 2 establishes the optimum over all shared projectors, rather than assuming block-aligned solutions. The resulting pairing-to-spreading transition explains how overlap and noise determine the best use of a shared rank budget. The information gap also has an operational interpretation as excess optimal Gaussian predictive log loss. This is a useful, precise characterization within the stated model.

  2. The unequal-overlap results establish a meaningful boundary of the main theorem. Proposition 3 demonstrates that cross-block mixing can strictly improve on every block-aligned allocation, while Proposition 8 provides an exact solution for two outputs. Together, these results show why the equal-overlap solution is special and provide additional understanding beyond its symmetric setting. The learned-subspace experiment supplies a complementary check that optimization from random initializations can recover the predicted behavior.

Critical weaknesses
  1. The contribution is relatively specialized, and its relationship to the closest classical determinant results needs sharper treatment. After the variational reduction, the equal-overlap optimization concerns a weighted sum of projectors. Classical prescribed-spectrum determinant inequalities, particularly Fiedler’s result, are directly relevant to the alignment step. The paper discusses several neighboring frameworks but does not explicitly establish this connection. This limits how much novelty I attribute to the equal-overlap proof itself; the stronger contribution is the combined tied-encoder characterization and unequal-overlap analysis.

  2. The presentation dilutes the central result and overstates the precision of the empirical transition check. Substantial discussion of selectors, capacity accounting, and forecasting remains despite those analyses providing no positive evidence for applying the frontier. Meanwhile, the learned experiment evaluates two overlaps on opposite sides of the theoretical threshold: it supports the predicted regimes but does not empirically locate the threshold. Section 3 should also explicitly state the noise variance and mathematically define the reported minus-bank mass, including its treatment of whitening. These issues make the central evidence harder to interpret and reproduce.

Questions for authors
  1. Clarification of novelty: Can you explain the relationship between the projector-alignment step in Proposition 2 and Fiedler’s determinant bounds, distinguishing the classical spectral ingredients from the additional results established here?

  2. Clarification of experimental evidence: What noise variance is used in Section 3, and what is the exact definition of the minus-bank mass for the learned raw encoder under the nonspherical covariance? Please also clarify that the two tested overlap values validate behavior on opposite sides of the theoretical transition, rather than estimating its location.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.