acceptodds
Under review as a conference paper at ICLR 2027

Slice Selection Matters: Structured and Adaptive Directions for Sliced Latent Regularization

Abstract

Sliced latent regularization typically uses random projections to pursue Gaussian-target matching, but these directions can conceal discrepancies along directions populated by the encoder. We investigate deterministic rotation and propose a simple radial-adaptive augmentation to better fulfill the regularization objective. For -dimensional latents, we append normalized minibatch vectors to random or rotating base directions, selecting vectors whose lengths are unusually small or large relative to the isotropic Gaussian target. Recomputed at each training step, this construction requires no additional learned parameters or inner optimization and supports both sliced Wasserstein- and Epps–Pulley losses. Across autoencoding and JEPA-style self-supervision on MNIST and ImageNet-100, matched-budget controls replace the adaptive directions with random directions, separating augmentation from base schedule and slice count. Over ten matched seeds, radial-adaptive augmentation lowers mean held-out latent-direction Kolmogorov–Smirnov discrepancy in all 16 comparisons spanning datasets, objectives, losses, and base schedules. Additional evaluation with disjoint samples preserves these mean improvements. Deterministic rotation alone has no universal advantage, whereas adaptation benefits both base schedules. A residual directional gap remains, so improved projected agreement does not establish complete Gaussian matching. We therefore recommend radial-adaptive augmentation as a simple complement to sliced Gaussian regularization when improved target matching is the goal, together with evaluation using both random and latent directions.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
sliced latent regularization, representation learning, self-supervised learning, autoencoders, JEPA, Gaussianity, sliced Wasserstein, Epps--Pulley, adaptive slice directions

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zmqSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how the choice of one-dimensional projection directions affects sliced latent regularization toward an isotropic Gaussian distribution. The authors consider autoencoder and JEPA-style training, with either sliced Wasserstein-2 (WS2) or Epps-Pulley (EP) as the latent regularizer. They compare independently sampled random directions with a deterministic rotating orthonormal frame. They further propose augmenting either base direction set with radial-adaptive directions obtained by selecting minibatch latent vectors whose squared norms are atypical under the target chi-square distribution.

The learned representations are evaluated using the average per-direction Kolmogorov-Smirnov discrepancy under two direction banks: uniformly random directions and directions obtained by normalizing held-out latent representations. The main empirical findings are that random-direction evaluation can report substantially smaller discrepancies than latent-derived evaluation, radial-adaptive augmentation reduces the mean latent-direction KS discrepancy across the tested configurations, and deterministic rotation alone does not consistently outperform random directions.

Critical strengths

The paper examines an important but often overlooked aspect of sliced latent regularization: the choice of projection directions. The proposed radial-adaptive augmentation is simple, introduces no learned parameters or inner optimization, and can be combined with both random and rotating base directions. The experimental design is generally careful: comparisons use matched slice budgets, a 2×22\times2 factorial construction, ten paired training seeds, two regularization losses, two learning objectives, and both random- and latent-direction evaluation. The authors additionally provide sample-disjoint controls and appropriately acknowledge that deterministic rotation is not consistently superior to random sampling. The radial-adaptive method lowers mean latent-direction KS discrepancy in all 16 matched comparisons, indicating a consistent improvement in matching the specified isotropic Gaussian target.

Critical weaknesses

The paper’s presentation is difficult to follow because it introduces several direction schedules, training losses, evaluation metrics, and learning objectives without clearly prioritizing the central contribution. More importantly, the motivation for the deterministic rotation schedule is weak: the golden-ratio-based angle construction is largely heuristic, receives no sensitivity analysis, and does not produce a consistent advantage over random directions. The radial-adaptive component also lacks essential ablations for KK, thresholding, and unranked latent-derived directions, making it unclear whether radial atypicality itself is important or whether any latent-derived augmentation would provide similar benefits. Comparisons with closely related data-informed methods, such as max-sliced and random-path sliced Wasserstein, are also missing. Finally, the evaluation primarily demonstrates improved Gaussian-target matching on MNIST and ImageNet-100; it does not establish meaningful improvements in reconstruction, representation quality, or downstream performance, and the linear probes are used only as diagnostics against complete collapse. These limitations leave the practical significance and generality of the method uncertain.

Questions for authors

Questions for the Authors

  1. Clarification of the overall objective: Could the authors provide an earlier and more intuitive explanation of what constitutes a “slice” and clearly separate the roles of the task loss, WS2 or EP regularization, KS evaluation, and linear-probe cross-entropy? The current presentation makes it difficult to determine which quantities are optimized and which are used only for evaluation.

  2. Motivation for the rotation schedule: What theoretical or empirical evidence motivates the specific golden-ratio-based angle schedule, the interval [π/8,3π/8)[\pi/8,3\pi/8), and the round-robin coordinate pairings? Equidistribution of the scalar phase does not necessarily imply uniform or low-discrepancy coverage of the unit sphere by the resulting orthogonal frames.

  3. Monte Carlo variation: Can the authors quantify the Monte Carlo variation introduced by independently sampled random directions and demonstrate that the deterministic rotating schedule actually reduces the variance of the sliced-loss or gradient estimator? The current paper states this motivation but does not directly measure it.

  4. Isolating the effect of rotation: Rand-D and Rot-D differ in orthogonality, randomness, temporal dependence, and directional coverage. How can the observed differences be attributed specifically to rotation? Could the authors include fixed-orthogonal and freshly sampled Haar-random orthogonal baselines to isolate these factors?

  5. Comparison with existing direction-selection methods: Why are there no experimental comparisons with established alternatives such as Max-Sliced Wasserstein, Distributional Sliced-Wasserstein, and Random-Path Sliced Wasserstein? Random-Path Sliced Wasserstein appears especially relevant because it also constructs optimization-free, data-dependent projection directions. PCA/eigenvector directions and prior parameterized orthogonal-projection methods would also be natural baselines.

  6. Evaluation-direction dependence: In the primary test-latent evaluation, the same held-out representations define the direction bank and supply the samples being projected. In particular, every sample is projected onto its own normalized direction, resulting in its norm. Since the proposed adaptive method explicitly targets radial atypicality, to what extent does this evaluation favor the proposed method? Could the authors make the disjoint-sample evaluation more prominent and include additional independent direction banks or multivariate Gaussianity diagnostics?

  7. Practical significance: Does the reduction in latent-direction KS discrepancy produce measurable improvements in reconstruction, downstream representation quality, linear-probe accuracy, or another practically meaningful outcome? Without such evidence, it is unclear why improved agreement with this particular Gaussianity diagnostic is valuable.

  8. Experimental generality: Do the conclusions hold under stronger architectures, longer training, full ImageNet, or additional self-supervised learning settings? The current MNIST and ImageNet-100 experiments are useful controlled ablations, but they provide limited evidence of general applicability.

  9. Statistical testing: The paper reports many paired hypothesis tests without correction for multiple comparisons. Do the principal conclusions remain statistically supported after an appropriate correction, and can the authors emphasize effect sizes and uncertainty rather than the number of nominally significant comparisons?

  10. Relationship to prior orthogonal latent-space methods: Could the authors more clearly discuss how the rotating orthogonal frame differs from prior methods that construct or learn orthogonal projection directions in latent space? This comparison is necessary to clarify the novelty of the structured-rotation component.

2 comments
AuthorsOpenReview ↗

Reviewer zmqS: Contribution Focus, Experimental Controls, and Statistical Evidence

We thank the reviewer for recognizing the matched design, disjoint controls, and consistent mean adaptive improvements. The presentation should prioritize the directional finding and adaptation.

Rotation showed no consistent advantage, whereas adaptation helps with either base schedule. This motivated emphasizing adaptation rather than extending the schedule study. Limited space explains that presentation choice, but does not replace missing controls.

Q1. Objectives and diagnostics. A slice is a unit direction onto which latent vectors are projected. Encoder training combines a task loss with WS2 or EP regularization toward standard Gaussian projections. The task loss is reconstruction MSE for AE or bidirectional prediction loss for JEPA. KS is evaluation only. A linear classifier is subsequently fitted by cross-entropy on frozen encoder outputs. These roles should be distinguished earlier.

Q2–Q4 and Q10. Rotation, variance, and novelty. The angle schedule is heuristic. Scalar phase equidistribution does not establish uniform or low-discrepancy sphere coverage. We have not demonstrated reduced loss or gradient-estimator variance. Removing fresh directional sampling alone does not establish variance reduction.

Rand-D versus Rot-D also changes orthogonality and temporal dependence, so it cannot isolate rotation causally. Fixed orthogonal and freshly sampled orthogonal frames would be appropriate controls. This schedule is a tested baseline, not universally superior. We do not claim novelty for orthogonal projections generally, an optimal schedule, or proven variance reduction.

Q5. Related methods and controls. Random-path sliced Wasserstein is particularly relevant as an optimization-free, data-dependent construction. Our method selects normalized minibatch vectors by radial atypicality relative to the target, but this difference does not establish superiority. Max-sliced, distributional, PCA-based, and other orthogonal constructions are also relevant comparisons. The present evidence is against matched random augmentation. Unranked latent-derived augmentation remains the most direct missing training control for isolating radial ranking. The experiments use K=8 without thresholding, so sensitivity to these choices remains unresolved.

Q6. Evaluation dependence. Appendix C splits held-out representations into two subsets. One supplies directions and the other projected observations, then their roles are exchanged. The directional gap remains positive for all 480 models, and all 16 mean adaptive improvements persist. Isotropic Gaussian reference controls produce gaps close to zero.

The paper's direct self-projection bound applies to removing the source observation from a fixed projection, not to all dependence from constructing a direction bank. Disjoint controls address sample reuse, but not the relationship between training and evaluation direction families. Finite projection tests also cannot certify full multivariate Gaussianity. Moment-matched Gaussian controls show that mean and covariance mismatch can explain substantial gaps.

Q7–Q8. Practical significance and generality. The recommendation applies when isotropic Gaussian matching is already a design objective. The augmentation improves latent-direction agreement without additional learned parameters or inner optimization, but does not establish downstream superiority.

Detailed comparisons of saved reconstruction and probe results, not included in the submitted paper, show mixed task effects, including lower probe accuracy despite improved latent-direction KS. Aggregate probe ranges are reported, but not these detailed comparisons. Above-chance accuracy indicates retained class information, not competitive SSL or the absence of partial collapse. Stronger architectures, longer training, and additional settings are needed to establish generality. Whether another target distribution better serves a downstream task is a separate question from how well the selected target is realized.

Q9. Statistical evidence. An additional Holm correction across the 16 principal adaptive comparisons retains 15 significant differences at family-wise 0.05, with adjusted p=0.03125, but not at family-wise 0.01. MNIST AE with WS2 and a rotating base has a lower mean but is not significant. This correction concerns those 16 comparisons, not every test in the paper.

Relative reductions in mean test-latent KS range from approximately 2.9% to 28.0% on MNIST and 17.6% to 53.1% on ImageNet-100. All 16 mean improvements also persist in disjoint evaluation, but a residual directional gap remains. Effect sizes and seed variability should accompany significance tests. The evidence supports improvement in the tested settings, not complete matching or universal downstream benefit.

Reviewer_zmqSOpenReview ↗

Follow-up: Design Justification and Remaining Concerns

Thank you for the detailed response and the additional clarification of the contributions. The disjoint-sample and Gaussian-reference controls strengthen the diagnostic finding, and the matched-budget results support the augmentation relative to random augmentation. I also appreciate the statistical correction and understand that downstream superiority is outside the central claim. However, several concerns remain:

  1. Presentation and organization. Although the response clarifies the roles of the objectives and diagnostics, the manuscript remains difficult to follow. An early overview of the complete training procedure would help. The presentation should prioritize the directional diagnostic finding and adaptive augmentation, while giving the exploratory rotation study an appropriate supporting role.
  2. Justification of rotation. The response acknowledges that the angle schedule is heuristic, scalar-phase equidistribution does not establish spherical coverage, and reduced loss or gradient-estimator variance has not been demonstrated. Furthermore, Rand-D versus Rot-D changes orthogonality and temporal dependence alongside rotation. Framing this construction as a baseline appropriately limits the claims, but the rationale for the particular golden-ratio schedule and angle interval remains insufficiently supported.
  3. Validation of radial selection. The strongest unresolved issue is the missing unranked latent-derived augmentation control. The reported improvements establish benefits over random augmentation, but do not determine whether radial-atypicality ranking adds value beyond selecting latent-derived directions generally. Sensitivity to the fixed K=8K=8 budget and comparisons with closely related methods, particularly Random-Path Sliced Wasserstein, would further clarify the contribution’s robustness and added value.

I recognize the supported empirical findings and the authors’ candid clarification of their limitations. Nevertheless, the distinctive benefit of the proposed radial-selection rule remains insufficiently established, and the presentation still obscures the central contribution. I therefore maintain my original assessment.

Reviewer_qcGqOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript is an investigation to understand if using a radial-adaptive augmentation during the slice-selection for the regularization itself, can lead to better representations than using standard random projections for Gaussian target matching. The authors perform experiments for auto-encoders and JEPA-style architectures on two low-res but popular image datasets, namely MNIST and ImageNET-100, and show that their proposed approach reduces the mean-latent-direction KS in all 16 directions they test. This is accompanied by finding of how compared to random projections, deterministic rotation is not helpful. Further, the authors also show that there is an eval issue when using random-direction eval, as there might be departures from the normal distribution along the test-latent directions that are hidden. Overall, the paper is a clean investigative study, but the fact that the metric being discussed in the results is designed to favor the proposed approach, and little to no discussion of the implications of the results in downstream tasks and utility makes the contributions seem not as substantive. The appendix and ablations do provide a handful of supporting arguments for the task but miss some key ones.

Critical strengths

[S1]: The investigative motivation is natural and the approach is clean: Examining slice selection during the regularization objective is a neat thought and is well articulated as to why one would think of something like that. The proposed approach is simple and effective, with no additional parameters or inner optimization tricks. The pseudocode and calibration provided in App. B are clear and provide reliability for the claims made by the authors.

Critical weaknesses

[W1]: Evaluation is not rigorous enough: The key metric cited everywhere in the paper and the abstract is the KS along the test-latent direction, and the proposed method explicitly adds latent-derived directions during training. So the only outcome fulfilled is a better alignment with the isotropic Gaussian under the exact same KS metric, but what the implications are for downstream use-cases are not discussed at all. As the authors acknowledge in Line 463, there is no comparison provided with unranked latent-derived augmentation either, so it is hard to understand the utility of the proposed approach because the test signal is intertwined with the training adaptation, it is hard to attribute where the gains are coming from. There is no ablation for the scores, or per-direction loss or gradient analysis. No natural mean-covariance penalty based baseline is evaluated. The related work mentioned in Sec. 2 for e.g. the sliced OT, or random-path SW are not included as baselines, which is also strange. Probe accuracy is very low for ImageNet as noted in Line 308 which is concerning.

[W2]: Contribution is minimal: This is tied to [W1] in the sense that a lack of metrics like say reconstruction MSE, or even FID to measure generation quality, leave the paper feeling incomplete and more like a blog-post almost in the sense that the contribution is unclear. Line 199 also confuses me, isn't the approach effectively performing an outlier penalty by choosing the direction, as the chosen direction would be defined by the extreme samples, so it feels like a contradiction. The positioning of the paper isn't very clear, it is a cool analysis and investigation, but to what end? Line 471 for example demonstrates that the authors provide a suggestion for improved agreement, but what is the bigger picture?

Questions for authors

[Q1] See [W1] for details. Are there more convincing results you could provide on a wider range of metrics and comparisons to other closely related baselines to isolate the contribution of the proposed approach, mainly the radial atypicality? Do the gains persist for different KK, more representative settings? Does selection by using say just the norm have the same gains? What if KK unranked latent-derived directions are used? All of these are natural questions that arise, but are left unanswered leaving the paper feeling incomplete.

[Q2] See [W2] for details. What are the downstream effects of employing the proposed approach? Does it lead to better representation quality, better reconstruction? What are the practical scenarios in which one would utilize your proposed method? Provide better applicability for your method compared to the current approaches, and how specifically the exact proposed approach is novel in its utility in achieving better representations would help strengthen the paper. Right now, it feels it is unclear what the implications are and even if they are determined what to exactly to pinpoint as cause of the improvements.

2 comments
AuthorsOpenReview ↗

Reviewer qcGq: Gaussian Target Matching, Evaluation Controls, and Practical Relevance

We thank the reviewer for recognizing the investigative motivation, simplicity, and clear computational descriptions. We agree that improved target matching must be distinguished from improved downstream representations.

W2. Contribution and positioning. The contribution combines an empirical diagnostic finding with a simple intervention. Random projections can indicate substantially closer agreement with the isotropic Gaussian target than directions populated by the encoder. This matters when isotropic Gaussianity is the intended regularization objective. The evidence concerns departures from that target, not necessarily higher-order non-Gaussianity. The moment-matched Gaussian controls show that mean and covariance mismatch can explain substantial directional gaps.

The augmentation reuses minibatch vectors, requires no additional learned parameters or inner optimization, and supports WS2 and EP. Its simplicity makes it a practical partial remedy, not evidence of universally better representations.

W1. Training and evaluation. KS is not optimized during training, and directions are not selected by their KS discrepancy. Training uses WS2 or EP, while selection uses radial atypicality. Nevertheless, both training and evaluation emphasize latent-derived directions, so using a different statistic does not fully resolve the concern.

Appendix C includes sample disjoint evaluation, where separate subsets supply directions and projected observations. The directional gap remains positive for all 480 trained models, and all 16 mean adaptive improvements persist. This rules out direct self-projection as the sole explanation, but does not establish that radial ranking is necessary or that every diagnostic improves. The 16 results are matched comparisons across datasets, objectives, losses, and base schedules, not 16 individual directions. ImageNet-100 uses 224 x 224 images.

Q1. Missing controls and radial ranking. Unranked latent-derived augmentation is the most direct missing training control. It would distinguish the benefit of including occupied directions from the benefit of radial ranking. Exploratory checks on frozen representations have mixed results and do not establish which selection rule is better during training. They are not a substitute for that ablation.

The experiments use K=8 without thresholding. Selecting only large norms, varying K, and thresholding also need controlled comparisons. The study concentrated on matched comparisons across datasets, objectives, losses, and base schedules, but this does not remove the value of these controls. Mean and covariance penalties and related data-informed methods, including random-path slicing, would be informative baselines. The current evidence supports improvement over matched random augmentation, not superiority over all alternatives.

W2. Outlier penalty. The construction is outlier-aware direction selection, but it does not directly penalize only the selected samples or their norms. Every minibatch representation is projected onto the selected directions and contributes to the sliced loss. The two-sided score can select both unusually small and unusually large norms. Using atypical samples to choose where to regularize therefore differs from applying a direct norm penalty to those samples.

Q2. Practical use and downstream effects. The question is how to better realize an already chosen isotropic Gaussian target, rather than which latent distribution maximizes downstream performance. For methods adopting this target, the paper identifies discrepancies that random-direction evaluation can miss and provides a simple augmentation that reduces them.

Detailed comparisons of the already collected reconstruction and probe results, not included in the submitted paper, show mixed task effects, including lower probe accuracy despite improved latent-direction KS. The paper reports aggregate probe ranges, not these detailed comparisons. For example, in ImageNet-100 JEPA with WS2 and a rotating base, adaptive augmentation lowers mean test-latent KS from 0.2234 to 0.1715 while mean probe accuracy decreases from 18.49% to 14.91%. Improvements do not occur under every evaluation bank.

The recommendation is therefore conditional on improved target agreement being the goal. Whether another latent distribution or geometric constraint better serves a downstream task requires a separate investigation. Observed tradeoffs could also reflect regularization strength, capacity, or training conditions rather than an unsuitable target. We do not claim downstream superiority or improved generation quality.

I thank the authors for their detailed response.

In the overall response the authors state

The intended contribution is therefore an empirical diagnostic finding and a practical partial remedy for methods that already seek isotropic Gaussian representations.

and further the authors mention in their response to [W2]

his matters when isotropic Gaussianity is the intended regularization objective. in their rebuttal, without providing stronger evidence of the utility. It would be helpful to see what exactly these applications are and quantitative outcomes for the same.

The clarification about [W1] and [W2] is helpful, but the fact that "both training and evaluation emphasize latent-derived directions", and App. C outcomes concluding "does not establish that radial ranking is necessary or that every diagnostic improves.", make the applicability of the proposed approach minimal in my opinion. This is seconded by the authors' own statement about "Detailed comparisons of the already collected reconstruction and probe results, not included in the submitted paper, show mixed task effects, including lower probe accuracy despite improved latent-direction KS."

Nothing shows that radial ranking itself matters, and the probe accuracy in fact drops for ImageNet-100 JEPA with WS2 and a rotating base.

Thus overall, having read the other reviews as well, I maintain my suggestion for rejection, as utility and contribution of the proposed approach does not meet the bar for ICLR. I appreciate the authors' frankness in stating the claims and limitations, but without downstream applications and the lack of consistent positive influence, with the mixed results re-affirmed by the rebuttal, it is hard to recommend acceptance.

Reviewer_es3DOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates slice selection as part of the regularizer for both generative modelling and self-supervised learning. The proposed method is validated using the autoencoding framework and JEPA-style self-supervision on MNIST and ImageNet-100 datasets.

Critical strengths

(+) The problem investigated in this paper is interesting.

Critical weaknesses

(-) The paper is not well prepared and difficult to follow. (-) The contributions of this paper are not clearly presented and justified.

Questions for authors

See the weaknesses

2 comments
AuthorsOpenReview ↗

Reviewer es3D: Clarifying the Contributions and Supporting Evidence

We thank the reviewer for recognizing that the problem is interesting. We take the concerns about presentation and justification seriously. The central question can be stated more directly. When sliced regularization is used to pursue an isotropic Gaussian latent distribution, do random projections provide a sufficiently informative assessment of whether that objective has been achieved? The first contribution is the observation that random-direction evaluation can conceal larger discrepancies along directions populated by the encoder. This is not merely a difference between two arbitrary reporting metrics. Under an exact isotropic Gaussian target, every fixed unit direction should yield a standard normal projection. The sample disjoint controls in Appendix C preserve the observed directional discrepancy, and the isotropic Gaussian controls produce gaps close to zero. Together, these results support the relevance of checking directions beyond independent random projections.

The second contribution is a simple augmentation that improves agreement in these latent directions. Selecting a small number of radially atypical minibatch vectors and using their normalized directions reduces mean held-out latent-direction KS in all 16 matched comparisons. The improvement also persists in the sample disjoint evaluation. The remaining gap is acknowledged, so the paper does not claim to establish complete Gaussian matching.

The deterministic schedule is an exploratory component rather than the central positive result. Its lack of a consistent advantage helps separate the benefits of adaptation from the choice of random or rotating base directions. We would appreciate clarification on whether the informativeness concern is that these findings are already established, or that their practical significance requires stronger justification. Likewise, identifying a specific methodological concern underlying the soundness assessment would help us address it directly. The contribution we ask the reviewer to consider is the combination of a directional diagnostic finding and a simple intervention that partially addresses it.

Thanks to authors’ rebuttal. I checked the rebuttal and other reviewers’ comments. I keep my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.