acceptodds
Under review as a conference paper at ICLR 2027

GrassMLR: Rethinking Multinomial Logistic Regression on Grassmann Manifolds

Abstract

Grassmann manifolds represent data as low-dimensional subspaces, effectively capturing the intrinsic geometric structure of high-dimensional data, thus achieving significant success in geometric representation learning. However, existing approaches typically map Riemannian features to Euclidean spaces for final decision, which inevitably introduces semantic distortions and separates the decision process from the underlying geometry, thereby overlooking the geometric power embedded in the Riemannian space. To address this issue, we propose **Grass**mann **M**ultinomial **L**ogistic **R**egression **(GrassMLR)**, which performs the classification decision directly on the Grassmann manifold. Specifically, to align classification decisions with the intrinsic geometry of the Grassmann manifold, GrassMLR introduces two learnable Grassmann anchors for each class. For an input subspace, it computes the squared geodesic distances to the two anchors. A distance-difference function is then defined as the difference between these two squared geodesic distances. This formulation induces a decision hypersurface with explicit geometric meaning, thereby ensuring that classification decisions remain consistent with the underlying manifold geometry. Furthermore, to obtain representations better suited to GrassMLR, we propose a Quotient-Space Representation Learning (QSRL) strategy. In particular, we exploit the quotient-space invariance of the Grassmann manifold to apply a learnable Lie-group transformation to subspace representations, preserving the quotient space while obtaining more effective geometric representations. To validate the generalised potential of our design, we apply GrassMLR to multiple downstream tasks. Extensive experiments demonstrate that GrassMLR and QSRL work synergistically to improve the performance of Grassmann neural networks and obtain more discriminative geometric representations. These results validate the effectiveness of geometry-aware decision making and representation optimisation under quotient space invariance.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Grassmann manifold;Multinomial Logistic Regression;Quotient Space

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EWV1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Grassmann Multinomial Logistic Regression (GrassMLR) and a Quotient Space Representation Learning (QSRL) strategy, providing a geometry aware unified framework for classification decision making and representation learning on Grassmann manifolds. Specifically, GrassMLR constructs decision hypersurfaces based on the differences between squared geodesic distances from an input subspace to learnable Grassmann anchors, thereby ensuring that classification decisions remain consistent with the underlying manifold geometry. Furthermore, QSRL exploits the quotient space invariance of the Grassmann manifold to adaptively transform orthonormal basis representations while preserving the underlying subspaces and their equivalence classes, thereby obtaining more effective geometric representations for decision making. Overall, this work extends Grassmannian learning from geometry aware representation learning to the joint modelling of geometry aware decision making and quotient-space representation learning.

Critical strengths
  1. This paper proposes GrassMLR, which constructs a distance-difference function to define a decision hypersurface with explicit geometric guarantee on the Grassmann manifold, thereby enabling classification based on its intrinsic geometry.

  2. The ablation study clearly shows that both GrassMLR and QSRL contribute to the reported gains.

Critical weaknesses
  1. The motivation for directly transplanting the multinomial logistic regression (MLR) decision rule onto the Grassmann manifold is not sufficiently developed. The paper argues that Euclidean or tangent-space classifiers introduce geometric distortion and “semantic errors,” but it does not concretely characterise what these semantic distortions are in existing Grassmann pipelines, nor does it clearly demonstrate how GrassMLR resolves them beyond preserving geodesic distances. In particular, the connection between the distance-difference formulation and the claimed preservation of “intrinsic geometric semantics” remains largely asserted rather than empirically or theoretically isolated.

  2. The experimental evaluation is limited to three datasets, two action-recognition datasets and one hyperspectral dataset. GrassMLR does not achieve the best HDM05 accuracy, and the improvements on HDM05 are modest.

  3. The computational comparison reports model size and FLOPs, but not training time or memory. Since QSRL introduces additional gating and Cayley transformations, the practical training overhead should be reported.

Questions for authors

See Weaknesses.

Reviewer_NkUkOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a formulation of multinomial logistic regression on Grassmann manifolds based on geodesic distances to learnable anchor subspaces. To deal with ambiguity of representation in the quotient space, the paper applies a lie group transformation to the subspace representation.

Critical strengths

Grassmann manifolds appear in low rank learning problems such as PCA where the latent variables are a subspace. Geometric ML methods have shown some useful applications and this paper is positioned in an interesting space. The derivations appear to be sound.

Critical weaknesses

The paper is dense, not very self-contained and difficult to follow. Part of this has to do with some missing background information, and a lack of motivation or narrative around a main idea. The paper could do a better job motivating why classification on Grassmann manifolds is an important open problem before delving into text and equations. Grassmann manifolds have applications in low rank learning problems where the latent variables are a subspace. Most readers are probably not as familiar with what Grassmann manifolds add for deep learning based classification or regression problems. There are a lot of geometric ML papers that have come out proposing classifiers that operate on Riemannian manifolds, for example to leverage connections between hyperbolic space and tree embeddings or product manifolds and graph embeddings. What do you stand to gain by using a Grassman manifold classifier instead of a Euclidean classifier, what kind of data is it useful for and whats the intuition behind it? In my opinion the paper doesn't sell or address this.

Section 3 (background) describes MLR by defining a hyperplane using a point p_k and a normal vector a_k perpendicular to it, but Section 4.1 (GrassMLR) defines a hyperplane as the set of points equally far from two reference points A_k and B_k so the classifier becomes a test of which is closer. This is an old formulation that lets MLR be applied to manifolds, why not use it in Section 3 instead of the point and normal vector definition? Alternatively just review hyperbolic MLR (Ganea et al) in the background section. Don't they also use this formulation together with the scale parameter and offset? I'm not sure whats novel in 4.1. Either way, it's probably important to include hyperbolic MLR as well as SPDMLR and RMLR in the background section since this work clearly builds off of it.

Don't SPDMLR and RMLR define the classifier on the manifold and compute the logit in the tangent space? Fig 1 hints the motivation of the paper is that working in the tangent space causes distortion. I think it's worth framing the paper around this and spelling this out if its the conceptual heart of the paper. Where does tangent space distortion come from (the tangent space error itself, the meaning of logits?) and how much of a problem is it? Doesn't tangent space error depend on the manifold and the data? If the curvature differs for different Grassmann manifolds, then how does tangent space error depend on k and n? How is the data itself embedded into a Grassmann manifold? Do you just take the SVD or eigendecomposition of the data, then operate on that directly? I’d guess this is one of the most consequential modeling choices. The paper doesn't explore any of this. The results and ablations seem to show marginal improvements. But after reading the paper, I don't really have a takeaway for what is actually gained by using the logarithmic map to class anchor approach in the paper vs the prior work, when it's applicable or why its important. This is a missed opportunity.

In my opinion Fig 1 itself isn't doing too much work, I'd suggest replacing it with something that illustrates the Grassmann manifold intuitively (the half circle with endpoints glued or hemisphere picture of Gr(1,3), maybe something to show the antipodal points and why the quotient-space is needed).

Some notation/terms are undefined or inconsistent (lg^A_k and d_G(\cdot, \cdot) for geodesic distance in Eq 13, later Log_X(A_k) in Eq 16 and 17).

AI statement is missing

Questions for authors

There are some common desirable properties of classifiers, namely 1) continuity and connectedness (do splits create disconnected regions?), 2) Geodesic convexity (do geodesics cross boundaries within regions?) and 3) Equivariance or equal spacing (are distances between splits equal?) Do each of these hold up under the log map to class anchor approach? Do they depend on manifold curvature, and as a result the embedding method? If so, how?

Are there any theoretical advantages to the log map to class anchor approach vs the tangent space or prior work (GrNet, SPDMLR, RMLR)? Why the focus on multinomial logistic regression on Grassmann manifolds instead of other classification methods on Grassmann manifolds?

How is the data itself embedded into a Grassmann manifold? Do you just take the SVD or eigendecomposition of the data, then operate on that directly? I’d guess this is one of the most consequential modeling choices. How does the choice of the embedding method itself influence performance? Doesn’t it affect the manifold curvature and as a result the tangent space distortion error?

Riemannian optimization can be a bit cumbersome and numerically unstable. Does the quotient space formulation introduce any degeneracy for optimization? How does Riemannian optimization error affect the ability to learn the anchor subspaces, and as a result downstream performance? Is this studied or ablated?

Reviewer_qQCtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a geodesic based multi-class classification method for Grassmann manifolds. Specifically, each class has two class prototype subspaces, and the logic is defined based on the difference of the squared geodesic distances. This approach is an extension of the Euclidean case. In addition, a quotient aware strategy is used to further improve the convergence. Experiments on the action recognition and hyperspectral image classification datasets show that the proposed method improves the accuracy.

Critical strengths

The extension from Eucldiean logits to Grassmannian is natural, and looks reasonable.

Critical weaknesses
  1. One important related work GCR [a] is missing, where the class prototype is a subspace, and the logit is defined by the norm of the projection from feature vector onto the class subspace. This formulation is geometric without mapping to Euclidean space, and achieved SOTA accuracy on ImageNet-1K using ResNet. The only difference in setting is that in this paper, the feature is also Grassmannian. An interesting question: Is the two-prototype class formulation for the logit necessary? What if simply use one class prototype and define the logit as the geodesic distance between feature and class? It has been shown to work in GCR.
  2. Some techincal details are unclear: (a) are the two class prototypes learnable? How to compute their gradients? (b) Why whould the QSRL impact the optimization at all? The formulation is claimed to be pure geometric, then a concequence is that the numerical representation should all be equivalent, correct? So its a contradiction: if a rotation changes optimization, then the formulation cannot be geometric. Please provide more details how and why QSRL improves accuracy.

[a] Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation, ICCV 2023.

Questions for authors
  1. The transformation R(U) is generated adaptively of each input.

Please provide details on how to generate rotation for each input.

  1. How many SVDs are needed for each forward pass and backward pass, per class? I assume that the each geodesic distance would invoke several calls to SVD, and the Riemannian gradient also calls some. And the follow up question is, are these SVDs counted in the FLOPs statistics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.