acceptodds
Under review as a conference paper at ICLR 2027

AOBE: Anchored Order-Isomorphic Bi-Lipschitz Embeddings

Abstract

A representation that faithfully encodes a partially monotonic input order should preserve prescribed comparisons within fixed-context fibers without introducing unsupported comparisons between inputs that are incomparable under that order. Forward partial monotonicity alone does not ensure this: it permits input collapse, comparability between incomparable inputs, and uncontrolled metric distortion. We introduce anchored order-isomorphic bi-Lipschitz embeddings (AOBE), constructed by concatenating a coordinate-wise anchor with bounded positive scales and a norm-controlled partially monotonic network. The anchor retains a positively scaled copy of every direction-aligned constrained coordinate and a signed pair of every context coordinate. For every admissible embedding parameter value, the embedding is an exact order isomorphism from the conditional product order onto its image, is injective, and satisfies parameter-uniform bi-Lipschitz bounds determined by fixed hyperparameters. Across six benchmarks spanning binary classification, regression, and ordinal classification, AOBE achieves competitive predictive performance. In a parameter-matched representation comparison, AOBE is the only evaluated embedding that combines zero audited order-preservation and false-comparability violation rates with positive normalized incomparability margins and a parameter-uniform bi-Lipschitz guarantee. Frozen-channel readouts support task-dependent utility of the nonlinear features, while direct-collapse and task-output audits examine input separation and bounded prediction changes. These results demonstrate that faithful order representation and geometric stability can be enforced while retaining predictive utility.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
monotonic neural networks, order-isomorphic embeddings, bi-Lipschitz representations, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_iVsUOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  1. This paper proposes AOBE, an anchored representation for partially monotonic data that preserves and reflects the conditional input order.

  2. The authors establish that AOBE is injective and satisfies parameter-uniform bi-Lipschitz bounds, while its readout guarantees monotone and Lipschitz predictions.

  3. Experiments on benchmarks show competitive predictive performance and strong order-preservation properties.

Critical strengths

The paper provides strong theoretical guarantees for faithful order-preserving representations while maintaining competitive predictive performance across diverse tasks.

Critical weaknesses
  1. The main guarantees largely follow from explicitly copying and sign-duplicating the input coordinates, making the technical novelty of the learned representation relatively limited.

  2. The empirical evaluation covers only six tabular datasets, leaving the scalability and practical value of AOBE on high-dimensional or large-scale problems unclear.

  3. AOBE does not consistently outperform existing methods, so the practical benefit of its additional structural constraints remains insufficiently demonstrated.

  4. The paper is overly exhaustive and formulaic, with many repetitive audits and claims but limited discussion of when exact order isomorphism is actually necessary in real applications.

Questions for authors
  1. Since the main guarantees rely heavily on copying and sign-duplicating input coordinates, what is the key technical novelty beyond this explicit anchor construction?

  2. How does AOBE scale to high-dimensional or large-scale datasets, particularly given the duplicated context coordinates?

  3. Given that AOBE does not consistently outperform existing methods, in which practical scenarios do its structural guarantees provide clear benefits?

  4. Could the authors provide real-world examples where exact order isomorphism is necessary, rather than standard monotonicity or approximate order preservation?

Reviewer_XXa4OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

My main understanding of the paper after a few iterations is the following.

The computational problem is the following, define an architecture that maps items from some initial space RdM+dU\mathbb{R}^{d_M+d_U} to some representation space Rdz\mathbb{R}^{d_z}. Functions from the architecture are denoted zθz_{\theta}, where θ\theta is the set of learnable parameters of the model. We now describe the desired properties that the authors want the architecture to have. The most important property is that the embedding must respect the coordinate wise order of certain subset coordinates but only when the other coordinates are equal, we explain this in the sequel. The coordinates of the input space are divided into two sets: for x∈RdM+dUx\in\mathbb{R}^{d_M+d_U}, we write x=(xM,xU)x=(x_M,x_U), where xMx_M are the "constrained coordinates" and xUx_U are the "unrestricted context coordinates". The embedding is required to have the property that for x,x′∈RdM+dUx,x'\in\mathbb{R}^{d_M+d_U} it holds that zθ(x)≤zθ(x′)z_{\theta}(x)\leq z_{\theta}(x') if and only if xM≤xM′x_M\leq x'_M and xU≤xU′x_U\leq x'_U (here we use ≤\leq to denote coordinate wise inequality). Furthermore, the embedding is required to be injective and bi-Lipschitz. A way that this can be achieved is to take the embedding to be z(x)=(x_M, x_U, -x_U). The paper presents a more general version of that embedding where xMx_M and xUx_U are multiplied with diagonal matrices with positive entries, an additional non linear embedding that does not affect things much is also appended. The architecture is also evaluated experimentally.

Critical strengths

The strongest part of the paper, to me, appears to be the experimental part where at in some datasets it appears to outperform other methods.

Critical weaknesses

As mentioned earlier the paper is very poorly written to the extent of it being almost incomprehensible. The proofs are straightforward so I don't see any technical contribution that the paper has. In the experiments, the methods slightly outperforms existing architectures in certain cases, this is not surprising since the new architecture does not deviate much from architectures known for decades.

It is hard to see any benefit of accepting this paper.

Questions for authors

I don't think I have any questions that would affect my assessment.

Reviewer_khk2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AOBE, an embedding for partially monotonic problems. It places a scaled copy of the input, plus a +/− copy of the context features, next to a norm-bounded monotone network. The authors prove that the embedding keeps and reflects the input order exactly, is injective, and has fixed upper and lower distance bounds. They add a monotone readout on top and test it on six small tabular datasets covering classification, regression and ordinal tasks.

Critical strengths
  1. The idea is simple, and the proofs are easy to follow. I went through them, and they look correct. I also like that the bounds hold for any trained weights.
  2. The experiments are done with care. The code is shared, the configurations are fixed in advance, the models are matched in size, and the ablations make clear what each part contributes.
Critical weaknesses
  1. All experiments use small tabular datasets (some have fewer than 400 samples) and models with about 9k parameters. The paper follows the LMN setup but leaves out its two larger tasks, Loan Defaulter and ChestXRay. I would like to see results for these, with deeper and wider networks. The upper bound is a product over layers, so it would also help to show how accuracy varies as the network deepens.
  2. The zero OPVR and FCR values are guaranteed by the design, and the baselines were never built to avoid false comparisons. So these audits mostly confirm the construction rather than test it. The paper says that faithful order and stable geometry are useful, but I could not find an experiment that uses the embedding itself, for example, in retrieval, transfer, fairness, or robustness to adversarial noise or distribution shift. On prediction, the gains are small: the average rank is 3.00 against 3.33 for LMN, AOBE is fifth on two datasets, and there are no significance tests.
  3. Some comparisons are missing: • BLNN/PBLNN are the closest prior works, but they are not in the numerical results. • A simple baseline would help: take LMN or SMNN and add the same scaled copy of the input. If that does just as well, the benefit comes from the anchor alone. • I would also like a sensitivity study for d̲, d̄, γℓ and γs, and a try with activations other than tanh. • Runtime and memory numbers are needed, since the context features are copied twice, and BlogFeedback has 268 of them.
Questions for authors
  1. Is there a task where the embedding itself, and not just the score, is used, and the guarantees make a visible difference? Nearest-neighbour search, counterfactuals or robustness would all work.
  2. What values of d̲, d̄, γℓ and γs did you end up using? I couldn’t find them in the paper. How sensitive are the results to them? A very small d̲ would make the lower bound almost meaningless, so this matters.
  3. How tight is the upper bound in practice? Comparing the product of the γℓ with the largest distance ratio you actually observe, at a few depths, would answer this.
  4. How much extra time and memory does the doubled context cost compared with LMN, for example, on BlogFeedback?
  5. BLNN/PBLNN has public code, even if its authors call it a skeleton. Could you run it on at least a few of the datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.