acceptodds
Under review as a conference paper at ICLR 2027

Learning to Space and Score Ordinal Labels Across Vision and Touch

Abstract

Ordinal labels specify class order, but not the distances between classes. Yet ordinal soft supervision must choose both how target probability is distributed across neighboring classes and how the resulting ordered distributions are scored. We make these two choices explicit. First, we formulate cumulative ordinal training as proper scoring over nested threshold events, connecting the commonly used squared cumulative objective (/RPS-type scoring) to cumulative Brier scoring and CDF-CE to cumulative logarithmic scoring. We prove cumulative strict propriety, derive their distinct cross-boundary gradient scaling, and show that soft and hard cumulative targets induce different probability-matching geometries. Second, we replace fixed equal spacing with a lightweight learnable supervision geometry in which positive gaps define strictly ordered, fixed-span class positions , optionally with a single global softness parameter . Across five benchmarks spanning vision and touch, Gaussian soft targets improve the mean task-primary metric over hard targets for every dataset under CE, , and CDF-CE. Under Gaussian supervision, CDF-CE improves over on all five tasks, whereas the ordering reverses on four of five under hard targets. Learning the target geometry further improves AADB, HCI, and PHAC-2, while the physically grounded equal spacing of Whisker remains preferable. Across the controlled target-and-scoring variants, a fixed or adaptive CDF-CE model achieves the best mean primary result on every benchmark. Together, these results show that ordinal soft supervision depends jointly on target construction and scoring.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
ordinal regression, soft labels, label distribution learning, proper scoring rules

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mDxnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

this work tackles ordinal classification task (OC). authors argue that in OC, labels only reflects order between classes. they dont reflect the distance/real distance between classes in real life. authors consider this an issue. they propose to learn the gap between classes contrary to uniform gaps. this is done within a cumulative scoring framework that uses CDF-CE for learning using soft-labels. some results are reported on 5 benchmarks.

Critical strengths

the paper tackles an important task, OC. some results are reported.

Critical weaknesses

writing: the writing is very bad. poor fluidity in the text. lack of justifications (all sec.3 is given as facts. no justification.). abstract is very quick. no clear problem. immediate jump to what has been done in the paper. introduction is poorly written. no real problem is discussed. no coverage of literature. immediate jump to contributions. related work section contains poor discussion to existing works. fails to highlight the problem and gaps, and aligning this work with prior works. method section is a series of facts. not clear what is new or old. lacks justification. what is the parag in line 290?? what is M0-M5?

method:

  • "Ordinal labels specify class order, but not the distances between classes" is not accurate. OC contains order via value of the index of class which also allows measuring distance between classes. the distance is not real but it is scaled down (usually) to the real distance to not 'blow up' learning.
  • there is no real problem discussed in the paper. the real distance between classes is already encoded in the labels (given, most of the time). usually, this distance is uniform. one can use the real distance like in whisker example - fig.1- with delta=30 between classes. but this will lead to learning issues (large error magnitude). instead, 'fictive' classes with uniform distance of 1 is used. one can multiply the model output to get the real 'class'. contrary to what has been said in this paper, one can adjust the classes labels to reflect distance in real life between classes. e.g. 0, 1, 3, 5, 6, 7 could be an encoding of classes to say that the 4th classes is twice further to 3rd and 5th classes compared to the rest of deltas. this will reflect in training scoring error. this is to say when the distance between real classes is known, it can be injected in the class labels directly, creating order between classes and reflecting their distance, which is the whole point of OC. sometimes, distance between class is inaccessible like in PHAC-2 (it still could be measured in lab or known using textbooks). in this case, the classes are fictive. is it even possible to learn/guess the real distance between classes in this case? why it is relevant to do so? the order and distance here are fictive to the model. they only make sense to humans. for the model, the best interpretation could be that it tries to order materials (wood to glass, no notion of hardness). is it relevant to know that iron is "1 million" times harder than wood? some scaled down deltas could still be used to reflect the distances between materials (not real ones). learning distances, as done in the paper, could lead to contradictions w.r.t real distances or even their scaled version.
  • caption fig.1, line 087: "HCI has an equal decade coordinate, although equal temporal spacing need not imply equal visual or perceptual change". who and why make such assumption that spacing implies same effect?

experiments: the experimental section is extremely poor. there is no comparison to existing works or SOTA.

overall, the problem is poorly posed. the proposed method is given as facts. no comparison to existing works in terms of results.

please stop using ai.

Questions for authors

please see weaknesses.

2 comments

We thank the reviewer for the feedback. However, several criticisms appear to arise from a different interpretation of our formulation, and some do not match the original manuscript. We clarify these points below.

1. Ordinal labels, numerical distances, and soft-target construction

The reviewer argues that class indices already encode distances and that nonuniform coordinates such as (0,1,3,5,6,7)(0,1,3,5,6,7) can be assigned. We agree that known distances can be encoded this way. However, such coordinates specify a chosen metric; ordinal labels alone do not necessarily provide an appropriate one, particularly for subjective ratings.

More importantly, our paper studies soft-target construction, not simply numerical label encoding. Renumbering classes does not change categorical cross-entropy if their assignments remain unchanged. In contrast, soft ordinal targets explicitly depend on class distances:

qj(y)(z,τ)∝exp⁡[−(zj−zy)22τ2].q_j^{(y)}(z,\tau)\propto\exp\left[-\frac{(z_j-z_y)^2}{2\tau^2}\right].

Changing zz therefore changes the supervision distribution while preserving class order and the prediction architecture. This is central to SORD (Díaz and Marathe, 2019) and label-distribution learning, already discussed in our original Section 2.

Our method learns task-adapted relative spacing for ordinal supervision when suitable distances are not predefined, rather than attempting to recover uniquely identifiable physical distances.

2. The Whisker example: 10-mm intervals and categorical prediction

The review refers to a 30-mm spacing between Whisker classes. However, the original Figure 1 explicitly specifies Δd=10\Delta d=10 mm. The values 0, 30, 60, and 90 mm are reference ticks on the physical depth scale, not adjacent class intervals.

The concern about large numerical labels causing training difficulties also does not apply to our formulation. All M0–M5 variants use a KK-class categorical softmax head (original Section 3.1). Whisker depth is discretized into nine classes with fixed coordinates zj=jz_j=j, not millimeter-valued regression outputs. The adaptive variants modify relative spacing while preserving order and total span (Equations 11–13).

Whisker deliberately represents a task with known physical spacing. The results support preserving it: fixed-spacing M3 achieves 6.778 mm MAE versus 7.182 mm for M4 and 7.114 mm for M5 (original Table 1). This directly illustrates when fixed geometry is preferable.

3. The HCI example and the uniform-spacing assumption

The reviewer asks who assumes uniform spacing between ordinal classes. This assumption is explicitly used in established methods. Hou et al. (2016, Section 2.2.2) define the ordered-class ground distance as Dij=∣i−j∣D_{ij}=|i-j|. Similarly, SORD (Díaz and Marathe, CVPR 2019, Section 4.1) explicitly treats Adience age groups as equally spaced categories when constructing ordinal soft targets.

Both papers also consider alternative distance constructions: Hou et al. estimate a feature-based distance matrix for EMD regularization, while SORD allows predefined task-specific distance functions. Our work addresses a different question: how to learn ordered class coordinates for soft-target construction while separately analyzing their interaction with cumulative scoring.

For HCI, equal chronological spacing need not imply equal visual changes, exactly as stated in our original Figure 1 caption. This motivates relaxing the fixed-spacing assumption. In the original experiments, learned geometry improves HCI MAE-EV from 0.693 (M3) to 0.666 (M4).

4. PHAC-2 and the interpretation of learned spacing

The review suggests that our model may simply rank materials such as wood and glass or estimate their physical hardness ratios. However, PHAC-2 predicts five-level human ratings of haptic properties from tactile recordings (original Section 4.1), not material identities or physical hardness values.

Predicting a hardness rating of 4 instead of 3 does not imply any particular physical hardness ratio. Our learned geometry determines how training probability is distributed among ordinal rating levels; it does not assign physical distances to materials. This distinction is central to learning task-adapted supervision when a distance scale is not provided.

5. Theoretical justification and related work

The review describes Section 3 as a series of facts without justification. However, the original manuscript already includes Theorem 1 establishing cumulative strict propriety (proof in Appendix A), the cross-boundary gradient derivations in Equations (7)–(9) and Appendix B, and the order-preserving geometry construction with proofs in Appendix C.

In particular, Equation (9) establishes Gksq=2Pk(1−Pk)GklogG_k^{\mathrm{sq}}=2P_k(1-P_k)G_k^{\mathrm{log}}, explaining the different gradient behavior of cumulative squared and logarithmic scoring near saturated probabilities. Our contribution is not to introduce soft labels or cumulative losses as entirely new concepts, but to analyze their interaction and separate target construction from scoring under controlled conditions.

The original Section 2 also covered ordinal-output methods (Niu et al., CORAL, CORN), soft-label and label-distribution approaches (SORD, TactONet, SALDL, OLDL), and cumulative scoring theory (Hou et al., RPS/CRPS, proper scoring rules). The revision further clarifies their relationships and how our approach differs.

6. Experimental comparisons

The review states that the paper contains no comparisons with existing approaches. The original Tables 1–2 already provided controlled comparisons of hard versus Gaussian soft targets under categorical CE, cumulative squared scoring, and CDF-CE across five vision and tactile benchmarks, using matched training protocols.

These experiments reveal the central target–scoring interaction: under Gaussian targets, CDF-CE improves the mean primary metric over cumulative squared scoring on all five datasets, whereas under hard targets the ordering reverses on four of five.

We acknowledge that direct numerical comparisons with specialized ordinal-output methods were absent from the original submission. We have therefore added CORAL and CORN across all five benchmarks using matched backbones, splits, and training budgets (revised Table 1; Appendix H), and explicitly identified M1 as Gaussian SORD.

For example, on HCI our M4 achieves 0.666 MAE-EV versus 0.702 for CORAL and 0.747 for CORN; on PHAC-2, M5 achieves 0.593 MAE$^M$ versus 0.664 for CORN. Fixed-Gaussian CDF-CE (M3) also improves over hard CE (M0) in mean primary performance on all five datasets. The results demonstrate that ordinal structure can be incorporated effectively through supervision and scoring without requiring a specialized prediction head.

7. Presentation and notation

We recognize that the original presentation could be more polished, particularly in introducing the method variants. The paragraph near line 290 referred to M0–M5 before their formal definition in Section 4.2. We agree that this ordering was confusing.

In the revision, Section 4.2 introduces M0–M6 before the experimental results, the detailed M6 formulation is moved to Appendix D, and Appendix E summarizes the method taxonomy. We have also substantially revised the abstract, introduction, related work, and method exposition to improve clarity, motivation, and overall flow.

These changes strengthen the presentation while preserving the theoretical analysis, literature foundations, and controlled experiments already present in the original submission.

We hope these clarifications address the concerns raised and welcome further specific technical feedback on the formulation, theoretical analysis, or experimental comparisons.

Reviewer_gZedOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies soft supervision for ordinal classification, and separates it into two choices: how the soft target is distributed across neighboring classes, and how the predicted and target distributions are scored. It formulates the commonly used squared cumulative loss and CDF-CE as cumulative Brier and logarithmic proper scores over the same ordinal thresholds, and proposes a learnable target geometry in which the class positions are learned jointly with the network. Across five vision and tactile benchmarks, Gaussian soft targets improve over hard targets, CDF-CE improves over the squared cumulative loss under soft targets, and the learned geometry further improves the task performance.

Critical strengths
  1. Clear and controlled analysis of ordinal supervision: The paper separates ordinal soft supervision into target construction and scoring, and compares them with a full factorial design under the experiment configurations. The proper-scoring view connects the commonly used squared cumulative loss with CDF-CE, and the gradient analysis gives an explanation of why the two scores behave differently under soft and hard targets. This gives a clear and simple recipe for training ordinal classifiers.

  2. Consistent benefit of soft targets across diverse tasks: Gaussian soft targets improve over hard targets on all five benchmarks under all three scorings, even though the tasks cover very different label types, including aesthetic ratings, age groups, historical decades, haptic ratings and physical depth. The proposed learnable geometry is also lightweight, since it only adds a few parameters and guarantees ordered classes and unimodal targets.

Critical weaknesses
  1. The improvements are small and not well supported statistically: Most of the claims are based on counting which mean is larger across datasets. However, many of these differences are smaller than the reported standard deviations, and the paired analysis in Appendix G shows that the confidence intervals of CDF-CE versus EMD include zero on four of the five datasets. The gains from learned geometry are similarly small, and no paired test is reported for them. In addition, the paper does not compare with existing ordinal methods such as CORAL, CORN or unimodal output distributions, so it is hard to tell how the proposed recipe performs relative to prior work.

  2. The learned geometry is not clearly justified: The paper argues that learning the class positions helps when the class spacing is not physically defined, and explains the drop on Whisker by its uniform physical bins. However, Adience has very irregular age groups and a large learned deviation from equal spacing, but its accuracy still decreases with the learned geometry. Moreover, since CDF-CE contains the entropy of the target (Eq. 10), learning the target jointly with the network can reduce the loss simply by making the targets sharper, which is consistent with the learned softness becoming smaller on all five datasets. This makes it unclear whether the learned positions reflect a meaningful class geometry or mainly an artifact of the objective.

Questions for authors

Corresponding to the weakness points above, I have two major questions:

  1. Would it be possible to report paired confidence intervals for all main comparisons, including the learned geometry variants, and to add existing ordinal methods such as CORAL, CORN or unimodal output distributions under the same pipeline? If the proposed CDF-CE and learned geometry still show significant improvements, it would support the claim that target construction and scoring are the key components of ordinal soft supervision.

  2. Would it be possible to compare the learned geometry with a fixed Gaussian target whose softness is tuned on the validation set, and to analyze how the learned gaps relate to the class frequency? If the learned geometry still outperforms the tuned fixed target and does not simply follow the class frequency, it would support the claim that the learned positions capture a meaningful supervision geometry.

2 comments

We thank the reviewer for recognizing the value of the controlled factorial design and its proper-scoring interpretation. In response to your questions, we have expanded the paired statistical analysis, added established ordinal baselines, and conducted further experiments on Gaussian softness and learned geometry.

Q1. Paired comparisons and ordinal baselines

We conducted additional paired bootstrap analyses for soft supervision (M1 versus M0), cumulative scoring (M3 versus M2), and learned geometry (M4/M5 versus M3). Each comparison uses matched seeds, folds, or splits, with two-way adjective–fold resampling for PHAC-2. Positive values favor the second method after accounting for metric direction. The intervals below are nominal 95% confidence intervals.

DatasetM1 over M0M3 over M2M4 over M3M5 over M3
AADB (Spearman ↑)+.0263 [.0169,.0358]+.0021 [−.0062,.0122]+.0023 [−.0043,.0089]−.0049 [−.0187,.0080]
Adience (Acc. ↑)+.0376 [.0208,.0544]+.0041 [−.0006,.0113]−.0056 [−.0139,.0022]−.0084 [−.0147,−.0037]
HCI (MAE-EV ↓)+.0306 [−.0030,.0672]+.0246 [−.0130,.0576]+.0271 [.0108,.0448]+.0043 [−.0212,.0308]
PHAC-2 (MAE^M ↓)+.0567 [−.0059,.1219]+.0105 [−.0069,.0261]+.0135 [−.0040,.0380]+.0256 [−.0062,.0600]
Whisker (mm ↓)+.7852 [.5595,1.0117]+.2862 [−.1392,.7409]−.4047 [−.5036,−.3212]−.3361 [−.4319,−.2403]

The soft-target comparison (M1 versus M0) yields positive paired intervals on AADB, Adience, and Whisker. Learned geometry (M4) also achieves a positive paired improvement on HCI, while Whisker favors its physically defined spacing. With identical Gaussian targets, CDF-CE (M3) improves the mean primary metric over EMD² (M2) on all five benchmarks. To further assess the scoring effect, we report paired comparisons on complementary ordinal metrics (revised Appendix G, Table 5):

DatasetSecondary metricM3 over M2 [95% CI]
AADBMAE-EV ↓+.0136 [−.0063,.0350]
AdienceOne-off accuracy ↑+.0070 [.0046,.0096]
HCIQWK ↑+.0436 [.0129,.0730]
PHAC-2Accuracy ↑+.0310 [.0035,.0673]
WhiskerQWK ↑+.0567 [−.0038,.1359]

All five complementary metrics favor CDF-CE in mean, with positive nominal paired 95% intervals on Adience, HCI, and PHAC-2. Together with the controlled factorial comparisons, these results provide further evidence that the relative performance of cumulative scoring rules depends on the target distribution.

We also added CORAL and CORN across all five benchmarks, using matched task-specific backbones, splits, and training budgets while retaining their ordinal heads and objectives (revised Table 1; Appendix H). On Adience, CORAL uses its native threshold-count decoder with validation-based checkpoint selection. We further clarify in Sections 3.1 and 4.2 that M1 is a Gaussian instance of SORD, while M2 and M3 apply different cumulative scores to its unchanged targets. M6 provides an additional direct monotone-CDF output-representation control.

On HCI, M4 achieves 0.6659 MAE-EV versus 0.7018 for CORAL and 0.7475 for CORN. On PHAC-2, M5 reaches 0.5926 MAE$^M$ versus 0.6643 for CORN, while M4 and CORAL achieve similar AADB Spearman correlations (0.6784 and 0.6768).

The target-and-scoring approach also performs consistently across the five benchmarks: fixed-Gaussian CDF-CE (M3) improves the mean primary metric over hard CE (M0) on every dataset, whereas CORAL performs below M0 on Adience and PHAC-2, and CORN performs below M0 on HCI. These comparisons demonstrate the broad applicability of incorporating ordinal structure through supervision and scoring while retaining a standard categorical softmax head.

(We address the Gaussian-softness and geometry diagnostics below)

Q2. Validation-tuned Gaussian softness, target sharpening, and class frequency

We conducted an additional HCI experiment to examine whether the improvement from learned geometry could be achieved by tuning global Gaussian softness (Section 4.5; Appendix H). For each of the ten HCI splits, M3 selects τ0∈{0.25,0.5}\tau_0\in\{0.25,0.5\} using validation MAE-EV. The resulting held-out MAE-EV is 0.69190.6919, compared with 0.66590.6659 for M4 with learned class spacing. The paired improvement is 0.02600.0260, with a 95% bootstrap CI of [0.0029,0.0512][0.0029,0.0512]. Thus, learned geometry retains its advantage over a validation-tuned fixed Gaussian target.

We also examined whether this improvement could be explained by target sharpening (Section 4.5; Appendix H). Under the original matched-softness HCI setting, the training-frequency-weighted mean cumulative Bernoulli target entropy is nearly identical for M3 (0.139180.13918) and M4 (0.139100.13910), despite their difference in prediction error. This indicates that the HCI improvement is not simply associated with a reduction in overall target entropy.

To assess the influence of class frequency on learned geometry, we conducted an additional Adience experiment applying class-frequency balancing specifically to the geometry updates, while leaving the prediction network training unchanged. Although balancing changes the learned class spacing, the resulting accuracy remains similar to the original M4 (0.6164 versus 0.6148). This suggests that predictive performance is not particularly sensitive to class-frequency weighting in geometry optimization.

We additionally conducted a six-condition PHAC-2 experiment using empirical human-rating distributions as supervision (Section 4.5; Appendix J, Table 7). Each object–adjective pair has multiple human ratings, allowing its empirical distribution to be used directly instead of a Gaussian target centered at the median. We compared the two target constructions under CE, EMD², and CDF-CE, using the same BioTac backbone, ten adjectives, and five object-disjoint folds per adjective.

Empirical human targets reduce MAE$^M$ under all three objectives: CE (0.6365→0.57730.6365\to0.5773), EMD² (0.6377→0.58780.6377\to0.5878), and CDF-CE (0.6204→0.58030.6204\to0.5803). With empirical targets, CDF-CE achieves the highest accuracy (0.65330.6533) and lowest CDF-MAE (0.12520.1252) and PMF-MAE (0.10660.1066).

Together, these experiments show that learned class spacing provides benefits beyond tuning global Gaussian softness on HCI, while empirical rating distributions offer an effective alternative when richer ordinal supervision is available.

Reviewer_Sh5vOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper analyzes ordinal soft supervision by separating target construction from cumulative scoring. It shows that common cumulative losses can be understood as proper scoring rules over ordinal threshold events, and that hard and soft targets lead to different optimization behavior. The paper also proposes a lightweight way to learn the spacing between ordered classes when equal spacing is not appropriate. Experiments across multiple vision and tactile benchmarks suggest that soft ordinal targets are generally beneficial.

Critical strengths

The paper cleanly separates target geometry, cumulative representation, scoring rule, and output representation. This is consequential because many ordinal-label papers conflate these choices; the paper’s factorial design makes it much easier to attribute gains to soft targets versus cumulative scoring.

Critical weaknesses

On AADB and Adience, the gains are very small; the appendix notes that paired intervals overlap zero on several datasets. The main controlled comparison intentionally keeps a shared softmax architecture, which is useful scientifically, but it also means the paper does not fully establish competitiveness against stronger ordinal-regression methods.

Questions for authors

1.How sensitive are M4/M5 to lambda_z, lambda_t, R, and t0? Would the conclusion about when to learn geometry change under a modest hyperparameter sweep? 2. Can you include comparisons or at least contextual numbers against representative ordinal-output methods or recent label-distribution methods under comparable backbones? 3.When should practitioners prefer fixed geometry over learned geometry? The Whisker results suggest that physically grounded equal spacing should be preserved. Could the authors provide clearer guidance or diagnostics for deciding when geometry learning is appropriate?

2 comments

We thank the reviewer for recognizing the value of our factorial design in separating target geometry, cumulative representation, scoring rules, and output representation. We appreciate the constructive suggestions and have conducted additional sensitivity analyses, evaluated established ordinal methods, and expanded the experiments on target geometry.

Q1. Sensitivity to λz\lambda_z, λτ\lambda_\tau, RR, and τ0\tau_0

We conducted additional sensitivity analyses for λz\lambda_z, λτ\lambda_\tau, and τ0\tau_0 (Appendix I, Table 6). Across ten HCI splits, the original λz=0.1\lambda_z=0.1 for M4 and λτ=1\lambda_\tau=1 for M5 achieve the lowest mean validation errors among their respective tested settings. This supports selecting the regularization coefficients using validation performance.

With λz=0.1\lambda_z=0.1, M4 achieves 0.66590.6659 test MAE-EV on HCI, outperforming fixed-Gaussian M3 even after its reference softness is selected using validation (0.69190.6919; Section 4.5 and Appendix H). The advantage of learned geometry therefore remains under the validation-tuned fixed-Gaussian comparison.

On PHAC-2, M4 achieves MAE$^M$ of 0.61360.6136 and 0.60470.6047 at τ0=0.5\tau_0=0.5 and 11, respectively. For M5, we use R=2R=2, allowing τ/τ0∈(0.5,2)\tau/\tau_0\in(0.5,2); the observed learned ratios lie within approximately [0.898,0.995][0.898,0.995], showing no tendency toward either bound. Together, these analyses support choosing the regularization and reference softness on validation data. The complete sweep results are reported in Appendix I.

Q2. Comparisons with established ordinal methods

We added CORAL and CORN to the evaluation across all five datasets, using the same dataset-specific backbones, data partitions, and training budgets while retaining their respective ordinal heads and objectives (Table 1 and Appendix H). Both baselines were evaluated over five AADB seeds, five Adience folds, ten HCI splits, fifty PHAC-2 adjective–fold runs, and five Whisker seeds.

We also clarify the connection to existing soft-label methods in Sections 3.1 and 4.2. M1 is a squared-index Gaussian instance of SORD (Díaz and Marathe, 2019), trained with categorical cross-entropy. M2 and M3 retain the same SORD-style targets while changing only the cumulative scoring objective, allowing a controlled comparison between squared and logarithmic scoring.

The task-primary means are summarized below; Table 1 additionally reports standard deviations. Bold indicates the best M0–M5 mean in each column. M6 is a separate output-representation control.

MethodAADB ρ↑\rho\uparrowAdience Acc.↑\uparrowHCI MAE-EV$\downarrow$PHAC-2 MAE$^M\downarrow$Whisker mm$\downarrow$
M0 Hard CE.648.578.715.6897.928
M1 Gaussian SORD.675.616.685.6327.143
M2 Gaussian EMD².674.616.718.6297.064
M3 Gaussian CDF-CE.676.620.693.6186.778
M4 Learned positions.678.615.666.6057.182
M5 Learned positions and softness.671.612.689.5937.114
M6 Direct monotone CDF.682.605.690.5947.293
CORAL.677.430.702.8687.443
CORN.648.587.747.6647.170

On HCI, M4 achieves 0.66590.6659 MAE-EV compared with 0.70180.7018 for CORAL and 0.74750.7475 for CORN. On PHAC-2, M5 achieves 0.59260.5926 MAE$^M$ compared with 0.66430.6643 for CORN. On AADB, M4 and CORAL achieve similar Spearman correlations (0.67840.6784 and 0.67680.6768). Importantly, our target-and-scoring approach also shows more consistent performance across benchmarks. Fixed-Gaussian CDF-CE (M3) improves over hard CE (M0) on all five datasets, whereas CORAL and CORN exhibit substantial performance degradation on certain tasks. For instance, CORAL reduces Adience accuracy from 0.5780.578 to 0.4300.430 and increases PHAC-2 MAE$^M$ from 0.6890.689 to 0.8680.868 relative to M0, while CORN increases HCI MAE-EV from 0.7150.715 to 0.7470.747. These results suggest that incorporating ordinal structure through soft targets and cumulative scoring provides a more consistent improvement across diverse tasks while retaining a standard categorical softmax head.

(We address the reviewer’s observation about the size and statistical support of the AADB and Adience improvements below.)

Regarding the AADB and Adience results, the factorial comparison distinguishes improvements from soft supervision itself from the additional effect of cumulative scoring.

Under categorical cross-entropy, replacing hard labels (M0) with Gaussian SORD targets (M1) improves AADB Spearman from 0.6480.648 to 0.6750.675 (paired improvement +0.0263+0.0263, 95% CI [0.0169,0.0358][0.0169,0.0358]) and Adience accuracy from 0.5780.578 to 0.6160.616 (+0.0376+0.0376, CI [0.0208,0.0544][0.0208,0.0544]). The complementary metrics also improve: AADB MAE-EV decreases from 1.3111.311 to 1.2281.228, and Adience one-off accuracy increases from 0.9140.914 to 0.9550.955 (Appendix K, Tables 8–9).

When the Gaussian target is held fixed (M2 versus M3), CDF-CE improves the mean task-primary metric across all five datasets. We additionally report paired comparisons for complementary metrics, which provide further evidence for the scoring choice (Appendix G, Table 5).

DatasetSecondary metricM2 → M3 improvementPaired 95% CI
AADBMAE-EV ↓+.0136[−.0063, .0350]
AdienceOne-off accuracy ↑+.0070[.0046, .0096]
HCIQWK ↑+.0436[.0129, .0730]
PHAC-2Accuracy ↑+.0310[.0035, .0673]
WhiskerQWK ↑+.0567[−.0038, .1359]

The complementary means favor CDF-CE on all five benchmarks, with positive nominal paired 95% intervals on Adience, HCI, and PHAC-2. These results complement the primary-metric analysis and clarify the respective contributions of soft targets and cumulative scoring. Appendix G reports the corresponding primary and secondary paired comparisons.

Q3. When should fixed or learned geometry be preferred?

We expanded the geometry analysis in Section 4.5 and conducted a new six-condition PHAC-2 experiment using empirical human-rating distributions (Appendix J, Table 7). Together, the HCI, Whisker, and PHAC-2 results illustrate three situations for constructing ordinal supervision.

For Whisker, the label spacing has a predefined physical interpretation, with penetration depth discretized into uniform 10-mm intervals. Fixed-spacing M3 achieves 6.7786.778 mm MAE, compared with 7.1827.182 for M4 and 7.1147.114 for M5. This supports preserving externally defined class spacing.

For HCI, labels represent equally spaced decades, but the visual differences between consecutive decades need not be uniform. Learned geometry (M4) achieves 0.66590.6659 MAE-EV, compared with 0.69190.6919 for fixed-spacing M3 after selecting its Gaussian width from τ0∈{0.25,0.5}\tau_0\in\{0.25,0.5\} using validation data. The paired 95% bootstrap interval for the error reduction is [0.0029,0.0512][0.0029,0.0512] (Section 4.5 and Appendix H). Thus, learning class spacing offers an advantage beyond selecting a single global Gaussian width.

For PHAC-2, we added an experiment using empirical target distributions constructed from multiple human ratings for each object–adjective pair. We compared Gaussian and empirical targets under CE, EMD², and CDF-CE, using ten adjectives and five object-disjoint folds per adjective. Replacing Gaussian targets with human-rating distributions reduces MAE$^M$ under all three objectives: CE (0.6365→0.57730.6365\to0.5773), EMD² (0.6377→0.58780.6377\to0.5878), and CDF-CE (0.6204→0.58030.6204\to0.5803). With empirical targets, CDF-CE achieves the highest accuracy (0.65330.6533) and lowest distributional errors (CDF-MAE 0.12520.1252; PMF-MAE 0.10660.1066), while CE achieves the lowest MAE$^M$ (0.57730.5773). The complete comparison is reported in Appendix J, Table 7.

These findings suggest retaining meaningful physical spacing when available, using empirical ordinal rating distributions when provided, and learning ordered class positions when suitable relationships for soft-target construction are not otherwise specified.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.