- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
this work tackles ordinal classification task (OC). authors argue that in OC, labels only reflects order between classes. they dont reflect the distance/real distance between classes in real life. authors consider this an issue. they propose to learn the gap between classes contrary to uniform gaps. this is done within a cumulative scoring framework that uses CDF-CE for learning using soft-labels. some results are reported on 5 benchmarks.
Critical strengths
the paper tackles an important task, OC. some results are reported.
Critical weaknesses
writing: the writing is very bad. poor fluidity in the text. lack of justifications (all sec.3 is given as facts. no justification.). abstract is very quick. no clear problem. immediate jump to what has been done in the paper. introduction is poorly written. no real problem is discussed. no coverage of literature. immediate jump to contributions. related work section contains poor discussion to existing works. fails to highlight the problem and gaps, and aligning this work with prior works. method section is a series of facts. not clear what is new or old. lacks justification. what is the parag in line 290?? what is M0-M5?
method:
- "Ordinal labels specify class order, but not the distances between classes" is not accurate. OC contains order via value of the index of class which also allows measuring distance between classes. the distance is not real but it is scaled down (usually) to the real distance to not 'blow up' learning.
- there is no real problem discussed in the paper. the real distance between classes is already encoded in the labels (given, most of the time). usually, this distance is uniform. one can use the real distance like in whisker example - fig.1- with delta=30 between classes. but this will lead to learning issues (large error magnitude). instead, 'fictive' classes with uniform distance of 1 is used. one can multiply the model output to get the real 'class'. contrary to what has been said in this paper, one can adjust the classes labels to reflect distance in real life between classes. e.g. 0, 1, 3, 5, 6, 7 could be an encoding of classes to say that the 4th classes is twice further to 3rd and 5th classes compared to the rest of deltas. this will reflect in training scoring error. this is to say when the distance between real classes is known, it can be injected in the class labels directly, creating order between classes and reflecting their distance, which is the whole point of OC. sometimes, distance between class is inaccessible like in PHAC-2 (it still could be measured in lab or known using textbooks). in this case, the classes are fictive. is it even possible to learn/guess the real distance between classes in this case? why it is relevant to do so? the order and distance here are fictive to the model. they only make sense to humans. for the model, the best interpretation could be that it tries to order materials (wood to glass, no notion of hardness). is it relevant to know that iron is "1 million" times harder than wood? some scaled down deltas could still be used to reflect the distances between materials (not real ones). learning distances, as done in the paper, could lead to contradictions w.r.t real distances or even their scaled version.
- caption fig.1, line 087: "HCI has an equal decade coordinate, although equal temporal spacing need not imply equal visual or perceptual change". who and why make such assumption that spacing implies same effect?
experiments: the experimental section is extremely poor. there is no comparison to existing works or SOTA.
overall, the problem is poorly posed. the proposed method is given as facts. no comparison to existing works in terms of results.
please stop using ai.
Questions for authors
please see weaknesses.
2 comments
5. Theoretical justification and related work
The review describes Section 3 as a series of facts without justification. However, the original manuscript already includes Theorem 1 establishing cumulative strict propriety (proof in Appendix A), the cross-boundary gradient derivations in Equations (7)–(9) and Appendix B, and the order-preserving geometry construction with proofs in Appendix C.
In particular, Equation (9) establishes , explaining the different gradient behavior of cumulative squared and logarithmic scoring near saturated probabilities. Our contribution is not to introduce soft labels or cumulative losses as entirely new concepts, but to analyze their interaction and separate target construction from scoring under controlled conditions.
The original Section 2 also covered ordinal-output methods (Niu et al., CORAL, CORN), soft-label and label-distribution approaches (SORD, TactONet, SALDL, OLDL), and cumulative scoring theory (Hou et al., RPS/CRPS, proper scoring rules). The revision further clarifies their relationships and how our approach differs.
6. Experimental comparisons
The review states that the paper contains no comparisons with existing approaches. The original Tables 1–2 already provided controlled comparisons of hard versus Gaussian soft targets under categorical CE, cumulative squared scoring, and CDF-CE across five vision and tactile benchmarks, using matched training protocols.
These experiments reveal the central target–scoring interaction: under Gaussian targets, CDF-CE improves the mean primary metric over cumulative squared scoring on all five datasets, whereas under hard targets the ordering reverses on four of five.
We acknowledge that direct numerical comparisons with specialized ordinal-output methods were absent from the original submission. We have therefore added CORAL and CORN across all five benchmarks using matched backbones, splits, and training budgets (revised Table 1; Appendix H), and explicitly identified M1 as Gaussian SORD.
For example, on HCI our M4 achieves 0.666 MAE-EV versus 0.702 for CORAL and 0.747 for CORN; on PHAC-2, M5 achieves 0.593 MAE$^M$ versus 0.664 for CORN. Fixed-Gaussian CDF-CE (M3) also improves over hard CE (M0) in mean primary performance on all five datasets. The results demonstrate that ordinal structure can be incorporated effectively through supervision and scoring without requiring a specialized prediction head.
7. Presentation and notation
We recognize that the original presentation could be more polished, particularly in introducing the method variants. The paragraph near line 290 referred to M0–M5 before their formal definition in Section 4.2. We agree that this ordering was confusing.
In the revision, Section 4.2 introduces M0–M6 before the experimental results, the detailed M6 formulation is moved to Appendix D, and Appendix E summarizes the method taxonomy. We have also substantially revised the abstract, introduction, related work, and method exposition to improve clarity, motivation, and overall flow.
These changes strengthen the presentation while preserving the theoretical analysis, literature foundations, and controlled experiments already present in the original submission.
We hope these clarifications address the concerns raised and welcome further specific technical feedback on the formulation, theoretical analysis, or experimental comparisons.
We thank the reviewer for the feedback. However, several criticisms appear to arise from a different interpretation of our formulation, and some do not match the original manuscript. We clarify these points below.
1. Ordinal labels, numerical distances, and soft-target construction
The reviewer argues that class indices already encode distances and that nonuniform coordinates such as (0,1,3,5,6,7) can be assigned. We agree that known distances can be encoded this way. However, such coordinates specify a chosen metric; ordinal labels alone do not necessarily provide an appropriate one, particularly for subjective ratings.
More importantly, our paper studies soft-target construction, not simply numerical label encoding. Renumbering classes does not change categorical cross-entropy if their assignments remain unchanged. In contrast, soft ordinal targets explicitly depend on class distances:
qj(y)(z,τ)∝exp[−2τ2(zj−zy)2].
Changing z therefore changes the supervision distribution while preserving class order and the prediction architecture. This is central to SORD (Díaz and Marathe, 2019) and label-distribution learning, already discussed in our original Section 2.
Our method learns task-adapted relative spacing for ordinal supervision when suitable distances are not predefined, rather than attempting to recover uniquely identifiable physical distances.
2. The Whisker example: 10-mm intervals and categorical prediction
The review refers to a 30-mm spacing between Whisker classes. However, the original Figure 1 explicitly specifies Δd=10 mm. The values 0, 30, 60, and 90 mm are reference ticks on the physical depth scale, not adjacent class intervals.
The concern about large numerical labels causing training difficulties also does not apply to our formulation. All M0–M5 variants use a K-class categorical softmax head (original Section 3.1). Whisker depth is discretized into nine classes with fixed coordinates zj=j, not millimeter-valued regression outputs. The adaptive variants modify relative spacing while preserving order and total span (Equations 11–13).
Whisker deliberately represents a task with known physical spacing. The results support preserving it: fixed-spacing M3 achieves 6.778 mm MAE versus 7.182 mm for M4 and 7.114 mm for M5 (original Table 1). This directly illustrates when fixed geometry is preferable.
3. The HCI example and the uniform-spacing assumption
The reviewer asks who assumes uniform spacing between ordinal classes. This assumption is explicitly used in established methods. Hou et al. (2016, Section 2.2.2) define the ordered-class ground distance as Dij=∣i−j∣. Similarly, SORD (Díaz and Marathe, CVPR 2019, Section 4.1) explicitly treats Adience age groups as equally spaced categories when constructing ordinal soft targets.
Both papers also consider alternative distance constructions: Hou et al. estimate a feature-based distance matrix for EMD regularization, while SORD allows predefined task-specific distance functions. Our work addresses a different question: how to learn ordered class coordinates for soft-target construction while separately analyzing their interaction with cumulative scoring.
For HCI, equal chronological spacing need not imply equal visual changes, exactly as stated in our original Figure 1 caption. This motivates relaxing the fixed-spacing assumption. In the original experiments, learned geometry improves HCI MAE-EV from 0.693 (M3) to 0.666 (M4).
4. PHAC-2 and the interpretation of learned spacing
The review suggests that our model may simply rank materials such as wood and glass or estimate their physical hardness ratios. However, PHAC-2 predicts five-level human ratings of haptic properties from tactile recordings (original Section 4.1), not material identities or physical hardness values.
Predicting a hardness rating of 4 instead of 3 does not imply any particular physical hardness ratio. Our learned geometry determines how training probability is distributed among ordinal rating levels; it does not assign physical distances to materials. This distinction is central to learning task-adapted supervision when a distance scale is not provided.