- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
OrdinalICL investigates whether explicitly modeling target order improves tabular foundation models under matched training conditions. The method proposes a shared ordinal interface for variable cardinalities, generating valid class probabilities via ordered thresholds and ensuring exact reversal symmetry through predictive averaging. A TabICLv2 adapter and a compact PFN trained from scratch validate the interface's ability to learn order. Next, using TabDPT-Turbo v1.2 with fixed backbone, initialization, data, optimizer, and compute, the authors compare equal-sized ordinal and nominal prediction-correction modules. Two evaluations on unseen synthetic mechanisms favor the ordinal module. Across a 40-family real benchmark, the final model outperforms both its matched control and published TabDPT, with confidence intervals excluding zero. The study supports ordinal adaptation in TabDPT while acknowledging limitations in context scale, metrics, and pretraining data overlap.
Critical strengths
- Ordinal prediction has important practical applications, making the research question meaningful.
- The title, abstract, and experimental narrative are well aligned, clearly conveying the motivation and overall approach.
- The AI Use Statement clearly discloses the scope of AI assistance, supporting transparency in the research process, thus authors don't overclaim their contribution.
Critical weaknesses
1.The experimental analysis lacks depth. Despite matched controls and reported subgroup trends, the main text primarily presents aggregate scores, with limited analysis and visualization of the effects of feature count, context size, and class count. 2. The four requirements in Section 2.1 are insightful. Although order scrambling experiments and reversal error checks are provided, the evaluation lacks systematic tests corresponding to all four requirements, particularly order-preserving recoding and unobserved categories. 3. Substantial space is devoted to initial implementations and backbone selection, leaving limited room for deeper analysis of the final method. As an adaptation of an existing backbone, the method needs stronger validation across different backbones and task settings. 4. The ordinal baselines remain incomplete. Table 4 includes proportional odds, LightGBM, and CORAL, but lacks sufficient classical methods and simple ordinal heads on the same backbone. These comparisons would clarify the method’s distinctive value, even without state-of-the-art performance.
Questions for authors
- Which components account for the gains? The ordinal module introduces rank encoding, risk-set processing, and ordinal decoding together. Could ablations separate their contributions and evaluate the performance gains and computational overhead of reversal averaging?
- Why is this interface needed instead of a simple ordinal prediction head? How would a standard cumulative-threshold head perform under the same TabDPT backbone, parameter budget, and training conditions? The nominal control supports the benefit of the complete ordinal module, but does not establish the necessity of its specific design.
Answer to Ws and Qs
Thank you for the detailed questions. We have run the reversal tests, checked the label-handling code, and broken down the real-data results. Here is what they show.
Why this design rather than a simple ordinal head? (W4, Q1–Q2)
We agree. Our comparison with an equally sized nominal module tests the complete change in label representation and prediction rule. It does not show that each part is needed, or that our design beats a standard cumulative-threshold head on the same backbone. We have not completed that head comparison or the separate training ablations. The existing proportional-odds and CORAL baselines do not answer this question.
Since both arms already use an order-sensitive loss, our experiment asks whether adding order to the architecture helps further.
What does reversal averaging add, and what does it cost? (Q1)
We tested both arms, all three seeds, and all 40 families. The averaged scores reproduce the original results exactly.
Lower is better. For OrdinalICL, averaging changes nRPS by +0.000049 compared with forward-only prediction; the 95% family-bootstrap interval is [−0.000308, +0.000399]. We therefore cannot claim an improvement over forward-only prediction. Averaging ensures the same answer whichever direction the scale is presented. We report both directions because the ordinal-minus-nominal difference is −0.000936 forward but +0.000501 reversed. Both arms use averaging in the main comparison.
Tests of the four requirements (W2)
We tested the declared-label API on ten real contexts using three model seeds and four feature permutations. Replacing labels with unevenly spaced numbers or strings changed probabilities by zero. Reversing the declared order and mapping outputs back gave a maximum difference of 2.22×10⁻¹⁶.
We also kept one interior category absent in each context: seven required row replacements and three already lacked it. All ten returned valid distributions over all K categories, including positive probability for the absent category. This verifies that the category is retained; it does not show that predictions for it are better than the nominal model's.
The submitted order-scrambling tests show that the earlier TabICL/PFN models use label order.
Where do the gains occur? (W1)
The real-data differences, ordinal minus nominal, vary with class count:
For 5–7, 8–10, and 11–12 selected features, the differences are −0.000464, −0.000157, and −0.000123 (4, 5, and 32 families). Contexts missing a rank average −0.000238, versus −0.000099 with all ranks present. These analyses were added after review. Families can appear in multiple groups, and some groups are small. The results are mixed; they do not establish how changing feature count or removing categories would affect performance. Larger-context tests are not completed.
Other backbones and presentation (W3)
The earlier TabICLv2/PFN studies show that the interface can be implemented on other models. The final matched design has been tested on TabDPT v1.2 and v1.3. For v1.3, the difference from its nominal control is −0.000429 [−0.000612, −0.000247], with 30/40 family wins. This repeats the result on another release of TabDPT, not another architecture.
We agree that the paper spends too much space on earlier implementations and backbone selection. The independent synthetic tests support the controlled comparison. The pooled real benchmark was examined during backbone and compute selection, so its results need that qualification. Our evidence supports a small benefit in the tested setting, without showing that every component is needed or that the method helps on every task.