acceptodds
Withdrawn from ICLR 2027

Useful Features, Backward Scores: OOD in Language-Model Trajectories

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Out-of-distribution (OOD) detectors prioritize inputs for closer inspection. Yet features that distinguish input groups need not yield a useful anomaly ranking. We analyze this gap in language-model trajectories under text-length control and fixed score directions. On Spam development data, an input adaptation of D^2HScore falls from raw AUROC 0.919 to 0.530 after length matching. On length-matched, held-out HateSpeech inputs, the same features yield AUROC 0.644 for a labeled linear classifier but 0.444 for an ID-fitted distance score. ToxicChat shows the same contrast. Feature-selection and backbone controls retain the main reversal pattern. Frozen Civil Comments and TweetEval irony tests also reverse (0.467 and 0.435), extending the finding beyond toxicity. In these contrasts, anomalous groups have farther centers but tighter spread. A labeled, fixed-center feature-space intervention changes rankings: equalizing spread helps some tasks and harms others. OOD evaluation must check the chosen score's ranking even when its features distinguish the classes.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
out-of-distribution detection, language models, representation geometry, Mahalanobis distance, score orientation, length-controlled evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_amLrOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how features that can distinguish between ID and OOD samples may still fail to produce reliable OOD detection scores in language-model trajectories. The authors present two main observations: 1) differences in text length can lead to misleading OOD detection, and 2) trajectory features that support supervised classification can still produce poor distance-based OOD rankings. The authors explain this through differences in within-class dispersion, showing that OOD representations can have more distant centroids but tighter distributions. I have several concerns on the paper quality, novelty, methodology, and experiments, which I do not believe the paper currently meets the acceptance standard and it requires substantial revision.

Critical strengths

None

Critical weaknesses
  1. The paper is very difficult to follow and understand, numerous metrics and technical terms are introduced without sufficient explanation and are used excessively throughout the paper. This makes it difficult to understand the overall research narrative and interpret the results. It is suggested for the authors to improve the manuscript without relying on purely AI-generated contents. The current quality of the manuscript clearly does not meet the standard for acceptance.

  2. ⁠The main observation that discriminative features do not necessarily produce effective distance-based OOD scores is genuinely well established (as explored in the discussed related work papers). The paper does not clearly explain what new insights are gained beyond demonstrating this phenomenon on language model trajectories. Moreover, while the paper presents extensive experiments and analyses, many results repeatedly support a basic observation without providing more indepth insights.

  3. ⁠The paper also remains unclear why language model trajectories are particularly important for studying this problem. For example, it is unclear why the particular trajectory features were selected, why these features are appropriate for investigating the proposed problem, or what makes language model trajectories a particularly meaningful setting for this analysis. The authors also do not convincingly demonstrate how the observed failure differs from similar issues in conventional feature-based OOD detection.

  4. ⁠A large amount of essential method and experiments information is put in the appendix (e.g., definitions and extended experimental results). Given that the main paper has enough space available (main text is only 7 pages), it is difficult to understand why so much essential information is deferred to the appendix, which significantly affects the readability and self-containedness of the paper.

Questions for authors

Please see the weaknesses.

Reviewer_PG9iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the mismatch between class information in language-model trajectory features and the anomaly rankings produced by an ID-fitted distance score. The authors combine length-matched evaluation, supervised classifiers using the same features, and fixed-center dispersion interventions to analyze this mismatch. The evaluation covers four development tasks across five model backbones, held-out confirmation sets, and frozen Civil Comments and TweetEval irony tests. The study provides empirical evidence that length matching can reduce apparent detection performance, that features supporting classification can produce below-chance distance rankings, and that equalizing feature dispersion does not consistently improve rankings across tasks.

Critical strengths
  1. The paper exposes a mismatch between class-discriminative trajectory features and OOD distance rankings, clarifying why informative representations alone do not establish useful anomaly detection.
Critical weaknesses
  1. Limited methodological novelty: The main contribution is a diagnostic analysis built on established distance-based OOD detection and statistical decomposition, without introducing a new detection method or a generalizable criterion for identifying unreliable rankings.
  2. Insufficient evidence of generalizability: The ranking reversals occur in selected task contrasts, while additional retrospective evaluations provide limited evidence that the phenomenon extends broadly across different OOD settings.
Questions for authors

see weaknesses

Reviewer_4JHaOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether informative language-model trajectory features yield useful anomaly rankings. It shows that length matching can substantially change apparent detection performance, and that features supporting above-chance supervised classification can produce below-chance rankings under a predefined ID-fitted distance score. The authors investigate this discrepancy through a centroid–dispersion decomposition and fixed-centroid feature interventions. Evaluations cover multiple backbones, feature-selection and metric controls, same-source confirmation splits, and additional tests on Civil Comments and TweetEval irony.

Critical strengths
  1. Comparing supervised readouts and ID-fitted distance scores on the same features and evaluation examples clearly separates the availability of class information from the effectiveness of a particular scoring rule.
  2. The evaluation covers length matching, feature-selection controls, reference fitting, distance geometry, and backbone sensitivity, with explicit reporting of inconclusive results and limitations.
Critical weaknesses
  1. The analysis closely follows established observations concerning reversed OOD scores and feature geometry, including Nalisnick et al. (2019), Müller and Hein (2025), and Janiak et al. (2026). Its extension to language-model trajectories provides useful empirical evidence, but the conceptual advance over these explanations remains modest.
  2. The centroid–dispersion decomposition is a standard second-moment identity concerning mean squared distance, which does not determine pairwise ranking. The feature interventions demonstrate sensitivity to dispersion without explaining why particular text classes contract or identifying conditions that predict AUROC reversal.
  3. Most backbone and sensitivity evaluations reuse the same four task contrasts, and the ToxicChat confirmation interval spans chance. HateSpeech, Civil Comments, and irony provide credible reversal evidence, but the variation across additional evaluations leaves the circumstances under which this phenomenon matters insufficiently characterized.
  4. The candidate trajectory features were developed on the original task suite, and the final subset was selected using development labels. Leave-one-task-out and all-candidate controls address subset-selection sensitivity, but retain the historically designed candidate pool, leaving transfer to independently specified trajectory representations uncertain.
Questions for authors
  1. What insight specific to language-model trajectories distinguishes these findings from existing explanations of reversed OOD scores and feature-geometry effects?
  2. Do the existing held-out evaluations reveal any consistent properties distinguishing tasks with reversed rankings from those with positive or inconclusive rankings?
  3. Were any trajectory representations specified independently of the original development suite evaluated, and, if so, did they preserve the observed reversal pattern?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.