- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies OOD detection when training data come from the ID distribution. The main result shows that for a chosen ID distribution, reaching a universal convergence rate over the countable-mixture completion of an OOD family is equivalent to attaining that rate uniformly over the original family. The paper also studies learning across different ID distributions and derives lower bounds using a single difficult distribution.
Critical strengths
- A connection between universal and uniform rates. For a fixed ID distribution, the main theorem shows that attaining a universal rate on the countable mixture completion of an OOD family is equivalent to attaining that rate uniformly over the original family. For each learner and proposed convergence rate, the paper constructs one fixed distribution on which that rate fails at infinitely many sample sizes. The difficult distribution remains the same as the sample size increases.
- A concrete correction to prior work. The counterexample to the necessity claim in Fang et al. is valuable as it identifies a gap between guarantees on expected risk for each domain and the existence of a single detector that performs well across domains.
Critical weaknesses
-
The presentation hides the understanding of the paper. Despite many careful readings, I found the exposition from the abstract to the statistical problem until the results very difficult to follow. I recognize that formal definitions are present, but the accumulation of terms such as “fiber”, “clock”, “schedule”, "slowness", "blind spot", “window” are used from the abstract, together with metaphors such as " "every star is lit in time, yet every row keeps stars in the dark" are amusing but makes the reading very difficult. Moreover, for example, the statement that a family admits “every rate or none” needs a much clearer explanation. In the positive case, each target rate may require a different learning procedure. The paper should make this dependence more explicit and better explain how these procedures are constructed. The metaphor that “a clock can run at any speed” is amusing but does not in any way explain the mathematical argument.
-
The extent of the main obstruction needs stronger motivation. Theorem 3.3 provides a link between universal and uniform rates under countable mixtures. However, the examples in Proposition 3.5 and Appendix D.6 both use detector classes that exclude a detector whose inclusion eliminates the obstruction. The paper acknowledges this and provides a useful compactness explanation but my remaining question is whether the same obstruction occurs in a detector class motivated independently of the construction.
Standard hypothesis classes in deep learning (e.g., neural networks with bounded weights and Lipschitz activations on bounded domains) typically enjoy relative compactness. Thus, it remains unclear to me whether the "moving blind spot" is an inherent barrier for realistic learning representations or an artifact of artificially non-compact families.
Overall assessment: Weak rejection. The paper contains very interesting theoretical results, including a precise rate equivalence and a valuable correction to prior work. However, I do not recommend acceptance in its current form. Indeed, my rassessment is primarily on two concerns: the exposition makes the central claims and their interpretation unnecessarily difficult to assess, and the manuscript does not sufficiently establish the relevance of its main obstruction to independently motivated detector classes . Resolving it would require a significant restructuring of the exposition, rather than local clarifications alone. I expect an ICLR submission to communicate its scientific contribution clearly, precisely, and above all accessibly to the research community. The text repeatedly shifts between figurative language and mathematical assertions without articulating the logical connections. Although formal definitions appear later, the overall exposition leaves the reader with the burden to reconstruct and to determine what is assumed, what is proved, and what the results imply. Addressing this concern requires a lot of rewriting of the exposition, beginning with the abstract and introduction. I do not consider the manuscript currently suitable for acceptance at ICLR.
Questions for authors
- Could you provide a self-contained explanation of the Lemma 2.1 and Proposition 2.2, using explicit (and simpler) mathematical statements? In particular, please distinguish the distribution of a learner’s output at a fixed ID law from the construction of a different learner by reindexing, and clarify which quantities may depend on the target rate.
- Beyond the correction to Fang et al., what specific problem does your theorem solve ? I would like to know how broadly it applies to standard detector classes and real-world OOD families. Please show a concrete example of a standard class where your result provides a new conclusion about what can be learned.
4 comments
Response to Reviewer EuUJ (2/3)
Weakness 2 (an independently motivated class, and bounded weights). Closed halfspaces: yes. Bounded-weight networks with one hidden ReLU unit define exactly the same detectors, so the answer carries over. In both cases a blind spot requires that the ID law and some hidden law each put mass on a hyperplane, and Proposition 3.6 gives one where they do.
Our submission's remark "A moving blind spot requires a noncompact class" (end of Section 3) dropped the lower-semicontinuity hypothesis of Proposition C.8, and without it the remark is false. No theorem or proof used it, and the statement of Proposition C.8 is unchanged. New Proposition 3.6 replaces the remark. It takes binary detection with zero-one loss and closed halfspaces , with on the unit sphere (VC dimension ).
- A blind spot (). Let the ID law be a point mass at and the hidden laws point masses at and . For any finitely many of these points, one halfspace accepts and rejects all of them. But a halfspace that accepts and rejects has , so it accepts for all large . If is the probability that a random halfspace accepts and rejects , its expected strong excess is on and at least in the limit on . So , and an even mixture of and attains . The countable family is learnable at every rate; its mixture completion is consistent but has no rate (Corollary 3.4, Theorem 3.3).
- No blind spot. If every hidden law, or the ID law, puts no mass on any hyperplane, every consistent fiber is uniformly learnable at every rate.
- One-unit networks. A bounded-weight network , accepting where it is nonnegative, accepts a closed halfspace, everything, or nothing, and every closed halfspace arises (Appendix D.7). As a detector class, this smallest network is exactly the closed halfspaces.
Relative compactness alone does not exclude a blind spot, because thresholding a score is not continuous. The separators of Appendix D.7 converge to the halfspace , whose boundary carries the ID atom and the hidden atom . Their pointwise limit is neither open nor closed, so no continuous score thresholded at zero, with or , accepts exactly . As with the prefix class, the class misses its pointwise limit. What changes is that the class is a standard one, fixed independently of the example; a larger class can still remove this particular blind spot (below). Proposition 3.6(2) identifies what any closed-halfspace blind spot needs: ID mass and hidden mass on hyperplanes.
For a linear head on a fixed feature map, such as a pretrained one, Proposition 3.6 applies to the feature laws, and these can charge hyperplanes: ReLU features have exact zeros, and features confined to a proper affine subspace lie in a hyperplane (Appendix D.7). So part (2) does not exclude a blind spot there. With strict thresholds, hidden laws with densities suffice, though the ID law must still charge a hyperplane (Appendix D.7).
Excess is class-relative, so enlarging the class can remove a given blind spot: two ReLU units accepting are exact on this family. For wider fixed architectures the question is open; what Proposition 3.6 shows is that compact parameters, a Lipschitz activation and a bounded domain do not by themselves exclude the obstruction. Proposition 3.6 is in Section 3, with proofs in Appendix D.7; the one-unit case and the open question are stated after it.
Question 2 (the problem solved, for a standard class). The problem is whether a consistent detector trained on ID data alone can come with a universal rate, that is, a rate whose constants depend on the OOD law. Theorem 3.3 answers that, for a fixed ID law and a family closed under countable mixtures, it can only if the family is uniformly learnable. To our knowledge, prior OOD learnability theory (Fang et al., 2022; 2024; Garov & Chaudhuri, 2025), which studies uniform and certified learnability, does not ask this. For closed halfspaces, and hence one-unit ReLU networks, with and zero-one loss, Theorem 3.3 reduces the question to whether , and Proposition 3.6 settles it except when both the ID law and some hidden law charge hyperplanes, where both outcomes occur.
Response to Reviewer EuUJ (3/3)
- No rate (Theorem 3.3). For one known ID law, a consistent family closed under countable mixtures can have no rate at all, but only if the ID law and some hidden law each charge a hyperplane (Proposition 3.6).
- Every rate (Proposition 3.6(2), the complementary side). With a known ID law, consistency already implies uniform learnability at every rate whenever the ID law, or every OOD law, charges no hyperplane; in particular for every ID law with a density.
- Finitely many ID laws. With finitely many ID laws that charge no hyperplane, every consistent family, together with its mixtures, is learnable at a uniform exponential rate (Proposition 3.6(2) with Theorem 4.1; Appendix D.7, "Finitely many ID laws").
For pooled deployments, the theorem says when per-source learning curves carry over to mixtures of the sources. Appendix I.1 discusses deployments pooled over sources, such as hospitals or OOD test sets, with unknown weights. For a fixed ID law, a learning curve that holds for each source, however fast, extends to all countable mixtures of the sources only if it holds uniformly over them. Finite mixtures inherit per-source curves, so the condition matters for open-ended families of sources. Appendix I.1 also gives a worst-case diagnostic built from existing per-source miss rates; it is not an estimate of deployed error.
If the revised pages and Proposition 3.6 address your two concerns, we would appreciate it if you would reconsider your rating.
Thank you for the care you put into your response and the revision. I found the contrast between the countable family and its mixture closure interesting as taking mixtures preserves consistency while ruling out every rate. The new halfspace example also answered one of my main questions. I think there are very good ideas here, and I appreciated the chance to understand them better through your response.
I also appreciate the detail you added to the AI use statement. Reading it left me less confident that I had understood the extent of the LLM contribution from the original declaration. I would welcome a clearer account of the role the models played in developing the main results that were already in the submission, and of how you checked those arguments yourselves. The detail you provided about the work done during the discussion period was helpful, and I would appreciate the same level of clarity about the original results.
I’m keeping my rating at 2 because I still found it difficult to follow the paper and see the scope of the results from the main text. I’d strongly encourage you to made the paper more accessible and easy to follow: what is the problem and why it it theoretically grounded, in order to solve this problem what are the naturel mathematical object and how they naturally appears. Then I strongly encourage the authors to resubmit it as soon as possible (if it doesn't go throught ICLR 2027). I think maybe, just maybe, putting the contrast between the countable family and its mixture closure near the beginning as a motivation, then building the exposition around it, would give readers a much simpler way into the paper as that example helped me see what makes the result interesting, and I’d like more readers to have that experience.
Thank you again for your thorought work for the revision.
Response to Reviewer EuUJ (1/3)
Thank you for the review, and for Question 1: the difference between the law of a learner's output at a fixed ID law and the learner built from it by reindexing is the step a reader most needs, and Proposition 2.2 now states that learner and what it depends on. On your first concern we agree: the exposition made the central claims hard to assess, and we have revised the abstract and introduction (below). On your second weakness, including its paragraph on bounded weights, the obstruction is not an artifact of non-compact parameters: closed halfspaces, whose parameters form a compact set, can have a moving blind spot. Halfspaces that each reject a given OOD point can converge to one whose boundary passes through that point and so accepts it; for closed halfspaces this produces a blind spot only when the ID law and some OOD law both put mass on a hyperplane (Proposition 3.6).
Weakness 1 and the overall assessment (presentation). What changed:
Question 1 and "every rate or none" (Lemma 2.1 and Proposition 2.2, explicitly). Fix the ID law P. A learner A maps a sample Sn∼Pn and an independent seed to a detector in H. The loss is bounded by M.
Lemma 2.1. Let μn be the law of the detector A(Sn) under Pn and the seed. It depends on A, P and n, not on the OOD law Q, and for every Q,
EE(P,Q)(A(Sn))=∫E(P,Q)(h)dμn(h).
On the set of problems with ID law P, the errors of A are therefore determined by the sequence (μn) alone.
Proposition 2.2. Suppose A attains a rate r for this P: every Q has CQ,cQ>0 with EE(P,Q)(A(Sn))≤CQr(cQn) for all n. Let s be any other rate. Choose integers φs(n) with r(φs(n)/n)≤s(n) (they exist because r(t)→0), and let the learner Bs, at sample size n, ignore its sample and output a detector drawn from μφs(n). Then, for every Q and n,
EE(P,Q)(Bs(Sn))≤max{CQ, M/s(1/cQ)}s(n).
Indeed, by Lemma 2.1 the left side equals EE(P,Q)(A(Sφs(n)))≤CQr(cQφs(n)). If n≥1/cQ, then cQφs(n)≥φs(n)/n, and since r is nonincreasing, r(cQφs(n))≤r(φs(n)/n)≤s(n). If n<1/cQ, the left side is at most M≤(M/s(1/cQ))s(n), since s is nonincreasing. What depends on what:
So yes, each target rate has its own learner: Bs depends on s. On particular families one learner can attain every rate at once (on the point-mass family of Proposition 3.5, the learner that outputs hn+1 does), but Proposition 2.2 does not claim this. Bs is a valid learner only because P is fixed: it samples from μ at P. Proposition 2.2 uses that P is known; this is why rates for one ID law are all or none, and why nontrivial rates need the sample to identify P (Sections 4 and 5). Theorem 3.3 is stated for each fixed learner and does not use reindexing, and Theorem 4.1(b) applies it when the ID law is unknown. Proposition 2.2 now states these dependencies.