- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
-
The paper proposes a new evaluation paradigm to test the generalization of large models (e.g. LLMs or vision-and-language models) beyond their training distribution. It is kindof an extension of the OOD paradigm, applicable to large models whose training distribution is not known. Their key insight is to "characterize the boundary [between in- and out-of-distribution] in terms of world semantics rather than observations". The framework is very general ("OOWD").
-
They then apply the framework to evaluate small and large models in 3 domains, and find that the large models show stronger generalization.
-
They then perform mechanistic analyses on these models to understand why they generalize better.
-
They finally propose a method to improve their form of generalization ""OOWDAlign".
Critical strengths
(S1) The evaluation of generalization capabilities is an important topic, and the issues of data contamination/data coverage (the core motivation for this paper) are very important concerns in the evaluation of ML systems in general.
(S2) The framework is applied to several domains.
(S3) The experiments/results are analyzed under multiple angles (behavioral, mechanistic).
Critical weaknesses
Main weaknesses
(W1) The first main issue is the clarify of the presentation and what the proposed framework is about. I had to go through the whole paper until I could make sense of what was initially stated in the abstract and intro. I now understand that the paper is about evaluating models after fine-tuning them on new tasks that have been handcrafted to tbe incompatible with the similar tasks from the (supposed) training data. I don't understand why this is not stated as such more explicitly upfront in the paper. My advice for the authors (maybe for next time) is to get feedback on their manuscript from a friend who doesn't know at all about the project. Right now it feels like the paper is written for someone who already knows everything about it.
(W2) The second main concern is whether the proposed framework is really doing what the authors claim it's doing. I understand that the initial motivation is to fix the issue with "traditional" OOD evaluations that they can't be guaranteed to be OOD because we don't exactly know the training distribution of large models. The goal of these traditional OOD evaluations is to evaluate the robustness of models to inputs that they haven't been extensively exposed to during training. So I assume that this is the goal here as well (I can't find a clear statement of a different goal in the paper). But after reading the paper, I understand that, what s being tested, is how robust the model is on new tasks after being fine-tuned on data from these new tasks. These experiments are interesting, but I think that the findings are answering very different questions, namely how much knowledge is reused across related-but-conficlting tasks, and across pretraining/fine-tuning phases.
Detailed or minor comments
(C1, related to W1) The first 4 pages of the paper are very vague and abstract. It's very difficult to understand concretely what the paper proposes. A concrete example (and possibly a "teaser" figure in the intro) to anchor the discussion from the start would be very helpful. The critical questions that the reader needs to know about are only answered much later in the text, like what does the framework prescribe exactly and what does it take to apply it to a new task/domain/model? What does iit entail exactly, like creating a new dataset? metrics? fine-tuning models? etc.
(C2, related to W2) I'm still trying to understand what the OOWD framework does exactly. Taking the frist (math) domain for example, I understand that the task is: "answer arithmetic questions". The pretrained model is fine-tuned on symbolic examples, and evaluated on natural language example. The trick of OOWD is simply to assign different meanings to the digits and operators. Is this correct? If so, for a given pretrained model with a given (fixed) arithmetic proficiency, I understand that the proposed setting is evaluating how well the model can learn (through fine-tuning) new meanings of the digits/operators that conflict with its pretrained knowledge. (This is my understanding but not really what is stated in the paper.)
(C3) Sentences like this in the formal description are really unhelpful unless you already understand the whole paper "/(...) then evaluates cross-distribution transfer under the same induced semantics" (across what distribution? same as what? induced by what?).
(C4) "conventional OOD evaluation requires source and target distributions to lie outside the training distribution": I don't understand, I think the source usually refers to the training distribution.
(C5) "scratch models": I don't know what this means. I suppose the authors mean "models trained from scratch", but this is not a good qualifier; all models are initially trained from scratch.
(C6) In the section on related work: "Existing literature often characterizes OOD generalization as a two-stage process: a model first recovers task-relevant latent variables from inputs, and then predicts the correct output based on these." I'm familiar with the OOD generalization literature but I don't see how this description fits the series of citations that follow this sentence. Maybe the citations on causal representation learning are related, but OOD generalization is kindof a second point to these works.
Questions for authors
Did I misunderstand the paper (in relation to W2/C2 above)?
Prospects w.r.t. a possible update of my rating:
-
Addressing W1 requires an extensive revision of the first ~4 pages of the paper, which I'm not sure can be addressed solely with the promise of an improved final version.
-
If the concern in W2 is correct, fixing W2 means changing the framing of the paper, but this would require an extensive rewrite of (at least) the first few pages of the paper. The experiments themselves are valuable and may be sufficient. My comments are rather about what the proposed setting/experiments address, and what the observations mean.
-
If the concern in W2 is correct, this also means that the main strength of the paper (evaluating capabilities with possible data contamination) does not hold.
(No need to reply to the minor comments C3-C6.)
2 comments
Reviewer's response
Thanks for the response. It mostly confirmed my initial concerns. I tried to read the new version with a set of fresh eyes but the description is about as inscrutable as the first version. I put anyone to the challenge of understanding what is going on from figure 1 alone. I reiterate my suggestion to the authors to test out their paper first on colleagues who do not know about this project, and see if they can understand it correctly.
This paper is clearly below the bar for the main conference hence I have updated my score to a strong rejection.
Response to Reviewer 6NuR
Thank you for taking the time to review our paper carefully. We especially appreciate such attentive engagement with our work in today's research environment. Thank you also for identifying the presentation problems so directly. Below, we present our revisions and responses.
W1 / C1: Making the procedure explicit in the first four pages.
We have substantially revised the opening presentation, with changes marked in blue in the revised manuscript. In particular, we added Figure 1 to the introduction to show the full procedure: construct a counter-world rule, fine-tune on source examples following that rule, and test the adapted model on a target domain with fixed weights. We replaced the abstract, formalization-led explanation in the introduction with a concrete account aligned with this figure, and reorganized Section 3 to make the distributional logic explicit. These revisions are already present in the revised manuscript, rather than being deferred to the final version.
Concretely, applying OOWD requires constructing or relabeling data under a specified counter-world rule, choosing different source and target input distributions, and adapting the model on source data only. For example, we can redefine addition as a+b↦a+b+3, teach this rule using symbolic expressions, and test it using natural-language word problems. Held-out source accuracy measures whether the rule was learned; target accuracy measures whether it transfers to the different input distribution. The OOWD ratio ρ=Acc-T/Acc-S summarizes that transfer, while the two accuracies are reported separately to expose failures of source fitting.
W2 / C2: Is this merely learning a new task after fine-tuning?
Your description of the arithmetic experiment is correct. We agree that the goal is to evaluate generalization to distributions that were not extensively represented during training. Where our interpretation differs is that learning the source examples and transferring to the target are distinct requirements. Symbolic expressions and natural-language word problems share computational structure, but their input distributions differ. Treating them as one undifferentiated new task would obscure the cross-distribution transfer being measured. Sharing an underlying task rule is also a standard feature of OOD generalization settings.
The revised Figure 1 makes this relationship explicit. In a conventional setting, a model is trained on A and evaluated on a different distribution A′. With an LFM, we cannot reliably determine whether the proposed target input distribution is already covered by pretraining. OOWD therefore introduces a controlled change in the input–output rule. Starting from pretraining on A, we train on a limited counter-world source distribution B, then evaluate on a target distribution B′ that uses the same counter-world rule but has a different input distribution. Relative to the task's regular-world pretraining reference and the added source distribution, respectively, the intended shifts are
\qquad p_{B'}(x)\neq p_B(x).$$ Here, pretraining and source fine-tuning together form the training stage. So we can describe OOWD as an operational approximation to conventional OOD testing through a controlled and mild extension of training. It does evaluate an adapted model, and we have made that fact explicit. However, adaptation does not remove the generalization question: models with similar source accuracy can differ sharply in target transfer. In the updated operator-overloading results, configurations with source accuracy of at least 90% yield almost identical mean source accuracy for Tiny GPT-2 and Qwen3-4B, **95.45% versus 95.23%**, but target accuracy of **46.89% versus 92.96%**. This difference cannot be explained simply by whether the new source rule was learned. We also agree that reusing pretrained knowledge is part of this process. Such reuse is compatible with OOD generalization: transfer generally depends on carrying useful structure from training to a different distribution. Our claim is therefore that OOWD exposes differences in cross-distribution rule transfer under controlled adaptation, while reducing the contamination ambiguity of ordinary benchmark testing. **C3–C6: Terminology and minor clarifications.** We have corrected and clarified the wording in the manuscript in response to these comments. By “scratch models,” we specifically mean the baselines initialized randomly and trained on our limited regular-world datasets before counter-world adaptation, as opposed to starting from broadly pretrained checkpoints. We welcome any further questions or comments and would be happy to continue the discussion. If our revisions and responses address your concerns, we would be grateful if you would consider raising your score.