- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper examines whether accurate in-context prediction implies recovery of the underlying task concept. It introduces a component-level evaluation framework that represents concepts through relevant attributes, their allowed value levels, and, where appropriate, a composition threshold. The framework is instantiated in a controlled STAGGER-style environment with AND, OR, and m-of-K rules. Using version-space analysis, the authors assess whether the demonstrations uniquely identify their target concepts. Experiments with four instruction-tuned LLMs compare implicit prediction with explicit recovery, characterize component-level errors, and study interventions that supply correct or perturbed structural information. Finally, fine-tuning experiments with Qwen2.5-7B test whether structured recovery can be learned while preserving implicit prediction and transferring to unseen composition forms.
Critical strengths
Identifiability-aware evaluation. Appendix C explicitly analyzes whether the demonstration sets uniquely determine the target concept within the specified hypothesis class. This is consequential because otherwise poor recovery could simply reflect ambiguity in the evidence. The version-space enumeration and identification-threshold analysis make the central prediction–recovery comparison more interpretable.
Useful error diagnosis and evidence of learnability. Table 1 identifies a concrete failure pattern—substantial over-inclusion of supposedly relevant attributes—rather than treating all incorrect recovered rules alike. The paired prediction comparison and concept-application controls help interpret this gap. Tables 5 and 6 additionally show large improvements from supervised recovery training, while joint training retains strong implicit-prediction performance. These results make the diagnostic framework potentially useful beyond merely reporting a new metric.
Critical weaknesses
The full-information interventions in Table 3 leave nothing to infer from the demonstrations. Because the composition form in Equation (1) is already given, providing both the relevant attributes A and their allowed levels L completely specifies an AND or OR concept; adding the threshold m does the same for m -of- K . High recovery under R ( A , L ) and R ( A , L , m ) therefore shows that the model can format and copy a rule already stated in the prompt, not that it combines partial cues with the demonstrations to induce the concept. Running the same prompt without demonstrations would immediately test whether the examples play any role at all. Separately, Appendix D.1 states that the K intervention tells the model there are exactly K relevant attributes, whereas Appendix B.1 allows | A | < K for AND concepts; in those cases, the prompt gives the model an incorrect attribute count.
The decoy experiment in Section 3.2 and Table 4 is ambiguously defined and has conflicting baseline numbers. Adding an allowed level to each relevant attribute changes the ground-truth Boolean function rather than inserting a harmless distractor. When recovery drops, it is impossible to tell whether the model is distracted by irrelevant information or simply following the modified rule it was given. The text is also unclear on whether the corrupted level sets replace the true ones or appear alongside them. Finally, the unperturbed R ( A , L ) baseline numbers on OR concepts do not match between Table 3 and Table 4 (e.g., Qwen2.5-72B shifts from 0.94 to 0.88, and Llama-3.1-70B shifts from 0.88 to 1.00), which needs to be explained before drawing conclusions from the perturbation results.
The identifiability guarantee in Appendix C does not match the output space allowed by the recovery prompt. Appendix B.1 generates concepts from a restricted family of ordinal threshold predicates, and the version-space analysis in Appendix C proves uniqueness only within that restricted class. Yet the recovery prompt in Appendix D.1 allows the model to output arbitrary subsets of levels for each attribute without stating the ordinal-threshold constraint. A demonstration set that uniquely pins down a concept within the generator's hypothesis class can still be compatible with multiple non-ordinal level subsets in the model's output space. Either the prompt needs to explicitly state the predicate restriction, or identifiability needs to be checked over the full grammar the model is permitted to produce.
The fine-tuning transfer claims and statistical tests are overstated. In Tables 5 and 6, training and evaluation share the exact same attribute vocabulary, level space, and output format, varying only the composition operator. This shows transfer across operators within a single synthetic generator, not general concept-recovery ability. The fine-tuning experiments also test a single model (Qwen2.5-7B) with no variance reported across random seeds, and Appendix D.7 omits basic training details such as epochs/steps, effective batch size, and checkpoint selection. Finally, Table 2 runs McNemar's test at the individual query level even though multiple queries are evaluated on the same sampled concept and demonstration set, violating the independence assumption; variance and significance should be computed at the concept or episode level instead.
Questions for authors
- In Table 3, when A and L (and m for m-of-K) are given, which component of the target concept still has to be inferred from the demonstrations? Do you have an existing control that supplies the same information without demonstrations?
- How exactly is the K intervention implemented for AND concepts with |A| < K? Does the prompt communicate nominal predicate capacity or the number of effective relevant attributes?
- In Table 4, is one extra level introduced per relevant attribute or per concept, and are the correct level constraints still shown alongside the perturbed ones? What accounts for the differences in the unperturbed R(A,L) results between Tables 3 and 4?
- Does the identifiability guarantee in Appendix C hold for the arbitrary allowed-level subsets apparently permitted by the recovery prompt in Appendix D.1, or is the restricted ordinal-predicate family explicitly communicated to the model?
3 comments
Response to Reviewer ny9X - Part II
-
W2-1: The decoy results may reflect following the modified specification rather than distraction by irrelevant information.
-
We clarify that the decoy intervention does not replace the ground-truth level constraints with modified constraints. Instead, it retains the ground-truth levels and adds one plausible but unnecessary level for each relevant attribute. The purpose is to examine whether models can distinguish the concept-defining levels from these additional candidates using the demonstrations.
- The augmented specification retains useful ground-truth information. Comparing in Table 4 with the attribute-only condition in Table 3, we find that the former yields higher recovery in multiple model--form conditions. Unlike , the augmented specification provides the ground-truth level constraints together with one additional decoy level for each relevant attribute. Thus, despite the presence of decoys, the provided level information can still offer useful guidance for recovering the target concept.
- As shown in Table 4, adding one decoy level per relevant attribute reduces whole-concept recovery. This indicates that models are sensitive to plausible but unnecessary structural information, even when the correct levels remain among the provided candidates.
- Overall, we interpret these results as evidence that models can benefit from partial structural information but remain sensitive to plausible yet unnecessary additions. The experiment therefore probes the difficulty of distinguishing concept-defining information from irrelevant structure, rather than treating the augmented specification as a new ground-truth concept.
-
-
W2-2 and Q3: The decoy experimental setting.
- In the level-perturbation condition (Table 4 in the paper), we add exactly one decoy level to each relevant attribute. Specifically, for each attribute , we construct , where is the first level outside the ground-truth set. Thus, the original ground-truth levels are retained rather than replaced. The model sees a single augmented candidate set for each attribute, without being told which level is the decoy. The prompt instructs the model to identify the true level constraints from the demonstrations, effectively requiring it to exclude the additional decoy level. The correct level constraints therefore remain present in the provided sets. The detailed design is presented in Sec. 3.3, and we have revised the Sec. 4.3 to avoid the ambiguity.
-
W2-3: Inconsistent values between Tables 3 and 4.
- We thank the reviewer for identifying these inconsistencies. We rechecked the experimental logs and found errors in the reported OR results in Table 3. We have corrected these entries to match the corresponding experimental results. The corrected values are Qwen2.5-72B: 0.88 and Llama-3.1-70B: 1.00. These corrections do not change our findings or conclusions.
-
W3 and Q4: Identifiability in the output space allowed by the recovery prompt.
- We thank the reviewer for the thoughtful question. The ordinal order of each predicate used to generate concepts is not sepcified in the recovery prompt in Appendix D.1. The prompt only specifies the three available levels, and our parser and scorer allow arbitrary level subsets.
- We extend the identifiability analysis in Appendix C to the full output space allowed by the prompt. For each attribute, we consider either omitting it or assigning it one of the six non-empty proper subsets of {LOW,MEDIUM,HIGH}. The full set is excluded for OR and -of- concepts because it makes an OR concept always positive or changes the threshold semantics of an -of- concept. For AND concepts, it is behaviorally equivalent to omitting the attribute. We perform this analysis on the same 48 concepts and demonstration streams used in our experiments.
- The results are identical to those obtained in the ordinal hypothesis class: 48/48 concepts are uniquely identified up to behavioral equivalence at (H=200), and 47/48 at (H=100), with the same single exception. In Appendix C.3, we check the demonstration streams used in our experiments, and the expanded analysis confirms that the target concepts remain identifiable even when considering the broader output space allowed by the recovery prompt. Therefore, the demonstrations provided to the LLMs in Appendix D.1 contain sufficient information to identify the target concepts, and recovery errors cannot be attributed to ambiguity in the demonstrations.
- We thank the reviewer for the thoughtful question. The ordinal order of each predicate used to generate concepts is not sepcified in the recovery prompt in Appendix D.1. The prompt only specifies the three available levels, and our parser and scorer allow arbitrary level subsets.
Response to Reviewer ny9X - Part III
- W4-1: Performance gains on the fine-tuned model.
- We conduct additional evaluations to examine generalization beyond the original representation. We evaluate on re-skinned -of- concepts by changing attribute names and value representations while keeping the concept grammar fixed. Joint training retains strong performance even when both types of changes are applied (Table 2 below). We further evaluate the same adapters on MONK-1/2/3 and UCI Mushroom, which involve different attributes, concept structures, and data distributions. Joint training improves implicit prediction on all four datasets and explicit concept application on three (Table 3 below). These results provide further evidence that the learned capabilities extend beyond the original synthetic setting. We have revised the manuscript and added these results in Appendix D.10.
Table 2: Evaluation of fine-tuned Qwen2.5 adapters under changes to attribute vocabulary and value representation. Orig: original attribute names and three ordinal levels; Names: renamed attributes; Values: four nominal values instead of ordinal levels; Both: renamed attributes and four nominal values. Trained-adapter results are mean standard deviation across three runs.
| Training | (Orig) | (Names) | (Values) | (Both) | (Orig) | (Names) | (Values) | (Both) |
|---|---|---|---|---|---|---|---|---|
| No training | ||||||||
| Recovery only | ||||||||
| Joint |
Table 3: Evaluation of fine-tuned Qwen2.5 adapters on MONK-1/2/3 and UCI Mushroom. Trained-adapter results are mean standard deviation across five seeds.
| Dataset | Training | ||
|---|---|---|---|
| No training | |||
| MONK-1 | Recovery only | ||
| Joint | |||
| No training | |||
| MONK-2 | Recovery only | ||
| Joint | |||
| No training | |||
| MONK-3 | Recovery only | ||
| Joint | |||
| No training | |||
| Mushroom | Recovery only | ||
| Joint |
-
W4-2: Training seeds and training details.
- Multiple runs. We report the mean and standard deviation across three training runs. The two additional reruns use explicitly fixed seeds to control LoRA initialization, data order, and dropout. As shown in Tables 5 and 6, the results exhibit relatively small variation across runs.
- Training details. We have revised Appendix D.7 to provide the missing training details, including the LoRA configuration, optimizer and learning-rate schedule, effective batch size, number of training epochs and optimizer steps, and checkpoint selection.
-
W4-3: Questions about McNemar's test.
- We replace the original tests with a concept-level paired analysis.
- For each of the 64 concepts, we compute the accuracy difference between implicit and explicit prediction. We then estimate 95% confidence intervals using a -stratified bootstrap and compute p-values using concept-level sign-flip permutation tests, with Holm correction across the four models.
- As shown below, the prediction–recovery gap remains statistically significant across all four models , with mean differences ranging from +0.16 to +0.24. The detailed breakdown in Appendix D.4 further shows that the gap is positive for both K=3 and K=4, and occurs on the vast majority of individual concepts.
- We replace the original tests with a concept-level paired analysis.
Table 4: Paired comparison of implicit and explicit predictions. denotes the mean paired accuracy difference across concepts, with CIs estimated via -stratified bootstrap. The pooled row averages paired differences across models for each concept.
| Model | 95% CI | |||
|---|---|---|---|---|
| Qwen2.5-72B | 0.84 | 0.61 | +0.23 | [0.20, 0.26] |
| Llama-3.1-70B | 0.73 | 0.57 | +0.16 | [0.12, 0.19] |
| Gemma-2-27B | 0.73 | 0.58 | +0.16 | [0.13, 0.19] |
| Mistral-24B | 0.77 | 0.53 | +0.24 | [0.21, 0.27] |
| Pooled (64 concepts) | -- | -- | +0.19 | [0.17, 0.22] |
Response to Reviewer ny9X - Part I
Thank you very much for your valuable feedback! We have revised the manuscript and provide our detailed responses below.
W1-1 and Q1: The full-information interventions in Table 3 leave nothing to infer from the demonstrations.
Table 1: Progressive revelation of structural information for concept recovery. Rj denotes the condition in which j of the ∣A∣ ground-truth relevant attributes are revealed to the model, with results averaged over all (j∣A∣) possible subsets. Qexp(Rj) measures behavioral recovery on held-out queries under this condition. Thus, R0 reveals none of the relevant attributes, whereas R∣A∣ reveals all of them. Implicit Prediction denotes direct query prediction from demonstrations without providing the relevant attributes. Levels for each attribute are not provided and blank entries correspond to undefined conditions.