acceptodds
Under review as a conference paper at ICLR 2027

Distilling Directional Verification

Abstract

Knowledge distillation aims to transfer the factual knowledge of large language models to smaller models for efficient deployment. Yet a teacher may recall a relation in one direction while failing to generate the answer in the reverse direction. Distillation from its generated answers can therefore propagate this directional limitation to the student. The same teacher can nevertheless recognize such an answer by scoring the relation in the direction it knows. We introduce directional label distillation, in which frozen teachers score candidate answers in that known direction and the best-scoring candidate becomes the student's training target. On facts about parents and their children, known-direction scoring yields more accurate labels than scoring the requested direction, even after tuned corrections for name priors. With prior-corrected scores, the better direction depends on the facts rather than the template, and reverses on mined facts whose notable entity is the parent rather than the child. With the evaluated children's forward facts withheld, students trained on known-direction labels improve open-ended accuracy on their trained queries by 13 to 15 points over students trained on prior-corrected reverse labels. After generated answers are matched to a fixed name list by lexical similarity, students reproduce nearly all selected labels. Their accuracy largely follows label quality. The label advantage holds on unscreened queries and when candidates are retrieved without inserting correct answers. Our findings show that directional verification mitigates the transfer of errors from teacher-generated answers to students by providing more accurate training targets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reversal curse, knowledge distillation, large language models, directional label distillation, pseudo-labeling, masked diffusion language models, factual knowledge

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KPfPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper distinguishes between generation and verification as two ways a pretrained teacher can expose its knowledge. When a teacher has learned a factual relation in one direction (A -> B), it may fail to generate the relation in the reverse direction (B -> A) while still being able to recognize the correct relation when evaluated in the direction it knows. Based on this observation, the paper introduces directional label distillation, where the teacher does not generate the reverse answer directly. Instead, it scores a set of candidate answers using the known direction, aggregates scores from multiple teacher channels, and selects the highest-scoring candidate as a pseudo-label for training the student.

The paper defines a known-direction scoring function in which the candidate changes while the continuation being scored remains in the direction where the teacher has stronger knowledge.

It also compares against reverse-direction scoring and addresses the effect of candidate name priors using PMI-like prior correction. The experiments further show that student accuracy largely follows pseudo-label quality: students reproduce the selected labels at very high rates, so improvements in label accuracy translate into improvements in student accuracy.

The method is evaluated on Wikidata parent–child relations. Known-direction scoring consistently produces more accurate labels than reverse-direction alternatives. Students trained with known-direction labels achieve approximately ~15 percentage points higher open-ended accuracy than students trained with prior-corrected reverse labels.

Critical strengths
  • Distinction between teacher generation and verification: The paper identifies an important phenomenon: a pretrained model may fail to generate a fact in one direction while still being able to recognize the correct fact when candidates are scored in a direction where its knowledge is better expressed.
  • Directional label distillation is a simple and effective formulation: The paper formalizes the above insight through a known-direction score, selects the candidate with the highest score, and uses the selected answer as a pseudo-label for student distillation. This turns the conceptual observation into a simple, practical distillation procedure, and the experiments show that the resulting labels consistently lead to substantially better student accuracy than reverse-direction alternatives.
Critical weaknesses
  • Computational cost: The candidate-scoring procedure may be substantially more computationally expensive than conventional distillation, since multiple teachers must score potentially many candidates. Although the paper acknowledges this limitation, it does not quantify the computational overhead or demonstrate that the proposed approach remains practical as the candidate inventory or number of teacher channels grows.
  • Potential pretraining exposure: The paper makes a commendable effort to prevent adaptation-time exposure by removing forward edges involving the relevant children before training, so the student does not see those facts in the forward training data. However, this does not establish that the student has never seen these facts during pretraining. The evaluation relies on relatively well-known people and parent-child relationships from Wikipedia, which could easily have appeared in the pretraining corpus of Qwen3 or other models. The warm-start controls show that reverse-label training is important, but do not establish that the observed gains represent acquisition of previously unknown knowledge rather than improved elicitation or memorization of knowledge already present in the student. This is particularly relevant given the paper's broader claim about knowledge transfer.
  • Hard-label SFT vs. distillation: The student is ultimately trained only on the selected final answer, making the training procedure resemble pseudo-labeling/SFT more than conventional knowledge distillation with a dense teacher signal. While the teacher's verification scores are used to construct higher-quality labels, the paper does not investigate whether retaining the teacher's score distribution or other soft supervision provides additional benefits. Clarifying the distinction between directional label distillation and pseudo-label-based SFT, and comparing against soft-distillation alternatives, would strengthen the methodological contribution.
  • Generality beyond the studied factual setting: It remains unclear whether directional verification continues to work for harder reasoning tasks and more general knowledge relations. The experiments focus on parent-child relations with a fixed candidate inventory, while the paper's central motivation is broader: that a model may be able to verify knowledge in a direction where it cannot generate it.
Questions for authors

Please address the concerns raised above (Critical Weaknesses Section).

16 comments
AuthorsOpenReview ↗

Response to Reviewer KPfP (1/9)

We thank the reviewer for the careful reading and for recognizing the important phenomenon the paper identifies and its simple and effective formulation. The revision answers each of the four weaknesses with a new measurement, and text revised in response to the reviews is set in blue. All four teachers label 1,000 queries in about seven minutes on one A100 card, at the same cost as reverse scoring (Appendix L). On invented facts that neither the student nor the untouched teacher holds, students trained on known-direction labels answer 96.8% of the reverse queries against 1.9% with summed-DC labels (Appendix M). Students trained on the selected label lead students trained on the teachers' score distribution in every paired seed at four temperatures (Appendix N). The advantage holds on three further relations (Appendix J) and on two-hop queries, where students trained on known-direction labels lead in all 24 seed-paired comparisons (Appendix O). New Section 4.5 on page 8 reports the results beyond parent and child facts in the main text, and each point below gives its place in the revised PDF.


W1. All four teachers label 1,000 queries of 64 candidates in 427 A100-seconds, about seven minutes on one card and less than a quarter of one MDM-0.6B training run, or less than half with teacher loading included, at the same cost as reverse scoring. With OLMo-2-7B as the single teacher, each correct label costs less than one seventh as much as with generation from the same teacher, and label accuracy peaks at eight channels, so the channel count need not grow.

"Computational cost: The candidate-scoring procedure may be substantially more computationally expensive than conventional distillation, since multiple teachers must score potentially many candidates."

"it does not quantify the computational overhead or demonstrate that the proposed approach remains practical as the candidate inventory or number of teacher channels grows."

We timed the label sources of the paper on one A100 80GB card, on the 512 uniform lists of the unscreened cohort with 64 candidates each, and scaled the times to 1,000 queries (Table R3, new Appendix L and Table 36 of the revision). Label accuracy is on the 512 uniform lists (Tables 13, 18 and 20).

Table R3. A100-seconds to label 1,000 queries of 64 candidates, scaled from the 512 uniform lists, and label accuracy (%) on the 512 uniform lists. Summed DC adds one context-only score for each distinct name, paid once.

Label sourceTeachersA100-secondsLabel accuracy
Known directionOLMo-2-7B10781.05
Summed DCOLMo-2-7B107, plus 9 once64.06
Generation, greedy and eight samples, best of 18 rules mapped onto the listOLMo-2-7B40.44.10
Known directionfour teachers42787.89
Summed DCfour teachers425, plus 36 once73.44
  • Overhead. Each candidate costs one forward pass of a 7 to 8B teacher, 1.6 to 1.7 ms for each candidate and teacher, with no decoding, so the four teachers label 1,000 queries in about seven minutes on one card, after a one-time loading of 456 seconds. Known-direction scoring costs the same as the reverse scoring that every reverse comparator of the paper uses. Against generation, each correct label costs less than one seventh as much, 0.13 A100-seconds against 0.99. The best of the 18 generation rules of Table 20 is the eight-sample majority, which breaks ties with the greedy label (Appendix F) and so needs the full 40.4 A100-seconds of greedy and eight-sample decoding with OLMo-2-7B. It gives 4.10% correct labels on these lists, against 81.05% from the same teacher's known-direction scores, so 107 seconds of scoring yield about twenty times as many correct labels as 40.4 seconds of generation.
AuthorsOpenReview ↗

Response to Reviewer KPfP (2/9)

W1, inventory size and channel count. Scoring a candidate list costs the same at any inventory size, the full-inventory scan grows only linearly and is shared by every label source, and label accuracy stops rising at eight channels.

  • Inventory size. Label selection scores only the candidate list, so its cost does not depend on the inventory, and the 64-name lists of the unscreened cohort and of the three new relations need no teacher scan. Building the acquired pools, where three teachers scan all 8,218 names, takes about 41 A100-seconds for each query and scoring direction at the same rate (three teachers times 8,218 names at 1.6 to 1.7 ms, new Appendix L) and grows linearly with inventory size and teacher count (Appendix E, page 21, page 22 in the revision). This scan is shared by every label source that picks from these pools, including all reverse comparators and the list-mapped generation of Table 20, so every label source starts from the same scan. A full scan of this kind, run for the 512 unscreened parents with no inserted answer, gives Table 16, where the known-direction pipeline labels 66.21% correctly against 39.06% for the summed-DC pipeline. For inventories far larger than 8,218 names a cheaper proposal stage can run in front of the scorer (Section 7 of the revision), and the label-selection cost of Table R3 stays the same because it depends only on the list.
  • Number of channels. A channel is one teacher and template pair (Section 2), so each added channel is one more scoring pass, about 107 A100-seconds for 1,000 queries of 64 candidates for each 7 to 8B teacher we timed. On the 1,500 acquired pools label accuracy is 90.20, 91.53, 91.07 and 90.80% with 6, 8, 10 and 12 channels (Table 28), so it stops rising at eight. The eight-channel labels of the paper's default-exposure controls, four teachers with two templates each and so twice the four-teacher scoring passes of Table R3, already exceed each single-teacher pipeline, 91.53% against 81.13% to 83.67% (Table 26b).
  • The cost is paid once and offline. Labeling 1,000 queries with all four teachers takes 0.12 A100 GPU-hours of scoring, or 0.25 with the one-time loading of the four teachers, less than half of one MDM-0.6B training run, which took 0.53 A100 GPU-hours including model loading and evaluation (mean of eight runs on A100-SXM4 cards). Every student reuses the labels, and the teachers are discarded at inference (Section 2.3), so the label source does not change the student's inference cost. Two students that differ only in their label source both answer 1,000 queries in 15.34 seconds at batch size 64 on one RTX A6000 card, scaled from their 512 trained reverse queries (new Appendix L).
AuthorsOpenReview ↗

Response to Reviewer KPfP (3/9)

W2. On invented facts that neither the student nor the untouched teacher holds, students trained on known-direction labels answer 96.8% of the reverse queries, against 1.9% with summed-DC labels and 0.0% without reverse labels, so they learn the facts stated in their labels and do not recall them from pretraining.

"The warm-start controls show that reverse-label training is important, but do not establish that the observed gains represent acquisition of previously unknown knowledge rather than improved elicitation or memorization of knowledge already present in the student."

This question is central to understanding how language models acquire knowledge, because distillation helps only if the student comes to hold knowledge it lacked, and the reversal curse is the case in which the student cannot infer that knowledge on its own. The evidence below shows that the students learn the facts stated in their labels.

We ran the direct test of acquisition, with facts the student does not hold before training (Table R6, new Appendix M and Table 37 of the revision, to which Section 3 of the revision now points). We invented 1,000 parent and child pairs with pseudoword surnames, each used once and absent from our 10,505-pair corpus. The untouched Qwen3-8B completes neither direction for any pair (0 of 1,000 each) and picks the true child at chance (1.2% against 1/64 = 1.6%). After LoRA training only on child-first sentences, the teacher recalls 96.2% of parents yet names the true child for 0 of 1,000 parents. MDM-0.6B students then follow the paper's recipe, with forward training on the real corpus only, so the students see the invented facts only in the reverse labels.

Table R6. Invented facts, 1,000 trained reverse queries with 64-name lists. Teacher label accuracy and student open and matched accuracy (%, mean ± sample SD over three seeds).

Label sourceLabel accuracyStudent open accuracyStudent matched accuracy
Known direction100.096.8 ± 0.999.4 ± 0.3
Summed DC1.91.9 ± 0.01.9 ± 0.0

Reverse labels from mean and summed scores are 2.2% and 1.8% correct, and students trained without reverse labels answer 0.0% of the queries. The student therefore acquires facts that neither it nor the untouched teacher holds, and it learns them from the known-direction labels.

Three results already in the submitted paper show the same on real facts.

  • Students trained on wrong labels almost never produce the true child. The reverse-mean students see 645 wrong labels among the 1,390 exposure-free queries, and under inventory matching they name the true child for at most 5 of them in any run and for none under open scoring (Table 12, columns F, Appendix D, page 19, page 20 in the revision). Students reproduce wrong labels as faithfully as right ones. On the 666 exposure-free queries where the two label sources disagree, reverse-mean labels are 5.26% correct and known-direction labels 82.73% correct, and after inventory matching students return their selected label for 96.60% and 97.25% of these queries (Table 11, Appendix D, page 19, page 20 in the revision). None of the known-direction, reverse and summed-DC students turns a wrong label into a whole-answer match (Appendix D, same page). If training elicited knowledge already in the student, wrong labels would be corrected far more often.
  • Students trained without reverse labels, MDM-4B and AR-8B models larger than the MDM-0.6B students of Table 1, stay below 3% open accuracy on all 1,500 trained queries of the screened cohort, including with the any-order masked objective MLM-U (Table 3b, page 8), so their pretraining, even together with every true forward pair, does not make them generate these reverse answers. Their inventory-matched accuracy in the same table, 28.16% to 44.38%, stays below the 69.80% that matching on the surname alone reaches with the original inventory, so it reflects surname cues that the matching step can use. Enlarging the matching inventory with parent-only names cuts the warm-start MDM-4B students from 39.56% to 10.69%, while students trained on reverse labels change by less than one point (Section 5.2, Table 27b, page 28, page 29 in the revision).
  • Student answers follow the labels over the whole range of label quality, and matched accuracy stays within about one point of label accuracy from random labels, 14.13% correct and 13.91% matched, to gold labels, 100% correct and 98.91% matched (Table 3a, page 8). The same holds in the new student settings of Section 4.5. After inventory matching every student arm stays within 0.26 points of its label accuracy on the three new relations (new Appendix J), and within 0.46 points on three-seed means with teachers of up to 70B parameters (new Appendix K) and on two-hop queries (new Appendix O).
Reviewer_hTSZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the reversal curse in large language models and proposes Directional Label Distillation, a method for constructing reverse supervision labels by exploiting the teacher's knowledge in a more reliable direction. Instead of directly generating reverse answers, the method retrieves candidate entities and scores them using conditional probabilities in the known direction. The highest-scoring candidates are then used as pseudo-labels to train student models. The authors evaluate the approach on parent–child relations extracted from Wikidata, using multiple teacher models and masked diffusion students. The experiments show that known-direction scoring produces more accurate pseudo-labels than reverse scoring, including prior-corrected variants. These improvements are further reflected in student accuracy, suggesting that directional verification can provide useful supervision when direct generation is unreliable.

Critical strengths

(1)A meaningful perspective on addressing the reversal curse. The paper presents an interesting approach to addressing the reversal curse by exploiting the teacher model's existing relational knowledge rather than relying on its potentially unreliable reverse generations. Constructing reverse supervision through known-direction verification is a useful perspective on knowledge distillation.

(2)Consistent empirical improvements. The proposed approach achieves substantial improvements in pseudo-label accuracy and student performance over several reverse-scoring baselines. The additional analyses of scoring directions, prior correction, and candidate selection further strengthen the empirical findings within the evaluated setting.

Critical weaknesses

(1)Limited evaluation across relation types. The experiments focus exclusively on parent–child relations extracted from Wikidata. Although the authors consider different candidate construction strategies and additional controls, all experiments remain within the same relation type. It is therefore unclear whether the observed advantage of directional verification generalizes to other relational knowledge. This limitation substantially weakens the evidence for the broader applicability of the proposed approach.

(2)Dependence on a predefined candidate inventory. The method relies on a predefined inventory of candidate entities. Although Appendix E includes full-inventory retrieval experiments without inserting the correct answers, the evaluation still assumes that answers can be selected from a fixed entity inventory. This assumption may limit applicability to open-ended factual questions, where constructing a sufficiently comprehensive candidate set is difficult. The paper does not sufficiently demonstrate how the approach would perform when the candidate space is incomplete or unavailable.

(3)Limited evaluation of stronger teacher models. The main experiments rely on teacher models of approximately 7B–8B parameters. Although Appendix G provides additional teacher-scaling experiments, the evaluation remains limited in model scale. Since the approach depends on the reliability of teacher likelihood scores, it is unclear whether the same directional advantage persists with substantially stronger models or whether improved reverse-generation capabilities would reduce the observed gap.

(4)Insufficient analysis of computational efficiency. The proposed pipeline requires scoring numerous candidate entities using multiple teacher models. While the appendix describes the full-inventory scans and their computational complexity, the paper provides limited quantitative analysis of the practical computational overhead. Reporting GPU hours, inference latency, and computational cost relative to alternative pseudo-label construction methods would help establish the efficiency and scalability of the approach.

Questions for authors

(1)Generalization beyond parent–child relations. The experiments are restricted to parent–child relations. Do the authors have evidence that the advantage of known-direction verification extends to other relation types? Such results would substantially strengthen the generality of the central claim.

(2)Dependence on the candidate inventory. While Appendix E evaluates full-inventory retrieval without answer insertion, the method still operates over a fixed set of entity names. How sensitive is the approach to candidate inventory completeness, and how would it handle questions whose correct answers are not represented in the inventory?

(3)Teacher reliability and computational cost. Appendix G examines teacher scaling, but the experiments mainly involve relatively small models. Could the authors clarify whether stronger teachers would preserve the same directional advantage? Additionally, what is the practical computational cost of the multi-teacher candidate-scoring pipeline compared with direct generation or other pseudo-label construction approaches?

14 comments
AuthorsOpenReview ↗

Response to Reviewer hTSZ (1/11)

We thank the reviewer for the careful reading and for recognizing the perspective on the reversal curse and the consistent gains over reverse-scoring baselines. Each of the four weaknesses asks for evidence, and the revision adds a new experiment for each. Table numbers are consistent throughout this response, and text revised in response to the reviews is set in blue in the revised PDF.

Evidence for the four concerns.

Answer. Verification supplies more accurate reverse labels across relations and teacher scales, detects missing answers, and lowers the cost of a correct label relative to generation.

W1, Q1. Other relation types. Known-direction or fact-level-rule labels improve student answers on each further relation. The advantage therefore extends beyond parent-child facts (Appendix J, Tables R1 and R1b).

W2, Q2. Candidate completeness. The method selects reverse labels from proposed candidates rather than generating answers without candidates. The pipeline works without an inserted answer and uses teacher disagreement to flag missing answers. Its advantage therefore does not require a curated list containing the correct answer (Appendices E and H, Table R5).

W3, Q3. Stronger teachers. Larger teachers retain a greater known-direction lead within every family, and their students retain the gain. Better reverse generation therefore does not remove the value of verification (Appendix K, Tables R2 and R2b).

W4, Q3. Practical cost. Known-direction labeling uses the GPU time of reverse scoring and costs less for each correct label than generation. The gain remains practical without adding teacher work at student inference (Appendix L, Table R3).

PointResult and revision location
W1, Q1, other relationsOn author to work, director to film and place of birth the known direction leads the summed reverse score on all six lists by 22.07 to 36.33 points, and students trained on the more accurate labels lead in all 18 paired seeds (new Section 4.5 and Appendix J).
W2, Q2, candidate inventoryWith no inserted answer the known-direction pipeline labels 66.21% of queries correctly against 39.06% for summed DC, and when the answer is missing, teacher disagreement flags 95.70% of uniform lists without any gold label (Table 16 and new Appendix H).
W3, Q3, stronger teachersWith 32B and 70B teachers the known direction leads the tuned reverse comparator on every list by 8.98 to 26.76 points, and the lead is larger than with the 7 to 8B teacher of the same family in all six comparisons (Section 4.3 and new Appendix K).
W4, Q3, costKnown-direction labels for 1,000 queries of 64 candidates take 0.12 A100 GPU-hours with four teachers, the same as reverse scoring, and with one teacher each correct label costs less than one seventh as much as with that teacher's generation (new Appendix L).

Verification improves labels across relations and teacher scales, handles missing candidates through disagreement, and has measured practical costs.

The comparisons establish label and student gains across relations and stronger teachers, missing-answer handling and practical computational costs. Revised-PDF location. New Appendices H, J, K and L.

AuthorsOpenReview ↗

Response to Reviewer hTSZ (2/11)

W1, Q1. Other relation types.

"It is therefore unclear whether the observed advantage of directional verification generalizes to other relational knowledge."

"Do the authors have evidence that the advantage of known-direction verification extends to other relation types?"

Answer. Yes, the advantage holds on three further relations.

To test whether directional verification generalizes to other relational knowledge, we ran new experiments on author-to-work, director-to-film and place-of-birth relations. This question matters because the gap the paper targets is a serious one, central to understanding how language models acquire knowledge, and it is not specific to families. A model trained on text that states a fact mostly in one direction, such as who wrote a work or where a person was born, often cannot state it in the other (Berglund et al., 2024). Distillation from the teacher's generated answers passes that failure on to the student (Section 1). A person who knows one direction of such a fact answers the other at once. A teacher that can check the fact but not state it therefore holds knowledge that distillation from generated answers leaves unused.

We mined author to work, director to film and place of birth from Wikidata as for the unscreened cohort (Appendix E). Each relation has 512 queries and a hashed sample of 8,218 answers as its inventory. The same four teachers score its uniform and lexical 64-name lists.

Table R1. Label accuracy (%) on three further relations, four-teacher means, 512 queries for each list. Differences use counts before rounding.

RelationListKnown directionReverse, summedSummed DCFact-level ruleGeneration, best teacher
Director to filmUniform37.7012.5029.1038.871.95
Lexical32.8110.7424.6134.383.71
Place of birthUniform51.3715.0438.8750.392.54
Lexical39.069.7729.1040.042.34
Author to workUniform58.0134.7760.9467.381.56
Lexical55.8631.0557.0364.261.95

On author to work, director to film and place of birth, the most accurate labels on every list come from checking candidates in the known direction or from the label-free fact-level rule, which picks the direction for each fact. Students trained on known-direction or fact-level-rule labels lead students trained on summed-DC labels in all 18 paired seeds. Generation rarely supplies the reverse answer, while candidate verification recovers it.

The advantage of directional verification holds on all three further relations.

Revised-PDF location. Table R1, new Appendix J and Table 34 of the revision.

AuthorsOpenReview ↗

Response to Reviewer hTSZ (3/11)

W1, Q1. Other relation types, continued.

"Do the authors have evidence that the advantage of known-direction verification extends to other relation types?"

The fact-level rule also leads summed DC on all six lists, and teacher agreement identifies more reliable labels.

The known direction leads the summed reverse score on all six lists and summed DC on director to film and place of birth. The table below separates the direction-choice gains from the agreement signal.

The fact-level rule of new Appendix I of the revision reads only teacher scores. It labels each query with the direction in which more teachers agree with the four-teacher choice, and it is applied here unchanged.

Comparison on the three relationsValue
Known direction over summed reverse, all six lists22.07 to 36.33 points
Known direction over summed DC, director to film and place of birth, paired 95% intervals above zero8.20 to 12.50 points
Fact-level rule over summed DC, all six lists, every interval above zero6.45 to 11.52 points
Fact-level rule over the reverse score tuned on each list's own gold labels3.71 to 11.52 points
Fact-level rule against the known direction, director to film and place of birthwithin 1.56 points
Author to work, queries where at least one fixed direction is correct (uniform / lexical)74.22% / 70.51%
Author to work, fact-level rule over the better fixed direction, points [95% interval]6.45 [3.12, 9.77] / 7.23 [3.91, 10.55]
Four-teacher agreement under the known direction, agreed labels correct78.85% to 94.81%
All labels correct, same lists32.81% to 58.01%
Correct agreed labels relative to four-teacher agreement on summed DC1.7 to 4.8 times
Teachers stating the answer on all three relationsat most 3.71% of queries

A fixed direction can vary in usefulness across facts, while teacher agreement improves direction choice and identifies accurate labels.

  • On author to work the two fixed directions are statistically tied, with both intervals of their difference including zero, and they are right on different queries. This is the case that Section 7 of the revision describes, where the better direction varies from fact to fact, and the rule chooses it for each fact from teacher scores alone.
  • Where precision matters more than coverage, the labels on which all four teachers agree are far more often correct than the labels overall, and on every list there are more of them than under summed DC.

On all three relations the teachers state the answer rarely. Yet they pick it far more often when they check each candidate in the direction they know. Where the better direction changes from fact to fact, their own agreement chooses it without gold labels.

The gap between what a teacher can check and what it can state is not specific to families.

Revised-PDF location. New Appendix J.

Reviewer_mRGnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a directional label distillation method to mitigate the unreliable reverse answer generation problem caused by the reversal curse in LLMs. The method uses frozen teacher models to score candidate answers in a more reliable relational direction, selects high-quality pseudo-labels, and then fine-tunes student models to answer reverse queries. The authors conduct extensive experiments and detailed ablation studies to validate the effectiveness of the proposed approach. Experimental results demonstrate that directional verification improves pseudo-label accuracy and achieves approximately 13–15 percentage-point improvements in student open-ended accuracy over prior-corrected reverse-label baselines on trained queries.

Critical strengths
  1. Extensive experiments demonstrate the effectiveness of directional verification. The authors conduct comprehensive experiments across multiple teacher models, scoring strategies, candidate selection methods, and student configurations. The consistent improvements over various baselines provide convincing evidence.
  2. The paper provides meaningful insights into the relationship between verification direction and knowledge accessibility. The experiments demonstrate that the preferred verification direction depends on the underlying factual knowledge. The analysis of name priors and entity notability provides useful empirical insights into why directional scoring can outperform direct reverse scoring.
Critical weaknesses
  1. The novelty of the method is relatively limited. The core method relies on reverse conditional likelihood scoring, which is closely related to existing noisy channel formulations, followed by conventional pseudo-label selection and supervised fine-tuning.
  2. The experiments does not sufficiently demonstrate generalizable reversal capabilities. The student models are primarily evaluated on reverse queries whose pseudo-labels were explicitly included during training, making it difficult to distinguish improved relational understanding from memorization of selected labels. Furthermore, the experiments focus exclusively on parent-child relations, leaving the effectiveness of the approach on unseen facts and other relation types unclear.
Questions for authors
  1. Does directional label distillation improve generalization beyond explicitly supervised reverse queries? The current student evaluations primarily focus on queries whose pseudo-labels were included during training. Could the authors provide results on unseen reverse facts to clarify whether the improvements extend beyond memorization of training labels?
  2. How can the appropriate verification direction be determined automatically? The experiments suggest that the preferred scoring direction changes with entity notability. How could the method identify the more reliable direction for previously unseen facts without relying on ground-truth labels or prior knowledge of the relation?
  3. How does the proposed method generalize to other relation types? The experiments are limited to parent-child relationships, which may exhibit specific directional characteristics. Could the authors clarify whether similar improvements can be applied for other factual relations, such as person-birthplace, author-book or other relationships?
13 comments
AuthorsOpenReview ↗

Response to Reviewer mRGn (1/10)

We thank the reviewer for the careful reading and for finding that the consistent improvements over various baselines provide convincing evidence. The revision adds a main-text Section 4.5 (page 8) that reports students trained on invented facts that neither the student nor the untouched teacher holds (Appendix M) and three further relations (Appendix J), and Section 4.4 now reports a label-free rule for choosing the direction (Appendix I). Section 7 of the submission named a label-free choice of direction as the natural next step, and the revision adds it together with three further relations. Each point below gives its place in the revised PDF, where text revised in response to the reviews is set in blue.

Response overview.

Answer. Verification direction recovers labels that reverse scoring and generation miss, and those labels transfer previously unknown facts to students.

W1. Novelty beyond noisy-channel scoring. The method shares likelihood scoring and supervised fine-tuning with prior work and chooses the scoring direction for each fact, without claiming a new formula or student objective. Verification direction supplies more accurate reverse labels under the same student objective. The gain comes from knowledge recovered by the check, rather than from a new channel formula or student training recipe (Introduction, Related work and Appendix K).

W2, Q1. Acquisition and unseen facts. The student evaluation tests learning from teacher-inferred reverse labels rather than student inference on reverse facts receiving no label. The labeling step infers reverse facts without recorded answers, and students acquire previously unknown invented facts. The claim concerns label inference and knowledge transfer, rather than unaided student inference on unlabeled reverse facts (Appendix M).

Q2. Automatic direction choice. Teacher agreement selects the direction from scores alone, including separately for facts in a mixed batch. Direction choice therefore requires neither gold labels nor prior knowledge of the relation (Appendix I, Table R4).

Q3, W2. Other factual relations. Known-direction or fact-level-rule labels improve student answers on the further relations. The label and student gains therefore extend beyond parent-child facts (Appendix J, Tables R1 and R1b).

Verification supplies accurate labels across relations, students acquire previously unavailable facts, and teacher agreement chooses the direction without gold labels. Revised-PDF location. New Appendices I, J and M.

AuthorsOpenReview ↗

Response to Reviewer mRGn (2/10)

W1. Novelty and the noisy-channel connection.

"The core method relies on reverse conditional likelihood scoring, which is closely related to existing noisy channel formulations, followed by conventional pseudo-label selection and supervised fine-tuning."

Answer. The method shares likelihood scoring and supervised fine-tuning with prior work and chooses which sentence direction to score for each fact. It does not claim a new noisy-channel formula or student training objective. It studies which direction lets a teacher verify facts it cannot state and turns those checks into reverse labels.

Pretrained teachers are not coherent across the two directions of a fact. We compare the known direction with four-teacher reverse scores whose prior correction is tuned with gold labels in all three cohorts of the paper (Section 4.3, Tables 18 and 19).

To our knowledge, earlier analyses of language models have not asked in which direction a model can check a fact that it cannot state, or whether that check can supervise a student. Work on the reversal curse measures the failure and changes the training data or the objective (Berglund et al., 2024, and the work cited in Section 1). Distillation builds targets from what the teacher generates. For a person the two directions of a fact are the same knowledge. A model that confirms that Y is X's parent yet cannot name Y's child therefore holds knowledge it cannot use, and the paper turns that knowledge into correct labels.

The paper builds on the noisy-channel connection in Sections 1 and 6 (Min et al., 2022). Under one coherent joint distribution, a channel score and a direct score corrected for the candidate prior rank the candidates identically (Section 1, page 2). In few-shot classification the channel form is therefore a way to remove label priors. The paper measures how far pretrained teachers depart from that coherence.

The table below compares the two directions after gold-label prior correction, for the paper's teachers, larger teachers and their students.

Lead of the known directionPoints
Gold-tuned four-teacher reverse scores across the three cohorts8.92 to 14.06
Four-teacher reverse score, coefficient tuned on other cohorts and transferred, all three cohorts (Table 19)10.07 to 14.06
Same, coefficient tuned on each cohort's own gold labels (Table 18)8.92 to 12.50
Reverse labels tuned on the cohort's gold labels, Qwen3-32B, OLMo-2-0325-32B and Llama-3.1-70B-Instruct, every list8.98 to 26.76
Their students, open accuracy, all 18 seed pairs, three-seed means (new Appendix K, Table 35 of the revision)9.05 to 26.17
Unscreened students over summed-DC and transferred tuned-reverse students, every paired seed (Section 4.1, Table 1)13.09 to 15.36

Tuning the prior correction with gold labels does not close the verification-direction gap. More accurate labels also produce more accurate student answers under the fixed training recipe.

The noisy-channel form alone does not determine that reliable direction.

Revised-PDF location. New Appendix K.

AuthorsOpenReview ↗

Response to Reviewer mRGn (3/10)

W1. Novelty and the noisy-channel connection, continued.

"The core method relies on reverse conditional likelihood scoring, which is closely related to existing noisy channel formulations, followed by conventional pseudo-label selection and supervised fine-tuning."

The reliable direction also changes with the facts, and verification supplies better targets than generation.

The direction-choice rule reads only teacher scores, and the target comparison holds the student training recipe fixed.

  • The better direction coincides with the notable entity (Section 7). It switches from child-to-parent on corpus facts to parent-to-child on mined facts with a notable parent, for both query sides and both candidate lists, with the shift quantified below (Section 4.4, Figure 4b, Table 24).
  • The teachers' agreement chooses this direction for each fact without labels. The fact-level rule of new Appendix I exceeds the better fixed direction when the two fact sets are mixed. On the three further relations it leads summed DC and the reverse score tuned on each list's own gold labels on all six lists (Q2 and Q3 below, new Appendices I and J). Section 6 of the revision now sets this choice for each fact against channel scoring and calibration, which fix one score form and one correction for a task.
  • As a distillation target, verification succeeds where generation fails (table below). Even when generation runs in the direction the teacher knows and is inverted into votes, only a minority of queries receive a label (Table 21). The known direction leads this inverted generation.

Holding the recipe fixed isolates label quality (Section 1, pages 2 and 3, and Section 4.1). The paper separates candidate coverage, label correctness and label reproduction (Tables 2, 11 and 12) to show that the student gain comes from better targets. Sequence-level distillation (Kim and Rush, 2016) likewise trains the student on one teacher output for each input.

Direction and generation checksValue
Mined facts with a notable parent1,024 facts
Change in the better direction with entity notability18.68 points
Fact-level rule over the better fixed direction, mixed fact sets3.06 points
Inverted known-direction generation, queries receiving a labelfewer than 30%
Known direction over inverted generation54.49 to 70.90 points

The direction depends on the underlying facts and can be chosen without labels. Even generation in the known direction fails to supply labels for most reverse queries.

Label accuracy (%)Acquired poolsUniform listsLexical lists
Teacher completions of the reverse prompt mapped onto the lists, best rule (Table 20)at most 17.05at most 4.10at most 2.73
Known direction (Table 18)90.7287.8971.29

Candidate verification supplies far more correct reverse labels than teacher-generated answers across every candidate construction.

The paper studies verification direction as a source of reverse labels, choosing it for each fact and recovering labels that generation misses, while the student training recipe stays standard by design (new Appendices I and J). This matters because distillation can transfer knowledge the teacher cannot state, as the Introduction and the Related work section's Verification and weak supervision paragraph explain.

Section 2.3 of the revision now states this relation.

Comments on the paper

8 comments
AuthorsOpenReview ↗

General Response (1/7)

We thank the reviewers for their careful reading. The revised PDF adds eight new experiments, and four results answer the main concerns. On three further relations a rule that reads only teacher scores leads summed-DC reverse labels on all six lists by 6.45 to 11.52 points (Table R1). With teachers of up to 70B parameters the known direction leads the reverse score tuned on gold labels by 8.98 to 26.76 points. This lead is larger than with the 7 to 8B teacher of the same family in all six comparisons, while these teachers generate a true child for at most 3.91% of parents (Table R2). On 1,000 invented facts that no model held before training, a teacher that generates none of the reverse answers gives 100.0% correct known-direction labels. Students trained on them answer 96.8% of the reverse queries against 1.9% with summed-DC labels (Table R6). Known-direction labels cost the same as reverse scoring. With OLMo-2-7B each correct label costs less than one seventh as much as with its own generation (Table R3).

Generality beyond family relations. The fact-level rule improves reverse labels across the further relations, and the more accurate labels improve student answers. Verification therefore supplies usable reverse supervision beyond families (Table R1, Appendix J).

Stronger teachers. Larger teachers still verify answers they rarely generate, and their directional lead grows within every family. Scaling the teacher therefore preserves the need for directional verification (Table R2, Appendix K).

Acquisition of previously unknown facts. Verification supplies correct invented-fact labels that students then learn despite failed teacher generation. The gain therefore transfers facts absent from the student's prior knowledge (Table R6, Appendix M).

Cost relative to alternatives. Known-direction labels use the GPU time of reverse scoring and cost less for each correct label than generation. Their accuracy gain therefore comes without extra reverse-scoring work (Table R3, Appendix L).

The reviewers agree that the paper identifies an important phenomenon (Reviewer KPfP), offers a meaningful perspective on the reversal curse with consistent gains over reverse-scoring baselines (Reviewer hTSZ) and gives insight into how the preferred verification direction depends on the underlying factual knowledge (Reviewer mRGn).

Most questions below ask whether one finding holds in further settings. That matters because the reversal curse is a serious problem, central to understanding how language models acquire knowledge. A model that learns a fact in one direction often cannot state it in the other, and the failure persists in the strongest models. What a model generates therefore understates what it has acquired. Distillation from a teacher's generated answers passes this failure on to the student.

Evidence for the reversal curseResult
Berglund et al. (2024), GPT-4 names a celebrity's parent / the celebrity from the parent79% / 33% of the time
Students without reverse labels, even with any-order masked trainingBelow 3% open accuracy (Section 5.1, Table 3)
One teacher, known-direction labels / its own generations81.05% / 4.10% correct (Table R3 below)

Failure to generate an answer does not imply failure to check it. The paper shows that the same teacher can check the fact in the direction it knows and train students that answer reverse queries.

All three reviewers ask whether the advantage holds beyond parent and child relations, and two ask about scoring cost. The other questions concern stronger teachers, incomplete candidate sets, facts the student does not already hold, soft targets, an automatic choice of direction and multi-step queries. We ran new experiments for each, listed below with their place in the revised PDF.

Verification gives more correct reverse labels than generation, and students trained on these labels answer the reverse queries. Revised-PDF location. New Appendices J, K, L and M.

AuthorsOpenReview ↗

General Response (2/7)

Relation to noisy-channel scoring.

Answer. The method shares likelihood scoring with noisy-channel approaches and chooses which sentence direction to score for each fact. It does not claim a new channel score. It studies which direction lets a teacher verify facts it cannot state and uses that check to construct reverse labels.

The paper finds that pretrained teachers break the equivalence that would make the two score forms interchangeable. A channel score and a prior-corrected direct score rank candidates identically when both come from one joint distribution (Section 1).

Known direction against reverse scoringLead
Reverse score whose correction is tuned on each cohort's own gold labels (Table 18)8.92 to 12.50 points
With a 70B teacher (Table R2)Up to 26.76 points

Prior correction does not make the score forms interchangeable for pretrained teachers. The direction of verification still determines which reverse labels they recover.

Channel scoring has served few-shot classification at inference (Min et al., 2022). Here the check builds the training labels of a student that answers the reverse query with no teacher present. Channel scoring and calibration also fix one score form and one correction for a task.

The new fact-level rule instead chooses the direction of the check for each fact from teacher scores alone (Section 6, Appendix I).

The paper studies verification direction as a source of student labels, including a separate direction choice for each fact. This matters because teachers can check facts they cannot state, as the Related work section's Verification and weak supervision paragraph and Appendix I explain. The channel form alone does not determine that direction.


Evaluation and label selection.

Answer. The student evaluation tests whether students learn reverse facts from teacher-inferred labels. It does not test student inference on reverse facts receiving no label. Label selection reads no recorded answer.

The claim of the paper concerns where correct reverse labels come from. Evaluating students on the queries whose labels they learn isolates the contribution of label quality to their answers (Section 7).

Answering a reverse query that receives no label would require the very inference that the reversal curse blocks. The method supplies that inference through the teachers' labels instead of asking it of the student. The main student comparisons keep only queries whose parent appears in no retained forward example (Section 3). A correct answer therefore cannot come from a forward example in training and traces to the teachers' check.

Label selection never reads the recorded answer, so any new reverse query with a candidate list is labeled and trained in the same way. The new experiments do so unchanged for three further relations, two-hop queries and invented facts.

Invented-fact controlResult
Students without reverse labelsAnswer 0.0% of queries (Table R6)

What the students answer comes from the labels and not from what they held before.

The paper studies whether teacher-inferred labels transfer reverse facts to students. This matters because verification supplies the reverse inference that generation misses, as the Conclusion and Limitations section explains. Label selection infers each fact without its recorded answer, and student evaluation isolates whether that label is learned.

AuthorsOpenReview ↗

General Response (3/7)

New results on relations and teacher scale.

Answer. The advantage holds beyond parent-child facts and grows with the larger teacher of every family.

The full tables appear in the replies to the individual reviews, and the revised PDF holds each result at the place given.

  • Table R1, other relations. On author to work, director to film and place of birth, which include the author and book and the person and birthplace relations raised in the reviews, the label-free fact-level rule leads summed DC on all six lists, with every 95% interval above zero. Students trained on the more accurate labels lead in all 18 paired seeds (Table R1b, new Appendix J, Table 34).
  • Table R2, larger teachers. With Qwen3-32B, OLMo-2-0325-32B and Llama-3.1-70B-Instruct the known direction leads the tuned reverse comparator, whose coefficient is fit on this cohort's own gold labels, on every list. Stronger teachers do not close the gap by generation, and their students keep the lead in all 18 seed pairs (Table R2b, new Appendix K, Table 35).
ResultMeasureValue
Table R1Fact-level rule over summed DC / over the reverse score tuned on each list's gold labels, six lists6.45 to 11.52 / 3.71 to 11.52 points
Table R1Known direction over summed reverse, every list / over summed DC, director to film and place of birth22.07 to 36.33 / 8.20 to 12.50 points
Table R1Greedy reverse generation mapped onto the lists, correctat most 3.71%
Table R1Four-teacher agreement, agreed labels correct / correct agreed labels relative to summed DC78.85% to 94.81% / 1.7 to 4.8 times
Table R1bStudent lead in open accuracy, known direction on director to film and place of birth / fact-level rule on author to work7.62 to 12.57 / 5.53 and 5.73 points
Table R2Lead over tuned reverse, larger teachers / increase over the 7 to 8B teacher, six comparisons8.98 to 26.76 / 0.78 to 8.59 points
Table R2True child named by generation, larger teachers / Llama-3.1-70B-Instruct known-direction labels correct0.98% to 3.91% / 96.09% and 86.91%
Table R2bStudent lead over tuned reverse, open accuracy, three-seed means9.05 to 26.17 points

Both further relations and larger teachers preserve the label-quality advantage. Every paired student seed also favors the more accurate verification labels.

The advantage extends beyond family relations and remains with the stronger teachers used for distillation. Revised-PDF location. New Appendices J and K.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.