acceptodds
Under review as a conference paper at ICLR 2027

Beyond Fact Frequency: Controlled Evidence on Expression Allocation and Factual Callability

Abstract

Fact frequency does not specify how a fixed fact’s exposure is distributed across linguistic expressions. We isolate this within-fact expression-allocation estimand by matching fact identity, exposure count, answer occurrence, tokenizer and train- ing budgets, and corpus-level template marginals. In controlled synthetic settings, distributing fixed exposure across faithful expressions instead of repeating one sub- stantially improves held-out Forward callability: +28.75 percentage points across ten Qwen3-0.6B training seeds, with replication across independently generated worlds from the same programmed generator, larger Qwen and Llama models, natural held-out wording, and a same-family Qwen3 persistence test through the largest tested 8B checkpoint. We then apply the same intervention to a five-domain natural-source panel at two Qwen sizes, where generated behavior and teacher- forced target-answer likelihood are heterogeneous across domains and model sizes, with strict exact match remaining at floor. Dedicated stress tests do not support a universal relation-preservation mechanism. Thus, in our controlled construction, equal fact frequency need not imply behaviorally equivalent training exposure, while natural-data, bidirectional-transfer, scaling-law, and internal-mechanism claims remain bounded.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.