acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Perplexity with Transferable Semantic Reweighting

Abstract

Perplexity provides an efficient likelihood-based signal for assessing how strongly a language model supports a target generation, but it uniformly aggregates token likelihoods despite large differences in their predictability and semantic importance. Consequently, variation driven by lexical frequency, surface realization, or model-specific regularities can obscure likelihood changes associated with meaningful contextual evidence. We introduce semantic reweighting of contextual token likelihoods, which learns non-negative weights over token-level negative log-likelihoods from contextual representations, allowing the resulting score to emphasize more informative parts of generations. Across nine model–dataset settings, our method reduces average disagreement with document relevance annotations from 15.69% for standard perplexity to 3.55%. Moreover, weighting functions transferred across models and datasets retain 91.72% of the improvement achieved by weighting, suggesting that the informativeness of weighting exhibits transferable structure. These results show that a pre-trained contextual reweighting model can make perplexity more sensitive to semantic evidence while retaining its computational efficiency, with strong cross-domain generalization and simple plug-and-play deployment.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
perplexity, document ranking, transferability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bY4YOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how to improve perplexity-based evaluation of document with reference answers. The authors observe that standard perplexity score can favor an irrelevant document over a relevant one. It proposes a simple method that learns context-dependent token weights from labeled document pairs and uses them to reweight token-level negative log-likelihoods. The authors evaluate its effectiveness and cross-model and cross-dataset transferability across three QA datasets and multiple LLMs.

Critical strengths

[S1] The paper reports an interesting finding: standard perplexity can favor an irrelevant document over a relevant one.

[S2] The method is simple and easy to understand: it learns token weights from labeled document pairs and uses them to reweight token-level negative log-likelihoods for document scoring.

Critical weaknesses

[W1] Since the method requires labeled data for training, please compare it with supervised baselines trained on the same data. Without such comparisons, it is difficult to indicate the contribution of token reweighting.

[W2] It remains unclear whether the method can be transferred to closed-source models, as their APIs may not provide the token log-probabilities required to compute perplexity.

Questions for authors

[Q1] The evaluation relies on open-sourced LLMs, such as Gemma-2-9B and Llama-3.1-8B. Please include closed-sourced APIs.

[Q2] The evaluation only consider simple question answering. It remains unclear whether these findings generalize to code generation, mathematical reasoning, and agent tasks.

[Q3] The context budgets differ across the proposed method (18,000 tokens) and baselines (2,048-token). Please use same budgets to ensure a fair comparison.

[Q4] The evaluation measures document-pair ranking given a gold answer. It remains unclear how the method could be applied when the answer is unknown.

[Q5] It remains unclear whether improved ranking translates into higher answer accuracy on downstream tasks such as NQ and HotpotQA.

3 comments
AuthorsOpenReview ↗

Supervised Baselines, Matched Context Budgets, and Clarifications of Closed-Source Transfer and Downstream Applicability

We thank the reviewer for the thoughtful summary and encouraging comments, and are glad that they found the finding interesting and the method is easy to understand.

W1 Since the method…of token reweighting.

We thank the reviewer for this suggestion. We added token2score, a trained discriminative ranking head using Qwen3-8B backbone, NQ training splits, and LoRA configuration as our method in Appendix A.4. Token2score achieves lower disagreement than weighted PPL on 2 datasets: 0.97% vs. 2.34% on NQ, and 1.13% vs. 2.02% on MS MARCO, higher on HotpotQA: 8.40% vs. 7.27%. See VJ1y W1/Q2 for the complete baseline and ablation results, and further discussion.

We appreciate all reviewers' suggestions to build a more comprehensive baseline. However, as emphasized in Section 4.4 and the conclusion, our central objective is to improve likelihood-based assessment through transferable, context-sensitive token weighting, rather than to outperform dedicated document ranking models or discriminative ranking head. None likelihood-based methods can’t be applied to different models, nor other uncertainty estimation tasks.

W2/Q1 It remains …compute perplexity.

We thank the reviewer for highlighting this limitation. The biggest challenge is that these closed-sourced APIs like gpt-4o-mini don’t provide last hidden layer embedding but only top N result log probability. If a required gold answer token is absent from the returned probabilities, its contribution to the score cannot be computed exactly. We investigated gpt-4o-mini’s Chat Completions API, which exposes output token log probabilities and at most 20 alternatives per position. We send 10 query, documents and gold answer pairs per each dataset, only 2 of 30 pairs (1 NQ and 1 HotpotQA) were successfully evaluated. We therefore do not claim empirical validation of closed-source model transfer. We clarified that its applicability depends on whether the target API exposes the conditional token probabilities required by the scoring protocol in Appendix B.2.

Q2 We thank the reviewer for this suggestion.We have demonstrated an experiment in Section 4.5 shows that WeightedPPL better distinguishes correct from incorrect answers among multiple Monte Carlo samples, achieving a higher mean within-question AUROC than BasePPL (see our response to Reviewer EsPH, W1/Q1 for more detail). Although this extends our evaluation to generate answer correctness, it remains within question answering. Evaluating generalization to code generation, mathematical reasoning, and agent tasks is beyond the scope of the present study, but we consider it an important direction for future work.

Q3 We thank the reviewer for highlighting this potential confound. We reevaluated Base PPL and Weighted PPL using a shared 2,048 token document budget, matching the document cap used by SAR and CSL. Both scorers receive identical truncated documents and are evaluated on the same saved held out pairs, using the existing Qwen3-8B checkpoints without retraining.

The updated disagreement rates for WeightedPPL are 2.31%, 7.27%, 2.01% for NQ, HotpotQA and MSMARCO respectively, the difference is small compared to what we report in paper, 2.34%, 7.27%, 2.02%.

Q4 We thank the reviewer for raising this point. The experiment described in our response to Reviewer VJ1y, W3/Q3 uses WeightedPPL scores computed from gold answers to supervise a student document reranker through a recent self distillation idea. The student learns to predict these scores using only the query and document, enabling document reranking without gold answers at inference time.

Besides, in a more capable model, the knowledge needed to answer a query may already be encoded in their parameters. This motivates applying WeightedPPL directly to their generated answers, potentially improving uncertainty estimation without requiring gold answers at inference. We consider this as a promising direction.

Q5 We thank the reviewer for raising this point. Notice that our method could only be used as a more accurate uncertainty proxy to evaluate generated content, but not directly improve any generation accuracy itself.

We added a generated answer evaluation experiment in Section 4.5 using 1000 heldout NQ questions that Qwen3-8B answered wrong during an initial closed-book screening.We sample 10 closed-book answers at temperature 1.0 and score these answers using BasePPL and our WeightedPPL, conditioning both scorers on the same question, positive document, and gold reference answer. An external gpt-5.4-mini judge labels each answer’s correctness. We compute AUROC on 10 sampled answers within each question and average equally over all questions.

DatasetBaseWeighted
NQ0.67720.6864
HotpotQA0.82290.8268
MSMARCO0.54390.5541

The results show improvement without additional training in all 3 datasets, suggesting that the benefits of learned token weighting extend from document ranking can translate to assessment of generated-answer correctness.

Thank you for your rebuttal, it has addressed the bulk of my concerns.

Dear Reviewer bY4Y,

Thank you for your follow up and for taking the time to consider our rebuttal. We are glad that our response has addressed most of your concerns. We appreciate your constructive feedback and would be happy to provide any further information while time remains.

Thank you again for your time and consideration.

Reviewer_EsPHOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a learned Perplexity (PPL) scoring that aims to solve current PPL's uniform aggregation token likelihoods despite large differences in their predictability and semantic importance. The authors trained a LoRa finetuned LLM head to produce adapted current token last layer embedding and project it through one layer network to get current token's weighted. Experiment result shows it's advantage in distinguishing positive documents that support current query and negative documents as distractor.

Critical strengths
  1. This paper proposed an interesting topic exploring learnable PPL scoring.
  2. This paper suggested that neither prior-based weighting and sematic-based weighting can provide better PPL signal.
  3. The experiments shows large and consistent gain across three model families
Critical weaknesses
  1. Mismatch between the claimed scope and the evaluation

The title ("Rethinking Perplexity"), abstract ("assessing how strongly a language model supports a target generation"), and introduction (uncertainty estimation, generation evaluation) present the contribution as a better general-purpose perplexity. However, all training and evaluation hold the answer fixed and only rank two context documents. This tests are first proposed by SePer to eval retrieval quality and test only the score's sensitivity to the conditioning context in this paper. The paper never evaluates whether the new score better reflects the quality of model outputs.

  1. Unfair experiments and baseline selection

2.1 This paper compared trained weighted PPL with other training-free methods (only CSL need calibration). It would be better to compare with other parameter matched scoring algorithm.

2.2 The selection of two baselines need to justified: 1. SePer proposed a metric for measuring info retrieval quality by LLM's output using that retrieval. This matched with the evaluation protocol that to distinguish or rank two context document, but not relate to the paper's scope on perplexity based scoring. 2. Semantic entropy measures uncertainty over response, a typical usage of PPL, good fit for the scope but not match with the evaluation setting. The root cause of this mismatch should come from the previous weakness 1.

Questions for authors
  1. Could the authors please designed another experiment to evaluate the proposed weighted PPL other than ranking support document?

  2. Could the authors add one more parameter-matched baseline that also requires training?

  3. The method appears under many names across the paper: "semantic reweighting of contextual token likelihoods," "context-sensitive token weighting," "learned weighted PPL," "Token Weighted PPL," and "native weighting PPL." In addition, "weighted PPL" also describes several baselines. A single, distinctive name would avoid confusion.

  4. Section 3 should mainly illustrate the proposed new weighting algorithm but section 3.2 is distracted to discuss eval result of other baselines. And the content described in section 3.2 doesn't match the result of table 1 as prior based weighting and sematic based weighting didn't show systematically difference. However, the paper claimed "Table 1 shows that semantic weighting reduces disagreement more substantially than propensity-based correction. Could the authors further justify this?

1 comment
AuthorsOpenReview ↗

Generated Answer Evaluation, Trained Scoring Baseline, and Clarifications of Scope, Baseline Selection, and Presentation

We thank the reviewer for taking the time to review our work, and we are glad they found the learnable PPL scoring is interesting, and experiments shows large and consistent gain across 3 model families

W1/Q1 Mismatch between…model outputs.

We thank the reviewer for this helpful comment. We added a generated answer evaluation experiment in Sec. 4.5 using 1000 heldout questions that Qwen3-8B answered wrong during an initial closed-book screening. For each question, we sample 10 closed-book answers at temperature 1.0. We then score these answers using BasePPL and our trained WeightedPPL, conditioning both scorers on the same question, positive document, and gold reference answer. An external gpt-5.4-mini judge labels each answer’s correctness. We compute AUROC on 10 sampled answers within each question and average equally over all questions.

DatasetBaseWeighted
NQ0.67720.6864
HotpotQA0.82290.8268
MSMARCO0.54390.5541

The results show improvement without additional training. In this experiment, the conditioning information is fixed while the generated answers vary. The improvement therefore provides evidence that learned token weighting from semantic reweighting model with gold reference answer context helps distinguish correct model outputs, extending the evaluation beyond supporting document ranking to generation correctness assessment.

W2.1/Q2 This paper… scoring algorithm.

We thank the reviewer for this suggestion. We added token2score, a trained discriminative ranking head using Qwen3-8B backbone, NQ training splits, and LoRA configuration as our method in Appendix A.4. Token2score achieves lower disagreement than weighted PPL on 2 datasets: 0.97% vs. 2.34% on NQ, and 1.13% vs. 2.02% on MS MARCO, higher on HotpotQA: 8.40% vs. 7.27%.

Notice that as emphasized in Sec. 4.4 and the conclusion, our central objective is to improve likelihood-based assessment through transferable, context-sensitive token weighting, rather than to outperform dedicated document ranking models with parameter matched condition. None likelihood-based methods can’t be applied to different models, nor other uncertainty estimation tasks. Please see our response to VJ1y W1/Q2 for the complete baseline and ablation results and further discussion.

W2.2 The selection…weakness 1.

We thank the reviewer for asking for the clarification of our baseline selection: As discussed in Sec. 2.2, SePer and semantic entropy provide different ways of incorporating semantic information into uncertainty evaluation. SePer directly evaluates retrieval utility through changes in perplexity. Semantic entropy is included to examine whether uncertainty over response meanings is informative about contextual support.

Our approach learns context sensitive token weights from final layer contextual representations while retaining the original model’s fixed token NLLs. Our central objective is to investigate how semantic information in these representations can guide token likelihood aggregation. This motivates comparison with SePer and semantic entropy, which incorporate semantic information through response level probability aggregation. We agree these methods estimate different quantities and clarified their distinct roles and the scope of the comparison in Sec. 4.4.

Overall baseline selection is fair within likelihood-based methods scope to serve our central objective.

Q3 We appreciate the reviewers suggestions and have aligned these terms. We aligned to: our method “semantic reweighting of contextual token likelihoods” pre-train a ”semantic reweighting model”; weighted perplexity (Weighted PPL) for the broader family of scoring methods; and native for distinguishing from transfer settings.

Q4 We thank the reviewer for asking to clarify the role of Sec. 3.2 and the interpretation of Table 1. Sec. 3.2 introduces the weighted likelihood aggregation in Eq. (2) and examines the limitations of predefined weighting criteria, providing the motivation for learning context-sensitive weights in Sec. 3.3.

Regarding to Table 1, the three bias correction schemes: IDF, inverse log frequency, and model prior weighted algorithms, improve over BasePPL in only 2 of 9 algorithm–dataset combinations; Semantic entropy and SePer, which incorporate semantic information without directly reweighting token NLLs, improve in 2 of 6, while the semantic token weighting algorithms, SAR and CSL, improve in 4 of 6. These results show that semantic token weighting improves over BasePPL in more of the evaluated combinations, although its gains and its performance relative to other algorithms remain dataset dependent. This result motivates investigating if learning context sensitive token weights can overcome the limitations of predefined criteria. Our algorithm achieves lower disagreement than all these baselines across the three datasets, providing empirical support for this approach. We have clarified this reasoning in Sec. 4.4 to better support our original claim.

Reviewer_VJ1yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets a known PPL weakness: equal weighting of answer-token likelihoods ignores large differences in predictability and semantic salience, so PPL rankings often misalign with human relevance judgments. It learns non-negative token weights with a LoRA adapter and linear head on the same frozen LLM, using hidden states at answer positions to reweight token NLLs. Training uses a supporting-document vs hard-negative ranking loss plus entropy regularization, while NLLs remain from the original model.

Across 3 models × 3 QA datasets, disagreement with relevance labels drops from 15.69% to 3.55%, beating all seven likelihood baselines (including IDF, SAR, and CSL) and approaching an LLM judge. The weighting function transfers across datasets, models, or both in 72 directional tests, keeping 91.72% of native gains on average. Analyses of frequency-weight relations, contribution shifts, and semantic probes support interpretability. Evaluation uses 10000 held-out document pairs per dataset, character-overlap projection for transfer, and appendix details on data, hyperparameters, and protocol.

Critical strengths
  1. The paper identifies a real gap: existing weighting methods (SAR, CSL) weight tokens by generic semantic importance and never learn directly which likelihoods distinguish supporting from irrelevant contexts. The method is simple, requires no sampling, and improves consistently across 9 settings (15.69% → 3.55%).

  2. The transfer evaluation is solid in design: it covers dataset transfer, model transfer, and joint transfer across 72 directional combinations, and it handles cross-tokenizer weight alignment through character-overlap projection. Model transfer retains 95.38% of the native improvement. The weights are learned in one model's representation space, yet they barely degrade when moved to a model with a completely different tokenizer and internal representations, which suggests that what is learned has some generality.

  3. The case-study figures (Fig. 5–6) are clear and intuitive. The appendix gives reproducibility details (data splits, the mining pipeline, verbatim prompts) that support verification and follow-up work.

Critical weaknesses
  1. A standard document-ranking control is missing. The task is essentially document relevance ranking, yet the paper never compares with established published methods for that task. Document ranking has mature cross-encoder rerankers such as ColBERTv2 and BGE-Reranker. The paper compares with none of them, and the released code implements none. Two basic ablations are also absent. One is a discriminative ranking head trained on the same hidden states. The other scores with the weights alone and drops the NLL term. Together these controls would answer one question the work leaves open. What is gained by casting the scorer as weighted PPL relative to a plain discriminative ranker?

  2. Hard-negative construction contaminates both training labels and evaluation gold labels. Appendix B.1 states that answer-string exclusion is skipped during negative mining. A “negative” only means outside the human positive set, with no check that it truly fails to support the answer. The pipeline keeps the highest retrieval-similarity documents, which are exactly the most likely to contain the answer. The paper reports no rate of answer-containing negatives, no cleaning, and no robustness check. This same label set serves as the training signal and the sole evaluation metric. For the headline result 15.69% to 3.55%, how much reflects real semantic judgment and how much fits quirks of the labeling pipeline remains unknown.

  3. No downstream task is validated, so all evidence rests on one proxy metric. The three applications named in the introduction, uncertainty estimation, generation evaluation, and retrieval-augmented generation, remain undelivered. There is no end-to-end RAG experiment, no retrieval reranking, and no calibration analysis. The jump in §4.1 from low disagreement to good retrieval performance has no support.

  4. Scoring needs a gold answer, which limits deployment. In real retrieval settings the answer is the unknown target, so the scorer cannot run. Practical use reduces to offline evaluation and to a training signal for retrievers. This also conflicts with the “plug-and-play” claim in the abstract.

Questions for authors
  1. What fraction of the mined hard negatives actually contain or support the gold answer? Appendix B.1 states that no answer-string exclusion is applied, so a small human audit(e.g., 100 negatives per dataset) would directly bear on the reliability of the headline disagreement numbers. If the fraction is non-negligible, would answer-string exclusion or manual filtering change the results?

  2. Can you compare with a broader set of baselines, in particular established document-ranking methods such as cross-encoder rerankers (e.g., ColBERTv2, BGE-Reranker)? Relatedly, is the perplexity-based formulation itself necessary: how does it compare to a discriminative ranking head trained on the same hidden states, and to a scorer that uses the learned weights alone without the NLL term?

  3. How does the scorer perform in a downstream task? For example, does reranking retrieved documents with weighted PPL improve end-to-end QA accuracy, or does it improve calibration in an uncertainty-estimation setting? This would connect the disagreement metric to practical utility.

2 comments
AuthorsOpenReview ↗

Additional Baselines and Ablations, Hard-Negative Robustness, and Downstream Retrieval Tasks

We thank the reviewer for the thoughtful summary and encouraging comments, and are glad they found the paper identifies a real gap and solve it with a simple but effective method with high transferability and clear and intuitive case study.

W1/Q2: A standard…discriminative ranker?

We thank the reviewer for suggesting these controls. We added ColBERTv2 and BGEReranker-v2-m3, alongside two ablations: (1) a direct discriminative ranking head using Qwen3-8B backbone, NQ training splits, and LoRA settings; and (2) a weight only scorer averages our model’s unnormalized weights without NLL. Notice that as emphasized in Sec. 4.4 and the conclusion, our central objective is to improve likelihood-based assessment through transferable, context-sensitive token weighting, rather than to outperform dedicated document ranking models or discriminative ranking head. Comparison with none likelihood-based document ranking methods is not our central objective as they can't be applied to other uncertainty estimation tasks. Weight only ablation shows NLL term contributes to gain. We added all results into the Appendix A.3.

%↓ColBERTbge-rerankertoken2weight
NQ3.052.182.34
HotpotQA17.54.167.27
MSMARCO1.491.212.02
%↓token2scoreweightsumtoken2weight
NQ0.976.842.34
HotpotQA8.412.47.27
MSMARCO1.1311.112.02

W2/Q1:Hard-negative…remains unknown.

We thank the reviewer for raising this concern. We agree demonstrating the impact of string-inclusive and the human annotation will make our result stronger. We first investigated inclusive rate in 30k hard negative documents using three rules: literal, case-sensitive substring matching; normalized, substring matching after NFKC Unicode normalization, case folding, and whitespace normalization; and boundary, normalized matching with word-boundary constraints:

Dataset%LiteralNormBoun
NQ2.062.161.71
HotpotQA4.004.102.09
MSMARCO0.280.590.41

We then reevaluated the existing results on HotpotQA, which has the highest inclusion rate under every rule. For each rule, we removed all pairs associated with a query if any of its negative documents matched the gold answer. “None” applies no answer-string filtering but retains the original exclusion of pairs with no answer tokens left to score after token skipping. Tokenizer differences lead to slightly different unfiltered counts across models.The table below reports disagreement rates for BasePPL and our WeightedPPL. WeightedPPL retains substantial improvements over BasePPL under every rule than no filter (None) result. Thus, we conclude the improvement persists, even better after removing inclusion samples.

ModelRuleNBaseWeightedRel.drop
Gemma-2None995624.727.5469.48
Literal955823.816.2973.59
Norm954823.796.2573.71
Boun974924.186.9571.23
Llama-3.1None996314.465.3163.29
Literal956513.674.5966.44
Norm955513.684.5866.49
Boun975613.965.0363.95
Qwen3None996321.807.2766.67
Literal956520.105.4472.96
Norm955520.075.4173.04
Boun975621.186.8067.91

We further did a human audit of 240 hard negative documents by 6 reviewers in 3 datasets, found 227 (94.6%) unsupported, 5 supported, 6 partially supported, and 2 uncertain, providing evidence that most negative labels are aligned with human audit labels. All results are added to Appendix B.2

W3/Q3:No downstream…no support .

We agree that downstream evaluation is important. We added experiment for RAG and retrieval reranking in Appendix A.5, inspired by a recent self distillation idea (Zhao et al., 2026), we use our WeightedPPL as teacher to supervise a document reranker training: the teacher scores a gold answer conditioned on the query and document, while the student learns to predict these scores without gold answer. On NQ, WeightedPPL supervision improves student NDCG@1/@3/@5 from 0.450/0.564/0.597 under BasePPL supervision to 0.507/0.624/0.661, respectively. NDCG in the teacher score has a similar performance increase from 0.514/0.677/0.720 to 0.607/0.756/0.790. These results provide initial evidence that WeightedPPL offers a more useful training signal for document reranking, without requiring gold answers at inference.

We find it valuable to investigate two additional downstream tasks. First, weighted log probability signals could be applied in RL and on-policy distillation, measuring downstream task accuracy at matched training-token budgets. Second for uncertainty estimation and generation evaluation, we added an experiment using our weighted PPL to measure Qwen3-8B model closed-book answer correctness. See our response to Reviewer EsPH, W1/Q1 and Sec. 4.5 for detail. These evaluations could assess whether improved scoring translates into more effective training and more accurate answer selection.

W4: Scoring needs…abstract.

See downstream task train student model in W3/Q3, it does not require answer at inference time.

Dear Reviewer VJ1y,

Thank you again for your constructive feedback. As the discussion period is nearing its end, we wanted to check whether there is anything else you would like us to address or clarify. We would be happy to provide any further information while time remains.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.