acceptodds
Under review as a conference paper at ICLR 2027

OPENING LLM JUDGES: RECOVERING PREFERENCE SIGNALS BEYOND THE FINAL VERDICT

Abstract

LLM judges are widely used to evaluate model outputs, but their verdicts can be unreliable: a judge may favor the worse answer because of its position, length, or other surface features. When a judge gives the wrong verdict, is the information needed to make the right judgment absent from the model, or is it still present in its internal representations but not reflected in the final output? We study this question across 64 open-weight evaluators and 14 datasets, including causal interventions on 41 judges (editing a model's internal activations while it runs to see whether its verdict changes). On LLMBar, a benchmark built so that the superficially better answer is the worse one, the verdicts of 50 judges agree with human labels only 0.456 of the time, even after position bias is cancelled by scoring both answer orders. Yet a small probe trained on the same judges' internal activations, without changing the judges, reaches 0.846, and still 0.686 after the influence of surface features such as length and position is removed (0.507 with shuffled labels). This gap holds across eight benchmarks and across model families, but it is not universal. A simple score of how well surface features alone predict the human label, computed before any probe is trained, is strongly correlated with the size of the gain (Spearman rho = 0.90). On two further tasks where a judge scores one answer at a time against a rubric, leaving no surface cue to exploit, reading the internals gives no advantage. The interventions also show that editing activations in the middle of the network already changes the verdict, before it can be read off directly, and locate the pathways that carry position and length bias. With the same human labels, the recovered signal also lets a judge flag cases where it is likely to be wrong and yields better labels for preference learning. A wrong verdict, then, does not mean the judge lacks the information, and a simple diagnostic shows when it is worth recovering.

Primary area:
interpretability and explainable AI
Keywords:
LLM-as-a-judge, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6DSEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks whether an LLM judge that gives a wrong pairwise verdict still holds the information needed for the correct judgment in its internal activations. Across multiple open-weight judges and different preference benchmarks, the authors train small linear probes on the judges’ frozen activations, and compare them to the judges’ emitted verdicts. Probes recover human preferences that the verdict misses in the adversarial scenario, and the paper uses cf, a length-only classifier’s accuracy, to predict when this helps.

Critical strengths

S1. The experiments span many judges, model families and benchmarks, with appropriate controls and replication, so the main observation is unlikely to be an artifact of a few models.

S2. The paper distinguishes what its evidence supports from what it does not, and reports negative results openly.

Critical weaknesses

W1. The paper recommends reading internals when cf ≳ 0.75. But cf is itself the accuracy of a length-only classifier trained on the same labels, and on the two most confounded benchmarks it matches or beats the probe:

The lens probe used as the deployed reader in §7 loses on both (Table 2a).

The stronger residual-stream probe also loses on RB2, and beats the length classifier on LLMBar by only 0.046 (0.846).

So in exactly the setting the method targets, the simple classifier computed while obtaining cf is already a comparable or better reader. This undermines the practical case for reading internals.

W2. Out of distribution, the probe largely fails (Appendix A.7). The leave-one-benchmark-out probe beats the verdict on only 1 of 8 benchmarks. The read-out therefore needs labels from each new evaluation distribution. This limits its value as a general method and weakens the claim that the judge holds a general preference signal.

W3. The paper is quite hard to follow. The core argument and the presentation are unclear. Implementation and statistical details crowd the main text, while the concepts needed to read the results are deferred to the appendix, such as the probe’s input features and the column definitions of Table 2 ((\Delta r), (\delta_o)). Figure captions are very brief, and Figure 2 in particular is difficult to interpret based on the main text alone. Many implementation details should be moved to the appendix, leaving more space to explain the experimental setup, key takeaways, and how to interpret each figure.

Questions for authors

See weaknesses. I would be happy to improve my score if my questions could be reasonably resolved.

4 comments

Thank you for constructive review. Your W1 led us to an issue in our data, and we are grateful for it.

W1. The length-only classifier vs. the probe.

First, the facts. On LLMBar, the length classifier (0.799) beats the lens probe (0.758) and loses to the residual-stream probe (0.846; better for 48 of 50 judges). The revision shows the length classifier as a row in the main table, so readers can see this directly.

Second, RB2. While checking your RB2 numbers, we found a loading error on our side. RB2 stores one correct and three rejected answers per prompt; our loader joined the three rejected answers into one string, so the rejected side was longer in 98% of pairs (53% with proper pairs). RB2's cf of 0.96 was an artifact of this error, not a property of RB2. We removed RB2 (General Response, item a). So of the "two most confounded benchmarks", only LLMBar remains.

Third, why the length classifier is not a competing judge. It is fit to how a benchmark was built, not to answer quality.

  • It never reads the content, and its sign changes across benchmarks. On LLMBar-adversarial the better answer is shorter in 78% of pairs; on rb-easy it is longer in 79%, and on MT-Bench longer in 66%. On the four lowest-cf benchmarks it scores between 0.49 and 0.59.
  • Used as a reward or as a labeler for preference learning, it can be satisfied by changing length alone. An evaluator must avoid exactly this failure.
  • Our question is whether the judge holds information beyond surface cues. The length classifier cannot answer it; the length-residualized probe can. It reaches 0.686 against 0.507 for shuffled labels and 0.456 for the verdict. On LLMBar's GPTOut sub-set (47 items, so a small sample), where length is a weak cue (length-only 0.618), the probe reaches 0.661 and stays at 0.661 after length is removed, above the verdict (0.606) and the length classifier.

What we changed. We now read cf as you suggest. A high cf means the labels can be predicted without reading the answers, which is why the verdict fails there. The practical rule in the revision is: when cf is high, do not trust the verdict; use the internal read-out instead; report the length baseline next to it; and if the read-out will be used as a reward or labeler, use its length-residualized version. We removed any suggestion that the probe is the most accurate label predictor on confounded sets.

W2. Out-of-distribution transfer. We agree. The read-out is specific to a distribution: it needs a small labeled sample from the target distribution (256 labels in our budget experiments). Our appendix already lists cross-benchmark transfer as a claim we do not make. The revision now says this in the abstract and introduction, and removes wording that suggested "a general preference signal". The claim is narrower: for a given evaluation distribution, a judge's layers carry label information that its verdict loses.

Two points of context.

  1. Every label-using fix shares this need. Our LoRA baseline was also trained per benchmark.
  2. As Reviewer 6aHJ notes, the benchmark labels measure different things (instruction following, correctness, safety, preference). A single probe direction that transferred across all of them would be surprising.

What does transfer is across judges. A read-out fitted on 49 judges and applied to an unseen judge, with none of that judge's labels, improves on its verdict on LLMBar by +0.077 [0.033, 0.118], for 78% of judges.

W3. Presentation. We agree and made these changes:

  • the probe's input features, Δr\Delta_r and δo\delta_o are now defined in the main text, before the table that uses them;
  • every caption now says how to read the panel, and the Figure 2 caption explains each axis and marker;
  • implementation and statistical details moved to the appendix;
  • the mechanism section is shorter, which frees space for the explanations above.

Dear Reviewer,

With the discussion deadline approaching, we wanted to gently follow up. We addressed all of your points. We'd be grateful if you could take a look, and if your concerns are resolved, we kindly ask you to consider revising the score. We're glad to answer any remaining questions.

Reviewer_6DSEOpenReview ↗

Response by Reviewer 6DSE

Thanks for the detailed response and revision. The presentation is now much clearer. However, my main concern remains. On LLMBar-adversarial, the length-only classifier still outperforms the deployed lens probe and the length-controlled probe, and the gain over the recalibrated verdict is concentrated almost entirely on this benchmark. The practical case for reading internals therefore remains narrow, and I will keep my score.

Thank you, we are glad the presentation is clearer.

Three points on the remaining concern.

  • The residual-stream probe beats the length classifier on LLMBar (0.846 vs. 0.799, for 48 of 50 judges).
  • The length-controlled probe has, by construction, no access to length, so comparing it with a length classifier asks a different question: does content signal remain? It does (0.686 vs. 0.507 with shuffled labels).
  • The length classifier is accurate on LLMBar only because it learns how that benchmark was built. On rb-easy the better answer is longer in 79% of pairs, so LLMBar's "prefer shorter" rule would be right only about 21% of the time. Used as a reward or labeler, it can be satisfied by changing length alone. The length-controlled read-out does not invite this, since length is regressed out of its inputs.

We agree the practical case is conditional, and the paper says so. We believe a conditional result whose condition is measurable in advance and validated on held-out data is itself a contribution.

Reviewer_M8qEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the internal-output gap in LLM judges: the final verdict may be poorly aligned with human labels even when preference-relevant information is still recoverable from frozen internal activations. The authors support this with probing experiments, activation interventions, a length-based diagnostic for when internal read-out helps, and downstream tests on abstention and DPO labeling.

Critical strengths
  1. The paper is careful about claim strength. In particular, the L1/L2/L3 distinction helps separate decodability, causal influence, and actual causal use, which is important for avoiding over-interpretation of probing results.
  2. The cf diagnostic is interesting. It is a useful step toward making internal read-out conditional rather than universal. Even though its current form is limited to length confounding, the attempt to predict when internal activations are worth reading makes the method more practically grounded.
Critical weaknesses
  1. The basic idea of reading evaluator internals is not entirely new; prior work has already explored probes and layer-wise read-outs for LLM judges. The paper’s added value is mainly in scale, controls, and the internal-output-gap framing
  2. The proposed cf diagnostic is currently instantiated as a length-only confound score. While this is useful for the studied benchmarks, it is unclear whether the conclusion extends to other common sources of judge bias, such as style, verbosity, formatting, self-preference, or domain familiarity.
  3. The mechanistic intervention section is interesting, but its practical value for LLM-as-a-judge design is unclear. The identified causal pathways mainly support the paper’s explanatory narrative that internal states can influence the emitted verdict and carry surface biases. However, they do not translate into concrete design guidance for LLM-as-a-judge.
  4. Finally, the presentation is overly compressed. The main text introduces many concepts, metrics, and experimental variants in rapid succession, which makes the paper feel overloaded and sometimes obscures what each experiment establishes.
Questions for authors

See weakness above.

4 comments

Thank you for the review, and for seeing the value of the L1/L2/L3 distinction and of making the read-out conditional.

We agree that reading evaluator internals is not new. Maiya et al. (2025) and LAGER show it, and our appendix says we do not claim that observation. What we add is:

  1. We test the gain against a recalibrated verdict trained on the same labels, so calibration alone cannot explain it (+0.157 on LLMBar, 50 of 50 judges). We also test it under length controls and model-family clustering across 50 judges.
  2. A diagnostic, computed before any probe is fit, that predicts where the gain appears and where it does not. It holds on an independent seed and on held-out sub-sets (see our reply to Reviewer 6aHJ).
  3. The use of recovered labels downstream: DPO training labels and abstention. To our knowledge, prior work has not used the recovered signal as preference-learning labels.

Does cf extend beyond length? Partly, and we now say exactly how far.

  • Formatting and verbosity. We computed a nine-feature surface vector (length in tokens and characters, markdown marks, list items, code blocks, headers, bold, average sentence length, digits) on LLMBar, Arena and JudgeBench. A classifier on all nine features is within 0.03 of the length-only cf. Removing all nine from the probe inputs changes probe accuracy by at most 0.005 compared with removing length alone. On these benchmarks, formatting carries little label information beyond length.
  • Self-preference and domain familiarity. We did not test these, and you are right to ask. The score is defined as the accuracy of a classifier on features that do not depend on the judge; length is the instance we validated. Self-preference would use the identity of the model that wrote each answer. To avoid over-reach, the revision calls the score a length-confound score, and lists self-preference and domain familiarity as untested.

Practical value of the mechanism section. We agree it is mainly explanatory, and the revision states that it explains why the verdict fails. Two findings do give design guidance.

  1. Do not audit judges with attribution alone. Direct logit attribution agrees with activation patching at only ρ≈0.35\rho \approx 0.35 (median over the 41 judges in our causal suite; stable across three seeds). A bias audit of a judge should use interventions.
  2. Position bias is concentrated in a few heads. Ablating a few top-ranked attention heads (at most eight) cuts the A/B order gap by about half (seed medians 0.52 to 0.55), against 0.02 to 0.04 for random heads. These heads were ranked and tested on the same small set of items, so this is an upper bound, and we say so. Order-averaging remains the simple first fix; head ablation is a targeted option when running both orders is too costly.

We moved the remaining mechanism detail to the appendix and corrected several descriptions there (General Response, item e).

We alo agreem about presenttaion, the main text now introduces one comparison, one table and one diagnostic, and defines every term before it is used (General Response, item 2).

We hope the clearer framing and the added analyses address your concerns. If so, we would be grateful if you would consider raising your score.

Dear Reviewer,

With the discussion deadline approaching, we wanted to gently follow up. We addressed all of your points. We'd be grateful if you could take a look, and if your concerns are resolved, we kindly ask you to consider revising the score. We're glad to answer any remaining questions.

Thank you for the detailed clarification. The recalibrated-verdict baseline, the nine-feature analysis, and the honest disclosure of the data issues partially address my concerns. However, after carefully reading other reviewers' comments and the rebuttal, I choose to keep my score, for that the practical gain is concentrated almost entirely on LLMBar-adversarial, and the errors uncovered during the rebuttal reduce my confidence.

Thank you for acknowledging the new baseline, the nine-feature analysis and the disclosure.

We respectfully disagree that the concentration on LLMBar-adversarial narrows the contribution much. That is the regime where a deployed judge is most dangerous. Its verdict is below chance for 36 of 50 judges, and sampling or prompting does not help (0.451 and 0.456 at 256 labels). Its own confidence is also inversely related to correctness (abstention AUC 0.431), so the failure cannot be seen from the output. The internal reader raises that AUC to 0.822 for all 50 judges. The practical result is being able to tell, before fitting anything, whether one is in this regime. Claiming a gain everywhere would have been wrong, and our own null results show it. The corrections did not touch this result.

Reviewer_6aHJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks whether we can recover a better judgment from what is represented inside an LLM judge, even when its final answer is wrong. The authors train small classifiers on the model’s internal activations and compare their judgments with the model’s output. They also change some activations to investigate what affects the verdict and whether biases such as answer length and position can be reduced.

They find improvements in some settings, especially on datasets where surface features can mislead the judge, but the improvements are not consistent across datasets. My understanding is that the paper is trying to explain when looking inside the model helps and why, although I found it difficult to connect the different experiments into a clear answer.

Critical strengths

The main question is interesting and easy to understand. It is worth investigating whether a model contains information that could lead to a better judgment than the answer it gives. The improvements reported in some settings suggest that there is something useful to investigate here. However, I struggled to understand what these results establish more generally. The style of writing suggests that there was a lot of LLM use in the writing or execution that makes the paper hard to read and atypical compared to how papers are usually structured and presented.

Critical weaknesses
  1. I do not think the paper gives a clear enough answer to its main question.

The paper shows several cases where a classifier trained on internal activations performs better than the judge’s output. However, the explanations seem to be given on a case-by-case basis, and I can see alternative interpretations of these findings.

For example, the classifier might learn patterns that predict the dataset labels. This does not necessarily mean that the judge had reached the correct judgment internally but failed to express it. The activation experiments show that changing parts of the model can change its verdict, but I do not clearly see how this connects to the information used by the classifier to improve its predictions. The paper acknowledges this distinction, but I still find the overall interpretation unclear.

I also have concerns about the description of the labels. The paper broadly refers to human preferences, but the datasets measure different things. JudgeBench uses correctness-based labels, while LLMBar focuses on instruction-following. UltraFeedback, used in the downstream experiments, originally contains GPT-4-generated feedback. I would like the authors to explain exactly which labels they use and what agreement with those labels tells us.

  1. The results are difficult to follow because they are often reported without enough explanation.

There are many experiments and metrics, but I struggled to identify the main result and understand how the other results support it. The paper moves between judgment accuracy, correlations, changes in bias, and downstream training results. These may each be useful, but their relationship is not clear enough to me.

If the main claim is that looking at internal representations gives better judgments, I would like to see that improvement measured consistently against a strong output-based baseline, with the same label budget and uncertainty reported across datasets. The other experiments could then help explain when this improvement occurs and what its limitations are.

Figure 1 adds to this difficulty: it introduces several research questions rather than making the main contribution clear. Table 1 reads as a related-work comparison and would fit better in that section or in the appendix.

Overall, the paper feels like a collection of experimental results rather than a clear argument. I would need substantial effort to be put into interpreting the results, considering alternative explanations, and explaining the main takeaway before I would be comfortable recommending acceptance.

Questions for authors
  1. What evidence distinguishes recovering a judgment that the model already represents from training a classifier to use features that happen to predict the dataset labels? How do the activation experiments help resolve this distinction?

  2. Does the proposed surface-confound score predict improvements on genuinely new datasets, rather than only across different runs on the same datasets?

  3. Could you specify the dataset versions, subsets, and label sources used in each experiment? Which labels come directly from humans, which are based on correctness, and which were generated by AI? Were any examples relabelled for this study?

5 comments

Thank you for your review. You asked for a clear answer to the main question, a test of an alternative explanation, and a precise account of the labels. We answer each below. Your presentation points are addressed in the General Response.

The main takeaway, in one sentence. When a judge's verdict is pulled by surface cues, its intermediate layers still favor the labeled answer, and a small linear read-out of those layers recovers much of the lost accuracy; when there are no such cues, the read-out adds nothing over a recalibrated verdict, and cf tells the two cases apart.

Q1. Is the probe recovering the judge's judgment, or learning features that predict the labels? We agree this is the key question, and we agree that probing shows decodability (our "L1"), not that the judge "reached" a correct judgment. We removed wording that suggested more. Within L1, three results show that the probe is not just learning dataset patterns.

  1. The lens probe only sees the judge's own verdicts. Its input is the judge's A-vs-B logit margin read at each layer, using the judge's own unembedding and averaged over both answer orders: one number per layer. It cannot read the text. It can only re-weight what the judge's own layers say about A vs. B.
  2. An equal-label output baseline. The same logistic model fit on the final-layer margin alone, with the same labels and cross-validation, is a recalibrated verdict. On LLMBar it reaches 0.582. Adding the intermediate layers reaches 0.738: +0.157 [0.135, 0.177], positive for 50 of 50 judges. A label prior or a flipped verdict cannot explain this gain; it comes from intermediate layers that favor a different answer than the final layer does. This is the equal-label comparison you asked for, and the revision reports it with CIs on every benchmark (General Response, item 1).
  3. Surface controls. With length regressed out, the lens probe keeps 0.610 and the residual-stream probe 0.686, against 0.500 and 0.507 with shuffled labels, and each beats its shuffled control for 50 of 50 judges. In the 16-layer analysis, adding eight more surface features (formatting, lists, code, sentence length and others) to the regression changes the length-residualized result by only 0.005 (0.602 vs. 0.597).

How do the activation experiments connect? They do not explain what the probe reads, and the revision says so. They answer a different question: why the verdict fails. They show that position bias passes through a few attention heads that can be switched off, and that attribution scores are a poor guide to which components matter (agreement with patching: ρ≈0.35\rho \approx 0.35). The section is now shorter and scoped to this role.

Q2. Does cf predict gains on new datasets? We ran two checks that do not reuse the fitted relation.

  • Leave-one-benchmark-out. We fit the gain on cf using six benchmarks and predict the seventh. The mean absolute error is 0.033 accuracy points, and the predicted gains rank the benchmarks at ρ=0.86\rho = 0.86 against the observed ones.
  • Sub-sets never used in the fit. LLMBar-adversarial has four parts built in different ways. Their gain (probe minus verdict, medians over 50 judges) follows their cf in order: Neighbor (cf 0.84, gain +0.51), GPTInst (0.74, +0.30), Manual (0.72, +0.22), GPTOut (0.62, +0.05).

We agree that these do not replace new benchmarks. We present cf as a diagnostic validated on these sets, not as a general law, and we say so in the paper.

Q3. Dataset versions and label sources. No item was relabeled. Ties were dropped. A/B position was randomized, and every reader was scored on both orders.

BenchmarkSourceItemsWhat the label measuresWho set the label
LLMBar-adversarialRewardBench (filtered), llmbar-adver-*319instruction followingLLMBar authors, manually checked
rb-easyRewardBench (filtered), alpacaeval-easy + llmbar-natural200overall qualitystrong vs. weak model by construction; LLMBar authors
MT-Benchlmsys/mt_bench_human_judgments230pairwise preferenceexpert human judges
Chatbot Arenalmarena-ai/arena-human-preference-55k400pairwise preferencecrowd human votes
HH-RLHFAnthropic/hh-rlhf400helpful/harmless preferencecrowd human annotators
PKU-SafeRLHFPKU-Alignment/PKU-SafeRLHF400the safer responsethe dataset's safety annotation (human, AI-assisted)
JudgeBenchScalerLab/JudgeBench400correctnessground-truth answers

You are right that UltraFeedback was listed as the DPO pool. That was an error: it enters no reported result. The DPO pools were LLMBar-adversarial and RB2, and RB2 is now removed (General Response, item a).

What agreement means. On LLMBar and JudgeBench it means picking the answer that is correct or follows the instruction. On MT-Bench, Arena and HH-RLHF it means agreeing with human preference. The largest gap is on LLMBar, where the label is the least ambiguous. So the gain is not mainly about matching annotator taste.

On the writing. We used AI tools to polish text and draft code, as stated in our AI-use statement, and we take full responsibility for the paper. We agree the result was too dense. The revision rewrites the main text in plain language around the single comparison above.

We hope these answers and the revision address your concerns. If they do, we would be grateful if you would reconsider your score.

Dear Reviewer,

With the discussion deadline approaching, we wanted to gently follow up. We addressed all of your points. We'd be grateful if you could take a look, and if your concerns are resolved, we kindly ask you to consider revising the score. We're glad to answer any remaining questions.

Comments on the paper

4 comments

We thank all three reviewers for their careful reading. The reviews agree on three points: the paper is hard to follow, the main result needs a clearer baseline, and the role of the surface-confound score (cf) must be sharper. We revised the paper around these three points.

1. The main text now asks one question: does a judge's own internal state carry label information that its verdict misses? The main table now reports, per benchmark, with 95% CIs over 50 judges:

  • the verdict (raw, and averaged over both answer orders);
  • the recalibrated verdict: a logistic model fit on the judge's final-layer A-vs-B margin, with the same labels and the same cross-validation as the probe. This is the fair output-based comparison: same labels, and no weights changed;
  • the length-only classifier (cf);
  • the probe on the judge's per-layer margins, before and after length is regressed out;
  • a shuffled-label control.

The key numbers on LLMBar-adversarial (50 judges): the recalibrated verdict reaches 0.582; adding the intermediate layers reaches 0.738 (this analysis reads 16 evenly spaced layers, so it sits slightly below the all-layer probe's 0.758), a gain of +0.157 [0.135, 0.177], positive for 50 of 50 judges. On rb-easy and MT-Bench the same gain is small (+0.02). On Arena and PKU-Safe it is zero within the CI, and on JudgeBench and HH-RLHF it is slightly negative (-0.01). So the judge's intermediate layers add information where its verdict is pulled by surface cues, and not elsewhere. This is the paper's main result, and cf is the measure that tells the two cases apart.

2. Table 1 moved to the appendix (as Reviewer 6aHJ suggested). Figure 1 now shows only the main comparison. The probe's inputs, Δr\Delta_r and δo\delta_o are defined in the main text. Captions now say how to read each panel. Implementation and statistical details moved to the appendix. The mechanism section is shorter and says clearly what it explains (why the verdict fails) and what it does not (what the probe reads).

3. We now say "benchmark label", not "human label", and add a table giving the source and meaning of every label (see our reply to Reviewer 6aHJ).

4. While preparing this response we re-checked every number in the paper against the code. We found errors, and we list all of them in part 2/2. The main LLMBar result is not affected.

Following issues are also fixed, or clearly stated, in the revision.

(a) RB2 stores one correct answer and three rejected answers per prompt, as lists. Our loader turned each list into a single string. So every RB2 "pair" compared one answer with three rejected answers joined together. The rejected side was longer in 98% of pairs (53% with proper one-to-one pairs). This produced RB2's cf of 0.96. All RB2 numbers are invalid, and we removed RB2 from every analysis.

(b) The script for the two Indic rubric sets (Samiksha, Pariksha) used a data loader that silently fell back to placeholder items when the real file was not passed to it. Its "no advantage" result carries no information. We removed it from the abstract, introduction and Section 6.

(c) The 0.610 and 0.686 results regress out length only, not nine features, and position is handled by scoring both answer orders, not by regression. A nine-feature control (token and character length, markdown marks, list items, code blocks, headers, bold, average sentence length, digits) on the lens probe (16-layer version) gives 0.597 against 0.602 for length alone, so the conclusion is unchanged. The text now says exactly this.

(d) About half of the MT-Bench human judgments refer to the second turn, but judges were shown first-turn answers. This adds the same label noise for every reader, so comparisons between readers stay fair, but absolute accuracies are noisier. We state this in the paper.

(e) The "67% steering flip" number came from a rationale-injection test; steering itself flips 20% to 33% of verdicts. The length-subspace example (0.41 to -0.11) is one judge; the median reduction across judges is 0.09. The "tuned lens" is a logit lens with a per-layer sign fitted on labels. The causal-depth definition now matches the code.

(f) It enters no reported result. The DPO pools were LLMBar-adversarial and RB2 (now removed).

What still holds, on the seven correctly loaded benchmarks:

  • LLMBar-adversarial (319 items, 50 judges) is unchanged: order-averaged verdict 0.456, lens probe 0.758, residual-stream probe 0.846, and 0.686 with length removed (shuffled labels: 0.507; 50 of 50 judges above their shuffled control).
  • The probe beats the raw verdict on 7 of 7 benchmarks (mean 0.571 to 0.673; residual-stream probe 0.695).
  • cf still orders the gains: Spearman ρ=0.89\rho = 0.89, exact permutation p=0.006p = 0.006 over all 5,040 orderings. cf from seed 0 predicts the gains on the independent seed 1 at ρ=0.96\rho = 0.96.
  • At 256 labels on LLMBar (6 judges), the probe reaches 0.729, against 0.451 to 0.456 for the raw verdict, self-consistency and the calibrated prompt.
  • DPO on LLMBar (n=80n = 80): verdict labels give 0.463 [0.35, 0.58]; probe labels give 0.788 [0.69, 0.88]; gold labels give 0.825.

The full fine-tuning comparison used RB2 items in both training and test, so we moved it out of the main text.

Dear Reviewers,

Thank you again for taking the time to read our paper and write your reviews. Your feedback helped us see where we needed to explain our work more clearly, and the paper is better for it. If you’ve had a chance to read our replies, is there anything you’re still unsure about or would like us to address? If explained satisfactory, we would be grateful if you would reconsider your score.

We thank the reviewers for engaging with the revision. Two concerns are shared, and we respond to both briefly.

"The gain is concentrated on one benchmark." This is what the paper predicts, not a failure of it. Our claim is conditional: a judge's internals help where surface cues mislead its verdict, and a score computed before any probe is fitted says where that is. A uniform gain would contradict this account. The evidence for the condition does not rest on one benchmark. The score orders the gains across seven benchmarks (Spearman 0.89, exact p = 0.006), predicts an independent replication (0.96), and is also followed by four LLMBar subsets that were never used in the fit. The predicted nulls appear where the score is low.

"The errors reduce confidence." We found and disclosed every error ourselves. Each correction removed results; none reversed a remaining result, and none touched the LLMBar pipeline. Every number in the revision is recomputed from released artifacts with released scripts, the run logs confirm real data for every remaining result, and the replication run agrees at the cell level (r ≥ 0.96). We hope the corrections are judged by what they changed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.