acceptodds
Under review as a conference paper at ICLR 2027

Data-Efficient Criteria Steering for LLM Judges

Abstract

LLM judges are increasingly used to enforce safety and compliance policies, but adapting them to new languages, regulations, and risk taxonomies remains costly. Fine-tuning requires substantial labeled data and model updates, while prompt optimization relies on representative evaluation data and can overfit when data are scarce and the signal is limited, reducing held-out performance. We introduce criteria steering, a data-efficient, training-free method for specializing LLM judges from small labeled datasets without updating model weights. Our pipeline mines and ranks candidate criteria, constructs a contrastive activation direction from diverse criteria, and applies it only to the criterion-token span during prompt encoding, adding no decoding-time computation. On Bengali toxicity, Japanese content safety, and Indian financial-services safety and compliance (FinProof), criteria steering improves held-out performance for small general-purpose as well as purpose-built guardrail models. It outperforms text-only prompt optimization and, on FinProof, matches or exceeds a strong automated prompt-optimization reference depending on the operating point. Gains persist with relatively small labeled samples. These results position criteria steering as a practical adaptation mechanism for low-resource languages and specialized domains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-as-a-judge, activation steering, inference-time adaptation, prompt optimization, guardrail models, AI safety, low-resource adaptation, representation engineering

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mirgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes criteria steering for adapting small LLM judges without updating their weights. It generates judging criteria from labeled examples, ranks them on validation data, and constructs an activation direction from stronger and weaker criteria. This direction is applied only to criterion tokens during prompt processing. Experiments on Bengali toxicity, Japanese safety, and financial compliance report improvements across several models.

Critical strengths
  • The intervention is simple and practical. Steering only the criterion tokens leaves model weights unchanged and requires no additional intervention during decoding. This offers a useful way to adapt existing judges to specific policies.

  • The improvements cover several models and domains. The Bengali results are particularly encouraging: Granite Guardian F1 increases from 0.882 to 0.951, with improvements across all five tested models (Table 1). This provides evidence that the method can improve judgment accuracy beyond one model or dataset.

Critical weaknesses
  • The label budget and confidence grouping need clarification. In Table 2, N counts only the examples used to generate criteria, while Sections 3.3-3.5 describe using additional validation labels to rank criteria and select steering settings. Section 3.5 specifies 353 validation examples for FinProof, so the N=16 result does not demonstrate adaptation from 16 labels overall. In Sections 3.1 and 3.3, assigning yes/no answers scores of 1/0 makes every parsed answer satisfy the confidence threshold. The grouping therefore cannot distinguish confident from uncertain valid answers. With no parsing failures, AUROC computed from these scores also reduces to balanced accuracy.

  • The comparisons do not establish superiority over prompt optimization or improvement on external data. In Section 3.5, the main prompt baseline uses a composed criterion, but the optimization budgets are not clearly matched. In Table 3, steering increases FinProof F1 from 0.957 to 0.964 relative to GEPA while also increasing the false-positive rate from 5% to 7.5%. This is a trade-off, and the manuscript provides no uncertainty estimates to establish whether the small F1 advantage is stable. In Table 4, the external Japanese result is below the published specialist average. Without an unsteered result using the same prompt, this comparison cannot establish whether steering improves performance on the external benchmark.

  • MINOR: The discussion of model abliteration should be expanded. In Section 2, refusal steering is briefly discussed through Arditi et al., but the connection to model abliteration and subsequent safety studies remains underdeveloped. The discussion could include Agnihotri et al. [1], Abu Shairah et al. [2], and Jain et al. [3]. A short comparison could explain the shared use of activation interventions, distinguish removing refusal-related components from adding a vector at criterion tokens, and clarify what is new about constructing that vector from ranked judging criteria.

References

  1. Agnihotri, Shashank, Jonas Jakubassa, Priyam Dey, Sachin Goyal, Bernt Schiele, Venkatesh Babu Radhakrishnan, and Margret Keuper. "A granular study of safety pretraining under model abliteration." arXiv preprint arXiv:2510.02768 (2025).

  2. Shairah, Harethah Abu, Hasan Abed Al Kader Hammoud, Bernard Ghanem, and George Turkiyyah. "An embarrassingly simple defense against LLM ablation attacks." arXiv preprint arXiv:2505.19056 (2025).

  3. Jain, Samyak, Ekdeep S. Lubana, Kemal Oksuz, Tom Joy, Philip H. Torr, Amartya Sanyal, and Puneet K. Dokania. "What makes and breaks safety fine-tuning? A mechanistic study." Advances in Neural Information Processing Systems 37 (2024): 93406-93478.

Questions for authors
  1. What is the total labeled-data budget for each method, including criterion ranking and all validation-based selection? Are these budgets matched across comparisons?

  2. Were Bengali/Japanese confidence groups and AUROC computed from continuous probabilities or parsed yes/no outputs? If the latter, what distinguishes an uncertain answer from a parsing failure?

  3. How were the prompt baselines and GEPA configured, and does steering retain an advantage at the same false-positive rate?

  4. What is the external Japanese performance with the same minimal prompt but no steering? How was the reported variant selected from the two precomputed steering variants?

  5. Are there parallels between the methodology of this work and works on model abliteration? If yes, could they please be discussed, and if not, then an elaborate response on why not would be very helpful.

Reviewer_QoryOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies a practical issue in adapting LLM judges into low-resource / specialized regimes. Full fine-tuning requires substantial labeled data and task-specific checkpoints; prompt optimization avoids weight updates but is known to overfit the evaluator and weaken transfer under scarce supervision. This paper instead asks whether one can change how the judge internally represents the criterion rather than its weights or its wording.

To this end, this paper propose criteria steering, a span-localized variant of Contrastive Activation Addition (CAA). From a balanced N-example sample, the method mines candidate judging criteria (teacher LM generates them from the subset of examples the judge gets right-and-confident vs wrong-and-uncertain, split by label), ranks candidates by validation AUROC, selects the top-k as the strong pole and the bottom-k as the weak pole. For each selected criterion, the judge's own mean-pooled criterion-token activations give per-layer vectors and their normalized difference is the steering direction. At inference, the judge builds a fixed prompt with the minimal criterion, and the direction is added only to the criterion-token span during the initial forward pass at the validation-selected layer; autoregressive decoding then proceeds from the steered KV cache with no further intervention.

On Bengali toxicity (Table 1), criteria steering improves held-out AUROC for all five target judges, with gains ranging from +0.073 (Granite Guardian 4.1-8B, already strong) to +0.352 (Granite-4.1-8B base, non-guardian).

On cross-domain data efficiency (Table 2), using Granite Guardian 4.1-8B across all three domains, criteria steering remains effective at N = 16 and improves monotonically in N over 16 to 256.

On FinProof (Table 3), different k choices trade off operating points: k=1 beats GEPA on both precision and FPR (0.979 / 0.042 vs 0.975 / 0.050) while retaining F1 0.950; k=4 reaches F1 0.964 and recall 0.964 (vs GEPA 0.957 / 0.940) at a higher FPR 0.075; k=8 interpolates between the two.

On Japanese safety (Table 4), criteria steering wins on all three held-out judges.

Critical strengths

1 Adapting binary guardrails to new languages / regulatory regimes with scarce labels is well motivated. This paper also provides substantive overfitting evidence in prompt optimization.

2 The method is simple, cheap, and data-efficient: no weights are updated, no decoding-time overhead is added. Table 2 empirically supports the data-efficiency claim.

Critical weaknesses

1 The paper's methodological novelty is limited. Criteria steering are composed of CCA, a span localization to criterion tokens, and a criterion-mining front-end. The paper cites all three ingredients, and the genuinely new elements are (a) the pole examples are judging criteria rather than paired instances and (b) the direction is applied only to the criterion-token span. Both are reasonable engineering choices, but neither carries a conceptual argument beyond "we tried it and it works."

2 Dependence on a 70B teacher substantially weakens the "data-efficient adaptation of small judges" claim. Section 3.5 states: "Llama-3.3-70B-Instruct generates candidate criteria, while the target judge ranks them and supplies all steering activations." So the small judges (≤ 8B) are the inference targets, but the generation pipeline requires a 70B model. The paper is honest about this in the Limitations, but the contribution statements in Section 1 still read: "Data-efficient adaptation of small judges. Criteria steering delivers substantial gains from small labeled samples across target judges of 8B parameters or smaller, without weight updates."

Questions for authors

Please refer to the weakness section.

Reviewer_7DjwOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method that does criteria steering by constructing steering directions from the internal representations of strong and weak criteria. Instead of fine-tuning the model or optimizing the criterion's wording, the method modifies the model's internal activations when processing the original criterion. Experiments demonstrate effectiveness of the method, including in settings where data is not abundant.

Critical strengths
  1. Criteria play an important role in LLM-based evaluation, and optimizing these criteria is I think indeed an important research direction.

  2. The research questions on reliability, data efficiency, and generalization are very good and well established by the authors.

  3. The methodology is intuitive and makes sense at a high level. Overall, I do not see any critical logical flaws in the proposed approach.

  4. The experiments demonstrate the effectiveness of the proposed method.

Critical weaknesses
  1. Although data-efficient, the method requires repeating the entire adaptation process for each new criterion. This could become costly and time-consuming when dealing with a large number of criteria. Thus, there are some concerens regarding the cost.

  2. The experimetns only show on limited evaluation domains and model sizes. It remains unclear whether the method would be equally effective across broader domains and larger models. I think more experiments should have been done, which makes me hesitate to recommend accpetance.

  3. I have included several questions below. Depending on the authors' responses, some of these concerns may also become weaknesses of the paper. I am willing to change my decision based on the answers of the authors, because most questions are asking in the fundamental level.

  • Note: I would encourage the authors to provide more details in the appendix. Although including supplementary materials is optional, additional information would help readers better understand the paper and clarify questions that may arise while reading. In particular, details about the datasets, the original and generated criteria, prompt templates, criterion generation process, and implementation settings would be helpful for understanding and reproducing the proposed method. However, I am aware that including an appendix is optional, so I reviewed the paper without considering the presence or absence of an appendix, and this did not affect the decision of my rating.
Questions for authors
  1. Could the authors include a baseline where a stronger/frontier LLM like GPT-6 directly generates clearer and more effective criteria, without activation steering? I am wondering whther just simply providing a better version of a critieria that can be unciversally well followed is better since I thnk that is just a naive solution, and I think frontier LLMs can be capable of generating them

  2. Does criteria steering improve the judge's adherence to the original criterion, or does it implicitly change the criterion's meaning to better match the validation distribution? Since strong and weak criteria are selected based on AUROC rather than how closely they preserve the original criterion's meaning, I was wondering whether the steering direction might introduce additional requirements or change what the judge considers safe or unsafe. We in fact dont know if the strong and weak are really presercing the orignal criteria. Could the authors clarify this or show some experiments to validate this?

  3. Moreover, could there be cases where the judge correctly follows the original criterion but achieves lower task performance because its predictions differ from the ground-truth labels? I am not sure whether task performance alone can prove that the model follows the criteria better. Thus, How do the authors distinguish criterion adherence from task performance?

  4. Another question is that whether the errors are caused by difficulty following the given criterion, or by a lack of domain-specific knowledge.

  5. Could the authors report the AUROC scores or performance gaps between the selected top k and bottom k criteria? I wonder how large these differences are and whether the effectiveness of the steering direction depends on the performance gap between the two groups.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.