- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AlignRadar predicts a dataset’s emergent-misalignment risk before fine-tuning. It learns from internal changes measured in historical fine-tuning runs, predicts a risk-associated change from dataset and base-model features, and combines this prediction with a base-model audit. Evaluation uses nested leave-one-family-out testing and includes a separate cross-domain outcome analysis.
Critical strengths
-
Forecasting risk before running fine-tuning is a meaningful problem. And using historical internal changes as training-only supervision is an interesting approach.
-
The evaluation is carefully designed. Nested family-held-out testing, fold-safe target selection, and permutation controls strengthen the evidence for the proposed supervision.
Critical weaknesses
-
The outcome labels need stronger validation. They depend on a small prompt suite, model judges, and a fixed deterioration threshold. Reversing answer order controls position bias, but not whether the judges correctly identify harmful changes. The secondary model review does not provide an independent estimate of label accuracy.
-
The evidence is still narrow: one base model, one fine-tuning recipe, and small, mostly constructed corpora. Family-held-out testing is useful, but it does not show whether the learned signal survives changes in model, training strength, or realistic dataset scale.
Questions for authors
Are any results available on another base model or fine-tuning recipe?
2 comments
Follow-up before the discussion closes
Dear Reviewer h1Y4,
Thank you again for your positive assessment. As the discussion period closes on November 18 (AoE), we wanted to check whether our response answered your questions. In brief:
- Labels: blinded Llama-3.3-70B and Gemma-3-27B judges reach 93% and 84% precision against blinded human annotators, and their scores rank the human labels at 0.92 AUROC with no threshold.
- Generalization: the cross-domain gain over the audit replicates on Llama-3.1-8B-Instruct (+16.1 AUROC) and at a 5× learning rate (+21.3).
We would be glad to answer any remaining questions. If our response addresses your comments, we would be grateful if you would consider raising your score.
Response to Reviewer h1Y4
We thank the reviewer for the positive assessment and insightful comments. Our new experiments confirm that (1) our outcome labels achieve 93% precision against blinded human annotators and remain robust without a fixed threshold (0.92 AUROC), and (2) AlignRadar’s gains replicate across a second base model and stronger fine-tuning recipes (5× and 10× learning rates).
1. Independent Label Validation, Thresholds & Prompt Suite (W1)
Upgrading to independent, blinded pointwise Llama-3.3-70B and Gemma-3-27B judges [1] establishes 93% and 84% precision against blinded human annotators, increases AlignRadar’s cross-domain gain to +30.6 AUROC, and achieves 0.92 AUROC against human labels without any binarization threshold.
Scoring each response independently eliminates pairwise comparison bias altogether and removes the variance floor of the original small judges, confirming that our labels accurately capture genuine harmful shifts.
2. Generalization Across Base Models, Recipes & Scale (W2, Question)
Yes. Re-running the complete 76-corpus pipeline confirms that AlignRadar’s cross-domain advantage replicates on Llama-3.1-8B-Instruct and under 5× and 10× learning rates.
(Note: AlignRadar also outperforms a supervised direct-EM classifier + audit baseline at 1× LR [93.3 vs. 75.2] and 5× LR [90.6 vs. 77.0], and matches its performance ceiling on Llama-3.1-8B [95.8 vs. 96.8].)
The learned internal-shift signal is robust across model families and training intensities: even when raw shifts begin to saturate under an extreme 10× LR for 5 epochs, zero-shot transfer of the standard-recipe AlignRadar forecaster still ranks outcomes best. Finally, our 100-150 interaction corpora directly target the practical few-shot fine-tuning vulnerability regime [1, 2], where safety degrades sharply while leaving minimal surface traces for static audits.
We hope these additional validations address your questions about label reliability and generalization. We will revise the manuscript to explain the threshold analysis more clearly and explicitly state the scope and limitations of our evaluation across models, fine-tuning recipes, and corpus sizes.
References [1] Betley et al. Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs. ICML 2025. [2] Qi et al. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! ICLR 2024.