- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Authors investigate how to prevent EM that appear when one finetune an LLM on a narrow task. they compare KL regularization with some other baselines in the literature. they show EM scores depend on the LLM judge. They also investigate if KL regularization should use prompts that are task specific or unrelated to the task of interest.
Critical strengths
s1) authors investigate the influence of data selection for KL divergence calculation when they have KL divergence as extra regularization term in SFT which normally lead to broad misalignment. which is novel contribution.
Critical weaknesses
w1) the set of baselines is quite limited when they compare their regularization with others. and the approaches they choose aren't even SOTA for mitigating emergent misalignment. see more recent works like block-em.
w2) I guess their KL regularization experiments are inconclusive? (i.e. they dont show any prompt selection is better than other one, so I dont see how is this interesting?)
Questions for authors
none