- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies whether frozen dialogue models remain behaviorally consistent when local speaker identifiers are renamed without changing their meaning. The authors address their claims by introducing a portability audit and evaluate several speaker-ID interfaces.
Critical strengths
1. The paper studies a question about whether arbitrary local naming can affect model predictions. The portability audit provides a simple and concrete way to expose this kind of representation sensitivity.
Critical weaknesses
1. The broader reliability framing seems overclaimed than what the current evidence supports. The setting only considers local identifiers that carry no task semantics, the experiments mainly test whether the model satisfies this particular interface contract. This is meaningful for the nominal-ID setting, but it can not support a broader claim about speaker-aware reliability
2. The main scientific insight appears fairly limited. Since the local IDs are defined to carry no semantics, prediction changes under renaming mainly reflect sensitivity to an arbitrary representation. The corresponding permutation invariance after removing this arbitrariness also seems fairly expected.
Questions for authors
1. What evidence supports extending the conclusions from nominal local identifiers to broader speaker-aware reliability?
2. For the canonical and relation-only interfaces, what empirical behavior is established beyond the invariance built into their construction?
3. At the individual-prediction level, how frequent are the changes underlying the reported conversation-level unit risk?
4. Which of the paper's conclusions are intended to extend to settings where speaker information carries task-relevant semantics?