acceptodds
Under review as a conference paper at ICLR 2027

InteractShift: Diagnosing Doctor LLM Consultation Degradation under Difficult Interactions via Fact-Anchored Paired Stress Testing

Abstract

Performance with cooperative patients does not establish how doctor LLM consultation capabilities change when interactions become difficult. We introduce InteractShift, a fact-anchored paired stress test that holds clinical facts fixed while varying interaction profiles. Across 147 clinical cases, one Cooperative and five Difficult interaction profiles, and nine doctor LLMs, we evaluate six consultation capabilities and separately audit doctor-side elicitation and patient-side disclosure at the fact level. Difficult interactions produce significant but uneven degradation: history-taking Coverage decreases by 6.3 points, whereas clinical information Elicitation decreases by 24.9 points on a 0-100 scale. Fact auditing shows more target facts receive explicit or partial inquiry under Difficult conditions despite lower domain-level coverage, yet correct disclosure is lower within each doctor-side elicitation status. Explicit inquiry is associated with smaller Cooperative-Difficult gaps in correct disclosure, although substantial deficits remain under Vague and Overwhelmed interactions. Different interaction profiles stress different consultation capabilities, while models differ in retained capabilities and shifts in doctor-side elicitation. Complementary expert studies reproduce the main degradation pattern, support patient simulation quality, and show agreement with automatic evaluation at both capability and fact levels. InteractShift provides a controlled basis for diagnosing where consultation capabilities degrade and information acquisition breaks down.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.