Can Individual Physician History Improve Dialogue-Action Prediction? A Personalization Benchmark with Four Profile Sources

Profile-based personalization uses individual statistical priors, history retrieval, dense profile representations, and user-specific parameter adaptation. These methods use history differently, but they are usually evaluated in the same way: predictions made with the target individual's profile are compared with predictions made without a profile. An improvement in this comparison does not by itself show that the model uses history specific to that individual. Relevant histories from similar individuals may already help, while a profile from another individual may harm prediction through mismatch. We study this issue in physician dialogue-action prediction. For each target physician reply, we hold the current dialogue fixed and vary only the profile source, comparing no profile, a department-pooled profile, another physician's profile, and the target physician's profile. The pooled profile excludes the target physician, and the other physician comes from the same department and has a similar number of profile consultations. We construct a benchmark with 244 evaluation physicians and 164,702 physician-reply events in the test set, labeled with 11 functional dialogue actions. We evaluate seven representative methods spanning statistical priors, history retrieval, dense profile representations, and user-specific parameter adaptation. Some gains remain after the target physician is excluded, while some Target-versus-Other differences arise because the other physician's profile increases the loss. Same-department history retains part of the profile gain for several methods, whereas the additional improvement from the target physician's history varies by method. Comparing all four sources shows whether an apparent gain remains with same-department history, depends on the target physician's records, or is enlarged by a mismatched profile.