acceptodds
Under review as a conference paper at ICLR 2027

Dataset Signatures in Human-LLM Interactions and User Modeling

Abstract

Human–LLM interaction datasets shape our understanding of AI use and provide a foundation for downstream research, including training and evaluation of user models. In recent years, a growing number of datasets have sought to capture a representative picture of human–LLM interactions. But how different are the pictures these datasets provide, and what do those differences mean for research built on them? We study these questions across seven conversation datasets, spanning in-the-wild chat logs and human preference data. We begin by revisiting the dataset classification experiment of Torralba & Efros: neural network classifiers identify a conversation's source from user messages alone well above chance, indicating distinctive dataset signatures. This separability persists after matching datasets on the dimensions of human-designed taxonomies, implying subtle differences that these taxonomies do not capture. We then examine the implications for user modeling: how dataset signatures propagate to the outputs of user models trained on these datasets; how dataset choice influences evaluations of user model quality; and how dataset classifiers can guide data selection for training user models. While each dataset is meant to capture a slice of “real-world” interactions, our findings reveal the extent to which these slices diverge, and the consequences of those differences for research built on these foundations.

Primary area:
datasets and benchmarks
Keywords:
Dataset signature, Human-AI interaction, User modeling

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_cZjoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether human–LLM conversation datasets contain identifiable source signatures and how these signatures relate to user modeling. Using English conversations from seven datasets, the authors remove selected dataset-specific markers, mask assistant responses, and train linear probes on frozen Qwen2.5 representations. The probes identify dataset sources well above chance, reaching 53.3% accuracy in the seven-way setting against a 14.3% chance baseline. Input ablations show that much of the signal is present in the first user turn, while turn and word counts alone yield substantially lower accuracy. On three datasets, matching selected taxonomy and structural summaries leaves substantial source separability, while matching classifier-score distributions reduces differences in taxonomy summaries.

The paper then trains intent-conditioned user models based on Qwen2.5-7B-Instruct and Llama-3-8B. In a controlled experiment with Qwen-based user models, classifiers trained on real conversations can identify the training sources of generated conversations even when the user models receive the same third-source intents and interact with a fixed assistant. Cross-dataset evaluation shows an in-domain perplexity advantage, and pairwise source-classification accuracy correlates with symmetric transfer gaps for both model families. Finally, classifier-guided importance resampling from other datasets improves target perplexity over uniform sampling at the two reported training budgets.

Critical strengths
  1. The paper provides clear empirical evidence that source signatures remain in human–LLM conversation data after several obvious classification cues are removed. The main classifiers use user messages with the listed dataset-specific markers removed and assistant responses masked. They still distinguish sources well above chance across seven datasets. The input ablations also help characterize the signal: the first user turn retains much of the classification accuracy, whereas turn and word counts alone perform substantially worse. These results extend dataset-bias research to human–LLM interactions and provide a useful basis for questioning how broadly findings from a single conversation source apply.

  2. The paper connects source separability to user-model generation, evaluation, and training through a coherent set of experiments. The real-to-synthetic experiment is particularly useful because it tests whether patterns learned from recorded conversations remain detectable in generated interactions under shared intents and a fixed assistant. Cross-dataset evaluation then shows that predictive-fit rankings depend on the evaluation source, while classifier-guided selection improves target perplexity over uniform sampling. Together, these results give the signature analysis practical relevance to research that trains and compares user models using public conversation datasets.

Critical weaknesses
  1. The taxonomy comparison does not establish the claimed distinctions beyond the taxonomy. Equation 3 in Appendix C.2 matches individual category totals and aggregate structural quantities, leaving category co-occurrences and the distributions of turn and token counts uncontrolled. The remaining source separability could therefore reflect differences already expressible through those variables. The score-matching experiment in Table 5 does not resolve this concern: its score definition and matching procedure are insufficiently specified, and the retained subset sizes and composition are not reported. Although Section 5 acknowledges that joint distributional differences remain uncontrolled, Section 2.3 still concludes that classifiers capture taxonomy information plus additional distinctions. This gap weakens the paper's explanation of what the identified signatures reveal beyond existing descriptions of dataset differences.

  2. The practical contribution of the signatures to user-model evaluation and training is only partly established. The propagation and perplexity experiments show source dependence, but do not establish whether the observed differences affect simulation fidelity, as the paper acknowledges. The data-selection experiment offers a concrete application, yet its comparison with uniform sampling leaves the source of the gains unclear. Selection can change both the proportions of donor datasets and the conversations chosen within each dataset, and the current comparison does not distinguish these effects. There is also no comparison with n-gram DSIR to assess what the representation-based classifier adds to an established selection approach. The undefined training-budget unit and missing SFT optimization settings further limit the interpretation of data efficiency. These gaps leave the paper with limited guidance on when its signature analysis improves user-model training or evaluation.

Questions for authors
  1. Please report the retained sample counts for each dataset pair and matching condition in Table 4, separately for training, validation, and test. How does matching change the composition of the selected conversations relative to the original datasets?
  2. Which classifier output is used as the score in Table 5, and how are its distributions matched? Given that Equation 3 leaves category co-occurrences and full structural distributions uncontrolled, what supports the conclusion that the classifiers capture distinctions beyond the taxonomy?
  3. What do the 50% and 100% budgets measure? A comparison with the original DSIR method across more data sizes would help assess whether the gains are consistent.
9 comments

We thank the reviewer for the careful reading and for questions that pointed to the gaps in our submission! We respond to each point below. Because each question / weakness point has several components, we break each into multiple parts.


Question 1. Retained sample counts for Table 4, and the change of the composition

The integer linear programming (Equation 3) selects the largest equal-sized subsets of conversations that satisfy the constraints. It is solved separately within each split, so the train, validation, and test sets remain disjoint. We will add the table below to Appendix C.

Table R1. Conversations retained per train / val / test, for each condition in Table 4. Both sources keep the same number of conversations by construction.

PairConditionTrain / Val / TestTotalKeptTest acc.
WC-1M / LMSYSOriginal40,000 / 3,000 / 7,00050,000100%78.7
F34,558 / 2,283 / 5,34642,18784.4%77.3
F+T32,229 / 1,586 / 4,91438,72977.5%75.4
F+T+M32,063 / 1,570 / 4,88238,51577.0%75.4
F+T+M+S31,403 / 1,529 / 4,74537,67775.4%74.9
WC-1M / ShareGPTOriginal40,000 / 3,000 / 7,00050,000100%86.2
F28,076 / 1,879 / 4,88934,84469.7%83.5
F+T27,932 / 1,504 / 4,86034,29668.6%82.9
F+T+M25,352 / 1,485 / 4,68631,52363.0%83.8
F+T+M+S25,352 / 1,441 / 4,61531,40862.8%83.2
LMSYS / ShareGPTOriginal40,000 / 3,000 / 7,00050,000100%83.4
F27,257 / 2,055 / 4,74234,05468.1%80.3
F+T26,705 / 2,010 / 4,64833,36366.7%79.9
F+T+M23,480 / 1,813 / 4,00329,29658.6%77.2
F+T+M+S23,458 / 1,809 / 4,00329,27058.5%77.4

How matching changes composition. Matching keeps the types of samples which two sources have in common; it removes conversations from whichever categories one source over-represents. Under F+T+M+S matching setup:

  • WC-1M / LMSYS (75.4% retained). The largest category gaps were Topic: Arts & Entertainment (62.1% vs. 36.3% → 46.1% in both) and Function: Content Generation (77.4% vs. 57.3% → 66.4%), i.e., the creative writing requests that are more common in WC-1M. Matching Structure: user-token counts mainly removes WC-1M's very long prompts: average tokens per user message go from 160.7 (WC-1M) and 50.1 (LMSYS) to 60.2 in both.

  • WC-1M / ShareGPT (62.8% retained) and LMSYS / ShareGPT (58.5% retained). The largest gap was Multi-turn: extend or deepen the prior task, 60.5% (ShareGPT) vs. 30.2% (WC-1M) and 22.4% (LMSYS) → 40.9% (matched ShareGPT / WC-1M) and 35.4% (matched ShareGPT / LMSYS). Matching removes most of ShareGPT's long sessions: user turns per conversation go from 7.45 to 3.21 (when matched with WC-1M) and 2.76 (when matched with LMSYS).

The accuracy drops in Table 4 results from both (1) change in composition of human-designed categories and (2) reduced training set size. We note that even after reduced training set size, which typically decreases test accuracy (Figure 3 left), the accuracy is well above the 50% chance.

Question 2 + Weakness 1. Classifier output used as the score in Table 5, distribution matching, score definition, and matching procedure

For each dataset pair (A, B), the score is the predicted probability P(B|x) from the binary classifier in Table 4's "Original" column. We compute P(B|x) for the test conversations of both sources.

How are the distributions matched? We split [0, 1] (the range of P(B|x)) into 20 equal-width bins. In each bin b, we keep min(n_{A,b}, n_{B,b}) conversations from each source, where n_{A,b} is the number of samples from dataset A whose P(B|x) is within the bin b. For example, while WC-1M has 4,221 test conversations with P(LMSYS) < 0.05, only the 548 that LMSYS can match are kept.

Table R2. Score matching on the test split: the binned histograms match exactly.

PairTest NRetained per sourceWasserstein-1 distance before → after
WC-1M / LMSYS7,0002,9620.555 → 0.004
WC-1M / ShareGPT7,0001,8800.709 → 0.006
LMSYS / ShareGPT7,0002,2590.660 → 0.004

Robustness checks:

  • Bin width. With 10 or 40 bins, the retained sizes are nearly unchanged: 2,968 / 1,906 / 2,276 and 2,941 / 1,858 / 2,240.
  • Sampling seed. Across 20 random seeds for the within-bin draw, the post-matching gaps stay within these ranges:
PairFunctionTopicMulti-turn
WC-1M / LMSYS3.6 - 4.43.1 - 3.95.0 - 5.4
WC-1M / ShareGPT4.3 - 5.72.6 - 3.54.8 - 6.7
LMSYS / ShareGPT1.2 - 2.21.5 - 2.53.8 - 4.8

Question 2 + Weakness 1. Uncontrolled category co-occurrance and the distributional control of the structure

We appreciate the reviewer for pointing out the uncontrolled category co-occurrance and the distributional control of the structure facet. It sparked two experiments ((i) and (ii) below) that corroborates our claim. Both use the 22 category indicators (9 Function, 9 Topic, 4 Multi-turn) together with the user-turn and user-token counts.

(i) How well can these variables alone separate the datasets?

We trained gradient-boosted decision trees (GBDT) on the 22 indicators plus the raw turn and token counts: trees can represent co-occurrence patterns and count distributions. We trained on the training split and evaluated on the test conversations.

Table R3. Test accuracy (%).

PairAdditive logistic regressionGBDT: categories onlyGBDT: turns + tokens onlyGBDT: categories + turns + tokensNeural classifier
WC-1M / LMSYS68.068.462.770.378.7
WC-1M / ShareGPT68.769.861.570.786.2
LMSYS / ShareGPT72.573.067.274.283.4

Thus, incorporating co-occurance patterns and structural distributions adds only 1.7–2.3 pp over an additive model. Even the most flexible taxonomy-based classifier stays 8.4 - 15.5 pp below the neural classifier.

(ii) Exact joint matching. We put every test conversation into a partition defined by three things:

  • full 22-dimensional category vector, so preserving co-occurrences
  • user-turn bin: {1, 2, 3, 4-5, 6+}
  • user-token octile (octiles computed on the pair's pooled test conversations)

We then keep min(n_{A,p}, n_{B,p}) conversations per partition p, making two sources have the same joint distribution over these variables. So a classifier that uses only these variables scores 50%. We then evaluate the frozen neural classifier from Table 4 on this subset, with no retraining.

Table R4. Classification accuracy (%) after exact matching of joint distributions.

PairRetained per sourceAcc. beforeAcc. afterRange over 100 sampling seeds
WC-1M / LMSYS1,15978.769.268.4 - 69.8
WC-1M / ShareGPT90086.277.977.1 - 78.8
LMSYS / ShareGPT1,04083.470.570.2 - 71.3

So even when two sets of conversations agree exactly on the category profile, its co-occurrences, and the binned length and turn distributions, the classifier still identifies their source well above chance.

Reviewer_JAghOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper analyzes datasets of human–LLM interactions and investigates whether they carry distinctive signatures that allow their sources to be identified. Using frozen language model representations with a multinomial logistic regression head, the authors find that dataset classification accuracy remains substantially above chance after removing dataset-specific markers and masking assistant responses. Separability also persists after matching datasets on selected dimensions of human-designed taxonomies. In a separate ablation involving WC-1M, LMSYS, and ShareChat, using only the preprocessed first user message yields 68.8% accuracy, compared with a chance level of 33.3%.

The paper further examines the implications of these signatures for user modeling. It shows that signatures persist in conversations generated by user models trained on different datasets, that model rankings depend on the evaluation dataset, and that dataset classifiers can guide training data selection to reduce target-dataset perplexity. Overall, the paper views interaction datasets as different slices of real-world interactions and demonstrates that their differences matter for user model generation, evaluation, and training. Whether these differences also change downstream conclusions about assistant behavior remains an open question.

Critical strengths

1, Whether different LLM interaction datasets carry distinctive signatures is a well-motivated and meaningful question. Understanding these differences has important implications for interpreting the reliability, reproducibility, and generalizability of research conducted using these datasets.

2, The paper provides a relatively rigorous investigation of dataset signatures. It includes extensive and well-motivated experimental analyses, making its central finding—that the datasets remain distinguishable under the examined controls—convincing.

3, In Section 3.3, classifier-guided training data selection reduces target-dataset perplexity relative to uniform random sampling at matched training budgets. This demonstrates the practical value of dataset classification for improving user models’ predictive fit to a target dataset.

Critical weaknesses

1, Although the authors establish through extensive experiments that different LLM interaction datasets carry distinctive signatures, the specific features underlying these signatures remain insufficiently characterized. Section 2.3 examines broad semantic categories, including Function, Topic, and Multi-turn behavior, but does not provide a deeper analysis of the finer-grained linguistic differences captured by the classifiers. Beyond the examined factors, such as capitalization, turn counts, and word counts, lexical style, grammatical accuracy, and other linguistic characteristics may also contribute to the observed separability.

2, The experiments use only the English-language subsets of the selected datasets, limiting the generalizability of the conclusions across languages, as the authors acknowledge in Section 5. Language-dependent assistant capabilities and differences in users’ language proficiency may influence interaction patterns, but these possibilities are not examined. In particular, whether differences in English proficiency contribute to dataset separability remains an untested hypothesis. It is therefore unclear how strongly the findings extend to other languages or whether separability would be more or less pronounced outside English.

3, The implications for user modeling discussed in Section 3 are important, but the experiments are restricted to intent-conditioned user models, leaving their applicability to other user modeling approaches unclear. Moreover, although Section 3.1 quantitatively compares generated conversations through dataset classification, the paper does not provide side-by-side qualitative examples of outputs generated under the same conditions by user models trained on different datasets. Such examples would make the observed differences more interpretable and help readers assess the conclusions.

Questions for authors

1, Could the authors investigate how finer-grained linguistic characteristics mentioned in W1 and W2—such as lexical style, grammatical accuracy, and proxies for users’ English proficiency—contribute to dataset classification, through analyses similar to those in Table 3?

2, Could the authors provide side-by-side comparisons of outputs sampled from user models trained on different datasets under the same conditioning information, such as the same intent and dialogue history? Alternatively, could they compare the per-token probabilities assigned to the same ground-truth user utterance under identical conditioning contexts?

3, Could the authors include additional text representations for dataset classification, such as term-frequency or TF–IDF vectors paired with a linear classifier? This would help assess how much of the observed separability can be captured by simple lexical features and what additional information the language model representations provide.

7 comments

We sincerely thank the reviewer for the careful reading, and for recognizing the value of our research question. We’d like to take the opportunity to discuss the weaknesses here:


W1 / Q1. Which linguistic characteristics contribute to dataset separability?

(a) Fine-grained linguistic features per dataset

We computed 29 interpretable features over 4 characteristics for each conversation's user messages. In the following, word frequencies come from the Python library rspeer/wordfreq lexicon on the Zipf scale (log10 of occurrences per billion words; 3 means one per million).

  • Structure (3):
    • user turns per conversation
    • words per conversation
    • mean words per user turn
  • Lexical style (11):
    • number of characters per word
    • vocabulary diversity (type-token ratio averaged over 50-word windows)
    • mean word frequency (Zipf scale)
    • number of rare words (Zipf < 3)
    • function words (we used Scikit-learn's English stop-word list)
    • first-person (I, me, my, we, ...) and second-person (you, your, ...) pronouns
    • politeness markers (please, thanks, sorry, ...)
    • hedges (maybe, perhaps, I think, kind of, ...)
    • shares of turns that open with a greeting (hi, hello, dear, ...)
    • turns containing "?"
    • turns opening with a request verb (write, explain, give, create, list, translate, ...)
  • Formatting (11):
    • uppercase share of characters
    • turns opening with a lowercase letter
    • number of all-capitalized words
    • punctuation marks per word
    • turns ending in end-of-sentence punctuation (./?/! or a closing quote or parenthesis)
    • a space before punctuation (e.g., "word ,")
    • curly quotes as a share of all quote marks
    • non-ASCII characters per 1K characters
    • lines formatted as markdown (bullet, numbered item, header, bold)
    • turns containing code (a fenced block, or lines ending in {, } or ;)
    • turns containing URLs
  • Proxy metrics for English proficiency (4):
    • out-of-vocabulary words: words of >=2 characters absent from the English lexicon. They are mostly misspellings, plus non-English words and rare names
    • very rare word forms (Zipf < 1.5)
    • Flesch–Kincaid grade level score
    • words per sentence

This table shows example features on the test split:

FeatureWC-1MWC-4.8MLMSYSArenaShareChatShareGPTHH-RLHF
User turns / conv. (mean)2.92.82.41.63.87.72.4
Words / user turn (median)30351417172010
Turns containing "?" (%)27273437413473
Turns opening with a request verb (%)2817292116274
Turns opening with a lowercase letter (%)2629333723367
Turns ending in sentence-final punctuation (%)57555150564989
Curly quotes among quote marks (%)159441161
Turns containing code (%)4934130
Non-ASCII characters per 1K characters8.814.03.64.02.04.80.2
out-of-vocabulary per 100 words2.53.82.12.61.52.00.1
Flesch–Kincaid grade (median)10.19.67.47.88.59.43.9

Some datasets have a distinctive style. For example,

  • HH-RLHF: crowdworkers write short, simple, well-punctuated first-person questions.
  • ShareGPT: conversations are long (mean 7.7 user turns), with many follow-ups that extend the previous request.
  • WildChat: both releases contain the most non-ASCII text; WC-4.8M also has the most out-of-vocab words and code.

(b) How much do these characteristics contribute?

We follow Table 3 (WC-1M + LMSYS + ShareChat) and test the effect of characteristic on classification accuracy via two ways:

  • Remove the characteristic. We retrain the classifier on text from which one characteristic has been removed:
    • Rewriting: gpt-5.6-luna rewrites every user turn. A minimal correction fixes spelling, grammar, punctuation, and capitalization errors, which removes grammatical cues. A neutral paraphrase keeps the content in new wording, which removes lexical-style cues.
    • Matching: we keep subsets whose proficiency proxies have identical joint distributions across datasets, following the prodecure taken in Table 4. For grammatical accuracy, this is the grammar-edit rate (the share of words the minimal correction changed).
  • Keep only the characteristic. Each conversation is represented only by its numbers from (a). A classifier (logistic regression or gradient-boosted trees) then predicts the dataset from these numbers alone.

Table. Table 3-style analyses on WC-1M + LMSYS + ShareChat (chance 33.3%).

Remove the characteristicAccuracy (%)
Original (baseline)72.5
Rewriting: minimal correction (spelling / grammar)70.8 (−1.7)
Rewriting: neutral paraphrase70.2 (−2.3)
Matching: on proficiency proxies ^69.2 (-3.3)
Matching: on proficiency + formatting proxies ^67.0 (-5.5)

^ Retained per dataset (train/val/test): 7,675 / 1,919 / 4,857 and 6,135 / 1,108 / 3,619.

Keep only the characteristic (features from (a))LinearGradient-boosted trees
Structure41.241.9
Lexical style45.351.0
Formatting41.648.2
Proxy metrics for English Proficiency42.842.8
Grammatical accuracy (grammar-edit rate)33.235.2
All features51.657.4

We observe the following:

  1. Grammatical accuracy is not what separates these datasets. Classifying from the edit rate is near chance (35.2%).
  2. Each interpretable features capture part of the signal, but not all of it. Each characteristic (structure -- proficiency) acheive accuracy beyond chance, and all features together even reach 57.4%, but short of the neural classifier's 72.5%.

W2. English proficiency and generalization beyond English

(a) English proficiency hypothesis

WildChat records each user's IP-derived country, letting us test the proficiency hypothesis directly within WC-1M. We compare users from English-speaking countries (US, UK, Canada, Australia, New Zealand, etc.; n=2,784 test conversations) with all other users (n=4,216). We observe:

  • Proxies behave as expected. Users from non-English-speaking countries have more grammar edits (17.0 vs. 15.4 per 100 words), more out-of-vocab (2.8 vs. 2.0 per 100), more lowercase openings (30% vs. 21%), and far more non-ASCII characters (13.2 vs. 2.3 per 1K).
  • Accuracy depends on them modestly. On the three-way task (WC-1M + LMSYS + ShareChat), the classifier recognizes WC-1M with 68.8% accuracy for samples from non-English-speaking countries, while 63.0% for English-speaking. Proficiency cues help identify WildChat, but WildChat conversations from English-speaking countries remain clearly identifiable.

(b) Do signatures exist in other languages?

We reran the classification task in the three non-English languages with the most conversations in both WC-1M and LMSYS: Chinese, Russian, and Spanish. ShareGPT has enough conversations only in Chinese, so we also test WC-1M + LMSYS + ShareGPT in the Chinese setting. The other datasets are too small in non-English.

  • Curation: the paper's pipeline (Section 2.1)
    • Language filter: the release's language label + fastText classification
    • Deduplication within and across datasets, and template filtering
    • Assistant responses are masked, and dataset-specific markers are deleted
  • Size: 4,000 / 500 / 1,500 train/val/test samples per dataset. An English (control setup) uses the same numbers, drawn from the English corpus.
  • Classifiers: the linear classifier with Qwen2.5-7B representations and the TF-IDF baseline (described in response of Q3).

Table. Test accuracy (%), linear classifier / TF-IDF (word+char).

DatasetsChanceChineseRussianSpanishEnglish (control)
WC-1M + LMSYS50.080.9 / 78.069.0 / 70.368.7 / 72.775.3 / 72.9
WC-1M + LMSYS + ShareGPT33.375.1 / 70.2––70.3 / 64.1
  • Signatures are not specific to English. In every language, classifiers separate the datasets far above chance.
  • Their strength varies by language.
    • Chinese is more separable than English: the probe is about 5 points higher in both settings. One visible difference is that Chinese LMSYS conversations are much shorter than Chinese WC-1M conversations (1.6 vs. 2.8 user turns); the English gap is smaller (2.4 vs. 2.9).
    • Russian and Spanish are 6-7 points below English for the linear classifier, but TF-IDF is within 3 points of English. We think this gap may reflect the LLM's weaker representations of these languages.
Reviewer_8xRaOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper performs a study on existing Human LLM conversation datasets, including many commonly used ones (wildchat, lmsys, sharegpt, HH-RLHF...), then the authors studies the signatures of these datasets --- whether a neural network is able to classify conversation sources. The author uses these signatures to study downstream impact on model training and evaluation.

Critical strengths
  • The paper studies an important topic. Broadly, the community should benefit more from a deeper dive into our datasets. In the context of this paper, it shows that different datasets have traits / signatures that is not simply explained by topic, and that training on such dataset inherits the signature.

  • The paper presents comprehensive ablations. For example, it has detailed ablations on the effect of training dataset size, and what is being fed into the classifier. The results clearly show that their finding that datasets has traits that are easily seperable.

  • The paper presents not just a study of dataset signatures, but also their downstream effect in model training and data selection, showing proof that we can leverage dataset traits to select better data.

Critical weaknesses
  • The core claim of the paper is that datasets can be easily separated / classified, which is not surprising. I was hoping to see which features did the classifier learn aside from lexical or topics? For example, some dataset have trait A whereas some dataset has trait B ... it would make the paper much more stronger if the authors can dig into what the classifier have learned, besides style, tone and topic.

  • The downstream effect is nice, but would be important to one usecase more solidified rather than two usecases (evaluation & selecting training data). For example, as I have mentioned, can we derive interpretable features of each dataset, and understand what makes each dataset good. Also it would be nice to connect it to how each dataset is collected, and understand the downstream impact on dataset collection choices (e.g. demographics). The current findings / takeaways / implications are not written clearly in a way that is easily adoptable by the community.

Questions for authors

Just minor suggestion: In the abstract the author wrote "we then examine ... implications (of dataset signatures) for user modeling: how dataset choice influences evaluations of user model quality" It would be better to just spell the takeaway out: dataset signatures can skew the evaluation quality ... just because you have achieved low ppl on a certain dataset doesn't mean that you did a good job in user modeling. The current phrasing is unclear to me what is the takeaway here.

3 comments

We sincerely appreciate the reviewer for the suggestions, and for recognizing the value of our research question. We’d like to take the opportunity to discuss the weakness points and questions:


W1. What the classifiers pick up beyond lexical or topics

We agree that some separability is expected, since the datasets were collected differently. To identify the traits involved, we examined interpretable features of user messages (details in our response to Reviewer JAgh) and the most predictive n-grams of a bag-of-words classifier. Table X1 summarizes what we found.

Table X1. Distinctive traits of each dataset and the collection process behind them.

DatasetCollectionDistinctive traits
WC-1MFree GPT chatbot on Hugging Face SpacesRecurring role-play and fan-fiction queries; many non-native English users
WC-4.8MSame service, later periodNewer models and period vocabulary (top n-grams o1, 4o, 2024); the most code among datasets
LMSYSVicuna demo, Chatbot ArenaUsers often test the model rather than use it (top n-grams vicuna, 13b; benchmark-style prompts)
ArenaModel-comparison votes (2025)Queries stating the user is comparing models; 2025 model names in user messages
ShareChatPublicly shared, several platformsPlatform and period vocabulary (top-n grams grok, on x)
ShareGPTShared via a browser extensionLongest conversations (avg. 7.7 user turns), extended with continue
HH-RLHFPaid crowdworkers with instructionsShort, first-person, well-punctuated questions

These traits reflect interaction behavior (how much of the task the first message states, and how users follow up and end conversations), the user's relation to the system (using, testing, or eliciting it), the time period and its assistants, and who the users are. Our ablations (response to Reviewer JAgh) indicate that no single factor fully accounts for the separability: controlling for topic, style, formatting, or keeping only the first user message, one at a time, leaves most of separability remained. The signature is spread across content, structure, wording, and formatting.


W2. What makes a dataset good, and collection choices

Thank you for the constructive suggestion. We agree, and would like to briefly discuss additional observations.

What makes a training dataset good. We think it is mostly a closeness to the target population the practitioner has on mind; no single dataset is best everywhere. To test this beyond our seven datasets, we evaluated all seven user models on the PRISM alignment dataset (Kirk et al., 2024), which none of them was trained on.

  • HH-RLHF's user model has the lowest in-domain perplexity (4.53 vs. 5.43–7.33 for the others), yet it transfers worst to the other six datasets and ranks 7th out of 7 on PRISM. The formulaic prompts of crowdworkers are easy to fit but far from users in other datasets.
  • LMSYS's user model transfers best: it has the smallest average gap to the in-domain user model (Table 7), and also the lowest perplexity on PRISM.

Collection choices and demographics. Table X1 links each signature to a collection process. Recruitment seems to matter most: crowdworkers (HH-RLHF) produce the most separable dataset and the worst-transferring user model. Platform purpose (testing vs. use), sharing-based collection (ShareGPT/Chat), and time period also leave signatures: a classifier separates even the two WildChat releases, collected on the same service at different times. Unfortunately, we noticed that demographics are mostly undocumented: only WildChat records users' (IP-derived) country. We will therefore recommend that dataset releases document recruitment, platform, collection period, and user metadata, so that the community can understand the population behind the datasets it uses.


Q1. The abstract's takeaway

We appreciate the suggestion. We will revise the sentence as follows, and we will keep refining it for clarity:

User models trained on different datasets produce distinguishable conversations even under identical conditions, and conclusions drawn from them depend on the user model’s training dataset: an assistant's measured performance can change with the user model, and a low perplexity on one dataset does not mean that a user model simulates users well in general.

I want to thank the authors for their rebuttal and I remain my positive score. One thing that I was curious about: does the authors expect a shift of how people are interacting with models from 2023 - now? and is there analysis conducted on more recent datasets? another question is that can this type of analysis be used to develop better user simulators?

Thank you so much for the kind follow-up and for keeping your positive score! Both questions get at where we hope this line of work goes next.


1. Shifts in how people interact with models

The datasets in our paper cover human–AI chats from 2022 to 2025, and we see some changes over time between datasets collected on similar platforms. (1) WC-1M vs. WC-4.8M: in the later release, more turns contain code, user turns are longer, and fewer turns open with a request verb such as 'write' or 'explain.' (2) LMSYS vs. Arena: in the later one, users more often say outright that they are comparing models.

After 2025, we expect larger shifts. As models improve, people may use them for more diverse and complex tasks. As assistants become agentic and multimodal, users may hand off longer tasks, even integreated to existing platforms (e.g., an assistant drafting email follow-ups in Gmail). One domain of particular interest is sessions with coding agents [1]. As more human-coding agent interactions are collected, we hope the community will revisit the dataset signature problem there.

2. Developing better user simulators

We believe so, in two ways:

Choosing training data. Classifier-guided selection improves a user model’s fit to a target population using samples from other datasets (Section 3.3). This can help when practitioners have too little data from the target population to train a simulator but enough to evaluate one (or have a privacy issue with using the target population data for training) : they can draw relevant samples from larger sources to improve it. We think recognizing that dataset signatures exist, and selecting data with them in mind, can make a meaningful difference in practice.

Measuring how realistic a simulator is. In our paper, the classifier distinguishes real datasets. A classifier trained to distinguish a real dataset from synthetic ones (generated by a user simulator) could instead test whether the simulator’s outputs resemble the target population. The same signal could also serve as a training reward for the simulator. This is closely related to adversarial training for text generation [2, 3] and to recent work on user simulation [4, 5]. We would welcome your thoughts on this direction.


[1] Baumann et al. SWE-chat: Coding Agent Interactions From Real Users in the Wild. COLM 2026.

[2] Zhang et al. Adversarial Feature Matching for Text Generation. ICML 2017.

[3] Wu et al. Generative Adversarial Imitation Learning for Text Generation. AAAI 2021.

[4] Wang et al. Learning User Simulators with Turing Rewards. EMNLP 2026.

[5] Gul et al. Towards User Simulators That Stay On Task And Still Sound Natural. 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.