- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether human–LLM conversation datasets contain identifiable source signatures and how these signatures relate to user modeling. Using English conversations from seven datasets, the authors remove selected dataset-specific markers, mask assistant responses, and train linear probes on frozen Qwen2.5 representations. The probes identify dataset sources well above chance, reaching 53.3% accuracy in the seven-way setting against a 14.3% chance baseline. Input ablations show that much of the signal is present in the first user turn, while turn and word counts alone yield substantially lower accuracy. On three datasets, matching selected taxonomy and structural summaries leaves substantial source separability, while matching classifier-score distributions reduces differences in taxonomy summaries.
The paper then trains intent-conditioned user models based on Qwen2.5-7B-Instruct and Llama-3-8B. In a controlled experiment with Qwen-based user models, classifiers trained on real conversations can identify the training sources of generated conversations even when the user models receive the same third-source intents and interact with a fixed assistant. Cross-dataset evaluation shows an in-domain perplexity advantage, and pairwise source-classification accuracy correlates with symmetric transfer gaps for both model families. Finally, classifier-guided importance resampling from other datasets improves target perplexity over uniform sampling at the two reported training budgets.
Critical strengths
-
The paper provides clear empirical evidence that source signatures remain in human–LLM conversation data after several obvious classification cues are removed. The main classifiers use user messages with the listed dataset-specific markers removed and assistant responses masked. They still distinguish sources well above chance across seven datasets. The input ablations also help characterize the signal: the first user turn retains much of the classification accuracy, whereas turn and word counts alone perform substantially worse. These results extend dataset-bias research to human–LLM interactions and provide a useful basis for questioning how broadly findings from a single conversation source apply.
-
The paper connects source separability to user-model generation, evaluation, and training through a coherent set of experiments. The real-to-synthetic experiment is particularly useful because it tests whether patterns learned from recorded conversations remain detectable in generated interactions under shared intents and a fixed assistant. Cross-dataset evaluation then shows that predictive-fit rankings depend on the evaluation source, while classifier-guided selection improves target perplexity over uniform sampling. Together, these results give the signature analysis practical relevance to research that trains and compares user models using public conversation datasets.
Critical weaknesses
-
The taxonomy comparison does not establish the claimed distinctions beyond the taxonomy. Equation 3 in Appendix C.2 matches individual category totals and aggregate structural quantities, leaving category co-occurrences and the distributions of turn and token counts uncontrolled. The remaining source separability could therefore reflect differences already expressible through those variables. The score-matching experiment in Table 5 does not resolve this concern: its score definition and matching procedure are insufficiently specified, and the retained subset sizes and composition are not reported. Although Section 5 acknowledges that joint distributional differences remain uncontrolled, Section 2.3 still concludes that classifiers capture taxonomy information plus additional distinctions. This gap weakens the paper's explanation of what the identified signatures reveal beyond existing descriptions of dataset differences.
-
The practical contribution of the signatures to user-model evaluation and training is only partly established. The propagation and perplexity experiments show source dependence, but do not establish whether the observed differences affect simulation fidelity, as the paper acknowledges. The data-selection experiment offers a concrete application, yet its comparison with uniform sampling leaves the source of the gains unclear. Selection can change both the proportions of donor datasets and the conversations chosen within each dataset, and the current comparison does not distinguish these effects. There is also no comparison with n-gram DSIR to assess what the representation-based classifier adds to an established selection approach. The undefined training-budget unit and missing SFT optimization settings further limit the interpretation of data efficiency. These gaps leave the paper with limited guidance on when its signature analysis improves user-model training or evaluation.
Questions for authors
- Please report the retained sample counts for each dataset pair and matching condition in Table 4, separately for training, validation, and test. How does matching change the composition of the selected conversations relative to the original datasets?
- Which classifier output is used as the score in Table 5, and how are its distributions matched? Given that Equation 3 leaves category co-occurrences and full structural distributions uncontrolled, what supports the conclusion that the classifiers capture distinctions beyond the taxonomy?
- What do the 50% and 100% budgets measure? A comparison with the original DSIR method across more data sizes would help assess whether the gains are consistent.
9 comments
Question 2 + Weakness 1. Classifier output used as the score in Table 5, distribution matching, score definition, and matching procedure
For each dataset pair (A, B), the score is the predicted probability P(B|x) from the binary classifier in Table 4's "Original" column. We compute P(B|x) for the test conversations of both sources.
How are the distributions matched? We split [0, 1] (the range of P(B|x)) into 20 equal-width bins. In each bin b, we keep min(n_{A,b}, n_{B,b}) conversations from each source, where n_{A,b} is the number of samples from dataset A whose P(B|x) is within the bin b. For example, while WC-1M has 4,221 test conversations with P(LMSYS) < 0.05, only the 548 that LMSYS can match are kept.
Table R2. Score matching on the test split: the binned histograms match exactly.
| Pair | Test N | Retained per source | Wasserstein-1 distance before → after |
|---|---|---|---|
| WC-1M / LMSYS | 7,000 | 2,962 | 0.555 → 0.004 |
| WC-1M / ShareGPT | 7,000 | 1,880 | 0.709 → 0.006 |
| LMSYS / ShareGPT | 7,000 | 2,259 | 0.660 → 0.004 |
Robustness checks:
- Bin width. With 10 or 40 bins, the retained sizes are nearly unchanged: 2,968 / 1,906 / 2,276 and 2,941 / 1,858 / 2,240.
- Sampling seed. Across 20 random seeds for the within-bin draw, the post-matching gaps stay within these ranges:
| Pair | Function | Topic | Multi-turn |
|---|---|---|---|
| WC-1M / LMSYS | 3.6 - 4.4 | 3.1 - 3.9 | 5.0 - 5.4 |
| WC-1M / ShareGPT | 4.3 - 5.7 | 2.6 - 3.5 | 4.8 - 6.7 |
| LMSYS / ShareGPT | 1.2 - 2.2 | 1.5 - 2.5 | 3.8 - 4.8 |
Question 2 + Weakness 1. Uncontrolled category co-occurrance and the distributional control of the structure
We appreciate the reviewer for pointing out the uncontrolled category co-occurrance and the distributional control of the structure facet. It sparked two experiments ((i) and (ii) below) that corroborates our claim. Both use the 22 category indicators (9 Function, 9 Topic, 4 Multi-turn) together with the user-turn and user-token counts.
(i) How well can these variables alone separate the datasets?
We trained gradient-boosted decision trees (GBDT) on the 22 indicators plus the raw turn and token counts: trees can represent co-occurrence patterns and count distributions. We trained on the training split and evaluated on the test conversations.
Table R3. Test accuracy (%).
| Pair | Additive logistic regression | GBDT: categories only | GBDT: turns + tokens only | GBDT: categories + turns + tokens | Neural classifier |
|---|---|---|---|---|---|
| WC-1M / LMSYS | 68.0 | 68.4 | 62.7 | 70.3 | 78.7 |
| WC-1M / ShareGPT | 68.7 | 69.8 | 61.5 | 70.7 | 86.2 |
| LMSYS / ShareGPT | 72.5 | 73.0 | 67.2 | 74.2 | 83.4 |
Thus, incorporating co-occurance patterns and structural distributions adds only 1.7–2.3 pp over an additive model. Even the most flexible taxonomy-based classifier stays 8.4 - 15.5 pp below the neural classifier.
(ii) Exact joint matching. We put every test conversation into a partition defined by three things:
- full 22-dimensional category vector, so preserving co-occurrences
- user-turn bin: {1, 2, 3, 4-5, 6+}
- user-token octile (octiles computed on the pair's pooled test conversations)
We then keep min(n_{A,p}, n_{B,p}) conversations per partition p, making two sources have the same joint distribution over these variables. So a classifier that uses only these variables scores 50%. We then evaluate the frozen neural classifier from Table 4 on this subset, with no retraining.
Table R4. Classification accuracy (%) after exact matching of joint distributions.
| Pair | Retained per source | Acc. before | Acc. after | Range over 100 sampling seeds |
|---|---|---|---|---|
| WC-1M / LMSYS | 1,159 | 78.7 | 69.2 | 68.4 - 69.8 |
| WC-1M / ShareGPT | 900 | 86.2 | 77.9 | 77.1 - 78.8 |
| LMSYS / ShareGPT | 1,040 | 83.4 | 70.5 | 70.2 - 71.3 |
So even when two sets of conversations agree exactly on the category profile, its co-occurrences, and the binned length and turn distributions, the classifier still identifies their source well above chance.
We thank the reviewer for the careful reading and for questions that pointed to the gaps in our submission! We respond to each point below. Because each question / weakness point has several components, we break each into multiple parts.
Question 1. Retained sample counts for Table 4, and the change of the composition
The integer linear programming (Equation 3) selects the largest equal-sized subsets of conversations that satisfy the constraints. It is solved separately within each split, so the train, validation, and test sets remain disjoint. We will add the table below to Appendix C.
Table R1. Conversations retained per train / val / test, for each condition in Table 4. Both sources keep the same number of conversations by construction.
How matching changes composition. Matching keeps the types of samples which two sources have in common; it removes conversations from whichever categories one source over-represents. Under F+T+M+S matching setup:
WC-1M / LMSYS (75.4% retained). The largest category gaps were Topic: Arts & Entertainment (62.1% vs. 36.3% → 46.1% in both) and Function: Content Generation (77.4% vs. 57.3% → 66.4%), i.e., the creative writing requests that are more common in WC-1M. Matching Structure: user-token counts mainly removes WC-1M's very long prompts: average tokens per user message go from 160.7 (WC-1M) and 50.1 (LMSYS) to 60.2 in both.
WC-1M / ShareGPT (62.8% retained) and LMSYS / ShareGPT (58.5% retained). The largest gap was Multi-turn: extend or deepen the prior task, 60.5% (ShareGPT) vs. 30.2% (WC-1M) and 22.4% (LMSYS) → 40.9% (matched ShareGPT / WC-1M) and 35.4% (matched ShareGPT / LMSYS). Matching removes most of ShareGPT's long sessions: user turns per conversation go from 7.45 to 3.21 (when matched with WC-1M) and 2.76 (when matched with LMSYS).
The accuracy drops in Table 4 results from both (1) change in composition of human-designed categories and (2) reduced training set size. We note that even after reduced training set size, which typically decreases test accuracy (Figure 3 left), the accuracy is well above the 50% chance.