acceptodds
Under review as a conference paper at ICLR 2027

AudCraft: Understanding the Audience-Oriented Creation Gap Between Human Creators and LLMs with Real-World Platform Data

Abstract

As low-cost LLM-generated content enters online platforms at scale, understanding how LLMs create for different audiences is increasingly important for assessing their potential impact on content ecosystems. Yet existing evaluations largely treat LLM-generated content as isolated textual artifacts, focusing on properties such as creativity and detectability while overlooking a key aspect of real-world content creation: creators target specific audiences and adapt content to their needs. To address this gap, we introduce AudCraft, an observational corpus of real-world human-created content spanning diverse audiences, and evaluate LLMs from two complementary dimensions of audience-oriented creation: audience orientation, which characterizes whom LLMs create for, and audience design, which characterizes how they adapt their expression to different audiences. Our analysis reveals systematic human-LLM gaps in both dimensions. For audience orientation, LLMs disproportionately concentrate their generations on a narrow subset of audience segments, a pattern that persists across models and prompting strategies. A Tullock contest-based simulation further shows that such concentrated orientation can be amplified through competition for limited audience attention, resulting in increasingly uneven content supply and intensified competition with human creators. For audience design, LLMs exhibit rich and audience-responsive expressions, but this flexibility does not necessarily translate into human-like adaptation: their changes are selective, often misaligned with human patterns, and less audience-specific in vocabulary. Code and data are available at https://anonymous.4open.science/r/AudCraft-54E4.

Primary area:
datasets and benchmarks
Keywords:
Audience-Centric Evaluation, Content Creation, Platform Ecosystem

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WUbZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AudCraft derives a human-content reference and audience taxonomy from Qilin, then compares LLMs with human posts on audience orientation and audience-conditioned expression. It distinguishes expression level, cross-audience variation, and alignment across 55 segments. Results show concentrated default generations, incomplete alignment with human expression, and a blinded preference advantage for human posts.

Critical strengths
  1. Separating expression level, adaptation magnitude, and adaptation alignment shows why lexical richness or large stylistic changes need not reproduce human audience patterns. Analysis across 55 segments and four models makes this more informative than a single overall quality score.

  2. The taxonomy uses real content and engagement queries, and the study adds human validation. The 275 blinded, randomized-order comparisons favor human posts in 58% of cases versus 35% for GPT-5.4-mini, while recognizing the model's contextual-usefulness advantage. This supports the descriptive findings without treating every difference as a deficiency.

Critical weaknesses
  1. The human reference may be selected independently of the model characteristics being evaluated. It removes 256,730 posts (13.9% of otherwise eligible content) with Fast-DetectGPT, without Chinese-platform calibration or filtering sensitivity. Removing human posts that resemble model output could enlarge the stylistic and originality gaps. This directly affects the central comparison.

  2. The audience-design comparison matches domain-intent labels but not concrete topic or genre. Content specificity and topic mix can therefore drive stylometric profiles and human-derived vocabulary scores. The human preference study is valuable but uses the same broad matching. The evidence supports differences under this generation setup more securely than differences specifically attributable to audience adaptation.

Questions for authors

Do the main profile and human-preference findings persist with an unfiltered or manually verified human subset? What detector threshold and Chinese-data calibration were used?

Can topic/genre-matched comparisons establish which profile differences remain, or can the audience-design interpretation be limited to the current broadly conditioned setup?

1 comment
AuthorsOpenReview ↗

Response for Reviewer WUbZ

Thank you for acknowledging our work and your constructive comments. Please kindly find our point-by-point responses below for your review.

1.Do the main profile and human-preference findings persist with an unfiltered or manually verified human subset? What detector threshold and Chinese-data calibration were used?

Thank you for this question. To assess the impact of AI-text filtering, we repeated our analyses using an unfiltered human reference and compared the results with those obtained after filtering.

(1) Stability of Audience Distributions. We compared the unfiltered and filtered human references, each containing 100 posts.

DistributionUnfiltered HHIFiltered HHIΔHHIMean category differenceMax category difference
Audience Domain0.10660.0955−0.01112.08 pp7.00 pp
Audience Intent0.15040.1369−0.01351.42 pp4.00 pp

Although filtering slightly reduces human concentration, the changes are small (ΔHHI = −0.0111 and −0.0135). Importantly, the unfiltered reference still exhibits substantially lower concentration than the LLMs, indicating that our main findings are not driven by detector-based filtering.

(2) Stability of Audience Adaptation Results. We further compared three adaptation results across the same 55 audience groups, each containing 56 estimates (4 models × 14 metrics).

Audience Adaptation resultMean absolute changeMax absolute changeEstimate correlation
Overall-level calibration0.0400.1410.999
Cross-audience adaptation magnitude0.0210.0710.997
Audience-order alignment0.0260.1360.990

The estimates remain highly consistent before and after filtering, with correlations of 0.990–0.999 and mean absolute changes of 0.021–0.040. None of these differences alters our qualitative conclusions, confirming the robustness of our findings to AI-text filtering.

(3) Detector Threshold and Chinese-data Calibration. We calibrated Fast-DetectGPT on Chinese data using 7,000 AI-generated texts from seven LLMs and 1,000 human-written posts. The threshold of 1.514155 was set to the median detection score of the AI-generated texts. At this threshold, the AI detection rate for LLM-generated texts was 4.46 times that for human-written posts, demonstrating discriminative ability on Chinese data. This calibration provides an empirical basis for filtering while limiting the removal of human-written content.

  1. Can topic/genre-matched comparisons establish which profile differences remain, or can the audience-design interpretation be limited to the current broadly conditioned setup?

Thank you for this suggestion. Our study focuses on audience-group-level comparisons, examining how LLM-generated content differs from human posts within the same Domain–Intent segment. Importantly, audience adaptation involves not only how content is expressed, but also what content is selected for a target audience. Therefore, our evaluation intentionally allows specific topics and genres to vary, capturing the overall content and expression choices under shared audience conditions.

We agree that our findings should not be interpreted as audience effects isolated from topical or genre variation. We will explicitly clarify this scope in the manuscript. Topic/genre-matched comparisons would offer a complementary analysis of which differences persist under finer-grained controls, which we leave for future work.

Reviewer_hWuNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper examines how LLM-generated content differs from human-created content in terms of audience-oriented creation. The authors introduce AudCraft, a dataset of approximately 1.59 million real-world social media posts, and propose two evaluation dimensions. First, audience orientation, which examines whom creators target, and second audience design, which examines how they adapt their writing to different audiences. For audience orientation, the authors find that LLMs tend to generate content for a narrower range of audiences than humans, a pattern that persists across models and prompting strategies. A competition-based simulation further suggests that these preferences may contribute to uneven content supply and increased competition with human creators. For audience design, the authors compare linguistic and stylistic features and find that although LLMs adapt their writing to different audiences, these adaptations often differ from human patterns, particularly in audience-specific vocabulary.

Critical strengths
  1. The paper introduces an interesting and relevant perspective on evaluating LLM-generated content, focusing on how models adapt their writing to different audiences. The real-world dataset, systematic comparisons across LLMs, and human evaluation are valuable contributions.

  2. I also liked the fact that the paper builds on real world social media posts, and that it evaluates multiple LLMs across different categories.

  3. It's also very positive that the authors complement their quantitative analyses with human evaluations, which help strengthen the empirical findings. It's was interesting to see that humnas show a preference for human-written content.

Critical weaknesses
  1. Although AudCraft contains approximately 1.59 million posts, the main audience-orientation experiment uses only 100 human posts and 100 generations per model. This may be insufficient to reliably characterize less frequent categories and differences in audience diversity. Could the authors (i) increase the number of annotated samples for both humans and LLMs, (ii) report a summary concentration measure with bootstrap CIs and (iii) show variability across repeated generation runs?

  2. Some posts were removed with Fast-DetectGPT. This could perhaps remove human posts that are structured or low in perplexity, which could inflate the reported human–LLM gaps. Could the authors report results with and without this filter, and state the collection period of the Qilin data?

  3. In the simulation, the two things that change when AI creators enter the platform is that they can produce content 10 times cheaper, and that they tend to target the audiences preferred by LLMs, However, the increase in content and the decrease in rewards for human creators might simply result from cheaper AI content, rather than from the concentration on particular audiences. Can the authors run an additional experiment where AI creators have lower production costs but no LLM-specific audience preference (λ = 0). This would help clarify which effects are actually caused by audience concentration.

Questions for authors
  1. Can the authors explain how the 120 audience segments and their reward values are obtained from AudCraft?

See "critical weaknesses" section for further questions.

4 comments
AuthorsOpenReview ↗

Response for Reviewer hWuN (Part 1)

Thank you for acknowledging our work and your constructive comments. Please kindly find our point-by-point responses below for your review.

  1. Although AudCraft contains approximately 1.59 million posts, the main audience-orientation experiment uses only 100 human posts and 100 generations per model. This may be insufficient to reliably characterize less frequent categories and differences in audience diversity. Could the authors (i) increase the number of annotated samples for both humans and LLMs, (ii) report a summary concentration measure with bootstrap CIs and (iii) show variability across repeated generation runs?

Thank you for these helpful suggestions. We expanded the audience-orientation evaluation from 100 to 500 posts per source, covering the human reference and all eight LLMs (4,500 annotated posts in total). We further conducted bootstrap analysis and repeated generation experiments to assess the robustness of our findings.

(i) Expanded evaluation sample. We increased the sample size fivefold, using the same Audience Domain and Audience Intent taxonomies. For each LLM, the 500 posts were obtained from five independent generation runs of 100 posts each, allowing us to evaluate both sample-size robustness and generation variability.

(ii) HHI with bootstrap confidence intervals. We report the Herfindahl–Hirschman Index (HHI, ∑kpk2\sum_k p_k^2) to quantify audience concentration, with 95% confidence intervals estimated using 10,000 bootstrap resamples. Higher HHI indicates greater concentration.

SourceDomain HHI [95% CI]Intent HHI [95% CI]
Human0.0954 [0.0920, 0.1038]0.1380 [0.1263, 0.1548]
GPT-5.4-mini0.3153 [0.3063, 0.3268]0.2789 [0.2639, 0.2977]
DeepSeek-V3.20.3674 [0.3312, 0.4089]0.2048 [0.1886, 0.2273]
Kimi-K2.50.5059 [0.4850, 0.5317]0.3415 [0.3235, 0.3634]
MiniMax-M2.50.2460 [0.2276, 0.2685]0.2142 [0.1939, 0.2405]
GLM-50.5221 [0.4710, 0.5763]0.2893 [0.2619, 0.3230]
Qwen3.5-35B0.4285 [0.3979, 0.4643]0.2771 [0.2666, 0.2908]
Qwen3.5-Plus0.4202 [0.3988, 0.4450]0.2589 [0.2371, 0.2861]
Qwen3-Max0.9149 [0.8814, 0.9489]0.7645 [0.7204, 0.8104]

The results remain consistent with the original 100-post analysis. Across all sources, the maximum absolute HHI change is only 0.0382 for Domain and 0.0103 for Intent. All eight LLMs remain more concentrated than humans in both dimensions, with non-overlapping 95% bootstrap confidence intervals.

(iii) Variability across independent generation runs. To assess generation stability, we independently generated five sets of 100 posts for each LLM and calculated HHI separately for each run. For comparison, we divided the 500 human posts into five non-overlapping subsets of 100. The table reports the minimum and maximum HHI across the five runs or subsets.

SourceDomain HHI rangeIntent HHI range
Human0.0890–0.11660.1318–0.1584
GPT-5.4-mini0.3048–0.33380.2476–0.3062
DeepSeek-V3.20.3354–0.40160.1770–0.2556
Kimi-K2.50.4706–0.52420.3262–0.3970
MiniMax-M2.50.2230–0.27740.2072–0.2248
GLM-50.4706–0.58920.2766–0.3228
Qwen3.5-35B0.4020–0.49360.2782–0.2894
Qwen3.5-Plus0.4038–0.48360.2182–0.2984
Qwen3-Max0.8862–0.94140.7146–0.8502

Despite run-to-run variability, every LLM exhibits higher audience concentration than the human reference across all five runs. Notably, even the lowest model HHI exceeds the highest human-subset HHI for both Domain (0.2230 vs. 0.1166) and Intent (0.1770 vs. 0.1584).

Together, the expanded samples, bootstrap confidence intervals, and independent generation runs consistently support our main finding that LLM-generated content exhibits substantially greater audience concentration than human-authored content in our evaluation setting.

AuthorsOpenReview ↗

Response for Reviewer hWuN (Part 2)

  1. Some posts were removed with Fast-DetectGPT. This could perhaps remove human posts that are structured or low in perplexity, which could inflate the reported human–LLM gaps. Could the authors report results with and without this filter, and state the collection period of the Qilin data?

Thank you for this concern. To assess whether Fast-DetectGPT filtering may inadvertently exaggerate human–LLM differences, we compared the key results before and after filtering.

(i) Stability of Audience Distributions. We compared the unfiltered and filtered human references, each containing 100 posts.

DistributionUnfiltered HHIFiltered HHIΔHHIMean category differenceMax category difference
Audience Domain0.10660.0955−0.01112.08 pp7.00 pp
Audience Intent0.15040.1369−0.01351.42 pp4.00 pp

Filtering leads to small changes in concentration (ΔHHI = −0.0111 and −0.0135) and mean category proportions (2.08 and 1.42 percentage points), suggesting that the overall audience distributions remain stable.

(ii) Stability of Audience Adaptation Results. We compared the three Audience Adaptation results across 55 audience groups, with 56 corresponding estimates per result (4 models × 14 metrics).

Audience Adaptation resultMean absolute changeMax absolute changeEstimate correlation
Overall-level calibration0.0400.1410.999
Cross-audience adaptation magnitude0.0210.0710.997
Audience-order alignment0.0260.1360.990

The results remain highly consistent before and after filtering, with estimate correlations of 0.990–0.999 and mean absolute changes of 0.021–0.040. Importantly, none of these changes alters our qualitative conclusions, suggesting that our findings are not driven by detector-based filtering.

(iii) Qilin Collection Date. The Qilin dataset was collected on November 27, 2024. We will explicitly include this information in the revised manuscript.

  1. In the simulation, the two things that change when AI creators enter the platform is that they can produce content 10 times cheaper, and that they tend to target the audiences preferred by LLMs, However, the increase in content and the decrease in rewards for human creators might simply result from cheaper AI content, rather than from the concentration on particular audiences. Can the authors run an additional experiment where AI creators have lower production costs but no LLM-specific audience preference (λ = 0). This would help clarify which effects are actually caused by audience concentration.

We thank the reviewer for this suggestion. To disentangle the effects of reduced production costs and audience targeting, we conducted an additional control experiment with reduced AI production costs but no audience preference (λ=0\lambda=0), and examined different targeting strengths.

(1) Cost reduction primarily drives increased production and declining human payoffs, while audience targeting further shifts content distribution. Cost reduction alone increases total production effort by 40.9% and reduces average human creator payoffs by 75.3%, while the share of content in LLM-preferred categories remains nearly unchanged (11.03% → 10.89%). Introducing audience targeting (λ=0.2\lambda=0.2) increases this share to 13.94%, demonstrating an additional effect beyond cost reduction.

(2) Audience targeting further intensifies competition within LLM-preferred categories. As λ\lambda increases from 0 to 0.2, the proportion of AI content allocated to these categories rises from 12.47% to 28.32%, while average payoff per item and human creator payoffs within these categories decrease by 22.0% and 4.6%, respectively. These results demonstrate that audience targeting independently reshapes local competition and payoff distributions.

We have incorporated these experiments into the revised manuscript to clarify the distinct effects of the two mechanisms.

  1. Can the authors explain how the 120 audience segments and their reward values are obtained from AudCraft?

Thank you for your question. We derive the 120 audience groups and their reward values from AudCraft as follows.

(1) Audience groups. We consider 12 intents × 12 domains, yielding 144 possible combinations, of which 120 are observed in AudCraft.

(2) Reward calibration. For each group, we calculate its content proportion and the mean and variance of post likes. The content proportion initializes creators' category selection probabilities, while the like statistics calibrate the group's attention reward pool. Following prior research [1], we use likes as a proxy for social attention and creator rewards, rather than directly observed monetary payoffs.

[1] A Computational Reward Learning Account of Social Media Engagement. Nature Communications, 2021.

I thank the authors for their detailed rebuttal and the effort they have put into improving the paper. My initial concerns are mostly addressed.

Regarding W3, the additional simulation is helpful and shows how the lower production costs are the primary driver of the decline in human creator reward and how the audience targeting has additional effects on specific categories. This is a distinction that I would to see in the paper and I encourage authors to include.

AuthorsOpenReview ↗

Thank You for the Further Feedback

Thank you for your positive feedback and constructive suggestions. We are pleased that our additional analyses have addressed your main concerns. We particularly appreciate your suggestion regarding W3 and will incorporate the additional simulation results into the revised manuscript, clearly distinguishing the effects of reduced production costs from those of audience targeting.

Reviewer_duMtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines which audiences LLMs generate content for, and how they adapt their expression to different audiences. They compare LM-generated content with human-generated content. They find that LMs generate for overly specific audiences and don’t vary their expression as much as humans do, e.g. wrt perplexity and sentence length, and don’t adapt to audiences as much as humans do, e.g. wrt sentence length and lexical diversity.

Critical strengths

The paper is well-motivated.

Critical weaknesses

One concern I have is that this paper might not be the best fit or get the rigorous review it deserves at ICLR, and may instead be more suitable for a different venue, perhaps one that has a human-centered track.

In section 3.1: is 100 samples enough for the results to be robust? What is the IAA between human annotators?

AI detectors vary in quality. How do I know I can trust Bao et al. 2024, the AI detector you use to filter your data?

We’re missing details in the main text on how you prompt AI to generate content in this study. This is an important part of your experiment setup. Without this information it’s difficult to judge the soundness of your work. You do say “independently generated by each LLM without specifying any target audience or topic.” Is this ecologically valid? Are people generating AI-generated content with highly underspecified prompts?

The paper is motivated by research in audience design, but it seems more like rather than measuring differences in audience, the paper is really just measuring topical differences. Generally the paper is dubious when it comes to construct validity, or at least does not do a very good job of justifying or explaining its operationalization of constructs, e.g. “tone fit, contextual usefulness, emotional resonance, nontemplate originality, and engagement potential”.

The paper only focuses on data from one social media site (Xiaohongshu), so the results around “humans” may be weak in external validity (e.g. there may be population biases).

Questions for authors

“Chinese UGC dataset” What is UGC? Maybe u should define it in line 136. Ditto for “HGC”.

4 comments
AuthorsOpenReview ↗

Response for Reviewer duMt (Part 1)

Thank you for acknowledging our work and your constructive comments. Please kindly find our point-by-point responses below for your review.

  1. This paper might not be the best fit or get the rigorous review it deserves at ICLR

We recognize the interdisciplinary nature of our work, but believe its central focus on evaluating LLM behavior in real-world content creation aligns well with ICLR's research interests.

Prior ICLR studies have examined content diversity in LLM-assisted writing [1], social intelligence evaluation [2], and human preference heterogeneity in LLM evaluation [3]. Building on this line of research, AudCraft introduces an audience-oriented evaluation framework that examines both whom LLMs create for and how they adapt their content to different audiences. Through systematic human-LLM comparisons and competition-based simulation, we further investigate the potential implications of these behavioral differences. We will clarify this positioning in the revised manuscript.

[1] Padmakumar, V., & He, H. Does Writing with Language Models Reduce Content Diversity? ICLR 2024.

[2] Zhou, X., et al. SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents. ICLR 2024.

[3] Petrova, N., et al. Unpacking Human Preference for LLMs: Demographically Aware Evaluation with the HUMAINE Framework. ICLR 2026.

  1. Is 100 samples enough for the results to be robust? What is the IAA between human annotators?

To assess robustness beyond the original 100-post samples, we expanded the evaluation to 500 LLM-annotated posts per source, covering the human reference and all 8 LLMs.

SourceDomain:Original 100 postsDomain:500 postsIntent:Original 100 postsIntent:500 posts
Human0.09550.09540.13690.1380
GPT-5.4-mini0.32300.31530.28820.2789
DeepSeek-V3.20.39410.36740.20020.2048
Kimi-K2.50.50910.50590.33250.3415
MiniMax-M2.50.21750.24600.22120.2142
GLM-50.56030.52210.29840.2893
Qwen3.5-35B0.43260.42850.27830.2771
Qwen3.5-Plus0.41810.42020.25880.2589
Qwen3-Max0.92260.91490.75420.7645

Sample-size robustness. The results remain consistent at 500 posts: all eight LLMs exhibit higher audience concentration than humans in both Domain and Intent, confirming that our main finding is not driven by the original sample size.

Inter-annotator agreement. As reported in Appendix D.2.1, three independent, source-blinded annotators achieved Fleiss’ κ of 0.91 for Domain and 0.85 for Intent, indicating high annotation agreement.

  1. How can I trust Bao et al. 2024, the AI detector you use to filter your data?

To address your concern, we (1) justify the reliability of Fast-DetectGPT, (2) verify the stability of Audience Domain and Intent distributions, and (3) assess the robustness of the three Audience Adaptation results before and after filtering. These analyses confirm that our conclusions do not rely on perfect detection accuracy.

Reliability of Fast-DetectGPT. Fast-DetectGPT (ICLR 2024) is a peer-reviewed zero-shot detector based on conditional probability curvature. It achieves an AUROC of 0.9887 across five models and 0.9338 for ChatGPT/GPT-4, with demonstrated robustness across text lengths, domains, languages, and decoding strategies. Although not error-free, it has been extensively validated across diverse settings.

Stability of the Audience distributions. We compared the unfiltered and filtered references, each containing 100 posts.

DistributionUnfiltered HHIFiltered HHIHHI changeMean absolute category differenceLargest category difference
Audience Domain0.10660.0955−0.01112.08 pp7.00 pp
Audience Intent0.15040.1369−0.01351.42 pp4.00 pp

Both distributions exhibit small HHI changes (−0.0111 and −0.0135) and mean category-level differences (2.08 and 1.42 percentage points), suggesting that filtering does not materially alter their overall concentration or composition.

Stability of the Audience Adaptation results. We compared the three Audience Adaptation results before and after filtering, covering the same 55 audience groups and 56 estimates (4 models × 14 metrics) per result.

Audience Adaptation resultMean absolute changeLargest absolute changeEstimate correlation
Overall-level calibration0.0400.1410.999
Cross-audience adaptation magnitude0.0210.0710.997
Audience-order alignment0.0260.1360.990

The estimate correlations remain exceptionally high (0.990–0.999), with small mean absolute changes (0.021–0.040). None of these differences alters the qualitative conclusions, demonstrating that our findings are robust to detector-based filtering.

AuthorsOpenReview ↗

Response for Reviewer duMt (Part 2)

  1. The main text lacks sufficient prompting details, raising concerns about experimental soundness and the ecological validity of generating content without specifying topics or target audiences.

Thank you for your concern. The complete prompt templates are provided in Appendices D.2–D.3. We agree that the key instructions should also appear in the main text and will add them in the revision.

Ecological validity. Open-ended prompting reflects a meaningful subset of real-world LLM usage. Shen (ICML 2025) analyzed approximately 2 million real user prompts and found that creative writing requests are often highly underspecified, including examples such as “please write me an inspiring story,” without specifying a topic or target audience [1]. Similarly, Xue et al. (2026) found that only 19.1% of participants used explicit prompting strategies in conversational information-seeking tasks [2].

Our RQ1 therefore examines a realistic, though not universal, generation setting: how LLMs select topics and audiences when these choices are left unspecified. This is also relevant to autonomous content-creation workflows, where LLMs may make such decisions themselves.

Supplementary experiments. To further address ecological validity, we conducted additional experiments with specified topics, examining whether our findings persist under more constrained generation settings, as detailed below.

[1] Shen, J. H. Position: Societal Impacts Research Requires Benchmarks for Creative Composition Tasks. ICML 2025.

[2] Xue, H., et al. Users’ prompting strategies and ChatGPT’s contextual adaptation shape conversational information-seeking experiences. Scientific Reports, 2026.

  1. The paper raises construct validity concerns, particularly whether it measures audience differences rather than topical differences, and lacks sufficient justification for the operationalization of key evaluation constructs.

Thank you for raising these concerns regarding construct validity. We clarify the theoretical foundations of our audience definition and the operationalization of our evaluation dimensions below.

1. Audience Definition. Prior research has characterized audiences through shared interests and consumption motivations, rather than demographics alone. Litt and Hargittai (2016) identified interest-based communities as a form of audience [1], while Gächter et al. (2024) demonstrated that audiences within the same topical domain can be segmented by distinct media consumption motivations [2]. Previous research has also used social media content to infer audience interests [3].

Following these studies, we operationalize audiences through two complementary dimensions: Audience Domain, capturing shared topical interests, and Audience Intent, reflecting underlying consumption needs (e.g., practical guidance versus emotional connection). Thus, our framework goes beyond topical classification by capturing not only what audiences are interested in, but also why they consume content. We acknowledge the overlap between audience and topic and will clarify this distinction in the revised manuscript.

2. Human-Evaluation Dimensions. As detailed in Appendix D.5, annotators evaluate posts with the target audience label and description provided. Tone fit, contextual usefulness, and emotional resonance assess audience-specific suitability, whereas non-template originality and engagement potential capture complementary aspects of content quality and perceived appeal. We will clarify these distinct roles rather than treating all five dimensions as direct measures of audience adaptation.

[1] Litt, E., & Hargittai, E. (2016). The Imagined Audience on Social Network Sites. Social Media + Society.

[2] Gächter, A., et al. (2024). A Uses and Gratifications-Based Market Segmentation Approach to Football Fans’ Online Media Usage Motivation. Communication & Sport.

[3] Henrich, V., & Lang, A. (2017). Audience Segmentation in Social Media. EACL.

AuthorsOpenReview ↗

Response for Reviewer duMt (Part 3)

  1. Focuses on data from one social media, so the results around “humans” may be weak in external validity.

Thank you for this concern. To assess cross-platform generalizability, we conducted an additional experiment and introduced an English-language corpus from Medium, a popular English long-form publishing platform, as an extra human reference and performed supplementary analyses of Audience Distributions and Audience Adaptation.

(1) Audience Distributions. We randomly sampled 1,000 Medium articles and annotated them using the same Audience Domain and Intent taxonomies. We compared their concentration (HHI) against the original Qilin reference and the average across eight LLMs.

DistributionHuman (Medium)Human (Qilin)Mean LLM
Audience Domain HHI0.12830.09550.4716
Audience Intent HHI0.17290.13690.3299

Both human datasets exhibit substantially lower audience concentration than LLMs in Domain and Intent, supporting the consistency of our main finding beyond Xiaohongshu.

(2) Audience Adaptation. We further examined cross-audience expression variability using the coefficient of variation (CV) of metric means across Domain–Intent segments, where higher CV indicates greater relative variation across audiences.

SourceSentence lengthText lengthPerplexityDistinct-1Distinct-2ModifiersAudience vocabulary
Human (Medium)0.34490.36300.28100.09440.02540.11370.4540
Human (Qilin)0.40650.27470.39870.06670.02340.17960.4132
GPT-5.4-mini0.09320.11620.16870.04140.00560.12970.5554
DeepSeek-V3.20.09990.17240.22120.04970.00730.10630.5270
Qwen3.5-Plus0.06250.07360.17550.04760.00690.09700.5290
Kimi-K2.50.20200.13320.18950.03950.00650.13300.6820

Both human references exhibit greater cross-audience variation than all four LLMs in five metrics (sentence length, text length, perplexity, Distinct-1, and Distinct-2), while showing lower variation in audience vocabulary. Although modifiers exhibit mixed patterns, these results suggest that several human–LLM expression differences persist across platforms.

We will incorporate these revisions into the manuscript to strengthen our work and expand its external validity. Thank you again for your insightful feedback.

AuthorsOpenReview ↗

Response for Reviewer duMt (Part 4)

  1. “Chinese UGC dataset” What is UGC? Maybe u should define it in line 136. Ditto for “HGC”.

Thank you for pointing this out. We will define UGC (User-Generated Content) and HGC (Human-Generated Content) at their first occurrences in the revised manuscript to improve clarity.

As the rebuttal period is drawing to a close, we would greatly appreciate it if you could let us know of any remaining questions or concerns before it ends. Thank you again for your valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.