- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how AI agents on Moltbook describe humans, comparing their posts with Reddit submissions. The authors use GPT-5.6 Luna to identify stereotypes and classify their themes and polarity. Moltbook posts more often frame humans in relation to AI, place more claims in the broad “other” category, and express more negative judgments about human competence. The paper also examines how five language models respond to stereotype-bearing posts. Replies to Reddit posts introduce new claims more often, while Gemini 3.5 Flash most often endorses and amplifies the original claim.
Critical strengths
-
Transparent, partly validated pipeline. The paper provides its three prompts in full (A.6). It compares the primary annotator with three human raters, reports an independent open-weight annotator, and examines author and community concentration.
-
Careful interpretation of community engagement. The apparent engagement advantage for outsiders largely disappears in the within-author analysis, and the authors appropriately temper their initial interpretation (Sec. 4.3, Fig. 2).
-
Useful measure of claim uptake. Separating a reply’s stance toward the original proposition P from its introduction of a new proposition Q captures forms of propagation that endorsement alone would miss.
Critical weaknesses
-
Unclear coverage of the sampling frame. The pipeline excludes sentences whose only human reference is “people,” although the audit finds stereotype-positive claims in 75.3% of the sampled Moltbook exclusions and 37.7% of the Reddit exclusions (Table 7). The paper does not quantify how this exclusion affects its main cross-platform comparisons.
-
Insufficient reporting of prevalence and retrieval coverage. The reported 15.1% prevalence gap lacks an explicit denominator and a corresponding numerical result for the Matched Reddit comparison (Sec. 4.1). The random-post audit finds stereotype-positive claims in 29.7% of sampled Moltbook posts and 9.3% of sampled Reddit posts, but the paper does not explain how these samples relate to the retrieved candidates or estimate retrieval recall (Table 7).
-
Potential bias in the main dimension comparison. In 72 of 78 disagreements with the human-majority dimension label, GPT-5.6 Luna assigns other where humans assign a primary dimension (A.5). Without agreement results by platform, it is difficult to assess how much this tendency contributes to Moltbook’s larger other share (Fig. 1a).
-
Limited generalizability and uncertainty estimates. One submolt accounts for 63.34% of Moltbook’s stereotype-positive posts (Table 6). The paper also cannot distinguish agent-generated views from operator or persona prompting, and it provides no uncertainty intervals or clustered statistical tests for its main comparisons.
-
Confounding in the propagation comparison. Moltbook replies are generated under an explicit AI-agent persona, whereas Reddit replies are not (A.6.2). Thus, the cross-platform differences in New Q and rejection rates cannot be attributed specifically to the source posts. The judgments in this experiment are not validated against human labels, and generated replies do not establish how claims spread through actual comment threads.
Questions for authors
-
What is the denominator for the 15.1% “prevalence” gap in Sec. 4.1? Please report the corresponding Matched Reddit result and uncertainty intervals, and clarify the population to which the prevalence claim applies.
-
What were the sampling frames and sample sizes for the Table 7 audits? Please assess how excluding “people”-only sentences affects the main cross-platform results.
-
What is the platform composition of the human validation set? Please report agreement and dimension confusion matrices separately by platform, particularly for assignments to other (Table 13, Fig. 1a).
-
Do the Table 3 differences persist when the reply persona prompt is held constant across platforms? Was the judge used for the propagation analysis validated against human annotations?
-
Sec. 4.3 would fit the paper better if the authors explained how community affinity relates to human-directed stereotypes; otherwise, consider moving this analysis to the appendix.
5 comments
W3 / Q3. Platform-Stratified Human Validation and Dimension Disagreements
We have added platform-stratified human validation in Appendix A.5 (pp. 18–19, Tables 13–14), covering 600 Moltbook and 300 Reddit human-annotated items.
The revised analysis reports both human–human and human-majority–GPT-5.6 Luna agreement separately for each platform.
Table 4. Platform-stratified annotation agreement (agreement percentage / κ).
| Field | Moltbook Human–Human | Reddit Human–Human | Moltbook Human–Luna | Reddit Human–Luna |
|---|---|---|---|---|
| Stereotype presence | 74.54% / .523 | 78.11% / .548 | 78.28% / .560 | 77.32% / .524 |
| Primary dimension | 71.33% / .603 | 75.00% / .677 | 71.94% / .604 | 83.33% / .787 |
| Polarity | 83.40% / .718 | 85.12% / .748 | 84.35% / .725 | 88.36% / .796 |
Human–human results report mean pairwise agreement and Fleiss' κ; human-majority–Luna results report exact-match agreement and Cohen's κ. Sample sizes vary across fields because the annotation schema conditions some judgments on preceding labels.
For stereotype presence, human–human agreement is comparable across platforms (κ = .523 on Moltbook and .548 on Reddit). Human-majority–Luna agreement is also similar (κ = .560 and .524, respectively).
For primary dimension, human-majority–Luna agreement reaches κ = .604 on Moltbook and κ = .787 on Reddit.
We additionally examine dimension disagreements, particularly assignments to Other. On Moltbook, 44 cases involve Luna assigning Other while the human majority selects one of the four primary dimensions, compared with 15 cases on Reddit.
W4. Community Concentration, Generalizability, and Uncertainty
We have added community robustness analyses and clustered confidence intervals in Section 4.1 (p. 6) and Appendix A.6 (p. 20, Table 16).
Although general contributes 63.34% of stereotype-positive posts, this closely matches its 63.13% share of candidate posts. Additional checks show that its stereotype-positive rate is comparable to other communities.
Table 5. Community concentration and exclusion analysis.
| Metric | Result |
|---|---|
| general share of all non-empty Moltbook posts | 51.43% |
| general share of candidate posts | 63.13% |
| general share of stereotype-positive posts | 63.34% |
| Stereotype-positive rate in general | 74.89% |
| Stereotype-positive rate in other communities | 74.24% |
| Competence-negative rate, all Moltbook | 11.84% |
| Competence-negative rate, excluding general | 14.41% |
| Competence-negative rate, Reddit Union | 7.72% |
The similar stereotype-positive rates in general and other communities indicate that the concentration largely follows the distribution of candidate posts.
Importantly, excluding general strengthens the competence-negative contrast. The Moltbook competence-negative rate increases from 11.84% to 14.41%, compared with 7.72% on Reddit. The difference therefore increases from 4.12 to 6.69 percentage points.
We further report confidence intervals using 2,000 cluster-bootstrap replicates.
Table 6. Clustered inference for the main comparisons.
| Outcome | Resampling | Difference (pp) | 95% CI |
|---|---|---|---|
| Stereotype-positive rate, Broad | Post | +15.07 | [13.59, 16.51] |
| Stereotype-positive rate, Broad | Author | +15.07 | [12.64, 17.35] |
| Stereotype-positive rate, Strict Matched | Matched pair | +5.28 | [3.71, 6.87] |
| Competence-negative rate, Strict Matched | Matched pair | +2.32 | [1.14, 3.57] |
These results demonstrate that the main competence-negative finding persists beyond the dominant community and remains positive under clustered statistical inference.
We position Moltbook as a large-scale observational case of an emerging agent-native social environment, providing an empirical foundation for understanding human-directed representations in persistent agent discourse.
W5 / Q4. Role-Matched Propagation and Human Validation
We have addressed the role-prompt asymmetry by adopting role-matched prompting as our primary propagation setting, reported in Section 5.2 (pp. 8–9), Appendix A.4 (pp. 17–18), and Appendix A.7.2 (pp. 23–24).
We retain the same 1,000 Moltbook source posts and five responding models, replacing the original explicit AI-agent persona with the neutral participant-role instruction used for Reddit.
This enables a direct comparison of propagation outcomes under matched participant-role instructions.
Table 7. Aggregate propagation results under original and role-matched prompting.
| Metric | Moltbook Original | Moltbook Role-Matched | Reddit Role-Matched |
|---|---|---|---|
| Endorse P (%) | 28.0 | 27.6 | 27.9 |
| Reject P (%) | 7.2 | 8.5 | 11.4 |
| New Q (%) | 42.70 | 46.29 | 58.66 |
| Amplify negative P (%) | 4.12 | 2.76 | 3.80 |
| Negative share among New Q (%) | 51.57 | 51.04 | 52.61 |
The main aggregate finding persists under matched prompting. Reddit still elicits more Luna-judged new human-directed propositions than Moltbook (58.66% vs. 46.29%), a difference of 12.37 percentage points (95% CI [10.06, 14.98]).
A supplementary comparison with the original Moltbook condition also reveals prompt sensitivity. Changing the role instruction increases New Q from 42.70% to 46.29%, while reducing amplification of negative source propositions from 4.12% to 2.76%.
The negative share among newly introduced claims remains similar across the three conditions (51.57%, 51.04%, and 52.61%, respectively).
We further compare the New-Q outcomes across all five responding models.
Table 8. Model-level New-Q rates (%) across prompting conditions.
| Model | Moltbook Original | Moltbook Role-Matched | Reddit Role-Matched |
|---|---|---|---|
| GPT-5.6 Terra | 28.1 | 25.3 | 53.0 |
| DeepSeek V4.1 Flash | 40.8 | 39.2 | 55.2 |
| Gemini 3.5 Flash | 45.6 | 52.2 | 61.9 |
| GLM-5.3-Flash | 56.7 | 58.1 | 68.8 |
| Claude Sonnet 5 | 42.3 | 56.7 | 54.4 |
| All models | 42.7 | 46.3 | 58.7 |
The role-prompt effects vary across models. Claude Sonnet 5 and Gemini 3.5 Flash show larger increases in New Q under neutral prompting, while the other models show smaller or opposite changes.
Under role matching, Reddit still elicits more New Q for four of the five models. Thus, role prompting influences claim expansion but does not fully explain the aggregate cross-platform difference.
Propagation annotation reliability. We have also added a 500-item human propagation audit, balanced across Moltbook and Reddit, in Appendix A.5 (Table 15).
Table 9. Human-majority–Luna agreement in the propagation audit (agreement / Cohen's κ).
| Judgment | Overall | Moltbook | |
|---|---|---|---|
| Addresses P | 89.0% / .765 | 89.6% / .781 | 88.4% / .748 |
| Relation to P | 77.8% / .690 | 79.4% / .708 | 76.2% / .671 |
| Transformation of P | 69.7% / .572 | 68.9% / .561 | 70.6% / .584 |
The audit shows substantial agreement for identifying whether replies address the source proposition, with similar results across the two platforms.
Our propagation experiment examines one-step claim transformation through controlled reply generation: how an existing human-directed proposition may be accepted, qualified, rejected, or extended into a new proposition. This complements the observational analysis of existing platform discourse by examining what happens when such claims enter another round of model interaction.
Together, the role-matched experiments and human annotation audit provide additional support for the propagation findings.
Ethics Concerns
We have expanded the Ethics Statement (p. 10) to clarify the safeguards implemented during human annotation and provide additional context for annotator compensation.
Regarding institutional ethics review, no prior IRB or institutional ethics-board approval was obtained for this study. The research analyzes publicly available online discourse, with human participation limited to the annotation and validation of textual content.
Before annotation began, participants were shown representative examples and informed that the dataset could contain hostile or disturbing human-directed material. We also conducted face-to-face well-being assessments and arranged consultations with university psychological counselors.
During annotation, participants could skip individual examples, pause, or withdraw from the task. To reduce unnecessary exposure, we removed identified instances of human biological reductionism from the human annotation pool and limited each annotator's workload to 100 samples per day.
Regarding compensation, the three student annotators received US$10 per hour, exceeding the applicable local statutory minimum hourly wage of approximately US$3.30 (Appendix A.5, Table 13).
We thank the reviewer for the detailed feedback and for recognizing the transparency of our annotation pipeline, the careful interpretation of community engagement, and the distinction between source-claim uptake and new-claim generation. We have strengthened the revised manuscript with additional sampling diagnostics, platform-stratified validation, statistical inference, and role-matched propagation experiments. We address the concerns below.
W1 / Q2. People-Only Exclusions and Retrieval Coverage
We have clarified the retrieval procedure in Section 3.2 (pp. 3–4) and Appendix A.3 (pp. 15–16), and further examined the impact of excluding people-only references.
We exclude people-only references because people may refer to other agents in Moltbook, making human-directed attribution ambiguous. To assess this decision, we audited 300 excluded samples per platform.
Table 1. Sensitivity analysis of people-only exclusions.
Importantly, the excluded samples exhibit similar directional patterns to our main findings: competence-negative judgments are more frequent on Moltbook (26/300 vs. 6/300), while positive-polarity claims are less common among stereotype-positive cases (10.2% vs. 21.2%).
These additional checks help characterize the sensitivity of our findings to the exclusion decision. The audit provides directional evidence, while the main quantitative comparison continues to use the consistently defined retained candidate pools.
W2 / Q1. Prevalence Denominators, Matched Comparison, and Statistical Inference
We have clarified in Section 4.1 (p. 5) and Appendix A.6 (pp. 19–20) that the reported prevalence refers to stereotype-positive rates among retrieved candidate sentences, rather than all posts in the platform corpus.
Specifically, Moltbook contains 12,818 stereotype-positive instances among 17,334 candidate sentences (73.95%), compared with 3,789 among 6,435 Broad Reddit candidate sentences (58.88%), yielding a difference of 15.07 percentage points.
Under strict matching, this difference decreases to 5.28 percentage points but remains positive.
Table 2. Stereotype-positive rates and uncertainty estimates.
We additionally use 2,000 cluster-bootstrap replicates, with post-, author-, and matched-pair resampling, to account for dependence among observations. The matched comparison remains positive under paired-post resampling.
Beyond overall stereotype-positive rates, the competence-negative difference also persists under strict matching (+2.32 percentage points, 95% CI [1.14, 3.57]).
We further clarify the interpretation of the random-post audit in Appendix A.3.
Table 3. Retrieval audits (300 samples per platform for each audit).
The random-post audit measures stereotype-positive cases identified in sampled posts outside the final candidate pool. It provides a complementary view of stereotype-relevant content beyond the retrieved candidates, while the principal prevalence comparisons are calculated over the annotated candidate sentences.
Together, these results clarify the comparison denominator, quantify the Broad and Matched differences, and provide statistical support for the remaining cross-platform contrasts.