- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper tackles diversity collapse in LLM-based social simulation, where agents differ only by persona prompts on a shared model. The proposed ReMap trains one LoRA module per source individual on their real posts, then builds new agents by assigning each depth block of the model the LoRA factors of a randomly sampled source, so that a small module library yields a much larger population. Across three domains and six backbones, the authors report that ReMap populations are closer to human text than prompted populations, contain far more distinguishable agents than the library size, and produce more diverse collective behavior in OASIS and YSocial, while remaining cheap to serve on a single GPU.
Critical strengths
- Important problem, clear motivation, and a simple, easy-to-follow method. Homogeneity of persona-prompted populations is a recognized threat to the validity of LLM social simulation. The moving individual variation into weights is a sensible, under-explored direction. Block-wise recombination of aligned LoRA factors requires no per-agent training, yields a standard adapter, and is easy to understand and plug into existing simulators.
- Some informative ablations and analyses in the experiments. The experiments cover three domains, six backbones and two external simulators. The ingredient study separates training on human text, distinctness of modules, and parametric vs in-context storage; the block-count and cross-block dependence sweeps, the context comparison, and the matching analysis with a K/N bound and Hoeffding lower bound (Appendix C) all provide useful evidence.
Critical weaknesses
1. Unfair comparison; insufficient baselines and related work . Table 1 compares ReMap only against untrained conditions, which any model fine-tuned on in-domain text should easily beat. The key fine-tuned baselines (single-source LoRA and a shared adapter on pooled source text) appear only in appendix analyses on one backbone, where they are as faithful as or more faithful than ReMap; they should be in the main table and be discussed in detail. Also, learned composition baselines (e.g., gated or routed LoRA modules, persona-conditioned shared adapters) are missing. Closely related multi-LoRA role-playing work such as Neeko and HIRPF [1-2] is not discussed, and the advantage of depth-wise hard splicing over learned same-layer composition is not explained.
2. "Distinguishable" is not the same as "a plausible new individual". Re-identifiability across sessions does not show that ReMap creates distinct, coherent individuals: it may reflect surface artifacts of block combinations, and there is no evidence that recombined agents hold consistent stances across related topics. Frequent self-repetition of ReMap agents in OASIS may also artificially inflate stance diversity and lower conformity, and the claimed tenfold expansion overstates the effective population size.
3. The "representativeness" claim conflicts with how source individuals are sampled. The sources are the 40 most prolific Reddit users, the 40 highest-volume congressional accounts, and 40 SCOTUS advocates. Highly active users are a notoriously unrepresentative slice of any platform, and 40 people cannot cover the variation of a society; recombination can only interpolate within this narrow base. The paper repeatedly claims to construct "more representative LLM populations" without measuring representativeness against any demographic or attitudinal target. The authors should either narrow this claim or show how fidelity and diversity change with larger K (e.g., 200 or 1,000 sources) and with less active users.
4. Limited insight into why block-wise recombination works. The method relies on the assumption that LoRA blocks trained independently for different individuals on the same backbone remain functionally compatible when spliced together. The paper offers no analysis of this, such as which depths carry stance versus style, how often recombination produces incoherent text, or how the choice of block boundaries interacts with model depth. The so-called "theoretical derivations" consist only of the K/N matching bound and a Hoeffding interval, which are correct but elementary. A layer-wise analysis would substantially strengthen the contribution.
[1] Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
[2] Identity-Driven Hierarchical Role-Playing Agents
Questions for authors
- What ReMap gains over single-source beyond population count? I suggest adding single-source LoRA and the shared all-source adapter as rows in Table 1 for all backbones and analyze the results.
- How do fidelity, Vendi, matching accuracy and attribution change as K grows (e.g., 40 → 500 → 1,000 sources), and when sources are sampled uniformly rather than as top-volume users?
- Do recombined agents hold coherent positions across related topics? For instance, does an agent's stance on gun policy predict its stance on related claims at a rate similar to human authors in the same corpus?
- Which depth blocks carry most of the stance signal versus stylistic signal? A swap experiment that varies one block at a time would answer this.
Response to the Authors
Thank you for the detailed response and the new experiments. Some of my concerns have been addressed, so I have accordingly raised my score. However, a few concerns remain. The new results show that the uniform recipe used in all main experiments yields low cross-topic stance consistency, and the proposed fix relies on group labels, which seems at odds with the claim that ReMap requires no explicit specification. In addition, the collapse of the learned gating baselines may stem from their setup (e.g., no per-agent gate inputs or load-balancing regularization) rather than an inherent limitation, so please clarify how these baselines were configured.