acceptodds
Under review as a conference paper at ICLR 2027

SCBO: Semantically coherent batching and ordering for LLM-based social surveys

Abstract

Large Language Models (LLMs) offer a scalable way to simulate survey respondents conditioned on demographic profiles and observed reference responses. However, the conventional one-question-per-prompt paradigm is limited in three respects: it repeatedly encodes the same context, incurring substantial token and inference costs; it restricts each target to its own narrow subset of observed responses, preventing reference evidence from being shared across targets; and it predicts every answer in isolation, preventing later predictions from leveraging information in earlier answers. To address these limitations, we predict multiple questions in a single prompt, which amortizes the shared context, allows multiple targets to share a broader pool of observed reference responses, and enables later predictions to condition on earlier ones. However, such a method faces two challenges: (1) how to form semantically coherent batches and select shared references, and (2) how to order questions and references to improve autoregressive generation. We propose Semantically Coherent Batching and Ordering (SCBO), a training-free framework that addresses these challenges through two modules, preceded by a Question Instantiation step in which an LLM extracts compact semantic representations from survey items to filter out template noise. (1) Semantic Batching and Selection: SCBO groups related questions into semantic batches and constructs a shared reference bank by combining target-specific retrieval with centroid-based completion. (2) Curriculum-based Ordering: It then applies a heuristic easy-to-hard ordering to target questions and orders references within the shared bank according to their semantic alignment with the ordered questions. Experiments on four large-scale survey datasets and four LLMs show that SCBO substantially reduces token consumption and inference time while generally improving prediction accuracy over the non-batched baseline. Code is available at https://anonymous.4open.science/r/SCBO-41D8.

Primary area:
generative models
Keywords:
large language models, social survey simulation, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jnoLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes the SCBO framework to address three limitations of the one-question-per-prompt paradigm in LLM-based social survey simulation: repeated encoding of shared context, the inability to share reference answers across questions, and independent answer prediction. SCBO batches questions through semantic clustering, constructs a shared reference bank, and orders questions and references from easy to hard, thereby reducing token costs and inference time while exploiting correlations among answers.

Critical strengths

1,The method is well targeted, addressing context amortization, reference-answer sharing, and dependencies among answers.

2,The experiments cover multiple survey datasets and LLMs, with a relatively large experimental scale.

Critical weaknesses

1, The Related Work section discusses batch prompting, semantic grouping, and Batch Permutation and Ensembling, but the experiments mainly compare SCBO with the one-question-per-prompt baseline.

2, SCBO changes question grouping, reference-bank size, reference visibility, question order, and reference order simultaneously. Therefore, even if the complete method outperforms the baseline, it is difficult to determine which factor is responsible for the gains.

3, The theoretical justification for semantic clustering and question ordering is relatively weak.

Questions for authors

How does the respondent-coverage filter affect the composition of the evaluation population, and how many respondents are excluded under each reference budget?

10 comments
AuthorsOpenReview ↗

Response for W1 (Part 1/2)

Thank you for pointing out this gap between the related-work discussion and the experimental comparisons. We have now added direct comparisons with BPE, BPE+SEAS, ADP, and OBP across four models and four survey datasets, using batch size 32 and a 1-budget setting. We report predictive performance, weighted token usage, and execution time, and will incorporate these comparisons into the revised main paper.

Empirical Advantages over General-Purpose Batch-Prompting Methods

Supplementary experimental setup. As some model versions used in the original manuscript have been updated, we conducted supplementary comparisons during the rebuttal phase using DeepSeek Flash (DeepSeek-V4.1-Flash-1017), DeepSeek Pro, MiMo-V2.6-Flash, and GPT-6.1-sol(low). The experiments cover four datasets—WVS, GSS, ANES, and BSA—with batch size 32 and a 1-budget setting, comparing SCBO against BPE, BPE+SEAS, ADP, and OBP.

Predictive performance. Using Acc over valid outputs, SCBO achieves the highest accuracy in all 16 model–dataset settings, outperforming the strongest baseline in each setting by 1.25–7.60 percentage points. For example, on WVS, DeepSeek Pro improves from 49.84% for the strongest baseline to 54.32%, while GPT-6.1-sol(low) improves from 50.58% to 58.18%. SCBO also achieves the lowest MAE and highest F1 in all 16 settings. These results demonstrate SCBO’s predictive advantages in direct comparisons with existing batch-prompting methods.

Each cell reports Acc↑ / MAE↓ / F1↑, with Acc and F1 expressed as percentages.

ModelMethodWVSGSSANESBSA
DeepSeek FlashBPE47.55 / 0.858 / 41.7346.21 / 0.725 / 41.2246.51 / 0.904 / 42.8545.92 / 0.693 / 37.96
DeepSeek FlashBPE+SEAS47.08 / 0.881 / 41.2046.46 / 0.722 / 41.6646.61 / 0.916 / 42.8845.08 / 0.720 / 37.23
DeepSeek FlashADP48.39 / 0.844 / 44.0549.44 / 0.674 / 45.3448.23 / 0.868 / 45.4048.75 / 0.663 / 43.63
DeepSeek FlashOBP47.55 / 0.841 / 42.9945.52 / 0.737 / 40.6648.49 / 0.898 / 45.1147.58 / 0.683 / 41.77
DeepSeek FlashSCBO52.25 / 0.816 / 45.3952.08 / 0.641 / 47.2853.68 / 0.819 / 50.3750.83 / 0.642 / 46.02
MiMo-v2.6-FlashBPE46.72 / 0.908 / 41.1647.22 / 0.705 / 42.1046.25 / 0.948 / 42.6447.33 / 0.689 / 40.01
MiMo-v2.6-FlashBPE+SEAS48.18 / 0.895 / 42.4247.29 / 0.717 / 41.7647.14 / 0.944 / 43.4248.08 / 0.698 / 40.80
MiMo-v2.6-FlashADP47.19 / 0.878 / 42.4747.46 / 0.703 / 43.1046.30 / 0.960 / 44.0247.75 / 0.701 / 41.83
MiMo-v2.6-FlashOBP47.92 / 0.842 / 43.4645.38 / 0.762 / 41.1446.25 / 0.941 / 43.6945.75 / 0.712 / 40.06
MiMo-v2.6-FlashSCBO51.41 / 0.804 / 48.4448.71 / 0.674 / 45.6250.42 / 0.869 / 48.3950.17 / 0.625 / 46.45
DeepSeek-V4-ProBPE49.84 / 0.811 / 44.6349.24 / 0.676 / 44.9446.30 / 0.906 / 43.2147.75 / 0.666 / 42.47
DeepSeek-V4-ProBPE+SEAS49.17 / 0.834 / 43.8749.17 / 0.684 / 44.5246.35 / 0.922 / 43.0448.92 / 0.671 / 43.64
DeepSeek-V4-ProADP49.06 / 0.816 / 44.8050.69 / 0.661 / 47.2647.29 / 0.899 / 44.7348.17 / 0.672 / 45.34
DeepSeek-V4-ProOBP48.33 / 0.817 / 44.0949.34 / 0.676 / 45.0848.07 / 0.890 / 45.5549.17 / 0.649 / 45.79
DeepSeek-V4-ProSCBO54.32 / 0.786 / 52.2352.47 / 0.637 / 49.8653.49 / 0.825 / 51.0252.50 / 0.597 / 50.47
GPT-6.1-sol(low)BPE49.43 / 0.826 / 44.7250.00 / 0.669 / 45.4149.17 / 0.870 / 46.2848.83 / 0.677 / 43.63
GPT-6.1-sol(low)BPE+SEAS49.84 / 0.814 / 44.2950.66 / 0.646 / 45.8948.75 / 0.866 / 45.4148.58 / 0.679 / 42.71
GPT-6.1-sol(low)ADP50.58 / 0.788 / 45.8048.34 / 0.715 / 45.0649.75 / 0.867 / 47.2650.71 / 0.635 / 46.54
GPT-6.1-sol(low)OBP50.52 / 0.774 / 47.5851.78 / 0.650 / 49.3849.63 / 0.899 / 47.3752.52 / 0.627 / 49.97
GPT-6.1-sol(low)SCBO58.18 / 0.716 / 53.8657.68 / 0.589 / 53.3456.51 / 0.788 / 53.3655.92 / 0.572 / 54.11

Computational usage advantages. SCBO achieves higher accuracy while consuming fewer weighted tokens. For example, with GPT-6.1-sol(low) on WVS at batch size 32, SCBO achieves an Acc of 58.18%, compared with 49.43% for BPE, while reducing weighted token usage by approximately 75%. Across comparisons with complete usage records, SCBO reduces weighted token usage by 5.1%–86.9% relative to the baselines. This metric is defined as input tokens + 4 × output tokens to compare input and output overhead.

AuthorsOpenReview ↗

Response for W1 (Part 2/2)

Execution time advantages. SCBO has a shorter recorded execution time than every baseline in all 16 model–dataset settings. For example, with GPT-6.1-sol(low) on WVS at batch size 32, SCBO takes 127.2 seconds, compared with 586.7 seconds for BPE and 712.6 seconds for BPE+SEAS. SCBO times on WVS/GSS include amortized preprocessing overhead, whereas recorded times on ANES/BSA exclude offline preprocessing. These results show that, under the evaluated workloads, SCBO achieves higher predictive accuracy with lower computational usage and shorter execution time.

Batch size = 32, 1-budget. Each cell reports weighted tokens (M) / execution time (seconds), where weighted tokens = input tokens + 4 × output tokens. Lower is better.

ModelMethodWVSGSSANESBSA
DeepSeek FlashBPE25.292 / 78.817.362 / 107.623.462 / 73.713.534 / 44.6
DeepSeek FlashBPE+SEAS18.175 / 59.612.664 / 70.816.807 / 56.09.427 / 33.0
DeepSeek FlashADP13.894 / 88.88.730 / 59.412.539 / 77.67.168 / 43.4
DeepSeek FlashOBP7.559 / 101.65.222 / 78.57.812 / 90.54.392 / 57.7
DeepSeek FlashSCBO6.935 / 23.54.823 / 40.26.777 / 22.34.018 / 11.4
MiMo-v2.6-FlashBPE24.900 / 383.016.358 / 420.423.531 / 508.013.429 / 289.9
MiMo-v2.6-FlashBPE+SEAS17.764 / 280.112.516 / 251.617.087 / 324.19.594 / 181.4
MiMo-v2.6-FlashADP13.359 / 453.18.433 / 355.611.959 / 465.76.928 / 256.4
MiMo-v2.6-FlashOBP7.997 / 738.75.248 / 312.17.649 / 398.64.631 / 270.4
MiMo-v2.6-FlashSCBO7.149 / 100.54.824 / 90.47.259 / 116.84.055 / 73.6
DeepSeek-V4-ProBPE25.289 / 169.917.137 / 165.823.303 / 121.813.532 / 74.2
DeepSeek-V4-ProBPE+SEAS17.727 / 117.011.767 / 101.816.155 / 91.59.292 / 58.1
DeepSeek-V4-ProADP13.893 / 238.28.732 / 155.312.538 / 144.17.167 / 80.3
DeepSeek-V4-ProOBP7.922 / 270.25.488 / 198.37.956 / 182.84.574 / 105.7
DeepSeek-V4-ProSCBO7.070 / 47.95.151 / 55.96.836 / 38.64.136 / 26.7
GPT-6.1-sol(low)BPE35.903 / 586.731.165 / 614.236.610 / 819.322.036 / 370.4
GPT-6.1-sol(low)BPE+SEAS36.676 / 712.636.412 / 695.236.786 / 1143.220.044 / 456.6
GPT-6.1-sol(low)ADP21.878 / 630.816.171 / 489.820.127 / 420.810.440 / 311.6
GPT-6.1-sol(low)OBP30.018 / 583.829.271 / 493.277.074 / 2080.524.281 / 1023.0
GPT-6.1-sol(low)SCBO8.999 / 127.27.197 / 163.110.062 / 218.45.676 / 109.0

Methodological Distinctions from General-Purpose Batch-Prompting Methods

  • Compared with BPE/SEAS: BPE mitigates sensitivity to ordering through multiple rounds of random permutations and ensembling, while SEAS reduces the number of ensemble rounds. SCBO retains target-specific relevant history, constructs a shared and deduplicated reference bank, and applies deterministic easy-to-hard question ordering and reference alignment. Under normal operation, SCBO requires only one call per batch, reducing the overhead of multi-round ensembling.

  • Compared with ADP: ADP uses previously generated question–answer pairs within a batch as demonstrations for subsequent predictions, requiring questions to be repeated in the output. SCBO instead retains target-specific relevant history, constructs a shared and deduplicated reference bank, orders questions according to historical support, and semantically aligns references with the question sequence. Both methods can typically operate with one call per batch. However, ADP’s question repetition increases output tokens, which are typically priced higher than input tokens, thereby increasing inference costs.

  • Compared with OBP: Beyond question grouping and demonstration selection, SCBO addresses survey-template noise, target-specific associations with personal history, and within-batch ordering. Question instantiation extracts topics, intents, and entities to support grouping and retrieval, while question ordering and reference alignment organize how evidence is used. Furthermore, in our adaptation to survey prediction, OBP leaves some target questions out during grouping, requiring additional single-question fallback calls and increasing both the number of calls and execution overhead.

AuthorsOpenReview ↗

Response for W2 (Part 1/2)

Thank you for highlighting this concern. We have supplemented the evaluation with an analysis of reference-bank size, a factorial experiment on question and reference ordering, separate ablations of batching and history selection, and prefix-answer interventions to better distinguish the roles of individual components.

(1) A larger shared reference bank does not guarantee higher accuracy

In the original manuscript, the shared reference-bank budget is defined as:

M=nC,M=nC,

where nn is the per-question reference budget and CC is the batch size. For example, with Qwen3.7-Max under 1-budget, increasing the reference-bank budget from 12 to 96 references decreases smoothed Acc from 54.43% to 52.66%. Under 3-budget, increasing the budget from 12 to 144 references decreases smoothed Acc from 55.42% to 54.01%. These results show that expanding the shared reference bank does not necessarily improve prediction, and SCBO's gains cannot simply be attributed to a larger reference bank.

(2) Separating question ordering and reference ordering

We evaluate all six combinations of three question orders and two reference orders, keeping batch membership, reference content, and reference counts unchanged. The fixed reference order is inherited from full SCBO and is not realigned when the question order changes, allowing the two ordering factors to be examined separately.

Easy-to-hard ordering with fixed references achieves the highest Acc on all three datasets. With easy-to-hard question ordering held fixed, randomizing reference order decreases Acc by 1.99, 1.00, and 1.20 percentage points on WVS, GSS, and ANES, respectively. With reference order held fixed, switching to hard-to-easy ordering (inverse curriculum) decreases Acc by 1.05, 1.18, and 1.96 percentage points, respectively. These results indicate that both question ordering and reference ordering affect predictive performance.

DeepSeek Flash, batch size = 32; Acc (%).

Question orderReference orderWVSGSSANES
Easy-to-hardFixed (full SCBO)52.2552.0853.68
Easy-to-hardRandom50.2651.0852.48
Hard-to-easy (inverse curriculum)Fixed51.2050.9051.72
Hard-to-easy (inverse curriculum)Random50.5250.9751.93
RandomFixed50.3650.4952.01
RandomRandom51.2549.9052.14
Reviewer_NMYfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that predicting survey answers one question per time wastes shared context and ignores how a respondent’s answers connect. It introduces SCBO, a training-free framework that groups related questions, gives each group a shared bank of the respondent’s observed answers, and orders questions from easiest to hardest. Each prediction can then build on the ones before it, with fewer prompts overall. The paper compares with a non batch baseline and conducts ablation studies to show the effectiveness of the method.

Critical strengths
  • Computational (token) efficiency is an important problem; the paper provides a working heuristic that can effectively reduce the token and time to simulate survey respondents, verified against multiple datasets and models;
  • Ablation studies also show that each component, Instantiation, Semantic, Ordering is useful.
Critical weaknesses
  • My major concern is the experiment results are not rigorous enough to support the claims
    • First, the baselines are very weak, only one non batch baseline, the ablations are not proper baselines, not even one baseline from other batch prompting works is compared to, e.g. Batch Permutation and Ensembling mentioned in the Appendix.
    • Second, the ablations are very naive and not robust. The only ablation is to remove a component but not to ablate it, e.g. in ordering part, prior works teach us inverse curriculum is also potentially helpful, and sometimes more helpful than curriculum.
    • Third, why cant we just input the whole survey as one input without batching? The model LLMs’ context window should be enough, but correct me if I’m wrong.
    • Overall, the experiments prove this is a working method, but not a method that works better than its neighbors.
  • Why related works are not discussed in the main paper, only in the Appendix? It is misleading to describe challenges in such a way, and it feels like batch prompting is a contribution of the paper, while it is already well studied. Many related batch prompting works are not discussed, e.g., Auto-Demo Prompting (https://arxiv.org/abs/2410.01724), Optimized Batch Prompting (https://www.vldb.org/pvldb/vol18/p2172-zhang.pdf)
Questions for authors

See weaknesses.

10 comments
AuthorsOpenReview ↗

Response for W1 & W4 (Part 1/3)

Thank you for highlighting the need for stronger baselines. We have added direct comparisons with BPE, BPE+SEAS, ADP, and OBP across four models and four survey datasets under matched batch-size and reference-budget settings. These experiments assess predictive performance, weighted token usage, and execution time, providing evidence of SCBO’s advantages over existing batch-prompting methods. Below, we clarify the methodological differences and present the supplementary results.

Methodological Distinctions from General-Purpose Batch-Prompting Methods

  • Compared with BPE/SEAS: BPE mitigates sensitivity to ordering through multiple rounds of random permutations and ensembling, while SEAS reduces the number of ensemble rounds. SCBO retains target-specific relevant history, constructs a shared and deduplicated reference bank, and applies deterministic easy-to-hard question ordering and reference alignment. Under normal operation, SCBO requires only one call per batch, reducing the overhead of multi-round ensembling.

  • Compared with ADP: ADP uses previously generated question–answer pairs within a batch as demonstrations for subsequent predictions, requiring questions to be repeated in the output. SCBO instead retains target-specific relevant history, constructs a shared and deduplicated reference bank, orders questions according to historical support, and semantically aligns references with the question sequence. Both methods can typically operate with one call per batch. However, ADP’s question repetition increases output tokens, which are typically priced higher than input tokens, thereby increasing inference costs.

  • Compared with OBP: Beyond question grouping and demonstration selection, SCBO addresses survey-template noise, target-specific associations with personal history, and within-batch ordering. Question instantiation extracts topics, intents, and entities to support grouping and retrieval, while question ordering and reference alignment organize how evidence is used. Furthermore, in our adaptation to survey prediction, OBP leaves some target questions out during grouping, requiring additional single-question fallback calls and increasing both the number of calls and execution overhead.

AuthorsOpenReview ↗

Response for W1 & W4 (Part 2/3)

Empirical Advantages over General-Purpose Batch-Prompting Methods

Supplementary experimental setup. As some model versions used in the original manuscript have been updated, we conducted supplementary comparisons during the rebuttal phase using DeepSeek Flash (DeepSeek-V4.1-Flash-1017), DeepSeek Pro, MiMo-V2.6-Flash, and GPT-6.1-sol(low). The experiments cover four datasets—WVS, GSS, ANES, and BSA—with batch size 32 and a 1-budget setting, comparing SCBO against BPE, BPE+SEAS, ADP, and OBP.

Predictive performance. Using Acc over valid outputs, SCBO achieves the highest accuracy in all 16 model–dataset settings, outperforming the strongest baseline in each setting by 1.25–7.60 percentage points. For example, on WVS, DeepSeek Pro improves from 49.84% for the strongest baseline to 54.32%, while GPT-6.1-sol(low) improves from 50.58% to 58.18%. SCBO also achieves the lowest MAE and highest F1 in all 16 settings. These results demonstrate SCBO’s predictive advantages in direct comparisons with existing batch-prompting methods.

Each cell reports Acc↑ / MAE↓ / F1↑, with Acc and F1 expressed as percentages.

ModelMethodWVSGSSANESBSA
DeepSeek FlashBPE47.55 / 0.858 / 41.7346.21 / 0.725 / 41.2246.51 / 0.904 / 42.8545.92 / 0.693 / 37.96
DeepSeek FlashBPE+SEAS47.08 / 0.881 / 41.2046.46 / 0.722 / 41.6646.61 / 0.916 / 42.8845.08 / 0.720 / 37.23
DeepSeek FlashADP48.39 / 0.844 / 44.0549.44 / 0.674 / 45.3448.23 / 0.868 / 45.4048.75 / 0.663 / 43.63
DeepSeek FlashOBP47.55 / 0.841 / 42.9945.52 / 0.737 / 40.6648.49 / 0.898 / 45.1147.58 / 0.683 / 41.77
DeepSeek FlashSCBO52.25 / 0.816 / 45.3952.08 / 0.641 / 47.2853.68 / 0.819 / 50.3750.83 / 0.642 / 46.02
MiMo-v2.6-FlashBPE46.72 / 0.908 / 41.1647.22 / 0.705 / 42.1046.25 / 0.948 / 42.6447.33 / 0.689 / 40.01
MiMo-v2.6-FlashBPE+SEAS48.18 / 0.895 / 42.4247.29 / 0.717 / 41.7647.14 / 0.944 / 43.4248.08 / 0.698 / 40.80
MiMo-v2.6-FlashADP47.19 / 0.878 / 42.4747.46 / 0.703 / 43.1046.30 / 0.960 / 44.0247.75 / 0.701 / 41.83
MiMo-v2.6-FlashOBP47.92 / 0.842 / 43.4645.38 / 0.762 / 41.1446.25 / 0.941 / 43.6945.75 / 0.712 / 40.06
MiMo-v2.6-FlashSCBO51.41 / 0.804 / 48.4448.71 / 0.674 / 45.6250.42 / 0.869 / 48.3950.17 / 0.625 / 46.45
DeepSeek-V4-ProBPE49.84 / 0.811 / 44.6349.24 / 0.676 / 44.9446.30 / 0.906 / 43.2147.75 / 0.666 / 42.47
DeepSeek-V4-ProBPE+SEAS49.17 / 0.834 / 43.8749.17 / 0.684 / 44.5246.35 / 0.922 / 43.0448.92 / 0.671 / 43.64
DeepSeek-V4-ProADP49.06 / 0.816 / 44.8050.69 / 0.661 / 47.2647.29 / 0.899 / 44.7348.17 / 0.672 / 45.34
DeepSeek-V4-ProOBP48.33 / 0.817 / 44.0949.34 / 0.676 / 45.0848.07 / 0.890 / 45.5549.17 / 0.649 / 45.79
DeepSeek-V4-ProSCBO54.32 / 0.786 / 52.2352.47 / 0.637 / 49.8653.49 / 0.825 / 51.0252.50 / 0.597 / 50.47
GPT-6.1-sol(low)BPE49.43 / 0.826 / 44.7250.00 / 0.669 / 45.4149.17 / 0.870 / 46.2848.83 / 0.677 / 43.63
GPT-6.1-sol(low)BPE+SEAS49.84 / 0.814 / 44.2950.66 / 0.646 / 45.8948.75 / 0.866 / 45.4148.58 / 0.679 / 42.71
GPT-6.1-sol(low)ADP50.58 / 0.788 / 45.8048.34 / 0.715 / 45.0649.75 / 0.867 / 47.2650.71 / 0.635 / 46.54
GPT-6.1-sol(low)OBP50.52 / 0.774 / 47.5851.78 / 0.650 / 49.3849.63 / 0.899 / 47.3752.52 / 0.627 / 49.97
GPT-6.1-sol(low)SCBO58.18 / 0.716 / 53.8657.68 / 0.589 / 53.3456.51 / 0.788 / 53.3655.92 / 0.572 / 54.11
AuthorsOpenReview ↗

Response for W1 & W4 (Part 3/3)

Computational usage advantages. SCBO achieves higher accuracy while consuming fewer weighted tokens. For example, with GPT-6.1-sol(low) on WVS at batch size 32, SCBO achieves an Acc of 58.18%, compared with 49.43% for BPE, while reducing weighted token usage by approximately 75%. Across comparisons with complete usage records, SCBO reduces weighted token usage by 5.1%–86.9% relative to the baselines. This metric is defined as input tokens + 4 × output tokens to compare input and output overhead.

Execution time advantages. SCBO has a shorter recorded execution time than every baseline in all 16 model–dataset settings. For example, with GPT-6.1-sol(low) on WVS at batch size 32, SCBO takes 127.2 seconds, compared with 586.7 seconds for BPE and 712.6 seconds for BPE+SEAS. SCBO times on WVS/GSS include amortized preprocessing overhead, whereas recorded times on ANES/BSA exclude offline preprocessing. These results show that, under the evaluated workloads, SCBO achieves higher predictive accuracy with lower computational usage and shorter execution time.

Batch size = 32, 1-budget. Each cell reports weighted tokens (M) / execution time (seconds), where weighted tokens = input tokens + 4 × output tokens. Lower is better.

ModelMethodWVSGSSANESBSA
DeepSeek FlashBPE25.292 / 78.817.362 / 107.623.462 / 73.713.534 / 44.6
DeepSeek FlashBPE+SEAS18.175 / 59.612.664 / 70.816.807 / 56.09.427 / 33.0
DeepSeek FlashADP13.894 / 88.88.730 / 59.412.539 / 77.67.168 / 43.4
DeepSeek FlashOBP7.559 / 101.65.222 / 78.57.812 / 90.54.392 / 57.7
DeepSeek FlashSCBO6.935 / 23.54.823 / 40.26.777 / 22.34.018 / 11.4
MiMo-v2.6-FlashBPE24.900 / 383.016.358 / 420.423.531 / 508.013.429 / 289.9
MiMo-v2.6-FlashBPE+SEAS17.764 / 280.112.516 / 251.617.087 / 324.19.594 / 181.4
MiMo-v2.6-FlashADP13.359 / 453.18.433 / 355.611.959 / 465.76.928 / 256.4
MiMo-v2.6-FlashOBP7.997 / 738.75.248 / 312.17.649 / 398.64.631 / 270.4
MiMo-v2.6-FlashSCBO7.149 / 100.54.824 / 90.47.259 / 116.84.055 / 73.6
DeepSeek-V4-ProBPE25.289 / 169.917.137 / 165.823.303 / 121.813.532 / 74.2
DeepSeek-V4-ProBPE+SEAS17.727 / 117.011.767 / 101.816.155 / 91.59.292 / 58.1
DeepSeek-V4-ProADP13.893 / 238.28.732 / 155.312.538 / 144.17.167 / 80.3
DeepSeek-V4-ProOBP7.922 / 270.25.488 / 198.37.956 / 182.84.574 / 105.7
DeepSeek-V4-ProSCBO7.070 / 47.95.151 / 55.96.836 / 38.64.136 / 26.7
GPT-6.1-sol(low)BPE35.903 / 586.731.165 / 614.236.610 / 819.322.036 / 370.4
GPT-6.1-sol(low)BPE+SEAS36.676 / 712.636.412 / 695.236.786 / 1143.220.044 / 456.6
GPT-6.1-sol(low)ADP21.878 / 630.816.171 / 489.820.127 / 420.810.440 / 311.6
GPT-6.1-sol(low)OBP30.018 / 583.829.271 / 493.277.074 / 2080.524.281 / 1023.0
GPT-6.1-sol(low)SCBO8.999 / 127.27.197 / 163.110.062 / 218.45.676 / 109.0
Reviewer_rPjxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SCBO, a training-free framework for improving the accuracy and efficiency of LLM-based social survey prediction. It combines semantic question batching, shared reference selection, and curriculum-based ordering to reduce redundant computation and exploit dependencies among survey responses. Experiments on four datasets and four LLMs demonstrate substantial reductions in token consumption and inference time, alongside accuracy improvements in most settings. Ablation studies further support the effectiveness of the proposed components.

Critical strengths
  1. The paper addresses a practical challenge in LLM-based survey prediction, with a clear motivation to reduce redundant computation and improve reference utilization.
  2. Experiments across four datasets, four LLMs, and multiple reference budgets demonstrate consistent efficiency gains and generally improved prediction accuracy.
  3. The training-free framework is easy to integrate into existing LLM pipelines. Its modular design and ablation studies also support its practicality and reproducibility.
Critical weaknesses
  1. The methodological novelty is relatively limited, as the core components largely rely on established clustering, retrieval, and ordering techniques. The distinction from existing batch-prompting methods remains insufficiently demonstrated.
  2. The ablation studies do not adequately isolate the contributions of question ordering, reference ordering, and shared reference selection, limiting the interpretation of individual components.
  3. The efficiency evaluation excludes preprocessing costs and does not report complete end-to-end execution time, potentially overstating the practical computational advantages.
  4. The experimental results lack statistical uncertainty estimates, while the respondent filtering criteria may limit the representativeness and generalizability of the findings.
Questions for authors
  1. Could the authors separately evaluate question ordering and reference ordering, and examine whether incorrect early predictions affect subsequent answers?
  2. Could the authors report total computational costs and end-to-end inference time, including preprocessing overhead?
  3. Could the authors provide confidence intervals and analyze the effects of respondent filtering and demographic differences on prediction performance?
8 comments
AuthorsOpenReview ↗

Response for W1 (Part 1/3)

Thank you for raising this concern. Below, we explain the task-specific requirements of social survey simulation, clarify the methodological distinctions between SCBO and existing batch-prompting methods, and present supplementary experiments comparing predictive performance and efficiency.

Distinctive Requirements of Social Survey Simulation

In typical few-shot NLP tasks, prediction can be expressed as:

\xrightarrow{\mathrm{LLM}} \text{Task answer}.$$ Social survey simulation, by contrast, is an **individual-conditioned prediction task**: $$\underbrace{\text{Persona}+\text{Personal response history}+\text{Target question}}_{\text{Evidence of individual characteristics and preferences}} \xrightarrow{\mathrm{LLM}} \text{Predicted response of the respondent}.$$ Here, historical question–answer pairs serve not only as task demonstrations but also as evidence of the respondent’s **personal preferences, experiences, and response patterns**. The objective is to approximate that respondent’s actual answer. **The same historical response may provide different levels of support for different target questions, and different respondents may require different historical evidence for the same question.** Therefore, social survey simulation requires careful selection and within-batch sharing of personal history, together with explicit associations between historical evidence and individual target questions. To address these needs, SCBO uses **clustering, retrieval, and ordering techniques** to group related target questions, select and share personal history, and organize the sequence of questions and reference examples, improving the utilization of historical evidence. ## Methodological Distinctions from General-Purpose Batch-Prompting Methods - **Compared with BPE/SEAS:** BPE mitigates sensitivity to ordering through multiple rounds of random permutations and ensembling, while SEAS reduces the number of ensemble rounds. SCBO retains target-specific relevant history, constructs a shared and deduplicated reference bank, and applies deterministic easy-to-hard question ordering and reference alignment. **Under normal operation, SCBO requires only one call per batch**, reducing the overhead of multi-round ensembling. - **Compared with ADP:** ADP uses previously generated question–answer pairs within a batch as demonstrations for subsequent predictions, requiring questions to be repeated in the output. SCBO instead retains target-specific relevant history, constructs a shared and deduplicated reference bank, orders questions according to historical support, and semantically aligns references with the question sequence. Both methods can typically operate with one call per batch. However, **ADP’s question repetition increases output tokens, which are typically priced higher than input tokens, thereby increasing inference costs.** - **Compared with OBP:** Beyond question grouping and demonstration selection, SCBO addresses survey-template noise, target-specific associations with personal history, and within-batch ordering. Question instantiation extracts topics, intents, and entities to support grouping and retrieval, while question ordering and reference alignment organize how evidence is used. **Furthermore, in our adaptation to survey prediction, OBP leaves some target questions out during grouping, requiring additional single-question fallback calls and increasing both the number of calls and execution overhead.**
AuthorsOpenReview ↗

Response for W1 (Part 2/3)

Empirical Advantages over General-Purpose Batch-Prompting Methods

Supplementary experimental setup. As some model versions used in the original manuscript have been updated, we conducted supplementary comparisons during the rebuttal phase using DeepSeek Flash (DeepSeek-V4.1-Flash-1017), DeepSeek Pro, MiMo-V2.6-Flash, and GPT-6.1-sol(low). The experiments cover four datasets—WVS, GSS, ANES, and BSA—with batch size 32 and a 1-budget setting, comparing SCBO against BPE, BPE+SEAS, ADP, and OBP.

Predictive performance. Using Acc over valid outputs, SCBO achieves the highest accuracy in all 16 model–dataset settings, outperforming the strongest baseline in each setting by 1.25–7.60 percentage points. For example, on WVS, DeepSeek Pro improves from 49.84% for the strongest baseline to 54.32%, while GPT-6.1-sol(low) improves from 50.58% to 58.18%. SCBO also achieves the lowest MAE and highest F1 in all 16 settings. These results demonstrate SCBO’s predictive advantages in direct comparisons with existing batch-prompting methods.

Each cell reports Acc↑ / MAE↓ / F1↑, with Acc and F1 expressed as percentages.

ModelMethodWVSGSSANESBSA
DeepSeek FlashBPE47.55 / 0.858 / 41.7346.21 / 0.725 / 41.2246.51 / 0.904 / 42.8545.92 / 0.693 / 37.96
DeepSeek FlashBPE+SEAS47.08 / 0.881 / 41.2046.46 / 0.722 / 41.6646.61 / 0.916 / 42.8845.08 / 0.720 / 37.23
DeepSeek FlashADP48.39 / 0.844 / 44.0549.44 / 0.674 / 45.3448.23 / 0.868 / 45.4048.75 / 0.663 / 43.63
DeepSeek FlashOBP47.55 / 0.841 / 42.9945.52 / 0.737 / 40.6648.49 / 0.898 / 45.1147.58 / 0.683 / 41.77
DeepSeek FlashSCBO52.25 / 0.816 / 45.3952.08 / 0.641 / 47.2853.68 / 0.819 / 50.3750.83 / 0.642 / 46.02
MiMo-v2.6-FlashBPE46.72 / 0.908 / 41.1647.22 / 0.705 / 42.1046.25 / 0.948 / 42.6447.33 / 0.689 / 40.01
MiMo-v2.6-FlashBPE+SEAS48.18 / 0.895 / 42.4247.29 / 0.717 / 41.7647.14 / 0.944 / 43.4248.08 / 0.698 / 40.80
MiMo-v2.6-FlashADP47.19 / 0.878 / 42.4747.46 / 0.703 / 43.1046.30 / 0.960 / 44.0247.75 / 0.701 / 41.83
MiMo-v2.6-FlashOBP47.92 / 0.842 / 43.4645.38 / 0.762 / 41.1446.25 / 0.941 / 43.6945.75 / 0.712 / 40.06
MiMo-v2.6-FlashSCBO51.41 / 0.804 / 48.4448.71 / 0.674 / 45.6250.42 / 0.869 / 48.3950.17 / 0.625 / 46.45
DeepSeek-V4-ProBPE49.84 / 0.811 / 44.6349.24 / 0.676 / 44.9446.30 / 0.906 / 43.2147.75 / 0.666 / 42.47
DeepSeek-V4-ProBPE+SEAS49.17 / 0.834 / 43.8749.17 / 0.684 / 44.5246.35 / 0.922 / 43.0448.92 / 0.671 / 43.64
DeepSeek-V4-ProADP49.06 / 0.816 / 44.8050.69 / 0.661 / 47.2647.29 / 0.899 / 44.7348.17 / 0.672 / 45.34
DeepSeek-V4-ProOBP48.33 / 0.817 / 44.0949.34 / 0.676 / 45.0848.07 / 0.890 / 45.5549.17 / 0.649 / 45.79
DeepSeek-V4-ProSCBO54.32 / 0.786 / 52.2352.47 / 0.637 / 49.8653.49 / 0.825 / 51.0252.50 / 0.597 / 50.47
GPT-6.1-sol(low)BPE49.43 / 0.826 / 44.7250.00 / 0.669 / 45.4149.17 / 0.870 / 46.2848.83 / 0.677 / 43.63
GPT-6.1-sol(low)BPE+SEAS49.84 / 0.814 / 44.2950.66 / 0.646 / 45.8948.75 / 0.866 / 45.4148.58 / 0.679 / 42.71
GPT-6.1-sol(low)ADP50.58 / 0.788 / 45.8048.34 / 0.715 / 45.0649.75 / 0.867 / 47.2650.71 / 0.635 / 46.54
GPT-6.1-sol(low)OBP50.52 / 0.774 / 47.5851.78 / 0.650 / 49.3849.63 / 0.899 / 47.3752.52 / 0.627 / 49.97
GPT-6.1-sol(low)SCBO58.18 / 0.716 / 53.8657.68 / 0.589 / 53.3456.51 / 0.788 / 53.3655.92 / 0.572 / 54.11
AuthorsOpenReview ↗

Response for W1 (Part 3/3)

Computational usage advantages. SCBO achieves higher accuracy while consuming fewer weighted tokens. For example, with GPT-6.1-sol(low) on WVS at batch size 32, SCBO achieves an Acc of 58.18%, compared with 49.43% for BPE, while reducing weighted token usage by approximately 75%. Across comparisons with complete usage records, SCBO reduces weighted token usage by 5.1%–86.9% relative to the baselines. This metric is defined as input tokens + 4 × output tokens to compare input and output overhead.

Execution time advantages. SCBO has a shorter recorded execution time than every baseline in all 16 model–dataset settings. For example, with GPT-6.1-sol(low) on WVS at batch size 32, SCBO takes 127.2 seconds, compared with 586.7 seconds for BPE and 712.6 seconds for BPE+SEAS. SCBO times on WVS/GSS include amortized preprocessing overhead, whereas recorded times on ANES/BSA exclude offline preprocessing. These results show that, under the evaluated workloads, SCBO achieves higher predictive accuracy with lower computational usage and shorter execution time.

Batch size = 32, 1-budget. Each cell reports weighted tokens (M) / execution time (seconds), where weighted tokens = input tokens + 4 × output tokens. Lower is better.

ModelMethodWVSGSSANESBSA
DeepSeek FlashBPE25.292 / 78.817.362 / 107.623.462 / 73.713.534 / 44.6
DeepSeek FlashBPE+SEAS18.175 / 59.612.664 / 70.816.807 / 56.09.427 / 33.0
DeepSeek FlashADP13.894 / 88.88.730 / 59.412.539 / 77.67.168 / 43.4
DeepSeek FlashOBP7.559 / 101.65.222 / 78.57.812 / 90.54.392 / 57.7
DeepSeek FlashSCBO6.935 / 23.54.823 / 40.26.777 / 22.34.018 / 11.4
MiMo-v2.6-FlashBPE24.900 / 383.016.358 / 420.423.531 / 508.013.429 / 289.9
MiMo-v2.6-FlashBPE+SEAS17.764 / 280.112.516 / 251.617.087 / 324.19.594 / 181.4
MiMo-v2.6-FlashADP13.359 / 453.18.433 / 355.611.959 / 465.76.928 / 256.4
MiMo-v2.6-FlashOBP7.997 / 738.75.248 / 312.17.649 / 398.64.631 / 270.4
MiMo-v2.6-FlashSCBO7.149 / 100.54.824 / 90.47.259 / 116.84.055 / 73.6
DeepSeek-V4-ProBPE25.289 / 169.917.137 / 165.823.303 / 121.813.532 / 74.2
DeepSeek-V4-ProBPE+SEAS17.727 / 117.011.767 / 101.816.155 / 91.59.292 / 58.1
DeepSeek-V4-ProADP13.893 / 238.28.732 / 155.312.538 / 144.17.167 / 80.3
DeepSeek-V4-ProOBP7.922 / 270.25.488 / 198.37.956 / 182.84.574 / 105.7
DeepSeek-V4-ProSCBO7.070 / 47.95.151 / 55.96.836 / 38.64.136 / 26.7
GPT-6.1-sol(low)BPE35.903 / 586.731.165 / 614.236.610 / 819.322.036 / 370.4
GPT-6.1-sol(low)BPE+SEAS36.676 / 712.636.412 / 695.236.786 / 1143.220.044 / 456.6
GPT-6.1-sol(low)ADP21.878 / 630.816.171 / 489.820.127 / 420.810.440 / 311.6
GPT-6.1-sol(low)OBP30.018 / 583.829.271 / 493.277.074 / 2080.524.281 / 1023.0
GPT-6.1-sol(low)SCBO8.999 / 127.27.197 / 163.110.062 / 218.45.676 / 109.0

Comments on the paper

4 comments
AuthorsOpenReview ↗

📌 Quick Navigation Index to Reviewer Threads & Responses — Feedback Welcome

Dear Area Chair and Reviewers,

As our rebuttal discussion now spans over 30 comments with detailed experiments and analyses, we provide direct links below to help you locate each review thread and all corresponding responses. We would appreciate hearing whether these responses address your concerns and whether any questions remain.

  • 👉 Jump to Reviewer rPjx’s Thread: Point-by-point responses to W1–W4 & Q1–Q3 covering batch-prompting baseline comparisons, separate question/reference ordering and history-selection ablations, prefix-answer error propagation, end-to-end runtime including preprocessing, statistical significance, and respondent filtering and demographic composition. Supplementary results: Chinese CGSS evaluation, embedding-encoder robustness, theoretical analysis, and context scaling.

  • 👉 Jump to Reviewer NMYf’s Thread: Point-by-point responses to W1–W5 covering comparisons against BPE, BPE+SEAS, ADP, and OBP; factorial ablations including inverse curriculum; full-survey prompting and historical-reference scaling; and clarification of SCBO’s contributions and related work. Supplementary results: Chinese CGSS evaluation, embedding-encoder robustness, statistical significance, complete runtime breakdowns, and theoretical bounds.

  • 👉 Jump to Reviewer jnoL’s Thread: Point-by-point responses to W1–W3 & Q1 covering batch-prompting baseline comparisons; shared reference-bank size and visibility; separate batching, history-selection, and ordering analyses; prefix-answer interventions; theoretical proofs for cross-batch information loss and conditional error propagation; and coverage-filter exclusion counts and demographic TVD analyses. Supplementary results: Chinese CGSS evaluation, embedding-encoder robustness, statistical significance, end-to-end overhead, and context scaling.

With the discussion period nearing its close, we would be grateful for a brief follow-up on any remaining concerns so that we can address them while there is still time for discussion. We are happy to provide further clarification.

Thank you again for your time and thoughtful engagement!

Best regards,
The Authors

AuthorsOpenReview ↗

Final 24-Hour Rebuttal Summary: Addressed Concerns & Request for Final Feedback

Dear Area Chair and Reviewers,

With less than 24 hours remaining in the discussion period, we provide a concise summary table mapping every reviewer concern (rPjx, NMYf, jnoL) to our targeted theoretical and empirical resolutions posted in this thread.

ReviewerCore Concern / QuestionSummary of Our Resolution & Key Quantitative Evidence
All Reviewers (rPjx W1, NMYf W1/W4, jnoL W1)Lack of batch-prompting baselines & unclear distinction from general batch prompting(1) Clarified task-specific modeling of individual-conditioned survey prediction vs. general few-shot batching. (2) Benchmarked against 4 batch-prompting baselines (BPE, BPE+SEAS, ADP, OBP) across 5 datasets (WVS, GSS, ANES, BSA + Chinese CGSS) and 4 LLMs. (3) SCBO achieves the highest or tied-highest Acc in 20/20 settings (+1.25–7.60 pp on English; up to +2.57 pp on CGSS), lowest MAE, highest F1, 5.1%–86.9% fewer weighted tokens, and shortest runtime.
All Reviewers (rPjx W2/Q1, NMYf W2, jnoL W2)Need for finer-grained ablations isolating components & testing inverse curriculum / error propagation(1) 6-way ordering factorial: Easy-to-hard + aligned references outperforms hard-to-easy (inverse curriculum, −1.05 to −1.96 pp) and random reference order (−1.00 to −1.99 pp). (2) 4-way batching × history ablation: Isolates semantic batching (+1.96–2.57 pp) and relevant history selection (+3.47–4.54 pp). (3) 10 prefix-intervention settings: Correct prefixes outperform wrong prefixes across all 10 cases, confirming error-propagation mitigation. (4) Encoder robustness: Tested 2 alternative encoders (ModernBERT-base, qwen3.7-text-embedding) across all 5 datasets; all 10/10 outperform the strongest baseline.
Reviewer rPjx (W3 & Q2)Missing preprocessing & end-to-end runtime breakdown(1) Reported full wall-clock breakdown across 8 model–dataset settings: semantic batching/ordering takes 1.53–17.88s and data processing takes 1.88–2.35s. (2) Total end-to-end time (23.5–163.1s) remains 2.5×–6.5× faster than BPE, BPE+SEAS, ADP, and OBP.
Reviewers rPjx & jnoL (rPjx W4/Q3, jnoL Q1)Statistical significance, sample filtering impact, & demographic representativeness(1) 50,000 respondent-level paired bootstrap resamples across 64 comparisons: SCBO wins 64/64 point estimates and is statistically significant (p<0.05p < 0.05) in 52/64 (13/16 per model). (2) Reported exact exclusion counts across budgets (3.20%–16.00%) and 6-attribute demographic TVDs (gender, age, education, marital status, income, region): all TVDs < 0.04 (< 0.02 for core demographics under 1-budget).
Reviewer NMYf (W3 & W5)Why not input the full survey in one prompt? & Main-text Related Work positioning(1) Evaluated context scaling up to the full survey (B=96B=96) and reference budgets up to k=15k=15 across 5 models: over-packing degrades accuracy due to attention dilution (e.g., Qwen3.7-Max drops from 54.43% to 52.66%; Qwen3-8B drops from 52.90% to 49.05%). (2) Committed to moving BPE, ADP, and OBP discussions into the main-text Introduction and Related Work, explicitly framing batch prompting as an established foundation.
Reviewer jnoL (W3)Need for stronger theoretical justification of clustering & orderingDerived 3 formal theorems & 2 corollaries: (1) semantic clustering eliminates cross-batch conditional total correlation under block separation; (2) descending historical-support ordering provably minimizes an explicit conditional error-propagation upper bound without assuming independent errors.

All detailed tables, statistical tests, and formal proofs are posted in our thread responses above and will be incorporated into the revised manuscript.

As the discussion window closes in less than 24 hours, we would be sincerely grateful if Reviewers rPjx, NMYf, and jnoL could briefly confirm whether these responses address your concerns and consider updating your scores accordingly. We remain actively online and ready to answer any final questions immediately.

Thank you very much for your time, guidance, and thoughtful consideration!

Best regards, The Authors

AuthorsOpenReview ↗

A Brief Follow-Up on Submission ☺

Dear Area Chair and Reviewers,

With less than 12 hours remaining in the rebuttal period, we would welcome any further questions. We have done our best to address the concerns raised, and our entire author team remains available to respond before the deadline.

We respectfully ask the Area Chair to take our responses into account when assessing the paper. If time permits, we would also be sincerely grateful if the Area Chair could send the reviewers a brief reminder to look over our replies while there is still time for discussion.

If our replies have addressed your concerns, we would be grateful if you would reconsider your assessments. Thank you again for your time and thoughtful review.

Best regards,
The Authors

AuthorsOpenReview ↗

Sincere Thanks as the Discussion Period Concludes

Dear Area Chair and Reviewers,

As the discussion period draws to a close, we would like to extend our heartfelt gratitude to all of you for dedicating your valuable time and thoughtful effort to reviewing our work.

First, we especially thank Reviewer NMYf for taking the time out of a busy schedule to read our responses and share constructive follow-up advice—we truly appreciate the helpful reminder regarding conciseness, and we will make sure to thoroughly incorporate the new baseline comparisons and clearer literature positioning into our revised manuscript as planned. At the same time, we completely understand that Reviewer rPjx and Reviewer jnoL have tight schedules during this busy end-of-semester and deadline season and regrettably have not had the chance to engage further in the discussion thread; nonetheless, your insightful initial reviews have guided us in strengthening this work and motivated us to thoroughly address every concern raised without leaving any loose ends.

Finally, we sincerely thank the Area Chair for carefully overseeing the entire review process, and we respectfully ask that you kindly take our supplementary empirical results and theoretical responses from the discussion period into consideration during your final assessment.

Thank you once again for your generous service to the research community, and we wish you all the very best!

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.