- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper uses cultural spheres to guide prompting and provide proxy supervision when a culture lacks training labels. Experiments across ten LLMs report better cultural alignment, particularly on WVS and for selected low-resource cultures. The fine-tuning experiments test whether supervision from related cultures transfers to held-out cultures and other benchmarks.
Critical strengths
-
The paper addresses a practical problem: improving alignment when a target culture has no annotations. Holding out the labels of eight low-resource cultures provides a meaningful test of whether supervision can transfer from related cultures.
-
Prompting improves WVS accuracy across all ten models, with the average rising from 52.25% to 58.51%. The ablations help explain the gain: same-sphere agreement provides most of the benefit, while distant-sphere contrast adds little.
Critical weaknesses
-
The controls do not separate the benefit of sphere membership from that of selecting agreeing examples. The experiments lack matched controls using agreeing countries selected without sphere labels or shuffled groupings. Agreement-only prompting already reaches 58.27%, compared with 58.51% for full SC-ICL. It is therefore unclear what the taxonomy adds beyond selecting and repeating supportive examples.
-
The VSM comparison requires a calibration that the paper does not provide. Appendix C.4 rescales model-derived scores using theoretical extrema, then compares them directly with published Hofstede scores. Putting scores in the same numerical range does not make their scales comparable; the VSM manual, §4 https://geerthofstede.com/wp-content/uploads/2016/07/Manual-VSM-2013.pdf explicitly cautions against such comparisons. This weakens the main evidence for transfer beyond WVS.
-
The open-ended results are less convincing. The PRISM judge sees each candidate's generation prompt, including sphere information supplied only under CSP. The comparison thus changes both the response and the context used to score it, with no blinded or human validation to separate their effects. Fine-tuning also reduces PRISM performance: every checkpoint falls below its untuned baseline, with relative declines of 9.4% to 15.9% (Table 6).
Questions for authors
- Do the gains persist when agreeing reference countries are selected without sphere labels, with the same demonstration questions, reference counts, and prompt length?
- What validates the mapping from model-derived VSM scores to published Hofstede scores? Do comparisons of country rankings support the same conclusions?
- Does the PRISM gain persist under blinded judging with identical evaluation context for both methods?
- How does proxy supervision affect performance on questions where a held-out culture disagrees with its sphere's consensus?
2 comments
W3: Potential Judge Bias in PRISM Evaluation
We conducted an additional Blind Judge Evaluation to eliminate the potential influence of generation prompts. Specifically, we retained the original Vanilla and CSP responses, GPT-4o judge, PRISM references, and scoring rubric, while removing the generation system prompts from the judge's input.
| Model | Vanilla | CSP |
|---|---|---|
| GPT-4o | 78.34 | 79.54 (+1.5%) |
| GPT-4.1 | 78.51 | 81.02 (+3.2%) |
| GPT-5 | 79.06 | 80.87 (+2.3%) |
| GPT-5.2 | 77.96 | 77.99 (+0.0%) |
| o3 | 76.56 | 78.37 (+2.4%) |
| Kimi-K2.5 | 79.33 | 79.47 (+0.2%) |
| DeepSeek-V3.2 | 77.81 | 79.14 (+1.7%) |
| Mistral-Large-3 | 81.08 | 81.33 (+0.3%) |
| Mistral-Medium-2505 | 79.62 | 79.91 (+0.4%) |
| Llama-3.3-70B | 74.21 | 75.40 (+1.6%) |
| Average | 78.25 | 79.30 (+1.4%) |
| 95% CI of relative gain | — | [+0.6%, +2.1%] |
The overall improvement trend remains consistent: CSP achieves gains across all 10 models, with an average relative improvement of +1.4% (95% CI: [+0.6%, +2.1%]). This further supports the effectiveness of cultural-sphere prompting under controlled evaluation conditions. We will include these results in the revision.
W4: Impact of Proxy Supervision on Country-Specific Preferences
We address this concern by clarifying the role of consensus supervision and providing further empirical evidence.
(1) Effectiveness of Proxy Supervision for Cold-Start Alignment. When target-culture training data are unavailable, cultural-sphere consensus provides a practical source of proxy supervision. As shown in Table 5, SFT-Aug improves held-out WVS accuracy across all three Qwen3 model scales, demonstrating its effectiveness in this setting.
(2) Improvements Across Consensus Levels. Following your suggestion, we conducted additional experiments to compare the results across high- and low-consensus questions. As shown in the table below, SFT-Aug achieves substantial gains for held-out low-resource cultures on high-consensus questions (+12.08, +4.35, and +7.23 pp for 4B/8B/14B). Importantly, performance does not decline on low-consensus questions, with further gains of +0.61, +3.03, and +3.89 pp, respectively. This suggests that consensus supervision may facilitate the transfer of shared cultural patterns while maintaining reasonable performance on less-consensual questions.
| Model | Method | High consensus | Low consensus |
|---|---|---|---|
| Qwen3-4B | Vanilla | 35.91 | 32.03 |
| SFT-Naive | 43.44 (+7.53) | 33.28 (+1.25) | |
| SFT-CSP | 45.84 (+9.93) | 30.19 (−1.85) | |
| SFT-Aug | 47.99 (+12.08) | 32.64 (+0.61) | |
| Qwen3-8B | Vanilla | 56.68 | 32.60 |
| SFT-Naive | 56.92 (+0.24) | 32.72 (+0.12) | |
| SFT-CSP | 60.63 (+3.95) | 34.66 (+2.06) | |
| SFT-Aug | 61.02 (+4.35) | 35.63 (+3.03) | |
| Qwen3-14B | Vanilla | 66.18 | 34.69 |
| SFT-Naive | 66.63 (+0.46) | 34.12 (−0.57) | |
| SFT-CSP | 68.54 (+2.36) | 33.22 (−1.47) | |
| SFT-Aug | 73.40 (+7.23) | 38.58 (+3.89) |
Thanks for your supportive review and valuable suggestions. Our responses to the weaknesses (W) and questions (Q) are provided below. A revision of our paper will be uploaded, with the revised parts marked in blue.
W1: Contribution of Cultural-Sphere Structure vs. Selected Examples
We would like to clarify the contribution of cultural-sphere priors versus the selected answers through existing and additional experimental evidence.
(1) Existing Evidence. Table 1 shows that adding cultural-sphere context without any reference answers (Vanilla → CSP) yields average relative gains of +1.5%, +3.7%, and +3.3% on WVS, VSM, and PRISM, respectively. On WVS, with the same demonstration questions and target-country answers, adding sphere context raises accuracy from 52.89% to 53.54% (Standard ICL → SP-ICL), while further incorporating sphere-based reference answers raises it to 58.51% (SC-ICL).
(2) Additional Reference Selection Ablation. To examine whether predefined spheres provide useful guidance beyond simply adding reference answers, we compared three strategies on GPT-4o, GPT-5, and GPT-5.2, keeping the demonstration questions, target-country demonstration answers, and number of references fixed.
Sphere-based selection outperforms both alternatives across all three models, supporting the value of cultural relationships beyond additional reference answers. Importantly, similarity-based selection requires sufficient human response data to estimate cultural similarities, which may be difficult to obtain for low-resource cultures. In contrast, established cultural-sphere priors provide a practical alternative when such data are scarce.
In the revision, we will incorporate these results to further clarify the contribution and applicability of cultural-sphere priors.
W2: VSM Calibration and Evaluation Validity
We conducted an additional analysis to address this issue and further clarify the robustness of our findings.
(1) Offset-Adjusted Similarity. To address the calibration concern, for each model, VSM dimension, and target country, we estimated an additive constant C from the mean difference between published Hofstede and Vanilla scores across the other 36 countries, applying the same C to both methods. Under this leave-one-country-out calibration, CSP achieves an average similarity of 79.91, compared with 79.51 for Vanilla (+0.40 pp)
(2) Existing Structural Evidence. Table 3 evaluates the consistency of pairwise cultural similarity rankings between models and human references. This metric is invariant to uniform additive offsets within each VSM dimension. CSP improves Spearman's correlation from 0.236 to 0.381, providing complementary evidence that does not depend on such calibration constants.
Together, these results further support the robustness of our findings. We will incorporate the additional analysis and clarify the limitations of absolute VSM score comparisons in the revision.