- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper collects cultural norms from CulturalBench, FORK, and NormAd, and uses a multi-agent process to instantiate them in social scenarios with implicit norms. The data are split by norm, with 3,521, 449, and 222 scenarios for training, validation, and testing, respectively; the NormCompass test set contains only norms from NormAd. Multiple models generate candidate answers; GPT-5.6 Sol assigns scores from 1 to 5, and untied within-scenario scores are converted into preference pairs. CRISP-RM trains a scalar reward head on Qwen3-0.6B or Qwen3-4B with the standard Bradley–Terry loss. During policy training, the 4B reward model reads the scenario and final answer only; its score is calibrated with validation-set statistics, mapped through a sigmoid, and used by GRPO. An optional 8B Norm Grounding Supervisor (NGS) additionally reads the gold norm and observable rationale and supplies a Grounded probability as an auxiliary reward. The paper evaluates the cultural reward with Best-of-N selection, GRPO for three policy models, CultureForest and CulShield, and controlled triplet analyses. These experiments support gains from culturally adapted reward data and provide evidence that NGS can improve observable norm grounding.
Critical strengths
- The data isolation and external evaluation are strong enough to make the empirical gains meaningful.
- The experiments are substantial and provide evidence for the effectiveness of each major design module.
Critical weaknesses
- The reward is validated against a supplied norm, but the independent evidence for “cultural appropriateness” remains limited. Preference annotation, the NormCompass final score, norm-matching diagnostics, and the construction and filtering of controlled triplets all rely on GPT-5.6 Sol. This is not test-label leakage, but it can make both training and analysis favor one evaluator's interpretation of the norm. The evaluation of 100 responses by two human annotators is useful support (human–GPT Krippendorff's α = 0.732), but the paper does not report the annotators' cultural experience, coverage across cultures, disputed cases, or whether optimized responses were included. Thus, the evidence mainly establishes agreement with scoring under the supplied norm; it does not establish that the norm should take priority in each scenario.
- The incremental value over culture-specific reward methods is not tested directly enough. CRISP uses a conventional scalar reward model, Bradley–Terry training, and GRPO; the main additions are contextual preference data and norm-grounding supervision. Therefore, outperforming general reward models that were not adapted to this domain establishes that specialized data are useful, but does not establish the relative advantage of the proposed design. The paper cites and discusses Think-as-Locals but does not provide an experimental comparison, even though that work also addresses cultural preferences and surface-cue interference. Although CuSiR is included, Appendix C.3 describes only a CuSiR-style three-dimensional reward and does not provide its time-weighting formula or enough adaptation detail for verification. CuSiR-style and Skywork degrade by roughly 19–34 points on CultureForest, and the degradation remains large on valid outputs. This important comparison needs a reproducible implementation and failure analysis rather than an emphasis only on parameter scale.
Questions for authors
- Could evaluators familiar with the relevant cultures blind-rate responses before and after optimization, while allowing them to mark the target norm as inapplicable, identify reasonable exceptions, or accept multiple actions? Can report the model sources, cultural distribution, and annotator expertise for the existing 100-response human evaluation, as well as human–model disagreements rather than only aggregate agreement?
- If a same-sized general reward model is adapted using the same preference data, how much of CRISP's advantage remains?
- NGS simultaneously adds the gold norm, an 8B model, and rationale supervision. What happens in controls that use only NGS, allow the answer reward to access the gold norm, or supervise only the norm rationale? After obtaining 88.31% macro-F1 on 117 test examples excluding other failure types, does NGS remain reliable for optimization-induced errors, generic platitudes, and responses whose rationale is correct but whose action is opposite to the norm?
- Figure 2 and Figure 6 use the test set to show training progress and λ sensitivity. Were the final checkpoint and λ selected entirely on an independent validation set? Could you specify the selection rule, repeated-training results, the actual default λ, and the additional training cost of NGS?
5 comments
Response to Questions For Authors-1
Q1: We thank the reviewer for this valuable suggestion. Following the reviewer's recommendation, we conducted an additional human evaluation comparing model responses before and after policy optimization to further validate the effectiveness of CRISP+NGS. Specifically, two human annotators independently rated responses generated before and after optimization with CRISP+NGS. As shown in the table, both annotators observed substantial improvements, with the average score increasing from 3.57 to 4.19. These results further demonstrate that CRISP+NGS not only improves automatic evaluation scores but also enhances culturally appropriate behavior as judged by human annotators, providing additional evidence of our method's effectiveness independent of the automatic evaluator.
Human Evaluation Results Before and After Policy Optimization
| Human Annotator | Before Optimization | +CRISP+NGS | Improvement |
|---|---|---|---|
| Annotator 1 | 3.62 | 4.28 | +0.66 |
| Annotator 2 | 3.52 | 4.10 | +0.58 |
| Average | 3.57 | 4.19 | +0.62 |
Q2: We thank the reviewer for this suggestion. The core contribution of CRISP lies in constructing situated cultural preference data and training a specialized cultural reward model using these data, rather than designing a new model architecture. To investigate whether our approach depends on a specific model initialization, we further use Skywork Reward V2 as the base model and train it on the same CRISP preference data. Experimental results show that models with different initializations achieve comparable performance at both the 0.6B and 4B scales. These findings suggest that our cultural reward modeling approach is not dependent on a particular model initialization and can be effectively applied to existing general purpose reward models.
Best-of-N Results with Different Reward Model Initializations
| RM | Size | NormCompass @8 | NormCompass @16 | CultureForest @8 | CultureForest @16 |
|---|---|---|---|---|---|
| Random | — | 3.6014 | 3.6104 | 56.6199 | 57.1828 |
| CRISP (Original) | 0.6B | 3.7838 | 3.8288 | 58.8661 | 58.7889 |
| CRISP (Skywork Initialization) | 0.6B | 3.7838 | 3.7995 | 58.7155 | 58.6499 |
| CRISP (Original) | 4B | 3.8896 | 3.9144 | 59.6382 | 60.2969 |
| CRISP (Skywork Initialization) | 4B | 3.9032 | 3.9347 | 59.8050 | 60.4017 |
Q3-1: To investigate the contributions of individual NGS components, we conducted additional ablation experiments using Qwen3-4B. The results show that, compared with the base Qwen3-4B model, using NGS alone improves performance on NormCompass (3.79) but leads to a noticeable decline on CultureForest (53.06). Adding only norm grounding rationale supervision (CRISP + rationale) improves performance on CultureForest (58.97), but underperforms CRISP alone on NormCompass. In contrast, the full CRISP + NGS achieves the best results on both benchmarks (3.86 / 60.28), suggesting that cultural reward signals and norm grounding supervision are complementary, and their combination leads to more comprehensive performance improvements.
| Setting | NormCompass | CultureForest |
|---|---|---|
| Qwen3-4B | 3.27 | 56.18 |
| +CRISP | 3.69 | 57.60 |
| +CRISP + Rationale | 3.45 | 58.97 |
| +NGS Only | 3.79 | 53.06 |
| +CRISP + NGS | 3.86 | 60.28 |
Response to Questions For Authors-2
Q3-2: We thank the reviewer for raising concerns about the reliability of NGS. To further evaluate its reliability in complex error scenarios, particularly its ability to detect norm recognition errors arising during policy optimization and distinguish different types of incorrect responses, we conducted three additional experiments to systematically assess the effectiveness and limitations of NGS.
(1) Norm Error Detection in Real Training Trajectories. To assess the reliability of NGS during policy optimization, we randomly sampled 160 responses from the reinforcement learning trajectories of Qwen3-4B. After GPT based verification, we identified 79 rollouts that failed to correctly recognize the target cultural norm. Among these, NGS assigned a p(pass) below 0.5 to 66 responses (83.54%), with an average p(pass) of 0.1873 across all 79 responses.
(2) Generic and Vague Responses. We constructed generic variants of 50 culturally appropriate responses that lacked specific guidance grounded in cultural norms. NGS assigned an average p(pass) of 0.8898 to the original responses, compared with only 0.0036 for the generic variants. Moreover, NGS preferred the original responses in all 50 paired comparisons. (Paired comparison refers to comparing the p(pass) values of the original and modified responses to the same question.)
(3) Inconsistency Between Norm Understanding and Action. Using the same 50 original responses, we further constructed variants that correctly identified the cultural norm but recommended actions that violated it. NGS assigned these variants an average p(pass) of 0.2566 and preferred the original responses in 94% (47/50) of paired comparisons.
These results demonstrate that NGS can detect most norm recognition errors arising during actual policy optimization and effectively distinguish generic responses from those providing specific norm grounded guidance. Even in the more challenging cases where the norm is correctly understood but the recommended action is inappropriate, NGS maintains strong relative discrimination ability.
Q4: We thank the reviewer for their attention to our experimental setup. We provide the following clarifications. Regarding checkpoint selection, all GRPO experiments use a fixed training budget of 880 steps and are evaluated using the final checkpoint, without selecting checkpoints based on validation or test set performance. Regarding the choice of λ, all main experiments use λ = 0.1. Figure 6 is intended solely to analyze the impact of different λ values on model performance, rather than to select the final hyperparameter. Regarding the computational overhead of NGS, NGS introduces an additional forward pass through the Qwen3-8B classifier at each GRPO training step.
Response to Weaknesses
We sincerely thank the reviewer for the thoughtful and constructive feedback. We appreciate the concerns regarding cultural appropriateness, the contributions of CRISP, and the reliability of our experimental evaluation. These comments have motivated us to conduct additional experiments and provide further clarification. We address each concern in detail below.
Concerns About the Cultural Appropriateness of the Data. We thank the reviewer for raising these concerns. We provide additional clarification from three perspectives. First, to reduce reliance on a single evaluator, we independently rescored 100 human annotated responses using Gemini 3.8 Flash and DeepSeek V4 Pro. Both evaluators show strong correlations with GPT-5.6 Sol (Spearman ρ = 0.684/0.690), indicating a degree of consistency across different models. Second, our original human evaluation primarily aims to examine whether model scores align with human judgments of whether a response adheres to a given norm, rather than directly validating the applicability of these norms in specific contexts. Third, regarding concerns about norm applicability, we acknowledge that the broad cultural coverage of our study makes it challenging to recruit annotators with relevant expertise across all represented cultures. To address this, we additionally employ two top tier models, Gemini-4-Argon and DeepSeek-V4-Pro, to filter our testbed based on the contextual applicability of the target norms. As shown in the table below, our filtered testbed contains 204 instances, and the experimental results remain comparable to those obtained on the original testbed. Furthermore, optimization using our constructed training data yields performance improvements on both our testbed and CultureForest, providing additional evidence supporting the quality of our training data.
Performance Comparison on the Filtered NormCompass Testbed
Concerns About the CRISP. We thank the reviewer for this suggestion. We would like to clarify that situated cultural preference data and the resulting trained reward model constitute the core contributions of CRISP. Our goal is not to introduce a new reward model architecture or optimization algorithm, but to develop a specialized reward model capable of distinguishing culturally appropriate behaviors in situated contexts.
We additionally compare against Think-as-Locals using Qwen3-4B as the policy model. Think-as-Locals achieves scores of 3.58 on NormCompass and 57.52 on CultureForest, both lower than those achieved by CRISP+NGS.
Regarding CuSiR, we implemented the three dimensional reward and dynamic curriculum weighting consistent with the original paper and adapted them to single turn GRPO training. We will provide additional implementation details in Appendix C.3.
Regarding the performance degradation on CultureForest, we believe it stems from reward models assigning excessively high rewards to responses that appear superficially plausible but violate cultural norms. Figure 3 has already shown that baseline models are susceptible to such superficial plausibility. Further analysis of the training logs reveals two representative cases: (1) In a Ukrainian flower giving scenario, Skywork assigned a high reward of 0.9985 to an incorrect response that insisted on giving six roses, rationalizing the number six as a symbol of luck, harmony, and love. (2) In a Korean workplace drinking scenario, CuSiR assigned a cultural reward of 0.95 to an inappropriate recommendation to maintain eye contact while drinking, which the policy justified as balancing humility and active engagement. These cases suggest that reward biases in baseline models may be further amplified during policy optimization.