acceptodds
Under review as a conference paper at ICLR 2027

SocialPrism: Multi-Dimensional Generative Reward Modeling for Video Social Reasoning

Abstract

Video social reasoning requires models to interpret dynamic, complex, and often incongruent multimodal signals to infer hidden mental states. Existing reward modeling approaches in reinforcement learning predominantly rely on blunt scalar scores, which collapse complex social reasoning processes into coarse feedback and make it difficult for models to identify or correct specific cognitive failures. To address this bottleneck, we introduce SocialPrism, a multi-dimensional and generative reward framework that decomposes holistic social judgments into five fine-grained cognitive dimensions. We construct SocialPrism-Data to provide multi-dimensional scores and rationales for reward model training, and SocialPrism-Bench, a human-verified benchmark for reward evaluation. Furthermore, we propose Gen-SocialPrism-7B, a generative reward model trained with a two-stage curriculum learning strategy. On SocialPrism-Bench, it achieves the lowest overall score prediction error among the compared reward models. It also outperforms standard binary-reward baselines in post-training and improves test-time reward-guided selection over standard sampling baselines.

Primary area:
applications to neuroscience & cognitive science
Keywords:
video social reasoning, theory of mind, reward model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XHRjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes SocialPrism, a generative reward framework for evaluating and improving video social reasoning. SocialPrism evaluates model responses along five dimensions, including contextual fact consistency, inference logical consistency, multimodal information utilisation, causal rationality, mental state inference. The authors construct a few datasets, including a 30+k annotated dataset and 300+ sample benchmark, and then train Gen-SocialPrism-7B. The main results include that the reward prediction error decreases compared to existing reward models, the model improves performance over binary reward GRPO.

Critical strengths
  • The paper identifies a clear limitation of scalar rewards in that they cannot distinguish failure cases such as perception, multimodal integration and social inference.
  • The authors also employ a comprehensive dataset construction pipeline, including diverse response generating models, multi-stage annotation, rubric-compliance verification etc.
  • The authors also demonstrate downstream utility by evaluating the reward modelling in RL post-training and test time selection.
Critical weaknesses
  • The methodological novelty is unclear. The method seems to be a combination of different components that are already individually established, such as multidimensional reward modelling, generative evaluator etc. There isn't any new or scientific intellectual insight here.
  • The work depends heavily on synthetic annotations. The training annotations are produced by Gemini 2.5 Pro as well as the benchmark. This creates a dependence on the teacher model's judgements.
  • Ablations are insufficient, in the sense that they do not convincingly establish which components are most necessary. The paper would benefit from stronger ablations on individual dimensions etc.
  • The paper also lacks mechanism analysis, to show that the method works as intended.
Questions for authors

Questions to be addressed are stipulated in the weaknesses above.

4 comments

We thank the reviewer for recognizing the motivation, dataset construction pipeline, and downstream utility of SocialPrism. Below, we clarify the contributions and supporting evidence, including an additional post-training ablation.

W1: Contribution and scientific insight.

Our primary contribution is the structured data and evaluation framework for training and evaluating verifiers and reward models for video social reasoning. Assessing the work solely through the novelty of its individual modeling components overlooks this contribution. SocialPrism-Data provides 37,087 responses annotated with dimension-specific scores and rationales, complemented by VSR-8K with 8,345 QA pairs, SocialPref-8K with 8,251 preference pairs, and SocialPrism-Bench with 339 human-verified evaluation samples. These resources support reward learning, preference learning, policy optimization, and reward-model evaluation within a common framework.

The scientific question is whether explicitly evaluating different aspects of social reasoning produces a more useful reward signal than evaluating only aggregate quality or answer correctness. This distinction matters when a response must reconcile conflicting audiovisual cues and justify an interpretation of hidden mental states: reaching the correct option does not establish that the supporting reasoning is sound. SocialPrism makes these aspects measurable through explicit rubrics and structured supervision. Table 2 and the additional experiment in W3 show benefits in both reward prediction and downstream policy performance.

The resulting data resources also provide a basis for future work on verifier-guided iterative self-improvement. Their practical utility is demonstrated here through post-training and test-time selection. The contribution therefore comprises reusable supervision resources, a domain-specific evaluation protocol, and empirical evidence that the structured reward formulation improves these applications.

W2: Dependence on synthetic annotations.

Gemini 2.5 Pro serves as the initial annotator, but the final scoring data undergo additional verification before use. Annotation quality should therefore be assessed through the full construction and validation procedure, which includes:

  • Diverse response generation: responses from 11 models provide coverage of different capabilities, response styles, and failure modes.
  • Two-stage annotation: Gemini 2.5 Pro first produces question-relevant video descriptions and then evaluates responses using both those descriptions and the original video.
  • Cross-model rubric verification: Seed1.8, Grok 4.1 Fast, and DeepSeek-Reasoner check whether the scoring rationales appropriately interpret the responses and whether the assigned scores follow the rubrics. Samples are retained by majority vote.
  • Video-grounded hallucination checking: Gemini 3 Flash separately checks the scoring rationales against the video to filter unsupported claims.

These stages separately address response coverage, video understanding, rubric compliance, and unsupported claims. Acceptance into the training corpus is thus subject to checks beyond the initial evaluator's judgment.

Crucially, the final reference annotations in SocialPrism-Bench are human-verified. Two annotators independently review the scores and rationales across all five dimensions. Samples are retained when both annotators accept the annotations or agree on the same correction. Section 4.4 reports 92.64% agreement between automatic annotations and human judgments, alongside 89.65% inter-annotator agreement.

The reported agreement with human judgments provides empirical evidence of annotation reliability. Moreover, reward prediction is evaluated against the final human-verified reference annotations, rather than unreviewed outputs from Gemini 2.5 Pro.

W3: Ablations and justification of the evaluation dimensions.

The paper provides both a theoretical basis for the five dimensions and empirical ablations of the reward model. Section 3 derives the dimensions from a capability hierarchy encompassing perception, multimodal integration, causal and social inference, and logical coherence, drawing on Theory of Mind, the Belief–Desire–Intention model, and social cognition research. Each dimension targets a distinct aspect of response quality.

The original ablations in Table 2 examine both curriculum learning and multidimensional modeling. Removing Stage 1 increases total-score NMAE from 0.2016 to 0.2069, while the total-score-only variant yields an NMAE of 0.2071. These results support the contributions of both components to reward prediction.

To further assess downstream utility, we conducted an additional post-training ablation by replacing Gen-SocialPrism-7B with the total-score-only reward model while keeping the training configuration unchanged. The total-score-only model directly predicts the aggregate score, whereas Gen-SocialPrism-7B explicitly evaluates the five dimensions. The results are:

Benchmark / subset+ CS + Gen-SocialPrism-7B+ CS + Total-score-only RMDifference (pp)
R³-Bench-Hard33.2331.33+1.90
EmoBench-M: SD67.8062.20+5.60
EmoBench-M: PEA75.2069.20+6.00
EmoBench-M: MPDER51.4051.60−0.20
WorldSense47.7947.76+0.03
OmniVideoBench: (0, 1] min49.1247.95+1.17
OmniVideoBench: (1, 5] min35.5934.12+1.47
FutureOmni52.4251.55+0.87
Mean51.5749.46+2.11

Values are accuracy (%). The mean weights the eight reported metrics equally. Differences are in percentage points.

Under this matched training configuration, Gen-SocialPrism-7B improves seven of the eight metrics, increasing mean accuracy from 49.46% to 51.57% (+2.11 percentage points). The largest gains occur on SD (+5.60 points) and PEA (+6.00 points), with small decreases on MPDER (−0.20 points).

Together with Table 2, this additional experiment demonstrates the benefit of the multidimensional reward formulation over direct total-score prediction in both reward prediction and downstream policy improvement.

W4: Analysis of how the method works.

Our evaluation explicitly tests whether Gen-SocialPrism-7B provides the quality signal required by its intended uses across four complementary settings:

  • Reward prediction: On SocialPrism-Bench, Gen-SocialPrism-7B achieves a total-score NMAE of 0.2016 and a Spearman correlation of 0.4247, outperforming the compared reward models on both metrics (Table 2). This tests agreement with human-verified quality judgments.
  • Post-training: With the same cold-start initialization, GRPO algorithm, and QA training data, adding the Gen-SocialPrism-7B reward improves all eight reported metrics over binary-reward GRPO (Table 3). The additional total-score-only comparison in W3 further shows the downstream benefit of the multidimensional reward formulation.
  • Inference: Figure 3 directly examines whether assigned rewards distinguish response reliability. Accuracy increases from 38.5% to 43.5% to 55.9% across the low-, medium-, and high-reward groups on WorldSense, and from 30.7% to 49.2% to 63.5% on FutureOmni. The high-reward group exceeds the overall accuracy on all five benchmarks.
  • Sampling: Figure 4 tests whether the reward signal improves answer selection by increasing the voting weight of high-reward candidates. Under the same sampling budgets, reward-guided voting improves upon or performs competitively with majority voting across the evaluated settings, with particularly clear gains on EmoBench-M, WorldSense, and OmniVideoBench.

These experiments examine agreement with human judgments, discrimination between responses of different reliability, and the use of that discrimination in training and selection. They provide direct evidence that the reward model performs its intended functions. We would welcome clarification of any specific mechanism or failure mode the reviewer considers unaddressed, so that we can provide a targeted response.

Reviewer_XHRjOpenReview ↗

Reviewer Response

I thank the authors for their rebuttal. I have decided to retain my score, for the following reasons.

  1. I believe the contribution is still unclear. The authors iterate that part of the contribution includes "whether explicitly evaluating different aspects of social reasoning produces a more useful reward signal than evaluating only aggregate quality or answer correctness". My opinion is that this is a rather "obvious" insight that may not be entirely new. Additionally, pitching this as a "benchmark/data" contribution may not help here - benchmark/data contributions typically involve a new axes of evaluation, new problem setting, and more demonstration of the utility of the data.

  2. I acknowledge the verification of the synthetic data, and also that of the analysis of how the method works. On the latter part, I think the intellectual insight of this method is still unclear - these appear to be robustness tests, which are welcome, but we aren't really testing any new principles introduced etc.

Overall while I acknowledge the author's efforts, I think the intellectual contribution of this work is still unclear.

I would suggest for the authors, to simplify the entire framework and focus on what is that "one" singular aspect of the methodology that leads to better results. It can't simply be that "one singular aspect" = our multi-dimensional framework. I hope the authors understand my point of view.

Thank you for clarifying your perspective. We understand your emphasis on identifying a specific methodological advance and appreciate your suggestion to sharpen the central contribution.

We respectfully maintain that the structured training and evaluation resources are substantive contributions in their own right. SocialPrism provides explicit scoring criteria, dimension-specific annotations, and human-verified evaluation data for training and assessing video social reasoning verifiers. Their utility is demonstrated through reward prediction, policy improvement, and test-time answer selection.

A reliable verifier is particularly valuable in the domain of video social reasoning: making the cognitive processes underlying social reasoning in the real, open world quantifiable. Resources that enable models to assess these distinctions support further research on social reasoning and verifier-guided improvement (even to support recursive self-improvement in this domain). We therefore believe the contribution should also be assessed through the capabilities these resources enable and their demonstrated downstream utility, alongside the novelty of the individual modeling components.

Reviewer_CZ8QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SocialPrism evaluates video social reasoning through five dimensions, producing a score and rationale for each. The authors construct scoring/preference data and a human-verified benchmark, then train a reward model with a single-dimension-to-joint curriculum. The model improves reward prediction and provides gains in GRPO post-training and test-time voting.

Critical strengths

Separating factual grounding from internal consistency makes sense: a coherent explanation can still misread the video. The annotation framework makes this distinction explicit, including for responses with correct final answers.

The GRPO comparison shares policy initialization and QA data with the binary-reward baseline. Improvements across the evaluated tasks give useful evidence that the learned reward helps (Table 3).

Critical weaknesses

I am not clear what the multidimensional design adds beyond a domain-trained scalar reward. Appendix D.1 aggregates dimension scores into a correctness-gated scalar; the policy never receives the diagnostic rationales. The Total-score-only variant in Table 2 has close prediction error, but no downstream results or ranking correlation are reported. Lower score error need not mean better response ordering. Seems the gain could mainly come from learning a task-specific verifier. Also, matching reference scores does not establish that the generated critiques correctly diagnose reasoning failures.

The MIU rubric is a bit troubling. It scores single-modality reasoning below flawed multimodal integration (B.2), without checking whether the question needs both modalities. Dialogue-rich videos do not guarantee this. A sufficient explanation can therefore be penalized on this dimension for omitting unnecessary cues. This favors a prescribed form of explanation, potentially at the expense of appropriate evidence use. Human agreement under the same rubric does not resolve this concern.

Questions for authors
  1. How does Total-score-only perform in GRPO and reward-guided voting with matched training budgets and reward scaling? Please also report its Spearman correlation.
  2. For responses to the same question with the same correct final answer, does your reward agree with blind human judgments of reasoning quality, especially when one modality already provides sufficient evidence?
1 comment

Thank you for the valuable suggestions. We provide additional downstream results and clarify the motivation behind the MIU rubric below.

W1&Q1 Multidimensional rewards versus a scalar reward.

We replace Gen-SocialPrism-7B with the total-score-only RM while keeping the training configuration unchanged and the results are:

Benchmark / subset+ CS + Gen-SocialPrism-7B+ CS + Total-score-only RMDifference (pp)
R³-Bench-Hard33.2331.33+1.90
EmoBench-M: SD67.8062.20+5.60
EmoBench-M: PEA75.2069.20+6.00
EmoBench-M: MPDER51.4051.60−0.20
WorldSense47.7947.76+0.03
OmniVideoBench: (0, 1] min49.1247.95+1.17
OmniVideoBench: (1, 5] min35.5934.12+1.47
FutureOmni52.4251.55+0.87
Mean51.5749.46+2.11

Values are accuracy (%). The mean weights the eight reported metrics equally. Differences are in percentage points.

Gen-SocialPrism-7B improves seven of eight metrics, raising mean accuracy by 2.11 percentage points. These results demonstrate a downstream benefit over a total-score-only verifier, despite their relatively close reward-prediction errors in Table 2.

W2: Scoring rubric for MIU.

MIU is designed to encourage cross-modal consistency checking, which we regard as a basic requirement of video social reasoning. Even when one modality appears sufficient, consulting the other can help rule out competing interpretations—for example, apparent verbal agreement may conflict with facial expressions or actions. A response can therefore emphasize the modality carrying the decisive evidence while still using other modalities to confirm or challenge its interpretation.

Accordingly, MIU assigns 0 when a response relies exclusively on one modality without meaningful cross-modal integration. This criterion does not require equal attention to both modalities or an exhaustive description of every cue.

Beyond the post-training ablation above, we will consider to add the remaining requested analyses in a future version. Thanks again for your effort and thoughtful suggestions.

Reviewer_AecXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SocialPrism, a reward model that scores video social reasoning responses along five dimensions and explains its scores. The authors build training data and a human-verified benchmark, then use the model for reinforcement learning and answer selection. They report improvements over binary-reward training and majority voting.

Critical strengths

The first strength is that authors use several models to generate responses and check the annotations for rubric compliance and hallucinations. The benchmark is also checked by human annotators. These steps help with data quality, though annotation bias may still remain. The second is that the paper tests the reward model in both RL training and test-time answer selection. The comparison with standard GRPO uses the same initialization and QA training data, which gives some evidence that the reward signal helps.

Critical weaknesses

I listed three weaknesses:

  1. Since the annotator sees the correct answer and captions generated using it, the scores may favor explanations that fit the known answer. How much do the scores change without this information, particularly for CR and MSI?

  2. I could not find any details on how the authors checked for overlap between the training and evaluation videos. The training data come from several datasets, so I would like to know whether the same videos, or clips from them, appear in both.

  3. Were the training experiments repeated with different seeds? The improvement on R³-Bench-Hard is only 0.32 percentage points. Without variation across runs or confidence intervals for the accuracy differences, it is hard to tell whether this gain is reliable.

Questions for authors
  1. Did you check whether any videos or overlapping clips appear in both the training and evaluation data? Please clarify this for both reward-model and policy training, and report any overlap you found.

  2. Were the Table 3 results repeated with different training seeds? The gain over standard GRPO on R³-Bench-Hard is only 0.32 percentage points, so I would like to know whether it holds across runs.

Thank you for clarify!

1 comment

We thank the reviewer for the time and valuable suggestions. As this review became available shortly before the discussion deadline, we have made every effort to address the concerns within the limited time available.

W1: Providing the correct answer during annotation

The correct answer and the captions help the annotator understand the video and the question before scoring a response. Without this information, the annotator may assign incorrect scores because it misunderstands the underlying scenario. Consequently, changes in scores (particularly for CR and MSI) could reflect differences in the annotator's understanding, rather than solely a tendency to favor explanations that fit the known answer. We therefore consider this information necessary for reliable annotation.

W2 & Q1: Overlap between training and evaluation videos

The data used for both reward-model training and policy training originate from SocialIQ-2.0 (2023) and LLaVA-Video-178K (2024). The evaluation benchmarks used in this work were introduced in 2025 or later. These datasets were constructed separately.

W3 & Q2: Results across different training seeds

We report results from a single training seed in Table 3. This follows the reporting format used in large multimodal-model technical reports.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.