acceptodds
Under review as a conference paper at ICLR 2027

ProCrit: Self-Elicited Multi-Perspective Reasoning with Critic-Guided Revision for Multimodal Sarcasm Detection

Abstract

Multimodal sarcasm detection requires reasoning over cross-modal incongruities between literal expression and intended meaning, yet the analytical perspectives needed to uncover these incongruities vary across samples. While recent methods make this analytical process explicit, they still rely on fixed, predefined perspectives that operate independently under hand-crafted routing rules. We argue that multimodal sarcasm detection instead calls for self-elicited multi-perspective reasoning, where a model autonomously generates the perspectives needed for each sample and progressively integrates them into a coherent analysis. To realize this goal, we propose ProCrit, a Proposal–Critic two-agent framework with a proposal agent for multi-perspective reasoning and a critic agent for external evaluation and targeted revision guidance. First, to overcome the lack of process-level supervision in existing sarcasm datasets, we synthesize reasoning annotations through a dynamic-role agentic rollout, in which a strong vision-language model sequentially spawns analytical roles within a shared context; the resulting trajectories are flattened into single sequences that preserve cross-perspective dependencies while enabling efficient autoregressive generation. Second, ProCrit adopts a draft–critique–revise paradigm in which the proposal agent dynamically elicits and progressively integrates sample-specific perspectives into a coherent analysis, while the critic agent evaluates the resulting reasoning and provides targeted natural-language feedback for directed revision. Finally, we develop a reciprocal training framework that jointly optimizes the proposal agent's drafting and feedback-guided revision via dual-stage reinforcement learning, while refining the critic agent according to the actual effectiveness of its feedback. Extensive experiments on three benchmarks demonstrate that ProCrit consistently outperforms reasoning-based methods, and human evaluation confirms substantially better-grounded rationales.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal sarcasm detection, Vision-language models, Self-elicited multi-perspective reasoning, Critic-guided revision, Process-level supervision, Reciprocal training, Reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Rs6uOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the issue that different samples in multimodal sarcasm detection require different analytical perspectives, which cannot be fully covered by predefined ones. It proposes a sample-adaptive dual-agent framework called ProCrit to achieve multi-perspective reasoning and revision, and presents a reciprocal training framework based on reinforcement learning. Extensive experiments were conducted on MMSD, MMSD2.0, and the cross-domain dataset RedEval to demonstrate the effectiveness of the framework.

Critical strengths
  1. To address the issue that different samples in multimodal sarcasm detection require different analytical perspectives that cannot be fully covered by predefined ones, the authors propose ProCrit, a framework with sample-adaptive analytical perspectives, and train it using reinforcement learning. Extensive experiments were conducted on MMSD, MMSD2.0, and RedEval, achieving promising results.

  2. Process-level reasoning trajectories are synthesized using GLM-4.6V, providing a feasible approach to constructing training data for multimodal sarcasm detection datasets lacking process-level annotations.

Critical weaknesses
  1. Although this paper conducts extensive comparative experiments and detailed ablation studies, the framework introduces substantial computational overhead, while its performance remains weaker than some existing task-specific models on in-domain datasets. In addition, the full training experiments are mainly based on Qwen2.5-VL-7B, lacking validation on newer open-source models such as Qwen3-VL. It is suggested to further evaluate the framework on newer backbone models and analyze the trade-off between performance improvements and computational cost.

  2. While this paper provides examples, human evaluations of perspective quality, and comparisons with fixed perspectives, but the sample adaptivity of the self-elicited perspectives lacks direct quantitative evidence. The differences between self-elicited and predefined perspectives in terms of adaptivity and diversity have not been sufficiently quantified.

Questions for authors
  1. Can the Critic's score truly reflect the quality of the reasoning process? How can the authors prevent the Critic's score from being overly influenced by the correctness of the final prediction? Could they report its agreement with human evaluations of reasoning quality?

  2. How can the correctness of process-level annotations be ensured? The process-level supervision in this paper was synthesized using GLM-4.6V and manually evaluated. However, for trajectories with correct final labels but incorrect reasoning steps, how can the authors prevent them from being treated as reliable supervision?

  3. How does the model achieve truly adaptive perspective selection? Can the diversity of perspectives across samples and the contribution of each perspective be quantified?

4 comments

We greatly appreciate the reviewer’s valuable comments and suggestions, which have helped us improve the clarity and quality of our manuscript. We address each point below.

W1: Newer backbone validation and performance–cost trade-off

Reply:

We have added experiments with Qwen3-VL-8B-Instruct, completing SFT and GRPO for the ProCrit proposal and three Reasoner baselines, and evaluating them alongside their zero-shot counterparts on MMSD2.0 and out-of-domain RedEval. ProCrit retains the existing frozen Qwen2.5-VL-7B critic; this comparison therefore evaluates adaptation to a newer proposal backbone. We follow the experimental setup used in our main experiments. The results are reported as Macro-F1 and Accuracy (%); Macro-F1 averages the F1 scores of the sarcastic and non-sarcastic classes.

ModelMMSD2.0RedEval
Macro-F1AccuracyMacro-F1Accuracy
Zero-shot Plain CoT72.373.181.882.4
Zero-shot Fixed-role70.570.579.079.1
Zero-shot Dynamic Draft71.071.983.483.9
Plain CoT Reasoner82.182.285.185.4
Fixed-role Reasoner81.781.884.184.9
Dynamic-role Reasoner82.882.886.086.3
ProCrit Draft82.282.386.486.7
ProCrit Revise83.783.885.785.9

Perspective design. On Qwen3, all three trained Reasoners outperform their zero-shot counterparts. Dynamic-role also outperforms both trained alternatives, exceeding Fixed-role by 1.1 and 1.9 points on MMSD2.0 and RedEval, respectively. These results support the usefulness of learned adaptive perspectives beyond the original backbone.

ProCrit training and revision. Relative to Dynamic-role, ProCrit Draft improves RedEval F1 by 0.4 points without an inference-time critic call, supporting cross-domain generalization through single-response inference. On MMSD2.0, ProCrit's advantage emerges after revision: F1 rises from 82.2 to 83.7, exceeding Dynamic-role by 0.9 points. ProCrit thus benefits initial prediction on RedEval and feedback-guided refinement on MMSD2.0, with different benefits across domains.

Performance–cost trade-off.

  1. Generalization and reasoning capabilities relative to task-specific discriminative baselines. The compared task-specific discriminative baselines primarily learn or enrich multimodal features for label prediction. ProCrit further incurs autoregressive generation to produce explicit, sample-adaptive reasoning that can be inspected and corrected, which is important for practical applications requiring human oversight. As shown in Table 13 (Appendix D.3), ProCrit draft achieves near-best cross-domain performance on RedEval, outperforming other discriminative baselines by 4.1–17.1 F1 points. These results demonstrate that ProCrit's benefits extend beyond in-domain accuracy to cross-domain generalization and explicit, revisable reasoning.

  2. Training and inference costs. Proposal training takes approximately 21.5 hours on eight GPUs, while single-response inference takes approximately 4.0 seconds per example. However, the trained critic can be reused across compatible reasoning backbones, as demonstrated above, and revision is optional, allowing users to balance performance requirements and computational budget.

We have added the Qwen3 results and analysis in Appendix D.12 (Table 25) .

W2 & Q3: Quantifying perspective adaptivity, diversity, and contribution

Reply:

Through training on dynamic-role trajectories (§3.1), ProCrit internalizes adaptive perspective selection into a single autoregressive sequence: each step is conditioned on the multimodal input and all preceding findings until the accumulated evidence suffices, while dual-stage GRPO and critic-guided revision (§3.3) reinforce perspectives that capture decisive sarcasm cues and recover omitted angles. Below, we quantify its adaptivity, diversity, and contribution relative to predefined perspectives.

Adaptivity. Adaptivity measures whether perspective selection responds to sample-specific needs. We quantify it as the average selection-rate gap between needed (Np=1N_p=1) and non-required (Np=0N_p=0) cases:

\frac{100}{|\mathcal{P}|} \sum_{p\in\mathcal{P}} \left[ \widehat{P}(S_p=1\mid N_p=1) - \widehat{P}(S_p=1\mid N_p=0) \right].$$

Using separate blind requests, GLM-5V-Turbo judged sample needs from image–text inputs and substantive perspective use (Sp=1S_p=1) from anonymized reasoning across 100 label-balanced teacher trajectories and valid MMSD2.0 test examples:

Trajectory sourceSamplesUse when needed (%)Use when not required (%)Adaptivity (points)
Teacher supervision (training set)10072.130.941.2
Predefined: ProCrit (Fixed) Draft (test set)10061.036.025.0
Self-elicited: ProCrit Draft (test set)10072.034.537.5

P\mathcal{P} covers pragmatic intent/tone, expectations/norms, external background knowledge, and humor vs. sarcastic intent; uncertain need judgments are excluded per category.

Even when predefined roles receive credit for flexible content within fixed steps, ProCrit Draft raises needed-perspective coverage by +10.9 points (72.0% vs. 61.0%, matching the teacher’s 72.1%) and improves Adaptivity by +12.4 points over ProCrit (Fixed) Draft (37.5 vs. 25.0).

Diversity. Unlike predefined methods that apply an identical 3-step schema to every input, clustering generated step titles across all 2,409 MMSD2.0 and 1,004 RedEval test drafts (with all-MiniLM-L6-v2, average-linkage cosine distance 0.35) shows substantial variation in both reasoning length and perspective combinations:

MetricMMSD2.0RedEval
Step-count distribution: 1 / 2 / 3 / 4+ steps (%)0.0 / 14.2 / 81.4 / 4.40.1 / 21.3 / 73.8 / 4.8
Mean steps: sarcastic / non-sarcastic2.98 / 2.842.96 / 2.75
Perspective clusters appearing in ≥10\ge 10 samples3219
Distinct perspective-cluster combinations451234
Between-sample Jaccard distance (3-step drafts)0.6160.627

Sarcastic samples elicit more steps on average, and the high Jaccard distances among 3-step drafts confirm that combination diversity persists at fixed reasoning length.

Contribution. We evaluate perspective contribution in both rationale quality and classification performance:

  • Step-level quality (Table 4; Appendix D.1, Table 10). Across 100 label-balanced MMSD2.0 test examples rated blindly by five annotators (0–5 scale), self-elicited perspectives achieve higher relevance, unique contribution, and necessity while nearly halving redundancy:
MethodRelevance ↑Unique contribution ↑Redundancy ↓Unsupported ↓Necessity ↑
Fixed-role Reasoner3.344.000.912.823.77
ProCrit Draft4.044.370.521.514.07
ProCrit Revise4.174.520.521.804.42
  • Task-level performance (Tables 1, 5, and 25). Under standard SFT+GRPO, Dynamic-role Reasoner exceeds Fixed-role Reasoner by +1.0, +0.5, and +2.1 F1 on MMSD2.0, MMSD, and RedEval. With identical critic and revision training, adaptive ProCrit still outperforms ProCrit (Fixed) by +0.93 F1 on both MMSD2.0 and RedEval.

We have included these analyses in Appendix D.13 (Tables 26–27), with the relevant content highlighted in blue.

Q1: Critic score validity and agreement with human evaluation

Reply:

The Critic is prevented from collapsing into a label-correctness proxy by three coupled designs: (1) SFT initializes evaluation from a 0–2 process rubric assessing interpretation accuracy, cross-modal reasoning, and coherence, explicitly instructing that a correct conclusion does not guarantee a high score (Prompt 6, Appendix C.3); (2) in RL, while ralignr_{\mathrm{align}} anchors scores to verifiable correctness, <score> is generated conditioned on <feedback>, which ractr_{\mathrm{act}} rewards only when it induces an actual error-fixing revision. Empirically, even though Proposal GRPO already includes binary label accuracy raccr_{\mathrm{acc}}, removing the Critic score reward revalr_{\mathrm{eval}} still drops MMSD2.0 Draft F1 from 81.1 to 79.9 (Table 17), showing that the score provides process-level signal beyond answer correctness.

Agreement with human evaluation. We matched the Critic’s scores (s∈{0,1,2}s \in \{0, 1, 2\}) against the five-annotator blind ratings (0–5 scale) on the 100 label-balanced MMSD2.0 ProCrit Drafts in Table 4 (Appendix D.1, Table 9), using per-example mean ratings and tie-corrected Spearman rank correlation:

Human-rated dimensionSpearman ρ\rho with Critic sspp-value
Overall quality (↑)0.36< 0.001
Evidence grounding (↑)0.34< 0.001
Logical coherence (↑)0.33< 0.001
Rationale–label consistency (↑)0.300.002
Core-cue coverage (↑)0.300.003
Hallucination severity (↓)−0.35< 0.001

Under the same rank-correlation metric, binary prediction correctness alone yields ρ=0.26\rho = 0.26 (p=0.010p = 0.010) with human Overall quality, whereas the Critic score reaches ρ=0.36\rho = 0.36, capturing reasoning quality variance beyond answer correctness.

Discrimination within correct predictions. Across all 2,409 MMSD2.0 test drafts, the Critic withholds the top score (s<2s < 2) on 13.3% (265 / 1,987) of correct predictions (s=0s = 0: 131; s=1s = 1: 134; s=2s = 2: 1,722). Within the 83 correct predictions in the 100-sample human-evaluated set, human quality ratings rise monotonically with ss (ρ=0.36\rho = 0.36, p=0.001p = 0.001):

Critic score ssnnEvidence grounding (↑)Hallucination (↓)Human Overall (↑)
061.371.800.80
162.301.131.80
2713.170.732.90

Even when final labels are identical and correct, drafts scored s=2s = 2 exceed those scored s<2s < 2 by +1.60 points in human Overall quality (2.90 vs. 1.30) with less than half the hallucination severity, confirming that the Critic retains sensitivity to reasoning quality within the same correctness level (see also Figure 8, Appendix E). We have added this analysis to Appendix D.1.

Q2: Quality and reliability of process-level supervision

Reply:

ProCrit promotes reliable process supervision through protocol-constrained synthesis and post-hoc label verification, without relying on final-label correctness alone. We first quantify the incidence of flawed steps in our synthesized supervision through human evaluation, and then summarize how ProCrit safeguards reasoning quality across data synthesis, reinforcement learning, and inference.

Human audit of label-correct trajectories. In our blind human evaluation of 100 label-balanced, label-correct GLM-4.6V training trajectories (Appendix D.1, Table 9), we group trajectories by their per-example mean ratings into three categories:

Category (n=100n = 100 label-correct trajectories)CountEvidence grounding ↑Logical coherence ↑Cue coverage ↑Hallucination ↓Overall ↑
Sound reasoning (Overall≥3.0\text{Overall} \ge 3.0)774.864.794.770.054.75
Grounded but incomplete cues (Overall<3.0, Hall.≤1.0\text{Overall} < 3.0,\ \text{Hall.} \le 1.0)142.102.230.330.290.36
Flawed / hallucinated steps (Hall.>1.0\text{Hall.} > 1.0)90.270.340.113.790.14
All audited trajectories (Table 9)1004.064.033.730.423.72

Overall, 91% of label-correct trajectories remain factually grounded (Hall.≤1.0\text{Hall.} \le 1.0), including 77% with thorough multi-perspective reasoning and 14% with valid observations that miss secondary cues, while 9% contain hallucinated or flawed intermediate steps despite reaching the correct label.

Safeguards across synthesis, training, and inference. Across the pipeline, three stages keep step-level errors low during synthesis and prevent the remaining 9% from degrading the final model:

  1. Protocol-constrained, label-blind synthesis and repair. During rollout, GLM-4.6V must contribute a distinct, context-conditioned perspective at each step without access to the gold label, which is verified only afterward to avoid post-hoc rationalization. Failed rollouts are regenerated using critic feedback rather than blind retries (Appendix B.1).
  2. Process-level differentiation during RL. SFT serves only as initialization. During GRPO, the critic’s quality score differentiates candidate drafts sharing the same correct prediction and assigns lower relative advantages to weaker rationales. Consistent with this design, RL improves MMSD2.0 Draft F1 from 78.1 to 81.1 over SFT alone (Table 2; Figure 9), and removing the critic evaluation reward lowers Draft F1 to 79.9 (Table 17).
  3. External critique at inference. At test time, drafts with weak reasoning—including correct predictions scored below 2 by the critic—are revised under targeted feedback, further reducing human-rated hallucination from 0.95 to 0.84 (Table 4; Figure 8).

We have added this analysis in Appendices B.2 and D.1.

Thank you again for your time and for your review. With the discussion period nearing its end, we would welcome any feedback on our response, especially if there are concerns that you feel still need to be addressed. We would be happy to follow up on those points.

Best regards,

The Authors

Reviewer_h8LvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProCrit for multimodal sarcasm detection. A proposal model generates sample-specific analytical perspectives, while an independent critic provides feedback for revising its reasoning and prediction. The authors synthesize reasoning trajectories with a teacher model for supervised fine-tuning, then use alternating reinforcement learning to optimize both models. The proposal learns drafting and feedback-guided revision jointly, and the critic is rewarded for feedback that helps correct predictions. Evaluation covers MMSD, MMSD2.0, and the out-of-domain RedEval benchmark, supported by ablations and human evaluation of explanations.

Critical strengths
  1. The method has a clear design rationale. To accommodate the different analytical perspectives that image–text pairs may require, the authors construct teacher-generated reasoning trajectories and convert them into output sequences for student training.

  2. The evaluation reports both the benefits and drawbacks of revision. Revision improves F1 on the two in-domain datasets but lowers it on RedEval. Human evaluation also shows slightly lower evidence grounding and logical coherence after revision, despite a higher overall rating. Reporting these trade-offs makes the empirical assessment more balanced.

Critical weaknesses
  1. The motivation for adaptive perspectives could apply to almost any reasoning task. It is unclear what particular difficulty in multimodal sarcasm detection leads to this design. Most components have close precedents in reasoning distillation, Critic-V, and RL Tango. The experiments also leave an important question unanswered: would a model using fixed perspectives perform similarly if given the same critic and revision training? Without this control, the gains cannot be clearly attributed to adaptive perspectives, which is the paper’s main claimed contribution.

  2. The paper does not compare with ConDi (TASLP 2025), a directly relevant baseline that also studies multi-view learning and cross-domain generalization on the same three datasets. Without this comparison, it is difficult to assess ProCrit’s advantages over recent strong methods.

Questions for authors
  1. What are the main differences between ProCrit and general-purpose reasoning frameworks such as Critic-V and RL Tango? Which specific challenges in multimodal sarcasm detection motivate these differences, and what evidence shows that the proposed designs address those challenges?

  2. How does ProCrit compare with ConDi (TASLP 2025) on the same data splits and evaluation metrics, particularly on RedEval? This comparison would help establish its advantage over a directly relevant method for multi-view learning and cross-domain generalization.

3 comments

We thank the reviewer for the careful and constructive comments. We address each point below.

W1&Q1: Task motivation and comparison with prior work

Reply:

Task motivation. Multimodal sarcasm detection requires identifying the incongruity between literal content and implied intent, whose evidence may lie in textual tone, visual context, or their interaction. Some samples require checking literal claims against visual evidence; others hinge on interpreting wordplay or exaggeration; still others require relating facial expressions and situational context to the speaker’s attitude. These analyses are also interdependent: establishing what the image depicts can provide the context needed to interpret a rhetorical expression and recover the intended attitude. The corresponding perspectives and how they should be combined therefore vary across samples.

These task requirements motivate the following methodological distinctions:

  • Beyond reasoning distillation. Reasoning distillation transfers teacher-generated rationales to a student. ProCrit makes perspective selection itself part of the learned reasoning process. Its distillation target explicitly represents successive perspective choices and their evidence-based analyses, supervising how the analytical focus develops from preceding findings.

  • Comparison with RL Tango [1]. RL Tango jointly trains a generator and verifier: verifier step judgments become process rewards for the generator, while the verifier is supervised by whether its final verdict matches the actual correctness of the generator’s answer. Instead, ProCrit makes critique part of a learned revision process: the proposal generates a new multi-perspective analysis conditioned on feedback, and the resulting revision outcomes supervise the critic’s corrective guidance.

  • Comparison with Critic-V [2]. Critic-V uses a trained critic to guide inference-time prompt updates. Instead, ProCrit makes the draft–critique–revise interaction part of proposal training, jointly optimizing drafting and feedback-conditioned revision. This teaches the proposal how to construct and revise a sample-specific multi-perspective analysis: which analytical angles to pursue and how to revisit missed or misread sarcasm cues using feedback.

Evidence. The perspective controls presented below support adaptive perspective selection under matched training. Within ProCrit, jointly optimizing the draft and revision objectives during GRPO improves in-domain F1 over optimizing either objective alone (Table 16). With the proposal held fixed, refining the critic using revision outcomes improves revision performance (Table 3). These results support learning both to construct multi-perspective analyses and to correct them using feedback.

We have expanded the discussion of the methodological distinctions in Appendix A.2–A.3.

[1] Zha et al. RL Tango: Reinforcing Generator and Verifier Together for Language Reasoning. NeurIPS 2025.

[2] Zhang et al. Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning. CVPR 2025.

W1: Contribution of adaptive perspectives

Reply:

To assess the contribution of adaptive perspectives independently of revision effects, we first compare three perspective settings under a common SFT+GRPO training protocol. We then train a fixed-perspective variant of ProCrit, denoted ProCrit (Fixed), with the same frozen critic, revision supervision, and dual-stage GRPO, to test whether fixed perspectives can match adaptive ProCrit. The results below report positive-class F1 and Accuracy (%).

ModelMMSD2.0RedEval
F1AccuracyF1Accuracy
Standard SFT+GRPO
Plain CoT Reasoner79.4080.3082.6084.80
Fixed-role Reasoner79.8080.8081.6083.90
Dynamic-role Reasoner80.8082.1083.7086.20
With critic and revision training
ProCrit (Fixed)†80.4681.6184.3986.55
ProCrit (Fixed)82.1782.9482.7784.86
ProCrit†81.1082.5084.8087.20
ProCrit83.1084.0083.7085.70

For ProCrit variants, † denotes Draft outputs; unmarked rows report outputs after one revision round.

Under standard SFT+GRPO, Dynamic-role Reasoner outperforms Fixed-role Reasoner by 1.0 and 2.1 F1 points on MMSD2.0 and RedEval, respectively, with higher Accuracy on both datasets. This comparison supports the contribution of adaptive perspectives without introducing critic-guided revision.

After one revision round, the reported adaptive ProCrit results exceed ProCrit (Fixed) by 0.93 F1 points on both datasets, alongside Accuracy differences of 1.06 and 0.84 points. In this comparison, the fixed-perspective variant remains behind adaptive ProCrit despite receiving critic and revision training. Together, these comparisons support a contribution from adaptive perspectives beyond the benefits of critic and revision training.

We have expanded the baseline comparisons and ablation analysis in Sections 4.1–4.3 (Tables 1 and 5).

W2&Q2: Comparison of multi-view learning in ProCrit and ConDi

Reply:

We have added a direct comparison with ConDi [3] in Appendix D.3 (Table 13), using its published MMSD2.0-trained results on the standard MMSD2.0 and RedEval test splits and reporting Macro-F1 and Accuracy (%).

On RedEval, ProCrit shows strong cross-domain generalization, with its draft achieving scores close to those of ConDi. Specifically, it obtains 86.9 Macro-F1 and 87.2 Accuracy, compared with 87.4 and 87.6 for ConDi, which leads on both benchmarks. Both ProCrit modes also outperform several task-specific classification baselines in both metrics.

The comparison also distinguishes two uses of multiple views: feature-based prediction in ConDi and explicit, revisable reasoning in ProCrit. ConDi learns incongruity features from three predefined views, namely modality disentanglement, global sentiment, and local description, and adaptively fuses their evidence for label prediction. ProCrit instead learns which analytical perspectives to introduce and how each builds on preceding findings, producing an explicit reasoning trajectory that is revised using critic feedback. ProCrit thus combines competitive cross-domain generalization with multi-view reasoning that can be inspected and corrected.

[3] Guo et al. Toward Generalized Multimodal Sarcasm Detection With Multi-View Learning. TASLP 2025

Thank you again for your time and for your review. Before the discussion period ends, we wanted to check whether there are any points in our response that you would like us to clarify. We would be glad to address any remaining questions.

Best regards,

The Authors

Reviewer_kYdEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Summary: ProCrit is a two-agent framework for multimodal sarcasm detection. Instead of relying on fixed analytical perspectives, a proposal agent generates sample-specific perspectives, while a critic evaluates the reasoning and guides revision.

Contributions: The paper introduces: (1) dynamic-role rollouts that synthesize process-level, multi-perspective reasoning annotations; (2) a draft–critique–revise workflow; and (3) reciprocal training that optimizes proposal drafting/revision and rewards critic feedback according to its corrective effect.

Validation: On MMSD2.0, MMSD, and out-of-domain RedEval, ProCrit generally outperforms zero-shot, fixed-perspective, and fine-tuned reasoning baselines. Ablations show benefits from adaptive perspectives, SFT/RL, external critique, and outcome-refined critic training. Human evaluation by five graduate annotators on 100 examples reports better grounding, coherence, cue coverage, and lower hallucination than CoT and fixed-role baselines.

Critical strengths
  1. Sample-adaptive reasoning design. ProCrit replaces fixed, manually defined analytical perspectives with perspectives generated according to each image–text pair and preceding reasoning steps. This is consequential because sarcasm cues vary substantially across samples, making adaptive analysis better suited to the task than rigid perspective templates.

  2. Strong and multifaceted validation. The evaluation spans three benchmarks, including an out-of-domain dataset, with controlled baselines, component ablations, and human rationale assessment. Results support both improved detection performance and better-grounded reasoning, providing evidence that the gains arise from adaptive perspectives and external critique rather than an additional generation pass alone.

Critical weaknesses
  1. Missing essential baseline comparisons. The experiments do not adequately isolate the benefits of the dual-agent architecture, specialized training, and teacher distillation. The paper lacks comparisons with
  • untuned Proposal–Critic agents using role-specific prompts,
  • and direct agentic inference using GLM-4.6V or another stronger model.

Because the reported zero-shot and proprietary-model baselines do not implement the same draft–critique–revise workflow, they are not directly comparable. Without these controls, it remains unclear whether ProCrit’s gains require two independently fine-tuned agents or primarily result from teacher supervision, additional computation, or model capacity.

  1. We agree with the authors that reasoning generation and reasoning evaluation should be decoupled. However, the paper does not sufficiently examine whether two independent Qwen2.5-VL-7B agents are necessary. It remains unclear how a unified model, jointly optimized to perform both the proposal and critic roles through role-specific prompts or objectives, would compare with the proposed dual-agent architecture in terms of performance, stability, and computational efficiency.
Questions for authors

See Weaknesses:

  1. untuned Proposal–Critic agents using role-specific prompts.

  2. direct agentic inference using GLM-4.6V or another stronger model.

  3. a unified model, jointly optimized to perform both the proposal and critic roles through role-specific prompts or objectives

4 comments

We are deeply grateful for the insightful comments and suggestions. We address each point below.

W1: Baselines under the Draft–Critique–Revise workflow

Reply:

To address the missing comparisons under a common Draft–Critique–Revise workflow, we evaluated two additional baselines: untuned Qwen2.5-VL-7B agents and direct agentic inference with GLM-4.6V, the teacher used to generate our training annotations. Both baselines use ProCrit’s role-specific prompts and one round of revision, with the corresponding model performing both proposal and critic roles for fair.

The results are reported below as positive-class F1 and Accuracy (%).

ModelMMSD2.0RedEval
F1AccF1Acc
Untuned Qwen2.5-VL-7B agents
Qwen2.5-VL-7B (Draft)61.3468.8569.5075.70
Qwen2.5-VL-7B (Revise)64.5068.2070.5974.60
Direct agentic inference with GLM-4.6V
GLM-4.6V (Draft)72.2774.4973.0276.22
GLM-4.6V (Revise)72.7474.3273.2775.98
ProCrit (Draft)81.1082.5084.8087.20
ProCrit (Revise)83.1084.0083.7085.70

On MMSD2.0 and RedEval, respectively, ProCrit Revise exceeds untuned Qwen2.5-VL-7B Revise by 18.60 and 13.11 F1 points, and GLM-4.6V Revise by 10.36 and 10.43 points. Even ProCrit Draft outperforms GLM-4.6V after revision on both datasets.

Thus, providing the untuned backbone or the annotation teacher with the same role-specific prompts and Draft–Critique–Revise workflow is insufficient to match the trained ProCrit agents. We have expanded the manuscript to include these comparisons alongside the existing proprietary-model baselines, as shown in Table 14 (Appendix D.3).

W2: Unified and dual-agent implementations

Reply:

We additionally train Unified ProCrit, a single Qwen2.5-VL-7B model jointly fine-tuned on proposal, revision, and critique examples, followed by staged reinforcement learning with role-specific objectives. The unified variant retains ProCrit’s supervision sources, training objectives, and Draft–Critique–Revise workflow, while sharing one parameter set across the proposal and critic roles.

The results are reported below as positive-class F1 and Accuracy (%); ΔF1 denotes Revise−Draft in F1 points.

Configuration / OutputMMSD2.0RedEval
F1AccΔF1F1AccΔF1
Unified ProCrit
Draft82.083.1—83.185.4—
Revise83.183.9+1.183.084.9−0.1
Dual-Agent ProCrit
Draft81.182.5—84.887.2—
Revise83.184.0+2.083.785.7−1.1

The two variants achieve broadly comparable performance. Unified ProCrit’s generation parameters also receive critique supervision. This broader supervision may contribute to Unified ProCrit’s stronger Draft performance on MMSD2.0. Dual-Agent ProCrit nevertheless achieves a larger revision gain on MMSD2.0 (+2.0 vs. +1.1 F1 points), reaching nearly identical Revise F1, and retains higher Draft and Revise performance on RedEval. These results suggest that a separately optimized critic may provide complementary feedback that benefits refinement and out-of-domain generalization.

For training stability, Unified ProCrit preserves sequential RL stages with frozen counterpart snapshots; both stages completed with finite losses and gradient norms throughout. For computational efficiency, both variants use the same backbone for three-stage inference, with comparable computation at matched sequence lengths. Parameter sharing halves distinct model-weight storage. Thus, Unified ProCrit offers a compact implementation while preserving ProCrit’s decoupled generation and evaluation roles. We have added this comparison to the manuscript, as shown in Table 15 (Appendix D.4).

Thank you again for your time and for your review. As the discussion period draws to a close, we wanted to follow up and ask whether our response has addressed your concerns. If any questions remain, we would be happy to clarify them.

Best regards,

The Authors

Apologies for my late reply, and thank you for the authors’ detailed responses. However, I do not believe the additional experiments adequately address my concern about whether two independent agents are necessary, given the very small performance gain. Moreover, I do not believe the paper currently warrants a score of 4. Therefore, I will maintain my original score: 3.

Dear Reviewer,

Many thanks for your reply and positive assessment. Your insightful suggestion of a unified model genuinely strengthened our paper: we agree that ProCrit’s core gain comes from decoupling generation and critique roles rather than duplicating parameters, and Unified ProCrit confirms this while providing a compact single-7B option. Meanwhile, a separate critic still offers distinct practical value—for example, our frozen Qwen2.5-VL-7B critic directly trains and guides a Qwen3-VL-8B proposal without critic retraining (Appendix D.12, Table 25), alongside higher out-of-domain performance on RedEval (84.8 vs. 83.1 Draft F1).

Following your feedback, we will explicitly clarify both regimes in the main text: Unified ProCrit for memory-efficient deployment on a fixed backbone, and Dual-Agent ProCrit for plug-and-play critic reuse across backbones and stronger cross-domain transfer.

Thank you again for your valuable feedback, and we would sincerely appreciate it if you might reconsider your evaluation in light of this.

Comments on the paper

1 comment
Area_Chair_9Bw7OpenReview ↗

Reminder: Discussion deadline approaching

Dear Reviewers,

Thank you for your time and effort in reviewing Submission 32939. As the discussion deadline is approaching, could you please read the authors’ response and any additional evidence, reply in the discussion to indicate whether your concerns have been addressed, and update your evaluation accordingly as soon as possible? Thank you for your contribution to the review process.

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.