- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper tackles the problem of LLM-based persuasion in a multi-receiver setting. Specifically, it proposes a Reinforcement Learning-based framework where the LLM can select multiple receivers, further integrating social context by means of a graph.
The framework is applied over three different domains, spanning political campaigns, academic rebuttal, and charity context. Method effectiveness is validated in terms of some features (e.g., clarity, success, persuasiveness) against a set of baselines.
Critical strengths
The underlying idea which forms the basis of this paper is intriguing. Effectively inducing LLM persuasion over multiple heterogeneous peers by further integrating social context is valuable and highly impactful.
Critical weaknesses
The writing quality of the paper is unfortunately poor, heavily affecting comprehension.
Overall, the text is full of overly generic and non-technically sound statements, with several concepts never properly defined (e.g., success signal , the "actor") or described in a shallow manner ("the environment validates the output", "the key is the corresponding context of the target receiver").
More crucially, it is not clear to this reviewer why the discrepancy in terms of stance should represent a good proxy in terms of persuasion. The paper claims that higher values of the stance score indicate greater persuasiveness, but why? Stance is merely a representation of the node inclination towards a given topic. If is maximized, this basically implies that the stance at converge will be higher than the original one, without taking account any target final value, or the semantics underlying the score.
Additionally, there is an important mismatch between the provided Figure and the main content. In the figures, to the best of my understanding, the graph G is actually labeled with labels representing the node inclination towards the given topic (e.g., supportive/opposing). However, the text never mentions such labels and how they are estimated.
Next, the findings reported in the experimental section are weak. The paper claims that integrating social context "improves" Pearson correlation and lowers MSE, but (i) most of the reported correlations are either low or even absent (approaching 0), and (ii) it is not showed that the reduction in terms of MSE is statistically significant. Finally, it is never clarified how the "quality" features used to validate the method are defined nor quantified. Similar statistical concern holds for the results reported in Table 1, whose values are often similar and reported intervals tend to overlap.
Response to Reviewer EqdF
Thank you for your review. We clarify the definitions and evidence below.
W1: Definitions and figures.
The actor is πθ; ηt denotes format and receiver validity in the transition/reward definitions. The evaluation appendix defines Success aggregation. Figure legends identify node stances and the selected target; directed edges denote observation links. Colors visualize stance values rather than a separate classification task.
W2: Stance and quality.
Stance denotes acceptance rating, policy support, or donation willingness. Thus, Δsi=sifinal−siinitial measures movement toward the domain objective. Quality separately averages Attitude, Clarity, Constructiveness, and Persuasiveness equally, using the appendix's rubrics and 0–10 scales; unexecuted steps receive zero.
W3: Statistical interpretation.
The matched ablation holds the model, examples, and other prompt inputs fixed. Social context raises Pearson correlation by 0.08/0.06/0.18 and lowers MSE by 0.02/0.04/0.41 on Paper/Political/Charity. These are observed improvements in all three domains; statistical significance of each difference has not been established.
Thank you again for your feedback.