acceptodds
Under review as a conference paper at ICLR 2027

Where Do Test-Time Scaling and Training Fall Short in Individual Stance Prediction?

14 views

Abstract

Test-time scaling and post-training have improved LLM performance in coding and mathematical reasoning, but their effectiveness for individual stance prediction remains unclear. We study this question by predicting a person's stance in a new discussion from their history. We evaluate widely used test-time scaling strategies and post-training methods, such as supervised fine-tuning and reinforcement learning, and identify four failure modes across generation, selection, and learning: (1) incorrect consensus, where repeated samples agree on the wrong stance; (2) selection failure, where generation covers the observed stance but selection misses it; (3) response overfitting, where supervised fine-tuning improves imitation but harms prediction; and (4) early plateau, where reinforcement learning shows modest initial gains followed by limited further improvement. We expose these failures using \benchmark, which contains 2499 prediction tasks from 500 Hacker News users. Guided by this analysis, we explore a simple approach that combines direct scores for all candidate stances with explicit assessments of support from the individual's history. On the 781-task test set, this approach achieves 21.83 discussion-specific Macro F1 with Qwen3-8B, compared with 19.27 for direct scoring. Our results motivate evaluating candidate generation, final selection, and person-specific evidence use separately.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large language models, Social simulation, Test-time scaling, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3FTaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper investigates the effectiveness of test-time scaling and post-training for individual stance prediction: given a user’s discussion history, a target discussion, and its candidate stances, the task is to predict the stance the user expresses conditional on participation. The authors introduce STANCE-BENCH, comprising 2,499 prediction tasks from 500 Hacker News users, and separately record candidate coverage and final selection. Across several inference-time strategies and learning methods, the paper reports four phenomena: incorrect consensus, selection failure, response overfitting, and an early plateau in reinforcement learning. It further explores a method that combines direct label scores with comment-level assessments of historical evidence. On the 781-task test set, this method achieves a discussion-specific Macro F1 of 21.83, compared with 19.27 for direct scoring, although the primary paired comparisons do not pass Holm correction. The main contribution is a diagnostic analysis of individual stance prediction, supported by a benchmark and an exploratory method.

Critical strengths

1.The paper separately evaluates candidate generation and final selection in individual stance prediction, providing a more informative analysis than final accuracy alone. Figure 3 shows that some methods are primarily limited by insufficient candidate coverage, whereas others improve coverage but fail to select the target user’s stance effectively. This decomposition helps explain distinct sources of failure behind similar aggregate performance and provides concrete directions for subsequent method development. 2.The experiments cover several inference-time strategies across three models, as well as verification, supervised fine-tuning, reinforcement learning, user profiles, and activation steering on Qwen3-8B. Several experiments report results over three seeds, and the paper provides matched controls for some comparisons. The evidence-fusion method includes branch ablations, history replacement, and prompt-concatenation controls. The authors also explicitly disclose that the primary comparisons do not pass Holm correction. These choices strengthen the study’s diagnostic value and help readers interpret the exploratory findings appropriately.

Critical weaknesses

1.The central findings depend on the reference labels and evaluation protocol of STANCE-BENCH, yet the current manuscript does not provide sufficient validation information. Section 3 describes model-assisted annotation and the use of target-user contributions to audit and revise stance taxonomies, but it does not report the size of the human audit, agreement measurements, or correction outcomes. Page 10 mentions volunteer annotators without supplying this supporting evidence.The main text and appendix also do not fully specify the training, validation, and test partitions; user and discussion overlap across partitions; the construction and isolation of additional training tasks for the profile methods; or the filtering statistics for OTHER examples. With only 781 test tasks distributed across 497 reference-supported discussion–stance classes, the resampling unit and treatment of within-user and within-discussion dependence are particularly important, but remain unspecified. These omissions limit the assessment of label reliability, generalization, and small performance differences. They do not establish that leakage occurred, but prevent readers from adequately ruling out such concerns. 2.Most test-time methods are evaluated only with four candidates, without an analysis across candidate budgets. The PlanSearch adaptation is particularly revealing: diverse generation followed by label voting results in selection of the first candidate on 774 of 781 Flash tasks (§4.2). This clearly demonstrates a limitation of that selection rule, but does not establish that a more suitable selector or a larger budget would also fail to improve prediction.For SFT, the training objective covers the comment-and-label response sequence, whereas evaluation uses only the label. The paper also reports more OTHER predictions, invalid outputs, and generation truncation. Although a matched base-model control is provided, the absence of an error breakdown and a direct label-supervision control makes it difficult to determine how much of the degradation reflects stance discrimination itself. For GRPO, sparse reward contrast co-occurs with a plateau within the 300-step training horizon, but a causal relationship is not established. Thus, the results primarily support descriptive findings for the evaluated configurations; broader interpretations of the failure mechanisms require stronger controls or more explicit qualification.

Questions for authors

1.Please provide the human-audit sampling procedure, sample size, number of annotators, agreement measurements, and correction outcomes. How are user contributions containing multiple comments or positions reduced to a single dominant label? How many discussion taxonomies were revised after inspecting target-user contributions, and do these revisions affect the reported diagnostic findings? This information would directly affect my assessment of benchmark validity. 2.Given that PlanSearch almost always selects the first candidate on Flash, do the main findings persist when the candidate pool is held fixed and selection is performed by a history-conditioned selector? Are results available for candidate coverage and final accuracy across different candidate budgets, to distinguish limitations of the fixed four-candidate setting from broader limitations of test-time scaling? 3.How much of the SFT degradation is attributable to incorrect listed stances, OTHER predictions, formatting errors, and truncation, respectively? Does the degradation persist under consistent output constraints or direct stance-label supervision? For GRPO, does the available evidence support only co-occurrence between sparse reward contrast and the plateau, or are there controls that distinguish the effects of reward structure, training budget, and optimization configuration?

3 comments
AuthorsOpenReview ↗

Thank you, the key question is how robust the findings are, and here we report the human audit, bootstrap, and new controls.

W1/Q1 · Benchmark validity. Five annotators collectively labeled all 881 test tasks with a listed stance or OTHER, using the same inputs as the LLM labeler. Agreement with reference labels was 93.3% (95% CI [91.5, 94.8]), compared with 12.8% expected by chance (Brennan–Prediger S = 0.92). All five labeled a common subset of 337 tasks, with 89.3% inter-annotator agreement. They resolved disagreements by discussion.

The labeler reads all of a user’s comments in a discussion and assigns the dominant position the user argues for. A position mentioned only to reject it is excluded. A second applicable stance is stored separately. In 92.8% of annotations, exactly one stance applies.

A discussion’s taxonomy was revised once if more than 15% of its contributions, including target-user contributions, were labeled OTHER. Revisions could add stances and relabel OTHER contributions. Before the human audit and model experiments, 66 of 1,100 discussions (6.0%) gained 81 stances and 295 of 8,980 annotations (3.3%) changed.

We split users into 200 training, 50 validation, and 250 test users. Each task follows its user’s split. Of the 781 listed-stance test tasks, 91.8% share a discussion with a development task. Test reference labels are held out from training, and visible target context excludes benchmark users’ comments. The profile methods use 694 extra training and 154 extra validation tasks from earlier labeled discussions of users in the corresponding splits. History ends before the user’s first comment in each new target, and these tasks do not duplicate benchmark tasks. Of 1,249 test tasks, 781 have a listed stance, 100 have OTHER, and 368 have NO_STANCE. The main metrics use the 781 tasks, and secondary Macro F1 uses all 881 listed-stance or OTHER tasks. Training retains OTHER and excludes NO_STANCE. On the 781-task subset, OTHER predictions count as errors.

Excluding the 123 tasks in revised discussions or the 52 tasks with visible comments posted after the user’s first target comment preserves the direction of all four Macro F1 comparisons below. Values are point differences.

ComparisonAllExclude revisedExclude later
Comment + label SFT vs base-2.18-2.30-2.70
Label-only SFT vs base-1.89-1.87-2.11
GRPO vs base+0.80+0.62+1.19
Ours vs direct scores only+2.83+2.32+2.31

The original bootstrap resamples users, keeping each user’s tasks together. We also ran 2,000 bootstrap replicates resampling the 245 test discussions, keeping each discussion’s tasks together. Below are the resulting 95% intervals for paired differences, in points.

ComparisonMetricDifference95% CI
Ours vs direct scores onlyMacro F1+2.83[0.67, 4.72]
Ours vs different-user historyMacro F1+1.89[0.13, 3.68]
GRPO vs baseMacro F1+0.80[-0.71, 2.35]
Comment + label SFT vs baseMacro F1-2.18[-4.59, 0.27]
Comment + label SFT vs baseAccuracy-5.63[-9.16, -2.22]

We will report both resampling analyses in the revision.

W2/Q2 · Selection and budgets. On fixed Qwen3-8B PlanSearch pools, we compared voting with two selectors that read the user’s history. All values are percentages.

kCoverageVote = firstVote acc.Likelihood acc.Reviewer acc.
472.097.023.126.124.2
896.849.022.225.923.5
1698.642.324.525.625.1

All selectors scored below direct generation (26.9%). At k = 4, likelihood accuracy was 25.1% without history and 23.9% with another user’s history. The reviewer stayed near 24%. With independent samples at k = 1/2/4/8/16, coverage was 28.0/37.5/45.8/53.8/61.6%, while majority and likelihood accuracy stayed within 26.4–28.6%.

Q3 · SFT and GRPO. Of comment-and-label SFT errors, 88.4% were wrong listed stances, 9.6% OTHER, 0.8% truncations, and 1.1% other invalid outputs. Using 713 training tasks, 270 updates, and three seeds per objective, generation accuracy was 20.4% for comment and label, 20.9% with 30× label weighting, and 20.5% for label only, versus 26.0% for the base. Label-only SFT produced no invalid or truncated outputs. Direct scoring of listed labels gave 25.5%, 26.6%, and 27.3%, versus 25.9% for the base.

We do not claim that sparse reward contrast caused the GRPO plateau. In the same three 300-step runs, most validation gains occurred by step 150 and reward contrast stayed sparse. In exploratory runs with NO_STANCE as a candidate, the exact-match with KL, prior-corrected, coverage-dual, Brier, and Dr. GRPO variants all scored below their matched bases. These runs did not isolate the effects of reward, optimization, and training budget. We will add the analyses and qualify the claims for the evaluated configurations.

AuthorsOpenReview ↗

Kind Reminder and Summary of Our Response

Dear Reviewer 3FTa,

Thank you again for your instructive review. As the discussion period is closing soon, we would like to kindly check whether our responses have addressed your concerns. We summarize them briefly below.

  • W1, Q1. Five annotators collectively labeled all 881 test tasks. Agreement with the reference labels was 93.3% (Brennan-Prediger S = 0.92), and inter-annotator agreement was 89.3% on a common 337-task subset. Exactly one stance applies in 92.8% of annotations, and taxonomy revisions affected 66 of 1,100 discussions (6.0%). Under a discussion-level bootstrap, our method keeps a +2.83 Macro F1 gain over direct scores only.
  • W2, Q2. On fixed Qwen3-8B PlanSearch pools at k = 4/8/16, coverage is 72.0/96.8/98.6%. Voting and two history-reading selectors all stay below direct generation (26.9%).
  • Q3. Of comment-and-label SFT errors, 88.4% are wrong listed stances, and label-only SFT reaches 20.5% accuracy versus 26.0% for the base. For GRPO, the evidence supports co-occurrence only, and we will qualify the claim for the evaluated configurations.

We would be glad to clarify any remaining point, and would sincerely appreciate it if you could consider updating your assessment.

Best regards,

The Authors

AuthorsOpenReview ↗

A gentle follow-up before the discussion closes

Dear Reviewer 3FTa,

We hope this message finds you well. We understand this is a busy time, and we just wanted to gently follow up on our response.

Any thoughts you could share before the discussion closes would be greatly appreciated. Thank you again for your time and helpful feedback.

Best regards,

The Authors

Reviewer_37ZaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines whether test-time scaling and post-training improve individual stance prediction from a user’s comment history, assuming participation in a new discussion. To investigate this question, the authors construct STANCE-BENCH, comprising 2,499 prediction tasks from 500 Hacker News users and 1,100 annotated discussions. They analyze candidate coverage and final prediction accuracy separately and track training behavior under SFT and RL. The results show that repeated sampling often favors an incorrect stance, while diverse candidate sets frequently include the reference stance without selecting it. Comment-and-label SFT also reduces prediction accuracy, and GRPO provides modest gains concentrated early in training. Motivated by these findings, the authors explore combining direct label scores with relevance-weighted evidence scores derived from historical comments. This approach achieves 28.04% accuracy and 21.83 discussion-specific Macro F1 with Qwen3-8B on 781 test tasks.

Critical strengths

Informative failure analysis

Figure 3 shows substantial differences between candidate coverage and final prediction accuracy. For example, Qwen3-8B PlanSearch reaches approximately 70.6% candidate coverage and 22.4% final accuracy. This distinction identifies opportunities to improve selection when the reference stance is already available. The training diagnostics also provide information beyond final scores by reporting response-token loss, validation performance, and within-group reward variation.

Transparent empirical evaluation

The experiments cover several inference and adaptation strategies, include repeated runs for multiple Qwen configurations, and provide decoding-matched base comparisons for important training results. History substitution and branch removal examine the contribution of additional personal evidence. Appendix C.4 explicitly reports confidence intervals and the failure of the two primary comparisons to pass Holm correction. This transparency supports a measured interpretation of the proposed method.

Critical weaknesses

Insufficient benchmark validation

The evaluation relies on GPT-5.5-assisted stance categories and reference labels, yet the paper provides no quantitative human-validation results. It also leaves important aspects of dataset partitioning, OTHER exclusions, and the timing of visible discussion comments unspecified. Since target contributions inform category revisions, the frequency and effect of these revisions require clarification. These omissions limit confidence in the benchmark and the resulting failure analysis.

Limited support for failure explanations

Several findings depend on specific implementation choices. PlanSearch selects the first candidate on 774 of 781 Flash tasks because its voting procedure frequently encounters ties. SFT errors include invalid outputs and truncation before label generation, leaving their contribution to the performance decline unclear. Most inference comparisons also use only four candidates, providing limited evidence about scaling behavior. For GRPO, sparse reward differences and limited later improvement occur together, but their causal relationship remains untested. Alternative selection rules, label-focused SFT, and varying computation budgets would strengthen the analysis.

Questions for authors
  1. What quantitative human-validation results support the stance categories and reference labels? Please report the sample size, agreement results, and disagreement-resolution procedure, including how frequently categories were revised after examining target contributions.

  2. How were users, discussions, and tasks divided into training, validation, and test sets? Please clarify overlap rules, OTHER exclusions, and whether all visible target-discussion comments preceded the target user’s first contribution.

  3. What proportions of SFT errors arise from OTHER predictions, invalid formatting, and truncation before label generation? This breakdown would clarify how much of the performance decline reflects incorrect predictions among the listed stances.


(I was unable to access the provided Anonymous GitHub repository, as it remained stuck at 0% during preparation ('The repository will take a few minutes to get ready...'). If my question is already answered in your code, please feel free to point it out.)

8 comments
AuthorsOpenReview ↗

Thank you, the key concern is benchmark validity, and here we report human agreement and clarify the evaluation protocol.

W1/Q1/Q2 · Benchmark validation and evaluation. Five annotators collectively labeled all 881 test tasks with a listed stance or OTHER, using the same inputs as the LLM labeler. Agreement with the reference labels was 93.3% (95% CI [91.5, 94.8]), compared with 12.8% expected by chance (Brennan–Prediger S = 0.92). All five annotated a common subset of 337 tasks, with 89.3% inter-annotator agreement. They resolved disagreements by discussion.

During labeling, we revised a discussion’s stance options once if more than 15% of its contributions, including those of target users, were labeled OTHER. Revisions could add options and relabel OTHER contributions. Before the human audit and model experiments, 66 of 1,100 discussions (6.0%) gained options and 3.3% of annotations changed.

We split users into 200 training, 50 validation, and 250 test users. Each task follows its user’s split. Discussions can span splits: 91.8% of listed-stance test tasks share a discussion with a development task. Test reference labels are held out from training. Target context excludes benchmark users’ comments. Of 1,249 test tasks, 781 have a listed stance, 100 have OTHER, and 368 have NO_STANCE. The main evaluation uses the 781 listed-stance tasks. Secondary Macro F1 includes the 100 OTHER tasks.

The user’s last comment in each history discussion precedes their first target comment. Of the 781 listed-stance tasks, 52 (6.7%) contain a visible target-thread comment posted after that first comment. Each of the following comparisons has the same direction after excluding those 52 tasks or the 123 tasks from revised discussions. Differences are in Macro F1 points.

ComparisonAll 781Exclude later commentsExclude revised discussions
Comment + label SFT vs base-2.18-2.70-2.30
Label-only SFT vs base-1.89-2.11-1.87
GRPO vs base+0.80+1.19+0.62
Ours vs direct scores only+2.83+2.31+2.32

W2/Q3 · Selection, SFT, and GRPO. With Qwen3-8B PlanSearch at k = 4/8/16, coverage was 72.0/96.8/98.6%, and the voted label matched the first candidate on 97.0/49.0/42.3% of tasks. Voting accuracy was 23.1/22.2/24.5%. On the same pools, likelihood reached 26.1/25.9/25.6% and an LLM reviewer reached 24.2/23.5/25.1%, all below 26.9% for direct generation. With independent samples, increasing k from 1 to 16 raised coverage from 28.0% to 61.6%. Majority vote and likelihood accuracy stayed within 26.4–28.6% across budgets.

For comment-and-label SFT, 88.4% of errors were wrong listed stances, 9.6% were OTHER, 0.8% were truncations before the label, and 1.1% were other invalid outputs. Across three seeds per objective, generation accuracy was 20.4% for comment and label, 20.9% with 30× label weighting, and 20.5% for label only, versus 26.0% for the matched base. Label-only SFT produced no invalid or truncated outputs. Direct scoring of listed labels with the same checkpoints gave 25.5%, 26.6%, and 27.3%, versus 25.9% for the base.

GRPO. We do not claim that sparse reward contrast caused the plateau. Section 4.4 reports two observations from the same three 300-step runs: most validation gains occurred by step 150, and reward differences within rollout groups remained sparse. Under an earlier protocol that included NO_STANCE, the prior-corrected, coverage-dual, and Brier variants each scored below their matched bases. We will add the analyses to the appendix and clarify the GRPO claim in Section 4.4 and Figure 1.


Thank you for flagging the access issue. The anonymous GitHub repository is now accessible. It may take a moment to load because it contains several large data archives.

AuthorsOpenReview ↗

Kind Reminder and Summary of Our Response

Dear Reviewer 37Za,

Thank you again for your insightful review. As the discussion period is closing soon, we would like to kindly check whether our responses have addressed your concerns. We summarize them briefly below.

  • W1, Q1. Five annotators collectively labeled all 881 test tasks. Agreement with the reference labels was 93.3% (Brennan-Prediger S = 0.92), and inter-annotator agreement was 89.3% on a common 337-task subset. Taxonomy revisions affected 66 of 1,100 discussions (6.0%) and 3.3% of annotations.
  • Q2. We split by user into 200 training, 50 validation, and 250 test users. All four main comparisons keep their direction after excluding the 52 tasks with later visible comments or the 123 tasks from revised discussions.
  • W2. With Qwen3-8B PlanSearch at k = 4/8/16, coverage rises to 98.6%, while voting, likelihood, and reviewer selection all stay below direct generation (26.9%). We do not claim that sparse reward contrast caused the GRPO plateau.
  • Q3. Of comment-and-label SFT errors, 88.4% are wrong listed stances. Label-only SFT produces no invalid or truncated outputs and still reaches 20.5% accuracy, versus 26.0% for the matched base.

The anonymous repository is also accessible now. We would be glad to clarify any remaining point, and would sincerely appreciate it if you could consider updating your assessment.

Best regards,

The Authors

Thank you for the rebuttal and additional analyses. The human validation results and new controls address my initial concerns, and I do appreciate your additional efforts.

One remaining question is whether the gains genuinely reflect person-specific prediction. While the different-user ablation is encouraging, it only replaces history in the evidence branch. I wonder whether a full-history substitution across both branches, keeping the target discussion fixed, could provide stronger evidence for the central claim.

I believe this would further strengthen the paper and positively influence my assessment. Please take your time if you decide to investigate this further. I will continue checking for additional responses until the rebuttal deadline.

Reviewer_MrU1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper evaluates test-time scaling and post-training methods for individual stance prediction, analysed different failure scenarios, and propose analysis guided method to improve the prediction task.

Critical strengths
  1. STANCE -BENCH is a useful benchmark as it links many users to the same naturally occurring discussion (500 users, 1,100 discussions, 2,499 tasks).

  2. The failure analysis is a reusable way to analyze where methods fail, where each of the four modes are backed by different evidences.

  3. The authors proposed a simple, lightweight method where the LLM stays frozen, and coefficients are fitted, which avoids the SFT and RL problems and is easy to reproduce. The authors also honestly reported that their method should be considered a probe, as comparisons did not pass Holm correction.

Critical weaknesses
  1. The stance prediction task is hard to understand as presented. The introduction describes it abstractly (predicting a person’s stance in a new discussion from their history) without a concrete example. A short worked example in the introduction, showing a user’s history, a new thread, the candidate stances, and the correct answer, would make the paper much easier to follow.

  2. The task predicts a user’s stance in one discussion from their comments in other discussions, so it depends heavily on how related those topics are. In the current setup, generation can pick up a user’s writing style and expressions from earlier comments, but this is not the focus of the work, and it does not appear to help stance evaluation. The low performance across all methods is consistent with this. Topic relatedness between history and target may have played a significant role (see Questions for Authors).

  3. The benchmark is useful. However, the task setup could be more logically centered on the user’s stance toward the target discussion topic, or at least present more clearly what each discussion thread contributes as input.

Questions for authors

The history given to the baselines is the 8 most recent annotated discussions, chosen by most recent discussions. Have you checked how these 8 selected discussions are similar to or different from the target discussion? If so, could you report it, along with performance split by low, medium, and high similarity?

5 comments
AuthorsOpenReview ↗

Thank you, the key question is how the task works and whether related history helps, and here we clarify the task with an example

W1 · Worked example. We will add this example to the introduction.

  • Target thread: “U.S. government will decide who gets to use GPT-5.6.”
  • Question: How should we interpret the government’s decision to limit access?
  • Candidate stances: L1 Political overreach, L2 Regulatory capture, L3 Understandable safety restrictions, L4 Pushes users toward open or Chinese models, L5 Needs clear rules for restricting access, and OTHER.
  • User history: In an earlier thread about the suspension of Fable 5 and Mythos 5, the user argued that U.S. providers could cut off access and had therefore lost their trust. The annotated stance was “US AI is now untrustworthy.” The model receives this and seven other annotated discussions.
  • Reference label: L4. In the held-out contribution, the user argued that U.S. providers would lose customers through many individual purchasing decisions and said they already buy non-American digital services.

W2/Q1 · Topic relatedness. We grouped all 781 test tasks by the number of history discussions in the target’s topic domain and by the mean TF-IDF cosine similarity between the target and each of the eight history discussions, using titles and central questions. We split text similarity into thirds. Values are accuracy (%).

GroupnGemini 3.7 FlashGemini 3.1 ProQwen3-8BOurs
Same domain: 016126.126.127.729.8
Same domain: 1–226224.822.930.526.7
Same domain: 3–420128.424.925.927.9
Same domain: 5–815732.524.824.828.7
Text similarity: low26128.024.925.826.1
Text similarity: medium26023.820.831.430.0
Text similarity: high26030.827.725.628.1

Accuracy shows no consistent trend across models and remains low even in the highest-overlap and highest-similarity groups. We will add these results to the appendix.

W3 · What each thread contributes. The target thread gives the model a question and a set of possible answers. The user’s earlier threads show what they said and which positions they took on other questions. The model uses that history to predict which answer best matches the user’s held-out comments in the target thread. In the example above, that answer is L4.

AuthorsOpenReview ↗

Kind Reminder and Summary of Our Response

Dear Reviewer MrU1,

Thank you again for your constructive review. As the discussion period is closing soon, we would like to kindly check whether our responses have addressed your concerns. We summarize them briefly below.

  • W1. We give a worked example of the task, with a user's history, the target thread, the candidate stances, and the reference label. We will add it to the introduction.
  • W2, Q1. We split all 781 test tasks by the number of same-domain history discussions and by low, medium, and high text similarity between history and target. Accuracy shows no consistent trend across models and remains low even in the highest-similarity group (25.6% to 30.8%).
  • W3. The target thread supplies the question and the candidate stances. The history shows which positions the user took on other questions.

We would be glad to clarify any remaining point, and would sincerely appreciate it if you could consider updating your assessment.

Best regards,

The Authors

AuthorsOpenReview ↗

A gentle follow-up before the discussion closes

Dear Reviewer MrU1,

We hope this message finds you well. We understand this is a busy time, and we just wanted to gently follow up on our response. We also posted a new control in our reply to Reviewer 37Za that may be relevant to your question on topic relatedness.

Any thoughts you could share before the discussion closes would be greatly appreciated. Thank you again for your time and helpful feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.