acceptodds
Withdrawn from ICLR 2027

SRLCF: Learning Controllable Schwartz Value Representations from Structured Interview Dialogues

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning value representations from interviews requires distinguishing respondents' expressed values from values introduced by interviewers and integrating evidence across turns. We introduce InterviewValues-100K, a corpus of 100,000 single- and multi-turn English interview units with machine-generated annotations for 20 Schwartz-based value categories, retained source records, and answer-side evidence. We propose SRLCF, a dual-encoder framework that combines semantic matching between dialogues and value definitions with imbalance-aware classification through label-specific gates. Gated residuals incorporate contextual answer-marker states, while fine-value and higher-order auxiliary supervision shape a dialogue-level value representation. Across three seeds, SRLCF achieves Macro-F1 of on the development benchmark, compared with for DeBERTa + ASL. In a retrospective fixed-model evaluation on 1,000 human-annotated MediaSum dialogues containing at least one expressed value, Macro-F1 is 0.494 versus 0.475 for a fixed-recipe baseline rerun, with the paired 95% confidence interval for the difference spanning zero. Representation analyses show that fine-value supervision improves frozen linear-probe performance without an observed gain in fused classification. Value-centroid distances also correlate with Schwartz's circular distances (Spearman ), without a circular training constraint. These findings distinguish the benefits of semantic–supervised fusion for value detection from those of fine-value supervision for representation learning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Human Values, Schwartz Value Theory, Value Representation Learning, Value Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8rzkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces INTERVIEWVALUES-100K, a corpus of 100,000 single- and multi-turn English interview units drawn from MediaSum, NPR-2P, Tennis Interviews, and NewsInterview, annotated with machine-generated labels for 20 Schwartz-based value categories, together with retained source records and answer-side evidence spans. It also proposes SRLCF, a dual-encoder framework that combines MPNet semantic matching between dialogue and value definitions with a DeBERTa direct classifier, fused by label-specific gates, plus an answer-marker residual and a dialogue-level value representation trained with fine-value and higher-order auxiliary supervision. SRLCF reaches 0.510 Macro-F1 on a development benchmark versus 0.483 for DeBERTa+ASL, and 0.494 versus 0.475 on a 1,000-dialogue human-annotated MediaSum set, with a paired 95% CI spanning zero. Representation analyses show fine-value supervision improves frozen linear probing (+0.0074 Macro-F1) without improving fused classification, and that value-centroid distances correlate with Schwartz circular distances (ρ = 0.502).

Critical strengths

A source-linked, evidence-annotated corpus for a genuinely under-resourced task. INTERVIEWVALUES-100K retains source identifiers, dialogue turns, and answer-side evidence spans, and it distinguishes question context from answer evidence. This is a meaningful resource contribution even if the labels require later audit.

An honest separation of representation quality from fused classification. The finding that fine-value supervision improves a frozen linear probe (+0.0074 Macro-F1) but not fused classification (Δ = −0.0003, all CIs spanning zero) is a useful negative result that is cleanly reported and correctly interpreted.

Critical weaknesses

No clean confirmatory evaluation supports the comparative method claim. The development test split is explicitly labeled a development benchmark because seed-42 diagnostics informed architecture development. The human evaluation is retrospective, positive-only, single frozen run per system, and the paired Macro-F1 difference is +0.020 with 95% CI [−0.012, +0.052], which crosses zero. The paper itself states this “leaves comparative uncertainty.” There is therefore no evaluation in the paper that establishes that SRLCF outperforms DeBERTa+ASL.

The 100k training labels are unaudited and cannot be treated as ground truth. The MediaSum portion is annotated by a single LLM family (qwen3.8-max) with two prompt variants and a third strict pass; agreement is defined as same-model-channel agreement, which is not semantic validation. No expert human override was applied, and no human audit of the training corpus was performed. The human evaluation uses different dialogues and does not validate these labels. This limits the corpus’s value as a benchmark and makes it unclear how much of the development-benchmark gain reflects annotation style rather than value understanding.

The human annotation protocol has no reported inter-annotator agreement, and the hierarchy auxiliary contradicts itself. The A/B decisions were not retained, so IAA cannot be reconstructed; annotators are 20 master’s students with no value-theory or English-proficiency screening; and the positive-only population cannot measure false alarms or prevalence. Separately, the paper reports that removing the hierarchy auxiliary improves the base configuration by +0.0024, yet the loss is retained in the final model; this inconsistency is not resolved.

Questions for authors

Can the authors add a human audit of a stratified sample of the 100k training labels (by source, rare label, polarity, question-only cue), with reported agreement against the machine labels? This is essential for treating the corpus as a benchmark.

Can the authors report inter-annotator agreement on the 1,000 human-annotated dialogues, or reconstruct it from retained A/B records?

Can the authors run a genuinely confirmatory comparison—e.g., a new model trained after all architecture decisions are frozen, evaluated on the retrospective human set and on a new blind sample—and report the paired interval? Without this, the comparative claim remains unsupported.

Why is the hierarchy auxiliary retained when removing it improves the base configuration by +0.0024? Can the authors either explain the choice or drop the loss?

Given that fine-value supervision improves frozen probing but not fused classification, what is the intended takeaway of SRLCF as a method? Is the contribution the representation, the fused classifier, or the corpus?

The paper states that checkpoints were deleted after inference. Can the authors clarify which artifacts are actually released, and whether the human-eval comparator rerun and the deployment checkpoint can be used to reproduce the main tables?

Reviewer_zdsyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper builds an LLM-labeled interview corpus for detecting Schwartz values and proposes SRLCF, a two encoder model that learns a dialogue level value representation.

Critical strengths

s1: Ways of getting the represenntation of values is important and crutial in the domain

s2: Authors are honest about many limitations of the data and evaluation

Critical weaknesses

w1: The paper computes a value representation, but its usefulness is only checked within its own framework. The evaluation of the representation is mianly confirming the training objective. Though the authors mentioned that conditioned generation is one of the future work, for a paper that proposes a methd tries to acuaire representation, downstream evaluation is definetly necessary. Experiments such as comparing the same value across different topics and different values within the same topic, transferring the representation to other value datasets or applying it in a downstream scenario would be needed to show that the representation actually captures values.

w2: The method relies on two encoders, in this case, which semantic space does the outcome representation belongs to? If it's in a mixed space, will there be noisy when try to apply it for downstream task?

w3: annotator agreement is not reported.

w4: writing reads more like an experiment report rather than a conference paper. This makes it hard to follow the motivation and the main contribution

Questions for authors

see weakness

Reviewer_zoFbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

this work introduces InterviewValues-100K and SRLCF, a dual-encoder model combining label-semantic matching with direct classification for detecting Schwartz values expressed in interview answers. SRLCF improves development benchmark Macro-F1, while auxiliary supervision improves frozen representation label recovery. Paper's advantage over an independent baseline remains inconclusive on human annotations.

Critical strengths

1.The corpus preserves dialogue roles, provenance, and answer-side evidence, with grouped splits, making it a useful resource for interview value detection. 2.The experiments distinguish classification from representation quality and transparently report evaluation limitations. Development benchmark improvements are consistent across the three seeds.

Critical weaknesses

1.The original test split informed architecture development. The human evaluation is retrospective, same-source, and restricted to value-positive dialogues. The Macro-F1 gain over ASL is 0.0198, with paired 95% CI: [−0.0117, 0.0523], leaving superiority unresolved and false alarms on entirely value-negative dialogues untested (§5Table14). 2.Learned gating improves over fixed gating by only 0.0011.And hierarchy supervision does not improve classification in its matched configuration. Answer-marker ablation provides some evidence, but no controlled test establishes robustness to interviewer-introduced value cues. The benefits of these mechanisms remain insufficiently separated from dual-encoder fusion (§4.1Table 2).

Questions for authors

1.Can you provide a prospectively fixed human test including value-negative dialogues, with paired comparisons across multiple seeds? 2.Can you add a stratified human audit of machine labels and independently assessable annotation agreement evidence?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.