- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Interesting paper - The paper studies whether LLM survey respondents can reproduce human "can't choose" response. Across three models, non-choice is underproduced. Output rescaling recovers much of the average deficit, while internal interventions show that later computation can either support or counteract a correction.
Critical strengths
-
I found the distinction between matching the average non-choice rate and matching which questions elicit non-choice particularly useful. The paper shows that these can come apart, rather than treating a better aggregate as a complete solution. Keeping midpoint responses separate is also important, especially given OLMo's behavior.
-
The intervention analysis goes beyond showing that a feature is decodable. Holding an edit fixed while allowing or clamping later computation gives a concrete way to examine what happens to it. I also appreciate the candid comparisons with random controls and simple output rescaling.
Critical weaknesses
-
The main correction results have a fairly narrow generalization scope. The controllers use question-specific coefficients, and the held-out people answer questions already seen during development. Changing answer order weakens the gains substantially in some cases. This is mentioned in the paper, but I would bring it closer to the main results so that readers do not mistake available control for a general correction method.
-
Another minor concern is the component-specific explanation is also less secure than the path-level observation. Random controls are competitive in Llama and OLMo, and some restoration comparisons have limited delivered-norm matching. These qualifications do not invalidate the live-versus-clamped effects, but they limit claims that the selected components have a unique role in non-choice.
Questions for authors
-
Can the main results more clearly separate transfer to new people, robustness to answer order, and transfer to new questions?
-
How robust are the key restoration comparisons to better norm matching and the fidelity gains to uncertainty in the human targets?
3 comments
Component-specific explanation (the second weakness). We agree, and the paper does not claim a unique role for the selected components: "The controls limit uniqueness" (§5.5, l.432–433), and competitive controls and incomplete norm matching restrict component specificity (§6, l.463–464). Retuned random controllers do not differ reliably from the selected ones on CC cross-entropy in Llama and OLMo (l.313–314), so the controllers show that a correction is accessible, not that the extracted candidates are superior (l.312–313). The §5.5 statement that a component supporting ordinary CC can oppose an upstream correction requires only that component's own live-versus-restored contribution, which uses all inputs (Table 26). We will clarify this scope in the revision.
New, the path-level observation at additional sites: under a design fixed in advance, we repeated the live-versus-clamped test at six additional MLP sites, the next two per model by the paper's development ranking. The prespecified rule (later layers adding in Llama and OLMo, contributing negatively in Qwen) held at 4 of the 5 sites it scores; the U-versus-N contrast raised CC at 4 of the 6 sites, and at all 4 the later-layer contribution had the predicted sign. The misses are reported in Table I1 (details in our reply to ZxQ4, Q2).
Q2. Norm matching and uncertainty in the human targets.
- Restoration. The comparison §5.5 relies on, the sign of each component's live-minus-restored contribution, uses all inputs and involves no norm matching (Table 26). The 7–61% applies only to the contrast between the selected component and its controls (G.2, l.1972–1977); we will clarify this in the revision.
- What we did (new). Keeping the matching criterion unchanged, we recomputed the difference between the selected component's contribution and the mean of its two controls on the inputs where the selected component and both controls pass.
- Result. Pooled over the four answer orders, the difference has the sign of the selected component's all-input contribution, with 95% CIs excluding 0: Llama +.022 (27% of inputs); OLMo −.044 (61%); Qwen ×.75 / ×1.25: −.032 / +.045 (7%).
- What it shows. On matched inputs, the selected components differ from their controls in the direction of their all-input contributions. Qwen's subset is small, so we treat it as a consistency check; the §5.5 conclusion rests on the all-input contributions.
- Direction specificity, where matching is nearly complete. For the fixed U edit, matching passes for 100% of Llama inputs and about 95–96% of OLMo inputs, and U yields lower CC cross-entropy than each displacement control matched to its norm (Table 27). New: among U and 40 random directions norm-matched per input, U gives the largest increase in the logit of mean CC on a 96-persona panel (Llama .502 vs at most .185; OLMo .127 vs at most .080; Table I2).
- Human targets. The paper's intervals condition on the empirical human targets and do not include the survey's full sampling uncertainty (A.4, l.826–827).
- What we did (new). We resampled human respondents within demographic cells (1,000 bootstrap resamples), recomputed the targets and re-evaluated the main fidelity gains; we also computed the agreement between two disjoint halves of the respondents.
- Result. Rescaling and the selected controllers lower CC cross-entropy in all 1,000 resamples. The baseline item-level correlations with human CC rates, computed on probabilities averaged over the four orders (point estimates .35 / .18 / .04; Q1 reports order-group values), have 95% percentile intervals of [.28, .41] / [.10, .24] / [−.03, .11]. The split-half reliability of the human item-level CC rates is .94 (Table I6).
- What it shows. The fidelity gains persist under respondent resampling within demographic cells, and the targets are reliable enough to evaluate which questions receive non-choice.
Thanks for the detailed response. This addresses most of my concerns.
We sincerely thank the reviewer for the careful and constructive review. We will add the new results to the revised PDF, which we will upload during the discussion period. Triplets of numbers refer to Qwen / Llama / OLMo.
Q1. Separating transfer to new people, answer order and new questions (also the first weakness).
We agree, and we will make this scope clear in the revision. The controller rows of Table 1 are for new people on the 42 known questions, with per-question coefficients, averaged over four answer orders (§3.2, l.134–140). Table 14 (App. D.1) separates the first two axes. New people: under the fitted order, the CC cross-entropy gain is preserved (development −.0141 / −.0496 / −.0319; held-out −.0141 / −.0520 / −.0329; §5.3, l.284–285). Answer order: the gain is smaller on the unfitted orders, most visibly in Llama (D.1, l.1281–1282). We will add the tables below.
New analysis: fitted versus unfitted orders, with and without question-specific coefficients.
Change in CC cross-entropy from baseline (lower is better):
Item-level Pearson correlation with human CC rates across the 42 questions, computed on the mean probability within each order group (the four-order values elsewhere, .35 / .18, average all four orders first and are not the mean of these columns):