- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the fidelity of local affine surrogates, , as explanations of LLM behavior. The design measures responses at symmetric endpoints along independent intervention directions. Empirically, the authors build eight benign task neighborhoods adapted from XSTest. Each has five prompt-level factors (authorization, consequence severity, ambiguity, evidence requirements, instruction conflict) at five levels. They collect responses from 22 open-weight LLMs and score caution and epistemic uncertainty with two fine-tuned reward-model scorers. Anchored and OLS affine surrogates underperform a constant training-mean baseline.
Critical strengths
- Clean separation of error sources. Distinguishing model-class limits from parameter mismatch is a useful framing for evaluating surrogate explanations. The exact three-term decomposition (Eq. 3) is easy to interpret.
- Concrete case study. The baseball certificate in Section 4.6 and Appendix C.5 includes full prompts and responses.
- Four-point certificate gives a human-inspectable witness of infidelity. Corollary 2 makes a useful practical point: a well-conditioned slope matrix does not certify fidelity.
Critical weaknesses
-
The experiments cannot separate model nonlinearity from measurement artifacts. The headline result is that every setting fails, but factors unrelated to LLM nonlinearity can inflate midpoint dispersion: a. No noise floor. Each prompt is generated once (in B.2 "Most models use greedy decoding"), and the paper explicitly does not estimate variability over repeated generations. Paraphrase sensitivity is also not measured. b. No positive control. The certificate is never shown to accept affine fidelity under realistic noise. A test that rejects in 155/155 cases needs evidence that it can pass. Examples would be simulated exactly affine responses with empirically estimated noise, or near-trivial interventions in real models.
-
Coordinates are assigned, not measured, and ± symmetry is not verified. The coordinate values are labels attached to discrete textual edits, and the paper states they do not imply a continuous semantic metric. The entire theory depends on the + and − edits being equidistant from the center in effect. If one edit is perceptually stronger than its opposite, a perfectly linear responder still shows a midpoint shift, which the paper would attribute to the affine class. No manipulation check verifies effect symmetry, for example through human ratings or a perception probe.
-
Lack of verification that target models registered the interventions. The rubric scores expressed caution and ignored correctness, so score shifts may reflect misreading or non-compliance rather than sensitivity to the intended factor. Compliance rates per model would strengthen the interpretation.
-
Explanation methods used in practice are not evaluated. The paper positions itself relative to LIME, MAPLE, and especially LAMP (Chen et al., 2026). LAMP fits local linear surrogates on LLM-stated factors and reports reasonably good linear fits. However, the experiments evaluate only the authors' own anchored and OLS surrogates on a custom symmetric design.
-
The title question is only partly answered. The paper gives a necessary condition on measured endpoints () and a failure certificate. Since no setting passes, there is no positive characterization of when local linear models are faithful. The conclusion also depends heavily on the tolerance (The value ).
Questions for authors
What is under (a) repeated sampling at temperature > 0, (b) paraphrased center prompts used as placebo ± directions, and (c) repeated or alternative scoring? How does this noise floor compare with ?
Can you show a positive control, with synthetic affine responses plus empirically estimated noise, or a real low-perturbation setting, where the certificate accepts affine fidelity?
Thank you for the detailed review. We added repeated-generation and rewording controls, compatibility checks, and an application to LIME to examine your central concern: what an exclusion tells us about model behavior. The anonymous repo has also been updated.
Noise floor. (a) All 22 models answered every r4 endpoint and base request in eight rounds at temperature 0.7 (Appendix C.23). All 201 complete, evaluable single-round tables exceed 0.25s; they are repeated measurements sharing models and prompts, not independent trials. As a no-edit control, ten responses to the unchanged base request placed at the endpoint coordinates also exceed 0.25s in all 27 evaluable settings (0.46s–1.29s). A single sampled table therefore cannot separate the edits from sampling variation. (b) Two raters judged that ten rewordings of the baseball base request keep its requirements (60/60). On greedy responses of all 22 models, any coordinate-only predictor has RMS error at least 0.72s on one of them (Appendix C.25); used as placebo ± directions for four models, they exceed the tolerance in every group (Appendix C.24). (c) Two scoring runs agree exactly (Appendix C.20); changing only the scorer's prompt moves the rewording radii by at most 0.033s4 (Appendix C.24). Even greedy responses changed across new engine processes for three of four models. Averaged over 32 sampled rounds in new processes, four models' endpoint mean tables are within tolerance for uncertainty and above it for caution. The four prespecified e-value tests leave open whether the mean response admits an affine fit (Appendix C.24). We therefore report the exclusions as exact limits of the recorded tables for this coordinate representation, not as evidence of a nonlinear response to the intended edits.
Positive control. By Theorem 1, an affine fit within ϵ exists exactly when R⋆≤ϵ, and the proof gives the fit, so the criterion can accept. Constructed tables with known affine signals and perturbations up to half the tolerance are accepted, while opposed midpoint shifts of twice the tolerance are rejected (Appendix C.18). Measured tables are also accepted: both 32-round uncertainty endpoint mean tables (Appendix C.24), the eight-round mean table for Witcher Nemotron uncertainty (0.242s; Appendix C.23), 10 of 16 word-deletion slice tables at tolerance 0.05 (Appendix C.26), and 25 of 31 r4 settings at 0.75s (Table 3). These are finite observed cases, not calibrated noise controls. In a further control, adding observed differences between sampled response tables to a known affine signal exceeds the tolerance in all eight cases (Appendix C.22).
Symmetry. We agree that a perceived asymmetry can create a midpoint shift. Raters found no reversed direction and no change outside the edited aspect in nine r4 pairs, but equal strength in only five and six (Appendix C.16). If each pair's magnitude ratio is shared by the 22 models, exclusion holds in 155/155 settings for ratios in [1/2, 2] and 151/155 for any ratio (§4.5, Appendix C.14). If each model may choose its own ratio, far fewer remain excluded (55/176 Nemotron caution combinations; Appendix C.17). The baseball bound persists in all 16 original and replacement tables, although raters rated some replacements stronger (Appendix C.21).
Registration of interventions. The prompt-preservation review assessed the edits. For generated answers, a literal audit of the instruction-conflict endpoints finds prohibited list markers in 136 of 176 positive-side responses (20 models) and in none of the 176 negative-side responses (Appendix C.19). Most models thus follow the lower-priority list request, so score changes can reflect non-compliance as you note; the audit does not assess semantic correctness or the other four factors.
Explanation methods. We applied the official LIME text explainer at 256 samples (Appendix C.26). M13 has complete fits: held-out MSE exceeds a constant's in three of four channels, and on two fixed slices LIME's largest error exceeds R⋆ in all eight evaluations. M10 has no fit because some scores are missing, so this covers one model and one prompt. Our bound covers every affine map of these coordinates and scores; LAMP uses other coordinates and targets, and we did not replicate LAMP or MAPLE.
Title question and tolerance. On measured endpoints the answer is exact: an affine explanation within ϵ exists if and only if R⋆≤ϵ, so R⋆ is the smallest attainable tolerance. Table 3 varies the tolerance from 0.15s to 0.75s, and §5 now explains which failures refitting can fix and which it cannot.
If these revisions address your concerns, we would be grateful if you would consider revising your rating upward.