- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces an expert annotation workflow that presents legal trajectories alongside natural-language explanations of surrogate-model activations. Kimi-K2.6 generates trajectories, Gemma3-27B processes them, an existing NLA verbalizes Gemma activations, and Kimi summarizes these explanations into token-cited claims. Three legal experts contributed 354 span annotations across 15 trajectories and explanation ratings for eight trajectories. Experiments examine explanation correspondence among three open models and classification of already identified legal errors. On 280 error spans in 14 task/document groups, aligned explanations improve macro-F1 from 0.2568 to 0.3243 relative to no explanations, with positive comparisons against text-only and shuffled controls.
Critical strengths
- A useful exploratory connection between interpretability and professional annotation. The interface combines localized legal-error judgments, written feedback, and separately rated explanation claims with links to their token-level sources. The study covers conversational and agentic tasks and documents expert coverage, compensation, and review effort. This is an interesting application of activation verbalization to a concrete professional workflow.
- Controlled initial utility evidence and careful reporting of limitations. The main classification experiment shares grouped splits and training settings across five conditions, matches explanation-field budgets, and uses paired document-bootstrap intervals. The aligned condition improves macro-F1 by 0.0675, with a 95% interval of [0.0195, 0.1009]. The authors distinguish diagnosis of known errors from detection and human improvement, report unsuccessful helpfulness filtering, and explicitly acknowledge that faithfulness to Kimi's internal processing remains unverified.
Critical weaknesses
- The specific value of activation-derived information remains confounded with the explanation-generation pipeline. Appendix D.1 generates text-only explanations with Qwen3-4B, whereas Appendix B.1 uses a Gemma3-27B NLA verbalizer followed by Kimi-K2.6 summarization. The text baseline receives local context and limited document excerpts; matching its accessible history to the surrogate pathway is not established. Matching output token budgets does not match model capacity, context, or summarization. Thus C>B supports this particular pipeline comparison, and C>E supports alignment, but neither isolates the activation pathway from a stronger text-analysis pipeline. The correspondence experiment does not settle this: both models process the same trajectory, shuffled explanations already reconstruct Gemma targets at 95–98% cosine similarity, and semantic agreement lacks a same-context text-explanation reference. Its binomial null of 0.5 is not an established chance-agreement baseline, and positions from the same source example may be dependent. I appreciate the explicit faithfulness caveats; the concern is insufficient evidence for the method's distinctive value, rather than an allegation that the authors claim to have established Kimi faithfulness.
- The legal-label resource lacks an independent reliability reference. Experts assign error labels while viewing both NLA claims and judge feedback, and these labels later serve as the prediction target. Improved agreement with such labels need not establish improved diagnosis against independently confirmed legal errors. This is not evidence of direct label leakage, but it makes annotation reliability important. The reported 82.26% agreement concerns 62 explanation-helpfulness claims, not legal-error localization or categories. One expert supplies approximately 73% of explanation ratings and 62% of legal-span annotations. Figure 3b also demonstrates that misleading ratings can reflect disagreement with the underlying legal judgment despite accurate paraphrasing. The paper acknowledges these distinctions, but the resulting resource still needs legal-label agreement or a limited independent audit. With only 14 evaluation groups and NLA coverage on 96 of 280 spans, the scope and distribution of the classification gains also remain important uncertainties.
Questions for authors
- Can you compare C against text-only explanations generated with matched observable history, comparable model capacity, and the same Kimi summarization procedure on the existing evaluation groups? Please specify the accessible context in each pathway and report results separately for the 96 NLA-covered spans and the other 184, together with per-category and document-level effects.
- In the existing open-model evaluation, how well do same-context text-only explanations agree with target NLA explanations? Can semantic comparisons retain source-example clustering, and can reconstruction include a mean-activation reference or centered variance-explained metric? These analyses would help distinguish shared-context correspondence from target-specific information without requiring access to Kimi activations.
- For the four trajectories with repeated expert review, what agreement is observed in legal-error localization and labels, and how are disagreements handled? Is any limited independent legal audit available? If not, please keep the resource's interpretation explicitly tied to assisted expert judgments and distinguish helpfulness from correctness and annotation improvement.
5 comments
Authors' response to Reviewer 2TTE [2/3]
W2. Controls for anchoring on claims/judge
To clarify, the rubric judge saw only output documents (LAB) or the final answer (PRBench), so its feedback never covered the reasoning, which holds 63% of all error spans (327 of 519).
For NLA claims, interaction logs from the third collection week onwards record whether an expert opened a span's NLA explanations before saving its label. Based on our annotation metadata, we split the 519 spans into three subsets:
- Exposed (244): activity metadata shows the expert opened, viewed, or rated NLA claims before labeling the span.
- Unexposed (64): activity metadata shows the expert did not open the NLA claims before labeling the span.
- Less exposed (211): in the first 2.5 weeks of annotation, before activity metadata was recorded, we did not require experts to view or rate NLA claims; they recorded only 1 claim rating across these reviews. Their annotation rate was also about double the later rate (~16 vs. ~7 spans/hour).
Thank you for allowing us to elaborate on the data composition and anchoring on NLA/Judge. We will include a flag on each span for their exposure/anchoring to NLA or rubric-judge claims in our final data release.
W1/Q2. Random control for judge agreement
- Chance baseline: we paired each target explanation with a surrogate explanation from a different example of the same legal task, so the legal task alone cannot drive agreement.
- Dependence between positions: confidence intervals resample whole source examples (cluster bootstrap over the 260 source examples), so positions from the same example are not treated as independent.
Rates are the share of pairs judged by Kimi to have the same or partially the same meaning, with 95% CIs:
| Target → Surrogate | Real pairs | Random pairs |
|---|---|---|
| Within family | ||
| Gemma-27B → Gemma-12B | 81.0% [77.5, 84.3] | 7.4% [5.1, 9.9] |
| Gemma-12B → Gemma-27B | 80.8% [77.3, 84.2] | 8.4% [6.0, 11.0] |
| Cross family | ||
| Qwen-7B → Gemma-12B | 55.6% [51.5, 59.9] | 3.6% [2.0, 5.5] |
| Qwen-7B → Gemma-27B | 53.8% [49.4, 58.2] | 3.8% [2.2, 5.6] |
| Gemma-12B → Qwen-7B | 56.8% [52.5, 61.0] | 4.2% [2.5, 6.0] |
| Gemma-27B → Qwen-7B | 55.2% [51.1, 59.4] | 4.8% [3.0, 6.7] |
Pooled, real pairs agree at 80.9% [78.5, 83.3] within family and 55.4% [52.7, 58.0] across families, against 5.4% [4.3, 6.6] for random pairs.
Thank you for requesting this addition; it helped us directly separate surrogate explanations from a clearly defined random baseline.
W1/Q2. Text-only control for surrogate-target agreement
We followed the same updated setup we used for text-only condition B from §3.2 into the semantic agreement setup, where we also give it the full causal context and ask it to explain the last token. Both judges favor the surrogate agreement with the target's NLA versus the text-only.
| Target → Surrogate | Kimi: surrogate NLA | Kimi: text-only | GPT-5.4-mini: surrogate NLA | GPT-5.4-mini: text-only |
|---|---|---|---|---|
| Within family | ||||
| Gemma-27B → Gemma-12B | 81.0% | 63.6% | 58.2% | 54.8% |
| Gemma-12B → Gemma-27B | 80.8% | 61.4% | 55.4% | 47.8% |
| Cross family | ||||
| Qwen-7B → Gemma-12B | 55.6% | 37.4% | 38.2% | 33.6% |
| Qwen-7B → Gemma-27B | 53.8% | 41.4% | 31.6% | 34.2% |
| Gemma-12B → Qwen-7B | 56.8% | 31.4% | 34.0% | 23.8% |
| Gemma-27B → Qwen-7B | 55.2% | 33.0% | 38.4% | 30.0% |
| Pooled | 63.9% | 44.7% | 42.6% | 37.4% |
We appreciate your ask for this addition; the surrogate explanations are directly shown to align better with the target than external explainers with full text context.
W1/Q2. High reconstruction for shuffled explanations and metric inflation
Thank you so much for highlighting the mean reference/centering for reconstruction, as cosine was indeed inflated by the mean direction of the activations (0.58-0.985). We updated the reconstruction metric to be Fraction of Variance Explained (FVE), following your comment and the original NLA reference, instead of cosine similarity. Using FVE (where 0 means no better than predicting the mean activation) shows how all shuffled explanations have negative FVE (−0.56 to −0.94), while surrogate explanations score close to the targets' own NLA (0.44–0.63 vs. 0.66–0.69):
| Target → Surrogate | FVE: target's own NLA | FVE: surrogate | FVE: shuffled |
|---|---|---|---|
| Within family | |||
| Gemma-27B → Gemma-12B | 0.69 | 0.58 | −0.65 |
| Gemma-12B → Gemma-27B | 0.66 | 0.63 | −0.67 |
| Cross family | |||
| Qwen-7B → Gemma-12B | 0.68 | 0.52 | −0.94 |
| Qwen-7B → Gemma-27B | 0.68 | 0.55 | −0.93 |
| Gemma-12B → Qwen-7B | 0.66 | 0.45 | −0.60 |
| Gemma-27B → Qwen-7B | 0.69 | 0.44 | −0.56 |
Thank you again for this helpful comment; this adjustment significantly improved the alignment of our results by removing the previous inflation.
Authors' response to Reviewer 2TTE [3/3]
W3,4. Utility to experts and label agreement and handling
We acknowledge not studying the direct utility to annotators in depth. We found the process itself is difficult at our scale in terms of getting the experts' feedback, time, and aligning them to samples in legal tasks they are familiar with while matching them with each other to collect label agreement data. This also limits our ability to do an independent legal audit within the rebuttal stage, so we will describe the labels as assisted expert judgments and distinguish helpfulness ratings from correctness and annotation improvement.
-
From collected feedback from the 3 experts, A and C found that the claims mostly restate the trajectory text, while B found them useful for checking and verifying it. We will include their exact feedback in the paper.
-
We will also include a crossover study on the direct utility of explanations for expert annotators as future work.
-
For data handling, we merge identical spans and use the union of their labels.
-
Claim-rating agreement: our submission reported 82% agreement between B and C on 62 claims ( = 0.60). Since then, A and B co-rated 1,568 claims, with 37% agreement ( = 0.00), because A rated 99% of claims neutral.
-
Span and label agreement: 7 trajectories have now been reviewed by two experts (4 in the submission). Each expert chooses where a span starts and ends; on these 7 trajectories, 32 of 210 spans overlap a span by the other expert (F1 0.15), and 3 of 21 overlapping span pairs share a label, likely because each expert highlights different reasoning issues given their background. We share two cases below:
- Same error, different label (playbook review). Expert B, tagged "other: missed rule": "Model fails to recognize the 72-hour notice rule in the playbook." Expert A, tagged incomplete reasoning: "Does not state that there is nothing in the agreement related to the 72 hour notice requirement as required by the playbook."
- Different focus (clinical-trial cost disclosure). Expert B flags incorrect citations, e.g., "should be 21 CFR 50.25(a)(7)". Expert A flags a missing framework: "45 CFR 46.117 is the overarching statute related to informed consent and should be included in the analysis."
Thank you for highlighting this limitation; these cases helped us demonstrate how each expert approaches the legal errors and how their perspectives can differ in some of these cases.
Authors' response to Reviewer 2TTE [1/3]
Thank you for your comprehensive feedback and suggestions! We appreciate that you found our idea of bringing interpretability into annotation promising. We address your comments and concerns below.
A main and shared concern was the size and composition of the expert data, and we understand they matter. We kept collecting expert annotations after submission. The data now covers 24 trajectories (15 in the submission, +60%), 519 error spans (354, +47%), 8,110 claim ratings (2,892, +180%) and 57.3 expert hours (34.3, +67%). Expert B's share of claim ratings fell from 73% to 57%. All new analyses below use this full data.
W1/Q1. Unclear gain from activations and unbalanced setup
We acknowledge the mismatch between the classifier conditions in section 3.2 and revised the setup so that they differ only in the source of the explanations (external text-based vs. internal NLA-based):
Results (macro F1):
We see this as a preliminary study of a new setting. Where downstream results are weak or mixed, we think they can be addressed or improved by:
We really appreciate highlighting the setup mismatch and helping us to show clearer results; the updated setup and data helped us isolate and clarify the gain from activations and NLA much more and clearly define what we can do in our next steps.