- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes AVPO, a two-stage activation verbalization framework that reconstructs text from hidden activations and further improves the inverter with DPO-based rewards. AVPO achieves strong gains over existing baselines across multiple models and evaluation settings.
Critical strengths
- The problem setting is clear, and the design is intuitive. By separating activation readout from downstream QA, the method makes the intermediate textual representation directly inspectable.
- The performance gains are substantial and consistent across settings. AVPO consistently outperforms existing activation verbalization baselines across multiple donor and decoder configurations.
- The experimental analysis is comprehensive. The paper includes analyses of reward design, layer choice, DPO iterations, and several control experiments.
Critical weaknesses
- It is unclear whether reconstructing the input text should itself be considered an interpretation of the activation. AVPO aims to provide a human-readable interpretation of hidden representations, but its actual objective is to reconstruct the original source text from the activation. However, the activation may compress or discard parts of the input information, while the inverter may infer missing details from the decoder’s prior. Therefore, strong reconstruction performance does not necessarily imply that the generated text faithfully reflects the information actually represented in the activation.
- AVPO may learn reconstructions that are more useful for QA rather than more faithful to the activation. The main DPO reward is based on QA performance over the reconstructed text. Therefore, it is unclear whether the observed gains come from more faithful recovery of the activation’s overall information or from selectively emphasizing information that is useful for downstream QA.
- There is limited direct evidence that the generated information actually originates from the activation. The current evaluation mainly measures source information recoverability, but it does not disentangle whether the reconstructed content is determined by the activation itself or supplemented by the inverter’s prior knowledge.
- The training reward and the primary evaluation rely heavily on the same QA and judge pipeline. Qwen3-32B is used both for the DPO reward and for the main gist/detail evaluation. Although the judge correlation analysis is useful, it does not directly demonstrate that AVPO’s relative gains over the baselines persist under an independent evaluator.
- Direct evaluation of hallucination is limited relative to the paper’s claims of “faithfulness” and “reducing hallucinations.” Anchor F1 only measures overlap in proper nouns and numbers rather than overall factual correctness, while the gist/detail metrics primarily measure information recoverability.
- Some closely related baselines are omitted from the main experiments. The paper discusses Rep2Text and NLA as closely related activation verbalization methods, but neither is included in the main comparison.
Questions for authors
Please see the weaknesses.
4 comments
Response to reviewer XdvT (2/2)
W5. Direct evaluation of hallucination is limited relative to the paper's claims of "faithfulness" and "reducing hallucinations." Anchor F1 only measures overlap in proper nouns and numbers rather than overall factual correctness, while the gist/detail metrics primarily measure information recoverability.
[ANS] We add two complementary evaluations on our curated implication dataset: (1) atomic fact grounding (2) implication QA evaluation that measure how well implied intentions are verbalized. All methods use Llama-3.1-8B from our main settings.
(1) In atomic fact grounding, we decompose reconstrcuted text from pre-dpo and dpo r4 into around 1000 atomic fact for each. We use GPT-4.1-mini to classify each fact as supported, partially supported, contradicted, or not in source. We separately compare reconstrcutions from each setting to the source. The result is shown in the table. We use absolute percentage gap here.
| Method | Same event ↑ | Supported ↑ | Partially supported | Contradicted ↓ | Not in source ↓ |
|---|---|---|---|---|---|
| Pre-DPO | 30.1% | 16.8% | 11.3% | 12.2% | 59.7% |
| Post-DPO | 51.0% | 23.3% | 10.5% | 11.3% | 54.9% |
DPO increases the same-event rate by 20.9%, showing great improvements at the top level. At the fact level, the supported proportion increases by 6.5%, while contradicted and not-in-source facts decrease by 0.9% and 4.8%, respectively. These results show concrete improvements in both event alignment and factual grounding.
2. Implication QA evaluation.
We additionally evaluate whether the reconstructed information supports correct answers to implied intentions. The dataset contains 300 short passages from SocialIQA (246), INLI (27), and five dialogue sources (27). We retain samples that Llama-3.1-8B answers correctly with the passage but fails on all six question-only attempts, reducing the likelihood of correct answers using model priors.
We use GPT-4.1-mini as readout and judge. GPT-4.1-mini scores all answers against reference answers on a 0–4 scale: 4 denotes fully correct, 2–3 partially correct, and 0–1 incorrect. The average score is normalized to [0, 1].
| Method | Fully correct ↑ | Partially correct↑ | Incorrect ↓ | Average score ↑ |
|---|---|---|---|---|
| UAV | 49 (16.3%) | 115 (38.3%) | 136 (45.3%) | 0.403 |
| AO | 58 (19.3%) | 99 (33.0%) | 143 (47.7%) | 0.407 |
| LatentQA | 59 (19.7%) | 116 (38.7%) | 125 (41.7%) | 0.441 |
| AVPO SFT | 54 (18.0%) | 103 (34.3%) | 143 (47.7%) | 0.380 |
| AVPO r1 | 62 (20.7%) | 130 (43.3%) | 108 (36.0%) | 0.481 |
| AVPO r4 | 69 (23.0%) | 122 (40.7%) | 109 (36.3%) | 0.497 |
As shown in the above table, DPO r4 increases the fully correct answer rate from 18.0% to 23.0% and reduces the incorrect answer rate from 47.7% to 36.3%.
So both experiments are consistent regarding fact-level results. DPO improves factual grounding in the reconstrcuted text and meanwhile improves downstream activation readout performance.
W6. Some closely related baselines are omitted from the main experiments. The paper discusses Rep2Text and NLA as closely related activation verbalization methods, but neither is included in the main comparison.
[ANS] Thanks for bring up this concern. Our inverter is only different on adapter design from Rep2Text. Our focus are two folds: (1) DPO can improve faithful inversion, which is established in W5. We add another experiment with Rep2Text using 1-round DPO. The result is below in the table and shows that DPO can also improve its inverter (2) Our two-stage approach can make information usage inspectable. Therefore, we focus on comparing with works related to QA-based information readout. Besides our baseline UAV is also a verbalizeation verion of Rep2text.
For NLA, We were hoping to include NLA and tried to replicate it in preliminary experiments. However, even for the smallest released setting (Qwen2.5-7B), a single NLA run is estimated to take about 9.5 days on 16 A100-80GB GPUs based on our measured training throughput, and this cost is incurred separately for every model and layer. Unfortunately, this is far beyond what our limited academic compute budget can afford, so we had to leave NLA out of the main comparison.
| QA gist | QA detail | Anchor F1 | |
|---|---|---|---|
| Rep2Text | 0.196 | 0.215 | 0.196 |
| Rep2Text + our DPO | 0.259 | 0.231 | 0.211 |
| Δ (95% CI) | +0.063 [+0.039, +0.088] | +0.015 [−0.007, +0.037] | +0.015 [+0.003, +0.027] |
Reference
- Zhao, Haiyan, et al. "Rep2Text: Decoding Full Text from a Single LLM Token Representation." arXiv preprint arXiv:2511.06571 (2025).
- Zhao, Haiyan, et al. "Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation." arXiv preprint arXiv:2605.25903 (2026).
Dear Reviewer XdvT,
As the author–reviewer discussion period is drawing to a close, we wanted to check whether our response has addressed your concerns, and whether there is anything further we can clarify or provide. We welcome any comments and suggestions you may have at any time, and would be glad to discuss with you how to further improve our paper.
Thank you again for your time and thoughtful review.
Thank you for your response. Regarding W1, I agree with reviewer sz9v. W2 and W3 have not been resolved. Therefore, I maintain my score.
Response to reviewer XdvT (1/2)
We thank reviewer XdvT's comprehensive and insightful questions. Please find our response below.
[ANS] (1) We thank the reviewer for raising this fundamental question. Li et al. (2026, §4) explicitly note that "verbalizers could respond based on (implicitly) reconstructed inputs." This possibility motivates examining the reliability of the underlying readout: if an interpretation builds on recovered context, errors in that context can undermine its conclusions. AVPO makes this intermediate step explicit and inspectable, allowing us to assess what was recovered before interpreting its implications.
(2) We agree that the inverter may infer from the decoder's prior, which is a shared challenge. Our goal is to encourage the reconstructed text to remain faithful to the input. With this, we can make sure the information usage during activation readout is inspectable.
[ANS] We do not claim that the inverted text preserves all information encoded in the activation. Our gist and detail questions are designed to encourage the inverted text to preserve the main meaning and important details of the input.
[ANS] Existing work on text inversion has investigated the information source in activation-conditioned decoding. For example, removing activation-conditioned adapter outputs in UAV while retaining the decoder-side LoRA reduces fact-retrieval Token F1 from 0.262 to 0.144 and gist Token F1 from 0.233 to 0.162 (Zhao et al., 2026, §4.5). These results provide evidence that activation information contributes to the generated content.
However, fully disentangling activation-derived information from decoder priors goes beyond the scope of our work. We acknowledge this limitation and will identify direct activation interventions on AVPO as future work in the revised manuscript.
We re-scored the main gist/detail evaluation on 978 sealed-test QA pairs using four additional judges: GPT-4.1-mini, Gemma-4-31B, Llama-3.3-70B, and Qwen2.5-32B, following the same evaluation protocol. The results are shown below. Although the performance gaps between AVPO and other methods vary across judges, they remain consistently positive, and the variation is smaller than the gap magnitudes.
Llama-3.1-8B L31 → Qwen3-4B — Ours = MR-DPO round 4 (n = 978 QAs)