- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses the efficiency of auditing large language models with natural language autoencoders. Because verbalizing every token position is costly, the authors study how well a wide range of signals predicts which positions are worth inspecting to find a threat. Their procedure verbalizes every position of each transcript and passes each verbalization to an LLM judge, which labels the position as 'on task' (relevant to the inspection) versus 'off-task' (irrelevant). These labels are then used to evaluate the signals. Across four datasets covering prompt injection and concealment they find that a simple structural baseline almost always outperforms signals derived from the internal representations of the model, and they conclude by recommending it as the practical recipe.
Critical strengths
The work provides a large and systematic evaluation study that is useful to guide future research on finding optimal predictive signals for efficient auditing.
The procedures used in the paper are well written and documented.
Critical weaknesses
The paper uses a large portion of its length to the analysis of the proposed signals. Although these signals carry some predictive information, they are consistently outperformed by the structural baseline and their behavior varies considerably across datasets and models, so they offer limited guidance toward a practical auditing recipe. While the systematic evaluation is valuable, it occupies a lot of space for a largely negative result. By contrast, the structural baseline that the authors ultimately recommend as the practical recipe receives much less analysis.
The paper aims to find position-selection strategies that improve on the heuristics used in prior works, yet it offers little to no comparison with those methods.
For these reasons I have concerns about the impact of the findings of the paper and I believe that the work needs some refinement, better addressing the task of providing a practical recipe for position-selection and I am inclined to decide weak rejection.
Questions for authors
At the same budget, does the structural baseline actually outperform the position-selection heuristic used in prior works, such as the final token before the answer or eight evenly spaced positions?
Does the structural baseline consistently select the same positions or chat roles, and does this selection generalize across datasets? Could a hardcoded rule, similar to those used in prior work, be extracted from the structural baseline?
The cost of the labeling procedure is unclear. Both the structural baseline and the signal directions must be fitted on labels, which requires verbalizing and then judging a portion of the dataset with an LLM. The experiments use four fifths of the transcripts for this. how many labeled transcripts are needed in practice, and how does this one-time cost compare with the savings at audit time.