acceptodds
Under review as a conference paper at ICLR 2027

Building Geometry-Aware Probes for Fine-Grained LLM Safety Classification

Abstract

Internal activation probes offer an efficient approach to safety monitoring in large language models, but most existing work focuses on general harmfulness rather than fine-grained safety categories. We study how harmful categories are organized in representation space and how this geometry affects classification. Category directions relative to safe inputs exhibit a strong shared orientation while retaining category-specific variation. Categories that align more strongly with this shared geometry also tend to be harder to classify, with Spearman correlations of -0.691 for mean inter-category directional similarity and -0.682 for shared-axis alignment. Motivated by this observation, we introduce Spatial-Sign-Conditioned Angular Kernel (SSCAK) probing. SSCAK estimates shared within-class directional scatter using spatial signs, conditions the representation space accordingly, and learns nonlinear category boundaries on the unit hypersphere using an angular kernel. Across seven instruction-tuned models and three token-aggregation strategies, SSCAK consistently improves macro-F1 over the corresponding standard probes. These results connect the geometry of internal safety representations to fine-grained classification and demonstrate the value of explicitly modeling within-class directional structure.

Primary area:
interpretability and explainable AI
Keywords:
large language models, AI safety, mechanistic interpretability, linear probing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ubNxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies fine-grained safety classification using the internal activations of frozen language models. It first analyzes harmful-category mean directions relative to a shared safe-category mean, finding substantial directional alignment and negative correlations between alignment and per-category classification F1. Motivated by these observations, the authors propose Spatial-Sign-Conditioned Angular Kernel probing (SSCAK). The method estimates within-class directional scatter using spatial signs, conditions and normalizes representations, and applies an RBF-kernel SVM with pairwise probability coupling.

Critical strengths

Strengths

  • Applying spatial-sign scatter estimation to fine-grained safety probing is a potentially useful methodological combination. Equation (4.5) prevents residual magnitude from directly dominating each sample’s contribution to the scatter estimate, while Equation (4.6) downweights directions with greater within-class directional variation. The construction is intuitive and implementable. Its novelty currently appears to lie primarily in task-specific integration rather than a new kernel or statistical estimation principle.
Critical weaknesses

Weaknesses

  • In Equation (3.2), every harmful-category direction subtracts the same safe mean, and the shared axis is constructed from those same harmful categories. These choices can induce alignment independently of a safety-specific representational mechanism. Positive pairwise cosines and large shared-axis fractions therefore require controls involving alternative reference categories, independently estimated safe means, random labels, and leave-one-category-out axes.
  • Table 1 compares standard probes against the complete SSCAK pipeline. It does not separate the effects of spatial-sign conditioning, feature normalization, kernel nonlinearity, or optimization. Missing comparisons include an ordinary RBF-SVM, an RBF-SVM on normalized features, covariance conditioning followed by the same classifier, and a well-tuned MLP. The current evidence establishes that the complete pipeline outperforms the selected standard probes, but not that spatial-sign geometry is responsible.
  • WildGuard contains only 50 test examples per category, and the AEGIS datasets contain 30 per category. Results are reported without repeated runs or confidence intervals. Learning rates, training duration, early stopping, full hyperparameter grids, selected configurations, and the rationale for extraction-layer selection are also absent.
Questions for authors

What exactly is the input to attention + SSCAK? Are category-specific pooled representations concatenated, averaged, or replaced by shared attention? How are attention parameters trained, and are identical frozen parameters used in both conditions?

How are dedicated guard outputs mapped and scored in Table 2? How are multiple predicted categories, unsupported categories, refusals, and malformed outputs handled? Do all methods receive equivalent input content and task instructions?

Reviewer_bDp5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors keep the language models fixed and train classifiers on their internal activations. They use within-class directions to adjust the activations. They then train a support vector machine (SVM) for safety classification. They report higher macro-F1 than the corresponding standard probes.

Critical strengths
  • Design based on directional structure. The authors consider the directional structure of the activations when they design the classifier. They use directions of within-class variation to adjust the activation space. They then use unit-sphere normalization and a nonlinear kernel. These choices give the classifier a way to use directional differences between categories.
  • Meaningful improvement in macro-F1. In Table 1, the classifier improves macro-F1 over the corresponding standard probes. I consider the improvement meaningful.
Critical weaknesses
  • Missing component ablations. The authors combine spatial-sign statistics, space adjustment, unit-sphere normalization, and an angular kernel. They report results for the full method in Table 1. They do not report component ablations.

    The authors do not report how much each part adds to macro-F1. They may get much of the gain from using a nonlinear classifier. They should remove one part at a time and report the change in macro-F1. Readers need to know which parts help and which parts they can leave out.

  • Small additions to existing methods. The authors use existing spatial-sign statistics. Their angular kernel is a Gaussian RBF kernel on unit-normalized activations. They use existing one-vs-one SVM training and probability coupling. The main added choice is how they combine these steps.

    The authors should compare with a standard Gaussian RBF SVM. They should compare with the same SVM using ordinary within-class covariance for the space adjustment. These baselines would test the added value of their design against existing methods.

  • Limited design for safety classification. The authors treat safety categories as ordinary class labels. They use standard label filtering and class balancing. They could use the same classification steps for other labeled tasks. The design adds little that is specific to safety classification.

    The authors should identify the safety problem that calls for these design choices. They should explain how their method addresses that problem.

Questions for authors

Please see the weaknesses.

Reviewer_DbdgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to distinguish fine-grained safety categories using internal representations from frozen LLMs. The authors find that harmful-category mean directions, measured relative to safe inputs, share a common orientation. Categories more closely aligned with this shared geometry tend to be harder to classify. Motivated by these observations, the authors introduce Spatial-Sign-Conditioned Angular Kernel (SSCAK) probing. SSCAK estimates shared within-class directional scatter using spatial signs, conditions and normalizes the representations, and learns nonlinear boundaries using a kernel SVM. On WildGuard, SSCAK outperforms the corresponding standard probes in all 21 comparisons across seven models and three token-aggregation strategies. It also achieves the highest macro-F1 on four of five datasets in comparisons with dedicated guard models.

Critical strengths
  1. The paper addresses a practically relevant problem in fine-grained safety classification. Distinguishing risk categories helps reveal what information safety representations retain and can support more specific monitoring policies than binary classification. Quantifying the relationships among shared directions, category differences, and classification difficulty provides useful observations.
  2. The method has a clear structure, and the main computational steps and their roles are explained directly. Classifying frozen-model activations also makes the approach easy to connect with existing work on representation analysis and safety monitoring.
Critical weaknesses
  1. Missing controls make it difficult to identify the source of the performance gains.

    SSCAK introduces within-class directional conditioning, final normalization, and nonlinear kernel classification together. The comparisons cover standard mean, last-token, and attention probes, but omit MLPs and ordinary RBF-SVMs on the same representations. Component ablations are also missing. Table 1 therefore supports the complete pipeline's advantage over the selected baselines, but does not establish the independent contribution of within-class directional modeling.

    These controls are particularly relevant because prior safety-monitoring research directly evaluates nonlinear probes. Beyond Linear Probes compares polynomial classifiers and MLPs for harmful-request classification. The paper should establish what additional value its geometric processing provides over such general nonlinear methods.

  2. The stability of the geometric observations across models and datasets remains unclear.

    Section 3 analyzes shared directions and their relationship with classification difficulty using Qwen3-4B on WildGuard. Subsequent experiments establish classification improvements across several models, but do not show whether those models exhibit the same geometric patterns. It is therefore unclear whether the observations reflect a general property of safety representations or depend on the particular model and category set. As a central finding of the paper, this analysis needs broader validation.

Questions for authors
  1. How much independent benefit does the proposed geometric processing provide over general nonlinear classifiers?

    How do MLPs and ordinary RBF-SVMs perform with the same model layer, aggregated representations, data splits, and comparable tuning budgets? How much does performance change when setting ρ=1\rho=1, while retaining centering, normalization, and the RBF kernel? Does the main advantage persist when replacing spatial-sign scatter with ordinary within-class covariance? These comparisons would identify which parts of the method contribute most.

  2. How stable is the shared geometry across models?

    Do the strength of the shared orientation and its relationship with classification difficulty remain consistent across model families, model sizes, and datasets? Which models or categories show clear exceptions? These comparisons would clarify the conditions under which the geometric pattern holds and what differences between models reveal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.