- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether the internal representations of large language models (LLMs) can capture different types of ill-posedness, including ambiguity, under-specification, and conflicting statements. Specifically, the authors represent the hidden states of prompt tokens as a point cloud and use zero- and one-dimensional persistent homology to extract seven compact topological descriptors for ill-posedness classification. Furthermore, the paper introduces a topology-conditioned steering mechanism that retrieves topologically similar positive and negative examples to construct query-specific steering directions, guiding models toward appropriate abstention or clarification. Experiments on two datasets and open-weight models from four families demonstrate the effectiveness of the proposed topological representations for ill-posedness classification and the benefits of topology-conditioned steering.
Critical strengths
- The paper addresses the challenge that LLMs often generate unsupported answers when faced with ambiguous, underspecified, or otherwise ill-posed questions. It aims to guide models toward more appropriate responses based on the nature of the questions.
- Unlike methods that directly use hidden-state vectors for classification, this paper employs persistent homology to characterize the connectivity and loop structures of prompt-token representations, offering a new perspective on how information is organized within LLMs.
Critical weaknesses
- The proposed method assumes that the topology of prompt-token hidden states can capture different sources of ill-posedness. However, it remains unclear why the connectivity and loop structures of these representations should correspond to ambiguity, under-specification, or conflicting information.
- Although the proposed topological descriptors achieve promising classification results, it remains unclear whether these improvements arise from topological information or more general geometric properties of hidden-state representations.
- Theorem 1 establishes class separation under specific reference-geometry and representation-distortion assumptions. However, the extent to which these assumptions hold for actual ill-posed questions remains unclear. Furthermore, the theoretical analysis focuses on particular geometric classes, whereas the experiments involve more diverse forms of ill-posedness.
- The proposed method uses topological descriptors to retrieve positive and negative examples, while the actual steering direction is constructed from their activation differences. Therefore, the observed improvements may arise from query-specific retrieval rather than topological information itself.
Questions for authors
See weakness
3 comments
Dear Reviewer Si43,
As the author discussion period comes to a close, we would appreciate your consideration of our rebuttal and revised manuscript. If our responses and additional evidence have resolved your concerns, we hope you will raise your score accordingly. Thank you again for your time and thoughtful consideration.
The Authors of Submission 37797
Thank you for your response. However, I still find the explanations unclear and remain unconvinced by the arguments. Therefore, I have decided to maintain my original score.
We thank the reviewer for the careful evaluation and constructive feedback. We address each concern below.
Question 1 & Weakness 1: Our motivation is that ill-posedness changes the contextual constraints among question tokens. Ambiguity leaves multiple interpretations viable, under-specification leaves relevant relationships unresolved, and conflicting information introduces incompatible requirements. Because hidden states integrate information across tokens through attention and subsequent transformations, these differences can alter which associations are reinforced and how contextual token representations are arranged.
Persistent homology characterizes this arrangement across distance scales. H0 captures when token groups become connected, distinguishing tightly connected neighborhoods from groups that merge only at larger distances. H1 captures persistent cycles in the proximity structure, describing relationships not captured by component merging alone. Together, these descriptors summarize how token representations are organized, rather than only their average activation or overall spread. This motivates TIPS's use of joint connectivity and loop signatures to identify class-dependent patterns associated with different sources of ill-posedness. We added this motivation to Section 3.1.
Question 2 & Weakness 2: We added a non-topological geometric baseline using the mean, standard deviation, and median pairwise distances from the same token clouds. Both methods use identical data splits, with layer and classifier hyperparameters selected using validation data only. This comparison tests whether simple summaries of token-cloud dispersion can explain the classification performance.
Across eight models and five seeds, average macro-F1 is 78.59%/39.44% for TIPS versus 72.51%/33.04% for the geometric baseline on AmbigQA/CLAMBER, respectively: gains of 6.08 and 6.40 percentage points. TIPS achieves higher mean macro-F1 in every model–dataset setting, showing that the advantage is consistent across the evaluated model families and tasks. We added these controls to Section 4.3 and Appendix D.5 (Table 12).
The distinction is how the same token-cloud geometry is summarized. Aggregate distance statistics describe typical separation and variability, while TIPS captures how connected components merge and cycles persist across distance scales. Clouds with similar aggregate distances can differ in these relational structures. These results support the predictive value of multiscale connectivity and loop descriptors beyond the tested aggregate geometric summaries.
Question 3 & Weakness 3: Theorem 1 provides a guarantee for binary ill-posedness identification under the stated assumptions. Specifically, when reference-class margins exceed representation distortion, the selected connectivity and loop descriptors separate well-posed questions from the analyzed nonunique classes with separated or cyclic structure. The theorem thus quantifies when these signatures are sufficiently robust to support identification.
Our experiments further demonstrate the utility of persistent homology for fine-grained ill-posedness classification. On CLAMBER, the seven descriptors support classification into a well-posed category and eight ill-posedness subtypes across eight models. The theoretical and empirical contributions therefore complement each other: the theorem establishes rigorous conditions for identification, while the experiments show that the representation is useful beyond binary detection for distinguishing a broader range of semantic categories.
Question 4 & Weakness 4: Topology and retrieval play complementary roles in TIPS: topology determines which positive and negative examples are selected, while their activation differences provide the intervention in the model's native space. Topology therefore conditions the steering direction rather than serving as the direction itself.
A local-smoothness interpretation motivates this design: if behavior-conditioned mean activations vary smoothly in topology space, nearby examples yield a local contrast whose approximation error depends on neighborhood radius and sampling error. This provides a rationale for using topological similarity to select relevant contrasts rather than averaging across heterogeneous examples.
The key contribution is therefore topology-conditioned construction of the local contrast, not query-specific retrieval alone. TIPS compares contrasting response behaviors within neighborhoods defined by similar connectivity and loop patterns. These neighborhoods determine which activations enter the contrast and consequently shape the resulting steering direction. Constructing and applying the direction in activation space is how the topological selection is translated into an intervention: topology specifies the comparison, while activation differences implement it.