acceptodds
Under review as a conference paper at ICLR 2027

TIPS: Topological Ill-Posedness Probing and Steering in Large Language Models

Abstract

Ill-posed questions, including those involving ambiguity, under-specification, or conflicting statements, may admit no valid answer or multiple plausible answers, posing a significant challenge for large language models (LLMs) despite their otherwise strong performance. Existing work often treats LLM reasoning as a black box and focuses on input-output analysis. Can a compact, unified topological representation of internal model states capture diverse sources of ill-posedness and steer reasoning toward responses appropriate to each source? To this end, we study the internal state from a topological perspective by treating the contextual hidden states of its prompt tokens at a single transformer layer as a point cloud, capturing the input problem’s relational structure as encoded by the model. Our analysis shows that zero- and one-dimensional persistent homology—tracking how connected components merge as the distance threshold increases and the emergence and filling of one-dimensional loops in the filtration , respectively—yield seven compact descriptors of ill-posedness that also serve as control signals for steering model reasoning. Thus, we develop a topology-conditioned steering mechanism. It retrieves topologically similar positive and negative examples for each query to form a local activation-space contrast. The resulting query-specific direction modifies internal states during prefill and decoding, guiding the model's responses toward suitable abstention or clarification tailored to the underlying source of ill-posedness. Empirical evaluations on two datasets and open-weight models spanning four families demonstrate the effectiveness of this compact representation for ill-posedness classification, while topology-conditioning significantly improved steering performance.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Persistent Homology, Ill-posedness Detection, Activation Steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Si43OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether the internal representations of large language models (LLMs) can capture different types of ill-posedness, including ambiguity, under-specification, and conflicting statements. Specifically, the authors represent the hidden states of prompt tokens as a point cloud and use zero- and one-dimensional persistent homology to extract seven compact topological descriptors for ill-posedness classification. Furthermore, the paper introduces a topology-conditioned steering mechanism that retrieves topologically similar positive and negative examples to construct query-specific steering directions, guiding models toward appropriate abstention or clarification. Experiments on two datasets and open-weight models from four families demonstrate the effectiveness of the proposed topological representations for ill-posedness classification and the benefits of topology-conditioned steering.

Critical strengths
  1. The paper addresses the challenge that LLMs often generate unsupported answers when faced with ambiguous, underspecified, or otherwise ill-posed questions. It aims to guide models toward more appropriate responses based on the nature of the questions.
  2. Unlike methods that directly use hidden-state vectors for classification, this paper employs persistent homology to characterize the connectivity and loop structures of prompt-token representations, offering a new perspective on how information is organized within LLMs.
Critical weaknesses
  1. The proposed method assumes that the topology of prompt-token hidden states can capture different sources of ill-posedness. However, it remains unclear why the connectivity and loop structures of these representations should correspond to ambiguity, under-specification, or conflicting information.
  2. Although the proposed topological descriptors achieve promising classification results, it remains unclear whether these improvements arise from topological information or more general geometric properties of hidden-state representations.
  3. Theorem 1 establishes class separation under specific reference-geometry and representation-distortion assumptions. However, the extent to which these assumptions hold for actual ill-posed questions remains unclear. Furthermore, the theoretical analysis focuses on particular geometric classes, whereas the experiments involve more diverse forms of ill-posedness.
  4. The proposed method uses topological descriptors to retrieve positive and negative examples, while the actual steering direction is constructed from their activation differences. Therefore, the observed improvements may arise from query-specific retrieval rather than topological information itself.
Questions for authors

See weakness

3 comments

We thank the reviewer for the careful evaluation and constructive feedback. We address each concern below.

Question 1 & Weakness 1: Our motivation is that ill-posedness changes the contextual constraints among question tokens. Ambiguity leaves multiple interpretations viable, under-specification leaves relevant relationships unresolved, and conflicting information introduces incompatible requirements. Because hidden states integrate information across tokens through attention and subsequent transformations, these differences can alter which associations are reinforced and how contextual token representations are arranged.

Persistent homology characterizes this arrangement across distance scales. H0H_0 captures when token groups become connected, distinguishing tightly connected neighborhoods from groups that merge only at larger distances. H1H_1 captures persistent cycles in the proximity structure, describing relationships not captured by component merging alone. Together, these descriptors summarize how token representations are organized, rather than only their average activation or overall spread. This motivates TIPS's use of joint connectivity and loop signatures to identify class-dependent patterns associated with different sources of ill-posedness. We added this motivation to Section 3.1.

Question 2 & Weakness 2: We added a non-topological geometric baseline using the mean, standard deviation, and median pairwise distances from the same token clouds. Both methods use identical data splits, with layer and classifier hyperparameters selected using validation data only. This comparison tests whether simple summaries of token-cloud dispersion can explain the classification performance.

Across eight models and five seeds, average macro-F1 is 78.59%/39.44% for TIPS versus 72.51%/33.04% for the geometric baseline on AmbigQA/CLAMBER, respectively: gains of 6.08 and 6.40 percentage points. TIPS achieves higher mean macro-F1 in every model–dataset setting, showing that the advantage is consistent across the evaluated model families and tasks. We added these controls to Section 4.3 and Appendix D.5 (Table 12).

The distinction is how the same token-cloud geometry is summarized. Aggregate distance statistics describe typical separation and variability, while TIPS captures how connected components merge and cycles persist across distance scales. Clouds with similar aggregate distances can differ in these relational structures. These results support the predictive value of multiscale connectivity and loop descriptors beyond the tested aggregate geometric summaries.

Question 3 & Weakness 3: Theorem 1 provides a guarantee for binary ill-posedness identification under the stated assumptions. Specifically, when reference-class margins exceed representation distortion, the selected connectivity and loop descriptors separate well-posed questions from the analyzed nonunique classes with separated or cyclic structure. The theorem thus quantifies when these signatures are sufficiently robust to support identification.

Our experiments further demonstrate the utility of persistent homology for fine-grained ill-posedness classification. On CLAMBER, the seven descriptors support classification into a well-posed category and eight ill-posedness subtypes across eight models. The theoretical and empirical contributions therefore complement each other: the theorem establishes rigorous conditions for identification, while the experiments show that the representation is useful beyond binary detection for distinguishing a broader range of semantic categories.

Question 4 & Weakness 4: Topology and retrieval play complementary roles in TIPS: topology determines which positive and negative examples are selected, while their activation differences provide the intervention in the model's native space. Topology therefore conditions the steering direction rather than serving as the direction itself.

A local-smoothness interpretation motivates this design: if behavior-conditioned mean activations vary smoothly in topology space, nearby examples yield a local contrast whose approximation error depends on neighborhood radius and sampling error. This provides a rationale for using topological similarity to select relevant contrasts rather than averaging across heterogeneous examples.

The key contribution is therefore topology-conditioned construction of the local contrast, not query-specific retrieval alone. TIPS compares contrasting response behaviors within neighborhoods defined by similar connectivity and loop patterns. These neighborhoods determine which activations enter the contrast and consequently shape the resulting steering direction. Constructing and applying the direction in activation space is how the topological selection is translated into an intervention: topology specifies the comparison, while activation differences implement it.

Dear Reviewer Si43,

As the author discussion period comes to a close, we would appreciate your consideration of our rebuttal and revised manuscript. If our responses and additional evidence have resolved your concerns, we hope you will raise your score accordingly. Thank you again for your time and thoughtful consideration.

The Authors of Submission 37797

Thank you for your response. However, I still find the explanations unclear and remain unconvinced by the arguments. Therefore, I have decided to maintain my original score.

Reviewer_XZVcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TIPS represents a question using seven features that summarize the connectivity and loops in a point cloud of its prompt-token hidden states. A supervised classifier uses these features to detect ambiguity or predict its subtype. The same features select nearby examples with contrasting response behaviors, whose activation difference is used to steer the model toward clarification or abstention. Classification is evaluated across eight models, while steering is evaluated on two. The paper also provides a conditional theorem showing that sufficiently separated reference geometries remain distinguishable under bounded distortion.

Critical strengths
  1. An interesting, compact representation.

  2. A positive fine-grained classification result.

Critical weaknesses
  1. The benefit of topology over simpler signals remains unclear. The features may capture question length, formatting, or general activation geometry rather than ambiguity itself, and matching seven aggregate descriptors to seven individual neurons does not resolve this concern. Comparisons with length/format features, pooled activations, and simple distance statistics on matched or paraphrased questions would better establish topology’s added value.

  2. The steering comparison changes several factors simultaneously. TIPS and AEN differ in example selection, direction construction, and intervention dimensionality, so the improvement cannot be attributed specifically to topological retrieval. A controlled comparison should use the same full-dimensional steering procedure while varying only the retrieval strategy, alongside a global mean-difference baseline and the full-vector variant already considered in the original AEN paper.

  3. More abstention does not necessarily indicate better clarification. The automated judge accepts generic disclaimers or refusals without separately assessing whether the response identifies the correct missing information, while the selected test subsets leave effects on clear questions insufficiently evaluated. The authors should measure clarification specificity and unnecessary abstention, and evaluate unsteered responses using the final judge to establish a consistent baseline.

  4. The theory assumes the key connection between language and geometry. The theorem establishes robust separation when questions have the assumed reference structures and hidden states preserve them, but the paper does not demonstrate that these conditions hold for the evaluated questions. Empirical evidence connecting semantic ambiguity to these geometric structures would strengthen the theoretical interpretation.

  5. Practical reliability and computational cost need fuller reporting. Important details are missing, including PCA dimension, selected layers, token handling, persistence thresholds, runtime, and steering sample sizes and run counts. Reporting these settings, paired uncertainty, and a small human or independent-judge validation would make the practical gains easier to assess, especially given the substantial variability in some steering results.

  6. Figure 1 is difficult to follow. The diagram is crowded, with small text and an unclear distinction between feature extraction, classification, and steering, making it difficult to understand how the topological features determine the intervention. Enlarging the labels, simplifying the panels, and clearly showing the connections between these stages would improve readability.

Questions for authors
  1. Does topology outperform length/format controls and pooled or simple geometric representations under the same data splits and selection procedure?

  2. Does topological retrieval remain better when all steering methods use the same direction construction, activation dimensions, and tuning budget?

  3. How much of the improvement consists of useful, question-specific clarification, and what happens on well-posed questions?

  4. Can you provide the missing feature-extraction settings, runtime costs, steering sample counts, and uncertainty analysis?

5 comments

We thank the reviewer for the careful evaluation and constructive feedback. We address each concern below.

Question 1 & Weakness 1: We added controls using identical data splits and validation-only hyperparameter selection, including layer selection where applicable. Our TIPS outperforms these added baselines. Across eight models and five seeds, average macro-F1 on AmbigQA/CLAMBER is 78.59%/39.44% for TIPS, 77.05%/28.12% for question length, and 72.51%/33.04% for pairwise-distance mean, standard deviation, and median from the same token clouds. We do not believe the full-dimensional mean pooling would be a fair comparison with TIPS. It relies on all hidden coordinates after token averaging (which has 4,096-8192 dimensions), whereas TIPS and other baselines (like AEN) only access a much smaller number of aggregated features (which has only 7 dimensions and only 0.09%-0.17% of full-dimensional mean pooling). Our key point is that TIPS achieves the strongest performance over low-dimensional controls, supporting the value of multiscale connectivity and loop descriptors beyond the tested length and distance summaries.

Question 2 & Weakness 2: We added global mean-difference and activation-neighbor steering controls. Global steering uses the positive–negative mean activation difference across both training banks as a fixed direction. Activation-neighbor steering retrieves positive and negative cases by mean-pooled activation similarity, then uses their mean activation difference as a query-specific direction. TIPS instead retrieves cases using seven topology descriptors. Across all four model–dataset settings, TIPS achieves the highest mean clarification/disclaimer/abstention rates among the evaluated steering methods, exceeding the two controls by 0.81–10.18 and 5.66–21.90 percentage points, respectively. These results support the effectiveness of TIPS to extract representative ill-posedness information to select similar reference cases and construct finer-grained, query-specific steering actions. We added these controls to Section 4.4 and Appendix C.6 (Table 10).

Question 3 & Weakness 3: We added a hierarchical GPT-4.1 audit separating question-grounded from generic accepted responses. Among accepted responses, the five-seed mean grounded fractions on AmbigQA/CLAMBER are 47.13%/15.54% for Llama and 77.25%/64.93% for Qwen, showing TIPS's high-quality steering performance that preserves the original information from the questions. For the steering impact on well-posed problems, among 4,878 well-posed problems wrongly classified as ill-posed and steered, clarification/disclaimer/abstention rates are 52.91%, versus 14.37% for unsteered. This shows our steering could also yield positive performance on the well-posed problem answers, verifying the generalizability of TIPS steering. We added this audit to Section 4.4 and Appendix C.5 (Tables 8–9).

Question 4 & Weakness 5: We thank the reviewer for these suggestions. We added PCA dimensions and selected layers (Table 4), token handling, persistence thresholds, and classifier settings in Appendix B.4; steering sample counts in C.3 (Tables 5–6), and paired uncertainty in C.4 (Table 7). Appendix E reports offline costs (Tables 13–14), classification latency (E.3, Table 15), and steering latency (E.4, Table 16). In particular, paired TIPS–AEN gains average 12.92–34.71 percentage points across settings, with seed SDs of 11.49–21.06, indicating positive mean gains for steering with variability. For run time, TIPS achieves multiple times faster classification latency following approximately 30 minutes of offline classifier training on Llama-3.1-8B, and only adds around 20% generation time during steering.

Weakness 4: Theorem 1 explains when geometric signatures support reliable ill-posedness identification. The reference structures model separated or cyclic organizations of plausible resolutions, while the distortion assumption bounds their discrepancy from observed token geometry. When class margins exceed the corresponding distortion bounds, the selected H0H_0 and H1H_1 descriptors guarantee separation for the analyzed classes. The assumptions thus specify the geometric signal and representation fidelity sufficient for identification. Our experiments complement this analysis by examining class-dependent patterns and classification performance on actual questions, including fine-grained subtypes. The theory provides a quantitative robustness guarantee, while the experiments establish predictive utility across the evaluated categories.

Weakness 6: Thank you for the helpful suggestions on improving Figure 1. We revised the figure by enlarging labels and boxes and removing redundant annotations. The steering panel now explicitly shows neighbor retrieval, activation-difference construction, and intervention. A direct connection from the topological features to retrieval clarifies how they guide steering, separately from classification.

Thank you for the additional baselines on question length and pairwise distance summaries. However, there are still some points not solved. Conceptual gap between topological compression and heterogeneous query ill-posedness: Prior geometric and topological representation methods are largely motivated by neighborhood structure, clustering, or similarity search. By contrast, query ill-posedness (e.g., lexical polysemy, missing referents, underspecified constraints, or presupposition failure) is semantically diverse. Compressing high dimensional hidden states down to 7 scalar summaries aggressively discards semantic and coordinate informatio, yet discarding information does not by itself isolate ill-posedness or guarantee sufficient representational capacity across diverse failure modes. What concrete mechanism allows 7 coarse connectivity and loop descriptors to capture diverse forms of semantic ill-posedness that are lost in simpler geometric projections, rather than merely acting as a low-capacity regularizer?

In addition, my original question asked for evaluation on matched or paraphrased questions (where length/formatting varies while ambiguity status is preserved, or vice versa).

Thank you for the follow-up. We clarify the mechanism underlying topology-based ill-posedness detection and report two paired evaluations using matched or paraphrased questions that hold either question length or the well-posedness label fixed.

1. Mechanism of topology-based ill-posedness detection

TIPS operates on token states already contextualized by the pretrained transformer. These states reflect both lexical content and information integrated from the surrounding context. The topological readout summarizes how these representations are organized in relation to one another, retaining structural signals useful for discrimination rather than reconstructing the question's full meaning.

Our hypothesis concerns shared effects on contextual relationships. Polysemy permits competing interpretations; missing referents or constraints leave associations unresolved; presupposition failures may conflict with learned contextual expectations. These effects can alter information integration and token neighborhoods. The classifier learns class-specific weights over the descriptors, allowing each category to be associated with a combination of connectivity, persistence, and concentration patterns across the token cloud.

Computationally, we examine token relationships across increasing distance thresholds. H0H_0 records component merging, while H1H_1 records cycles that persist before being filled. Persistence magnitudes, entropy, and concentration summarize this organization. Aggregate distance statistics describe overall separation and dispersion, while persistence additionally reflects how neighborhoods connect across scales. Clouds with similar aggregate distances can therefore have different merging and cycle-persistence patterns.

TIPS therefore provides the classifier with summaries of how contextual token neighborhoods connect and how loops persist across distance scales. Our claim is a useful compact structural representation, with empirically demonstrated discrimination, rather than universal sufficiency across all semantic failure modes.

2. Paired evaluations

We evaluated TIPS and the question length and pairwise-distance baselines on held-out AmbigQA questions across eight models and five seeds. AmbigQA’s annotations link ill-posed source questions to official well-posed rewrites, enabling comparisons within source groups. Question length is measured by the number of words. Distance statistics comprise the mean, standard deviation, and median pairwise token-cloud distances. Table entries are percentages: mean ± sample SD across five seeds, after averaging the eight models within each seed.

A. Different lengths, both well-posed. We paired well-posed rewrites from the same source with different question lengths, preserving the well-posedness label. There are 18,099 pair–seed instances (12,421 distinct pairs) and 13,701 question–seed instances. Question accuracy counts each question once per seed; the other metrics are pair-based.

MethodQuestion accuracyBoth questions correctPair disagreement
TIPS80.03 ± 1.0573.00 ± 3.5120.43 ± 2.38
Distance statistics73.54 ± 0.9965.50 ± 3.7022.29 ± 2.23
Question length70.70 ± 1.1556.53 ± 6.1832.50 ± 5.51

TIPS achieves higher question accuracy, a higher proportion of pairs with both questions correctly classified, and lower disagreement than both controls. Because all questions here are well-posed, higher accuracy corresponds to fewer false-positive ill-posedness predictions. Lower disagreement indicates more consistent classification across the tested different-length rewrites.

B. Same question length, different labels. We paired original ill-posed questions with official well-posed rewrites having identical question lengths, including eligible rewrites absent from the original classification dataset. There are 948 pair–seed instances (638 distinct pairs).

MethodAccuracyMacro-F1Both questions correct
TIPS59.77 ± 0.6557.60 ± 0.9721.65 ± 1.38
Distance statistics58.53 ± 0.8056.84 ± 0.5821.26 ± 1.18
Question length50.00 ± 0.0041.24 ± 1.400.00 ± 0.00

TIPS retains predictive information when question length cannot distinguish the labels. It also achieves slightly higher mean accuracy and macro-F1 than the distance-statistics baseline.

Together, these evaluations provide complementary evidence: TIPS is more consistent than the tested controls across same-source, well-posed rewrites of different lengths, and remains informative when ambiguous and disambiguated questions have identical question lengths. They support the advantage of TIPS beyond general comparison on the whole dataset.

If the explanation and the additional evaluations resolve the concerns underlying your assessment, we would be grateful if you would consider raising your score. Thank you again for your time and thoughtful consideration.

Reviewer_ReiZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TIPS, a method for ill-posed question classification. TIPS aggregates the topology of tokens into a compact feature vector, and use it for classification.

While the idea is interesting, a number of items could make the paper stronger:

  1. Why does semantic ambiguity result in differences in geometric patterns? Some logical justification would be more interesting than simple correlation. Perhaps more ambiguous or general terms result in a different pattern than specific terms, as former and latter trigger a different set of attention?
  2. Is there a reason that TIPS appears to have less of an advantage on newer models such as Gemma? It seems that for CLAMBER results, the TIPS performance is not too much stronger than zero-shot or few-shot prompting. Besides, since the proposed method involves training, it may make sense to post-train models on the same training sets for a more comprehensive comparison.
  3. Continuing from 2. How do the frontier models such as GPT-6 and Claude 5.5 perform on this task? Naturally you would not be able to replicate TIPS on their private weights, but the performance would still be useful in comparison.
Critical strengths

TIPS aggregates the topology of tokens into a compact feature vector, and use it for classification.

Critical weaknesses

While the idea is interesting, a number of items could make the paper stronger:

  1. Why does semantic ambiguity result in differences in geometric patterns? Some logical justification would be more interesting than simple correlation. Perhaps more ambiguous or general terms result in a different pattern than specific terms, as former and latter trigger a different set of attention?
  2. Is there a reason that TIPS appears to have less of an advantage on newer models such as Gemma? It seems that for CLAMBER results, the TIPS performance is not too much stronger than zero-shot or few-shot prompting. Besides, since the proposed method involves training, it may make sense to post-train models on the same training sets for a more comprehensive comparison.
  3. Continuing from 2. How do the frontier models such as GPT-6 and Claude 5.5 perform on this task? Naturally you would not be able to replicate TIPS on their private weights, but the performance would still be useful in comparison.
Questions for authors

See above

2 comments

We thank the reviewer for the careful evaluation and constructive feedback. We address each concern below.

Question 1 & Weakness 1: Semantic ambiguity can alter relationships among contextual token states. A well-posed question constrains the intended interpretation, whereas ambiguity permits competing resolutions and missing context leaves relationships unresolved. Through attention and subsequent transformations, these differences may change information integration, token distances, and neighborhood structure.

Your suggestion about attention is consistent with this intuition: competing interpretations or missing constraints may alter information integration across tokens and, consequently, hidden-state geometry. We added this motivation to Section 3.1.

Question 2 & Weakness 2: To provide a full picture, we added evaluations of two frontier models including GPT-6 Astra and Claude Opus 5.5 using the default reasoning-effort. Using the same prompt templates, the highest macro-F1 scores were 58.58%/48.36% for GPT-6 Astra and 60.47%/50.07% for Claude Opus 5.5 on AmbigQA/CLAMBER. Both are much lower than TIPS's 77.98%–79.32% across eight open-weight models on AmbigQA. On CLAMBER, through slightly better than TIPS, their macro-F1 scores are around 50%.

We would like to emphasize that for our evaluation spans Llama 3.1/3.3 (2024), Ministral 3 (2025), and Qwen3.5/3.6 and Gemma 4 (2026), TIPS’ remains stable across eight models: macro-F1 ranges from 77.98% to 79.32% on AmbigQA and 37.10% to 41.97% on CLAMBER. The advantage of TIPS seems to be model and data dependent. Smaller performance advantages in some cases is likely due to the fact that training data used by certain models have higher overlap with certain benchmarks. In general, TIPS outperforms the evaluated prompting baselines across all eight models.

Besides performance gains in detecting ill-posedness, we would also like to point out that TIPS reuses its topological features to retrieve contrastive examples and construct query-specific activation-steering directions whereas the evaluated prompting baselines provide classification decisions rather than internal interventions. This steering is a unique ability by TIPS. In runtime measurements on Llama-3.1-8B, TIPS is 3.8–6.0× faster than zero-/few-shot prompting and 29.8× faster than few-shot CoT, following approximately 30 minutes of offline classifier training.

Finally, we wanted to clarify that TIPS does not require training the LLM itself. It accesses its internal activations for detecting ill-posedness and steering, but the LLM itself is not trained or fine-tuned. Only the detection model outside LLM is trained in TIPS. This is consistent with related work like AEN. TIPS offers a lightweight alternative that trains only a compact classifier and reuses frozen-model representations for steering, without the model-parameter optimization required by post-training. I believe our new comparison with frontier models also helps provide a complete picture. We added the latency comparisons to Appendix E (Tables 13–16).

Question 3 & Weakness 3: This is an interesting point. As you noted, TIPS and AEN require internal activations that are unavailable through standard black-box APIs. We have added several prompting baselines for comparison on GPT-6 Astra and Claude Opus 5.5. We have compared TIPS with these models with respect to (1) detecting ill-posedness and (2) responding to the source of ambiguity or missing information through clarification, a disclaimer, or abstention. In summary, TIPS outperforms frontier models on both objectives. The results significantly strengthen our results. Please see further details below. Thanks again for the suggestion.

Using the same zero-shot, few-shot, and few-shot CoT templates, the highest macro-F1 scores were 58.58%/48.36% for GPT-6 Astra and 60.47%/50.07% for Claude Opus 5.5 on AmbigQA/CLAMBER. Both are much lower than TIPS's 77.98%–79.32% across eight open-weight models on AmbigQA. On CLAMBER, through slightly better than TIPS, their macro-F1 scores are around 50%. We also submitted the original questions to frontier models for answering. Under the same GPT-4.1 judge, GPT-6 Astra acknowledged ill-posedness through clarification, a disclaimer, or abstention in 10.55%/30.00% of responses on AmbigQA/CLAMBER; Claude Opus 5.5 did so in 17.22%/42.50%. For context, TIPS steering achieves a better 76.23%/57.02% with Llama-3.1-8B and 34.97%/36.38% with Qwen3.5-9B.

These findings highlight two objectives in our TIPS paper: detecting ill-posedness and using that signal to guide response behavior. TIPS addresses both objectives on open-weight models by extracting internal-state signals and using topology-conditioned contrasts to steer responses. It therefore offers a complementary approach that links compact ambiguity detection to targeted response steering. We added frontier-model results to Sections 4.3–4.4, with full details in Appendix F (Tables 17–18).

Dear Reviewer ReiZ,

As the author discussion period comes to a close, we would appreciate your consideration of our rebuttal and revised manuscript. If our responses and additional evidence have resolved your concerns, we hope you will raise your score accordingly. Thank you again for your time and thoughtful consideration.

The Authors of Submission 37797

Comments on the paper

2 comments

Dear Reviewers,

As the discussion period draws to a close, please read and acknowledge the authors’ rebuttal within the next 24 hours. I also encourage you to engage in the discussion and share any remaining questions or concerns.

Thank you for your time and thoughtful reviews.

Best,

Your AC

We thank the reviewers for the careful evaluation and constructive feedback. Your suggestions have helped us strengthen the evidence and presentation. We have provided point-by-point responses and revised the manuscript, highlighting main-text additions in blue and reporting detailed results in the appendix.

We also thank the area chair for facilitating the discussion and encouraging engagement with our rebuttal. Below, we connect the principal updates to the reviewers' questions (Q) and weaknesses (W).

1. Semantic motivation and theoretical scope — ReiZ Q1/W1; XZVc W4; Si43 Q1/W1, Q3/W3. We added an explanation of how competing interpretations, missing constraints, and conflicting information may affect contextual token relationships, motivating connectivity and loop descriptors. Our responses explain the theorem's conditional separation guarantee for the analyzed geometric classes and its relationship to experiments on broader ill-posedness subtypes. We added this motivation in Section 3.1.

2. Classification controls — XZVc Q1/W1; Si43 Q2/W2. We added question-length and pairwise-distance baselines using identical splits and validation-based selection. Across eight models and five seeds, TIPS achieves higher mean macro-F1 than both controls in all 16 model–dataset settings. Average gains over distance statistics are 6.08/6.40 percentage points on AmbigQA/CLAMBER, supporting the value of the seven descriptors beyond the tested length and aggregate-distance summaries. These results appear in Section 4.3 and Appendix D.5 (Table 12).

3. Steering controls — XZVc Q2/W2; Si43 Q4/W4. We added global mean-difference and activation-neighbor steering controls. The former uses a fixed activation contrast, while the latter retrieves cases by mean-pooled activation similarity to construct a query-specific contrast. TIPS achieves the highest mean clarification/disclaimer/abstention rates among the evaluated methods in all four settings, exceeding these controls by 0.81–10.18 and 5.66–21.90 percentage points, respectively. The query-specific activation-neighbor comparison provides additional evidence for topology as an informative retrieval representation. These controls appear in Section 4.4 and Appendix C.6 (Table 10).

4. Model dependence and frontier comparisons — ReiZ Q2/W2, Q3/W3. Our responses distinguish relatively stable TIPS classification performance across base models from improvements in prompting. We added GPT-6 Astra and Claude Opus 5.5 evaluations using the default reasoning-effort setting. Their best AmbigQA macro-F1 scores are 58.58% and 60.47%, much lower than TIPS's five-seed means of 77.98%–79.32% across eight open-weight models; frontier prompting performs slightly better on CLAMBER but only achieves macro-F1 scores around 50%. We also evaluate ordinary answering on the frontier models; only 10.55% to 42.50% of questions were answered with awareness of ill-posedness, showing that a stronger model does not automatically solve the ill-posed question-answering task and highlighting the importance of our TIPS method for ill-posedness identification and behavior steering. These results appear in Sections 4.3–4.4 and Appendix F (Tables 17–18).

5. Clarification specificity and well-posed questions — XZVc Q3/W3. We added a hierarchical GPT-4.1 audit separating question-grounded from generic accepted responses. Mean grounded fractions among accepted responses range from 15.54% to 77.25% across the four settings. We also showed that steering by TIPS has a positive performance on the well-posed questions. These analyses assess specificity and deployment generalizability beyond overall acceptance rates. The audit appears in Section 4.4 and Appendix C.5 (Tables 8–9).

6. Reproducibility, uncertainty, and efficiency — XZVc Q4/W5. We added PCA dimensions, selected layers, token handling, persistence thresholds, classifier settings, steering sample counts, and paired seed-level variability. In matched CLAMBER timing on Llama-3.1-8B, TIPS is 3.8–6.0× faster than zero-/few-shot prompting and 29.8× faster than few-shot CoT. Offline costs and steering latency are reported separately. These details appear in Appendix B.4 (Table 4), C.3–C.4 (Tables 5–7), and E (Tables 13–16).

7. Figure clarity — XZVc W6. We revised Figure 1 and its caption to clarify feature extraction, classification, neighbor retrieval, activation-difference construction, and intervention, making the role of topological features in steering easier to follow.

We would appreciate your feedback on whether these additions address your concerns. If the rebuttal and revised manuscript resolve the concerns underlying your assessment, we would be grateful if you would consider raising your score. Thank you again to the reviewers and area chair for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.