acceptodds
Withdrawn from ICLR 2027

Semantic Steering via Hyperbolic Geometry

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As modern text-to-image models approach photorealism, reliable semantic steering becomes essential to guardrail generation. Steering methods operate through linear manipulations in Euclidean spaces, which often provide limited stability and locality under strong interventions. We introduce HypSteer, a steering framework operating in hyperbolic space that controls semantic attributes through geometry-aware directions in a structured representation space. Using a state-of-the-art hyperbolic vision–language encoder, HypSteer defines semantic steering directions from positive and negative prompt sets and applies them to arbitrary inputs via parallel transport, yielding geodesic edits that follow the semantic organization of the space. A lightweight adapter connects these hyperbolic representations to frozen off-the-shelf T2I backbones. Across four safety benchmarks and four generative backbones, HypSteer achieves state-of-the-art safety results, retaining image quality, showing that hyperbolic steering is a practical and flexible alternative.

Primary area:
generative models
Keywords:
AI Safety, T2I generation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_3DagOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

HypSteer derives concept directions in a hyperbolic vision-language embedding space, transports them to new prompt embeddings, and applies geodesic edits. A learned adapter connects these embeddings to frozen image generators. Evaluation covers safety suppression, retention, retrieval, and human judgments across four backbones.

Critical strengths

The geometric formulation is interesting, and the evaluation spans several backbones and safety benchmarks. The reported gains, particularly on SD3.5, suggest that the approach can be useful in practice.

Critical weaknesses
  1. The geometric interpretation is stronger than the method establishes. Sections 3-4 argue that steering remains in meaningful entailment-cone intersections, but Eqs. (8)-(9) transport a direction and follow a geodesic without imposing a cone-membership constraint. Staying on the hyperboloid does not itself guarantee semantic validity or preservation of the original concept. The limited cone-retrieval examples do not establish this claim for arbitrary prompts or strong interventions.

  2. The claimed advantage over Euclidean steering is not consistently supported. In Table 9, HypSteer has worse FLUX NudeNet results (31.65 vs. 27.85); on SD3 it improves suppression but worsens FID (47.86 vs. 39.21) and LPIPS (0.69 vs. 0.45). These are mixed trade-offs. Since reducing T5 prompt strength also substantially improves suppression, the role of geometry needs clearer isolation under matched conditioning and tuning. The rasterized labels in Figures 2(a) and 3 are also difficult to read, making the supporting evidence harder to inspect.

Questions for authors
  1. What guarantees, or directly measures, that the transported geodesic stays in the intended cone intersection while retaining the original concept?
  2. Does the Euclidean control use the same encoder information, adapter, T5 scale, and tuning budget? Please reconcile the broad superiority claims with Table 9 and provide legible versions of Figures 2(a) and 3.
Reviewer_wxHoOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

HypSteer controls a target concept in text-to-image generation by editing text representations in hyperbolic space. A frozen HyCoCLIP encoder is used, and the editing direction is built from Fréchet means of prompt embeddings with and without the concept. The direction is carried to a new prompt by parallel transport and applied along a geodesic, and an adapter maps the edited representation into a frozen generator. The method is evaluated on four backbones and four safety benchmarks, plus retrieval, multi-concept control and human evaluation.

Critical strengths
  1. Concept suppression works across several frozen backbones without retraining the generator, and the effect is sometimes large: on SD3.5 with Ring-a-Bell, the NudeNet rate falls from 50.63 to 6.33. The adapter still needs training, so the framework is not entirely training-free.

  2. Four backbones and four safety benchmarks are covered, with text alignment, image quality and human judgments reported, which makes it easier to judge whether the approach transfers.

Critical weaknesses
  1. Extremely poor presentation: The quality of the figures and tables is exceptionally low. Due to severe compression, most figures are completely illegible, and nearly all legends are unreadable. Furthermore, the information conveyed by these figures is very limited, making the paper difficult to follow.

  2. It is hard to tell how much of the gain comes from hyperbolic editing. The method changes the encoder and its adapter, applies geodesic editing and scales down T5 conditioning, and the Euclidean baseline in the appendix does not clearly state whether it uses the same HyCoCLIP–adapter pair. Evaluation is also limited to HyCoCLIP, so transfer to other hyperbolic encoders is untested.

  3. HAC and DIET already study concept unlearning in hyperbolic space, and the paper does not explain what its formulation adds beyond them.

  4. Preservation is not compared at similar suppression levels. On FLUX the NudeNet rate is 31.65 for HypSteer against 27.85 for the Euclidean baseline, SD3 FID is 47.86 against 39.21, and SDXL CLIPScore drops from 0.32 to 0.27.

Questions for authors
  1. Which preservation properties follow from the geometry and which are empirical? Geodesic editing keeps points on the manifold but does not keep them inside the entailment cones. Please state what the localized-editing and monotonicity claims cover, with non-target preservation and monotonicity results against ordinary vector editing.

  2. How are CLIP pooled, CLIP token and T5 token conditions handled on each backbone? Appendix B.1 describes SDXL as using a single 768-d pooled CLIP embedding, which differs from the standard setup. Are the Flickr30k adapter split and the Table 1 evaluation split disjoint?

  3. Table 3 reports 67.00 on a 0–5 relevance scale: is this a normalized mean or a percentage above a threshold? Are the Table 4 numbers residual nudity or successful censorship? Please add the original-model and Euclidean-baseline results, sample counts and intervals.

Reviewer_Lw79OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HypSteer edits text representations in the Lorentz space of a pretrained HyCoCLIP encoder. It computes Frechet means of concept-present and concept-absent prompts, obtains a tangent-space concept direction, transports it to a new prompt, and applies a geodesic update. A learned MLP adapter maps the edited representation to the conditioning space of frozen text-to-image models. Evaluation covers concept retrieval, safety control across four generative backbones and four benchmarks, and multi-concept editing, with adapter, steering-strength, prompt-scale, and Euclidean-baseline analyses.

Critical strengths
  1. The method gives a concrete, reusable way to apply a concept direction at different prompt locations without retraining the generative backbone. Retrieval and multi-concept experiments provide useful evidence that the interface can support semantic control beyond a single safety setting.

  2. The evaluation includes meaningful controls rather than only comparisons with unedited generators. Table 1 checks adapter fidelity, Figure 5 examines steering strength and T5 conditioning, and Table 9 compares with Euclidean steering. For SDXL, Table 9 reports improvements in both NudeNet (30.51 to 26.58) and FID (63.87 to 60.43), supporting practical value.

Critical weaknesses
  1. The methodological novelty appears limited. The hierarchical representation and entailment structure come from HyCoCLIP, while HySAC already explores safety-directed traversal in hyperbolic vision-language space; both are appropriately cited. HypSteer's distinct contribution is the construction and transport of a contrastive concept direction and its connection to frozen generators. This is a meaningful difference in operation and task, but the Frechet mean, logarithmic/exponential maps, and parallel transport are established tools. The paper does not sufficiently establish that this combination provides a substantial advance over simpler editing rules within the same representation and interface.

  2. The independent contribution of hyperbolic geometry remains insufficiently isolated. The description of Table 9 does not clearly establish whether the Euclidean baseline matches the encoder, adapter, T5 prompt scale, and strength-selection budget. Table 1 supports adapter fidelity for ordinary conditioning, and the lambda sweep shows that editing matters within HypSteer; neither isolates geometry from these other factors. Figure 5b confirms that T5 rescaling itself materially changes safety and alignment. Moreover, Table 9 shows trade-offs: on SD3, NudeNet improves from 31.65 to 27.85 while FID worsens from 39.21 to 47.86 and LPIPS from 0.45 to 0.69; on FLUX, NudeNet worsens from 27.85 to 31.65 despite better FID. These results do not establish a consistent advantage at matched suppression or preservation.

I recommend weak rejection because the methodological advance is limited and the evidence does not yet establish the independent benefit of its central geometric construction.

Questions for authors
  1. Clarification: For Table 9, please specify which encoder, adapter, T5 prompt scale, concept-prompt sets, and strength-selection procedure are used by each method. Which factors are identical? This would determine whether the comparison already isolates the editing geometry.

  2. Decision-relevant evidence: With HyCoCLIP, the adapter, conditioning weights, and generation settings fixed, can you compare the transported geodesic update with a simpler tangent-space edit on one representative backbone, using equal tuning budgets and reporting preservation at matched suppression? If existing results already support this comparison, please identify them. This would help establish whether the specific geometric construction provides a substantive contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.