- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MindCraft, a dynamic framework based on full-sequence flow operators, designed to investigate how concept representations emerge across the depth of large language models. By analyzing the propagation of counterfactual perturbations through stretching and turning operations, the authors seek to unify the perspectives of linear and manifold-based representations. Furthermore, they propose a branching layer to construct hierarchies of concepts. Experiments conducted across various models and tasks demonstrate the potential value of this approach in mitigating hallucinations and biases.
Critical strengths
1.The paper introduces a dynamic framework for understanding how concept representations evolve across model layers, offering an interesting perspective beyond static representation analysis. 2. The proposed full-sequence flow framework provides a systematic mathematical treatment of counterfactual propagation, spectral dynamics, and representation geometry. 3. The experiments cover multiple model families and task domains, demonstrating consistent correlations between branching layers and causal effects. 4. The proposed branching-layer-guided steering method shows promising improvements in reducing hallucinations and social biases, highlighting the potential practical value of the framework.
Critical weaknesses
- Theorem 4.3 relies on strong growth-gap assumptions, but it is unclear whether these assumptions hold in practice. The experiments show a decreasing off-target/on-target ratio, but do not directly demonstrate convergence to the task read-out subspace.
- The link between turning patterns and linear or manifold representations is not fully established. The current results are suggestive, but more direct evidence is needed to support this claim.
- The steering experiments lack comparisons with fixed-layer, random-layer, and layer-sweeping baselines. Without these comparisons, it is hard to tell how much of the improvement comes specifically from choosing the branching layers.
- The branching-layer definition depends on a similarity threshold, which may affect the selected layers. It would be useful to see how sensitive the results are to this threshold, especially for downstream steering.
Questions for authors
- Can the authors provide more direct evidence that the growth-gap assumptions in Theorem 4.3 hold in the evaluated models? How should we interpret the gap between the theoretical convergence result and the observed off-target/on-target ratios?
- What evidence supports the claim that different turning patterns lead to linear versus manifold representations? Could the authors provide a more controlled experiment to test this relationship?
- How does MindCraft compare with fixed-layer or random-layer steering under the same experimental settings? Is selecting the branching layer necessary to achieve the reported improvements?
- How sensitive are the branching layers to the choice of similarity threshold? Does changing this threshold significantly affect the downstream steering performance?
The authors did not engage in a discussion, so my concerns were not addressed and thus I am keeping my score.