- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Prefix Semantic Dynamics (PSD), which represents an evolving response as a trajectory of cumulative-prefix embeddings and characterizes it using correlation dimension, semantic recurrence, and tortuosity. It further proposes a generation controller that uses these descriptors and prompt alignment to select among candidate continuations, with experiments demonstrating improvements in the targeted geometric metrics.
Critical strengths
-
A useful process-oriented perspective. The paper examines how semantic content develops during generation rather than evaluating only the completed response. Cumulative-prefix trajectories provide a unified representation for both analysis and generation control.
-
Relatively comprehensive analysis. The experiments include controlled text perturbations, sensitivity analyses, multiple generator–representation configurations, and cross-representation evaluation. These analyses help characterize the behavior of the proposed descriptors and clarify their dependence on measurement settings.
Critical weaknesses
-
Insufficient evaluation of real-world writing quality. The paper does not evaluate performance on dedicated writing benchmarks, such as WritingBench or LongBench-Write. Although evaluation with a different representation model reduces concerns about using the same model for selection and evaluation, it still relies on a similar embedding-based metric framework. Consequently, the reported improvements do not adequately establish gains in coherence, informativeness, completeness, or overall writing quality.
-
Incomplete assessment of computational cost. Generating four candidates per controlled round and encoding their extended prefixes introduces additional inference overhead. The reported latency ratio of approximately 1.33 in one configuration reflects parallel execution and should not be interpreted as only a 33% increase in total computation. Moreover, Table 12 shows that, with the 9B generator and Harrier representation, median runtime increases from 59.7 to 121.1 seconds, approximately doubling. The paper should report total generated tokens, representation-processing costs, or GPU time, and compare quality gains under matched computational budgets.
-
Insufficient evidence for the necessity of the geometric descriptors. The current baselines do not establish whether the full PSD formulation offers substantial benefits over simpler candidate-selection criteria, such as prompt similarity or semantic novelty relative to previous content. Furthermore, TextOnlyRouter receives only the latest 900 words, making it an imperfect comparison with a method that uses cumulative-prefix history. Stronger information-matched baselines and descriptor-level ablations are needed to isolate the contribution of the proposed geometry.
Questions for authors
-
Evaluate the method on dedicated writing benchmarks and include blinded assessments of coherence, semantic repetition, informativeness, and overall quality.
-
Add simple relevance-and-novelty selection baselines and ablate each geometric descriptor, reporting both output quality and computational cost under matched budgets.
2 comments
Thank you for your response. However, my main concerns about soundness remain unresolved, so I will maintain my original score.
We thank Reviewer K5iX for the specific comments. Our response to each point is below, with the experiments described in the general response.
Evaluation of writing quality. Of the qualities listed, semantic repetition is the one the controller is designed to reduce, and the revision evaluates it with a model outside the pipeline. PSD Control's responses reuse fewer already-active features than Continue's on the same prompts, in all 13 held-out domain groups (general response, part 1, Figure 4a). Content-word coverage of the prompt increases by 2.6–4.9 percentage points, and the factual-support audit excludes a loss larger than 0.057. Table 8 also applies PSD to LongBench-Write answers, to check that the measurement carries over to that benchmark.
Computational cost. We agree that the 1.33× ratio should not be read as total compute. It was wall-clock latency under parallel candidate generation. Table 20 reports about 34 generation calls per response versus 10 for Continue, 3.4–3.5× the generated tokens and 3.85–4.05× the representation tokens. PSD Control, PhaseAgnosticPareto and TextOnlyRouter all generate four candidates per controlled round, so they are compared at matched candidate budget.
Necessity of the geometric descriptors.