- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper models repeated language-model rewriting as a stochastic transmission process. Semantics are operationalized through the action selected by a frozen receiver in a fixed task context: messages inducing the same action belong to the same executable fiber. Strong lumpability characterizes when the quotient process has closed Markov dynamics. A killed operator, which removes trajectories on their first exit from the initial fiber, is used to analyze first-exit times and survival probabilities. The paper also studies observable repair based on receiver outputs and uses two receivers to report intersection and union events. Experiments cover multiple tasks, rewriter scales, transmission conditions (conservative, mixing, and compression), and held-out trajectories and receivers.
Critical strengths
- The paper clearly separates surface change from receiver-defined executable behavior. Its point that identical current actions can have different future survival is important and is formalized rather than treated only as an intuition.
- The theoretical results are tied to explicit assumptions. Strong lumpability, killed-process survival, and repair domination are stated as mathematical properties with conditions, rather than as unconditional empirical claims.
- The paper distinguishes receiver agreement from correctness against task labels and explicitly preserves disagreement between two receivers through intersection/union reporting.
Critical weaknesses
- The central notion of semantic survival is defined by a fixed candidate set and the actions of particular receivers. The paper does recognize receiver dependence, but broad statements about semantic preservation should remain explicitly qualified as receiver-defined executable preservation. Candidate-set changes and unconstrained user interpretations are outside the current operational metric.
- The operator estimates use design trajectories and are evaluated on held-out trajectories, which is a sound design. Still, some estimated rows may be based on limited observations, and longer-horizon predictions compound estimation error. The paper would benefit from a concise summary of effective per-state sample sizes and uncertainty for the most important language-model estimates in the main text, with sparse-state behavior made easy to locate.
- Repair is evaluated with two receivers that trigger when their public actions disagree or either action is invalid, and a repairer that receives the current and initial messages. This is a specific feedback protocol. The results support claims under that protocol; they do not establish that arbitrary repair systems improve semantic survival. The paper should keep this scope prominent when summarizing practical implications.
- Because a receiver chooses among eight candidate actions in the main relay tasks, semantic changes outside the supplied candidate set are not measured. This is a limitation of the measurement setup rather than a flaw in the Markov analysis; an explicit sensitivity study over candidate-set construction would strengthen external validity.
Questions for authors
- Could the authors summarize the distribution of per-state design counts and uncertainty in the estimated transition rows used for the language-model survival predictions?
- How sensitive are the main results to the construction of the eight-candidate action set, including distractor difficulty and near-duplicate candidates?
- Which repair mediation outcomes are most common across the main task conditions, including cases where repair changes a pre-repair survivor into an exit?
4 comments
Response to Reviewer 56dJ
W2: Sparse states and uncertainty in estimated transition rows
Critical weakness 2:
The operator estimates use design trajectories and are evaluated on held-out trajectories, which is a sound design. Still, some estimated rows may be based on limited observations, and longer-horizon predictions compound estimation error. The paper would benefit from a concise summary of effective per-state sample sizes and uncertainty for the most important language-model estimates in the main text, with sparse-state behavior made easy to locate.
Question 1:
Could the authors summarize the distribution of per-state design counts and uncertainty in the estimated transition rows used for the language-model survival predictions?
A2 — Response:
Thank you for highlighting sparse transition-state occupancy. We audited the effective source-transition exposures used to estimate the killed operators. The counting procedure includes a source transition only while its trajectory has survived through the source generation, matching the first-passage event.
Among design-observed rows of the three-variable empirical state representation, 5.64% have zero at-risk source exposures, and 23.38% have only 1–9 exposures. These counts show that the empirical transition matrix contains both well-supported and sparsely supported rows.
We also performed a trajectory-level design bootstrap with 2,000 resamples per condition/evaluator group. The mean width of the resulting horizon-grid prediction intervals is 0.111. Resampling complete trajectories respects the dependence among successive generations and provides a direct measure of finite-design uncertainty.
Alongside these diagnostics, we included an exposure-matched pooled hazard baseline and observable-state ablations. In the current manuscript, pooled hazard achieves MAE 0.0461 and the three-variable killed operator achieves 0.0382 across 720 held-out groups. The state ablation reports A0 = 0.0461, A3 = 0.0403, and A7 = 0.0382. These results suggest that observable state information improves aggregate prediction, while the sparse-row and bootstrap statistics explain why uncertainty needs to be reported together with point estimates.
The exact killed-operator survival identity continues to describe the specified microstate process. The fitted operator is a finite-data approximation, and its precision depends on available at-risk exposures. We have brought this statistical distinction into the appendix discussion and the main prediction summary.
We appreciate this constructive suggestion, which gives readers a clearer picture of how operator estimates are supported by the actual trajectory data.
Revised manuscript: Section 9, Observation 3 and Figure 4; Appendix N, prediction-error construction; Appendix P, State occupancy and uncertainty, Figure 11, and Microstate interpretation.
Response to Reviewer 56dJ
W3: Protocol-specific repair and induced exits
Critical weakness 3:
Repair is evaluated with two receivers that trigger when their public actions disagree or either action is invalid, and a repairer that receives the current and initial messages. This is a specific feedback protocol. The results support claims under that protocol; they do not establish that arbitrary repair systems improve semantic survival. The paper should keep this scope prominent when summarizing practical implications.
Question 3:
Which repair mediation outcomes are most common across the main task conditions, including cases where repair changes a pre-repair survivor into an exit?
A3 — Response:
We appreciate the request for a clearer characterization of the implemented repair protocol. Our mechanism evaluates each proposed rewrite using two frozen receivers. A disagreement or invalid output triggers intervention, and the repairer receives the current message together with the stored initial seed. The paired trajectory analysis measures the full downstream effect of this policy.
For immediate mediation, the proposed and repaired messages form four mutually exclusive outcomes: rescued exit, induced exit, preserved survival, and preserved exit. Across the original study's 3,287 triggered repair events, there are 1,313 rescues and 116 induced exits. The remaining 1,858 triggers preserve the current event status. The detailed mediation breakdown gives 1,071 preserved-survival and 787 preserved-exit cases.
In the 35B, 32-generation mixing panel, the task-level rescue/induced-exit counts are 22/0 (Attribute), 66/0 (Banking77), 39/0 (Intent), and 79/1 (Tool). These are immediate pre/post-repair comparisons, and they help explain why favorable immediate interventions are more frequent in that panel.
The first-passage policy effect is measured separately across the full subsequent relay. A rescued proposal can still be followed by later semantic exit, and a repaired message changes the input received by subsequent rewriters. Theorem 6.1 supplies an elementwise-domination condition for a guarantee at every horizon; the observed mediation outcomes and paired survival curves characterize the actual intervention implemented in our study.
The new direct seed-restoration comparison provides an additional practical control. Under matched 35B mixing trajectories, both LLM repair and seed restoration improve horizon-32 survival relative to unmodified transmission, with rates 0.8281 and 0.7653, respectively, compared with 0.6602. LLM repair has the higher observed survival by 0.0628, while both interventions improve over the unmodified policy.
We have made the protocol, immediate event categories, and downstream policy estimand clearer in the revised manuscript. The complete four-cell task-level breakdown is provided here as an additional response statistic; the paper presents aggregate mediation counts and per-condition trigger/net measurements.
Revised manuscript: Section 6, Theorem 6.1; Section 9, Observation 5; Appendix S, Immediate and downstream effects and Relation to domination; Appendix R, direct seed-restoration comparison, Figure 15 and Table 49.
Response to Reviewer 56dJ
W4: Sensitivity to distractor difficulty
Critical weakness 4:
Because a receiver chooses among eight candidate actions in the main relay tasks, semantic changes outside the supplied candidate set are not measured. This is a limitation of the measurement setup rather than a flaw in the Markov analysis; an explicit sensitivity study over candidate-set construction would strengthen external validity.
Question 2:
How sensitive are the main results to the construction of the eight-candidate action set, including distractor difficulty and near-duplicate candidates?
A4 — Response:
Thank you for proposing a candidate-set sensitivity analysis. We conducted a predeclared hard-distractor experiment for Banking77 and manual intent routing, with 64 episodes per task. Each episode retains eight candidate actions, while the harder condition uses ontology-near alternatives designed to create closer operational distinctions.
We evaluated the same unrepaired message trajectories under the original and harder candidate sets. Because the candidate context defines the executable partition, the initial receiver action is computed independently under each candidate set before constructing its first-passage survival event. Holding the rewrite trajectory fixed isolates the effect of the changed receiver context on the measurement.
At generation 32, the results are:
| Task | Original candidates | Hard distractors |
|---|---|---|
| Banking77 | 0.562 | 0.344 |
| Manual intent | 0.609 | 0.578 |
| Two-task average | 0.586 | 0.461 |
The paired average change is −0.1250, with 95% interval [−0.1953, −0.0547]. Banking77 has a larger reduction (−0.2188); the manual-intent reduction is −0.0313, with an interval spanning zero. This task heterogeneity is informative because the candidate distinctions differ in how closely they overlap semantically.
The experiment confirms that executable survival depends on the receiver's declared action context. It also provides a concrete sensitivity measurement supporting the scope discussion.
We appreciate the suggestion, which strengthens the paper's treatment of external validity and clarifies how action-set geometry affects the observed survival event.
Revised manuscript: Section 8, Executable relay tasks; Section 9, Table 1 and Scope and deployment cost; Appendix K, task construction; Appendix R, Candidate-set difficulty sensitivity, Figure 16 and Table 50.
Response to Reviewer 56dJ
We sincerely thank the reviewer for the thoughtful and constructive assessment. We appreciate your recognition of the operational semantic definition and the separation between executable preservation and gold task correctness. Your suggestions led us to clarify the fixed-context setting and add analyses of sparse transition estimates, repair mediation, and candidate-set difficulty.
W1: Receiver-defined meaning and limits of the action set
Critical weakness 1:
A1 — Response:
We agree with the importance of this qualification. Executable semantic survival is defined with respect to a frozen receiver and a fixed task context. The receiver's initial selected action determines the basin, and the survival event requires every intermediate message to remain in that basin.
This operational definition makes changes in tool routing, banking intent, and referent selection directly measurable. Gold task correctness is reported as a separate quantity. For example, an initially incorrect receiver may consistently preserve its original choice, producing high receiver-defined survival while remaining incorrect relative to the task label. Retaining both measurements makes their interpretation clear.
We have strengthened the description of the eight-candidate-per-episode benchmark and the context dependence of the induced fibers. The sensitivity analysis with harder distractors, discussed in A4, further quantifies how changing the candidate action set affects survival. For richer action spaces, the same mathematical formulation requires an explicit executable outcome criterion suited to the task.
We appreciate this suggestion and have made the operational scope prominent in the formal definitions, experiment description, and discussion.
Revised manuscript: Section 3, Executable Semantic Transmission; Section 8, Executable relay tasks; Section 9, Scope and deployment cost; Appendix K, task construction; Appendix U, Correctness and semantic survival.