- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies inference-time refinement of the internal latent resolution of byte-level models with learned tokenization. Prefix-level decisions about restoring full native resolution are more reliably effective than fine-grained boundary placement, improving over gate-count-based allocation on Python code at matched latent budgets.
Critical strengths
I find the idea of predicting continuation-likelihood gain per added latent from the observed prefix interesting. Such predictability is not obvious, and the matched-budget comparisons and replication on independent Python test sets support the usefulness of this approach.
Critical weaknesses
The main evaluation relies on restoring positions after artificially thinning the native support. It is unclear how well this setting represents practical use under a limited compute budget, making the practical relevance of the results harder to assess.
Questions for authors
Q1. The paper motivates prefix-level allocation under a latent budget. Could you illustrate this with a simple, possibly idealized use case, explaining how the artificial thinning and per-prefix restoration decisions relate to the compute constraints in that setting?
Q2. How do you interpret the larger observed gains on Python than on JavaScript? Could this reflect the router being trained on Python or differences in the evaluation data?
Thank you for your thoughtful review!
Why this question matters. Learned byte models choose which byte positions enter their expensive middle network. Once a model is trained, those positions also define a possible test-time choice: we can retain a reduced subset for a prompt and restore the model’s own native positions when finer resolution is valuable. The central question of ActiveMesh is whether the observed prefix tells us when that restoration is worth its cost.
The distinction between boundaries and requests matters here. Attention-based boundary scores appear plausible, yet they do not reliably identify which individual positions improve future prediction. Across requests, however, the value of returning to native resolution varies substantially. Our prefix router captures part of that variation and improves BPB over count-based allocation on two Python test sets at the same added-latent budget. This suggests a useful role for learned tokenization beyond setting one compression pattern during training: its resolution can be allocated according to the needs of each prompt.
Downstream performance. We agree that better likelihood should be tested against a task decision. For a code-continuation ranking task with two available candidates, we added a simple task-aware rule: score both candidates on the reduced mesh and restore native resolution when their scores are close. On 1,024 choices, it reaches 81.69% accuracy, compared with 81.74% for full native and 80.96% for count allocation at the same added-latent spend. It uses 22.7% of the extra native states available beyond the reduced mesh. The route’s advantage over count is +0.73 percentage points [0.15, 1.32]. Candidate ranking gives the policy a direct signal of uncertainty; extending this result to free-form generation is an important next experiment.
Measured efficiency. We also measured prefill quality and time across 10%, 25%, 50%, and 75% escalation. As the route spends more time, its BPB improves from 2.17891 at 93.99 ms to 2.14788 at 101.85 ms.