RelayTTS: Structure-Aware Verifier Serving for Edge Test-Time Scaling

Verifier-guided test-time scaling (TTS) offers a practical route to stronger reasoning on memory-constrained devices, where a larger model may not fit. Its search procedure naturally produces a live frontier of parent groups, each defined by a shared parent prefix. Existing TTS serving systems discard this structure by scoring candidates independently and switching between generator and verifier at model granularity. The result is repeated parent-prefix computation within scoring passes and repeated model movement across them. We present RelayTTS, which makes the live frontier the unit of verifier serving. Parent-complete score packing evaluates shared parent prefixes once across sibling candidates, while a frontier-aware planner trades score-pack capacity against the resident verifier prefix. Layer relay supplies the nonresident blocks only during scoring. Across search procedures and model scales, RelayTTS achieves a 1.91× gain in effective-token throughput and cuts end-to-end latency by 47.8% on average, with a maximum reduction of 88.3%.