acceptodds
Under review as a conference paper at ICLR 2027

ReLay: Are We Ready for Latent Routing for Multi-Agent Systems?

Abstract

Large language model (LLM)-based multi-agent systems rely on communication and orchestration for collaborative reasoning. Existing methods often exchange reasoning traces in a fixed textual or latent form. Text preserves precise details but can be verbose, whereas latent representations offer semantic richness at the risk of obscuring critical information. Beyond representation choice, complete traces often contain essential information alongside erroneous, irrelevant, or redundant evidence. These limitations motivate segment-wise routing that combines complementary representations and filters low-utility content. Our empirical analysis further shows that three internal trace signals—entropy, maximum probability, and hidden-state norm—carry predictive information about downstream correctness, suggesting their utility for adaptive routing among TEXT, LATENT, and SKIP. We introduce ReLay, a two-level routing framework for effective multi-agent collaboration. For intra-agent routing, ReLay employs a Signal-Conditioned Utility Router (SCUR) to make segment-wise transmission decisions. SCUR aggregates internal trace signals to estimate the downstream utility of each communication choice. The router is trained on counterfactual receiver outcomes, and its predictions guide the construction of mixed-format messages. For inter-agent routing, an Adaptive Representation Orchestrator (ARO) conditions on the routed message and task state to select the next compatible agent or terminate the interaction. Experiments across reasoning and long-horizon decision-making tasks demonstrate that ReLay achieves the strongest overall performance–efficiency trade-off, outperforming baselines while substantially reducing communication cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Systems, LLM Communication, Latent Communication, Adaptive Routing, Efficient Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_U5soOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper finds that entropy, maximum probability, and hidden-state norm carry predictive information about downstream correctness, suggesting their utility for adaptive routing among TEXT, LATENT, and SKIP. Therefore, this paper proposes ReLay, a two-level routing framework for effective multi-agent collaboration. Experiments on reasoning and long-horizon decision-making tasks show that ReLay achieves an optimal balance between overall performance and efficiency, outperforming various baselines while significantly reducing communication costs.

Critical strengths
  • Segment-level representation selection addresses a meaningful communication problem.
  • Experiments cover multiple tasks, backbones, and coordination structures, with useful component ablations.
  • The paper examines latent information transfer and acknowledges its bandwidth and fidelity limitations.
Critical weaknesses
  • The paper states that the results are derived from the average of three complete evaluation runs; this is highly unrigorous. For a dataset like AIME2026, the variance is significant, and it is standard practice within the community to use k=32k=32 or k=64k=64 for evaluation.
  • The main novelty appears to lie in receiver-utility-based segment selection among TEXT, LATENT, and SKIP. Please clarify the advances over existing hybrid communication and confidence-based switching, like HyLaT and ThinkRouter.
  • SCUR’s supervision depends on a background mask, but this mask is absent from its inputs. How reliable are these utilities when multiple segments are jointly routed?
Questions for authors

None

5 comments

We sincerely thank Reviewer for the constructive feedback. We address comments below

W1: Evaluation Reliability Our initial evaluation was 3 seeds due to computational constraints. We conducted additional 5-seed evaluations of competitive baselines on both benchmarks

AIME26 — Qwen3.5-9B
MethodAccuracy (%)Mean ± Std. (%)95% CI (%)Tokens ± Std. (k)
CoT[76.67, 80.00, 80.00, 76.67, 80.00]78.67 ± 1.83[76.40, 80.93]32.4 ± 0.7
Self-Refine[83.33, 86.67, 80.00, 83.33, 86.67]84.00 ± 2.79[80.54, 87.46]60.0 ± 1.5
TextMAS_GV[80.00, 83.33, 76.67, 83.33, 76.67]80.00 ± 3.33[75.86, 84.14]44.2 ± 1.2
TextMAS_OS[90.00, 86.67, 86.67, 90.00, 86.67]88.00 ± 1.83[85.73, 90.27]206.2 ± 2.8
SDE[80.00, 80.00, 76.67, 83.33, 76.67]79.33 ± 2.79[75.87, 82.80]46.2 ± 3.2
CIPHER[76.67, 83.33, 80.00, 73.33, 73.33]77.33 ± 4.35[71.94, 82.73]196.2 ± 1.3
ReLay_OS[93.33, 93.33, 90.00, 86.67, 93.33]91.33 ± 2.98[87.63, 95.04]161.4 ± 5.6
HMMT Feb — Qwen3.5-9B
MethodAccuracy (%)Mean ± Std. (%)95% CI (%)Tokens ± Std. (k)
CoT[57.58, 66.67, 66.67, 63.64, 57.58]62.42 ± 4.60[56.72, 68.13]42.4 ± 2.8
Self-Refine[63.64, 69.70, 66.67, 66.67, 57.58]64.85 ± 4.60[59.14, 70.55]81.3 ± 7.2
TextMAS_GV[66.67, 60.61, 63.64, 60.61, 66.67]63.64 ± 3.03[59.87, 67.40]64.7 ± 2.1
TextMAS_OS[69.70, 72.73, 75.76, 72.73, 75.76]73.33 ± 2.54[70.19, 76.48]296.7 ± 5.5
SDE[51.52, 54.55, 60.61, 57.58, 51.52]55.15 ± 3.95[50.25, 60.06]47.3 ± 3.7
CIPHER[63.64, 54.55, 63.64, 66.67, 66.67]63.03 ± 4.98[56.85, 69.21]260.1 ± 13.5
ReLay_OS[69.70, 72.73, 72.73, 69.70, 75.76]72.12 ± 2.54[68.97, 75.27]200.8 ± 14.2

The results show consistent performance across seeds. ReLay outperforming the strongest baseline on AIME26 and remaining competitive on HMMT, provide further evidence of evaluation reliability.


W2: Advances over Prior Work We appreciate these insightful works. Unlike prior hybrid communication protocols and token-level reasoning switches, ReLay learns to selectively transmit reasoning segments based on their predicted downstream utility to the receiver. We will add comparisons in Appendix F. The key distinctions are summarized below.

DimensionHyLaT (EMNLP 2026)InterLat (ACL 2026)ThinkRouterSeLaR (ACL 2026)ReLay (Ours)
SettingMulti-agentMulti-agentSingle-agentSingle-agentMulti-agent
GranularityReasoning traceReasoning traceToken-levelToken-levelSegment-level
Decision spaceTEXT + LATENTLATENTTEXT / LATENTTEXT / LATENTTEXT / LATENT / SKIP
Routing criterionInformation typeFixed latent channelToken confidenceToken entropyPredicted receiver utility
TrainingTwo-stage co-trainingReceiver adaptation + compression trainingTraining-freeTraining-freeCounterfactual receiver supervision
Downstream feedbackTask-trained protocolTask-trained latent processingNoneNoneReceiver outcomes
Efficiency mechanismHybrid latent–textLatent compressionConfidence-based switchingLatent activationUtility-Guided Adaptive Segment Routing

W3: SCUR under Joint Routing For efficiency, we adopt streaming processing, routing each segment upon generation and retaining only its selected transmission form. Since future routing decisions are unavailable, SCUR estimates each segment's weighted-average downstream utility across sampled contexts (Eq. 7). Finer-grained joint-context modeling remains a promising direction.

Experiment. We evaluate joint routing on 298 MATH/GPQA problems. SCUR jointly transmits the top-kk segments ranked by Eq. 4, while the baseline randomly selects kk segments (averaged over two draws). Both use matched token budgets and either TEXT or LATENT transmission. We use 5-fold problem-level cross-fitting to obtain out-of-fold scores.

kkTEXT SCURTEXT RandomGain (pp)LATENT SCURLATENT RandomGain (pp)
188.369.6+18.686.268.1+18.1
293.680.0+13.691.978.0+13.9
394.685.5+9.193.384.0+9.3
595.490.5+4.995.189.0+6.0

SCUR outperforms random selection by 9–19 pp for k≤3k\leq3, with all improvements statistically significant (95% bootstrap CIs). The advantage persists at k=5k=5, beyond the training context sizes. Moreover, average segment utility predicts joint-message success (within-problem AUC 0.723 ± 0.009 for TEXT and 0.697 ± 0.002 for all-LATENT, mean ± std over 3 seeds). We will add these results to the Appendix.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with less than 4 days remaining, we wanted to ensure that we have addressed all your concerns satisfactorily.

If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Reviewer U5so, Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. Should there be any remaining questions requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.

We sincerely look forward to your feedback.

Reviewer_4KMcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ReLay, a two-level routing framework for LLM multi-agent systems. At the communication level, its Signal-Conditioned Utility Router (SCUR) divides reasoning traces into segments and routes each as text, probability-weighted token embeddings, or SKIP. SCUR uses uncertainty signals and content features and is trained using counterfactual receiver correctness under each channel. At the coordination level, the Adaptive Representation Orchestrator (ARO) selects the next compatible agent or terminates execution using either an LLM or a lightweight learned policy.

The method is evaluated on mathematical and scientific reasoning benchmarks and on long-horizon ALFWorld and WebShop tasks. The authors report improved accuracy–token trade-offs over text, latent, hybrid, and adaptive-agent baselines. Ablations support segment-level routing, complementary router inputs, and the separate contributions of SCUR and ARO; additional analyses examine routing granularity, channel usage, and internal-signal behavior.

Critical strengths

1/ Fine-grained, utility-based communication routing. ReLay goes beyond choosing one representation for an entire message by routing individual reasoning segments through text, latent, or skip channels. This is an insightful conceptual advance, as reasoning traces contain heterogeneous information, and the strong whole-trace-routing ablation supports the importance of this granularity.

2/ Downstream-grounded supervision with broad empirical support. SCUR learns from counterfactual receiver outcomes rather than heuristic uncertainty labels. Across reasoning and long-horizon tasks, the reported gains and component ablations consistently support improved performance–efficiency trade-offs, making the contribution consequential beyond a purely architectural proposal.

Critical weaknesses

1/ The efficiency claim is based on incomplete cost accounting. Reported savings count discrete tokens (in Section C.3; Tables 1 and 4) but omit the bandwidth, memory, and computational costs of dense latent sequences. As the authors acknowledge in Section D.3 and Table 14, each 2,560-dimensional bfloat16 latent position requires roughly 40.96 kbits, versus tens of bits for a token ID, while FLOPs and memory traffic are not measured. Without end-to-end latency, compute, or communication measurements, the claimed performance–efficiency advantage is not established at the system level.

2/ The comparative evidence is not sufficiently robust. Section C.2 acknowledges that several latent-communication baselines were adapted outside their intended operating conditions and may therefore underperform. Important gains are also reported on only 30 AIME2026 and 33 HMMT problems in Table 11. Although Section C.3 mentions averaging over three runs, the main results in Tables 1–4 provide no variance. Consequently, modest improvements over the strongest baselines may reflect evaluation variance or implementation choices rather than ReLay itself.

Questions for authors

1/ In Tables 1 and 4, are latent positions entirely excluded from the token metric despite incurring receiver-side computation? Can the authors provide end-to-end latency, FLOPs, memory, or transmitted-byte measurements under a common deployment setup? This would materially determine whether ReLay is system-efficient rather than only discrete-token-efficient.

2/ Can the authors provide per-run results, confidence intervals, or significance tests for the main comparisons—especially AIME2026 and HMMT—and specify how each latent baseline was changed from its recommended configuration? This is necessary to assess whether the reported improvements are robust to sampling variance and baseline implementation choices.

Suggestions for future work: None; these questions concern the validity and interpretation of the paper’s current claims.

3 comments

We thank the reviewer for recognizing our novelty and empirical support

Q1&W1: System-Level Efficiency We agree that discrete-token counts alone do not fully capture latent communication costs. ReLay's efficiency gains primarily stem from segment-level selective routing, rather than latent communication

We further evaluate system-level efficiency using mean end-to-end latency. Across AIME and HMMT, ReLay reduces latency under Generator–Verifier and Orchestrator–Subagent compared with their corresponding TextMAS baselines

MethodAIME26 (s/task)HMMT Feb(s/task)
CoT12481862
Self-Refine15082207
SDE17891875
CIPHER16222100
TextMAS_GV18122421
ReLay_GV735842
TextMAS_OS45245729
ReLay_OS18102689

For further discussion of latent costs, please refer to our response to ReviewerYgmF (W1).


Q2&W2: Robustness and Baseline Fairness We thank the reviewer for raising concerns about reliability and baseline comparability 1. Reliability. We conducted additional five-seed evaluations of competitive relevant baselines on both benchmarks

AIME26 — Qwen3.5-9B
MethodAccuracy (%)Mean ± Std. (%)95% CI (%)Tokens ± Std. (k)
CoT[76.67, 80.00, 80.00, 76.67, 80.00]78.67 ± 1.83[76.40, 80.93]32.4 ± 0.7
Self-Refine[83.33, 86.67, 80.00, 83.33, 86.67]84.00 ± 2.79[80.54, 87.46]60.0 ± 1.5
TextMAS_GV[80.00, 83.33, 76.67, 83.33, 76.67]80.00 ± 3.33[75.86, 84.14]44.2 ± 1.2
TextMAS_OS[90.00, 86.67, 86.67, 90.00, 86.67]88.00 ± 1.83[85.73, 90.27]206.2 ± 2.8
SDE[80.00, 80.00, 76.67, 83.33, 76.67]79.33 ± 2.79[75.87, 82.80]46.2 ± 3.2
CIPHER[76.67, 83.33, 80.00, 73.33, 73.33]77.33 ± 4.35[71.94, 82.73]196.2 ± 1.3
ReLay_OS[93.33, 93.33, 90.00, 86.67, 93.33]91.33 ± 2.98[87.63, 95.04]161.4 ± 5.6
HMMT Feb — Qwen3.5-9B
MethodAccuracy (%)Mean ± Std. (%)95% CI (%)Tokens ± Std. (k)
CoT[57.58, 66.67, 66.67, 63.64, 57.58]62.42 ± 4.60[56.72, 68.13]42.4 ± 2.8
Self-Refine[63.64, 69.70, 66.67, 66.67, 57.58]64.85 ± 4.60[59.14, 70.55]81.3 ± 7.2
TextMAS_GV[66.67, 60.61, 63.64, 60.61, 66.67]63.64 ± 3.03[59.87, 67.40]64.7 ± 2.1
TextMAS_OS[69.70, 72.73, 75.76, 72.73, 75.76]73.33 ± 2.54[70.19, 76.48]296.7 ± 5.5
SDE[51.52, 54.55, 60.61, 57.58, 51.52]55.15 ± 3.95[50.25, 60.06]47.3 ± 3.7
CIPHER[63.64, 54.55, 63.64, 66.67, 66.67]63.03 ± 4.98[56.85, 69.21]260.1 ± 13.5
ReLay_OS[69.70, 72.73, 72.73, 69.70, 75.76]72.12 ± 2.54[68.97, 75.27]200.8 ± 14.2

Across seeds, ReLay outperforms strongest baseline on AIME2026, achieving statistical significance at the 90% one-sided level (paired t-test, p = 0.071). On HMMT, ReLay remains competitive

2. Baseline Adaptations

  • SDE: We stream hidden-state extraction, retaining only the designated edit layer's token hidden state and discard the rest. This enables generation up to 100K tokens without changing SDE's state-delta computation or editing mechanism
    • Rationale: The original SDE implementation retains hidden states from all layers at every decoding step, causing OOM beyond ~30K tokens. This is insufficient for AIME26 and HMMT. The adaptation affects only memory management, ensuring a faithful comparison
  • CIPHER: We retain probability-weighted embedding representation but construct it from standard token-space decoding (T = 0.7) using renormalized top-3 token probabilities. We select 3,000 tail-biased vectors per generator (6,000 total) for a single generator→judge round
    • Rationale: Full-length communication (~90K soft tokens) exceeds available GPU memory, making original multi-round protocol infeasible. Our adaptation preserves embedding communication while enabling evaluation under matched generation budgets and judge prompts
  • LatentMAS: We retain the original implementation, updating only the KV-cache handling to support the newer transformers API required by corresponding backbone model
    • Rationale: LatentMAS already retains only the last hidden state from the final layer, avoiding the memory overhead encountered by SDE
  • HyLaT: We applied KV-cache, SDPA attention, and chat-template compatibility fixes for corresponding model, using bf16 instead of fp32 to reduce memory usage during long-horizon generation
    • Note: These changes address API compatibility and memory constraints while preserving HyLaT's communication architecture
  • InterLat: On MATH, we compress the sender's hidden states into 8 latent slots using a trained adapter and actor LoRA, then inject them into the receiver's embeddings
    • Note: Computational constraints limited the extent of training, potentially affecting the latent channel's performance We will add details in Appendix C.2

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with less than 4 days remaining, we wanted to ensure that we have addressed all your concerns satisfactorily.

If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Authors,

Thank you for your efforts! My concerns and questions had been fully addressed. I will maintain my score.

Reviewer_YgmFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how agents in LLM-based multi-agent systems should communicate intermediate reasoning. Specifically, the authors argue that all-text communication is verbose, all-latent communication may lose precise information, and full reasoning traces contain redundant or misleading content. To address this, they propose ReLay, which splits each reasoning trace into fixed-size segments and uses a router trained on counterfactual receiver outcomes to send each segment as text, latent soft embeddings, or skip it, together with an orchestrator that selects the next agent. Experiments on math, science QA, ALFWorld, and WebShop show that ReLay improves accuracy over text and latent MAS baselines while using fewer tokens.

Critical strengths
  1. The problem is interesting. Choosing the communication form at the segment level, rather than fixing it for the whole message, is a reasonable and under-explored direction.
  2. The idea is straightforward and easy to understand.
  3. The evaluation is broad. It covers foundational reasoning and long-horizon decision-making tasks, multiple backbones, two MAS structures, and many text and latent communication baselines.
Critical weaknesses
  1. The role of latent communication is unclear, and the gains mainly come from skipping content. Specifically, according to Table 12, LATENT accounts for only 7.8–13.2% of segments, versus 41.8–44.4% for SKIP, and the authors state that the token reduction is driven primarily by omission (Appendix D.3). Moreover, based on Table 7, randomly retaining only 50% of the message already preserves 98–99% of performance, the token metric excludes the cost of latent embeddings, and latent messages can only be sent to agents sharing the same backbone.
  2. The results are not solid enough to claim the contributions. For example, ReLay underperforms the best baseline on HMMT for both backbones (−4.3% and −2.8%, Table 1). And the test sets are small, HMMT and AIME2026 have only 33 and 30 problems, so one problem changes accuracy by about 3%. I am not sure how about the variance.
  3. Some negative results are unexplained: whole-trace routing drops AIME2026 accuracy to 26.7% (Table 3), far below even single-agent CoT (76.7%, Table 1). This make me concerning the effectiveness of the proposed method.
Questions for authors

Please see my weaknesses.

5 comments

We appreciate the recognition of segment-level communication form and the clarity of our approach. We address comments below W1: Latent and Skip Gains LATENT and SKIP are complementary components of ReLay with distinct roles: LATENT enhances effectiveness by preserving richer reasoning information. SKIP improves efficiency by omitting low-value content. Their combination reflects an trade-off (Table 14)

LATENT Gains. Channel frequency (Table 12) reflects usage rather than contribution. Ablation results (Table 3, w/o embedding) show that removing LATENT reduces accuracy by 3.1–4.5 pp across benchmarks. Figure 4 and Appendix I further demonstrate cases where LATENT succeeds but TEXT over identical retained segments fails. These findings highlight LATENT's contribution to effectiveness

SKIP Gains. Table 7 is an exploratory experiment under a simplified protocol, illustrating the existence of communication redundancy. It does not establish that random skipping is sufficient. Effective SKIP requires utility-guided selection to preserve critical information. Ablations (Table 3, w/o skip) show that SKIP reduces token consumption by 38.7–50.3%, while our method consistently outperforms Random routing in accuracy (Table 3). We will clarify Table 7 in the revision

Table 7Main experiments
Evaluation SetCorrectly answered problemsAll problems
MetricAccuracy relative to full-message baselineEnd-to-end task accuracy
Topology1 generator → 1 receiver, no routingMethod-specific topology
Receiver taskRead-off: thinking off, ≤256 tokensReasoning: thinking on, ≤100k tokens
StrategyRandom TEXT/SKIP~Bernoulli(p)Learned TEXT/SKIP/LATENT per segment

Token and Compute Cost. Under identical BF16 serving, we compare sequential LLM calls, input/output tokens, transmitted bytes, and estimated FLOPs (2×parameters×tokens2\times\text{parameters}\times\text{tokens}) per episode

BenchmarkMethodSuc/ScoreCallsTokens (In/Out, k)Wire (KB)PFLOPs
ALFWorld-SeenZero-shot26.737.9266.1/1.39072.14
CoT31.731.9246.7/2.48281.99
Self-Refine30.069.5439.4/3.914503.55
TextMAS29.2111.2802.0/3.126346.44
AFlow28.138.2267.5/1.39132.15
ReLay-LLM45.332.2183.2/1.339151.48
ReLay-Policy43.330.5198.0/1.240231.59
ALFWorld-UnseenZero-shot15.037.8237.8/1.38341.91
CoT29.230.4199.4/2.76901.62
Self-Refine13.360.0293.8/3.49722.38
TextMAS30.0104.9645.1/3.021215.19
AFlow25.037.1231.5/1.38131.86
ReLay-LLM33.127.0156.3/1.238821.26
ReLay-Policy36.724.3142.6/1.133201.15
WebShopZero-shot31.7/46.48.343.2/1.21360.36
CoT30.8/44.818.175.2/2.42240.62
Self-Refine28.3/49.417.982.6/1.22740.67
TextMAS32.9/59.222.8118.6/2.13950.97
AFlow31.7/48.610.959.3/1.41990.49
ReLay-LLM33.2/56.120.045.0/1.231450.37
ReLay-Policy37.2/57.510.349.5/1.229770.41

Despite higher latent transmission volume, ReLay generally incurs lower estimated FLOPs. Combined with the latency reductions reported to Reviewer 4KMC (Q1&W1), these results demonstrate ReLay's competitive system-level efficiency despite increased communication volume

Heterogeneous Latent Communication. Latent compression and cross-model alignment (e.g., InterLat) are promising but often require substantial training. ReLay enables segment-level routing without adapter training, supporting scalable communication over long reasoning trajectories on challenging benchmarks

W2: HMMT and AIME26 Results We thank the reviewer for highlighting the reliability concerns on small benchmarks. Our original evaluation used 3 seeds due to GPU constraints. We conducted additional five-seed evaluations of competitive baselines on both benchmarks

AIME26 — Qwen3.5-9B

MethodMean ± Std. (%)95% CI (%)
CoT78.67 ± 1.83[76.40, 80.93]
Self-Refine84.00 ± 2.79[80.54, 87.46]
TextMAS_GV80.00 ± 3.33[75.86, 84.14]
TextMAS_OS88.00 ± 1.83[85.73, 90.27]
SDE79.33 ± 2.79[75.87, 82.80]
CIPHER77.33 ± 4.35[71.94, 82.73]
ReLay_OS91.33 ± 2.98[87.63, 95.04]

HMMT Feb — Qwen3.5-9B

MethodMean ± Std. (%)95% CI (%)
CoT62.42 ± 4.60[56.72, 68.13]
Self-Refine64.85 ± 4.60[59.14, 70.55]
TextMAS_GV63.64 ± 3.03[59.87, 67.40]
TextMAS_OS73.33 ± 2.54[70.19, 76.48]
SDE55.15 ± 3.95[50.25, 60.06]
CIPHER63.03 ± 4.98[56.85, 69.21]
ReLay_OS72.12 ± 2.54[68.97, 75.27]

Across five seeds, ReLay achieves the highest mean accuracy on AIME26 and remains competitive on HMMT, while reducing token usage by 22–32% relative to TextMAS_OS. Details table in our response to Reviewer4KMc (Q2&W2)


W3: Whole-Trace Routing Degradation. Whole-trace routing is an extreme ablation that applies a single adaptive decision to the entire reasoning trace. The degradation stems from two factors: truncation caused by computational constraints and reduced routing flexibility at coarser granularity. Truncation effect. Due to computational constraints, our original approximation used a 2,000-token window. While shorter MATH and GPQA-D traces are less affected, AIME2026 traces (5k–60k tokens) suffer substantial truncation. Expanding the window improves accuracy on AIME26 and GPQA-D

Variant | Acc / TokMATHAIME26GPQA-D
ReLay_Core77.5 / 14.986.7 / 103.382.5 / 17.3
Whole-trace routing (Table 3)75.0 / 10.426.7 / 201.970.0 / 10.3
Whole-trace routing (5k window)75.0 / 22.340.0 / 166.773.0 / 36.4

Granularity effect. Fig. 5 shows that larger routing windows reduce routing flexibility and degrade accuracy. Both factors explain AIME26 degradation, while the revised results reaffirm fine-grained routing benefits.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end, with less than 4 days remaining, we wanted to ensure that we have addressed all your concerns satisfactorily.

If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.