acceptodds
Under review as a conference paper at ICLR 2027

TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision

Abstract

Agents with smaller language-model backbones are less expensive but can drift into persistent failure modes, whereas those with larger backbones are generally more reliable but more costly. This reliability-cost trade-off motivates routing methods that decide when to invoke an agent with a larger backbone: before execution, after a fixed trajectory prefix, or locally at individual steps. Our method, TACIT-SWITCH, learns permanent handoff policies from accumulated trajectory evidence and Teacher-Annotated Censored Intervention Times (TACIT). It represents each annotation as an interval-censored observation on a cumulative-risk scale. The resulting mixture-cure threshold model estimates the probability that the paired Strong rollout succeeds and, conditional on success, the handoff threshold; no teacher is required at deployment. In a mechanism-based multi-step simulation, TACIT-SWITCH improves success by 7.4–11.1 percentage points over task-level, step-level, and fixed-prefix routing baselines at comparable cost. Within that controlled simulation, ablations show that task features and cumulative trajectory risk provide complementary information. With operating points selected on development data, TACIT-SWITCH achieves the highest held-out success among learned policies on both ALFWorld (48.5% with 4B Cheap; 45.5% with 9B Cheap) and DABench (73.1%).

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, model routing, adaptive escalation, censored supervision, cost-aware inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Q9eFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TACIT-Switch, a method for deciding when an LLM agent should permanently switch from a cheaper model to a stronger model during a multi-step task.

The method uses paired Cheap and Strong rollouts and teacher-annotated intervention times as training supervision. It combines a model of Strong's success probability with a learned threshold over cumulative trajectory risk, using a mixture-cure survival model and interval censoring.

Experiments on simulated tasks, ALFWorld, and DABench show improved success-cost trade-offs compared with task-level, step-level, and fixed-prefix routing baselines.

Critical strengths
  1. Useful and practical problem. Deciding when to switch from a cheaper to a stronger model is an important problem for multi-step agents. The idea of learning an adaptive, permanent handoff policy is interesting, especially since it avoids expensive model-based decisions during deployment.

  2. Promising experimental results. The method shows meaningful improvements over existing routing baselines in both simulations and real-agent benchmarks. The larger ALFWorld results are particularly encouraging, with TACIT-Switch achieving 58.0% success compared to 42.0% for the SWE-Router-style baseline at lower reported cost. The paper also includes reasonable experimental controls, ablations, and a larger-scale evaluation.

Critical weaknesses
  1. It is unclear how much of the improvement comes from the proposed statistical model. The paper shows that cumulative trajectory risk and task features improve routing, but does not establish why the mixture-cure survival model is necessary. A simpler classifier trained with the same features and teacher annotations might perform similarly. The baselines also differ in important ways. TACIT-Switch uses richer trajectory information than some baselines and continues from the current state, while SWE-Router restarts Strong. These differences make it difficult to attribute the improvements to the proposed learning method rather than the additional information or handoff strategy.

  2. The supervision does not directly establish whether switching is beneficial. The model uses Strong's success from the initial state and teacher-estimated intervention times to learn switching decisions. However, Strong succeeding independently does not necessarily mean it can recover from Cheap's intermediate state. Similarly, Strong might fail independently but succeed after Cheap has made useful progress. The teacher annotations are not validated through actual handoffs on real-agent tasks. While the simulations provide some evidence, it remains unclear how accurately the learned score captures the benefit of switching at a particular point in a real trajectory.

Overall, I find the problem useful and the results promising. However, the paper does not yet convincingly establish that its particular statistical formulation is better than simpler adaptive switching approaches, or that the supervision reliably identifies useful intervention points.

Questions for authors
  1. Why is the survival-analysis formulation necessary? Can the authors compare against a simple switching classifier or learned cumulative-risk threshold using the same features, training data, and teacher annotations? This would help establish whether the statistical formulation provides additional value.

  2. How reliable are the teacher-annotated intervention times? Have the authors tested whether Strong actually succeeds when resumed from these checkpoints on real-agent tasks? It would be useful to compare actual success when switching at the annotated checkpoint versus continuing Cheap or switching at nearby checkpoints.

  3. Are the baseline comparisons isolating the right factors? In particular, could the authors compare against a fixed-prefix policy that continues from the current state instead of restarting Strong, and a step-level policy using the same trajectory information? This would help separate the benefits of adaptive timing, richer features, and preserving progress.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Q9eF (1/2): handoff timing and teacher supervision (W1, W2)

W1. What accounts for the gains?

Does the gain persist when both policies keep Cheap’s progress? Yes, relative to a fixed handoff on the original ALFWorld 9B-to-27B evaluation. Under the Dev budget, the best fixed delay was 16 Cheap actions. Strong continued from the current state under both policies. TACIT solved 159/274 tasks versus 137/274 for fixed handoff at similar mean weighted call cost (182.79 vs. 181.09). That is 22 additional successes, or 8.03 percentage points (paired 95% bootstrap interval: 3.28 to 12.77). Preserving progress alone therefore does not explain this difference.

Where should a handoff label come from? A Cheap trajectory shows what happened while Cheap remained in control, not what would have happened had Strong taken over. A direct way to estimate the best handoff time under a given budget is to save the state after each Cheap action, run Strong several times from each state, and compare success and cost across handoff times and against continuing with Cheap. Repeating these counterfactual rollouts at every step of every training trajectory is expensive in both time and compute.

What is new in our supervision? Using an LLM to review a past trajectory is not itself new: for example, Zhang et al. (2025) use execution logs to identify the step responsible for a failure. Our offline teacher instead reviews existing paired Cheap and Strong runs to mark when a handoff might help. If several adjacent steps are plausible, asking for one exact step would turn the teacher’s uncertainty into an arbitrary label. We retain the whole window as an interval-censored observation. To our knowledge, this is the first use of teacher-marked windows as interval-censored supervision for learning permanent handoff between LLM agents.

Is mixture-cure required? No. We chose it because it handles censoring and separates Strong’s chance of success from the start from the timing of handoff conditional on that success. A practitioner could instead use a nonparametric model or another machine-learning method that learns from interval-censored labels, without changing how the trajectories are annotated.

We fitted a discrete-time logistic hazard model to the same training tasks and teacher windows. Its observed success rate was similar to that of the TACIT refit, at a higher mean cost (Q1). We therefore distinguish the evidence for choosing when to switch from evidence for any particular model of timing. Q3 separates the choice to continue or restart from the choice to hand off permanently or for just one action.

W2. Does supervision identify useful interventions?

Can Strong rescue a task at the teacher’s recommended step? We tested four handoff times on the same 34 ALFWorld tasks where Cheap failed, Strong succeeded from the start, and all four checkpoints were available. Strong rescued 20/34 tasks six steps before the recommendation, 23/34 three steps before, 21/34 at the recommendation, and 18/34 three steps after. Three steps early had the highest observed rescue rate on this subset. Q2 reports the corresponding costs.

Should we calibrate the fitted policy’s timing? At Reviewer F6yD’s suggestion, we compared TACIT’s actual handoff step with the teacher’s recommendation on 117 annotated ALFWorld test tasks. TACIT handed off 2.31 steps later on average. This suggests testing whether an earlier handoff can compensate for that lag. The 34-task comparison above supports exploring that direction, but it set the checkpoints directly rather than shifting TACIT’s decisions. In the revision, we will make timing calibration an explicit step: select the training-window offset and handoff threshold on Dev under a fixed cost budget, then freeze both before testing. On the full 300-task Dev set, this procedure selected no shift.

Initial success is neither a guarantee nor a prerequisite for rescue. Although Strong succeeded from the start on all 34 tasks, it failed on 13 when it took over at the recommendation. Conversely, on a separate set of 65 tasks where both models had failed from the start, Strong rescued 12 after 16 Cheap actions. We therefore use Strong’s initial outcome as a training signal, not as a label of whether an intermediate state is recoverable.

Why use teacher windows? Testing every candidate step on every training trajectory would require a separate Strong continuation from each checkpoint. The teacher instead marks a plausible window on existing paired runs. The actual handoffs above show that many recommended checkpoints were recoverable; they do not establish that the learned score measures the causal gain from switching.

AuthorsOpenReview ↗

Response to Reviewer Q9eF (2/2): timing models, checkpoint rescues, and baselines (Q1–Q3)

Q1. Is survival analysis necessary?

Our contribution is how we obtain and learn from the labels, not a particular survival model. An offline LLM teacher reviews paired Cheap and Strong trajectories and marks a range of plausible handoff steps. For example, it may be confident that a handoff is warranted around steps 6-8 without being able to justify step 7 over either neighbor. We fit a statistical model to the whole window as an interval-censored observation. This avoids the time and compute expense of generating counterfactual training labels by running Strong separately from every candidate checkpoint. To our knowledge, this is the first use of LLM-teacher handoff windows as interval-censored supervision for learning permanent handoff between LLM agents.

Can a different timing model learn from the same windows? Yes. We chose mixture-cure for interpretability. To test an alternative, we kept the component predicting Strong’s success from the initial state fixed and replaced the timing component with a discrete-time logistic hazard fitted to the full teacher intervals. Both models used the same 200 training tasks, 89 teacher windows and paired outcomes. We selected each handoff threshold on Dev300 under the same cost budget and froze it before testing.

ControllerSolved / 274Mean weighted call cost
TACIT mixture-cure refit148188.38
Interval-censored logistic hazard151199.56

On these 274 previously evaluated ALFWorld tasks, the logistic hazard had a similar observed success rate to the TACIT refit (55.1% vs. 54.0%), at a higher mean cost. As you suggest, the timing module can be replaced without changing how the teacher labels are collected or reannotating the trajectories.

Q2. Can Strong rescue teacher-marked states?

Yes. We ran Strong from four saved checkpoints on each Cheap trajectory. Of 75 ALFWorld tasks where Cheap failed and Strong succeeded from the start, 34 had all four checkpoints available. We compare handoff times on those same 34 tasks. Continuing with Cheap solved 0/34 by selection. Mean weighted call cost includes both the Cheap prefix and the Strong continuation.

Offset from teacher recommendationSolved / 34Mean cost
6 steps earlier20 (58.82%)202.83
3 steps earlier23 (67.65%)189.13
At recommendation21 (61.76%)191.32
3 steps later18 (52.94%)191.32

Strong rescued 21/34 tasks at the teacher’s recommendation. Handing off three steps earlier rescued two more at a slightly lower mean cost; moving six steps earlier offered no further benefit. On these trajectories, the teacher often identified a state from which Strong could recover, but not necessarily the best step to switch.

Q3. What do the baseline controls isolate?

Fixed prefix: restart or continue? We kept the SWE-style router’s four-action prefix and its learned scoring head. First we changed only what happened when the router called Strong: it resumed from Cheap’s state instead of restarting. We then allowed the continuation variant to reselect its gate on Dev. Results below are from the original ALFWorld 9B-to-27B scale-up evaluation.

PolicySolved / 274Mean cost
SWE-style restart, original gate115189.85
SWE-style continuation, same gate106181.37
SWE-style continuation, gate reselected on Dev147194.83
TACIT permanent handoff159182.79

Keeping progress alone did not improve the SWE-style router at its original gate. Retuning the gate made continuation more competitive, but TACIT still solved 12 more tasks at lower mean cost. The separate fixed-16 comparison in W1 also retains progress in both arms.

One-action deferral with trajectory information. We gave a step-level controller the same 12 task descriptors and five underlying trajectory diagnostics rather than just action uncertainty. At each step, it could delegate one action to Strong and then return control to Cheap. It solved 87/274 tasks at mean cost 143.37, charging every Cheap proposal and Strong call. This controller was cheaper but solved fewer tasks than TACIT. Because its Strong-action scoring procedure also differed, we do not attribute the entire gap to permanent handoff.

Reviewer_jWr3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cost-aware escalation from a cheaper to a stronger LLM during interactive agent tasks, using a permanent handoff from the current trajectory state. TACIT-Switch learns from paired full-episode outcomes and offline teacher annotations of intervention windows. It combines a model of initial-state Strong success with an interval-censored threshold distribution on learned cumulative trajectory risk, selecting a deployment threshold on development data. No teacher is required at inference. Contributions include the supervision scheme, the statistical handoff model, and a local stability analysis under annotation noise. Validation comprises model-recovery simulations, a separate mechanism-based trajectory simulation, and held-out ALFWorld and DABench experiments using parameter-weighted call costs. The paper reports improved success–cost trade-offs against task-level, step-local, and fixed-prefix routing baselines.

Critical strengths
  1. A practical permanent-handoff design with an economical learning and deployment procedure. The paper studies transferring the remainder of an episode to Strong from the current state after accumulating trajectory evidence. This control structure differs from the uncertainty-triggered local deferral of ReDAct and from fixed-prefix decisions to restart. Sections 2–3 turn paired full-episode outcomes and coarse teacher windows into a compact incidence–threshold model, avoiding exhaustive checkpoint rollouts and any deployment-time teacher. The treatment of successful Cheap episodes as right-censored records also uses information beyond failure annotations. The authors appropriately identify the likelihood as a working model when annotation coarsening is nonignorable, and Theorem 1 makes a limited local parameter/score-stability claim. These are useful practical design choices, with the statistical interpretation stated cautiously.

  2. The strongest real-agent result is a meaningful held-out improvement. Table 2 reports ALFWorld 9B–27B success of 61/134 (45.5%) at cost 165.11, versus 48/134 (35.8%) at 205.95 for SWE-Router. Appendix B.2 gives a paired 95% bootstrap interval of [2.2, 17.9] percentage points for this success difference. Thus the result improves both reported objectives, after development-only operating-point selection. With 4B Cheap, the method reaches 48.5% success versus pure Strong's 50.7%, at cost 191.10 versus 230.51. The larger ALFWorld study is directionally consistent for both model pairs. The 100-replicate mechanism simulation further separates policy utility from recovery under a correctly specified likelihood. DABench's 49/67 versus 45/67 advantage remains suggestive because its interval includes zero; the paper explicitly acknowledges this. These qualifications strengthen the credibility of the empirical contribution.

Critical weaknesses
  1. The experiments do not establish the incremental value of the proposed learned handoff model. Appendix B.4 gives the ALFWorld task router five binary features, whereas TACIT-Switch receives a 12-dimensional task representation and five trajectory diagnostics. Step Deferral receives only selected-action NLL. TACIT-Switch also uses paired Strong outcomes and teacher timing supervision that these baselines do not use. The comparisons therefore change features, supervision, and intervention structure together. They lack a simple permanent-handoff classifier or cumulative-risk threshold using matched information, and a development-tuned fixed-delay handoff that continues from the current state. These controls are particularly consequential in Table 3, where 95.6% and 92.7% of tasks invoke Strong. The present results leave open whether economical delayed takeover accounts for much of the improvement; they do not demonstrate that it does. Component ablations are confined to simulation. This is the principal reason for my rejection recommendation: the observed system-level gains do not adequately establish the additional contribution of the survival formulation and adaptive timing over simpler permanent-handoff policies.

  2. The supervision's connection to actual rescue remains empirically under-validated. As Section 2.2 correctly states, B is the outcome of one Strong rollout from the initial state, rather than the probability that Strong can recover a Cheap-generated state. A failed initial Strong rollout need not imply an unhelpful intervention, and a successful one does not validate a teacher's proposed recovery window. Appendix B.2's teacher inspects both complete trajectories, but the real-agent study does not report checkpoint continuations testing whether the annotated windows correspond to recoverable states. Appendix B.1 provides such a useful check only in simulation, where intervention at a marked checkpoint succeeds in 85.6% of relevant cases. The working-likelihood qualification avoids a mathematical overclaim, but neither it nor the local stability theorem supplies the missing empirical validation. This limits confidence in the proposed supervision as the reason for the observed gains and in the interpretation of the incidence cap.

Questions for authors
  1. Validation of current claims: How does the method compare with a simple one-way classifier or monotone cumulative-risk threshold using the same task features, diagnostics, and available paired/teacher supervision? Please also compare a development-tuned fixed-delay continuation from the current state. These controls would determine whether the claimed learned timing contribution persists beyond the permanent-handoff structure itself.

  2. Validation of current claims: What are Strong rescue success and incremental benefit over continued Cheap at checkpoints before, within, and after teacher windows, accounting for intervention cost? A representative real-agent audit, including some B = 0 episodes, would directly assess the current use of paired initial-state outcomes and teacher windows as intervention supervision. This would materially affect my confidence in the model's central learning premise.

  3. Clarification: In DABench, are discarded Cheap proposals charged in the Cheap call count in Equation (20), included when enforcing the 12-call limit, and included in the denominator of Strong share? Appendix B.2 describes generating and discarding such proposals, whereas the accounting description does not explicitly resolve these cases. The answer matters for interpreting the reported cost advantage; I am not assuming the calls were omitted.

3 comments
AuthorsOpenReview ↗

Response to Reviewer jWr3 (1/3): matched baselines and the handoff framework (W1)

W1. Matching Task Router’s initial features. You are right that the original Task Router had five task features while TACIT had twelve. We corrected this mismatch by refitting Task Router with the same twelve inputs. It still chooses Cheap or Strong before the first action. We kept its training labels and Dev300 threshold-selection rule unchanged and froze the threshold before evaluating on the original Test274. Each cell gives tasks solved (success rate) and mean weighted call cost.

Cheap / Strong12-feature Task RouterTACIT-Switch
4B / 27B106/274 (38.69%); 187.30151/274 (55.11%); 176.87
9B / 27B127/274 (46.35%); 191.88159/274 (58.03%); 182.79

Matching the inputs improved the 9B router (106 to 127 successes) but not the 4B router (113 to 106). TACIT still solved 45 more tasks with 4B Cheap and 32 more with 9B Cheap, at lower mean cost in both settings.

Why the other baselines use different inputs. Step Deferral uses the uncertainty of Cheap’s proposed action (its negative log-likelihood) to delegate just that action. Our SWE-Router-style baseline instead scores the task and a fixed prefix of Cheap’s trajectory to decide whether to continue with Cheap or restart with Strong. We kept these decision rules rather than adding TACIT’s hand-designed diagnostics to them.

Adaptive versus fixed handoff. We first tested your suggestion directly: could handing off after a fixed number of Cheap actions match TACIT? We chose the fixed delay on Dev (16 actions) and let Strong continue from Cheap’s current state under both policies. On the original ALFWorld 9B–27B test, TACIT solved 159/274 tasks versus 137/274 for fixed handoff at similar mean weighted call cost (182.79 vs. 181.09). Adaptive timing thus gained 8.03 percentage points in success rate in this comparison (paired 95% bootstrap interval [3.28, 12.77]).

Our contribution is the supervision framework. An offline LLM teacher can mark a window of plausible handoff steps instead of a specific step. We then fit a statistical model that treats each window as an interval-censored observation of handoff timing. Asking for one exact step often demands more precision than the teacher can provide: it may confidently identify steps 6–8 but have no basis for choosing step 7 over either neighbor. By learning from the full windows and paired Cheap and Strong outcomes, we avoid running Strong from every training checkpoint. To our knowledge, no prior work has used LLM-teacher annotations as interval-censored data to learn when one LLM agent should permanently hand control to another.

Why a mixture-cure model? The next component in our framework is the timing model, which learns when to hand off from the interval-censored data produced by the teacher. We chose an interpretable mixture-cure threshold model from survival analysis for this component: its likelihood uses the probability assigned to the whole interval, and its two components describe Strong’s chance of success from the initial state and, conditional on that success, the cumulative-risk threshold for handoff. To test an alternative, we kept the Strong-success component fixed and replaced only the timing component with a discrete-time logistic hazard. Its coefficients relate step-level signals to the conditional odds of reaching the handoff point. We fitted both models on the same 200 training tasks with the same 89 teacher intervals and chose their thresholds on Dev. The hazard solved 151/274 test tasks at mean weighted call cost 199.56; the TACIT refit solved 148/274 at 188.38.

The mixture-cure model is a replaceable component of our framework: it learns from the teacher’s interval-censored labels but does not define how those labels are collected. For example, a practitioner who needs a more flexible fit could use a nonparametric timing model trained on the same intervals. We did not test that option; neither of the two parametric implementations we tested needs the teacher at deployment.

AuthorsOpenReview ↗

Response to Reviewer jWr3 (2/3): teacher supervision and timing-model controls (W2, Q1)

W2. Why use teacher windows instead of testing every training checkpoint? You are right: B records whether Strong succeeded from the start; it cannot tell us when Strong should take over. To label the handoff point from actual outcomes, we would need to compare what happens when Cheap continues with what happens when Strong takes over at different steps on the same trajectory. Doing this for every training trajectory would require a separate Strong rollout from each candidate checkpoint. This cost motivated our use of an offline teacher to annotate existing paired rollouts. The teacher may be confident that a handoff is warranted within a few steps yet unable to distinguish one step from its neighbors. We therefore ask it to mark a window it is confident contains a suitable handoff point. We use the whole window as interval-censored supervision, retaining the timing information the teacher can provide without forcing a potentially incorrect point label.

Does the timing information in the teacher labels matter? We introduced a systematic timing error by shifting the teacher’s training windows three steps later. We then refit TACIT on the same tasks, keeping the paired outcomes unchanged. If a shifted window was unusable, we kept the episode outcome but omitted its timing label. At the threshold selected on the original windows (0.20), the original model solved 189/300 ALFWorld development tasks (63.0%); the shifted model solved 168/300 (56.0%). Mean weighted call cost rose from 180.05 to 184.06. The drop in success after this perturbation suggests that the original teacher windows carry useful information about when to hand off.

How should Strong-only performance be interpreted? We apologize for the ambiguity in our use of “cap.” We will describe Strong-only performance as a baseline, not an upper bound. Running Strong alone or Cheap alone from start to finish gives two benchmarks for evaluating handoff; the policy’s success rate need not fall between them. In the model, the incidence term estimates the probability of Strong-only success given the task features and caps only the handoff score.

Q1. Does a fixed handoff step suffice? We selected a fixed delay of 16 Cheap actions by maximizing the success rate on Dev300 subject to the cost budget. Under both policies, Strong took over from Cheap’s current state and never switched back. On the original ALFWorld 9B–27B test, TACIT solved 159/274 tasks versus 137/274 for fixed handoff, at similar mean weighted call cost (182.79 vs. 181.09). Adaptive timing thus solved 22 more tasks without restarting.

How does TACIT compare with a discrete-time logistic hazard model? In a separate comparison, we refit TACIT and trained the hazard model on the same 200 tasks and 89 teacher windows, keeping the initial Strong-success model fixed. The hazard model uses step-level diagnostics and elapsed time to estimate when to hand off. It learns directly from the teacher intervals without assigning a binary label to each step. Both policies make a permanent handoff from Cheap’s current state; we selected their thresholds on Dev300 under the same cost cap and froze them before evaluation. On the 274 previously evaluated test tasks, the hazard model solved 151 at a mean weighted call cost of 199.56, versus 148 at 188.38 for the TACIT refit.

Our contribution is the supervision framework. To our knowledge, this is the first framework to learn when one LLM agent should permanently hand off to another by treating an offline teacher’s handoff windows as interval-censored labels. We chose a mixture-cure model because its parameters are interpretable; the framework does not require that choice. In the comparison prompted by your question, replacing its timing component with a discrete-time logistic hazard gave a similar observed success rate at higher cost. A practitioner who values flexibility over interpretable coefficients could fit a nonparametric timing model to the same intervals, an option we have not tested.

AuthorsOpenReview ↗

Response to Reviewer jWr3 (3/3): checkpoint rescues, timing calibration, and call accounting (Q2, Q3)

Q2. Can Strong rescue a task at the teacher window? We resumed Strong from saved ALFWorld states, counting both the Cheap prefix and the Strong continuation in the mean weighted call cost. On 50 unseen tasks selected because Cheap failed and Strong succeeded from the start, handoff just before the window, at the recommendation, and just after the window rescued 38, 37, and 37 tasks (costs 164.48, 171.54, and 171.81). Cheap alone solved none at 112.50: handoff at the recommendation rescued 37/50 for an additional 59.04 per task.

Does handing off even earlier help? As reported in our response to Reviewer F6yD, Strong rescued 56/75 tasks three steps early, 52/75 at the teacher’s recommendation, and 47/75 three steps late. We extended this comparison to six steps early. Only 34 of the 75 trajectories had all four checkpoints, so we compare all four timings on those same 34 tasks.

Handoff relative to teacher recommendationRescued / 34Mean cost
6 steps earlier20/34 (58.82%)202.83
3 steps earlier23/34 (67.65%)189.13
At recommendation21/34 (61.76%)191.32
3 steps later18/34 (52.94%)191.32

Three steps early had the lowest cost and rescued three more tasks than six steps early and five more than three steps late. These are observed differences on this subset, not evidence of a universal best step.

What does this imply for calibration? As also noted in our response to F6yD, TACIT handed off 2.31 steps later than the teacher recommended on average across 117 annotated test tasks. That lag motivated checking the timing offset on Dev. In a separate model comparison with thresholds chosen on Dev, moving the training windows one step earlier and refitting TACIT raised test success from 148/274 to 151/274 and lowered mean cost from 181.65 to 178.15. A joint search over offsets and thresholds under a cost cap on all 300 Dev tasks retained the original windows. The table tests handoff times directly; it does not establish that the three-step offset corrected the fitted model’s lag.

What if Strong also failed from the start (B = 0)? On 21 such tasks where Cheap also failed, Strong rescued 5/21 three steps before the lower window endpoint, 3/21 at the recommendation, and 4/21 three steps after the upper endpoint (costs 267.32, 246.21, and 225.32). Cheap alone solved 0/21 at 112.50. Thus an initial Strong failure did not rule out rescue.

Q3. Are discarded Cheap proposals charged? Yes to all three. The accounting unit is a model call, not an executed tool action. Generating the proposal consumes one Cheap call even if the handoff rule discards it. That call is charged in Equation (20), counts toward the shared 12-call budget, and is included in the denominator of Strong’s call share. Handoff does not reset the budget; Strong receives only the remaining calls.

We checked the call records for all 67 tasks at the reported TACIT operating point. They contain 236 Cheap calls (including discarded proposals) and 118 Strong calls. Strong’s share is 118 / (236 + 118) = 33.33%. The mean weighted call cost is (2.25 × 236 + 6.75 × 118) / 67 = 19.81, and the success count is 49/67. The recorded figures already include the discarded calls.

Reviewer_F6yDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when an LLM agent should permanently hand control from a cheap model to a strong model. TACIT-Switch runs the cheap model first; at each step, hand-designed diagnostics from the cheap model’s own outputs (repeated actions, cycles, NLL, invalid outputs, no progress) are combined with learned weights into a cumulative risk score. Once a fitted score crosses a threshold, the strong model takes over the same trajectory from the current state. The scorer is trained from paired cheap/strong rollouts and offline teacher labels of roughly when to switch, modeled as interval-censored data in a mixture-cure model. On ALFWorld and DABench, it reports higher success than routing baselines at lower or comparable cost, though the claimed mid-trajectory drift and recoverability window are shown only in simulation.

Critical strengths

S1. Practical and well-motivated control structure. The paper frames routing as a one-way handoff in which the strong model continues the same trajectory from the current state, rather than restarting (SWE-Router) or deferring single steps and returning control to the cheap model (ReDAct). The deployed policy is also very cheap: the strong model is not called before handoff, and the switching decision uses only diagnostics computed from the cheap model’s own outputs plus a small fitted formula, with no teacher, value head, or extra LLM at deployment. This matters to my assessment because it makes the method easy to deploy, and the separation between “the strong model cannot rescue this task” and “handoff is not needed yet” is a useful way to think about the problem.

S2. Consistent gains under a careful evaluation protocol. All operating points are selected on development data and frozen before test, and costs are charged consistently across methods (including SWE-Router’s prefix, value-head call, and restart). Using Tables 2 and 3, I compared each method with a matched-cost random-mixing baseline (assigning each task to the strong model with a fixed probability). TACIT-Switch exceeds this baseline by roughly 6 to 14 points in all five real-agent settings, including the larger-scale ALFWorld check. This matters because it suggests the trajectory signals carry real information about when to switch, beyond simply spending more budget on the strong model.

Critical weaknesses

W1. The core premise is not shown in real data, and the gains cannot be attributed to adaptive timing. The paper’s motivation is that the cheap model drifts mid-trajectory and that there is a window in which handoff can still rescue the task. In ALFWorld and DABench, this is never measured: there is no analysis of when failures such as loops or invalid actions start, no check that handing off at the teacher’s marked step actually rescues the task, no counterfactual curve of success versus handoff step, and no report of the learned diagnostic weights, the test-time switch steps, or a per-task win/loss analysis. The only direct evidence (85.6% rescue at the marked step) comes from the authors’ simulator, whose synthetic teacher is itself defined by cumulative-risk threshold crossing, the same structure the model assumes. In addition, the comparison does not isolate the key mechanism. There is no fixed-step permanent-handoff baseline, so it is unclear whether the gain comes from adaptive risk or simply from continuing instead of restarting, as SWE-Router does. This matters most in the 4B setting, where the cheap model succeeds on only 1.5% of tasks and TACIT-Switch hands off on most of them. Several baselines also fall below a matched-cost random-mixing baseline (computed from Tables 2 and 3), which suggests they are weakly configured; for example, the SWE-Router value head is trained on 100 episodes versus about 1.7k trajectories in the original paper. This is central to my assessment because the paper’s contribution is a timing mechanism, but the evidence shows only aggregate success and cost, not that the mechanism works for the reasons claimed.

W2. Limited generality and a narrow experimental regime. The diagnostics and task features are hand-designed separately for each benchmark (5 diagnostics and 12 task features for ALFWorld, 8 diagnostics and 2 task features for DABench); only their weights are learned. The weights are also constrained to be non-negative, so signs of progress cannot lower the risk, and failure modes not anticipated by the designers cannot trigger a handoff. There is no per-diagnostic ablation and no description of how the rules were designed or whether only the training split was used. Applying the method to a new environment therefore requires substantial manual feature engineering, which weakens the claim of a general routing framework. The experiments are also small and single-run (100/40/134 episodes for ALFWorld, 140/50/67 for a custom DABench split), the strong-model outcome labels come from a single stochastic rollout, and the chosen model pairs (a 4B cheap model at 1.5% success) make step-local deferral fail almost by construction. Savings relative to always using the strong model are modest (roughly 12 to 28% cost on ALFWorld, at a 1.5 to 5.2 point drop in success), and the cost proxy counts model calls weighted by parameter count rather than tokens, even though the strong model must read the full cheap-model history after handoff. This matters because it limits how far the results support the paper’s practical claim of cost-efficient deployment.

Questions for authors

Q1. What exactly does the strong model see at handoff: the full cheap-model history including its reasoning, or only actions and observations? SWE-Router restarts partly because continuing may bias the strong model toward the cheap model’s errors. Do you observe this, and how often does the strong model inherit a state from which recovery is impossible?

Q2. Can you add a fixed-step permanent-handoff baseline, where the strong model takes over the same trajectory at step k (k selected on development data), and a SWE-Router variant that continues from the current state instead of restarting? Without these, it is unclear whether the gain comes from adaptive risk-based timing or simply from not restarting, especially in the 4B setting, where the cheap model succeeds on only 1.5% of tasks.

Q3. Please clarify the full deployment cost of monitoring. How is each diagnostic computed? In particular, how is the DABench “semantic mismatch” diagnostic computed, and if it requires an embedding model or LLM call, is that cost included in Equation 20? Can you also report token-based cost, including the strong model’s prefill of the cheap-model history after handoff?

Q4. How were the diagnostics and task features chosen, and was this done using only the training split? Please report the learned diagnostic weights and a per-diagnostic ablation. How would a practitioner obtain these diagnostics for a new environment?

Q5. Can you show in ALFWorld or DABench that the claimed recoverability window exists? For example, on a subset of AB=01 tasks, hand off at the teacher-marked step and at several earlier and later steps, and report the strong model’s success rate at each. How often does handoff at the marked step actually rescue the task in real environments?

Q6. Please report the distribution of test-time switch steps, how these compare with where the teacher would mark, and a per-task win/loss analysis against SWE-Router. Are the tasks TACIT-Switch wins those in which the cheap model first makes progress and then drifts?

Q7. The strong-model outcome B, which decides whether a task is treated as rescuable, comes from a single stochastic rollout, and Appendix C shows that flipping B hurts the estimator much more than shifting teacher intervals. How stable are the results if B is estimated from several strong rollouts? How much do the real-agent results vary across random seeds or sampling runs?

2 comments
AuthorsOpenReview ↗

Response to Reviewer F6yD (1/2): real trajectories, timing controls, and deployment cost

Thank you for recognizing the value of a one-way handoff and for asking whether its gains come from choosing when to switch or simply from preserving Cheap’s progress. We address the two weaknesses and Q1–Q3 here; Q4–Q7 follow in Part 2.

W1. Drift on real trajectories. Of 274 ALFWorld 9B Cheap-only runs, 237 (86.5%) repeated an action three times in a row and 245 (89.4%) repeated a cycle. Among runs with each pattern, it first appeared after 8.4 and 11.2 actions on average, respectively. The groups overlap; repetition alone need not mean failure. In one task, Cheap picked up a ladle at action 2, then repeatedly put it down and picked it up without washing it (actions 3–9). TACIT handed off after action 9; Strong washed the ladle and placed it on the dining table within six more actions. Cheap alone never finished. Across 254 handoffs, switching turned 113 Cheap failures into successes and two Cheap successes into failures. Q5 reports actual rescue at teacher-marked checkpoints.

W1 / Q2. Does adaptive timing help when both policies preserve progress? On the original ALFWorld 9B–27B test, development data selected a fixed handoff after 16 Cheap actions. Both policies let Strong continue from the state Cheap had reached at their respective handoff times. All gates were frozen before test.

PolicySolved / 274Mean weighted call cost
Fixed handoff after 16 actions137181.09
SWE restart, original gate115189.85
SWE continuation, original gate106181.37
SWE continuation, gate reselected on Dev147194.83
TACIT159182.79

At similar mean cost, TACIT solved 22 more tasks than fixed handoff (+8.03 percentage points; paired 95% bootstrap interval [3.28, 12.77]). For SWE, continuation alone did not improve success; the better result required retuning its gate. The 100 training tasks cited in the review apply to the small study; the scale-up SWE head used 3,992 prefixes from 1,000 distinct tasks. A random Cheap/Strong mixture selected under the same Dev budget gave 51.33% expected test success at expected cost 186.89: above SWE restart in success, but below TACIT (58.03% at 182.79). The timing controls are for 9B, not 4B.

W2. What transfers? Paired episode outcomes and an offline teacher’s interval avoid running Strong at every checkpoint or asking the teacher to identify one exact step. No teacher or routing LLM is used at deployment. Hand-designed signals and nondecreasing cumulative risk are choices of our controller, not requirements of the interval supervision. Holding incidence fixed, we fitted two timing models on the same 200 training tasks with the same 89 teacher intervals: the discrete-time logistic hazard solved 151/274 at mean cost 199.56, versus 148 at 188.38 for the mixture-cure refit. The framework accommodates either model. Appendix B.3 (Table 3) reports the 1,000 Train / 300 Dev / 274 Test scale-up. With 4B Cheap at 1.5% success, we do not infer that permanent handoff is generally preferable. In the original 4B–27B test, TACIT solved 65/134 vs. 68/134 for always-Strong, at 17.1% lower parameter-weighted call cost, not measured dollar or time savings.

Q1. What does Strong inherit? In ALFWorld it sees the current state, task, observation, admissible actions and up to 4,000 characters of action–observation history, not private reasoning. In DABench it inherits tool state and bounded logs (including logged rationale), but not the discarded proposal. Of 26 handoffs immediately following an A–B–A–B cycle, Strong repeated A–B in 9; this does not establish that seeing the history caused it. Strong solved 140/254 after handoff. The other 114 exhausted the 50-action budget; 114/254 is an upper bound, not an estimate, of the fraction of irrecoverable states.

Q3. Monitoring and token costs. ALFWorld checks repeated actions, short cycles, invalid actions/fallback, action uncertainty (one minus the geometric mean token probability from the same call), and stagnation (word-set overlap between observations at zero reward). DABench checks format, parsing/routing errors, tool failures, repeated code/output, delayed submission, Python budget, steps since progress, and semantic mismatch. The latter uses schema/type/operator rules, e.g., flagging p_value = result.statistic. Neither monitor calls an embedding model or LLM. We did not time the local checks. Counting inputs and outputs, including Strong’s prefill, TACIT averaged 9,893 tokens per DABench task. On ALFWorld 9B–27B, reconstructed means were 40,701 for TACIT and 48,455 for SWE restart (including the value head); after weighting tokens by model size, TACIT used 2.49% more.

AuthorsOpenReview ↗

Response to Reviewer F6yD (2/2): diagnostics, teacher windows, and stability

Q4. Feature selection, weights and ablations. We took task descriptors from the task specifications and chose diagnostics to capture recurring errors. Feature design was not restricted to Train: we observed repeated actions in both the ALFWorld valid_seen pilot and Train, and explored DABench tool errors before fixing the split. We fitted weights on Train and chose the handoff threshold on Dev. In a new environment, we would identify recurring failures in training traces, turn them into signals observable before the decision, and freeze those definitions before evaluation.

The first four numeric columns below are original ALFWorld weights. The last two come from a separate 9B experiment in which both the full model and each drop-one refit used teacher intervals shifted one step earlier and the same 274 test tasks.

DiagnosticMain 4BMain 9BScale-up 4BScale-up 9BSolved without itMean cost
Repeated action.0008.0003.0017.0002157169.72
Short cycle.0024.0003.2087.0007157169.74
Action uncertainty.6506.6196.5624.4915159163.26
Invalid action/fallback.0009.0106.0025.3125157170.07
Stagnation.3453.3693.2246.1952155169.97
None omitted————157169.72

No single omission changed the success count by more than two. DABench weights were .3846 (format), .1028 (protocol), .0365 (execution errors), .1287 (loops), .1746 (delayed submission), .0004 (tool budget), .0002 (lack of progress), and .1722 (semantic mismatch). We did not run DABench drop-one ablations; weight magnitude alone does not establish a diagnostic’s contribution.

Q5. Does the teacher mark states Strong can rescue? Thank you for suggesting direct checkpoint rollouts. We compared the same 75 ALFWorld tasks on which Cheap (9B) failed, Strong (27B) succeeded from the initial state, and all three checkpoints were available. Costs include the Cheap prefix and Strong continuation.

Handoff timeSolvedMean weighted call cost
Three steps before the recommendation56/75 (74.7%)162.84
At the recommendation52/75 (69.3%)174.81
Three steps after the recommendation47/75 (62.7%)175.26

On these tasks, advancing by three steps rescued four more than the recommendation at lower mean cost; delaying by three rescued five fewer. In a separate comparison on 50 eligible unseen tasks, Strong rescued 38 just before the window, 37 at the recommendation and 37 just after. Your question and the observed timing difference below also prompted a Dev check. On the 98 Dev tasks where Cheap failed and all seven checkpoints were available, handing off one step before the recommendation (selected on this subset) rescued 82 instead of 65 at mean cost 136.31 instead of 176.79.

Q6. Actual switch times and paired wins. In the original 9B–27B evaluation, TACIT switched on 254/274 tasks; 20 never switched. The 254 handoffs followed 1–5 Cheap actions in 61 cases, 6–10 in 106, 11–15 in 48, and 16 or more in 39 (median 8; IQR 6–13). On 117 annotated tasks, 24 switches were before, 34 within and 59 after the teacher window; the fitted policy handed off 2.31 actions later than the recommendation on average. Against SWE restart, TACIT alone solved 68 tasks, SWE alone 24, both 91 and neither 91. An interaction that changed the observation followed by repetition or a short loop occurred in 23/68 TACIT-only wins and 11/24 SWE-only wins. It occurred in some TACIT wins but was not more common among them.

Q7. Variation across Strong runs. The original Strong labels came from one greedy run per task. On the same 100 training tasks, greedy Strong solved 55; three sampling runs (temperature 0.7, top-p 0.9) solved 60, 58 and 64. Outcomes differed on 26 tasks. We replaced those 100 labels with their within-task sampled means and refitted only the component predicting Strong success; the other 900 labels, timing model and handoff threshold stayed fixed. Handoff timing changed on 47/300 Dev tasks; the same 13 never switched, and the mean absolute change among the other 287 was 0.21 actions. We measured sensitivity of handoff decisions, not end-to-end success or cost across seeds.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.