acceptodds
Under review as a conference paper at ICLR 2027

Co-Batching Can Change What Agents Do: Tool Execution and History Effects

Abstract

Tool-using agents turn model outputs into operations that read information, modify records, and complete tasks. Inference servers batch unrelated requests for efficiency, but can these shared execution conditions change an agent’s behavior as well as its output text? We study this question in AppWorld by holding the target request fixed, varying co-batch conditions, executing the resulting actions, and following their consequences and action–observation histories. Repeatable operational forks occur in four of five tested deployment panels. Paired continuations differ on task checks in 36 of 97 evaluable selected states within an eight-decision window, including completed tasks. Restoring common measured database contents still leaves different supplied histories capable of changing subsequent actions. These differences persist across reversed comparison orders, while matched same-history controls agree. On new tasks, average task-check disagreement beyond repeated-condition background is positive but uncertain on both deployments. Deterministic configurations eliminate the tested initiating forks in controlled Qwen and DeepSeek audits; MiniMax shows partial prevention alongside residual and induced differences. The findings show that serving variation can affect executable operations and that database agreement alone is insufficient for behavioral agreement. Evaluating agent reproducibility therefore requires observing serving conditions, executed behavior, and retained interaction history together.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM agents, LLM inference, co-batching, tool use, agent reproducibility, deterministic inference, serving nondeterminism

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kxmPOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how co-batching in LLM inference serving affects agents’ tool calls, executed outcomes, and subsequent interaction histories through controlled experiments in AppWorld. It further examines how these behavioral differences impact task success and evaluates the effectiveness of deterministic inference configurations in mitigating them.

Critical strengths
  1. Unlike prior work that primarily focuses on differences in model outputs or tool calls, this paper further examines executed tool outcomes, database states, and task success. It provides valuable empirical evidence that co-batching-induced behavioral differences can propagate to actual tool execution and affect task outcomes, contributing to our understanding of agent execution reliability.

  2. The paper employs a rigorous experimental design, including history interventions under common database states, reversed execution orders, and same-history controls. These experiments provide convincing evidence that interaction histories can influence subsequent agent behavior.

Critical weaknesses
  1. Insufficient isolation of the co-batching effect. The comparison between singleton and co-batch conditions simultaneously changes batch occupancy and companion request composition, making it difficult to disentangle their individual contributions to the observed behavioral differences.

  2. Limited evidence for the generality of the findings. The most prominent task-outcome differences (36/97) are observed on states preselected for behavioral forks, which do not represent the prevalence of such effects in general agent workloads. Furthermore, the evaluation on 16 unselected tasks in Section 6 yields inconclusive average effects for both models, limiting the generalizability of the findings.

  3. Limited incremental novelty over prior work. Previous studies have already investigated batch-conditioned output variation, agent trajectory divergence caused by inference serving conditions, and the influence of interaction histories on subsequent behavior under common environment states. This paper primarily extends these findings to tool execution and task outcomes. While this provides valuable empirical evidence, it offers limited new mechanistic insights and does not introduce a new mitigation method or general-purpose benchmark.

Questions for authors
  1. How can the authors disentangle the effects of batch occupancy and companion request composition on the observed behavioral differences?

  2. Given the preselection of fork states and the inconclusive results on unselected tasks, what evidence supports the generalizability of these findings to broader agent workloads?

  3. What are the key novel insights of this work beyond prior studies on batch-conditioned output variation, agent trajectory divergence, and history-dependent behavior?

1 comment
AuthorsOpenReview ↗

Reply to kxmP

K1. What happens when only the companions change? Changing only the companions produced no observed excess. We kept the focal request fixed and disabled prefix reuse in the AppWorld comparison. At a batch size of four, replacing three identical-input companions with three length-matched unrelated requests produced zero observed excess action disagreement on Qwen and MiniMax. Changing from singleton to four identical inputs gave zero on Qwen and 0.167 on MiniMax on four H100s (family-cluster interval 0 to 0.500, six families). These controls test the particular companion choices in Table R1.

K2. How should we interpret 36/97? We selected states where singleton and co-batched serving had already produced a repeatable action difference that executed cleanly. In 36 of 97 evaluable states, the two branches differed on at least one task check within eight decisions. This rate describes the selected forks. The two branches differed in official success in 17 cases. Both branches had finished in ten of those cases. In the other seven, at least one branch reached the decision limit (§5.1). The ten completed pairs establish observed task consequences. For the separate screen in K3, we used a fixed rule to choose early states before seeing any controlled-batch outcome. That screen required a valid source action and an environment we could reconstruct.

We extended the capped pairs to a longer decision limit on H20 GPUs. This gave seven DeepSeek pairs with complete, repeat-consistent endpoints. Six reached the same final success outcome and one still differed. These are continuations of the same selected pairs, so the original denominator of selected states does not change. This longer-window comparison is new in this response and is reported separately from the capped outcomes.

K3. What evidence extends beyond selected forks? We also ran a separate AppWorld screen whose states were chosen before observing batch outcomes. In 50 of 200 eligible states on each of DeepSeek and MiniMax, the tool return or the measured state after execution differed. On fresh tasks, both average excesses in requirement disagreement are positive, with intervals that include zero. DeepSeek averaged 0.098 (95% CI −0.036 to 0.233, four positive estimates and twelve zeros). MiniMax averaged 0.089 (−0.035 to 0.214, three positive, eleven zero and two missing). A second environment, τ²-bench retail, also showed pre-execution action changes on DeepSeek (22.0%, 95% CI 12.8–31.2%) and MiniMax (34.2%, 23.1–45.4%). Action changes therefore appear in both tool environments and on both main deployments.

K4. What did we learn beyond earlier work? Earlier batch studies measure changes in outputs, labels or correctness. Earlier common-state comparisons, such as compressed versus full contexts at the same AppWorld state (Min et al.), vary the history by design. In our test, both histories come from the agent's own responses to a batch change. On selected states, we restored the same measured database contents and switched which of these histories the agent saw. Its next action changed in 7 of 8 DeepSeek states and all 8 MiniMax states. The same-history controls agreed, including after we reversed the order (§5.3). Restoring the application data alone can therefore miss a consequence of a batch change. Our contribution is this empirical finding, not a new stabilization method.

Reviewer_3hNnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether changing co-batch serving conditions for a fixed request changes a tool-using agent's executable behavior, rather than just its generated text. Using AppWorld, the authors compare singleton inference against co-batching with three unrelated saved requests while holding the target prompt, tool interface, parsing, model, and documented decoding settings fixed. They execute the resulting tool calls and track observations, database contents, task checks, official success, termination, and retained interaction history.

Critical strengths
  1. Execution-grounded measurements. The study operationalizes the problem beyond text mismatches. It parses ordered tool calls, executes alternatives from restored checkpoints, measures real tool observations and application rows, and separates requirement checks, official success and finite decision caps. This makes the selected-state claim substantially more informative than a simple generation-level discrepancy.

  2. Careful history and repeat controls. Restoring the same measured database contents, reversing the order of source histories across restarted service blocks, and using time-matched same-history controls directly address repeat and ordering artifacts. The authors also make reused states, exclusions and correction procedures relatively transparent. I regard the within-selected-state history evidence as a genuine positive result.

Critical weaknesses
  1. Incremental scientific contribution remains insufficiently distinguished. The broad phenomena motivating the paper - batch-dependent inference variability and divergent agent trajectories despite nominally identical tasks - were already demonstrated in pre-cutoff work (Yuan et al., NeurIPS 2025; Kadadekar, ICML 2026 workshop; Bjarnason et al., ICLR 2026 workshop). Earlier preprints additionally explore structured tool-call inconsistency (Yagubyan, 2026) and agent replay interventions (Shah, 2026). I recognize that this paper connects these strands in a new co-batch-origin history design. However, it is not yet clear what unexpected or generalizable insight this combination establishes beyond showing that serving-dependent action differences can propagate through a history-dependent agent.

  2. Limited external validity and incomplete causal isolation. The strongest downstream results are obtained conditional on selected, repeat-qualified operational forks, not on an unfiltered traffic/task distribution. The 16-family new-task experiment yields mean requirement excess confidence intervals spanning zero for both deployments (DeepSeek: 0.098 [-0.036, 0.233]; MiniMax: 0.089 [-0.035, 0.214]), and MiniMax contributes only 14 complete task estimates. The positive sums are driven heavily by one task per deployment (63.5% / 62.2%). The history counterbalancing has eight complete selected states per deployment. Finally, co-batch occupancy and companion composition change together, and deterministic comparisons alter a configuration bundle. These limitations preclude both a well-isolated serving cause and a convincing estimate of the effect on typical agent tasks.

Questions for authors
  1. What is the central non-obvious scientific finding that is uniquely enabled by the co-batch-origin history intervention, beyond known batch-dependent inference variability and repeated-run agent divergence? Please compare the precise estimand and contribution against pre-cutoff work such as Yuan et al. (2025), Kadadekar (2026), and Bjarnason et al. (2026), rather than only distinguishing experimental settings.
2 comments
AuthorsOpenReview ↗

Reply to 3hNn (1/2)

H1. What does the history intervention add? Two branches can end up with the same application data even though the agent saw different tool calls and returns along the way. We tested whether that distinction matters on the next step. We restored a common database and supplied one or the other action–observation history produced by the original batch comparison. On selected states, the next admissible action changed in 7 of 8 DeepSeek states and all 8 MiniMax states in each of two blocks. The second block reversed each state's history order. Replaying the same history twice gave the same next action in every control. A database snapshot alone can miss this effect (§5.3).

Section 2 compares the interventions and outcomes in the closest studies.

  • Yuan et al. vary inference conditions and measure output divergence and correctness. Kadadekar measures batch-conditioned label flips and tests invariant kernels. We follow the altered tool call into execution and then test the history it leaves behind.
  • Bjarnason et al. measure how repeated coding-agent runs differ in trajectory and resolution. Yagubyan compares the sequences and arguments of repeated tool calls. We vary the batch for a fixed target request and subtract the disagreement between repeats within each condition.
  • Shah replays interventions on agent steps to attribute failures. Patodiya varies cache conditions in stateful tool episodes at batch size one. Min et al. compare compressed and full contexts at a common AppWorld state. In our comparison the two histories come from the agent's own responses to a serving perturbation. We vary the supplied action and observation messages together, hold the measured database contents fixed, and repeat the comparison in reverse order.

Unlike these comparisons, our history test isolates batch-origin histories at matched database contents. It concerns the supplied action and observation messages, not hidden runtime state or carry-over through the model's original reasoning.

H2. How much depends on the largest task? Removing the largest task leaves both estimates positive. In the fresh-task panel, the largest contributor accounted for 63.5% of the positive sum on DeepSeek and 62.2% on MiniMax. The full-panel mean excess in requirement disagreement was 0.098 (95% CI −0.036 to 0.233) and 0.089 (−0.035 to 0.214), respectively. Removing the largest task from each deployment reduced the estimates to 0.0383 (−0.0069 to 0.0835, N01 removed) and 0.0363 (−0.0174 to 0.0900, N09 removed). As in the full panel, both intervals include zero. Figure 5 shows the task-level distribution, and Supplementary Table R2 gives every native-task deletion.

AuthorsOpenReview ↗

Reply to 3hNn (2/2)

H3. Why are there only 14 MiniMax task estimates? We include a co-batched trajectory only when its requests are verified to overlap during the specified window. Seven trajectories lacked complete evidence of this overlap, which left two of the 16 tasks without an estimate. We computed the MiniMax average over the other 14. Figure 5 marks both missing tasks (§6).

H4. How many independent states support the history result? Eight states on each deployment, each with complete measurements in both blocks. We restarted the service and reversed the order of the two histories in the second block, using the same eight states. In each block, switching histories changed the next action in seven DeepSeek states and eight MiniMax states. Repeating the same history never changed the next action. The second block checks the effect of order on these same states (§5.3). App. F gives family-level sensitivity.

H5. Can occupancy and companion composition be separated? On AppWorld, replacing three copies of the target input with three unrelated inputs of matching length kept the batch size at four. The observed excess action disagreement was zero for both Qwen and MiniMax under the request-level tool check. Changing from one request to four identical-input requests gave zero for Qwen and 0.167 for MiniMax on four H100s (family-cluster interval 0 to 0.500, one of six families). Table R1 reports the two comparisons separately. In τ²-bench retail the two four-request arms also agreed in every measured state (59 DeepSeek and 76 MiniMax, Table R4).

H6. Which part of the deterministic configuration helped? Changing attention partition alone suppressed all five qualifying Qwen forks under the request’s tool definitions and all four under the full tool definitions. Changing matrix multiplication alone suppressed three and two, respectively (§7, Table R3).

Reviewer_GvteOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether co-batching changes tool-using agents’ behavior in AppWorld. It follows fixed-request action differences through execution and task checks, examines history effects at common database contents, and evaluates existing deterministic configurations.

Critical strengths

The experiments connect serving variation to executable actions and downstream task outcomes. The common-database history interventions and same-history controls strengthen the evidence for history-dependent behavior.

Critical weaknesses
  1. The underlying mechanism remains insufficiently explained. The experiments establish behavioral differences, but do not identify which numerical operations produce the initiating forks. The configuration comparison changes multiple settings together, leaving the sources of improved action agreement unclear.

  2. The practical scope remains uncertain. Average excess disagreement on new tasks is inconclusive, and the deterministic configuration has mixed results on MiniMax. These findings limit conclusions about how frequently the problem matters and when the tested mitigation should be used.

  3. Evaluation is limited to AppWorld. Although the paper evaluates multiple model deployments, all experiments use AppWorld. It remains unclear whether the observed effects on tool execution and task outcomes extend to other agent environments, such as coding or web browsing. Evaluation in another environment would strengthen the generality of the findings.

Questions for authors
  1. What causes the performance cost of deterministic serving?
  2. Why does the mitigation behave differently across deployments?
  3. What practical mitigation follows from these findings?
3 comments
AuthorsOpenReview ↗

Reply to Gvte (1/3)

G1–G2. Which component matters, and where do the outputs diverge? Your question about the configuration bundle prompted us to change one component at a time. On Qwen3-32B in SGLang on one H100, changing attention partition alone suppressed all five reproducible forks that passed the request’s tool definitions. Four forks passed the full tool definitions, and it suppressed all four. These are two checks on the same 12 selected states. Changing matrix multiplication suppressed three of the five and two of the four. Changing the sampler or log-softmax suppressed none (§7, Table R3).

To see where the outputs first differed, we examined stored runs from 12 selected positive and 12 matched control states. This is a case-control panel, so the counts below describe these 24 states rather than population fork rates. Under standard serving, 88 of 192 singleton/co-batched pairs diverged in output (86 from positive states and two from controls). In 82 of those 88 pairs, the action changed. At the first differing token, the gap between the two highest singleton log-probabilities distinguished those positions from eligible nonfork positions with AUROC 0.9994 (interval 0.9987–0.9998). Using the smallest gap over the singleton output to distinguish forked from nonforked pairs gave AUROC 0.890 (0.766–0.976). Both intervals use a state-cluster bootstrap. The deterministic configuration produced no forks in the same 192 pairs (App. I).

G3. What about the average on new tasks? Both point estimates are positive, and both intervals include zero. The estimated mean excess in requirement disagreement was 0.098 for DeepSeek (95% CI −0.036 to 0.233) and 0.089 for MiniMax (95% CI −0.035 to 0.214) (§6).

G4. Why does mitigation differ across deployments? DeepSeek and MiniMax are different MoE models and use different tensor parallelism (TP8 and TP4). On dense Qwen running on one GPU, changing attention partition alone suppressed every qualifying fork in our panel. Attributing the difference between the two MoE deployments to model, tensor parallelism or serving backend needs component tests on those deployments. On MiniMax, the deterministic configuration prevented 21 of 38 existing action differences. Seventeen persisted, and four of 36 previously stable controls developed a difference. Two of the 40 targets and four of the 40 controls were excluded. In a separate diagnostic on four H200s, 11 of 20 selected states had all four common-prefix margin measurements. Their state-level median top-two margin under a common prefix, served alone, was 0.125 under standard serving and 0.344 under deterministic serving.

AuthorsOpenReview ↗

Reply to Gvte (2/3)

G5. Does the effect extend beyond AppWorld? Yes for DeepSeek and MiniMax, at the action level. We ran DeepSeek and MiniMax in τ²-bench retail, with both arms on H20 GPUs and the frozen serving configuration. For each target request, we compared the action the model chose when served alone with the action it chose in a four-request batch. We included a state when all its experimental conditions finished, regardless of the actions produced.

  • DeepSeek chose a different action in 22.0% of the measured states (95% CI 12.8–31.2%, 59 states from 37 tasks).
  • MiniMax chose a different action in 34.2% (95% CI 23.1–45.4%, 76 states from 40 tasks).

The intervals account for clustering by task. Adding three copies of the target input or three unrelated, length-matched inputs gave the same rates. The two four-request arms agreed at every measured state.

We also ran a smaller Qwen3-32B pilot on one H100. Across 12 tasks and 216 decisions, we observed no difference in the action or resulting database (exact one-sided 95% upper bound of 22.1% per task). This is consistent with AppWorld, where Qwen also has few action differences (12 of 400 screened states, Table 1). The pilot's first decisions were read-only, so it measures first-decision agreement. The DeepSeek and MiniMax results show that action changes carry over to another tool interface. Later task checks and official success are measured in AppWorld (§8, Supplementary Table R4).

AuthorsOpenReview ↗

Reply to Gvte (3/3)

G6. What does deterministic serving cost, and when would we use it? On the Qwen stack we tested, attention partition is the first change we would try. It suppressed all the eligible forks and raised median latency by 14.3% (mean latency by 8.4%) while reducing token throughput by 4.4%. Changing matrix multiplication alone suppressed fewer forks, raised median latency by 21.2%, and reduced throughput by 7.4%. We measured both changes on the same single-H100 workload (Table R3). These single-H100 costs are not comparable with the H200 costs in Table 13.

Table 13 gives the cost of switching the entire configuration on the original H200 deployments. Its values are transcribed from the original submission. Mean latency rose by 7.7% on Qwen, 173.8% on DeepSeek and 78.5% on MiniMax. Token throughput fell by 17.0%, 62.0% and 46.7%, respectively. Qwen and DeepSeek used dynamic arrivals. MiniMax used sequential requests. The measurements exclude cold loading and are specific to these workloads (§7, Table 13).

On a new deployment, we would replay known forks alongside requests whose actions had been stable, using the intended workload. MiniMax illustrates why both checks matter. The configuration fixed some existing differences and introduced a difference on four previously stable controls. We would weigh both outcomes against the measured latency and throughput before enabling it.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General response (1/3)

We are sorry this response comes late in the discussion period, and we thank the reviewers for their patience.

New experiments in this revision.

  • Component tests of the deterministic configuration (Gvte, 3hNn). In a selected Qwen panel, changing attention partition alone suppressed every qualifying fork (§7, Table R3).
  • Occupancy versus companions (3hNn, kxmP). Changing only the companions at a fixed batch size gave zero observed excess action disagreement on Qwen and MiniMax (§4, Supplementary Table R1).
  • A second tool environment (Gvte). In τ²-bench retail, co-batching changed the actions chosen before execution on both DeepSeek and MiniMax (§8, Table R4).
  • Where outputs first diverge (Gvte). A case-control analysis of the token positions where Qwen outputs first differ (§7, App. I).
  • Task-level results (all reviewers). We report the individual task estimates and show how the mean and its confidence interval change when we remove the largest contributor for each model (§6, App. F–H).

Revised passages are marked in blue. Figures 1–8 were redrawn, and Fig. 9 shows the Qwen score margins at nonfork positions.

How often do batch-induced differences arise? (Gvte, 3hNn, kxmP) Before seeing the batch outcomes, we used a fixed rule to choose states with a valid source action and an environment we could reconstruct. In this screen, changing the batch changed the tool return or the measured state after execution in 50 of 200 states on each of DeepSeek and MiniMax (Table 1). On fresh tasks, every task estimate is zero or positive. DeepSeek had four positive estimates and twelve zeros. MiniMax had three positive estimates, eleven zeros and two missing tasks (§6, Fig. 5).

What is new beyond earlier work? (3hNn, kxmP) Restoring the application data alone can miss a consequence of a batch change. On selected states at the same measured database contents, switching to the history from the other serving condition changed the next action in 7 of 8 DeepSeek states and all 8 MiniMax states. Replaying the same history never changed it (§5.3, Fig. 4).

AuthorsOpenReview ↗

General response (2/3)

Does the number of requests matter, or does it matter which requests share the batch? (3hNn, kxmP) We served the target alone (S), added three copies of its input (4T), or added three unrelated inputs matched to its length (4U). The target stayed the same. Comparing S with 4T changes the number of requests. Comparing 4T with 4U keeps that number at four and changes the companions.

In AppWorld, changing companions at fixed occupancy gave zero observed excess action disagreement on both Qwen and MiniMax. We checked actions against the tools listed in the request and against the full tool definitions. For Qwen, 13 task families qualified under the request-level check and all 24 under the full check. Qwen also had zero excess disagreement between S and 4T. All six MiniMax families passed both checks. One of the six showed a difference between S and 4T, giving an estimated excess of 0.167 (family-cluster interval 0 to 0.500). All three MiniMax arms ran on four H100s. Its original AppWorld deployment used four H200s (§4 and Supplementary Table R1).

In the earlier equal-length instruction-companion controls, offline replay with the frozen tool list produced identical output sequences in all 432 cross-pairs. The remaining unknown-function cases traced to a gap in that tool list, not to batching. In the new τ²-bench retail experiment, 4T and 4U produced the same action in every measured state on both DeepSeek and MiniMax (Table R4).

AuthorsOpenReview ↗

General response (3/3)

Which change to the serving configuration helped? (Gvte, 3hNn) We tested individual components on Qwen3-32B on one H100. The original Qwen controlled requests also used one H100, and its dynamic latency workload used H200 (Table 4). The panel contained 12 selected states, six with clean forks and six where a tool error contributed to the fork.

Under the request’s tool definitions, five forks qualified (three clean and two involving errors). Changing attention partition alone suppressed all five. Changing matrix multiplication alone suppressed three. The sampler and log-softmax changes suppressed none. Using the full tool definitions on the same states gave four qualifying forks. Attention partition suppressed all four and matrix multiplication suppressed two. Norm/mean and the full bundle have no component-level result in this panel.

Of the two changes that suppressed forks, attention partition cost less on the matched workload. Median latency rose by 14.3% and token throughput fell by 4.4%. For matrix multiplication, median latency rose by 21.2% and throughput fell by 7.4% (§7, Table R3).

AuthorsOpenReview ↗

Summary of the discussion

Dear AC and reviewers,

Thank you for your time and your careful reviews. Your questions led to several new experiments that strengthened the paper. As the discussion period closes, we summarize the paper and the discussion below.

What the paper shows.

  • Co-batching in LLM serving can change what a tool-using agent does, not only the text it generates.
  • In a pre-specified screen, changing the batch changed the tool return or the measured state after execution in 50 of 200 states on each of DeepSeek and MiniMax (Table 1).
  • On fresh tasks, all 30 task estimates (16 DeepSeek, 14 MiniMax) are zero or positive (§6, Fig. 5).
  • Restoring the application data alone can miss a consequence. At the same measured database contents, switching to the history from the other serving condition changed the next action in 7 of 8 DeepSeek states and all 8 MiniMax states. Replaying the same history never changed it (§5.3, Fig. 4).

Main concerns and what we added.

  • Number of requests versus companions (3hNn, kxmP): at a fixed batch of four, changing only the companions gave zero observed excess action disagreement on Qwen and MiniMax (§4, Supplementary Table R1).
  • Beyond AppWorld (Gvte): in τ²-bench retail, co-batching changed the actions chosen before execution on both DeepSeek and MiniMax (§8, Table R4).
  • Which part of the deterministic configuration helps, and at what cost (Gvte, 3hNn): in a selected Qwen panel, changing attention partition alone suppressed every qualifying fork. Median latency rose by 14.3% and token throughput fell by 4.4% (§7, Table R3).
  • Dependence on single tasks (all reviewers): we report every task estimate, and the mean and its interval after removing the largest contributor for each model (§6, App. F–H).

How the paper changed during the discussion. Before the discussion, three questions were open: whether the effect comes from the number of requests rather than the companions, whether it holds beyond AppWorld, and which deterministic setting removes it at what cost. Each is now answered by a new experiment (Supplementary Table R1, Table R4, Table R3). All task-level estimates and the leave-one-out analysis are now reported in full.

We believe these additions address the main concerns raised in the reviews. Revised passages are marked in blue. If our responses have addressed your concerns, we would be grateful if you would take them into account in your final assessment and score. Thank you again for your time and engagement.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.