acceptodds
Under review as a conference paper at ICLR 2027

When to End, Which to Invoke Next: Transition-Aware Tokenized Procedural Memory for LLMs

Abstract

() has recently emerged as a parameter-efficient paradigm for storing and reusing task-solving procedures in large language models (LLMs), where each procedure is encoded as a compact memory token. This paper investigates a more challenging scenario in which . We identify a of TokMem-style procedure chaining: while memory tokens provide explicit entry points to individual procedures, the transitions between procedures remain implicit. This overlooked handoff mechanism causes next-procedure selection to become increasingly fragile as procedural chains grow longer. To address this issue, we propose (), a lightweight framework that explicitly models inter-procedure handoffs. The key insight behind TapMem is to formulate each procedure transition as a boundary-conditioned selection problem. Since the next procedure should be selected only after the current procedure-guided span ends, TapMem first introduces xplicit ermination ignal, to make this completion boundary observable. Building on the resulting boundary representation, TapMem further incorporates a ransition-onditioned outing dapter to generate a context-dependent correction over candidate memory-token logits. By intervening only at transition positions and only over memory-token logits, TapMem preserves the original full-vocabulary autoregressive generation process while improving the sparse procedural decisions required for reliable multi-step orchestration. Extensive experiments across diverse procedure-recall settings demonstrate that TapMem achieves promising performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Procedural Memory, Memory tokens, Tool Calling, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KqmdOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies errors in chaining tokenized procedural memories. TapMem augments TokMem with an explicit end-of-call token (ETS) and a transition-conditioned routing adapter (TCRA). The adapter adjusts memory-token logits at the initial selection and after generated call boundaries while retaining full-vocabulary autoregressive decoding. During memory learning, the backbone is frozen; the added embeddings and linear adapter are trained with autoregressive and routing losses. The evaluation covers APIGen compositional recall, SNI atomic recall, longer and dependency-structured trajectories, InterCode-Bash execution, and training-free initialization of unseen procedures. Results generally improve over TokMem, with larger gains in several unadapted and longer-chain settings.

Critical strengths
  1. On unadapted Llama 3.2 1B, Table 1 improves average tool-selection F1 from 82.4 to 88.8 and argument F1 from 67.0 to 74.0. Gains also appear on larger Llama models and the Qwen models in Table 5. Table 10 reports only 0.21M–0.41M trainable parameters for TapMem. These results support its practical use for improving frozen-backbone procedural memory without training many parameters.
  2. The explicit boundary mechanism is simple, and Algorithm 2 makes clear that inference uses generated boundaries rather than oracle boundaries. Tables 2/8 provide component ablations, while Figure 11 and Appendix E.5 report an increase in mean correct-prefix length from 2.78 to 3.43. Larger gains at longer call counts support the relevance of the intervention to chained recall.
Critical weaknesses
  1. The comparisons add a linear classifier and an auxiliary routing loss along with transition conditioning. Table 10 approximately doubles the trainable parameter count relative to TokMem, although the absolute budget remains small. Tables 2/8 establish that ETS helps and that the complete system improves further, but do not isolate added routing capacity and supervision as alternative explanations. A particularly relevant control is TokMem+DC, which already separates routing and generation embeddings; its original experiments do not settle this paper's compositional setting. A routing-capacity and supervision-matched control would help determine the distinct benefit of boundary conditioning. This matters to the recommendation because the main contribution is an explanation and remedy for a specific transition bottleneck, beyond the empirical gain from adding a stronger router.
  2. Regarding time cost, the paper reports no direct comparison of inference latency, throughput, or end-to-end execution time. Table 10 measures trainable parameters rather than runtime. Algorithm 2 introduces additional routing computation at the initial selection and after generated tokens; for the same call chain, each call's also adds an autoregressively generated token. The net time effect requires measurement, since changes in output length or fewer errors and retries could offset some overhead. Consequently, the time cost of the accuracy gains is undetermined, making the practical value of the small InterCode success gain particularly difficult to assess. This is not a claim that the method is slower; the concern is that the tradeoff between accuracy and time cost has not been quantified.
Questions for authors
  1. On a representative existing compositional setting, can you add a routing-capacity and supervision-matched control, such as TokMem+DC with matched routing supervision, and explain the additional benefit of explicit boundary conditioning? Separating initial-selection errors from subsequent-transition errors would make the comparison more informative. One focused control would suffice; I am not requesting a new benchmark suite.
  2. On one existing representative configuration, can you compare per-task latency and total generated tokens for TokMem and TapMem under the same hardware, precision, batch size, prompting protocol, and decoding budget, while also reporting accuracy? Please specify warm-up, timing boundaries, and repeated measurements. If using InterCode, distinguish model generation time, command or environment execution time, and end-to-end task time, retaining failed tasks to avoid bias from comparing only successful cases. One controlled comparison would support assessment of the time cost; a performance sweep across all models is unnecessary.
  3. Do the F1 metrics account for call order and repeated calls? Please specify the matching and averaging rules, and report an order-aware or dependency-valid trajectory measure on existing TaskBench/TRAJECT-Bench predictions, allowing equivalent valid orders where appropriate. Also clarify whether APIGen atomic examples are partitioned before composition and how TRAJECT-Bench source trajectories are partitioned. These clarifications establish the generalization scope; the current description alone does not demonstrate leakage.
5 comments
AuthorsOpenReview ↗

Official Comment [1/4]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W1}\text{\\&}\mathbf{Q1:}} Can you add a routing-capacity and supervision-matched control?

A1: Thank you for your comment! We conduct additional ablation experiments to verify the effectiveness of TapMem. The results on Llama-3.2-1B are presented below.

ETSTCRAMethodAdditional Parameters (M)Tool Selection F1 (%)Argument Generation F1 (%)
TokMem0.1082.467.0
\textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\dagger}}0.2087.568.9
\textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\ddagger}}0.2087.765.4
✓ETS-Only0.1084.068.6
✓\textcolor[rgb]{0.357,0.608,0.898}{\textsf{TCRA-only}}0.2086.670.0
✓✓TapMem0.2188.874.0

Additional ablations. We introduce three new variants:

  1. \textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\dagger}}: To match TapMem's parameter budget, following TokMem, we decouple the input and output memory-token embeddings. This is exactly the same method as TokMem+DC following TokMem.
  2. \textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\ddagger}}: To test whether routing supervision alone is sufficient, we further apply routing supervision based on TokMem$^{\dagger}$ to the output memory-token logits without explicit boundary-conditioned routing.
  3. \textcolor[rgb]{0.357,0.608,0.898}{\textsf{TCRA-only}}: To test whether the routing module alone accounts for the gains, we retain the same routing module and routing supervision in TCRA as TapMem but remove ETS. During decoding, routing correction is applied whenever a memory token appears among the base LM's top-kk candidates.

Analysis. With the almost same parameter budget as TokMem, ETS-only improves both F1 scores by 1.6%, demonstrating the benefit of explicit procedure-boundary identification. With a parameter budget comparable to TapMem, both TokMem$^{\dagger}$ and TokMem$^{\ddagger}$ underperform TapMem, indicating that extra parameters or routing supervision alone cannot obtain the gains. Moreover, TapMem outperforms TCRA-only, despite sharing the same routing module and supervision. These results demonstrate the effectiveness of TapMem.

AuthorsOpenReview ↗

Official Comment [2/4]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W2}\text{\\&}\mathbf{Q2:}} Compare inference latency, throughput, end-to-end execution time, per-task latency and total generated tokens for TokMem and TapMem under the same hardware, precision, batch size, prompting protocol, and decoding budget, while also reporting accuracy. Please specify warm-up, timing boundaries, and repeated measurements.

A2: Thank you for your valuable question! We compare TokMem and TapMem using Llama-3.2-1B on APIGen, reporting per-task inference latency, total generated tokens, accuracy, throughput, and end-to-end prediction time. We also clarify the shared experimental settings and the details of warm-up, timing boundaries, and repeated measurements.

Batch sizeMethodRouting Acc (%)Generation latency (s/batch)Total generated tokensThroughput (tasks/s)End-to-end time (s)
1TokMem69.741.97355990.507986.51
1TapMem83.822.00359510.4981003.55
256TokMem69.7491.21355992.724183.57
256TapMem83.8292.90359512.674186.95

Experimental Setup. We evaluate Llama-3.2-1B on the APIGen. Both methods run on one exclusively reserved NVIDIA A100-SXM4-80GB GPU. We measure inference time for both methods using FP32 precision. We compare matched batch sizes of 1 and 256, using greedy decoding with a maximum of 512 newly generated tokens. Routing accuracy is the number of correct tool predictions at matching positions divided by the total number of reference calls.

  • Warm-up: Before measuring performance on the full test set, we run inference on two batches to reduce the impact of one-time initialization on timing. Warm-up is excluded from all reported metrics.
  • Repeated measurements: Each configuration is evaluated three times using the same checkpoint and inference seed of 42. The table reports the means of these timing repetitions.
  • Timing boundaries: Generation time is measured from when tokenized inputs are ready on the GPU until generation is complete, including prefill and decoding. End-to-end time covers processing the full test set, including input preparation, generation, and output parsing. Model loading, warm-up, accuracy scoring, and saving benchmark results are excluded.
  • Prompting protocol: Following TokMem, both methods use user requests from the user_input field of our APIGen compositional test set. We apply the same user-to-assistant template, without additional instructions or few-shot examples.

TapMem achieves higher routing accuracy at a small additional time cost in this setting. Accuracy increases from 69.74% for TokMem to 83.82% for TapMem, while per-task generation time changes from 1.97s to 2.00s. The time difference remains small at batch size 256, where processing the full test set takes 183.57s with TokMem and 186.95s with TapMem.

AuthorsOpenReview ↗

Official Comment [3/4]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{Q3:}} Separating initial-selection errors from subsequent-transition errors would make the comparison more informative.

A3: Thank you for this suggestion! We group samples by initial-selection correctness to examine gains beyond the first choice, the results are shown below. We also assess sequential routing through correct-prefix length (Figure 11 and Appendix E.5).

MethodFirst procedure correctFirst procedure incorrect
TokMem97.4 / 80.953.9 / 41.0
TapMem98.6 / 82.859.8 / 45.3

(1) Gains beyond the first choice. We analyze Llama-3.2-1B on the APIGen. We compute F1 over the complete call sequence within each group. Here each entry reports Tool Selection F1 / Argument Generation F1. These results demonstrate that TapMem improves procedure generation and mitigates the impact of initial-selection errors. The results are shown above.

Number of proceduresTokMemTapMem
21.481.57
32.122.55
42.452.66
52.853.42
63.223.51
73.263.93
83.624.80
93.044.49
103.004.39
Overall2.783.43

(2) Sequential routing reliability. We further evaluate correct prefix length, which measures the number of consecutive correctly selected procedures before the first routing error. As shown in Figure 11, TapMem increases the average correct prefix length from 2.78 to 3.43. The improvement is more pronounced for longer chains, averaging 1.34 over the three longest chain lengths versus 0.24 over the three shortest. These results further demonstrate TapMem's effectiveness in maintaining correct procedure transitions across longer sequences. The results are shown above.

Reviewer_QtieOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a scenario in which a sequence of procedures is needed within a single generation, and the previous one-token procedural memory approach is fragile because the transitions between different procedures are implicit. This paper proposes transition-aware procedural memory, in which it first adds a termination signal at the end of each phase to directly train when to terminate. It also adds a router module to correct the candidate memory-token logits. Experiments and ablation studies show that these two modules make TapMem perform better than TopMem when the number of procedure calls increases.

Critical strengths
  1. The approach is simple and easy to implement. The performance is significantly better than TopMem when the base model is not fine-tuned, and the performance gap is increasing when the number of procedure calls increases.

  2. There are ablation studies showing that both modules have benefits, and the performance is not sensitive to the hyperparameters.

Critical weaknesses
  1. TapMem is not clearly better when the model is fine-tuned (Topmem+adaptation vs Tapmem + adaptation in Table 1). Since both methods require training the model, the practical significance of TapMem’s gains with adaptation seems more important, while the gains without adaptation are unclear. Hence, given that TapMem's improvement is not clear in the adaptation setting, the practical significance of TapMem is relatively weak.

  2. The novelty of the approach is somewhat limited. The explicit termination signal can be regarded as an end token in each procedure, and the TCRA is a learned routing module. Neither of these two modules has novel techniques.

  3. (minor) The results in Figure 6 (multi-turn execution setting) cannot be regarded as a clear improvement, since the performance gap is less than 1% and evaluation noise cannot be ruled out.

Questions for authors
  1. Since TopMem and TapMem require training, is the adaptation setting more important? What is the training cost of fine-tuning the backbone model on compositional recall data, compared to the training cost of TopMem and TapMem?

  2. In both Table 1 and Table 4, the performance is not increasing when the number of the procedure calls increases. Then, what is the motivation to study multiple calls within one generation? Could you find one experiment setting that multiple call is necesssary for better performance, to illustrate the practical significance of TapMem?

7 comments
AuthorsOpenReview ↗

Official Comment [1/3]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W1}\text{\\&}\mathbf{Q1:}} Is the adaptation setting more important? TapMem's improvement is not clear in the adaptation setting. The training cost of fine-tuning the backbone model?

A1: Thank you for raising these questions. We clarify that adaptation is an optional extension. Following TokMem, adaptation trains the backbone to acquire general knowledge of procedure orchestration, improving its initial performance. However, the gains of adaptation are limited because autoregressive prediction only implicitly models procedure transitions. Nevertheless, TapMem outperforms TokMem with adaptation. We then confirm the gains through additional experiments on TaskBench and longer chains. Overall, we address these questions from three aspects: the role of adaptation, performance improvements, and training costs.

(1) Role of adaptation. Following TokMem, adaptation teaches the backbone to learn general patterns of compositional memory recall. Adaptation facilitates memory token prediction, leaving argument generation as a greater challenge, whereas TapMem explicitly models procedure transitions to improve subsequent procedure generation. We therefore focus on the setting without adaptation to evaluate TapMem's contribution independently, while also reporting results with adaptation.

(2) Improvement of adaptation. We acknowledge that the improvement in Tool Selection F1 under the adaptation setting is modest in Table 1. However, the improvement in Argument Generation F1 is clearer: TapMem outperforms TokMem by 1.6% on APIGen with Llama-3.2-1B. The gain in Tool Selection F1 is smaller because both methods already achieve around 98% or higher under adaptation, leaving little room for improvement. Argument generation remains more challenging, and TapMem continues to improve this metric.

Performance with adaptation on TaskBench. We further evaluate TapMem and TokMem with adaptation on TaskBench. The results are shown below.

ModelMethodTool Selection F1Argument Generation F1
Llama-3.2-1BTokMem95.571.7
TapMem96.973.4
Llama-3.2-3BTokMem96.073.4
TapMem97.374.6
Llama-3.1-8BTokMem94.871.2
TapMem96.073.6

Across all three backbone sizes, TapMem outperforms TokMem. Accordingly, TapMem achieves a clear improvement of 2.4% in Argument Generation F1 on Llama-3.1-8B. These results demonstrate that TapMem remains effective for interdependent procedure orchestration even after adaptation.

Performance with adaptation on longer chains. For long chain with up to 10 calls, we further evaluate TapMem and TokMem with adaptation on APIGen. The results are shown below.

MethodTool Selection F1Argument Generation F1
TokMem w/ adaptation97.3482.64
TapMem w/ adaptation98.3584.13

To examine how the advantage varies with chain length, we report results for 3, 5, and 10 calls below.

CallsTokMem Tool F1TapMem Tool F1TokMem Argument F1TapMem Argument F1
398.198.986.286.7
597.598.484.085.4
1093.696.178.779.8

TapMem improves both metrics at all three reported lengths. Its Tool Selection F1 advantage increases from 0.8% at 3 calls to 2.5% at 10 calls. These results indicate that TapMem provides more pronounced improvements over TokMem on longer procedure chains, even with backbone adaptation.

(3) Training Cost. Backbone adaptation introduces additional costs in trainable parameters, training time, and data requirements. To quantify these costs, we conduct three independent runs on APIGen using Llama-3.2-1B. The average results are shown below. Here training time measures the time spent in the training loop, while end-to-end time includes the entire process, from model loading and data preparation to training and checkpoint saving.

MethodTrainable Parameters (M)Training Time (s)End-to-End Time (s)
TokMem w/o Adaptation0.10406.05483.21
TapMem w/o Adaptation0.21409.82480.26
TokMem w/ Adaptation1.06665.41772.97
TapMem w/ Adaptation1.26682.57796.93

Adaptation requires an additional training stage with more training time. As shown in Table 10, fine-tuning the backbone also involves more trainable parameters. On the 8B model, fine-tuning uses 3.41M trainable parameters, compared with 0.20M for TokMem and 0.41M for TapMem. In our experiments, adaptation requires an additional 5,000 training samples.

AuthorsOpenReview ↗

Official Comment [2/3]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W2:}} The novelty of the approach is somewhat limited. The explicit termination signal can be regarded as an end token in each procedure, and the TCRA is a learned routing module. Neither of these two modules has novel techniques.

A2: TapMem provides a simple yet effective solution to a critical bottleneck in procedural memory orchestration. Our motivation is that existing methods overlook procedure transitions, while our innovation lies in explicitly separating procedure termination and next-procedure selection through boundary-conditioned routing. This approach avoids complex architectures while introducing only modest additional costs.

(1) Evidence. We observe that TokMem can effectively recall individual procedures but struggles with transitions between them. Figure 1(b–c) shows that TokMem's transition errors increase as procedure chains grow longer, while TapMem consistently reduces these errors. Table 2 then shows that ETS and TCRA improve their respective decisions. These observations reveal a critical bottleneck in existing tokenized procedural memory methods and motivate explicit transition modeling.

(2) Effectiveness. The lightweight design introduces only 0.11M additional trainable parameters compared to TokMem. On APIGen with Llama-3.2-3B, TapMem improves Tool Selection F1 from 74.2% to 81.9%, achieving a 7.7% improvement. These results demonstrate that TapMem effectively addresses the transition bottleneck through a simple and lightweight design.

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W3:}} The results in Figure 6 (multi-turn execution setting) cannot be regarded as a clear improvement, since the performance gap is less than 1% and evaluation noise cannot be ruled out.

A3: Thank you for your valuable comment! To provide a more comprehensive evaluation, we introduce τ-bench[1] to evaluate TapMem in multi-turn user interactions involving stateful tool execution, and extend the maximum number of command execution rounds to 10 on InterCode-Bash. These results further demonstrate TapMem's effectiveness in more complex multi-turn execution scenarios and longer interactions.

(1) Additional evaluation on real execution. We further evaluate TapMem on τ-bench, which involves user interactions and executable tools.

MetricTokMem (%)TapMem (%)Δ\Delta (%)
Task Success Rate ↑29.2835.94+6.67
Database Match Rate ↑33.0438.84+5.80
Invalid Action Rate ↓5.863.24+2.63

Experimental settings. We train Llama-3.2-3B on an APIGen-MT[2] subset, comprising multi-turn trajectories from the τ-bench Airline and Retail environments. We use Airline data for backbone adaptation and Retail data for training TokMem and TapMem. Here, database match rate measures the proportion of tasks whose final database states exactly match the reference states. Invalid action rate measures the proportion of decisions that fail action validation.

Analysis. TapMem outperforms TokMem across all three metrics. These results demonstrate that TapMem's improvements extend beyond action validity to better task completion in multi-turn execution environments.

(2) Longer interactions on InterCode-Bash. We extend the two-round execution limit to 3, 4 and 10 rounds to evaluate TapMem under longer interactions. The results of success rate are shown below.

Maximum Execution RoundsTokMem (%)TapMem (%)
232.5033.33
335.8337.00
436.1737.17
1037.8338.50

Analysis. TapMem achieves higher success rates than TokMem under all five execution budgets. Although the overall gains remain modest, these results provide additional evidence of TapMem's effectiveness and stability under longer execution budgets.

[1] Shunyu Yao, Noah Shinn, et al. "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains" International Conference on Learning Representations (ICLR), 2025.

[2] Akshara Prabhakar, Zuxin Liu, et al. "APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay" Advances in Neural Information Processing Systems (NeurIPS), 2025.

AuthorsOpenReview ↗

Official Comment [3/3]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{Q2:}} In both Table 1 and Table 4, the performance is not increasing when the number of the procedure calls increases. Then, what is the motivation to study multiple calls within one generation? Could you find one experiment setting that multiple call is necesssary for better performance, to illustrate the practical significance of TapMem?

A4: We sincerely apologize if the description of our contributions caused any misunderstanding. Our goal is to improve the reliability of procedural orchestration in tasks that inherently require multiple procedure calls, rather than introducing additional calls to improve performance. We clarify this distinction from three perspectives:

  1. Motivation for multiple procedure calls. Many tasks require multiple procedures to accomplish interdependent operations. In such scenarios, the number of calls is determined by the task requirements rather than chosen to improve performance. Our work focuses on making such necessary procedure transitions more reliable. Therefore, all of our experiments need multiple calls except for atomic recall.
  2. Performance degradation with longer chains. As each additional call introduces risks of selection errors and error propagation, longer procedure chains become more challenging. As illustrated in Figure 9, TokMem selects an incorrect procedure at the first step and terminates prematurely, failing to complete the subsequent procedure. Therefore, performance degradation with longer chains is expected. The key question is whether a method can maintain higher reliability as task complexity increases.
  3. Effectiveness of TapMem. To further evaluate multi-call reliability, we analyze the consecutive correct prefix length in Figure 11, which measures the number of correctly selected procedures before the first routing error. TapMem increases the average correct prefix length from 2.78 to 3.43. More importantly, its advantage becomes more pronounced for more calls, with an average gain of 1.34 over the three longest chain lengths, compared with only 0.24 over the three shortest. These results demonstrate the effectiveness of TapMem on multiple calls.
Reviewer_c8DROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper builds on TokMem and targets a specific failure: when the model calls several procedures in a row, it often picks the wrong next one. The authors propose TapMem. The model first generates an <EOC> token to mark that the current procedure has finished, and a small routing module then adjusts the scores of the candidate procedure tokens to help the model choose the right next step. During training, the base model is frozen and only the new token embeddings and the routing module are updated. In the experiments, TapMem beats TokMem on both tool selection and argument generation, and the gap grows on longer call chains. On the real multi-turn Bash execution task, however, the success rate only goes from 32.50% to 33.33%.

Critical strengths

(1) The paper focuses on an important problem and the motivation is very strong.

(2) The proposed method is very easy to follow, which make the contribution in this paper clear.

(3) Broad experimental settings. The experiments cover several model families, as well as long call chains, multi-turn execution, and newly added procedures. Compared with testing only on standard procedure recall, this gives a fuller picture of how the method behaves.

(4) Ablations support the two-part design. Adding the end marker alone gives a small gain, while adding the routing module on top of it leads to a much larger improvement.

Critical weaknesses

(1) The ablation study in this paper is not enough. The authors compared TokMem, TokMem with ETS only and TokMem with both ETS and TCRA. However, the comparison with TCRA-only parameter-matched baseline are missing. It's unclear whether the performance gain is from modeling transitions or from the extra parameters and routing supervision.

(2) The performance gain in the multi-turn execution is very small. For example, on InterCode-Bash, the success rate is improved from 32.50 to 33.33, and the reward is from 0.777 to 0.780. In addition, the evaluation uses at most 2 rounds of commands, which will limit the contribution and raise concerns about reliability in longer interactions.

(3) Most experiments focus on procedure recall and argument prediction. The compositional samples in APIGen are built by concatenating independent queries, which may not reflect real dependencies between calls. Execution tests involving intermediate results, conditional branches, and error recovery would better support the paper’s practical value.

(4) Continual learning results are still preliminary. Even with the best cold-start configuration, Argument F1 on unseen procedures is only 22.8%. There is still a lot of room for improvement before the method can reliably support newly added procedures.

Questions for authors

N/A

8 comments
AuthorsOpenReview ↗

Official Comment [1/3]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W1:}} There is no setting with the routing module alone and no parameter-matched baseline. So it is still hard to tell whether the gains come from explicitly modeling procedure transitions, or simply from the extra parameters and routing supervision.

A1: Thank you for your comment! We conduct additional ablation experiments to verify the effectiveness of TapMem. The results on Llama-3.2-1B are presented below.

ETSTCRAMethodAdditional Parameters (M)Tool Selection F1 (%)Argument Generation F1 (%)
TokMem0.1082.467.0
\textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\dagger}}0.2087.568.9
\textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\ddagger}}0.2087.765.4
✓ETS-Only0.1084.068.6
✓\textcolor[rgb]{0.357,0.608,0.898}{\textsf{TCRA-only}}0.2086.670.0
✓✓TapMem0.2188.874.0

Additional ablations. We introduce three new variants:

  1. \textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\dagger}}: To match TapMem's parameter budget, following TokMem, we decouple the input and output memory-token embeddings.
  2. \textcolor[rgb]{0.357,0.608,0.898}{\textsf{TokMem}^{\ddagger}}: To test whether routing supervision alone is sufficient, we further apply routing supervision based on TokMem$^{\dagger}$ to the output memory-token logits without explicit boundary-conditioned routing.
  3. \textcolor[rgb]{0.357,0.608,0.898}{\textsf{TCRA-only}}: To test whether the routing module alone accounts for the gains, we retain the same routing module and routing supervision in TCRA as TapMem but remove ETS. During decoding, routing correction is applied whenever a memory token appears among the base LM's top-kk candidates.

Analysis. With the almost same parameter budget as TokMem, ETS-only improves both F1 scores by 1.6%, demonstrating the benefit of explicit procedure-boundary identification. With a parameter budget comparable to TapMem, both TokMem$^{\dagger}$ and TokMem$^{\ddagger}$ underperform TapMem. Moreover, TapMem outperforms TCRA-only, despite sharing the same routing module and supervision, indicating that extra parameters or routing supervision alone cannot obtain the gains. These results demonstrate the effectiveness of TapMem.

AuthorsOpenReview ↗

Official Comment [2/3]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W2:}} Small gains on the real execution task. On InterCode-Bash, the success rate only goes from 32.50% to 33.33%, and no error bars are reported for this comparison. The evaluation also allows at most two rounds of command execution, which says little about reliability in longer interactions.

A2: Thank you for your valuable comment! To provide a more comprehensive evaluation, we introduce τ-bench[1] to evaluate TapMem in multi-turn user interactions involving stateful tool execution, and extend the maximum number of command execution rounds to 10 on InterCode-Bash. These results further demonstrate TapMem's effectiveness in complex tool-use scenarios and longer interactions.

(1) Additional evaluation on real execution. We further evaluate TapMem on a subset of τ-bench, which involves user interactions and executable tools. The results are shown below.

MetricTokMem (%)TapMem (%)Δ\Delta (%)
Task Success Rate ↑29.28 ± 3.2935.94 ± 1.00+6.67
Database Match Rate ↑33.04 ± 3.7938.84 ± 0.50+5.80
Invalid Action Rate ↓5.86 ± 3.113.24 ± 2.32+2.63

Experimental settings. We train Llama-3.2-3B on an APIGen-MT[2] subset, comprising multi-turn trajectories from the τ-bench Airline and Retail environments. We use Airline data for backbone adaptation and Retail data for training TokMem and TapMem, followed by evaluation on τ-bench. Here, database match rate} measures the proportion of tasks whose final database states exactly match the reference states. Invalid action rate measures the proportion of decisions that fail action validation.

Analysis. TapMem outperforms TokMem across all three metrics, improving task success rate by 6.67% and database match rate by 5.80%, while reducing invalid action rate by 2.63%. These results demonstrate that TapMem's improvements extend beyond action validity to better task completion in interactive execution environments.

(2) Longer interactions on InterCode-Bash. We extend the 2-round execution limit to 3, 4 and 10 rounds to evaluate TapMem under longer interactions. The results of success rate are shown below.

Maximum Execution RoundsTokMem (%)TapMem (%)
232.50 ± 2.2933.33 ± 1.89
335.83 ± 0.7637.00 ± 0.50
436.17 ± 0.7637.17 ± 0.76
1037.83 ± 1.5338.50 ± 0.87

Analysis. TapMem achieves higher success rates than TokMem under all five execution budgets. TapMem also exhibits lower standard deviations in most settings, indicating greater consistency across runs. Although the overall gains remain modest, these results provide additional evidence of TapMem's effectiveness and stability under longer execution budgets.

[1] Shunyu Yao, Noah Shinn, et al. "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains" International Conference on Learning Representations (ICLR), 2025.

[2] Akshara Prabhakar, Zuxin Liu, et al. "APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay" Advances in Neural Information Processing Systems (NeurIPS), 2025.


\textcolor[rgb]{0.086,0.467,1}{\mathbf{W3:}} The compositional samples in APIGen are built by concatenating independent queries, which may not reflect real dependencies between calls.

A3: We apologize for the unclear presentation. Our paper already includes evaluations on TaskBench and TRAJECT-Bench (Figure 3 and Table 6), which involve interdependent tool calls. Specifically, TaskBench models structured workflows with explicit dependencies defined by directed tool graphs, while TRAJECT-Bench provides realistic execution trajectories with interdependent procedure calls. For clarity, we summarize the results on Llama-3.2-1B below.

BenchmarkMetricTokMem (%)TapMem (%)Δ\Delta (%)
TaskBenchTool Selection F188.993.8+4.9
Argument Generation F166.670.7+4.1
TRAJECT-BenchTool Selection F176.781.9+5.2
Argument Generation F145.952.6+6.7

We further evaluate TapMem on τ-bench, which involves multi-turn user interactions and stateful tool execution, reflecting real dependencies between calls. Detailed experimental settings and analysis are provided in A2. For convenience, we summarize the results below.

MetricTokMem (%)TapMem (%)Δ\Delta (%)
Task Success Rate ↑29.28 ± 3.2935.94 ± 1.00+6.67
Database Match Rate ↑33.04 ± 3.7938.84 ± 0.50+5.80
Invalid Action Rate ↓5.86 ± 3.113.24 ± 2.32+2.63

These results demonstrate that TapMem's improvements extend beyond independent procedure compositions to workflows involving interdependent procedures and interactive tool execution.

AuthorsOpenReview ↗

Official Comment [3/3]

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W4:}} Execution tests involving intermediate results, conditional branches, and error recovery would better support the paper’s practical value.

A4: Thank you for your helpful suggestion! As described in A2, the additional τ-bench evaluation includes intermediate execution results, conditional branches, and error recovery. For convenience, we summarize the results below, which can support the paper's practical value.

MetricTokMem (%)TapMem (%)Δ\Delta (%)
Task Success Rate ↑29.2835.94+6.67
Database Match Rate ↑33.0438.84+5.80
Invalid Action Rate ↓5.863.24+2.63

For a concrete example, consider task ID 30 in the official τ-bench test set, involving customer Olivia Lopez. She received a damaged tablet and wants an identical replacement. If no replacement is available, she wants to return the tablet and cancel its accompanying charger. The customer also wants to return a pair of sneakers. A successful TapMem rollout handles this request as follows:

  • Intermediate results. The model queries the customer’s orders and uses the returned order and item IDs in later tool calls.
  • Conditional branches. The requested tablet is out of stock, so the model submits a return and cancels the associated charger after customer confirmation. If an identical replacement were available, the task would instead call for an exchange.
  • Error recovery. When handling the sneakers, the model incorrectly attempts to cancel an already delivered order. The environment rejects both attempts. After further dialogue, the model switches to return_delivered_order_items, and the return request succeeds.

\textcolor[rgb]{0.086,0.467,1}{\mathbf{W5:}} Continual learning results are still preliminary.

A5: Thank you for your comment! Cold-start adaptation in tokenized procedural memory is a challenging problem. In this paper, our experiments represent an initial exploration rather than a proposal for a new continual learning algorithm. Our primary contribution is explicitly modeling procedure transitions to improve procedure orchestration. Furthermore, the cold-start methods studied in our paper are all plug-and-play and require no gradient updates, making them suitable for settings with limited data. When more data are available, new procedures can be learned directly through training while freezing parameters of existing procedures. We then implement the experiment with enough data. TapMem outperforms TokMem in both training-based and cold-start continual learning settings. Results of training-based method are shown below.

MethodTool Selection F1 (%)Argument Generation F1 (%)
TokMem64.69 ± 1.1054.63 ± 2.15
TapMem73.78 ± 1.9963.12 ± 1.93
Δ\Delta+9.09+8.49

Results of representative cold-start methods are shown below, which can be found in Figure 10. Here each entry reports Tool Selection F1 / Argument Generation F1.

MethodTokMemTapMemΔ\Delta
Cluster Mean39.00 / 30.7340.43 / 40.30+1.43 / +9.57
Ridge50.20 / 35.1554.95 / 38.97+4.75 / +3.82
Top-3 Cosine46.28 / 31.3954.71 / 36.53+8.43 / +5.14
Top-3 Softmax54.64 / 43.9064.33 / 49.60+9.69 / +5.70

Comments on the paper

1 comment

Dear All,

We sincerely thank all the reviewers for their thoughtful and constructive feedback. Within the rebuttal period, we have made our best effort to address every concern with new experiments, new analyses and detailed clarifications.

Our main efforts are summarized as follows:

  • Substantiate the contribution of explicit transition modeling (\textcolor[rgb]{0.086,0.467,1}{\textbf{c8DR W1; Qtie W2, Q2; Kqmd W1/Q1:}}) We clarify the motivation behind TapMem and provide additional ablations to examine the source of its improvements:

    • Contribution and motivation (\textcolor[rgb]{0.086,0.467,1}{\textbf{Qtie W2, Q2}}): We clarify that TapMem treats inter-procedure transitions as boundary-conditioned routing problems, explicitly separating procedure termination from next-procedure selection. Multiple calls arise from task requirements, and our objective is to improve the reliability of these necessary transitions.
    • Additional ablations (\textcolor[rgb]{0.086,0.467,1}{\textbf{c8DR W1; Kqmd W1/Q1}}): To distinguish the benefits of explicit transition modeling from those of additional parameters or routing supervision, we add a parameter-matched TokMem baseline, a TokMem variant with additional routing supervision, and a TCRA-only variant. The results show that TapMem outperforms all three variants, supporting the benefit of explicit boundary conditioning.
  • Expand evaluation across interactive execution, backbone adaptation, and continual learning (\textcolor[rgb]{0.086,0.467,1}{\textbf{c8DR W2, W3, W4; Qtie W1/Q1, W3}}). We supplement the original experiments to examine TapMem under the practical settings:

    • Interactive execution and longer interactions (\textcolor[rgb]{0.086,0.467,1}{\textbf{c8DR W2, W3; Qtie W3}}): To assess reliability in interactive execution and longer interactions, we add a τ-bench evaluation involving stateful tool execution and extend the execution budgets on InterCode-Bash. We also provide a rollout illustrating intermediate results, conditional branching, and error recovery. On τ-bench, TapMem improves task success rate from 29.28% to 35.94% over TokMem.
    • Effectiveness with backbone adaptation (\textcolor[rgb]{0.086,0.467,1}{\textbf{Qtie W1/Q1}}): To evaluate TapMem with backbone adaptation, we add experiments on TaskBench and longer APIGen chains. TapMem improves both F1 metrics on TaskBench and achieves larger Tool Selection F1 gains on longer APIGen chains, demonstrating further benefits from explicit transition modeling.
    • Learning newly introduced procedures (\textcolor[rgb]{0.086,0.467,1}{\textbf{c8DR W4}}): To assess TapMem’s ability to incorporate new procedures, we add experiments on training-based memory expansion. Together with existing cold-start results, TapMem outperforms TokMem both with and without additional training, showing that explicit transition modeling remains effective as procedural memory expands. We further clarify that our cold-start study is an initial exploration rather than a primary contribution of this paper.
  • Extend the evaluation of procedure orchestration (\textcolor[rgb]{0.086,0.467,1}{\textbf{Kqmd Q1, Q3; Qtie Q2}}). We provide additional analyses and clarify evaluation details:

    • Sequential routing and dependency preservation (\textcolor[rgb]{0.086,0.467,1}{\textbf{Kqmd Q1, Q3; Qtie Q2}}): To examine errors in procedure transitions, we report performance by initial-selection correctness and revisit correct prefix length. We also report order-aware routing accuracy and introduce Dependency Satisfaction Rate. These analyses show that TapMem maintains correct procedure transitions over longer chains and better preserves task dependencies.
    • Metric definitions and data partitioning (\textcolor[rgb]{0.086,0.467,1}{\textbf{Kqmd Q3}}): To clarify the evaluation protocols, we specify the F1 matching and averaging rules. We also explain the data splits for APIGen and TRAJECT-Bench.
  • Quantify computational costs (\textcolor[rgb]{0.086,0.467,1}{\textbf{Kqmd W2/Q2; Qtie Q1}}). We measure inference costs and the additional cost of backbone adaptation:

    • Inference costs (\textcolor[rgb]{0.086,0.467,1}{\textbf{Kqmd W2/Q2}}): Reviewer Kqmd notes the need to quantify the computational cost of the accuracy gains. Our results show that TapMem achieves clear performance gains at only a small additional cost. The comparison covers latency, throughput, generated-token counts, and end-to-end prediction time under matched settings. We also specify the warm-up, timing boundaries, and number of repeated measurements.
    • Backbone adaptation costs (\textcolor[rgb]{0.086,0.467,1}{\textbf{Qtie Q1}}): To assess the additional cost of backbone adaptation, we report its trainable parameters, training time, and data requirements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.