- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SAGE, which studies sequential AscendC kernel generation with a frozen backbone model. It tracks whether retrieved experiences are adopted, assigns them outcome-based credit, and consolidates useful records into bounded resident memory. On a sequential 88-operator NPUKernelBench stream, it improves execution rate over controlled memory baselines across three backbones; for GLM-5.2 the reported rate is 84/88, versus 74/88 for Value Memory. On 16 unseen xCCL tasks, it obtains 11/16 versus 9/16, with memory updates still enabled. A separate frozen-bank cross-backbone experiment reuses the same 88 operators that built the bank.
Critical strengths
- The online protocol scores each task before updating memory from that task, directly testing whether accumulated experience helps later tasks. Shared-harness baselines, component ablations, and a fixed-bank ranking control make the main comparison informative.
- Adoption-aware credit and utility-gated consolidation address distinct parts of memory management. A fixed-bank reranking control isolates credit quality, and the frozen cross-backbone experiment provides evidence that learned memory is usable by another generator on the same task suite.
- The hardware timing protocol is comparatively clear: five warmups and 100 device-event-timed repetitions are reported.
Critical weaknesses
- The xCCL result combines source-memory reuse with continued learning on xCCL, exactly as the authors state. It shows adaptation to a new task library but cannot quantify how much benefit the initial source memory provides before xCCL updates. The separate frozen-memory cross-backbone test uses the same 88 operators that contributed to the bank, so direct frozen-memory transfer to unseen operators remains outside the demonstrated scope. This is a boundary on the transfer interpretation, rather than a flaw in the stated online-adaptation experiment.
- Table 7 freezes the final SAGE bank and replays historical SAGE adoption traces to create ranking variants. No separate evaluation set is identified, so its rates appear to come from the same 88-operator suite that produced the bank and traces. Holding contents fixed is useful, but this retrospective setup does not preserve the online pre-task information boundary and cannot alone isolate credit assignment's prospective contribution; the online Figure 5 ablation is more relevant for that claim. The paper does not specify clearly whether Table 7 reruns generation or how target-operator and future-operator records are handled during evaluation.
- The paper's own order comparison shows substantial Level-3 sensitivity: execution rate is 38.9% for Level-3 only, 61.1% for a mixed stream, and 77.8% when Level-2 precedes Level-3. The principal stream has only one fixed order. Appendix B.4 provides SAGE's per-operator xCCL speeds for GLM-5.2/5.3, but not a task-by-task SAGE-versus-Value-Memory correctness comparison behind the two-task xCCL margin. Order or seed uncertainty is also absent.
- A 10K-token resident-memory cap does not by itself establish efficiency. Actual prompt tokens, retrieval/model calls, and end-to-end time across baselines are not reported. Implementation and memory traces are also unavailable in the supplied materials.
Questions for authors
- Can you show per-task SAGE-versus-Value-Memory xCCL correctness outcomes and mark when each first retrieves or adopts xCCL-derived memory? Appendix B.4 already provides SAGE's per-operator speeds across two backbones; the requested comparison would explain the two-task execution-rate margin. A small frozen-source-memory control, if feasible, would separate initial transfer from later adaptation.
- For Table 7, were the three ranking variants evaluated in new generation runs or by replaying saved runs? When scoring a target operator, could its own records or outcomes from later operators in the stream appear in the frozen bank or ranking utilities? Please make the information boundary explicit so this control can be interpreted separately from the online ablation.
- Can you report actual resident/input tokens, retrieval and model calls, and wall-clock time for the compared methods? What order or seed variation is available for the main 88-task stream?
2 comments
Thank you again for your detailed questions. Our response clarifies the transfer and memory-evaluation protocols, identifies which conditions were rerun and which results were reused, and provides per-operator xCCL outcomes and synthesis-cost statistics for nine matched tasks.
As the discussion period draws to a close, we would appreciate hearing whether any concerns remain about the experimental setup or interpretation of the results. We would be happy to provide further clarification on any specific point.
Thank you for requesting the transfer protocols and per-task evidence.
W1 · Transfer scope
Fig. 3 measures online xCCL adaptation starting from benchmark-evolved memory. Fig. 4 tests whether another backbone can use a frozen donor bank on the same 88-operator suite.
The Initial and GLM Transfer conditions use the same recipients, tasks, round cap and hardware evaluator. They compare a frozen seed bank with a frozen donor bank built on the same suite. Native provides the recipient’s online reference. We will clarify these settings in §3.5, Fig. 4 and Appendix D.3.
W2 · Interpretation of Table 7
The two new ranking runs use only records preceding the target operator. The ATU row comes from the online run in Table 1. Q2 gives the protocol for each row.
W3 · Order sensitivity and the xCCL margin
Table 2 compares interleaving with a sequential L2→L3 schedule on the same source and target sets. In the fixed-order GLM-5.2 xCCL run, SAGE solves two additional operators,
sparse_flash_attentionandgmm_deq_swiglu_quant_gmm_deq. Q1 gives the per-operator results.W4 · Efficiency and reproducibility
The 10K cap applies to resident context. Appendix D.4 describes on-device kernel timing with 5 warmups and 100 timed repetitions. Appendix G gives example traces and memory record schemas. Q3 reports synthesis costs for nine matched xCCL tasks.
Q1 · Per-operator xCCL results and frozen-source control
Of the 16 GLM-5.2 xCCL tasks in Fig. 3, both methods solve nine and fail on five. SAGE alone solves the other two.
lightning_indexersparse_flash_attentionmla_prolog_v3_xcompute_and_cache_digestapply_rotary_pos_emb_partialgmm_deq_swiglu_quant_gmm_deqhc_post_xhc_postfused_gelu_sigmoidfused_scatter_update_and_compute_digestrms_norm_ropegrouped_matmul_swiglu_quant_xmoe_dist_dispatch_mul_expmoe_gating_top_k_remaprotary_pos_emb_partialdequant_swiglu_clamp_quantSAGE’s first recorded cross-task adoption was item 10121, written at round 23 of
sparse_flash_attentionand used at round 3 ofgmm_deq_swiglu_quant_gmm_deqto avoid a cast on an intermediate activation. For Value Memory, the available record establishes only that this item was absent on the GMM task.A run with the benchmark-end bank frozen would estimate the contribution of xCCL updates when paired with the online run. Comparing that frozen bank with the frozen initial bank on the same tasks would estimate the contribution of the source experience.
Q2 · What was rerun, and what could the ranker see?
In each new run, the bank and utility replay at operator t used only records from earlier operators. The reused ATU row comes from the online run, where each task’s record becomes available after that task is scored. We will identify ATU as the online reference in Table 7.
Q3 · What cost and run variation can be reported now?
The cost records cover nine matched GLM-5.2 xCCL tasks, including eight shared successes and one shared failure (
rotary_pos_emb_partial).On these tasks, SAGE used 8.0% fewer estimated prompt tokens and 6.8% fewer model calls. The prompt estimates count cached prefixes incrementally within each task and include SAGE’s resident context, which held 818 tokens at the median and 832 at peak. The elapsed totals sum the agent-side intervals from each task’s first prompt to its logged verdict or round limit. Device stages have separate timing.
The principal 88-task stream uses one fixed-order run per configuration. Table 2 varies the curriculum on 18 Level-3 targets, with 7/18, 11/18 and 14/18 solved under L3-only, mixed L2+L3 and L2→L3, respectively.