acceptodds
Under review as a conference paper at ICLR 2027

Collective Algebra Above a Closed Library: A Research-Loop Agent Finds the Rewrites a Search Misses

Abstract

Collective-communication libraries on closed accelerator stacks such as AWS Trainium expose a fixed set of primitives and provide little control over how they are scheduled. OverlayCCL searches above this library layer: a language model uses an on-device-calibrated simulator to generate, implement, and evaluate different collective-communication strategies, then refines the most promising ones. However, enumerating structural rewrites alone misses important optimizations. For example, OverlayCCL may miss cases where a value that appears to depend on the current computation is actually a constant determined by the routing pattern, allowing several communication operations to be replaced by a single one. It may also miss opportunities to replace a Python loop that repeatedly prepares data for an unavoidable collective operation with a vectorized operation that performs the same preparation in one step. We therefore introduce SorcarCCL, a research agent that extends OverlayCCL by proposing and evaluating such optimizations using the same simulator and hardware constraints. SorcarCCL compares ideas before implementation, retains validated improvements, records failures, and runs a separate adversarial pass. On the eight collective problems evaluated by OverlayCCL, SorcarCCL matches its results without regression. On 40 more complex production collective problems, we search every problem with nine seeds. Across all 40 problems, the median-seed rewrites run on 224 Trainium cores at 1.12×to 4.37×the performance of OverlayCCL’s solutions, with every problem improved by SorcarCCL. Integrated into end-to-end training of 9.7-billion-parameter GPT- and Llama-style transformers on 224 cores, SorcarCCL’s rewrites further reduce steady-state step time by 2.13×and 2.15× relative to OverlayCCL’s solutions.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Collective Communication, LLM-Guided Optimization, Distributed Training, AWS Trainium

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_y3LyOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The authors introduce introduce SorcarCCL which is a research agent that extends related work (OverlayCCL) by proposing and evaluating optimizations on a modification of overlayccl's simulator. The authors indicate that their loop finds optimizations that overlay missed. They then show that SorcarCCL improves upon OverlayCCL's performance across multiple problems, with and end-to-end training of 9B+ models improved by 2.13× and 2.15× on a per-step basis.

Critical strengths

A few strengths:

  • The authors have measured results on hardware (from the paper) "7 AWS trn1.32xlarge instances in a single placement group, with 32 NeuronCores per node and 224 ranks".
  • The authors indicate that there are five xm.*collective primitives that are available.
  • Amongst their contributions, the authors increase the fidelity of their simulator to capture pre-/post-processing time and use a single, predetermined set of operations as the calibration probe for the simulator.
  • The programs come with ablations to help make the outputs more inspectable.
Critical weaknesses

There's just a LOT of this paper which reads AI-written which makes parsing many of the things presented just a tad onerous, e.g., "This oracle, not the wording of the prompt, defines correctness" -- which to be honest, I'd never assumed that the wording of the prompt would have defined correctness so this just feels like empty padding of pages? What exactly was validated?

  • The primary weakness from my perspective is that I'm not sure I understand the takeaway here? If forced, I'd say assume that the authors seem to trade-off an adaptive search, that uses a cheap proxy to do wider initial searches, and then uses an adaptive loop to get better results. This is good, but I don't think the authors have actually shown me this? I need to put in a lot of effort to even get that insight/intuition?

  • https://pytorch.org/blog/kernelagent-hardware-guided-gpu-kernel-optimization-via-multi-agent-orchestration/ seems similar but for kernel level optimizations? How might you be able to distinguish your results from this -- beyond simply saying kernel optimizations are different from collectives? I'm not sure it's apparent to me why this might be fundamentally different?

  • I'd also like details on baselines? How might the adaptive version of Overlay compare on the other problems that you're comparing against it on. My understanding was that the adaptive search in overlay didn't bring significant marginal utility vs. the fixed search budget, and hence was not recommended. But you do have an adaptive search -- is your method helpful due to adaptivity? Due to wider search? I don't see it yet?

  • How would SorcarCCL perform on the older OverlayCCL simulator? I see that you gave overlay the newer simulator, but I don't think that the older method might be well suited to the new sim? The replication work seems to be a tad iffy too?

"The isolated-amplified bolt-on avoids this: the 8 emitted family kernels run repeat passes over fixed native payloads in a graph that is disjoint from the model graph (xm.mark_step() every inner_seg passes → one reused compiled graph whose count is independent of repeat). The model loss is then bit-identical across backends, while amplification (repeat=224) makes the family dominate the step so the overall step ratio tracks the per-call family ratio (~3×). A discarded global-scalar "sink" keeps the collectives alive against dead-code elimination without feeding the loss."

^ From the readme of the repo -- this makes it seem like the speedup is over an amplified extra set of elements? Additionally, it seems like the outputs here do not feed the loss? So I don't think the bit-identical part matters?

Questions for authors

See weaknesses?

4 comments

We thank the reviewer for the direct questions.

The takeaway. We apologize for not underscoring the key takeaways in an explicit way. The high-level takeaway is indeed a trade-off: favoring an adaptive search strategy that uses a cheaper proxy to explore a broader set of candidates initially, then iteratively refines promising candidates through an adaptive feedback loop. At a finer level, our ablations reveal 3 additional insights in the "adaptive feedback loop": (a) Pairwise judging of ideas helps reduce search costs by filtering out unpromising candidates early, as in the "noA" ablation. (b) LLM-proposed adversarial testing is essential for "saving" promising strategies with minor bugs or missing edge cases. Without it, these candidates would be rejected by the one-shot correctness gate. (c) The global ledger, which tracks validated improvements and records failure modes, is essential for avoiding redundant exploration and reducing search costs.

Relation to KernelAgent. We've added it to Related Work in the revised paper, together with the boundary distinction below and a row in the new comparison table (Table 2). The distinction is not simply the topic label of "kernels vs. collectives," but rather what the vendor boundary permits and where performance gains come from. KernelAgent authors new GPU kernels under an open programming model that provides low-level control. KernelAgent could indeed be ported to computational kernels on Trainium, as Trainium supports programmable custom kernels through the Neuron Kernel Interface (NKI). However, it cannot be trivially extended to collective communication on Trainium because the underlying collective implementations are exposed as black boxes. Nothing below the five xm.* collective APIs is writable, so authoring collective kernels on Trainium is unavailable by construction.

Our line of optimization is a combination of the surrounding computation (where we have greater control) and the sequence of collective calls (which must be treated as black boxes). Finding optimization opportunities requires both modeling black-box collective operations and learning from a faithful simulator to understand how we better pre-process/post-process our data around the calls to these black boxes.

Adaptive OverlayCCL as the baseline. Adaptivity vs. wider search. Adaptivity and wider search are not the sole differentiator. Removing selection and iteration from SorcarCCL (in our "NoB" ablation) still shows 33/40 wins over OverlayCCL. Pairwise judging of ideas, LLM-proposed adversarial testing, and the global ledger are all crucial parts of the loop.

SorcarCCL on the older OverlayCCL simulator; fairness of giving OverlayCCL the newer one. Note that our extension only adds new cost terms. It does not change the terms inherited from OverlayCCL's released Phase-1 simulator. OverlayCCL's candidates receive the same scores as before in the new simulator. Using both (old and new) versions of the simulator, SorcarCCL reproduces OverlayCCL's emitted code (logically equivalent) on Overlay's own eight problems (Section 4.1), providing a replication check.

The amplified e2e setup and problem that outputs do not feed the loss".

We did perform an experiment where the collective outputs directly into the model's computation and loss. Putting the emitted collective inside the transformer layer (as part of the model's own tensor-parallel communication) does preserve the about 2× performance ratio (as documented in the artifact README in the /e2e folder). However, it breaks bit-identical loss parity: when Sorcar and Overlay collectives are integrated into the same compiled model execution, the Neuron compiler changes the execution order of the model's own BF16 reductions. Because floating-point addition is sensitive to operation order, this introduces small numerical differences that accumulate over a few training steps, causing the loss to drift (~0.1% to at most 3%). This is a consequence of the non-associativity of floating point numbers, not a bug of the code generated by SorcarCCL.

We chose to report isolated-amplified bolt-on numbers to enforce loss parity, following the established approach in this line of research (e.g., MSCCL, TACCL, OverlayCCL). We claim this setup satisfies our main purpose:

  1. Validate execution under real training conditions. The emitted rewrites should compile and run in a real 224-rank training process, exercising the actual compiler, runtime, memory pressure, and EFA network. This captures effects that isolated per-problem benchmarks cannot fully account for, such as compilation overhead and hardware-level computation–communication overlap.

  2. Verify that isolated speedups survive co-location. The per-call family speedup (2.94~2.96×) matches the isolated per-call measurements, showing that the gains persist when multiple collective strategies run alongside one single training step.

Thank you for your response. Unfortunately, I still don’t quite see what my main takeaway should be. The broader writing is still too opaque. My understanding is that the workflow is to:

  1. Use a cheap simulator to explore candidate rewrites.
  2. Have the LLM compare ideas before implementing them.
  3. Test code with adversarial inputs and use failures to repair it.
  4. Keep a record of what worked and failed, so the agent can use that history.

Although there are ablations, I still have difficulty in understanding, as an example -- how I should generalize this to my own workflows. What's the most impactful single step from your paper? I don't see it? Is it that everything is so tightly coupled that we can't generalize/decompose to it's composing units?

At present, my takeaway feels like “implement a good workflow” I don't see how this translates to a methodology contribution?

Thanks for the followup! We believe we can answer “What is the single most impactful step?” directly using data already in the paper.

The single most impactful step is proposing multiple ideas at once and comparing them before implementing any of them (Stage A).

Your 4-point summary of the workflow is accurate. And we'll say that point 2 is the key takeaway. The NoA baseline represents what a practitioner would normally do. It uses the standard generate -> evaluate -> refine workflow with the same search space, proposer model, and budget as OverlayCCL. Adding Stage A improves the results on 7 more (net) problems out of 40.

This gives us a direct comparison between a reasonable default agent workflow and our method. The key difference is simple. When testing all implementations of all possible ideas is expensive (which is true in distributed training), the LLM agent should first generate several ideas and compare them against one another and against previously observed failures. Only the most promising idea is then implemented and tested. This avoids spending time and resources on weak candidates.

Regarding whether our workflow "generalizes/decomposes to it's composing units", we believe the workflow is modular, and the ablation study shows this.

We remove each stage separately and measure how much performance drops. Every version still runs from beginning to end, which shows that the stages can be separated and studied independently.

The results also show that each stage contributes measurable value:

  • Removing proposal batching and pairwise comparison costs 7 net wins.
  • Removing selection and iterative refinement costs 7 net wins.
  • Removing adversarial testing costs 6 net wins.
  • Removing the convergence check costs 5 net wins.

Importantly, NoB still outperforms OverlayCCL on 33 of the 40 problems. Removing a stage makes the workflow less effective, but does not cause it to break down. This is evidence that the stages are modular, not too tightly coupled to separate.

The stages also work well together because they address different problems. One helps avoid wasting resources on weak ideas. Another helps recover candidates that are almost correct. Others help detect failures and avoid repeating unsuccessful attempts.

For a methodology contribution, we frame it as a set of four design principles that apply beyond this particular system.

The paper is not simply about putting together a better agent workflow. It identifies four practical principles for searching when testing candidates is expensive. Each principle addresses a specific problem, and the ablation study measures what happens when we remove it.

  1. Focus on ranking candidates correctly, not predicting their exact performance. The simulator does not need to predict exact running times. It just needs to identify which candidates are better. Our simulator ablation shows this.

  2. Spend reasoning effort before spending evaluation resources. Generate multiple candidates and compare them before implementing the most promising one. Removing this step costs 7 wins under a same budget.

  3. Test and repair candidates instead of simply accepting or rejecting them. Some candidates are almost correct and can become successful after small fixes. A one-time correctness check would reject these candidates without giving them a chance to improve.

  4. Keep track of what has already worked and failed. Record validated results and known failure cases so the agent can reuse them instead of repeating the same unsuccessful experiments.

These principles are not specific to collective optimization. They can also apply to other tasks where the system being optimized is difficult to understand, a simulator or other cheap evaluation method is available, and final validation is expensive. For example, this workflow can be extended to measure compilation time on AWS Trainium. Other tasks in other hardware could also benefit from the workflow, like kernel autotuning and query rewriting.

We would also like to highlight one final observation from the ablation. On most easy problems, all versions of the workflow eventually find the same solution. The biggest benefits appear on the harder problems, where simply generating and testing one idea at a time often fails.

Thanks, I'll raise my score in response to this.

Reviewer_vfviOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposed SorcarCCL that generates fused collective operators that provide speed ups, specifically on AWS Trainium. Compared to prior baseline, OverlayCCL that enumerates all strategies on ordering and structure, this new tool is a research loop agent that finds algebraic and vectorization optimizations. Experiement shows on 40 selected production problems at 224 ranks, it beated OverlayCCL on all, with 1.46x speed up median and 1.86x geometric mean. The model end to end training latency reduced by 2.13-2.15x.

Critical strengths
  • comparison is well controlled between baseline and proposed method. the author mentioned "use the same closed stack, calibrated simulator, hardware gates, and proposer model". The comparison is valid because it isolates the effect of search method, and the hardware gains are easier to attribute than in a cross-stack comparison.

  • The paper reported the latency measured on real hardware for e2e model training over 120 steps, "The model loss stays bit-identical to the OverlayCCL backend at every step"

Critical weaknesses
  • the mismatch between rewrite and the original pairs on functionality. Appendix D says each pair "condensed to the collective structure and the code that surrounds it". In D.6, Listing 11 applies the rank-1 update L times, scaling the v direction by (1 + β) to the power L, while Listing 12 applies it once. For example, with 4 ranks, inputs (r + 1) times [1, -1], β equal to 0.4 and L equal to 2, they return [19.6, -19.6] and [14, -14]. In D.4 with β equal to 0.02, the correction in Listing 7 cancels exactly and returns the initial s, while Listing 8 returns s plus 0.02 times its sum, and D.5 collapses its loop the same way. The paper says "rewrites that drop or double-count communication fail" the oracle, so either the listings are wrong or the oracle missed real errors. These may affect the core claim of semantics-preserving.

  • the reported 40 are selected which does not reflect the full capability of baseline and the proposed method. It should also report several other cases that baseline or other method is comparable or faster (if any). Appendix A says "problems on which OverlayCCL and SorcarCCL agree are not reported here". Also, the measured number is from one warm-cache run of the median seed, so there is no variance reported and the smallest gain of 1.12x is not proven out of normal noise region.

Questions for authors
  • Please provide full math details for F4, F5, F6 and oracle results on rank-distinguishable inputs. What omitted operations or constraints make Listings 7 and 8, 9 and 10, and 11 and 12 equivalent?

  • Could you tell more about the overall program candidates instead of the 40 subset? Or is 40 all of the programs? What is the accept rate for both systems?

1 comment

We thank the reviewer for the technically engaged review.

Q1 — full math for F4, F5, F6, and oracle results on rank-distinguishable inputs. We thank the reviewer for spotting the inconsistencies between Listings 7, 9, and 11 and Listings 8, 10, and 12. We double checked and found the error is from our manually condensed code listings, not in the SorcarCCL emitted programs or the oracle. Please refer to the committed median-seed files in the artifact (results/<family>/<problem>/{overlay,sorcar}/median.py) for a proof that the actual programs are logically equivalent.

  • F6 (Listing 11. Correct code in results/F6/r67b_vself_b0p4_p2048_d8/overlay/median.py). After every all-reduce except the last, the OverlayCCL committed program executes a line our listing omitted: s = acc − (β/(1+β))·v·(v·acc).mean(). In the OverlayCCL solution, each of the first L−1 rounds is couple -> all-reduce -> decouple, so the first L-1 rounds can be dropped. We rewrote the OverlayCCL solution into a loop for conciseness, but accidentally dropped the decoupling logic.
  • F4 (Listing 7. Correct code in results/F4/r63_gsum_b0p02_p2048_d8/overlay/median.py). The committed OverlayCCL program applies the inverse correction s = acc − 0.02·acc.sum()/(1 + 0.02·D) after rounds 1..L−1. But after the final round, it simply returns acc = s + 0.02·s.sum() without the correction. So the correct program's net effect is just one application of the coupling, equal to Listing 8 (SorcarCCL's solution). When condensing the OverlayCCL solution to a loop, our listing wrongly applied the correction to every round including the last.
  • F5 (Listing 9. Correct code in results/F5/r65b_gnorm_b0p6_p2048_d8/overlay/median.py). The committed OverlayCCL program recovers the pre-clipping scale at every round using the Möbius inverse M = A/(1 − β·A). Specifically, let a = mean|s| be the pre-clipping scale. After clipping, acc = s/(1 + β·a), so A = mean|acc| = a/(1 + β·a). Inverting this relation gives M = A/(1 − β·A) = a. Notice that each intermediate round starts from the same pre-clipping value and is therefore stationary. Only the final round retains the clipping operation, producing s/(1 + β·mean|s|), exactly matching SorcarCCL's all_reduce(s / ((1 + β·mean|s|)·W)). When condensing our listings, we mistakenly omitted the Möbius-inverse recovery step, but the emitted code is equivalent.

The results above are reproducible from the artifact by running the two median.py files of each problem through harness/score_service.py (the single gate both controllers used). The reviewer can verify from the GitHub repo that we made no changes to the emitted code (i.e., they are all faithful, and the bugs are in our writeups for the paper). In the revised paper we have corrected the Appendix D listings to be operation-faithful to the committed code.

Q2 — the problem set and accept rates.

The 40-problem set was constructed by extracting collective-communication-related code segments from AWS production code. We filtered for code segments containing more than one collective communication call (xm.*) and non-trivial computation either between or within these calls. We then parameterized the resulting problems by block count, loop depth, payload size, and coupling width, as reflected in log names such as r60d_b16_L5_p2048_d8. We evaluated every instance using nine independent search seeds per controller. The "problems on which OverlayCCL and SorcarCCL agree" are the 8 problems from OverlayCCL's evaluation set. We did not list the emitted code for these problems in the appendix, as both OverlayCCL and SorcarCCL converged to logically equivalent implementations, resulting in a tie.

Acceptance rate through the correctness gate is essentially saturated for both systems. Our committed artifact records 360/360 accepted candidates for both OverlayCCL and SorcarCCL across 40 problems * 9 seeds, so acceptance does not differentiate them.

Single warm-cache run and the 1.12× tail. We would like to emphasize three points: (1) All 40 problems (SorcarCCL vs OverlayCCL) show a one-sided Mann–Whitney U p-value below 0.05 and a bootstrap 95% lower bound above 1.0, which shows statistical significance. (2) For the smallest confirmed ratios, the family structure is itself a replication: the 1.12× F5 instance sits among ten F5 siblings (whose rewrite strategies are conceptually similar) at 1.12–1.47×, and the 1.12× F7 instance among five at 1.12–1.61×. (3) Run-to-run stability of the warm-cache protocol is also checkable in the public e2e logs in the Github repo, where two fully independent runs of the same backend comparison (25-step and 120-step) reproduce the step ratio within 1.5% (2.139 vs. 2.125 and 2.171 vs. 2.147).

Reviewer_4nHNOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Optimizing communication during distributed training when the collective implementations are not accessible as in closed accelerator stacks. These applications can be optimized by changing the code around the collectives. The authors propose to use an agentic framework to do these rewrites automatically.

Critical strengths
  • Tackles a well scoped out problem in optimizing in the context of collective communication libraries in closed accelerator stacks.
  • The demonstrated results are quite strong (2x speedups)
Critical weaknesses

My major concern with this work is that the research contributions are not very clearly substantiated.

First, given that prior works like OverlayCCL exist, the key contribution of this work really is in how to use LLMs to do rewrites. It would be more of a contribution if the authors discussed in more detailed scenarios where this is needed and why LLMs are the best option for performing these edits. I was not able to fully understand scenarios where code rewrites are needed and how many of these scenarios exist. These would be key contributions of the work.

Second, I would imagine that correctness is a critical challenge here. It was not clear from the paper how this is addressed. Many parts of the paper are difficult to read, especially this part in the appendix. The authors should give significantly more importance to how correctness is handled.

Third, I was unable to gauge significant novelty in the design of the LLM search/edit framework itself, which seemed quite straightforward. The authors simply discuss the three phase pipeline, but the rationale behind its design, the novelty in design, and the challenges that drove each choice are simply not clear. The main loop combines idea generation, implementation, feedback, a history of previous attempts, and adversarial testing. The paper should explain which challenges specific to collective optimization motivate these choices and what insight distinguishes this design from a conventional generate–evaluate–refine agent.

Overall, there needs to be more substance in the research contributions before the paper is ready for acceptance.

Questions for authors
  • How did you select the problems for evaluation?
  • Was there a specific type of rewrite that produced most of the benefits?
1 comment

We thank the reviewer for pressing us to clarify where the contribution lies. We have made this point explicit in the revised paper, as detailed below.

Q1 — How were the problems selected?

The 40-problem set was constructed by extracting collective-communication-related code segments from AWS production code. We filtered for code segments containing more than one collective communication call (xm.*) and non-trivial computation either between or within these calls. We then parameterized the resulting problems by block count, loop depth, payload size, and coupling width, as reflected in log names such as r60d_b16_L5_p2048_d8. We evaluated every instance using nine independent search seeds per controller. The remaining 8 problems we evaluated were directly drawn from OverlayCCL's evaluation set. On all eight problems, OverlayCCL and SorcarCCL converged to logically equivalent implementations, resulting in a tie.

Q2 — Which types of rewrites produced the benefits?

The "wins" come from two main mechanisms (Table 1):

  • Algebraic reduction in collective count: 33 problems (F1-F2 and F4–F7).
  • Vectorization of per-block data handling, with no change in collective count: 7 problems (F3, F8).

The largest gains come from F2 (11 problems; median 2.65×, best 4.06×, where SorcarCCL reduces the amount of collective calls), and F3 (6 problems; median 4.19×, best 4.37×, where SorcarCCL replaces a per-block Python slicing loop with device-vectorizable view(B,S) and mean(dim=1)).

Scenarios

The eight families represent recurring patterns in AWS production data-parallel and MoE training code, including weighted bucketed gradient reduction (F1), MoE routing-coverage renormalization (F2), per-block RMS/variance normalization (F3), global-scalar feedback (F4), global-norm clipping (F5), rank-1 Sherman–Morrison-style updates (F6), banded neighbor coupling (F7), and blockwise rotations (F8).

why an LLM is needed

An LLM is needed because there is no intermediate representation below the five xm.* collective calls. The optimizable code is therefore the surrounding Python/XLA program. Finding the winning rewrites requires deriving problem-specific algebraic identities. For example, the LLM can spot when a repetitive calculation simplifies itself because its data sizes don't change. It then automatically writes the correct parallel code to run this calculation across multiple processors at once. Standard compiler passes such as CSE, DCE, and XLA fusion do not reason across collectives with rank semantics. A template-based approach would need a separate template for each identity.

Correctness

We check correctness at 3 levels, from small-scale testing to the full deployment settings:

  1. Every candidate is tested on every rank at world sizes 4 and 8, using a fixed sample of admissible shapes and deterministic, rank-distinguishable inputs. Results are checked in fp32 (atol = rtol = 1e−3) and bf16; shape mismatches and NaN/Inf are rejected.
  2. An adversarial subtask generates inputs designed to expose incorrect rewrites before performance scoring.
  3. Phase 3 requires successful compilation and execution at 224 ranks, inside an 8-layer model step.
  4. End-to-end runs must preserve bit-identical loss over 120 steps.

We've included this four-level validation ladder in the revised paper (Section 3.2, "The correctness oracle").

Why this particular search loop?

Our contribution is mapping the specific failure modes of collective optimization to the search stages, and showing that each stage matters. The ablations quantify this effect (we define "net" as losses minus wins relative to the full loop, over 600 runs):

  • Incorrect algebra can appear valid on symmetric inputs. This motivates adversarial testing with rank-asymmetric inputs. Removing adversarial testing results in a regression of 6 net problems.
  • Many candidates are near ties, while validation is expensive. This motivates proposing multiple ideas and pairwise judging before implementation. Removing it costs a regression of 7 net problems.
  • Some strategies require multi-step derivation, while many initial ideas fail for simple reasons. This motivates tracking rejected ideas and iteratively refining them. Removing it costs a regression of 7 net problems.
  • The search has a fixed budget. This motivates an explicit convergence check. Removing it costs a regression of 5 net problems.

We've added this challenge-to-stage mapping to the revised paper as a dedicated "Why these stages" paragraph in the research-loop section (Section 3.2), making clear why each part of the loop is needed.

Reviewer_JmT4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes SorcarCCL, a research-loop agent that optimizes collective communication on closed accelerator platforms by discovering algebraic rewrites and local computation optimizations beyond the structural search space of OverlayCCL.

Critical strengths
  1. The paper addresses collective communication optimization under the constraints of closed accelerator libraries, which is a practically relevant problem.

  2. The paper provides real Trainium hardware evaluations, end-to-end training experiments, multi-seed search analysis, and ablation studies.

Critical weaknesses
  1. The introduction mainly relies on textual descriptions and two examples. A motivating figure comparing the original computation, existing structural optimizations, and the proposed rewrites would help clarify the task.

  2. The introduction illustrates the limitations of existing methods through two examples and identifies a broader optimization gap. However, the underlying technical challenges could be more clearly articulated. The authors should further clarify the difficulties in discovering and validating equivalent computations beyond predefined structural transformations and explain how the proposed method addresses these challenges.

  3. The paper does not clearly distinguish whether the performance gains come from the expanded search space or the proposed search strategy. A comparison with simpler LLM-based search methods under the same search space and budget would help clarify the methodological contribution.

  4. The related work could more directly compare SorcarCCL with approaches such as CoCoNet, FunSearch, and AlphaEvolve in terms of optimization space, search mechanisms, and correctness validation, thereby clarifying its distinct contributions.

  5. Figure 1 uses repeated numbering (“1” and “2”) within Phase 1, making the hierarchy difficult to follow. Consistent numbering and a simpler layout would improve readability.

Questions for authors
  1. How can the authors distinguish performance gains arising from the expanded search space from those attributable to the proposed research-loop strategy? Would a simpler LLM-based search method under the same search space and budget achieve similar results?

  2. How does SorcarCCL ensure the correctness of algebraic rewrites beyond the tested inputs and configurations, particularly when equivalence depends on assumptions such as step-invariant routing patterns?

2 comments

We thank the reviewer for the careful reading and constructive suggestions on how to improve the presentation.

Q1 — Separating the search space from the search strategy, and comparing against a simpler LLM-based search under the same space and budget.

OverlayCCL and SorcarCCL use exactly the same formal search space by construction. Both models generate arbitrary Python programs using the same five collective primitives. They are evaluated by the same simulator (we have shown in Section 3.1 that our fixed simulator preserves per-problem ranking as in OverlayCCL), correctness gate (harness/score_service.py in the artifact), proposer model (Claude Sonnet 4.5), and per-problem budget (30 steps and $5). Thus SorcarCCL's gains cannot be attributed to having a larger search space.

A comparison with a simpler LLM-based search under the same space and budget is already included in our methodology ablation (Section 4.6 and Appendix C), which evaluates five ablated variants over 600 runs while keeping the search space, model, simulator, and budgets fixed. In particular, NoA removes idea proposal and pairwise judging, making it a traditional "generate -> evaluate -> refine" agent under the same conditions. NoA finds a net 7 fewer problems than the full system (losses minus wins, counting only differences outside a +/-20% performance band). The other ablations also perform worse: NoB (no selection or iteration) by 7 problems, NoAdv by 6, and NoC by 5. Also, none of the ablated variants discovers a rewrite that the full system misses. We've made this comparison explicit in the revised paper: Section 4.6 now identifies NoA as the simpler generate->evaluate->refine LLM-search baseline under the identical space, model, and budget, and Related Work points the reader to it.

Q2 — Correctness beyond the tested inputs and configurations, including the assumption of step-invariant routing.

We check correctness at 3 levels, from small-scale testing to the full deployment settings:

  1. Every candidate is tested on every rank at world sizes 4 and 8, using a fixed sample of admissible shapes and deterministic, rank-distinguishable inputs. Results are checked in fp32 (atol = rtol = 1e−3) and bf16; shape mismatches and NaN/Inf are rejected.
  2. An adversarial subtask generates inputs designed to expose incorrect rewrites before performance scoring.
  3. Phase 3 requires successful compilation and execution at 224 ranks, inside an 8-layer model step.
  4. End-to-end runs must preserve bit-identical loss over 120 steps.

We've included this four-level validation gate in the revised paper (Section 3.2, "The correctness oracle").

On routing specifically, the rewrite relies only on properties stated in the problem specification. The specification identifies which inputs are fixed and which may vary. The oracle checks correctness by independently computing the reference output from the original formula on fresh rank-distinguishable inputs. For example, a rewrite that incorrectly treats a step-varying quantity as constant would disagree on the oracle's rank-distinguishable inputs and the candidate would simply be rejected, which is what happens to many incorrect candidates during search. Moreover, each rewrite is generated and validated for one specific problem specification and is never applied beyond that.

Motivating figure and clearer explanation of the technical challenges.

We agree that the paper needs both a motivating figure and a clearer explanation of the underlying technical challenges. We've added an introductory figure to the revised paper (new Figure 1, built on the F2 case study) comparing three computations side by side:

(a) the baseline computation;
(b) the best structural rewrite found by OverlayCCL
(c) the algebraic rewrite found by SorcarCCL, which telescopes the computation into two collectives.

Related work: CoCoNet, FunSearch, and AlphaEvolve.

We've added a concise comparison table to the revised paper (Table 2, Related Work), organized around the three dimensions highlighted by the reviewer: optimization space, search procedure, and correctness validation.

The key distinctions are:

  • CoCoNet fuses computation into communication kernels within an open software stack. It searches compiler transformations and relies on correctness by construction.
  • FunSearch and AlphaEvolve search program text using evolutionary procedures and evaluate candidates with an exact scorer. Their oracles do not model SPMD execution semantics.
  • SorcarCCL searches above a closed collective-library boundary. It uses a research-loop search procedure, a rank-level execution oracle, and training-scale hardware validation.

Thank you for the detailed response. The added related-work comparison, and clarification of the search-space controls and correctness validation address most of my concerns. However, the evidence for correctness beyond tested configurations remains limited. Based on the authors' response, I have updated my rating accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.