acceptodds
Under review as a conference paper at ICLR 2027

CARTA: Training-Free LLM-Driven Tensor Mapping for Spatial Accelerators

Abstract

Tensor mapping coordinates parallel execution with data reuse across an accelerator's memory hierarchy. Soter's per-task RL Tuner relearns it for each layer-accelerator pair. In CARTA, a frozen LLM given a short strategy text picks spatial allocations, loop priorities, and tile caps for Soter's deterministic mapping builder, and we ask what transfers across accelerators. On identical problems with an energy-delay product (EDP) objective, the Tuner has 1.85x CARTA's EDP at batch 16 across three independent runs with DeepSeek as the LLM and 1.61x with pinned open-weight Qwen3-14B. However, random search beats this weak incumbent on the main suite by 1.66x at 500 evaluator calls. At 200 evaluator calls, random search has 1.21x DeepSeek CARTA's EDP, while a Gaussian-process (GP) surrogate with a structured encoding and an LLM-free annealer seeded with the same text each have 1.06x. These margins lose support under workload resampling, reach parity at 500 calls, and reverse with Qwen3-14B, which both controls beat (0.92x). Porting changes about 30 lines of strategy text without retraining, but writing them costs at least 3,000 diagnostic evaluator calls per port plus expert reading. Without the text, DeepSeek loses to the Tuner on both held-out ports. With it, Tuner/CARTA EDP is 1.13x on GemminiLike and 1.02x on NVDLALike, a tie, and the annealer has 1.03-1.05x DeepSeek's EDP. Qwen3-14B fails the port prediction registered before the port's agent runs, with 1.45x the annealer's EDP on NVDLALike. The strategy text carries the port, and the model determines the increment, 1.03-1.06x for DeepSeek and negative for Qwen3-14B. We also measure how budget accounting, baseline alignment, and run selection inflate an LLM optimizer's apparent advantage.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
tensor mapping, large language models, spatial accelerators, training-free optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_59g2OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

CARTA proposes a "training-free" LLM agent to map tensor programs onto spatial accelerators, aiming to replace per-task RL tuning (like Soter). The LLM uses a human-authored strategy text to propose spatial allocations, loop priorities, and tile caps. These proposals are snapped to valid hardware constraints by a deterministic builder and evaluated analytically via Timeloop to minimize the EDP. The authors rigorously evaluate this against RL and heuristic baselines across 90 layers on three modeled architectures. They find that while the LLM beats the RL baseline, the real heavy lifting is done by the human-written strategy text, with the LLM adding only a marginal improvement over a non-LLM annealer given the exact same text.

Critical strengths
  • They systematically deconstruct why LLM wins by introducing controls like the strategy-seeded annealer. This effectively proves that the human-authored text is doing the heavy lifting, serving as a much-needed reality check on the current hype around LLMs natively optimizing spatial architectures.

  • The paper convincingly demonstrates that the Soter RL Tuner is an incredibly weak baseline. By showing that an unconstrained random search at 500 evaluations easily beats the Tuner, the authors provide a valuable recalibration for the community regarding how we evaluate mapping frameworks.

Critical weaknesses
  • The framework relies entirely on Timeloop's analytical scores, assuming perfectly fluid spatial fanouts that ignore physical routing constraints. Treating a 64-wide spatial fanout as a frictionless allocation ignores the routing congestion and multiplexing overhead inherent in driving an interconnect or NoC. Furthermore, the claim of search efficiency relies entirely on minimizing evaluator calls. In wall-clock time, the LLM API latency makes this approach drastically slower than local heuristic searches, which can execute roughly 2,000 evaluations in the same timeframe and reach parity.

  • The paper framed in the introduction as a novel LLM framework, but the actual narrative is that LLMs are mostly just parsing human intuition and barely beat traditional search heuristics. This narrative is buried under an overwhelming details and the writing makes it exhausting to extract the core points. For example, the text unleashes a barrage of unanchored ratios (1.85x, 1.61x, 1.66x, 1.21x, 1.06x, 0.92x) across different evaluation budgets (200 calls, 320 calls, 500 calls) in the very first paragraph. Also, the paper switches between differently constrained baselines, comparing the LLM to the "fair Tuner," "latency-directed Tuner," "unported agent," "ported agent," and "strategy-seeded annealer". This forces the reader to memorize highly specific terms to understand the contents.

Questions for authors
  • Why is the paper framed as an LLM systems paper when the ablation data strongly supports an anti-LLM, pro-heuristic-with-priors conclusion?
  • Can you provide a wall-clock time vs EDP Pareto frontier figure in the main text, plotting the LLM against the random/annealer baselines?
  • Have you tested these LLM-proposed mappings on cycle-accurate simulators to see if the spatial allocations hold up against actual hardware that includes non-ideal assumptions such as interconnect routing congestion?
1 comment

Thank you for the careful review and the clear reading of our results. We agree with it, and the revision adopts it: we priced the on-chip network, added the wall-clock Pareto figure to the main text, reframed and rewrote the paper as a measurement study, and counted which mappings fit real hardware; new content is marked in blue in the revised PDF. Throughout, a ratio is the other method's EDP divided by CARTA's, so above 1 means CARTA is better; brackets are 95% intervals.

1. Network realism (W1)

Once the on-chip network is priced, simple search ties or beats CARTA. On Simba, the 64-wide level you mention is the multiply lanes inside each processing element, not a routed network; the other wide levels are real networks, and our Timeloop build charges every method nothing for them (Sec. 4). In a pre-registered experiment we charged wire energy, reduction adders and link bandwidth to first order and re-scored every method's best mapping on all 270 cases (intervals in App. F.5):

Control vs. CARTANetwork freeNetwork pricedVerdict once priced
Same-text annealer, 200 calls1.0581.007 [0.959, 1.055]tie
GP, 200 calls1.0690.968 [0.909, 1.031]tie
Same-text annealer, 500 calls0.9870.928 [0.887, 0.969]annealer better

The paper now states every CARTA lead over these controls as holding only for a free network. Two caveats: this is a first-order model and does not capture congestion; and the re-run controls matched only 94.9-97.9% of stored values against our 99% bar, so by our own rule they are flagged not reproducible, although their ratios agree with the stored ones within 0.23% (Sec. 5.2, App. F.5).

2. Wall-clock time and the Pareto figure (W1, Q2)

Added: Figure 9 in the main text plots EDP against wall-clock time per layer for CARTA, random search, SA, the same-text annealer and the GP. CARTA has no established wall-clock lead over the annealer: at equal time the annealer has 1.030× CARTA's EDP at the mean call cost and ties at the median-layer cost (0.996; post hoc, modelled). An evaluator call takes a median 0.34-0.56 s on our host, not the 0.1 s the submission assumed. You were also right about parallel search: with 4 parallel evaluators random search is within the margin of error, and with 16 it reaches parity (Sec. 5.5, App. F.7).

3. Framing and writing (W2, Q1)

The paper should not have been framed as an LLM systems paper, and the revision changes that. It is now a measurement study, retitled "CARTA: The Strategy Text, Not the LLM, Carries Most of the Gain in Tensor Mapping". The abstract opens with this finding, and CARTA is described as a measurement instrument, not a new method. Our pre-registered tests of what the LLM adds are inconclusive on two current models (Sec. 5.3). The writing is rewritten: three ratios in the abstract, one name per concept, no internal codes in the main text, and a table of five questions and answers at the start of Sec. 5.

One correction on "human-authored text": the two port texts were drafted by an LLM coding assistant in one call each, in sessions that had seen some test results (so the port results may be optimistic, though CARTA and the annealer share the same text). A pre-registered test of DeepSeek writing the text itself did not pass its main test: its text was worse than the assistant-drafted one (Sec. 5.4).

4. Cycle-accurate simulation (Q3)

Not run; we claim no cycle-level fidelity. Besides the network pricing, we counted mappings that fit published designs. On GemminiLike almost none fit Gemmini's fixed dataflow (CARTA 0/60, Tuner 0/30 per seed), because our model relaxes it for all methods. On Simba, the annealer and the GP fit about as many as CARTA (53-57 against 48-60 of 90 per run; same 95–98% re-run match note; App. F.6). For the camera-ready we commit to RTL simulation of every method's best mappings on 10 GemminiLike layers. This is planned for the camera-ready, not current evidence.

Thank you again for your time and feedback. We are happy to answer any further questions.

Reviewer_6wTfOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

CARTA replaces Soter’s per-task reinforcement-learning proposer with a LLM as a backend that generates compact tensor-mapping decisions for an existing builder. Experiments study several aspects, such as strategy-text transfer, and evaluation biases. The results show that supplied strategy text accounts for most gains, while the LLM’s additional benefit is small and model-dependent.

Critical strengths
  • The paper investigates an important question: whether an LLM-based optimizer contributes useful knowledge to tensor mapping beyond the supplied domain knowledge.
  • The experimental controls are rigorous and unusually honest.
  • The evaluation reflects substantial engineering effort, with multiple runs, multiple model backends, and workload-level uncertainty reported.
Critical weaknesses
  • The paper’s presentation is difficult to follow.
  • The technical contribution is limited.
  • The paper’s own evidence undermines its claimed methodological contribution.
Questions for authors
  • The abstract and introduction lack a clear logical flow and are hard to follow. The abstract lists many numbers without making clear what the reader should conclude from them or how they support the paper’s claims. The contribution list is internally contradictory: bullet 2 claims that CARTA “beats the fair Tuner,” while bullets 1 and 4 effectively show that this comparison is not very informative, because the Tuner is a weak baseline and most of the gain comes from the supplied strategy text. The reader is left to reconstruct the paper’s actual message from scattered statements across roughly 50 pages.
  • The evaluation reads more like an experimental log than a self-contained paper. The same concept is variously called “addendum,” “strategy text,” “prior,” and “recommended strategies,” and runs are labeled “canonical,” “traced,” “pooled,” “September,” “seed 1/2,” “same-day,” “stored,” and “first-completed-attempt.”
  • The method section does not explain the design clearly or thoroughly. Section 3 introduces exploration, refinement, and polishing, but does not explain the rationale for each stage, the design motivations behind them, or the challenges they are intended to address.
  • The key experiment that could have demonstrated that the LLM itself matters was never conducted. The method is not novel: the “LLM proposes, evaluator scores, repeat” loop is a standard propose–evaluate–repeat pattern, and the legality-fixing builder is inherited from Soter. More importantly, the paper’s own controls show that the apparent gain disappears under proper comparison. A simple annealer seeded with the same strategy text—without any LLM—matches CARTA within 6% after 200 calls and outperforms it after 500 calls. A structured GP performs better on TensorCore, and Qwen3-14B underperforms the annealer on NVDLALike.
1 comment

Thank you for the careful review, and for judging the controls rigorous and unusually honest. You were right on the main points, and the revision follows them: we ran the key experiment you asked for on two models, rewrote the main text around one message, and removed the contradictory method claim; new content is marked in blue in the revised PDF. Throughout, a ratio is the other method's EDP divided by CARTA's, so above 1 means CARTA is better; brackets are 95% intervals.

1. The paper's message and the contradictory contribution list (W1, Q1)

The paper now has one message: given the same strategy text, an LLM adds no established gain over plain search. The "beats the fair Tuner" contribution is deleted. The abstract opens with the question and its answer and keeps three ratios. The introduction follows one example (GemminiLike) to the question, states one key observation, and lists three consistent contributions (Sec. 1).

2. The key experiment (Q4)

We ran it, pre-registered, on two current DeepSeek models: start from the LLM's first 18 calls, then anneal ("LLM-first search"), against the annealer given the same text.

Annealer EDP / LLM-first EDPRatioVerdict
deepseek-flash, 200 calls1.031 [0.996, 1.063]inconclusive
deepseek-v4-pro, 200 calls1.027 [0.994, 1.060]inconclusive
deepseek-flash, 500 calls0.997parity

Pooling both models (post hoc), the 95% interval rules out a gain above about 6% (8% by workloads) (Sec. 5.3; App. F.8, F.9). The submission called this experiment untested; in fact an earlier run on a now-retired model existed, and Sec. 5.3 now reports it (post hoc).

3. The paper's own controls undermine the claimed gain (W3)

Agreed; these readings are now the paper's stated result (Secs. 5.2-5.3):

Control vs. CARTARatioReading
Same-text annealer, 200 calls1.06tie by workloads
Same-text annealer, 500 calls0.986parity
Same-text annealer, network cost priced, 500 calls0.928annealer better (re-run matched 95-98% of stored values, below our 99% bar)
GP on TensorCore0.93GP better
Same-text annealer vs. Qwen3-14B CARTA0.92annealer better

4. The evaluation reads like an experiment log (Q2)

The main text is rewritten. Each concept has one name ("strategy text", "the three runs", "the same-text annealer", "LLM-first search"), and no run or experiment codes appear in Secs. 1-6. Sec. 5 opens with a table of its five questions and answers, and Sec. 5.3 puts the LLM-first tests in a small table. The appendix keeps short codes, explained in a glossary (App. F.10).

5. Why three search stages (Q3)

Sec. 3.2 now explains each stage with a small table and a four-step worked example. Exploration chooses the spatial split, where a wrong choice is costly; refinement tunes loop priorities and tile caps of the best few candidates; polishing makes small local changes. Measured afterwards, refinement improves 35-43% of cases by more than 1%, and polishing 11-15%.

6. The technical contribution is limited (W2)

The loop and the builder are not new, and the paper says so. The contribution is the measurement: (i) how much of the gain comes from the text and how much from the model (from −8% with Qwen3-14B to +6% with DeepSeek, the +6% not established); (ii) how weak the RL Tuner is (random search with 500 calls also beats it, 1.66×) and how much common evaluation choices inflate CARTA's advantage (Table 2). Practical rule: before crediting an LLM optimizer, compare it with search given the same knowledge, at equal evaluator calls and wall-clock.

Length: Apps. A-E now take 20 pages (38 in the submission); the new App. F adds 16 (49 pages in all). A correction: the two port texts were drafted by an LLM coding assistant in one call each, not hand-written, in sessions that had seen some test results (so the port results may be optimistic, though CARTA and the annealer share the same text).

Thank you again for your time and feedback. We are happy to answer any further questions.

Reviewer_C1sEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces CARTA, a training-free LLM-based framework for tensor mapping on spatial accelerators. CARTA uses a frozen LLM to propose mapping configurations, which are validated and completed by a deterministic mapping builder. Experiments across multiple accelerators and workloads show that CARTA outperforms Soter's RL-based tuner in EDP while requiring fewer evaluator calls.

Critical strengths
  1. CARTA is evaluated extensively across multiple accelerators, DNN workloads, and LLM backends.
  2. Evaluation against baselines is also comprehensive and includes a wide range of baselines, including random search, simulated annealing, and genetic algorithms.
Critical weaknesses
  1. The portability experiments rely on manually constructed strategy texts derived from random-search winners on development layers. The improvements primarily demonstrate the transfer of explicitly supplied domain knowledge rather than the LLM's ability to independently discover effective mapping strategies.
  2. The strategy-seeded annealer, given the same domain knowledge, performs within 3–6% of DeepSeek CARTA, and structured search reaches parity at 500 evaluator calls. Furthermore, this advantage does not consistently transfer across LLM backends. With Qwen3-14B, the comparison reverses: both structured GP and the annealer beat CARTA.
  3. CARTA requires fewer evaluator calls than the RL Tuner, this does not translate into demonstrated wall-clock improvements. DeepSeek CARTA takes approximately 279 seconds per layer, compared with 99.6 seconds for the traced latency-directed Tuner, although these timings are not fully comparable. Moreover, random search reaches parity with pooled CARTA at 2,000 evaluations, which could be affordable within CARTA's execution time when evaluation is inexpensive. This raises questions about the practical benefit of repeated LLM inference.
  4. The paper is very hard to follow. The abstract alone packs roughly a dozen ratios into a few sentences, and method, main results, alternate approaches, and limitations are interleaved throughout. A clearer separation of these, together with a single end-to-end worked example showing a strategy text, the LLM’s proposed decisions, the builder’s completed mapping, and the resulting EDP, would greatly improve readability.
Questions for authors
  1. Can the translation of random-search winners into strategy text be automated, for example by having an LLM or a simple rule extractor summarize the winners spatial allocations, loop priorities, and tile caps? How many expert hours did each port require, and how sensitive are results to the text?
  2. Given that strategy-seeded annealing achieves nearly identical performance, what specific capability does the LLM contribute beyond applying the supplied strategies?
  3. Could the authors report CARTA vs the best conventional search as a function of evaluator latency, identifying the crossover cost at which repeated LLM calls become worthwhile?
  4. When the 3,000 diagnostic evaluations, the roughly 2,500 validation calls on GemminiLike, and the expert effort are counted, how many subsequent mapping tasks on a given accelerator are needed to amortize the porting cost relative to strategy-seeded annealing or structured search?
  5. Since the alternative approaches are also training-free, could the authors comment on what advantage the “training-free” framing is meant to convey beyond the comparison with the RL Tuner?
  6. Since GemminiLike approximates the Gemmini architecture, could the authors comment on whether CARTA’s mappings could be validated using RTL simulation (e.g., Verilator or FireSim)? To what extent do they expect the performance gains observed in Timeloop to translate to cycle-accurate execution?
1 comment

Thank you for the careful review and the six concrete questions. For your review we ran the key test of what the LLM adds (on two models), tested whether the strategy text can be written automatically, measured wall-clock cost and the crossover point, priced the on-chip network, and rewrote the main text; new content is marked in blue in the revised PDF. Throughout, a ratio is the other method's EDP divided by CARTA's, so above 1 means CARTA is better; brackets are 95% intervals.

1. What the LLM adds beyond the strategy text (W2, Q2)

Given the same strategy text, an LLM-free annealer comes within 1.06× of CARTA at 200 calls and ties it at 500; we could not establish any extra gain from the LLM. The LLM's role is to read the free-form text and choose each layer's spatial split, loop order and tile caps; the annealer needs the text turned into templates first. Our pre-registered test starts from the LLM's first 18 proposals and then anneals ("LLM-first search"):

Annealer EDP / LLM-side EDP, 200 callsRatioVerdict
LLM-first, deepseek-flash (pre-registered)1.031 [0.996, 1.063]inconclusive
LLM-first, deepseek-v4-pro (pre-registered)1.027 [0.994, 1.060]inconclusive
Both pooled (post hoc)the 95% interval rules out a gain above about 6% (8% by workloads)no gain established

In practice, with the strategy text in hand, run the LLM-free annealer (Sec. 5.3; App. F.3, F.8, F.9).

2. The ports transfer supplied knowledge (W1)

Agreed, and the revised paper now says so (Sec. 5.4). Without the text, CARTA loses to the Tuner on both new accelerators; with it, the annealer given the same text is within 1.05× of DeepSeek CARTA. A correction: the two port texts were drafted by an LLM coding assistant in one call each, not hand-written, in sessions that had seen some test results (so the port results may be optimistic, though CARTA and the annealer share the same text).

3. Can the text be written automatically? (Q1)

Partly. A text that DeepSeek wrote from development data alone helps, but less than the assistant-drafted one; a simple script does about as well overall.

Text used with CARTA (two new accelerators)Outcome
DeepSeek's text vs. no textDeepSeek's text better (1.378×)
DeepSeek's text vs. assistant-drafted (pre-registered primary)worse: the assistant's text has 0.938× its EDP; equal on GemminiLike
Script-built rule text vs. assistant-drafted (post hoc)within 5% overall (0.991); the assistant's text is better on GemminiLike (1.028), NVDLALike inconclusive

An automatic text costs one DeepSeek call (7-8 s). Human time was not measured (Sec. 5.4; Apps. F.4, F.8).

4. Wall-clock time and the crossover cost (W3, Q3)

At the evaluator cost we measured, CARTA has no established wall-clock lead over the annealer (Figure 9, Sec. 5.5). CARTA would lead only if one evaluator call cost more than 0.632 s; our median call costs 0.34-0.56 s. At equal wall-clock the annealer has 1.030× CARTA's EDP (post hoc, modelled; App. F.7). Run one call at a time, random search gets about 295 calls in CARTA's time, not 2,000; with 4 parallel evaluators its workload-level interval includes parity (Sec. 5.5).

5. When does a port pay for itself? (Q4)

Counting evaluator calls, after 9 layers against random search (16 with GemminiLike's 2,500 validation calls) and 81 layers against the GP on GemminiLike (149 with the validation calls), but never against the same-text annealer or the GP on NVDLALike. Human and assistant time are not counted (App. F.8, post hoc).

6. "Training-free" (Q5)

We dropped it from the title. It now only contrasts CARTA with per-task trained methods such as the Tuner; search needs no training either (Sec. 1).

7. RTL and cycle-accurate checks (Q6)

We did not run RTL simulation. Two partial checks: with a first-order network cost (pre-registered), the annealer ties CARTA at 200 calls and leads at 500 (a re-run matched 94.9-97.9% of stored values exactly, below our 99% bar, so we flag it as not reproducible, though ratios agree within 0.23%; Sec. 5.2); and almost no method's mappings fit Gemmini's fixed dataflow (CARTA 0/60), because our model relaxes it for all methods (App. F.6). For the camera-ready we commit to RTL simulation of every method's best mappings on 10 GemminiLike layers. This is planned for the camera-ready, not current evidence.

8. Readability (W4)

The main text is rewritten: the abstract keeps three ratios, each concept has one name, and Sec. 5 opens with a table of its five questions and answers. Sec. 3.2 gives the worked example you asked for in four steps: strategy text, LLM decision, builder's mapping, EDP.

Thank you again for your time and feedback. We are happy to answer any further questions.

Comments on the paper

1 comment

We thank the AC and the reviewers. The three reviews read the same message in our results, and the revision now states it plainly: given the same strategy text, an LLM-free annealer comes within 1.06× of CARTA at 200 evaluator calls and ties it at 500, so most of the gain comes from the text, not the LLM.

What we did

  • Ran the key test the reviewers asked for, pre-registered, on two current DeepSeek models: starting from the LLM's first 18 calls and then annealing is statistically indistinguishable from the annealer alone (1.031 [0.996, 1.063]; 1.027 [0.994, 1.060]).
  • Priced the on-chip network (pre-registered) and added a wall-clock figure to the main text (Figure 9). With the network priced, the annealer ties CARTA at 200 calls and leads at 500; at equal wall-clock CARTA has no established lead.
  • Tested automatic writing of the strategy text (pre-registered): DeepSeek's own text underperformed the assistant-drafted text.
  • Retitled the paper ("CARTA: The Strategy Text, Not the LLM, Carries Most of the Gain in Tensor Mapping") and rewrote the main text: three ratios in the abstract, one name per concept, no internal codes, a five-question table opening Sec. 5, and a worked example in Sec. 3.2.

New content is marked in blue in the revised PDF. Every number the reviews cite is unchanged. Details are in each reviewer's reply and in App. F of the revised PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.