acceptodds
Under review as a conference paper at ICLR 2027

From Estimation to Experience: Building Reusable Deployment Knowledge for LLM Parallel Training

Abstract

Training large language models (LLMs) in data center networks (DCNs) requires exploring a large space of parallelization and resource configurations. Yet real-system observations are costly to obtain and therefore sparse, making deployment experience scarce and slow to accumulate. A natural alternative is to use performance estimators for low-cost exploration, but existing estimators typically produce scenario-specific solutions rather than reusable deployment knowledge. We therefore ask whether low-cost estimation can be converted into reusable deployment knowledge, and whether accumulated experience can in turn guide subsequent estimation. To this end, we propose Est2Exp (Estimation to Experience), a unified framework that couples a performance estimator with an experience bank through bidirectional feedback. The estimator uses a hybrid-granularity directed acyclic graph (DAG) to capture computation-communication dependencies, and LLM-assisted calibration with limited real-system measurements to refine latency functions, including irregular behaviors that are difficult to capture with conventional physical models. The experience bank distills estimator outputs into reusable deployment knowledge, which is reused to warm-start subsequent exploration and jointly refined with the estimator as new real-system measurements become available. Experimental results show that Est2Exp achieves 99.3-99.9% DAG-level latency accuracy across evaluated configurations with limited real-system measurements. Reusing accumulated deployment experience further improves cross-scale search success from 73.3% to 98.3% while substantially reducing search overhead.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Large Language Models, Parallel Training, Performance Modeling, Deployment Knowledge, Experience Bank

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.20 over 5 reviews, above 96% of the others.
Reviewer_nDJNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to accumulate and reuse deployment experience to improve the efficiency of searching for LLM parallelization configurations when real training measurements are scarce. The authors propose Est2Exp, which combines a performance estimator with an experience bank. It first calibrates a mixed-granularity DAG estimator using limited real profiling data, then distills estimated results into experience entries with applicability conditions and supporting evidence through controlled single-factor comparisons and FunSearch-based interpretable policy search. The resulting experience constrains the search space and initializes subsequent searches, while additional measurements resolve conflicts between the estimator and stored experience.

The main contribution is a bidirectional feedback framework between the estimator and the experience bank, together with mechanisms for latency calibration, experience extraction, and refinement. The evaluation uses real training traces from Ascend clusters to validate estimation accuracy and a heterogeneous simulation environment to construct the experience bank and evaluate search transfer from 128 to 256 devices. The authors report DAG latency prediction accuracy of 99.3%–99.9%. Experience-based warm starts increase search success from 73.3% to 98.3% and reduce candidate evaluations from 12 to 4 among successful runs.

Critical strengths
  1. Combining low-cost evaluation with experience reuse across tasks has practical value. Experience entries with applicability conditions and supporting evidence provide a more structured way to organize deployment knowledge than storing only a best configuration. Measurement-driven conflict resolution also accounts for the possibility that experience may become invalid.

  2. The estimator is validated using real large-scale training traces. The evaluation covers models with 30B–505B parameters and clusters of 128–16K NPUs, including extrapolation to large-scale settings not used for calibration. This provides valuable empirical support for the estimator.

Critical weaknesses
  1. The added value of experience extraction over ordinary warm starts has not been established. Table 4 compares only cold starts with policy warm starts, without a baseline that directly reuses previously identified high-performing configurations. It therefore does not distinguish the benefit of initialization with historical information from that of the proposed experience extraction and policy representation.

  2. The quality of the selected configurations has not been validated on real deployments. The latency extrapolation in Section 6.2 and conflict resolution in Section 6.4 have measurement support, but the ranking of multiple candidate configurations within the same new scenario remains unverified. Experience generation and search evaluation depend on the same estimator, so shared biases may prevent simulation-based search success from translating into real deployment gains.

Questions for authors
  1. Regarding Weakness 1: Under the same search budget, how much additional benefit does policy-based experience provide over warm-starting directly from previously identified high-performing configurations?

  2. Regarding Weakness 2: Could you measure multiple candidate configurations in the same new deployment scenario to validate the estimator's ranking and the quality of the configuration selected by experience-guided search?

6 comments
AuthorsOpenReview ↗

Response to Reviewer nDJN (Part 1/2)

W1 & Q1: Added Value of Policy-Based Experience Extraction and Reuse

Thank you for this insightful suggestion. We agree that reusing a previously identified high-performing configuration is an important baseline for assessing the value of extracting reusable policies. In the revised manuscript, we compare these two forms of reuse by examining the quality of their initial configurations and how quickly each search improves them.

(1) Evaluation metrics. We use two metrics, defined in Section 6.1 (Metrics, page 8) of the revised manuscript.

  • Optimality Gap (OG) measures the relative latency difference between the best parallelism configuration identified through experience-guided search and the optimal configuration. Lower OG indicates better configuration quality.

  • Search Iterations measures search efficiency by recording the number of iterations required to first identify the reference-optimal configuration.

(2) Experimental setup. We evaluate 100 heterogeneous 256-GPU deployment scenarios using three random seeds per scenario. The new Figure 3 (Section 6.3, page 9) compares the following four settings .

  • Cold-start policy-based search (blue) learns policies from scratch at the 256-GPU scale and uses the resulting policies to select a configuration for each scenario.
  • Experience-guided policy-based search (orange) starts from policies learned at the 128-GPU scale, evolves them at the 256-GPU scale, and uses the resulting policies to select a configuration for each scenario.
  • Experience-guided simulated annealing (green) starts from a configuration derived from the 128-GPU experience bank and searches for a better configuration in the target scenario.
  • Cold-start simulated annealing (purple) starts from a randomly selected feasible configuration.

(3) Results and implications. Figure 3 tracks the optimality gap as the iteration budget increases, with each curve reaching 0% once all 300 runs for that setting have found the reference optimum (100 scenarios with three random seeds each). Table 5 (page 10) reports the maximum and average numbers of iterations that runs needed to first reach a 0% gap. Since an iteration involves different operations in the two search methods, we compare each experience-guided variant with its own cold-start baseline.

Policy-based search. In Figure 3, the experience-guided variant (orange) starts with a smaller optimality gap and reaches 0% sooner than the cold-start variant (blue). As reported in Table 5, experience reuse reduces the maximum iteration count from 19 to 10 (about 47% fewer) and the average from 7.0 to 1.7 (about 76% fewer).

Simulated annealing. The experience-guided variant (green) reaches small optimality gaps sooner than the cold-start variant (purple). A few runs take longer to reach 0%, extending the tail of the green curve. The average iteration count drops from 14.3 to 2.9, a reduction of 11.4 iterations (about 80%).

Both methods reach the reference optimum in fewer iterations on average when initialized from prior experience. Policy-based search starts from a previously learned scoring policy, and simulated annealing starts from a configuration derived from the experience bank.

Compared with simulated annealing, which returns the best configuration it finds for the target scenario, Est2Exp’s policy-based search yields both a configuration and a reusable scoring rule. Est2Exp first distills deployment experience from the scoring rule, then stores the resulting findings together with their applicability conditions in the experience bank. The stored experience can guide candidate ranking in similar scenarios and warm-start the search for new policies. The advantage of Est2Exp lies in automatically turning the results of individual searches into reusable deployment knowledge, reducing the need to rediscover effective strategies across scenarios.

AuthorsOpenReview ↗

Response to Reviewer nDJN (Part 2/2)

W2 / Q2. Hardware validation.

Thank you for raising this point. We agree that measuring multiple configurations under identical conditions in a new scenario would provide an independent check of both the estimator’s ranking and the quality of the configuration selected by experience-guided search.

Our estimator-based evaluation covers 100 heterogeneous 256-GPU scenarios. Even a focused follow-up on 256 NPUs would require coordinating cluster access with our industry partner, reproducing and verifying the device slowdowns and network bandwidth limits, and testing several candidate configurations under identical conditions in each scenario. This setup and measurement effort requires more time than the rebuttal window allows.

For follow-up validation, we plan to test the configuration recommended using transferred experience alongside several strong alternatives in a few representative new scenarios. We will compare the hardware results for memory feasibility, training-step latency, and configuration ranking with the deployment findings distilled from the estimator’s outputs. If discrepancies arise, we can use these measurements in the existing Loop 3 (Figure 1) to determine whether to recalibrate the estimator or refine the stored guidance.

These experiments would extend the hardware validation of the existing Est2Exp architecture. The core contribution is the automated workflow already implemented in Est2Exp. It distills deployment findings from estimator outputs, and reuses them to guide configuration search in new scenarios. This gives engineers a practical way to reuse deployment knowledge across settings, start new configuration searches from prior findings, and reduce the need for costly hardware trials.

Thank you for the additional comparisons and clarification. The results provide further evidence for the benefits of warm starts, but the added value of policy extraction over direct reuse of strong historical configurations under a matched budget remains unclear.

Reviewer_c3UPOpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper considers the problem of developing a cost model for LLM training in a large-scale data center network. There are two core components of the Est2Exp system. First is the estimator: “It provides low-cost quantitative results for comparing candidate configurations and exploring deployment experience” and the experience bank: “The experience bank distills estimator results into reusable deployment knowledge.” There are three loops: one loop to update the estimator, one to generate experiences (estimator outputs) and one to handle conflicts between real-life system measurements and existing experiences.

How does the estimator work? Each training workload is represented as a DAG. For data parallelism and pipeline parallelism, model-level DAGs are used. For tensor parallelism, an operator level DAG is used. However, the paper doe not say how performance estimation is performed on this DAD, other than at a very high level… for example: “End-to-end performance estimation: estimating training latency through critical-path analysis”. No math or formulas are given. Latency of each “execution unit” (though it is unclear what that is) is computed as a sum of regular and irregular communication behaviors. The irregular behaviors are described by an LLM which appears to fit the observed data to an interpretable function.

How does the experience bank work? Each experience is represented as a triple: C specifies its applicability conditions, F the deployment finding, and V its supporting evidence and validation status. The paper also mentions that experiences can be extracted in two ways. When there is a single factor, the other conditions can be fixed ad the effect of the factor measured. When many factors interact, a search mechanism for interpretable programs is used. Experiences may be consolidated.

The technical section of the paper finishes with how experiences are used and refined.

Critical strengths

This paper tackles a difficult and very well-justified problem. The authors have obtained a very impressive training set, with runs on a cluster with 16,000 AI accelerators. That is truly impressive! It appears that they have built a complex, non-trivial and very useful system. The system-level integration of a performance estimator with an LLM-driven experience bank appears to be novel.

Critical weaknesses

This may be wonderful work, but the paper itself as submitted needs a lot of work—far more than can be accomplished during a paper revision. I would suggest that the authors re-write the paper and send to the next top-tier AI venue. The research may be good enough to be publishable in a top-tier venue, but right now, I am concerned that the paper is not.

Overall, this paper reads like a design document or blog post. There are few/no technical details and it would likely be impossible to reproduce these results from the details in the paper. Take section 5.2. Statements such as “Known local relationships narrow the search space. For example…” and “Prior policy experience provides a warm start by prioritizing candidate configurations. For example...” do not provide a precise algorithm or mathematical description of the search space or how similarity between scenarios is measured. They are merely high-level English descriptions of the system that leave almost all details open to interpretation. To be fair, a technical paper cannot give all details about everything, especially with the system constructed is complex and intricate (as it is here). But there are few/no technical details in the paper. The paper relies on an LLM to discover latency equations and policy programs, but provides no system prompts, mutation operators, convergence criteria, or LLM specifications in the main paper. For the critical path analysis the paper does not state whether it uses an event-driven discrete simulator, a topological longest-path solver, or a scheduling heuristic. I could go on with various examples of how details are missing.

Even the experiments, while impressive, are very difficult to parse. Table 1 uses unexplained, internal profiling shorthand (crazy terms such as Z3 MUON G-AG DP-EP, Z3 next P-AG, All GPU AR 0/1) without defining the operations, message sizes, tensor shapes, or communication process groups, making it impossible to understand the exact benchmark workloads.

It is very difficult or impossible to judge the quality of the experiments. The paper provides no evidence of a train/test split, k-fold cross-validation, or held-out training steps/batches. I am concerned that the parameters are fit to the 30B-128 dataset and then in the results shown in Table 1, evaluated on that exact same dataset.

All of this needs to be fixed, and if it is, the authors could have a wonderful paper, but this is a complete re-write.

Questions for authors

The authors could address my questions regarding the experiments, but my concerns about the current state of the writing and lack of technical details in the writing would remain.

5 comments
AuthorsOpenReview ↗

Response to Reviewer c3UP (Part 1/3)

Thank you for the careful review and for recognizing the novelty of integrating a performance estimator with an LLM-driven experience bank. Your comments helped us identify where the original manuscript needed more detail for readers to follow and reproduce the workflow. We hope the revisions and clarifications address your concerns.


1. Presentation of the system. When profiling real systems is costly, deployment experience is slow to accumulate. Our proposed framework, Est2Exp (Estimation to Experience), addresses this problem through an automated workflow that couples a performance estimator with an experience bank. The estimator lets engineers explore configurations with limited profiling data, and the experience bank turns the resulting findings into guidance for future deployments.

We develop and evaluate Est2Exp in large-scale distributed LLM training, where these profiling constraints arise in practice. Because our system-level contribution lies in how the estimator and the experience bank work together, we organize the main text around the workflow that connects them. This structure makes the design easier to follow. It also helps engineers facing similar constraints see how they might adapt the framework to their own systems. Reviewer Ecch highlighted how clearly the introduction explains the deployment problem and motivates the proposed solution, describing the paper as “refreshingly clear and well-motivated.”

We agree that pursuing this system-level presentation also calls for precise implementation details so readers can reproduce the workflow. The revised manuscript expands the explanations in Sections 4.1 and 5.2 and directs readers to the relevant appendices for the corresponding technical details.


2. Technical details in Appendices A–C. The revised manuscript adds formal definitions and detailed implementation procedures to help readers reproduce the Est2Exp workflow.

  • Appendix A: Hybrid-granularity DAG and its applications. Formally, defines the hybrid-granularity DAG (A.1) in mathematical way and provides mathematical formulations for its applications (A.2).
  • Appendix B: LLM-guided calibration. Covers identifying latency prediction errors, selecting corrections and setting fitting bounds, validating corrections, and recording evidence for reuse (B.1–B.4).
  • Appendix C: Experience bank construction and refinement. Describes solution-based experience (C.1), policy-based experience (C.2), and how conflicts are attributed and resolved through targeted updates (C.3).
AuthorsOpenReview ↗

Response to Reviewer c3UP (Part 2/3)

3. Technical Details

How prior experience guides deployment search. Solution-based experience narrows the search by applying prior findings to specific deployment decisions. For example, when the tensor parallelism (TP) degree exceeds the number of GPUs per node, some exchanges must cross nodes. Given the volume of TP communication, lower-bandwidth links can increase training-step latency. Where this finding applies, we restrict the search to placements that keep each TP group within a node when feasible. Policy-based experience can initialize the search for a new scoring policy. For example, a policy learned at 128 GPUs can seed policy evolution at 256 GPUs when key hardware parameters are comparable. Prior experience can also supply an initial configuration for heuristic search. We have expanded these explanations in Section 5.2 (Page 7).

Search space and scenario matching. The revised manuscript defines the feasible search space in Appendix C.2.1 and details the two retrieval procedures in Appendices C.1 and C.2.

We represent each candidate configuration as a tuple (PP,TP,EP,DP,MB)(PP, TP, EP, DP, MB) specifying the pipeline, tensor, expert, and data parallelism degrees and the number of microbatches. A feasible configuration must satisfy constraints on the device budget, global batch size, expert capacity, and available device memory. For example, in the 128-GPU setting, the device-budget constraint is PP×TP×DP=128PP \times TP \times DP = 128.

For solution-based retrieval, we check whether the target scenario satisfies an experience entry’s applicability conditions. When it does, we apply the entry’s finding to narrow the relevant deployment choices. See Table 8 (Page 21) for the conditions and corresponding findings.

For policy-based retrieval, we match the target scenario to a stored slow-device pattern using four descriptors. These capture the fraction of slow devices, the fraction of affected affinity groups, the maximum imbalance within an affinity group, and the highest fraction of slow devices on any single node. We derive a deployment configuration directly from the Retained Deployment Experience in the matching row. This configuration can serve as the starting point for heuristic search. See Table 10 (Page 25) for the descriptor definitions and Table 12 (Page 26) for the scenario patterns and their associated deployment experience.

Critical-path analysis. We use a topological longest-path solver. Each dependency edge carries the estimator’s predicted latency for its source execution unit, and the longest-path weight gives the predicted training-step latency. To identify bottlenecks, we perturb one unit’s predicted latency at a time and recompute the longest path. The resulting change in predicted training-step latency measures sensitivity to that unit. Appendix A.2 gives the algorithm and formulations for all three applications introduced in Section 4.1, including end-to-end latency estimation, bottleneck analysis, and computation–communication overlap analysis.

Profiling notation in Table 1. We added a note below Table 1 (Page 9) to define the profiling abbreviations.

  • Z3 denotes ZeRO-3, and MUON identifies the optimizer.
  • AR, RS, and AG denote AllReduce, ReduceScatter, and AllGather, respectively.
  • P-XX and G-XX indicate parameter and gradient communication, respectively. DP-XX identifies the data-parallel communication domain.
  • Example labels.
    • Z3 MUON G-AG DP-EP denotes a gradient AllGather in the DP-EP domain of a ZeRO-3 run using MUON.
    • Z3 next P-AG denotes the next scheduled parameter AllGather in the ZeRO-3 run.
    • All GPU AR 0/1 labels two AllReduce events across the full device group.
AuthorsOpenReview ↗

Response to Reviewer c3UP (Part 3/3)

4. Evaluation protocol and results

(1) Execution-unit and DAG-level calibration. We jointly calibrate Est2Exp using profiling measurements from both the 30B-128 and 30B-256 Ascend configurations. Table 1 reports a before-and-after calibration comparison for selected execution units from the 30B-128 calibration traces. After calibration, all regular units listed in the table exceed 99% accuracy, and most irregular units exceed 94%. Table 2 reports the calibration results for both configurations. Calibration improves average execution-unit accuracy and reduces WAPE in both settings, with end-to-end training-step latency prediction accuracy reaching 99.30% on 30B-128 and 99.83% on 30B-256.

(2) Extrapolation test across deployment scales. For the extrapolation test, we hold out entire deployment configurations from calibration. We apply the same fitted estimator to three unseen Ascend deployments ranging from 2K to 16K devices. Table 4 reports end-to-end training-step latency prediction accuracy above 99.6% in all three deployments.

Deployment scenarioEnd-to-end latency prediction accuracy (%)
30B-2K99.87
90B-6K99.70
505B-16K99.61

What we found encouraging is that the hardware parameters and communication characteristics learned at smaller scales remains useful as the same hardware architecture is deployed in larger cluster topologies. Since collecting profiling traces becomes increasingly costly as the number of devices grows, this points to a practical way to estimate communication costs and training-step latency at larger scales using measurements from smaller clusters.

The revised manuscript defines the evaluation metrics in Section 6.1 (Page 8) and describes the calibration and extrapolation evaluations in Section 6.2 (Pages 8–9).

(3) Search efficiency with experience reuse. We evaluate 100 heterogeneous 256-GPU scenarios with three random seeds per scenario. We use two metrics to assess configuration quality and search efficiency. The optimality gap measures the relative latency difference between the best parallelism configuration identified through experience-guided search and the optimal configuration. The search iterations metric records the number of iterations a run takes to first identify the reference-optimal configuration.

For policy-based search, Figure 3 (Section 6.3, Page 9) compares learning policies from scratch (blue) with adapting policies learned at the 128-GPU scale (orange). The experience-guided variant starts with a smaller optimality gap, and its curve reaches 0% sooner. Table 5 (Page 10) shows that the maximum iteration count falls from 19 to 10 (about 47% fewer) and the average from 7.0 to 1.7 (about 76% fewer). Est2Exp can also distill deployment experience from the resulting scoring policy and store that experience with its applicability conditions for future searches.


5. Code and data release. Following the reviewers’ suggestions, we have prepared the initial Est2Exp release to support reproducibility. It includes estimator code and skills for constructing the deployment experience bank. We will make these materials publicly available at Est2Exp repository and continue to update and refine them.

The profiling traces collected through our industry collaboration contain commercially confidential information. We plan to work with our partner to identify a subset of the traces, remove sensitive information, and seek approval for its public release.

Thank you again for your feedback. It helped us explain the methodology more clearly and document the implementation more fully in the revised manuscript.

Reviewer_uDsvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how to convert performance estimates for distributed LLM training into reusable deployment knowledge, reducing repeated search when model sizes, cluster resources, or device distributions change. Its goal extends beyond predicting training latency to accumulating evidence-backed, context-dependent experience that can be revised using new observations. Using real profiling data spanning 30B–505B models and 128–16K Ascend NPUs, the authors report approximately 99.3%–99.9% DAG-level latency accuracy after calibration. Transferring policies from heterogeneous 128-GPU scenarios to 256-GPU scenarios raises reported search success from 73.3% to 98.3% and reduces candidate evaluations among successful runs from 12 to 4. These evidence sources should be distinguished: real profiling validates the estimator, while deployment experience and what-if results rely primarily on simulation.

Critical strengths
  1. The analysis of Ascend systems across model sizes and configurations up to 16K NPUs is valuable. Explicitly reserving 30B-2K, 90B-6K, and 505B-16K as extrapolation tests provides stronger evidence than fitting a single setting, particularly through the improvements in execution-unit accuracy.
  2. Organizing one-off configuration-search results into reusable knowledge addresses a meaningful systems problem beyond performance prediction.
Critical weaknesses
  1. The initial estimator already achieves 98.59%–99.74% DAG accuracy in Table 2. For 90B-6K, calibration slightly decreases this metric from 99.74% to 99.70%, while execution-unit accuracy remains only 78.73%. Aggregate accuracy can therefore mask local errors and does not establish correct ordering of closely performing configurations. The paper needs explicit accuracy formulas, aggregation weights, confidence intervals, ranking correlations, and selection regret measured using real latency. The initial 30B-128 DAG accuracies of 93.6% in Table 1 and 98.62% in Table 2 also require a scope explanation. 2.Table 4 does not sufficiently specify the target threshold, test-scenario count, random seeds, search budget, or aggregation used for iterations and evaluations. Reporting evaluation counts only for successful runs can introduce selection bias. Total costs for bank construction, LLM calls, function search, profiling, and retrieval are also missing, preventing assessment of how many reuse episodes are needed to amortize the upfront investment.
Questions for authors
  1. What are the exact definitions of Acc_unit and Acc_total, the raw predicted/measured latencies, and the calibration sample counts? Why do Tables 1 and 2 report different initial 30B-128 accuracies? What are ranking error and real selection regret on unseen configurations?
  2. Can the transferred 256-GPU recommendations and strong alternatives be executed to validate latency, memory feasibility, and ranking? Do the retained findings survive computation–communication overlap or nonuniform expert routing?
  3. What are Table 4's success criterion, number of test instances, seeds, and treatment of failed runs?
  4. Can you provide complete algorithms for function search, experience retrieval, and conflict attribution, and disclose which calibrations or experience entries were manually revised?
3 comments
AuthorsOpenReview ↗

Response to Reviewer uDsv (Part 1/3)

Thank you for reading our paper so carefully and for taking the time to share such thoughtful and constructive feedback. Your comments helped us explain Est2Exp more clearly in the revised manuscript. We respond to each of your main concerns below.

W1 & Q1: Accuracy Metrics, Table Consistency, and Configuration Ranking

1. Accuracy Metrics The original manuscript described the accuracy metrics only briefly. In Section 6.1 (Metrics, page 8) of the revised manuscript, we provide full mathematical definitions for the two metrics used to evaluate the estimator at the execution-unit and end-to-end levels.

  • Accunit\mathrm{Acc}_{\mathrm{unit}} is execution-unit latency accuracy, which measures the average prediction accuracy across all execution units.
  • Acctotal\mathrm{Acc}_{\mathrm{total}} is end-to-end latency accuracy, which measures the prediction accuracy of total training latency.

We also report WAPE (weighted absolute percentage error) to quantify the aggregate prediction error across execution units. WAPE weights each unit’s relative error by its measured latency, so longer-running units contribute more to this metric.

The raw predicted latencies are the estimator outputs, and the raw measured latencies are the corresponding values obtained through profiling. For each execution unit, the calibration sample count is the number of profiling observations used to calibrate its latency model.

2. Execution-Unit Accuracy and Configuration Ranking

(1) Accuracy at the execution-unit and end-to-end levels. We agree that high end-to-end accuracy can mask local prediction errors because overestimates and underestimates along the DAG’s critical path can partially cancel. Our calibration therefore aims for accurate latency predictions at both levels.We first calibrate the latency model for each execution unit and assess its predictions using Accunit\mathrm{Acc}_{\mathrm{unit}} and WAPE, as defined above. We then use Acctotal\mathrm{Acc}_{\mathrm{total}} to check how closely the end-to-end latency predicted from the DAG's longest path matches the measured latency. As shown in Tables 1 and 2, calibration improves execution-unit accuracy, and end-to-end accuracy exceeds 99% in both the 30B-128 and 30B-256 configurations.

(2) Cross-scale extrapolation. The original manuscript reported calibration and extrapolation results together, making it difficult to tell which configurations were used for calibration. In the revised manuscript, Table 2 reports calibration results for the 128- and 256-device Ascend configurations. Table 4 reports results from applying the same calibrated estimator to three unseen deployments ranging from 2K to 16K devices, including 90B-6K.

Across Tables 2 and 4, the estimator achieves high accuracy at both levels in most configurations. The 90B-6K case is an exception at the execution-unit level, with moderate unit-level accuracy and 99.70% end-to-end accuracy. This gap arises in part because most large relative errors occur in short, irregular operations, so their absolute latency errors are small. Overestimates and underestimates along the critical path can also partially offset each other. We added this explanation to the second paragraph under “Cross-scale extrapolation” in Section 6.2 (page 9) of the revised manuscript.

(3) Ranking candidate configurations. The high end-to-end prediction accuracy supports using estimated training-step latency to prioritize promising parallelism configurations. We agree that when two candidates have similar predicted latencies, even small errors can change their ranking. Loop 3 (Figure 1, page 4) uses real-system measurements to resolve such discrepancies and determine whether to recalibrate the estimator or refine the corresponding deployment experience.

3. Correction of Table Inconsistency. Thank you for pointing out this discrepancy. We apologize for inadvertently reporting an intermediate result from an earlier calibration run as the initial 30B-128 DAG accuracy in Table 2. We have corrected the entry in the revised manuscript and rechecked all reported results.

AuthorsOpenReview ↗

Response to Reviewer uDsv (Part 2/3)

Q2. Hardware validation, computation–communication overlap, and expert routing

(1) Hardware validation. Thank you for raising this point. We agree that running the transferred recommendations alongside strong alternatives under identical conditions would directly test memory feasibility, training-step latency, and the estimator’s ranking. Our estimator-based evaluation covers 100 heterogeneous 256-GPU scenarios. Even a focused follow-up on 256 NPUs would require coordinating cluster access with our industry partner, verifying device slowdowns and network bandwidth limits, and testing several configurations per scenario. This would take longer than the rebuttal window allows.

For follow-up validation, we plan to test the configuration recommended using transferred experience alongside several strong alternatives in a few representative new scenarios. We will compare the hardware results for memory feasibility, training-step latency, and configuration ranking with the deployment findings distilled from the estimator’s outputs. If discrepancies arise, we can use these measurements in the existing Loop 3 (Figure 1) to determine whether to recalibrate the estimator or refine the stored guidance.

These experiments would extend the hardware validation of the existing Est2Exp architecture. The core contribution is the automated workflow already implemented in Est2Exp. It distills deployment findings from estimator outputs, and reuses them to guide configuration search in new scenarios. This gives engineers a practical way to reuse deployment knowledge across settings, start new configuration searches from prior findings, and reduce the need for costly hardware trials.

(2) Computation-communication overlap. The 256-NPU results already account for overlap under the evaluated schedules. The hybrid-granularity execution DAG in Figure 2 captures dependencies between execution units, allowing the estimator to model when computation and communication can run concurrently. For example:

  • In Data parallelism (DP), once a set of gradients is ready, its AllReduce overlaps with the remaining backpropagation.
  • In Pipeline parallelism (PP), computation for one microbatch overlaps with activation transfers for another. Using more microbatches also reduces the fraction of time the pipeline spends idle.

(3) Nonuniform expert routing. Thank you for raising this. It’s a practical question for MoE models such as DeepSeek-V4-Pro and Kimi K3. Uneven routing can change both how much work each expert does and how much data devices exchange, which in turn can affect which parallel configuration performs best. Understanding how these computation and communication loads interact calls for a systematic study beyond the scope of the rebuttal period. We see this as an important direction for future work.

We plan to collect and compare training traces under uniform and nonuniform expert routing. We’ll then use Est2Exp’s automated experience distillation to identify which of our existing recommendations still hold and derive suitable parallel configurations for different routing conditions.

AuthorsOpenReview ↗

Response to Reviewer uDsv (Part 3/3)

W2 & Q3: Search Success Criteria and Evaluation Protocol

(1) Success criterion and evaluation protocol. A search trial is successful if the best feasible parallelism configuration found within the search budget has a predicted latency within 3% of the reference-optimal latency, i.e., Tbest≤1.03,TrefT_{\mathrm{best}} \leq 1.03,T_{\mathrm{ref}}, where TrefT_{\mathrm{ref}} is the minimum predicted latency among all feasible configurations. The success rate is the proportion of trials that meet this criterion.

We evaluate 100 heterogeneous 256-GPU deployment scenarios with three random seeds per scenario, yielding 300 trials. Each trial runs for up to 15 iterations and stops early after three consecutive iterations without improvement. All trials count toward the success rate. In the original manuscript, we reported iteration and candidate-evaluation statistics only for successful trials.

(2) Evaluation metrics and search results. To better assess configuration quality and search efficiency, we added two metrics to Section 6.1 (Metrics, page 8) of the revised manuscript:

  • Optimality Gap (OG) measures the relative latency difference between the best parallelism configuration identified through experience-guided search and the optimal configuration.
  • Search Iterations measures search efficiency by recording the number of iterations required to first identify the reference-optimal configuration.

Using these two metrics, we compare cold-start policy-based search and experience-guided policy-based search in the new Figure 3 and Table 5. In Figure 3, the experience-guided method (orange) reaches a 0% optimality gap sooner than the cold-start method (blue). Table 5 shows that it reaches the reference-optimal configuration in 10 iterations on average, down from 19 with a cold start, a reduction of about 47%.


Q4. Algorithm Details and Disclosure of Manual Revisions

(1) Algorithm details. Thank you for raising this question. We expanded Appendix C (Experience Bank Construction and Refinement) in the revised manuscript. Sections C.1 and C.2 explain how solution-based and policy-based experience are constructed and retrieved, including the FunSearch procedure in C.2. Section C.3 details conflict attribution in Loop 3.

(2) Manual revisions and automation. We made no manual revisions to calibration results or experience entries in our experiments. Once the prompts, search spaces, and validation rules are set, Est2Exp automates estimator calibration, experience bank construction, and subsequent updates to stored experience.

Looking ahead, we see value in incorporating human feedback into this workflow. For example, deployment practices validated in real systems and initial guidance from domain experts could make estimator calibration and experience bank construction more efficient and accurate. We leave the design and systematic evaluation of these human-in-the-loop mechanisms to future work.

Reviewer_VHyPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Est2Exp, a framework for optimizing parallelization and resource configurations for LLM training in data center networks. Existing work falls into two categories: performance estimation, which often fails to capture irregular behavior; and experience banks, which can be expensive for large-scale LLM training. The proposed system, Est2Exp, couples estimator and experience bank components with bidirectional feedback in between. The estimator uses an LLM to model regular and irregular latency factors. The experience bank uses an LLM to convert deployment evidence into reusable, structured experience to guide future deployments. Conflicts between experiences are handled by estimator recalibration or applicability condition refinement.

The evaluation uses real-system training traces across various LLM and infrastructure scales. LLM calibration improves latency estimation accuracy for all reported regular units, but reduces accuracy on two of six irregular units. The authors further present a case study using 128-GPU deployment experiences in a 256-GPU setting to improve the target-reaching success rate. Additionally, the authors show that this 128/256-GPU experience does not generalize to larger-scale (2K) settings, and refine their applicability configurations.

Critical strengths

S1. The paper discusses a critical problem of improving large-scale training efficiency. As models and infrastructures continue to scale, the importance of this problem rises.

S2. In Table 2, the unit-level estimator accuracy gains generalize to large infrastructure scales (up to 16K GPUs), where the calibration data is limited to at most 256-GPU settings. This result highlights the effectiveness of the framework for practical deployment.

Critical weaknesses

My current assessment is weak reject, but I am open to raise my score if the weaknesses and question are addressed.

W1. The authors claim that the LLM-guided calibration provides reliable estimation from sparse real-system measurements. However, table 1 shows that two irregular units drop in accuracy after the calibration, especially on All GPU AR 1. The authors claim that insufficient evidence in available traces leads to these inaccuracies, which seems to contradict the claimed strength. It would be helpful to provide analysis on accuracy gains as a function of profiling data.

W2. The estimation accuracy component lacks comparison baselines from related work. Section 2.1 introduces multiple families of performance estimators for LLM training, yet none is cited for comparison in Section 6.2. If it is possible to reproduce those baselines on the profiling data used in the paper, the authors should include a few of them to demonstrate the strength of their estimator designs; if not applicable, the authors should provide a brief justification.

W3. In Section 6.3, the metric "target-reaching success rate" is not defined before presenting the results and needs to be inferred from the context.

Questions for authors

Q1. The paper discusses interesting observations that are only observable in a large-scale cluster: e.g., the order of DP-CP AllGather and DP-EP AllGather reverses as the cluster scales from 256 to 2K GPUs. Publishing (a subset of) the profiling traces is critical for reproducibility and facilitation of future research. I hope the authors can clarify whether there are plans to release the traces, as the paper currently does not mention this.

4 comments
AuthorsOpenReview ↗

Response to Reviewer VHyP (Part 1/2)

Thank you for taking the time to read our paper carefully and for sharing your thoughtful feedback. We really appreciate your openness to reconsidering your assessment and the opportunity to address your concerns. We hope the clarifications and additional results below help answer the questions you raised.

W1: Low Prediction Accuracy for Two Irregular Communication Units

Thank you for pointing out these issues. We acknowledge that the initial calibration reduced the accuracy of two irregular communication units. Our default calibration uses profiling data from only a single training step per configuration to minimize measurement overhead on large-scale Ascend clusters. For these two units, the limited profiling data made it difficult to separate communication latency from variable synchronization and runtime overhead and to identify reliable physical latency models.

Together, these two units account for only 0.15% of the critical-path duration, and each occurs once per training step. Their prediction errors therefore have little effect on the end-to-end latency estimate, which achieves 99.3% accuracy (Table 1, page 9).

To see whether more profiling data would help, we collected profiling data from four additional training steps under the same 30B-128 configuration. These observations gave the LLM more evidence to identify suitable latency functions and fit their coefficients. As shown below, accuracy improved by more than 45 percentage points for both units when using multi-step profiling.

Execution unitInitial (%)Single-step profiling (%)Multi-step profiling (%)
DP Scalar AR tail32.131.277.9
All GPU AR 155.042.287.6

We added this analysis and the new results to the second paragraph under “Calibration accuracy” in Section 6.2 (page 8) and highlighted the additions in blue.

These results motivate an adaptive profiling strategy that starts with a single training step and collects additional measurements for units with large prediction errors or insufficient data to fit a reliable latency model. This strategy needs to be integrated into the profiling workflow on real clusters to assess the cost of collecting those extra measurements. We therefore leave its implementation and a systematic evaluation of the trade-off between profiling overhead and prediction accuracy to future work.


W2: Comparison with Existing Baselines

Thank you for this valuable suggestion, which motivated us to extend our evaluation with two representative simulators, ATLAHS [1] and ASTRA-Sim [2].

  • ATLAHS [1]: It constructs separate model-level and P2P-level DAGs and predicts step latency using analytical node-latency models. Unlike our proposed Est2Exp, it neither integrates multiple granularities into a unified DAG nor employs LLM-guided physical-model calibration.

  • ASTRA-Sim [2]: It converts profiled computation and HCCL communication events into Chakra execution traces and estimates latency using analytical compute and network backends. Unlike Est2Exp, it does not support LLM-guided discovery or calibration of interpretable physical latency models.

We evaluate both baselines on the 30B-128 and 30B-256 Ascend configurations. Because training steps follow similar computation and communication patterns under a fixed configuration, we use a single training step per configuration, which is sufficient for comparing latency prediction accuracy.

The table below reports end-to-end training-step latency prediction accuracy (Acctotal\mathrm{Acc}_{\mathrm{total}}) for the two Ascend configurations. All baselines fall below 90% accuracy at 128 devices and lose accuracy at 256 devices, with the largest drops for the two ATLAHS variants. Est2Exp gives the closest predictions to the measured latencies in each configuration, exceeding 99% accuracy in both. It also closely tracks the measured increase in training-step latency from 128 to 256 devices. Together with our calibration experiments, these results support the effectiveness of our LLM-guided automatic function calibration method.

Method30B-128 Acctotal\mathrm{Acc}_{\mathrm{total}} (%)30B-256 Acctotal\mathrm{Acc}_{\mathrm{total}} (%)
ATLAHS-model-level83.062.7
ATLAHS-P2P-level89.965.9
ASTRA-Sim86.583.1
Est2Exp99.399.8

In the revised manuscript, Section 6.1 (Page 8) describes the baselines and evaluation setup. In Section 6.2 (Page 9), Table 3 compares measured and predicted training-step latencies, and the new paragraph “Comparison with existing estimators” discusses the results. These changes are highlighted in blue in the revised manuscript.

[1] S. Shen et al., “ATLAHS: An application-centric network simulator toolchain for AI, HPC, and distributed storage,” 2025.

[2] W. Won et al., “ASTRA-sim2.0: Modeling hierarchical networks and disaggregated systems for large-model training at scale,” 2023.


AuthorsOpenReview ↗

Response to Reviewer VHyP (Part 2/2)

W3. Definition of Evaluation Metrics

Thank you for pointing out this ambiguity. By target-reaching success rate, we meant the proportion of search trials in which experience-guided search finds the reference-optimal parallelism configuration within the allotted budget. This rate measures how reliably the search succeeds at a given budget, leaving open how performance changes across budgets and how many iterations it takes to reach the optimum.

To better assess configuration quality and search efficiency, we added two metrics to Section 6.1 (Metrics, page 8) of the revised manuscript:

  • Optimality Gap (OG) measures the relative latency difference between the best parallelism configuration identified through experience-guided search and the optimal configuration.
  • Search Iterations measures search efficiency by recording the number of iterations required to first identify the reference-optimal configuration.

Using these two metrics, we compare cold-start policy-based search and experience-guided policy-based search in the new Figure 3 and Table 5. In Figure 3, the experience-guided method (orange) reaches a 0% optimality gap sooner than the cold-start method (blue). Table 5 shows that it reaches the reference-optimal configuration in 10 iterations on average, down from 19 with a cold start, a reduction of about 47%.


Q1: Profiling Trace Availability and Release Plans

Thank you for your interest in our profiling traces. We would also be happy to make these data available to the research community. However, since the traces were collected through our collaboration with an industrial partner, the current dataset contains commercially confidential information and cannot be publicly released at this stage.

Looking ahead, we will explore the possibility of releasing a subset of the profiling data after removing sensitive information and obtaining the necessary approval from our partner. We believe this could help other researchers reproduce our observations and further investigate large-scale training behaviors. This is particularly important as model sizes continue to grow and GPU clusters in data centers scale to thousands of accelerators. We fully recognize the value of sharing such data with the research community and will certainly consider this as part of our future plans.

Reviewer_VHyPOpenReview ↗

Reviewer's response

Thanks to the authors for the sensitivity analysis on the volume of profiling data and additional baselines. I appreciate the authors' openness in open-sourcing the data. Overall, I believe the additional evidence is sufficient to positively change my assessment of the paper, and I have adjusted my score accordingly.

AuthorsOpenReview ↗

Response to Reviewer VHyP

Thank you for your encouraging feedback and for taking the time to reconsider our work! We truly appreciate your constructive suggestions, which have helped us strengthen the evaluation and improve the paper. We believe Est2Exp can become increasingly valuable as LLMs and training clusters continue to scale. We also plan to continue expanding the data we make publicly available over time and hope it can contribute to future research on AI infrastructure.

Reviewer_EcchOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Est2Exp, a promising framework that addresses the limitations of existing performance estimators for LLM training parallelism. Since existing estimators often produce scenario-specific solutions that are difficult to reuse as parallelism strategies and deployment environments evolve, Est2Exp couples a performance estimator with an experience bank to accumulate and reuse deployment knowledge. The authors demonstrate high latency estimation accuracy across various model and cluster scales and show that reusing accumulated knowledge significantly improves the success rate and efficiency of cross-scale configuration search.

Critical strengths

[S1] Clear and well-motivated presentation. The paper is exceptionally well-written and easy to follow. In particular, the introduction clearly explains the problem and motivates the proposed approach.

[S2] Promising approach to reusable deployment knowledge. Est2Exp addresses an important challenge by transforming performance estimation into reusable deployment experience, which is particularly valuable given rapidly evolving parallelism strategies and deployment environments.

Critical weaknesses

[W1] Overly strong claims about existing estimators. The paper somewhat overstates the limitations of existing estimators without sufficiently acknowledging their original design goals and target environments. The positioning could be more balanced.

[W2] Lack of direct comparison with existing estimators. The paper lacks comparisons with representative baselines such as Metis, making it difficult to assess Est2Exp's relative advantages in latency estimation accuracy, parallelism selection quality, and cross-environment extrapolation.

Questions for authors

Thank you for submitting this paper to ICLR! I really enjoyed reading it. In particular, I found the paper refreshingly clear and well-motivated, and I especially appreciated how clearly the introduction explains the problem and motivates the proposed solution.

I am generally very satisfied with the paper's current presentation, but I have the following two concerns.

1) Positioning against existing estimators.

The paper states that existing estimators typically produce scenario-specific solutions rather than reusable deployment knowledge. I understand the motivation behind this argument, but I feel that the claim is somewhat too strong.

In my view, existing estimators were often designed to address the parallelism strategies and deployment environments available at the time. However, these environments and parallelism strategies evolve rapidly, making it difficult to directly reuse previously obtained solutions in new settings.

I believe Est2Exp is a promising solution to this problem, as it systematically accumulates and reuses deployment knowledge across changing environments. Therefore, slightly softening the discussion of existing estimators would not weaken the paper's contribution. In fact, I believe it would make the motivation and contribution even more convincing.

TL;DR: Please consider softening the claims about existing estimators in the abstract and introduction, while emphasizing the importance of reusable deployment knowledge in rapidly evolving environments.

2) Comparison with existing estimators.

In line with my first concern, could the authors compare Est2Exp against existing approaches, such as Metis or other representative recent baselines?

I believe a direct comparison would help readers better understand where Est2Exp stands relative to existing solutions and make its advantages more convincing.

For example, it would be interesting to compare latency estimation accuracy, the quality of the selected parallelism configurations, and, most importantly, the ability to extrapolate to unseen environments or larger scales.

Of course, I understand that many existing estimators rely on extensive profiling, and their latency estimates can be very accurate within the environments they target. Even if Est2Exp achieves comparable estimation accuracy, I believe this would still be a strong result, especially if it can demonstrate better extrapolation or more effective reuse of previously accumulated knowledge.

I also understand that implementing existing baselines on Ascend NPUs may not be straightforward. Nevertheless, even a limited comparison or a more detailed discussion of these differences would help clarify Est2Exp's advantages.

Thank you again for this valuable contribution. My current assessment is somewhere between 3 and 4, leaning toward 4. If the authors can satisfactorily address the above concerns, especially the comparison with existing approaches, I would be happy to raise my score to 4 (Clear Acceptance).

5 comments
AuthorsOpenReview ↗

Response to Reviewer Ecch

Thank you for your kind words about our paper. Your positive assessment is a great encouragement to us.


W1 & Q1: Desciption of the Estimator. We truly appreciate this thoughtful and constructive suggestion. We agree with your point that existing estimators were often designed for the deployment settings “available at the time.” Their contributions within those settings should be acknowledged.

Following your suggestion, we carefully revised the relevant wording in the Abstract and Introduction (Page 1), with all changes highlighted in blue. Your feedback helped us present the paper’s positioning more clearly and in a more balanced way!


W2 & Q2: Comparison with Representative Estimators and Extrapolation Capability.

1. Comparison with existing baselines. Thank you for this valuable suggestion, which motivated us to extend our evaluation with two representative simulators, ATLAHS [1] and ASTRA-Sim [2].

  • ATLAHS [1]: It constructs separate model-level and P2P-level DAGs and predicts step latency using analytical node-latency models. Unlike our proposed Est2Exp, it neither integrates multiple granularities into a unified DAG nor employs LLM-guided physical-model calibration.

  • ASTRA-Sim [2]: It converts profiled computation and HCCL communication events into Chakra execution traces and estimates latency using analytical compute and network backends. Unlike Est2Exp, it does not support LLM-guided discovery or calibration of interpretable physical latency models.

We evaluate both baselines on the 30B-128 and 30B-256 Ascend configurations. Because training steps follow similar computation and communication patterns under a fixed configuration, we use a single training step per configuration, which is sufficient for comparing latency prediction accuracy.

The table below reports end-to-end training-step latency prediction accuracy (Acctotal\mathrm{Acc}_{\mathrm{total}}) for the two Ascend configurations. All baselines fall below 90% accuracy at 128 devices and lose accuracy at 256 devices, with the largest drops for the two ATLAHS variants. Est2Exp gives the closest predictions to the measured latencies in each configuration, exceeding 99% accuracy in both. It also closely tracks the measured increase in training-step latency from 128 to 256 devices. Together with our calibration experiments, these results support the effectiveness of our LLM-guided automatic function calibration method.

Method30B-128 Acctotal\mathrm{Acc}_{\mathrm{total}} (%)30B-256 Acctotal\mathrm{Acc}_{\mathrm{total}} (%)
ATLAHS-model-level83.062.7
ATLAHS-P2P-level89.965.9
ASTRA-Sim86.583.1
Est2Exp99.399.8

In the revised manuscript, Section 6.1 (Page 8) describes the baselines and evaluation setup. In Section 6.2 (Page 9), Table 3 compares measured and predicted training-step latencies, and the new paragraph “Comparison with existing estimators” discusses the results. These changes are highlighted in blue in the revised manuscript.

[1] S. Shen et al., “ATLAHS: An application-centric network simulator toolchain for AI, HPC, and distributed storage,” 2025.

[2] W. Won et al., “ASTRA-sim2.0: Modeling hierarchical networks and disaggregated systems for large-model training at scale,” 2023.


2. Extrapolation Capability. We calibrated Est2Exp using measurements from the 128- and 256-device Ascend configurations, then applied the calibrated estimator to configurations ranging from 2K to 16K devices. prediction accuracy of end-to-end training-step under different network scales are summarized in the following table. As shown, all accuracy exceeded 99.6% in all evaluated configurations.

Target configurationAcctotal\mathrm{Acc}_{\mathrm{total}} (%) ↑
30B-2K99.87
90B-6K99.70
505B-16K99.61

What we found encouraging is that the hardware parameters and communication characteristics learned at smaller scales remained useful as the same hardware architecture was deployed in larger cluster topologies. Since collecting profiling traces becomes increasingly costly as the number of devices grows, this points to a practical way to estimate communication costs and training-step latency at larger scales using measurements from smaller clusters.

In the revised manuscript, Table 4 (page 9) compares measured and predicted training-step latencies, and the new paragraph “Cross-scale extrapolation” (pages 8–9) in Section 6.2 discusses these results. The additions are highlighted in blue.

Dear authors,

Thank the authors for the detailed response!

Most of my concerns are addressed, but I read the comments from other reviewers.

I think some concerns are valid, regarding technical details/presentation. Of course, the authors improved the paper during the revision, and it addresses those concerns significantly. However, since some reviewers have concerns regarding the technical detail presentation, I want to ask a quick question.

Will the authors release code to public? Since some reviewers have concerns in terms of technical details, if the code will be released, it will be very helpful for me to raise my score.

Thank you! I'm waiting for the authors' reply!

Best, Reviewer Ecch.

AuthorsOpenReview ↗

Response to Reviewer Ecch

Thank you for your thoughtful follow-up and for reconsidering our work! We agree that releasing the code would make the technical details easier to inspect and reproduce.

In the revised manuscript, we have substantially expanded Appendices A–C to provide more technical details. Appendix A explains DAG modeling and its applications, Appendix B describes the profiling-guided calibration process using LLM, and Appendix C covers experience construction and retrieval, FunSearch, and conflict attribution.

We have now made an initial release of Est2Exp available at Est2Exp repository. Because this work was developed in collaboration with an industry partner, the raw profiling data and trace-related components remain confidential. The current repository includes the estimator code and reusable skills for constructing the deployment experience bank. We will continue updating and refining them with materials cleared for public release.

We hope these additions make Est2Exp easier to understand and reproduce.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.