acceptodds
Under review as a conference paper at ICLR 2027

LaDiM: Layered Diagnosis for Multi-Agent Code Migration

Abstract

Migrating code across deep learning frameworks enables the reuse of training programs in new software and hardware environments. Different frameworks, however, might compute gradients or update parameters in different ways, leading to wrong results even when a translated program runs without error. Current methods are mostly designed for language transfer, neglecting the subtle differences in framework level. To address this, LaDiM preserves the underlying training computation order, e.g., forward, gradient computation, update, and uses the dependencies among these stages to detect and localize silent computational errors. LaDiM includes Translator, Verifier, and Repair agents working under a shared Orchestrator. The Translator produces an initial output. The Verifier traces discrepancies along the computation order and hands its findings to the Repair Agent, which revises the candidate using this evidence and a persistent repair history. For repository level migrations, LaDiM further introduces repository context management which tracks cross file dependencies and preserves diagnostic context as agents move across the repository. Evaluation across framework migration, language migration, and repository migration tasks shows that LaDiM achieves comparable or better migration results with up to 80.8% fewer tokens than state-of-the-art methods, demonstrating its effectiveness.

Primary area:
applications to robotics, autonomy, planning
Keywords:
code migration, large language models, multi-agent systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TR5WOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LaDiM is an agent framework for migrating deep-learning code across frameworks and languages. It combines translation, staged diagnosis, and repair, using comparisons of forward values, gradients, and parameter updates to guide debugging. Repository-level extensions manage dependencies and reconstruct relevant context. The paper reports improved acceptance or reduced token usage on several migration collections.

Critical strengths
  1. The diagnostic signals address a real failure mode. The Tanh example in Figure 4 illustrates that nearly identical forward values and losses can hide a broken differentiation path. Comparing per-parameter gradients provides actionable information that execution-only testing would miss.

  2. Some efficiency results are substantial. On the main MindSpore collection, LaDiM and MatchFixAgent both achieve complete acceptance, while LaDiM uses 57.4% fewer tokens. The time-series repository also shows considerable token savings at equal acceptance.

Critical weaknesses
  1. The evidence for the proposed orchestration is inconsistent. The method mainly combines LLM role separation, persistent repair history, numerical testing, and context management. These are reasonable engineering choices, but the ablations do not consistently justify the complete design. In Table 7, removing repair history preserves 50/50 acceptance while reducing tokens from 5.159M to 4.566M. Although history helps the separate ten-program repair study, this benefit does not extend to the main collection.
  2. The evaluation is small and heavily constructed. The advertised 50 MindSpore tasks correspond to only 29 distinct source-program/evaluation combinations, and 20 of these already pass before repair. Thus, the main repair comparison involves only nine distinct failing inputs. Repository evaluation covers just two repositories, with 865 and 2,086 nonblank lines. The training-signal ablation uses four deliberately constructed fault types that align directly with the four feedback categories. These experiments demonstrate feasibility, but provide limited evidence of reliable gains on a broader migration workload.
  3. The gains do not consistently transfer to other settings. In the six-case JAX repair study, both LaDiM and Direct repair achieve 6/6 acceptance, but LaDiM uses 0.674M tokens versus 0.353M. In the native JAX study in Appendix H, all methods reach 9/12 overall acceptance, while LaDiM uses 1.980M tokens versus 0.901M for Direct repair. The headline 80.8% saving comes from the time-series repository; on the recommendation repository, LaDiM consumes more tokens than either main baseline. The evidence therefore supports workload-specific advantages rather than a broadly effective accuracy–cost improvement.
  4. The comparisons do not sufficiently isolate diagnostic value from basic agent execution behavior. In the Tanh case, both baselines identify the suspected operator, and MatchFixAgent even drafts the relevant registration, but neither applies the patch within the budget. LaDiM’s advantage here therefore includes enforcing that an edit is actually applied and tested, rather than uniquely identifying the fault. The paper does not compare with coding-agent harnesses such as Claude Code or Codex supplied with the same numerical checks. Such a comparison would help establish whether specialized multi-agent coordination adds value beyond providing useful tests and a competent edit–test loop.
Questions for authors

Can the authors compare LaDiM with a coding-agent harness (e.g., codex, claude code) given the same source programs, initial translations, numerical checks, and repair budget? Please distinguish a same-model orchestration comparison from an end-to-end system comparison.

2 comments
AuthorsOpenReview ↗

Response to Reviewer TR5W (1/2): Comparison with Claude Code and the Value of Diagnosis

Thank you for the constructive feedback. In the uploaded revision, modified text is in blue and the added tables appear in Appendix J. This comment answers Q1&W4, and Part 2 answers W1, W2, and W3.

Q1&W4. Comparison with a coding-agent harness, and isolating diagnostic value from the edit and test loop.

Following your suggestion, we compare LaDiM with Claude Code (version 2.1.322) given the same source programs, initial translations, and numerical checks. Its repair budget is the same 40 model calls, 120,000 output tokens, 1,800 s, and four submissions. A wrapper ends a session at the first limit reached and counts each run of the acceptance script as a submission. Both systems call the same model on one service with the reasoning setting of Table 14, every model call made by Claude Code counts toward the 40, and scratch tests do not count as submissions for either. Because our claim concerns orchestration, we hold the LLM fixed across LaDiM and Claude Code, so every number below is a same-model result. Pairing Claude Code with its native model would change the model and the orchestration together, and a difference could then come from either. The diagnosis stage does not rely on a particular model, and we leave its evaluation with stronger models to future work. We also include two LaDiM controls with identical signals and tools: flat diagnosis presents the measurements as one flat shuffled list without signal-type labels or check names, while the second control removes the Verifier stage. Unlike Direct repair in Table 2a, which allows two calls and one edit per attempt in one conversation and has no completion condition, this second control retains the repair stages, repair history, and completion condition of LaDiM. Every failing translation receives three independent runs, and the table counts repaired attempts.

CollectionAttemptsLaDiMClaude CodeFlat diagnosisWithout diagnosis stage
Table 2a programs (5)15127116
Public PyTorch programs (21)6338243421
Main-collection translations (26)7875707369

On the public programs, the repair rate of LaDiM is 22.2 points above that of Claude Code (95% paired bootstrap interval over source files, 11.1 to 31.7). Table 15 in Appendix J sets the configuration of LaDiM, the two controls, Claude Code, and Direct repair side by side, and Table 17 lists the repaired runs of every program under each method. On these 63 attempts, Claude Code uses 126.3M tokens versus 104.6M for LaDiM, and auxiliary calls outside its main loop make up 4% of its calls.

We agree that the Tanh result includes applying and testing an edit, which is the completion condition of the Repair Agent in Section 3.3. The variant without the diagnosis stage keeps the repair stages, repair history, and completion condition and removes the Verifier, so it measures the gain from adding the diagnosis stage. On the Table 2a programs it repairs 6 of 15 attempts, while Direct repair, with two calls per attempt, repairs 1 of 15 in three runs and none in the single run of Table 2a. The two differ in the length of the repair stages and in the completion condition. The diagnosis stage raises the count from 6 to 12, and over the three rows of the table, LaDiM repairs 29 more attempts than the variant without it. Against flat diagnosis, LaDiM repairs 7 more attempts. On the autoencoder of Figure 4, LaDiM repairs 3 of 3 runs, flat diagnosis 2, the variant without the diagnosis stage 1, and Claude Code 1. Claude Code applies an edit and runs the checks after it in 151 of its 156 sessions (97%), so sessions without a tested edit explain few of its failures. Of its 55 unrepaired sessions, 35 end at the call limit and 8 at the submission limit (Appendix J). We also measure fault localization before any edit under the layered and flat records. Of the 50 controlled faults of Appendix C, the injection changes one function in 38 and several functions in the other 12. On these 38, with one run per fault, the Verifier names the faulty function in 34 cases for LaDiM and 32 for flat diagnosis.

AuthorsOpenReview ↗

Response to Reviewer TR5W (2/2): Orchestration Design, Evaluation Scale, and Transfer

This comment continues our response with W1, W2, and W3.

W1. Evidence for the orchestration design, and repair history in Table 7.

Repair history takes effect when a program needs more than one submission. In the main collection, 20 of the 29 inputs pass before repair and seven of the nine failing inputs are repaired at the first submission, which leaves only two inputs where history can act. Removing it saves tokens at 29/29 acceptance. By contrast, on the Table 2a programs, where repairs take up to three submissions, removing history lowers the repaired programs from 4/5 to 0/5 (Table 3b). On the public programs, where 25 of the 38 repairs by LaDiM are accepted at the second or a later submission, it lowers the repaired attempts from 38/63 to 20/63. On recommendation, the conditions without repository context management stop at the context limit after 59 to 68 calls, and with it the run uses the full 80-call budget (131/145 checks in every condition of Table 11), whereas the token saving of the repository components appears on time series (Table 3c). The revised paper states these conditions next to Tables 7 and 11.

W2. Evaluation scale and constructed tasks.

We agree that the nine failing inputs of the main collection form a small repair set. The repair evaluation now spans 52 failing translations from three collections (composition and sources are in our comment to all reviewers). On the 21 failing translations of public PyTorch programs, LaDiM repairs 38/63 attempts versus 3, 2, and 3 of 21 in one run each for Direct repair, SWE-agent, and MatchFixAgent. These failures arise during translation and therefore complement the four constructed fault types of Table 3a. Among the 52 failing translations, 18 are silent failures (11 first caught by the gradient comparison). For repository scale, we add a third public repository with 69 code files and 7,827 nonblank lines, almost four times the recommendation repository. On this repository, LaDiM passes 116 of 131 behavior checks, MatchFixAgent 113, and SWE-agent 99 (Table 21).

W3. Transfer of the gains to other settings.

The repair gain is large on the Table 2a programs and the public failing translations, where the baselines of Table 2a repair none of the five programs and at most 3 of 21, and the token saving is largest where LaDiM finishes before the budget. In Table 2b, both methods repair all six supplied JAX faults, which leaves the Verifier stage no headroom for further repairs while it still adds tokens. On 16 failing JAX translations from 63 public workloads, LaDiM repairs 9, 8, and 9 in three runs versus 7, 8, and 7 for Direct repair. The repair-token ratio is 1.9 there, close to the 2.4 and 1.9 of the separate studies of Table 13 and Table 2b, so on JAX the additional repairs come at a higher token cost. On time series, LaDiM stops after 35 of the 80 calls (Table 1c). On recommendation, all three agents use the full 80 calls and their token totals lie within 6% of each other. LaDiM restores reward model inference and retrieval, which stay unresolved in both baselines, and reaches the highest coverage of 130/145 checks (130, 131, and 130 in three independent runs versus 129, 130, and 128 for MatchFixAgent). The third repository of W2 shows the same picture: all three agents use the full 80 calls, their token totals lie within 11% of each other in every run, and LaDiM reaches the highest coverage in all three runs. The revised abstract removes the single 80.8% figure and distinguishes the three main comparisons that reduce token use from the one that reaches the highest coverage at comparable token use.

Reviewer_PJ7ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to locate and repair the true cause of semantic drift after initial translation. The authors propose LaDiM (Layered Diagnosis for Multi-Agent Code Migration), a multi-agent frame work that follows the training computation order of deep learning programs. The authors propose Layered Diagnosis, inspecting the translated candidate stage by stage, e.g., forward computation, gradient computation, and parameter update, in the order of how training actually executes. According to the results, LaDiM achieves comparable or better migration results with up to 80.8% fewer tokens than state-of-the-art methds.

Critical strengths
  1. The research problem is practically significant and clearly focused.
  2. The experimental evaluation is comprehensive, with clear results.
Critical weaknesses
  1. The contributions, in terms of novelty relative to existing research and practice, are not clearly articulated.
  2. The effectiveness of the core method has not been directly demonstrated.
  3. The scale of the evaluation is limited, and the comparative experiments lack sufficient control settings.
Questions for authors

Section 1:

  1. It is said that “… As a result, a translated program can run without raising error while still producing incorrect results due to silent failures in computing losses, gradients or parameter updates.” However, it fails to provide any examples of " silent failures " that actually occurred during the migration from PyTorch to MindSpore, nor does it indicate how frequently such errors arise. This point is central to the research motivation of this work. Section 2:
  2. At the end of each paragraph in the Related Work section, the authors summarize their own work; however, they should have first outlined the limitations of existing approaches—specifically, the aspects that these methods fail to address. Consequently, readers are unable to discern the innovative contributions of LaDiM based on this presentation. Section 3
  3. The title of section 3.2 is “VERIFIER AGENT WITH LAYERED DIAGNOSIS”; however, this layering is not reflected in the algorithm.
  4. It is said that “Based on this inventory, the agent partitions the target files into work units. Each unit specifies its editable files, a repair goal, its tests, and the units it depends on.” What is the agent's accuracy? Furthermore, there is no explanation regarding how to ensure the absence of cycles in the graph, nor how to handle cases where circular dependencies arise. Section 4:
  5. The discrepancies in Table 1b, Table 1d, and Table 3b each involve only a single procedure or check; furthermore, as this section does not report results from repeated runs or provide measures of variance, it is impossible to distinguish these discrepancies from random fluctuations.
  6. Appendix A.1 states that the 50 identifiers “correspond to 29 distinct combinations of source program and evaluation settings” and that identical conditions “are executed once and associated with the corresponding identifiers.” How do these identifiers differ, and why does Table 1a count all 50 rather than the 29 distinct inputs, over which “LaDiM and MatchFixAgent each accept 29/29”?
3 comments
AuthorsOpenReview ↗

Response to Reviewer PJ7R (1/2): Silent Failures, Novelty, and Evidence for the Core Method

Thank you for your detailed comments. Our revised paper marks all changes in blue and places the added tables in Appendix J. This comment answers Q1, Q2&W1, and W2, and Part 2 answers W3, Q3, Q4, Q5, and Q6.

Q1. Examples and frequency of silent failures in PyTorch to MindSpore migration.

Two of the five failing translations in Table 2a are silent failures. In the autoencoder of Figure 4, the program executes and its loss differs from the source by 1.43×10−61.43\times10^{-6}, while both gradients of the first linear layer are zero. In a gated convolutional model, the loss differs by 4.77×10−74.77\times10^{-7} while the six parameter tensors of its two branches and its GroupNorm layer receive zero updates. In both programs, a PyTorch function without a MindSpore mapping runs on copied tensors, which keeps the forward values and disconnects differentiation. For frequency, we classify every initial translation in three collections. A silent failure executes every step without an error and fails a numerical check.

CollectionTranslationsPassExecution failureSilent failure
Table 2a programs10532
Main-collection programs, three translations each8761179
Public PyTorch programs4524147

Silent failures account for 18 of the 142 translations and 18 of the 52 failing translations. Under the loss-only threshold of 0.02 used in Appendix A.6, 15 of these 18 programs pass the first-step loss check. Four of them, including the two examples above, also pass at the 10−510^{-5} threshold of Table 6. The gradient comparison is the first failing check in 11 of the 18, the forward values in five, and the parameter updates in two. We have added the first example and the overall frequencies to Section 1, with both examples, the threshold results, and the full classification in Appendix J (Table 18).

Q2&W1. Novelty relative to existing research and practice, and the Related Work section.

We agree that each paragraph of Related Work should first state what prior approaches leave open, and we have restructured the section accordingly. Specifically, translation methods and repository translation agents validate output behavior through tests, which can pass whenever forward values match even if gradients or parameter updates diverge. Differential testing tools such as TensorScope and NablaFuzz flag API-level discrepancies across frameworks but stop at detection; in practice, MindSpore TroubleShooter compares loss curves and MindConverter and MSAdapter map APIs, without localizing a gradient or update discrepancy. Repair agents (SWE-agent, MatchFixAgent) act on test failures, yet a failed numerical training check only flags the comparison and does not identify the faulty computation. To address this, LaDiM organizes its diagnosis along the dependency order of training (execution, forward values, gradients, and parameter updates): agreement in earlier layers and disagreement in later layers direct the investigation to the earliest inconsistent computation before any edit. W2 measures the contribution of the diagnosis stage and of the layered record.

W2. Direct evidence for the effectiveness of the core method.

We compare LaDiM with two controls that receive identical verification signals, tools, and budgets. Flat diagnosis is the non-layered counterpart and presents the same measurements as one flat shuffled list without signal-type labels or check names (Table 15). The second control removes the Verifier stage while keeping the repair stages and repair history. We also run Claude Code on the same LLM. The comparison uses 21 failing translations of public PyTorch programs with three independent runs each, and the table counts repaired attempts.

Failing translations of public programsAttemptsLaDiMFlat diagnosisWithout diagnosis stageClaude Code
All 21 programs6338342124
Seven silent failures among them21121035

The repair rate of LaDiM is 27.0 points above that of the variant without the diagnosis stage (95% paired bootstrap interval over source files, 14.3 to 39.7) and 6.3 points above that of flat diagnosis (1.6 to 12.7), and two of the four additional repairs over flat diagnosis fall on the silent failures (per-program results in Table 17, Appendix J).

AuthorsOpenReview ↗

Response to Reviewer PJ7R (2/2): Evaluation Scale, Algorithm Details, and Repeated Runs

This comment continues our response with W3, Q3, Q4, Q5, and Q6.

W3. Scale of the evaluation and control settings.

We expanded the repair evaluation to 52 failing translations across three collections, with their composition detailed in our comment to all reviewers. Every comparison holds the source programs, initial translations, checks, and budgets fixed across flat diagnosis, the variant without the diagnosis stage, Claude Code on the same LLM, and the three baselines of Table 2a. On the 21 failing public translations, Direct repair, SWE-agent, and MatchFixAgent repair 3, 2, and 3 in one run each. At repository scale, we add a third public repository with 69 code files and 7,827 nonblank lines, almost four times the recommendation repository. On this repository, LaDiM passes 116 of 131 behavior checks versus 113 for MatchFixAgent and 99 for SWE-agent (Table 21).

Q3. Layering in the algorithm of Section 3.2.

We agree, and the revised Algorithm 1 shows the layering explicitly. It adds a step, AssignLayers, that associates each measurement in S.mS.m with its layer of D\mathcal{D} (execution, forward values, gradients, or parameter updates) through its name and its field in the observation. This step makes existing behavior explicit, since the submitted code already names each numerical measurement by its signal and reports the execution status separately. Each Verifier call receives this record, and the Verifier selects the computation to investigate from the agreement and disagreement between layers (Table 15). The returned (ℓ,h)(\ell,h) records the computation that the Verifier selects and its hypothesized cause. Flat diagnosis in W2 removes the signal type and the check name from each record, keeps its parameter name, values, and threshold, and shuffles the order, and its repair count falls from 38/63 to 34/63.

Q4. Accuracy of the work-unit partition, acyclicity, and circular dependencies.

Acyclicity is enforced when a plan is stored. The planning tool traverses the prerequisite relations depth first and rejects any plan with a cycle or an unknown prerequisite, while the previous plan stays in place. The agent receives the rejection and can revise and resubmit the plan. On a test repository with mutually importing files, the agent places these files in one work unit in 3 of 3 runs, which is how circular dependencies are handled. For accuracy, we compare the final plans with the import structure reported by Repository Structural Analysis. In the repositories of Tables 1c and 1d, every file belongs to exactly one unit. Of the 26 imports between files of different units, 25 lead from a unit to one of its prerequisites, so shared code is repaired first. The remaining import joins two units without a declared dependency, and the checks of both units pass at the last submission. As Appendix G shows on recommendation, without planning a change to a shared training entry point drops the passed checks from 128 to 103 at the second submission, and the last submission regresses from 133 to 130. With planning, the run recovers from execution failures at the first submission to 122 and then 131 checks, and stays there.

Q5. Repeated runs and variance for Tables 1b, 1d, and 3b.

Following your suggestion, we added two independent agent runs to each comparison under the same configuration, where Run 1 is the run reported in the paper.

ComparisonMethodRun 1Run 2Run 3
Table 1b, accepted of 18 programsLaDiM9910
MatchFixAgent898
SWE-agent878
Table 1d, passed of 145 checksLaDiM130131130
MatchFixAgent129130128
Table 3b, repaired of 5 programsLaDiM444
Continuous conversation323

LaDiM has the highest mean in each of these comparisons, and its lowest run matches or exceeds every baseline run. Table 19 in Appendix J reports the three runs with their ranges. The component study of Table 11 is a separate recommendation run that passes 131 checks. The third repository and the JAX workloads also receive three runs (Tables 21 and 22), in which LaDiM has the highest mean.

Q6. The 50 identifiers and the 29 distinct inputs.

The 50 identifiers follow the task index of the controlled repair collection in Appendix C, where tasks that share a source program differ in the injected fault. End-to-end migration starts from the source program, so identifiers with the same source program and evaluation settings form one input that is executed once. We agree that the 29 distinct inputs are the right unit. Accordingly, Table 1a, Figure 1, Table 7, and the revised text report them as the primary unit, with LaDiM and MatchFixAgent accepting 29/29, SWE-agent 25/29, and Direct LLM 20/29. The 57.4% token saving over MatchFixAgent is unchanged, since each call was already counted once.

Thank you for the thorough response and the substantial additional experiments, especially the silent-failure analysis, the flat-diagnosis control, and the repeated runs, which make the paper clearer and more convincing. However, the gains of layered diagnosis over flat diagnosis and the baselines remain modest, so I am keeping my score of 2.

Reviewer_7VqtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces LaDiM, a multi-agent system for deep learning code migration that explicitly diagnoses and repairs semantic drift across frameworks by following the training execution order: execution, forward values, gradients, and parameter updates. The framework comprises a Translator, a Verifier that performs Layered Diagnosis, a Repair agent with evidence handoff and repair history, and repository-level coordination via structural analysis, dependency-graph planning, and context reconstruction. Experiments on PyTorch to MindSpore, Java/DJL to Python/PyTorch, two repositories, controlled ablations, and JAX related repair tasks show higher or comparable acceptance with substantially fewer tokens than strong agentic baselines (SWE-agent, MatchFixAgent).

Critical strengths
  • The motivation is well explained, and the role decomposition across agents (Translator/Verifier/Repair/Orchestrator) is clear.
  • This paper introduces a diagnostic strategy based on computational dependencies, sequentially examining execution, forward computation, gradients, and parameter updates. This structured approach helps localize discrepancies and reduces unnecessary debugging attempts.
  • This paper demonstrates promising token efficiency while maintaining comparable acceptance rates, highlighting its practical value. Furthermore, the proposed repository planning and context management mechanisms may be applicable to other agent based code tasks.
Critical weaknesses
  • The ablation studies mainly demonstrate the benefits of additional verification signals, including forward values, gradients, and parameter updates. However, they do not clearly isolate the contribution of the layered diagnostic strategy itself. A comparison with a non-layered diagnostic approach using identical verification signals and tools would help establish its independent effectiveness.
  • Although this paper provides detailed token usage statistics and time budgets, it lacks a systematic comparison of actual wall-clock runtime and computational resource consumption among agent-based methods
  • The reported 50 MindSpore migration tasks correspond to only 29 distinct program evaluation settings, and the repository-level evaluation covers only two relatively small repositories. Furthermore, in the six controlled JAX repair cases, Direct Repair achieves the same acceptance rate with fewer tokens, raising concerns about the generalizability of LaDiM's advantages across frameworks.
  • Most numerical comparisons cover only two or three consecutive training steps, which may not reveal accumulated numerical differences over longer training trajectories. Therefore, the current evaluation provides limited evidence that migrated programs preserve equivalent training behavior over extended training.
Questions for authors
  • Could the authors further clarify how much of the performance improvement comes from the dependency-guided diagnostic strategy itself, rather than from the additional verification signals? A more controlled ablation study would help distinguish these contributions.
  • Could the authors report the wall-clock runtime and computational resource usage of LaDiM and the agent-based baselines? This would help assess whether the reported token savings translate into practical efficiency improvements.
  • Could the authors extend the JAX evaluation to include more end-to-end migration tasks requiring repair and longer training trajectories? This would provide stronger evidence for the generalizability of layered diagnosis across frameworks.
3 comments
AuthorsOpenReview ↗

Response to Reviewer 7Vqt (1/2): Layered versus Non-Layered Diagnosis and Runtime

Thank you for the careful review. In our revised manuscript, changes appear in blue and the added tables are in Appendix J. This comment answers Q1&W1 and Q2&W2, and Part 2 answers W3 and Q3&W4.

Q1&W1. Contribution of the layered diagnostic strategy under identical verification signals and tools.

To isolate the diagnostic strategy, we compare LaDiM against two controls under identical verification signals, tools, and budgets. Flat diagnosis is the non-layered counterpart. It retains the Verifier stage and the evidence handoff and receives the same measurements as one flat shuffled list without signal-type labels or check names (Table 15). The Verifier instructions stay the same. The second control removes the Verifier stage and shifts its eight calls to the first repair stage. Unlike Direct repair in Table 2a, which allows two calls and one edit per attempt in one conversation, this second control retains the repair stages of LaDiM. All variants keep the completion condition of Section 3.3, and every failing translation receives three independent runs per variant. The table counts repaired attempts.

Failing translationsAttemptsLaDiMFlat diagnosisWithout diagnosis stage
Table 2a programs (5)1512116
Public PyTorch programs (21)63383421
Main-collection translations (26)78757369

On the public programs, the repair rate of LaDiM is 27.0 points above that of the variant without the diagnosis stage (95% paired bootstrap interval over source files, 14.3 to 39.7) and 6.3 points above that of flat diagnosis (1.6 to 12.7). Table 17 in Appendix J lists the repaired runs of every program under each method, and the intervals resample its 21 source files 10,000 times. On the public attempts, LaDiM uses 3.4% fewer tokens (104.6M versus 108.3M for flat diagnosis). LaDiM has the highest repair rate in every collection, with the smallest gap on the main-collection translations, where MatchFixAgent also repairs all nine failing inputs in Table 1a. While the signal ablation in Table 3a shows which faults each signal exposes, this comparison holds the measurements fixed and varies the diagnosis stage and the organization of the record.

Q2&W2. Wall-clock runtime and computational resources.

We measure wall-clock time for all methods under one job scheduler with equal concurrency, keeping each agent's native controller. These timed runs are additional runs under the configurations of Tables 1a and 2a, and their call and token totals remain within 3% of the reported figures. The table below gives the mean investigation and repair time per program.

CollectionLaDiMMatchFixAgentSWE-agent
Main collection (29 inputs)217 s463 s486 s
Table 2a programs (10)612 s783 s641 s

Wall-clock time is consistent with the generated output tokens and the number of calls. On the main collection, LaDiM makes the fewest calls (Table 1a) and generates the fewest output tokens (Table 5) among the three agents, so it needs 53.1% less time than MatchFixAgent and 55.3% less than SWE-agent. The first saving is close to the 57.4% token saving. On the Table 2a programs, the three agents use similar token totals and LaDiM makes the fewest calls (Appendix D), but each call there carries a longer context, so the three times lie closer together. In these timed runs, LaDiM repairs four of the five failing programs, while SWE-agent and MatchFixAgent repair none, as in Table 2a. All runs use four CPU cores per program, and peak memory is 1.8 GB per evaluation process for every method. Table 20 details the mean, median, and maximum time and the CPU time of each method.

AuthorsOpenReview ↗

Response to Reviewer 7Vqt (2/2): Evaluation Scale, JAX Repair Tasks, and Longer Training

This comment continues our response with W3 and Q3&W4.

W3. Evaluation scale and generality across frameworks.

We agree that the 29 distinct inputs are the right unit for the main collection. Table 1a, Figure 1, Table 7, and the text of the revised paper now report them directly, with LaDiM and MatchFixAgent accepting 29/29 and SWE-agent 25/29. Because token totals already counted each call once, the 57.4% token saving is unchanged. To enlarge the repair evaluation, we translate 45 public PyTorch programs once and each of the 29 main-collection inputs three times in independent requests. Together with Table 2a, this gives the 52 failing translations evaluated in Q1 (their composition is in our comment to all reviewers). For repository scale, we add a third public repository, awesome-semantic-segmentation-pytorch, with 69 code files and 7,827 nonblank lines, almost four times the recommendation repository. All three agents use the full 80 calls of the same repository budget. Starting from the shared initial translation, the evaluation follows the protocol of Appendix E and runs 105 numerical checks and 26 execution, data handling, and boundary behavior checks. In three runs, LaDiM passes 116, 118, and 114 of 131 behavior checks, MatchFixAgent 113, 115, and 112, and SWE-agent 99, 102, and 97, with 12.106M, 12.913M, and 11.706M tokens in the first run. On recommendation, three independent runs give 130, 131, and 130 checks for LaDiM versus 129, 130, and 128 for MatchFixAgent. In Table 2b, both methods repair all six supplied JAX faults, which leaves the Verifier stage no headroom for further repairs while it still adds tokens. Q3 therefore turns to JAX translations that fail without any supplied fault.

Q3&W4. More end-to-end JAX tasks that require repair, and longer training trajectories.

We extend the native JAX study of Appendix H from 12 to 63 public workloads, which include the original twelve. Initial translation passes 44 workloads, three return no code, and 16 translated programs fail evaluation and enter repair. In the first of three runs, LaDiM repairs 9 of these 16 programs, the variant without the diagnosis stage 8, Direct repair 7, SWE-agent 7, and MatchFixAgent 5. LaDiM uses 13.8M investigation and repair tokens in this run versus 7.4M for Direct repair, a ratio of 1.9, and it repairs two more programs. In the separate studies of Table 13 and Table 2b, this ratio is 2.4 and 1.9. In the other two runs, LaDiM repairs 8 and 9 programs versus 8 and 7 for Direct repair (Table 22). Over the three runs, LaDiM completes 26 repairs and Direct repair 22, at 1.59M and 1.01M tokens per successful repair.

For longer trajectories, we train the source and each repaired program for 100 steps from identical initial states, with each side keeping its own state. From the Table 2a and public collections, we take the first accepted candidate of every program that LaDiM repairs in at least one run, and from the JAX collection every program repaired in the first run. This gives four Table 2a programs, 15 public PyTorch programs, and 9 JAX programs. All 28 programs stay within every acceptance threshold at steps 10, 50, and 100 on the three evaluation seeds of their protocol. The table gives the largest difference from the source over these steps and seeds, with the programs that pass before repair as the reference. The MindSpore rows follow the protocol of Appendix D and give the absolute loss difference, the L2 difference of the gradient vector, and the relative parameter update difference. The JAX rows give the largest elementwise difference as a percentage of the tolerance of Appendix H, which adds an absolute term of 10−710^{-7} to a relative term of 10−510^{-5}.

ProgramsCountLossGradientsParameter updates
MindSpore, acceptance threshold0.020.050.03
MindSpore, repaired by LaDiM199.5×10−69.5\times10^{-6}4.6×10−54.6\times10^{-5}3.4×10−43.4\times10^{-4}
MindSpore, pass before repair298.1×10−68.1\times10^{-6}6.1×10−56.1\times10^{-5}2.2×10−42.2\times10^{-4}
JAX, repaired by LaDiM9below 0.1%58%44%
JAX, pass before repair44below 0.1%52%61%

For the JAX programs in float64, the largest loss difference is 2.6×10−92.6\times10^{-9} for the repaired programs and 3.3×10−93.3\times10^{-9} for those that pass before repair. The largest percentages occur on gradient and update entries close to zero, where the absolute term sets the tolerance. Table 23 adds the JAX outputs, parameters, and buffers, which stay below half of the tolerance. Repaired programs and programs that pass before repair thus stay within the same thresholds at steps 10, 50, and 100. Sections 4.2 and 4.3 of the revised paper summarize these results, and Tables 21 and 22 give the third repository and the JAX workloads.

Thank you for the detailed response and the additional experiments. I appreciate the efforts to address my concerns. However, the incremental improvement of layered diagnosis over flat diagnosis remains relatively modest, and the JAX results still raise concerns about the efficiency of the proposed approach. Therefore, while I acknowledge the improvements, I maintain my rating of 2: Weak Rejection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their comments. We have uploaded a revised paper in which text changes are in blue. The main text stays within ten pages, and Appendix J contains the controls, collections, and measurements from the discussion period. The added runs use the same model version as the submission, served by a second provider in November 2026 (Appendix I). Below we list each change with its location and the comments it addresses.

  • Layered and flat diagnosis with identical signals (7Vqt Q1, PJ7R W2, TR5W Q1). Section 4.3 and Tables 15 to 17 compare LaDiM with flat diagnosis, a variant without the diagnosis stage, and Claude Code on the same LLM. On 21 failing translations of public PyTorch programs, LaDiM repairs 38/63 attempts, flat diagnosis 34/63, and the variant without the diagnosis stage 21/63.
  • Scale of the repair evaluation (7Vqt W3, PJ7R W3, TR5W W2). The repair evaluation covers 52 failing translations, with the five of Table 2a, 26 from three translations of each main-collection input, and 21 from 45 public PyTorch programs. The 26 main-collection translations span 12 of the 29 inputs and all differ in code, while the public programs come from PyTorch examples, torchvision, minGPT, makemore, nanoGPT, and pytorch-tutorial. Table 18 classifies all 142 initial translations. Table 21 adds a third public repository, awesome-semantic-segmentation-pytorch, with 69 code files and 7,827 nonblank lines, almost four times the recommendation repository. In three runs under the same repair budget, LaDiM passes 116, 118, and 114 of 131 behavior checks, MatchFixAgent 113, 115, and 112, and SWE-agent 99, 102, and 97. Table 22 extends the native JAX study from 12 to 63 workloads, with three repair runs per method.
  • Unit of the main collection (7Vqt W3, PJ7R Q6). Table 1a, Figure 1, Table 7, and the text report the 29 distinct inputs. LaDiM and MatchFixAgent accept 29/29, and SWE-agent accepts 25/29.
  • Wall-clock time (7Vqt Q2). Section 4.2 and Table 20 report the time of investigation and repair per program.
  • Repeated runs (PJ7R Q5, TR5W W3). Table 19 reports three runs of the comparisons in Tables 1b, 1d, and 3b.
  • Longer trajectories (7Vqt Q3). Table 23 reports 100-step trajectories of the repaired MindSpore and JAX programs.
  • Silent failures, Related Work, and Algorithm 1 (PJ7R Q1, Q2, Q3). Section 1 adds an example and the frequency of silent failures, Related Work clarifies what each line of prior work leaves open, and Algorithm 1 makes the layer assignment explicit.
  • Work-unit plans (PJ7R Q4). Appendix J describes the cycle check on stored plans and compares the final plans with the import structure.
  • Repair history and repository components (TR5W W1). Appendices B and G state the conditions under which each component contributes.
  • Abstract and Conclusion (TR5W W3). The abstract removes the single 80.8% figure and separates the comparisons with a token saving from the one with a coverage gain at comparable token use, and the conclusion names the same settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.