acceptodds
Withdrawn from ICLR 2027

Guided Reasoning Across ISA Dialects: Inference-Time Control for GPU Assembly Translation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

GPU software is increasingly shipped as compiled NVIDIA binaries, while the hardware that runs it is diversifying. Porting such software to another vendor usually requires source code that is often unavailable, or a hand-written binary translator for each instruction set. Large reasoning models can translate native GPU assembly without task-specific training, but not reliably: programs are long, reasoning and translation share one output budget, and the compiler can confirm that a translation is legal but not that it is correct. We treat translation as inference-time control of a stochastic model and present GRIT, which translates the native assembly in an NVIDIA executable into an AMD executable. GRIT splits programs into kernel translation units, grounds the model in context recovered from the binary, repairs rejected candidates with compiler diagnostics, and uses a learned Reasoning Controller to choose effort, context, repair budget, and attempts per program. On 375 programs from three benchmark suites, GRIT raises Gemini 3.8 Flash from 4.0% for whole-program translation to 77.3% in a single attempt, and the controller reaches 88.0% on a held-out test set with repeated attempts. We also find that more reasoning is not free: high effort exhausts the response limit six times as often as medium.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
reasoning models, large language models, program synthesis, code generation, inference-time reasoning, compiler feedback, cross-ISA translation, GPU assembly, binary translation, stochastic sampling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tyQ6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes GRIT, an test-time scaling framework for translating NVIDIA GPU assembly into executable AMD GPU assembly (SASS) without access to CUDA source code or PTX. Rather than fine-tuning a translation model, GRIT improves the model through 1) decomposing programs into kernel translation units, 2) providing executable-derived context, 3) using compiler diagnostics for iterative repair, and 4) learning a reasoning controller that adaptively selects reasoning effort, context, repair budget, and the number of repeated samples. The authors evaluate GRIT on three benchmark suites. The paper shows GRIT can improve end-to-end correctness from 4% to 77% using Gemini 3.8 Flash.

Critical strengths
  • Translating GPU binaries across different hardwares without access to source code is very challenging. The paper explores an interesting application of LLMs and test-time scaling methods for addressing this problem.

  • The empirical results are strong. The improvement from 4% to 77% is quite a lot, and the ablations demonstrate the effectiveness of each component in improving the correctness.

  • The analysis highlights an important limitation of LLMs on long structured outputs.

Critical weaknesses
  • A big challenge here is the verifiers, both for correctness and legality. The paper lacks detailed description of these two. For example, there's a short writeup only in Appendix B about Oracle O, but that's insufficient. The viability of this approach relies on evaluating the correctness quickly and in a robust way.

  • There are so many papers (perhaps many thousands at this point) on test time scaling method, so in terms of novelty from a methodology perspective, it is unclear to me how this stands against the wealth of prior literature.

Questions for authors
  • More explanation of the verifier, both from a functional and systems (legality) perspective.

  • What does it take to apply this approach to other pairs of hardware?

  • The ablations show that compiler-guided repair and kernel-level decomposition lead to the largest performance gains. Could the authors provide more insights into why these two components are particularly effective compared to other techniques?

  • Under a fixed budget, would generating more samples with lower reasoning effort outperform fewer samples with higher reasoning effort?

  • GRIT combines many techniques, including task decomposition, iterative repair, repeated sampling, and adaptive test-time compute allocation. While the ablations show that compiler-guided repair / kernel-level decomposition lead to the largest gains, it would be helpful if the authors could further explain why these two components are particularly effective compared to the others.

2 comments
AuthorsOpenReview ↗

Response to Reviewer tyQ6 (1/2)

We thank the reviewer for the positive assessment and constructive questions, and we ran new experiments for each point below. All numbers use the unit tests described under W1/Q1, on 244 programs (123 CASS-v2, 107 HeCBench and 14 Rodinia). Where we compare settings, we use three repeated runs made with an earlier version of GRIT. These runs sit below GRIT's current single-attempt accuracy of 75.4%.

W1/Q1. How do the legality checks and unit tests work, and how reliable are they?

The two have separate roles. The legality checks run on each candidate during translation, and their messages are the only feedback the model sees. The unit tests run a finished program and compare it with the NVIDIA original. They score our results but take no part in generation. A single attempt is repaired until it passes the legality checks and is then output. Only resampling and the controller use unit tests during translation, to decide whether to try again.

Legality checks. These run per kernel translation unit and need no GPU. GRIT's parser first rejects truncated or malformed answers. LLVM's AMDGPU assembler (llvm-mc) then rejects illegal instructions, registers, modifiers and operands. Some errors pass the assembler but would break at link or run time, so GRIT adds its own checks for undefined labels, reads past the kernel arguments, missing kernels or helper functions, and barriers or atomics that are in the SASS but missing from the translation. Each failure becomes a message naming the check, the function and the faulty instruction, which goes back to the model for repair.

One attempt, up to 3 repairsCASS-v2HeCBenchRodiniaTotal
Accepted by legality checks1199414227
Correct under unit tests1145812184
Accepted and correct96%62%86%81%

Legality is a strong signal for the small CASS-v2 programs but a weaker one for the larger HeCBench programs, which is why we score each result with unit tests.

Unit tests. Rodinia ships with its own verifiers, and we use 14 of its 15 programs. The 15th, myocyte, has its result check disabled in the default build. CASS-v2 and HeCBench do not ship unit tests, and many HeCBench programs print only timings, which say little about whether the kernels are correct. We therefore sorted the other 361 programs, including myocyte, by what their output shows.

  • 230 programs print values computed from the result arrays that the kernels produce and copy back to the host, such as sums, averages, extrema, checksums or a comparison with a CPU result. Their printed output works as a unit test, and with Rodinia's 14 programs they form our 244 programs.
  • 57 programs print only performance numbers but still copy their result arrays back to the host. These arrays could be recorded in an NVIDIA run and compared element by element with the AMD run. We leave this group out of the results for now.
  • 74 programs neither print values computed from the kernels nor copy their results back, so we have no output to compare. We leave them out as well.

For each of the 244 programs, the unit test also fails when the kernel launches are skipped, on both NVIDIA and AMD GPUs. A unit test is only as good as the code it runs, so we also measured instruction coverage. We compiled the programs for sm_120 at -O3 and ran them under NVBit on an RTX 5090. The tests execute 85.5% of the SASS instructions in the launched kernels on average (95% CI 82.9 to 88.0), and the median program executes 100%. A test runs the AMD program with its unmodified host code. Its printed output must match the NVIDIA run after timings and addresses are normalized. One test takes a median of 3.5 seconds, against a median of 141 seconds to translate a program.

AuthorsOpenReview ↗

Response to Reviewer tyQ6 (2/2)

W2. Novelty relative to test-time scaling methods

We agree that each technique has precedent, and we do not claim a new test-time scaling algorithm. Our contribution is a system that makes LLM translation of whole GPU binaries work, and a study of where common test-time scaling assumptions break when the output is long and must be exact.

  • Kernel translation units. Asked to translate a whole program in one response, the same model gets 3.7% of the 244 programs right (4.0% averaged over ten random held-out splits). Splitting each binary into units of one kernel plus the functions it calls raises one-turn accuracy to 28.7%. It also makes repair practical, because each error points to one small unit.
  • More reasoning has a cost. Reasoning and the answer share one output budget. At high effort, 47 of the 230 CASS-v2 and HeCBench programs have a call cut off at the output limit, against 8 at medium effort, and those 47 fail.
  • Legality is the only feedback during generation. There are no per-kernel tests, and correctness is known only after the whole program runs. Even so, assembler and structural feedback raises a single attempt from 28.7% to 62.5%, the largest gain in the paper.
  • The controller reacts to the type of failure. Unlike Snell et al., it does not rely on statistics of the model's own samples. It stops on programs that do not run and resamples the others, which gives higher accuracy than a fixed policy at lower cost (80.4% against 78.7%, at 1.39 against 1.59 times the cost of one translation).

Q2. Applying GRIT to other hardware pairs

GRIT moves to a new target without retraining the model. Only the hardware-specific parts change, and they involve no machine learning. The source side needs a reader for the binary format. The target side needs the assembler, a short description of the target, such as which registers hold which values at launch, and a small runtime. These are written once per target. Adding hardware features the suites lack worked the same way. Textures, indirect calls and tensor cores needed only runtime support and a few target notes. Texture programs translate reliably, while indirect calls and especially tensor cores remain harder. Extending GRIT to more AMD targets, including wave64 GPUs, is our next step. A non-AMD target would also need an assembler with clear error messages, since repair depends on them.

Q3/Q5. Why repair and kernel units help most

Both address the same problem. A translation is long and must be exact, so one wrong instruction can break the whole program.

Repair. Most errors are small and local, such as an instruction written in NVIDIA's style instead of AMD's. In affine, 1 of 85 instructions was wrong. The assembler message points to that instruction, so the model fixes it and keeps the rest, while resampling would rewrite the whole output and could add new errors. With Gemini 3.8 Flash, 80.0% of the units that needed repair were fixed in one repair turn (81.5% in the submission's setup), and a repair call uses far less reasoning than the first call (median 8.9k against 36k reasoning tokens). Repair does need a model that can use feedback, and it gives no gain for CASS-7B.

Kernel units. Each request covers one kernel, so the answer stays short, fewer answers are cut off at the output limit (Fig. 3), and the model can spend more reasoning per instruction. The model also writes only kernel bodies, while GRIT produces the surrounding structure, which is where whole-file answers fail most.

Q4. More low-effort samples or fewer high-effort ones?

More low-effort samples do not help. In one turn without repair, low, medium and high effort reach 3.0%, 28.7% and 36.5%. At low effort Gemini does not produce reasoning tokens, and it solves only 1 of 107 HeCBench programs, so extra low-effort samples add little. At about the cost of one high-effort attempt, two medium-effort samples reach 38.7%, which is not significantly different from 36.5% (p = 0.55). Extra samples also help only when a unit test can tell which one is correct. Repair beats both options, reaching 62.5% in one attempt at about 1.1 times the cost of one high-effort attempt.

We hope these answers address the reviewer's questions, and we would be grateful if the reviewer would reconsider the score in light of them.

Reviewer_N5PyOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents GRIT (Guided Reasoning for ISA Translation), a framework that treats source-free GPU binary translation from NVIDIA SASS to AMD AMDGPU assembly as an inference-time control problem for stochastic reasoning models. By combining Kernel Translation Units (KTUs), executable-derived context, compiler-guided repair using LLVM diagnostics, and a learned Reasoning Controller (RC), GRIT successfully raises whole-program translation accuracy from 4.0% to 77.3% in a single attempt, and up to 88.0% on held-out test sets.

Critical strengths
  1. Innovative application of an inference-time control paradigm for low-level GPU binary translation using general-purpose reasoning models without task-specific fine-tuning.

  2. Effective compiler-guided repair loop using LLVM assembler diagnostics, which resolves 81.5% of rejected translation units in a single turn.

  3. A learned Reasoning Controller that predicts optimal inference configurations (effort, repair budget, context mode, and sample count) from static binary features.

  4. Rigorous evaluation across 375 programs from three diverse benchmark suites.

Critical weaknesses
  1. Proprietary Model Dependency: Empirical evaluations are restricted to a single commercial model (Gemini 3.8 Flash), leaving open-weight model generalization untested.

  2. Oracle Dependency: Final validation and selection rely on an execution-based correctness oracle requiring the original NVIDIA reference environment, which may be unavailable in practical deployment settings.

  3. Lack of Efficiency and Cost Analysis: The paper lacks a comprehensive analysis of wall-clock execution time, monetary API costs, and token overhead compared against traditional or intermediate tools (like ZLUDA or SCALE). Given that multi-turn repairs and high reasoning efforts incur heavy token consumption, a clearer cost-benefit efficiency analysis is necessary.

Questions for authors
  1. How well does the GRIT framework generalize to open-weight reasoning models (e.g., DeepSeek-R1 or Qwen-QwQ) that manage reasoning tokens differently under shared-token budget limits?

  2. In real-world deployments where the original NVIDIA hardware or oracle is missing, how can candidate translations be reliably selected without execution feedback?

  3. Can the authors provide a direct comparison of average wall-clock time and estimated monetary API costs per translated program relative to traditional toolchains like ZLUDA?

2 comments
AuthorsOpenReview ↗

Response to Reviewer N5Py (1/2)

We thank the reviewer for the constructive review and questions. All numbers below come from our updated evaluation on 244 programs (123 CASS-v2, 107 HeCBench and 14 Rodinia).

W1/Q1. Can GRIT run other models, including open-weight ones?

We thank the reviewer for this suggestion. GRIT's pipeline does not depend on a specific model. Context recovery, kernel translation units, legality checks and packaging are deterministic, so switching models only changes the translation call. We tested other Gemini models, proprietary models from other providers and open-weight models, with no change to GRIT.

All Gemini runs below use the same 14 Rodinia programs, compiled for sm_120 at -O3, with up to three repairs.

Gemini modelEffortCorrect of 14
Gemini 3.8 Flashmedium7
Gemini 4 Hydrogenhigh8
Gemini 4 Argonhigh10, 12, 11 (three runs)
Gemini 4 Argonmedium7, 9, 7 (three runs)

Argon also shows the effort effect from the paper. At high effort it passes 33 of 42 runs, against 23 at medium effort.

We also ran GRIT unchanged with 15 models on lud and srad_v1, two of the hardest Rodinia programs, with one attempt each. Eight of these models are open-weight (marked †) and were served through OpenRouter.

OutcomeModels
Both programs translated exactlyGemini 3.8 Flash, GPT-6 Astra, Claude Fable 5.2
lud exact, srad_v1 not finished within our time limitClaude Opus 5.5, GPT-6.1 Sol
lud runs but gives wrong valuesKimi K3.1 Pro†, Mistral Large 4†, Grok 4.7, Muse Wonder
lud still rejected by the assembler after 3 repairsQwen 3.8 (2.4T)†, GLM 5.3†
Output budget used up by reasoning on ludDeepSeek V4 Pro†, DeepSeek V4.1 Flash†, Nemotron 3.5 Super VL†, Dots3 Note Preview†

So GRIT is not tuned to Gemini, and with a capable model it produces exact translations. The open-weight models we tested are limited by their own translation ability rather than by the harness. Their shared budget for reasoning and answer is also tight, and short kernel units help keep each answer within it.

W2/Q2. Selecting a translation without the NVIDIA reference

GRIT does not need an NVIDIA reference to produce a translation. During translation the model only sees feedback from the assembler and our legality checks. A single attempt is repaired until it passes these checks and is then output as is. This translation is correct for 184 of 244 programs (75.4%) without any reference.

Only resampling and the controller use a reference run, to decide whether another attempt is needed. GRIT addresses a porting problem. Someone has an NVIDIA binary and wants to run it on an AMD GPU, so the original binary exists and one recorded run of it is usually easy to obtain. This is the same check a developer would run before relying on any port. When no reference run is possible, GRIT outputs its single checked translation.

AuthorsOpenReview ↗

Response to Reviewer N5Py (2/2)

W3/Q3. Cost and wall-clock time compared with ZLUDA and SCALE

GRIT pays the translation cost once per binary, and the translated program then runs without the model. The costs below apply Gemini 3.8 Flash list prices to the input, reasoning and output tokens we recorded, averaged over ten random held-out splits, each holding out a quarter of the 244 programs.

Method (ten held-out splits)CorrectCost per correct program
Gemini, whole file (baseline)4.0%$4.75
GRIT controller80.4%$1.14

Each whole-file call is cheaper, but only 4.0% of them are correct, so a correct program costs about four times more than with GRIT. Translation takes a median of about 140 to 150 seconds per program for both the baseline and GRIT, and one unit test takes a median of 3.5 seconds.

ZLUDA and SCALE have no per-program model cost. ZLUDA's just-in-time compiler adds a median of 0.1 to 0.2 seconds at first launch, and SCALE compiles a program from source in under one second (median). Neither can run SASS-only binaries, because ZLUDA needs embedded PTX and SCALE needs the CUDA source. GRIT's cost buys portability in the cases these tools cannot cover.

Once translated, GRIT's kernels run close to native speed. On RDNA3, rocprofv3 kernel traces give a median ratio of 1.01 to native HIP -O3 builds (95% CI 1.00 to 1.03).

We hope these results address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the score in light of them.

Reviewer_99woOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper translates native NVIDIA SASS assembly into executable AMD GPU code without CUDA source, PTX, or further training of the reasoning model. It supplies context recovered from the binary and uses compiler diagnostics to guide repairs. A learned controller selects the reasoning effort, context, repair budget, and number of independent translation attempts for each program.

Critical strengths

The paper holds the model fixed and isolates the effects of inference configuration. Using compiler feedback to guide repairs is informative.

Critical weaknesses
  1. The 78.9% pass@3 and 88.0% controller results rely on a correctness oracle, with the controller stopping at the first passing attempt. No mechanism is provided to select a correct translation when reference execution is unavailable. In one HeCBench run, 50 of 153 validated translations still produce incorrect results.
  2. For 360 of 375 programs, correctness is assessed through normalized printed output under fixed inputs. Only the 15 Rodinia programs compare direct result arrays. Agreement on these executions does not establish correctness on other inputs or for results not exposed by the output checks.
  3. All binaries are compiled with -Xptxas -O0 for SM80 or SM89, and the corpus contains no tensor core, texture, surface, or indirect branch instructions. The evaluation does not establish effectiveness on optimized production binaries or test the method’s handling of indirect control flow.
  4. Table 2 combines results over 375 programs with controller results over 92 held-out programs, while also mixing pass@1, pass@3, and adaptive pass@N. The claimed 2.7× improvement compares different populations and inference budgets. The controller’s 100% Rodinia result is based on a single program.
  5. Most experimental conditions are run once, and no confidence intervals are reported. Yet 20.5% of programs, including 36.5% of HeCBench programs, change verdict across runs with identical settings. The robustness of smaller reported gains remains unclear.
  6. Table 6 reports 66.3% for the controller against 58.9% for the best fixed configuration, but the 88.0% headline additionally uses variable sampling. No strong fixed policy or simple size heuristic is evaluated under a matched total token budget on the same held-out programs. The benefit of learned allocation over additional sampling remains unclear.
  7. The reasoning effort and controller findings rely on Gemini 3.8 Flash, and the controller is trained only on that model’s outcomes. No transfer experiments establish whether the learned policy or reasoning budget findings generalize to other reasoning models.
Questions for authors
  1. How would the paper’s method select a final translation when reference execution is unavailable, and what accuracy would that selection achieve? Can incorrect but validated translations be detected without the correctness oracle?
  2. How often do translations that pass the printed output checks fail on additional inputs, boundary cases, or direct comparisons on result arrays? Can the CASS-v2 and HeCBench checks be extended to compare device result arrays?
  3. How does the paper’s technique perform on optimized binaries and kernels containing tensor core or indirect branch instructions? How does KTU construction handle indirect calls and unresolved call targets?
  4. What is the composition of the test set with 92 programs? Can the main methods in Table 2 be compared on this same set with matched budgets of one and three attempts per program?
  5. Could the authors repeat the main configurations across multiple independent runs and report confidence intervals and paired comparisons? How stable are the smaller reported accuracy gains?
  6. Does the controller outperform a fixed policy with rich context, three repairs and a simple size heuristic, at equal token budgets on the same test set? Please report costs without using the oracle to stop early.
  7. Do the reasoning effort findings hold for other reasoning models? How well does the controller trained on Gemini transfer to another model without retraining, compared with a controller trained separately for each model?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 99wo (1/3)

We thank the reviewer for the careful review and ran the experiments the reviewer asked for. A program counts as correct only if it passes its unit test, meaning fixed inputs with expected outputs (W2/Q2). All numbers below come from our updated evaluation on 244 programs.

W1/Q1. Selecting a translation without reference execution

GRIT does not run unit tests while it translates. The model only sees feedback from the assembler and our structural checks. These checks show that the code is legal for the AMD target, but not that it is correct. This is why some translations pass them and still give wrong results, as the reviewer points out, and why we do not count a translation as correct on the checks alone.

A single translation with up to three repairs needs no reference, and it passes the unit tests for 184 of 244 programs (75.4%). Picking among several samples does need a test. GRIT uses the program's own unit test for this, similar to how a developer checks a port on known inputs before relying on it. Stopping at the first pass only saves cost. Producing N translations first and testing them afterwards gives the same accuracy at a higher cost.

W2/Q2. How strong are the unit tests?

Rodinia ships with its own verifiers, and we use 14 of its 15 programs. The 15th, myocyte, has its result check disabled in the default build. CASS-v2 and HeCBench do not ship unit tests, and many HeCBench programs print only timings, which say little about whether the kernels are correct. We therefore sorted the other 361 programs, including myocyte, by what their output shows.

  • 230 programs print values computed from the result arrays that the kernels produce and copy back to the host, such as sums, averages, extrema, checksums or a comparison with a CPU result. Their printed output works as a unit test, and with Rodinia's 14 programs they form our 244 programs.
  • 57 programs print only performance numbers but still copy their result arrays back to the host. These arrays could be recorded in an NVIDIA run and compared element by element with the AMD run. We leave this group out of the results for now.
  • 74 programs neither print values computed from the kernels nor copy their results back, so we have no output to compare. We leave them out as well.

For each of the 244 programs, the unit test also fails when the kernel launches are skipped, on both NVIDIA and AMD GPUs. A unit test is only as good as the code it runs, so we also measured instruction coverage. We compiled the programs for sm_120 at -O3 and ran them under NVBit on an RTX 5090. The tests execute 85.5% of the SASS instructions in the launched kernels on average (95% CI 82.9 to 88.0), and the median program executes 100%.

Under these unit tests and with up to three translations per program, GRIT translates 202 of the 244 programs correctly. CASS, the only other method that works from SASS alone, translates 60, and 57 of them come from CASS-v2.

SuiteGRITCASS
CASS-v2 (123)117 (95.1%)57 (46.3%)
HeCBench (107)73 (68.2%)3 (2.8%)
Rodinia (14)12 (85.7%)0 (0.0%)
Total (244)202 (82.8%)60 (24.6%)
AuthorsOpenReview ↗

Response to Reviewer 99wo (2/3)

W3/Q3. Optimized binaries, special instructions and indirect control flow

We recompiled all 244 programs for Blackwell (sm_120) with ptxas -O3 and translated them to RDNA3 with Gemini 3.8 Flash. With one translation per program, GRIT passes 172 programs at -O3 against 181 at -O0. Optimized kernels are denser, so they are harder to translate.

The 375 programs do not contain tensor-core, texture or surface instructions in their own kernels. Such work usually runs inside libraries like cuBLAS, which GRIT forwards to the AMD equivalent. We therefore tested the 14 programs below, which use these instructions, from sm_120 -O3 to RDNA3 with Gemini 3.8 Flash and three translations each. Their unit tests compare the result arrays with an NVIDIA run, element by element. Supporting them needed small additions to GRIT's prompt notes and runtime, with the same model.

Program (suite)Exercises
simpleTexture, simpleLayeredTexture, simplePitchLinearTexture, simpleCubemapTexture (CUDA Samples), 4bba410b48af (CASS)2D, layered, pitch-linear and cubemap textures
simpleSurfaceWrite (CUDA Samples)surface write, texture read
simpleSeparateCompilation, newdelete (CUDA Samples)function-pointer and virtual calls
wmma-cuda (HeCBench), cudaTensorCoreGemm, immaTensorCoreGemm (CUDA Samples), two CUTLASS examplesfp16, int8, fp64 and tf32 tensor cores
DeepBench GEMM (cuBLAS)library call forwarded to rocBLAS

Texture and surface programs translate reliably. Indirect calls pass in some translations but not all. Tensor cores are the hardest case. The fp16, int8 and fp64 kernels pass in some translations, and the tf32 kernel failed in all three.

With an indirect call, the target is only known at run time, for example a virtual function picked through a vtable. When GRIT builds kernel units, it collects the device functions whose addresses are stored in the binary, because only those can be called indirectly. A unit with an indirect call receives this set, so each possible target is translated together with the call. Jump tables, which the suites already contain as compiled switch statements, stay inside one function, so the existing kernel units already cover them.

W4/Q4. Same programs and matched budgets

The held-out set in the paper has 92 programs (48 CASS-v2, 43 HeCBench, 1 Rodinia). A single split is fragile, so we now use ten random splits that each hold out a quarter of the 244 programs, and each method runs on the same held-out programs. The baseline gives the same model the whole SASS file and the original host code. The table below replaces the 2.7× comparison.

Method (mean of 10 splits)Translations per programCorrect
Gemini, whole file14.0%
GRIT, best fixed configuration173.3%
Same, up to 2 translations1.378.7%
GRIT controller1.380.4%

At the same budget of 1.3 translations, the controller is 1.7 points ahead of the fixed configuration (95% CI 1.1 to 2.3) and is ahead in 10 of 10 splits. Most of the gain over the whole-file baseline already appears with a single GRIT translation, so it comes mainly from the pipeline rather than from extra samples.

Once both inputs include the program's initialized device data, the compact and rich inputs perform the same. On the same 39 programs, the compact input gets 32 to 34 right across three runs and the rich input gets 33. GRIT therefore uses the cheaper compact input, and the controller only picks the reasoning effort and the repair budget.

AuthorsOpenReview ↗

Response to Reviewer 99wo (3/3)

W5/Q5. Stability and paired tests

Re-running a finished translation does not add noise. We ran 135 translations three times each, and each one gave the same result in all three runs. Translation itself does vary. Three full rounds over the 244 programs, run before the device-data fix in W4/Q4, pass 62.3%, 62.7% and 63.1%. This spread is small next to the 95% interval from resampling programs (56.5 to 68.5%). 127 programs pass in all three rounds, 70 fail in all three and 47 pass in one or two, and those 47 are where extra samples help. The first three comparisons below use these rounds.

Paired comparison, same programsDifference (points)95% CIp (exact McNemar)
Whole file to GRIT units, one turn without repair+25.219.1 to 31.3<0.0001
Without repair to up to 3 repairs+33.527.0 to 40.0<0.0001
Medium to high effort, one turn+7.81.7 to 13.90.02
Fixed (up to 2) to controller, same budget+1.71.1 to 2.310 of 10 splits

W6/Q6. Controller against a fixed policy at equal cost

The fixed policy is the best single configuration (medium effort, three repairs), chosen on the training programs, and it retries whenever a translation fails. Cost comes from recorded tokens, relative to one fixed translation.

Policy (mean of 10 splits)TranslationsRelative costCorrect
Fixed, 1 translation1.001.0073.3%
Fixed, up to 21.271.5978.7%
Fixed, up to 31.482.1181.6%
GRIT controller, up to 31.301.3980.4%

The controller stops after a translation that does not even produce a running program, since more attempts rarely fix those. It costs less than the fixed policy with two translations and scores 1.7 points higher. The fixed policy needs a third translation and 52% more cost to pass it by 1.2 points. A tuned size heuristic reaches 73.1%, since the inputs it chooses between now perform the same. Without stopping on a pass, the controller still reaches 80.4% at a relative cost of 2.00, against 2.74 for a fixed three translations per program. On Gemini 3.8 Flash the controller ends up picking medium effort with three repairs, so its gain comes from its rules on when to try again and when to stop.

W7/Q7. Can GRIT run other models, and does the controller transfer?

We thank the reviewer for this suggestion. GRIT runs other models with no change to the pipeline, and we tested other Gemini models, proprietary models from other providers and open-weight models.

All Gemini runs below use the same 14 Rodinia programs, compiled for sm_120 at -O3, with up to three repairs.

Gemini modelEffortCorrect of 14
Gemini 3.8 Flashmedium7
Gemini 4 Hydrogenhigh8
Gemini 4 Argonhigh10, 12, 11 (three runs)
Gemini 4 Argonmedium7, 9, 7 (three runs)

Argon also shows the effort effect from the paper. At high effort it passes 33 of 42 runs, against 23 at medium effort.

We also ran GRIT unchanged with 15 models on lud and srad_v1, two of the hardest Rodinia programs, with one attempt each. Eight of these models are open-weight (marked †) and were served through OpenRouter.

OutcomeModels
Both programs translated exactlyGemini 3.8 Flash, GPT-6 Astra, Claude Fable 5.2
lud exact, srad_v1 not finished within our time limitClaude Opus 5.5, GPT-6.1 Sol
lud runs but gives wrong valuesKimi K3.1 Pro†, Mistral Large 4†, Grok 4.7, Muse Wonder
lud still rejected by the assembler after 3 repairsQwen 3.8 (2.4T)†, GLM 5.3†
Output budget used up by reasoning on ludDeepSeek V4 Pro†, DeepSeek V4.1 Flash†, Nemotron 3.5 Super VL†, Dots3 Note Preview†

Controller transfer, replayed offline on Argon's runs. The stopping rules transfer as they are. The learned choice does not, because the controller trained on Gemini 3.8 Flash learned to choose the input, while Argon needs the right effort level. Retraining it took one medium and one high effort run on 10 programs.

Policy on ArgonTranslationsCorrect
Best fixed setting (high effort, 3 repairs)178.6%
Controller trained on Gemini 3.8 Flash, unchanged1.663.5%
Controller retrained to also pick effort1.288.1%
High effort with the controller's rules only1.188.1%

We hope these results address the soundness concerns, and we would be grateful if the reviewer would reconsider the scores in light of them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.