- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes GRIT, an test-time scaling framework for translating NVIDIA GPU assembly into executable AMD GPU assembly (SASS) without access to CUDA source code or PTX. Rather than fine-tuning a translation model, GRIT improves the model through 1) decomposing programs into kernel translation units, 2) providing executable-derived context, 3) using compiler diagnostics for iterative repair, and 4) learning a reasoning controller that adaptively selects reasoning effort, context, repair budget, and the number of repeated samples. The authors evaluate GRIT on three benchmark suites. The paper shows GRIT can improve end-to-end correctness from 4% to 77% using Gemini 3.8 Flash.
Critical strengths
-
Translating GPU binaries across different hardwares without access to source code is very challenging. The paper explores an interesting application of LLMs and test-time scaling methods for addressing this problem.
-
The empirical results are strong. The improvement from 4% to 77% is quite a lot, and the ablations demonstrate the effectiveness of each component in improving the correctness.
-
The analysis highlights an important limitation of LLMs on long structured outputs.
Critical weaknesses
-
A big challenge here is the verifiers, both for correctness and legality. The paper lacks detailed description of these two. For example, there's a short writeup only in Appendix B about Oracle O, but that's insufficient. The viability of this approach relies on evaluating the correctness quickly and in a robust way.
-
There are so many papers (perhaps many thousands at this point) on test time scaling method, so in terms of novelty from a methodology perspective, it is unclear to me how this stands against the wealth of prior literature.
Questions for authors
-
More explanation of the verifier, both from a functional and systems (legality) perspective.
-
What does it take to apply this approach to other pairs of hardware?
-
The ablations show that compiler-guided repair and kernel-level decomposition lead to the largest performance gains. Could the authors provide more insights into why these two components are particularly effective compared to other techniques?
-
Under a fixed budget, would generating more samples with lower reasoning effort outperform fewer samples with higher reasoning effort?
-
GRIT combines many techniques, including task decomposition, iterative repair, repeated sampling, and adaptive test-time compute allocation. While the ablations show that compiler-guided repair / kernel-level decomposition lead to the largest gains, it would be helpful if the authors could further explain why these two components are particularly effective compared to the others.
2 comments
Response to Reviewer tyQ6 (2/2)
W2. Novelty relative to test-time scaling methods
We agree that each technique has precedent, and we do not claim a new test-time scaling algorithm. Our contribution is a system that makes LLM translation of whole GPU binaries work, and a study of where common test-time scaling assumptions break when the output is long and must be exact.
- Kernel translation units. Asked to translate a whole program in one response, the same model gets 3.7% of the 244 programs right (4.0% averaged over ten random held-out splits). Splitting each binary into units of one kernel plus the functions it calls raises one-turn accuracy to 28.7%. It also makes repair practical, because each error points to one small unit.
- More reasoning has a cost. Reasoning and the answer share one output budget. At high effort, 47 of the 230 CASS-v2 and HeCBench programs have a call cut off at the output limit, against 8 at medium effort, and those 47 fail.
- Legality is the only feedback during generation. There are no per-kernel tests, and correctness is known only after the whole program runs. Even so, assembler and structural feedback raises a single attempt from 28.7% to 62.5%, the largest gain in the paper.
- The controller reacts to the type of failure. Unlike Snell et al., it does not rely on statistics of the model's own samples. It stops on programs that do not run and resamples the others, which gives higher accuracy than a fixed policy at lower cost (80.4% against 78.7%, at 1.39 against 1.59 times the cost of one translation).
Q2. Applying GRIT to other hardware pairs
GRIT moves to a new target without retraining the model. Only the hardware-specific parts change, and they involve no machine learning. The source side needs a reader for the binary format. The target side needs the assembler, a short description of the target, such as which registers hold which values at launch, and a small runtime. These are written once per target. Adding hardware features the suites lack worked the same way. Textures, indirect calls and tensor cores needed only runtime support and a few target notes. Texture programs translate reliably, while indirect calls and especially tensor cores remain harder. Extending GRIT to more AMD targets, including wave64 GPUs, is our next step. A non-AMD target would also need an assembler with clear error messages, since repair depends on them.
Q3/Q5. Why repair and kernel units help most
Both address the same problem. A translation is long and must be exact, so one wrong instruction can break the whole program.
Repair. Most errors are small and local, such as an instruction written in NVIDIA's style instead of AMD's. In affine, 1 of 85 instructions was wrong. The assembler message points to that instruction, so the model fixes it and keeps the rest, while resampling would rewrite the whole output and could add new errors. With Gemini 3.8 Flash, 80.0% of the units that needed repair were fixed in one repair turn (81.5% in the submission's setup), and a repair call uses far less reasoning than the first call (median 8.9k against 36k reasoning tokens). Repair does need a model that can use feedback, and it gives no gain for CASS-7B.
Kernel units. Each request covers one kernel, so the answer stays short, fewer answers are cut off at the output limit (Fig. 3), and the model can spend more reasoning per instruction. The model also writes only kernel bodies, while GRIT produces the surrounding structure, which is where whole-file answers fail most.
Q4. More low-effort samples or fewer high-effort ones?
More low-effort samples do not help. In one turn without repair, low, medium and high effort reach 3.0%, 28.7% and 36.5%. At low effort Gemini does not produce reasoning tokens, and it solves only 1 of 107 HeCBench programs, so extra low-effort samples add little. At about the cost of one high-effort attempt, two medium-effort samples reach 38.7%, which is not significantly different from 36.5% (p = 0.55). Extra samples also help only when a unit test can tell which one is correct. Repair beats both options, reaching 62.5% in one attempt at about 1.1 times the cost of one high-effort attempt.
We hope these answers address the reviewer's questions, and we would be grateful if the reviewer would reconsider the score in light of them.
Response to Reviewer tyQ6 (1/2)
We thank the reviewer for the positive assessment and constructive questions, and we ran new experiments for each point below. All numbers use the unit tests described under W1/Q1, on 244 programs (123 CASS-v2, 107 HeCBench and 14 Rodinia). Where we compare settings, we use three repeated runs made with an earlier version of GRIT. These runs sit below GRIT's current single-attempt accuracy of 75.4%.
W1/Q1. How do the legality checks and unit tests work, and how reliable are they?
The two have separate roles. The legality checks run on each candidate during translation, and their messages are the only feedback the model sees. The unit tests run a finished program and compare it with the NVIDIA original. They score our results but take no part in generation. A single attempt is repaired until it passes the legality checks and is then output. Only resampling and the controller use unit tests during translation, to decide whether to try again.
Legality checks. These run per kernel translation unit and need no GPU. GRIT's parser first rejects truncated or malformed answers. LLVM's AMDGPU assembler (llvm-mc) then rejects illegal instructions, registers, modifiers and operands. Some errors pass the assembler but would break at link or run time, so GRIT adds its own checks for undefined labels, reads past the kernel arguments, missing kernels or helper functions, and barriers or atomics that are in the SASS but missing from the translation. Each failure becomes a message naming the check, the function and the faulty instruction, which goes back to the model for repair.
Legality is a strong signal for the small CASS-v2 programs but a weaker one for the larger HeCBench programs, which is why we score each result with unit tests.
Unit tests. Rodinia ships with its own verifiers, and we use 14 of its 15 programs. The 15th, myocyte, has its result check disabled in the default build. CASS-v2 and HeCBench do not ship unit tests, and many HeCBench programs print only timings, which say little about whether the kernels are correct. We therefore sorted the other 361 programs, including myocyte, by what their output shows.
For each of the 244 programs, the unit test also fails when the kernel launches are skipped, on both NVIDIA and AMD GPUs. A unit test is only as good as the code it runs, so we also measured instruction coverage. We compiled the programs for sm_120 at -O3 and ran them under NVBit on an RTX 5090. The tests execute 85.5% of the SASS instructions in the launched kernels on average (95% CI 82.9 to 88.0), and the median program executes 100%. A test runs the AMD program with its unmodified host code. Its printed output must match the NVIDIA run after timings and addresses are normalized. One test takes a median of 3.5 seconds, against a median of 141 seconds to translate a program.