acceptodds
Under review as a conference paper at ICLR 2027

RVVCoder: ISA-Grounded LLMs for High-Performance RISC-V Vector Kernel Generation

Abstract

Open RISC-V hardware is spreading rapidly, yet every new vector core needs efficient kernels, and hand-writing RISC-V Vector (Rvv) assembly is costly. Rvv is vector-length agnostic, so a kernel must stay correct across vector lengths and fast on each core, which challenges both compilers and large language models (LLMs). We formalize Rvv kernel generation under instruction-set architecture (ISA) and hardware constraints and propose RvvCoder. RvvBench pairs RvvCoT, a specification-guided corpus, with a 38-operator suite verified across vector lengths and timed on real boards. RvvCoder learns ISA rules by supervised fine-tuning, then improves by reinforcement learning on kernels executed in QEMU and on hardware, where Decomposed Execution Verification (DEV) rewards partial progress. RvvCoder reaches 59% ER@10 correctness versus 13% for GPT-5.5, and its verified kernels run up to faster than the strongest applicable baselines on two Rvv boards, confirming the value of hardware feedback.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Large Language Models, Code Generation, Program Synthesis, RISC-V Vector Extension, Kernel Generation, Execution-Grounded Reinforcement Learning, Hardware-Aware Optimization, Instruction Set Architecture

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mn2COpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies RVV assembly kernel generation using Qwen3-14B. It first applies supervised fine-tuning on the four-layer RvvCoT corpus, followed by GRPO with Decomposed Execution Verification (DEV). DEV assigns partial credit through operator-specific diagnostic tests. The authors report an average ER@10 of 59% on a benchmark covering 38 operator types and real-hardware performance results for seven operators on two boards.

Critical strengths
  1. The RVV setting poses concrete challenges. RvvBench distinguishes compilation success from functional correctness and checks each candidate under three VLEN settings. This directly tests whether generated code depends on a fixed vector length.
  2. The diagnostic examples make DEV easy to understand. Table 13 separates SGEMM coverage, loading, accumulation, and addressing checks, while Figure 10 illustrates relevant code failures. These examples show why two kernels that fail the full harness might still deserve different feedback.
Critical weaknesses
  1. The current comparisons do not isolate the additional benefit of DEV’s semantic decomposition. Figure 5 reports 59% average ER@10 with DEV, compared with 30% using a whole-kernel reward. This is an interesting gain, but the paper does not directly compare DEV with an ordinary test-pass-fraction reward. DEV’s operator-specific semantic tests are not identical to these approaches. The current evidence does not establish whether the benefit comes from that semantic decomposition or from providing finer-grained test feedback. This distinction is central to assessing the method’s contribution.
  2. Key data and experimental details behind the main gains are missing. Appendix B describes corpus construction and filtering, but the final corpus size and composition, the actual number of evaluation tasks, and the exact training settings behind the main results are not reported in sufficient detail. The 38 operators are operator types, not task counts, and Table 11 gives ranges for several hyperparameters. The main ablation also lacks reported independent training repetitions or uncertainty estimates. These omissions do not show that the results are wrong, but they make it difficult to assess the supervision and evaluation conditions behind the gains and their stability. Since the corpus and reward design are central contributions, these gaps materially affect my recommendation.
Questions for authors
  1. Clarification of the contribution: Do you have an existing comparison between DEV and an ordinary test-pass-fraction reward, or another analysis that separates their effects? Please clarify whether the starting checkpoint and the training and verification budgets are comparable, and what this evidence establishes about semantic decomposition itself.
  2. Clarification of the experimental setup: What are the per-layer sample counts, mixture weights, teacher identities, and training/validation/test split sizes for RvvCoT? How many shape–hardware tasks are in the main evaluation, and what exact training configurations produced the results in Table 1 and Figure 5?
  3. Clarification of result stability: How many independent training runs underlie the 59% and 30% results in Figure 5? Are there existing repeated-run results or uncertainty estimates showing how this gain varies across training runs?
2 comments

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


Q1. DEV vs. ordinary test-pass-fraction rewards

Thank you for this important question. DEV’s distinguishing design is semantically structured execution feedback; its incremental learning benefit over ordinary dense feedback remains unestablished.

1. Methodological distinction.

Conventional test-pass-fraction rewards count successful end-to-end tests, whereas DEV evaluates executable semantic obligations. For RVV kernels, these obligations include output coverage, pointer stepping, vector loads, accumulation, and reductions. A candidate may satisfy an intermediate obligation while failing complete numerical verification. DEV can therefore distinguish failure modes that ordinary full-output tests treat identically.

RewardFeedback mechanism
BinaryWhole-kernel pass/fail
Test-pass fractionPassed end-to-end tests
DEV (ours)Executable semantic partial progress

2. Empirical evidence.

Our original experiments report:

MethodAvg. ER@10
ISA-grounded SFT34%
GRPO + binary reward30%
GRPO + DEV59%

Notably, binary-reward GRPO does not improve upon SFT, whereas DEV achieves +25 percentage points over SFT and +29 points over binary-reward GRPO. These results highlight the importance of our reward design.

3. Supplemental investigation and validation.

We corrected the supplemental reward implementation in a separate working copy while preserving DEV's diagnostic definitions and weights. Both comparator arms now use identical compilation shaping, differing only in the source of their progress scores.

ValidationResult
Regression test reproducing the missing compilation rewardPassed
Reward replay against preserved execution logsPassed
Compilation failure, partial credit, crash, timeout, and normalization testsPassed
Native C harness validation with correct and incorrect controls84 executions completed
Overall test suite16 passed, 2 PyTorch tests skipped

In summary, DEV provides semantically structured execution feedback, and the original experiments report substantial gains over binary rewards.


Q2. Corpus and experimental reproducibility

Thank you for emphasizing reproducibility. We recovered exact supplemental configurations:

1. Structured supervision.

RvvCoT contains four complementary layers: ISA/ABI rules, verified kernels, failure repairs, and hardware-performance reasoning, addressing distinct challenges of RVV generation.

2. Recovered experimental details.

Supplemental configurationValue
Source / filtered corpus rows964 / 611
Train / validation split581 / 30
Length-valid training sequences331
BackboneQwen3-14B
SFT steps / learning rate100 / 8×10−78 \times 10^{-7}
RL steps / group size60 / 4
RL learning rate / seed3×10−73 \times 10^{-7} / 11
KL coefficient / clipping0.05 / 0.2
Recovered evaluation tasks30 (19 L1, 8 L2, 3 L3)

Overlength training targets were excluded rather than truncated, preserving complete target sequences.


Q3. Statistical stability across independent runs

Thanks for your emphasis on reliable performance estimates.

  • Reported improvement. The original experiments report 59% versus 30% Avg. ER@10, a substantial 29-percentage-point difference between DEV and binary-reward GRPO.
  • Available stability evidence. The supplemental experiment used one seed (11) per condition. The number of independent runs underlying the original comparison remains unverified.
  • Statistical interpretation. We agree that multiple generated candidates or task-level bootstrapping cannot substitute for independent training repetitions. We therefore report the original point estimates without claiming independently verified cross-seed robustness.

Thank you again for identifying these important distinctions. We hope the clarified contribution, explicit implementation findings, and narrower interpretation help you reassess the work. We have carefully considered all your suggestions and we would be deeply grateful if you could kindly reconsider raising the score to 3 or above. Thank you very much!

Reviewer_mn2COpenReview ↗

Official Comment by Reviewer mn2C

Thank you for the clarification. The additional checks concern implementation correctness, but they do not establish DEV’s learning benefit over ordinary test-pass-fraction rewards. It also remains unclear how the recovered configuration relates to the original 59% versus 30% results, and whether the reward correction affects that comparison. The new material therefore does not resolve the main concerns behind my review, and I maintain my original recommendation.

Reviewer_NcGqOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a new task focused on generating correct and efficient operators for RISC-V hardware. It presents two main components: RvvCoder, a framework for code generation, and RvvBench, a benchmark comprising 38 operators and the RvvCoT dataset. Experimental results show that RvvCoder outperforms GPT-5.5.

Critical strengths
  1. Interesting Topic. The paper explores a relatively underexplored area of LLM-based code generation for RVV hardware and introduces a dedicated benchmark for evaluating correctness and performance.

  2. Good Illustration. Figure 1 provides a clear and intuitive illustration of the problem and the proposed approach.

Critical weaknesses
  1. Limited methodological novelty. The proposed approach appears to rely primarily on dataset construction and model fine-tuning. I would expect more methodological contributions beyond applying an established fine-tuning pipeline to a new hardware target.

  2. Limited evaluation of optimization capabilities. I am not convinced that evaluating LLMs under a fixed generation budget adequately reflects their practical kernel optimization capabilities. I would encourage the authors to consider agentic kernel optimization, where models can iteratively generate, test, and refine kernels based on execution feedback. This would provide a more realistic assessment of their optimization potential.

  3. Outdated baselines. GPT-5.5 is no longer the latest available model. The authors should consider including more recent and stronger baselines, such as GPT-5.6 Sol, which was released approximately three months ago.

Questions for authors
  1. Unclear description in the introduction. In line 80, the sentence "RvvBench: To supply supervision and evaluation, specification-guided synthesis builds RvvCoT..." is difficult to follow. It is unclear what the sentence's subject is and how RvvBench relates to constructing RvvCoT. Please clarify how these components relate.

  2. Confusing organization of contributions. The paper's presentation of RvvCoder, RvvBench, and RvvCoT is somewhat confusing. Based on the introduction, I would expect these to be three distinct contributions. However, the paper presents RvvCoT as part of RvvBench. The authors should clarify the relationships among these components and organize the contributions more consistently.

2 comments

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


W1. Methodological novelty beyond fine-tuning

Thank you for this question. Our contribution is not a new SFT or GRPO algorithm, but an ISA-grounded generation problem and an executable-feedback framework for learning to produce correct, portable RVV assembly.

Unlike higher-level kernel generation, RVV assembly directly exposes several coupled requirements:

RequirementCore challenge
ISA/ABI complianceLegal instructions, register constraints, calling conventions
Functional correctnessCompilable kernels may compute incorrect results
VLEN portabilityRuntime-dependent vector lengths, tails, and strip-mining
Hardware efficiencyPerformance varies across physical architectures

We address these through three complementary contributions:

  1. RvvCoT: Four-layer supervision spanning ISA rules, verified kernels, repairs, and hardware-performance reasoning.
  2. RvvBench: An evaluation framework combining compilation, cross-VLEN correctness, and physical-board timing.
  3. RvvCoder with DEV: ISA-grounded policy learning with executable, operator-specific partial feedback.

DEV's distinguishing design is semantic decomposition. Instead of assigning credit solely for successful complete-output tests, it evaluates intermediate obligations such as coverage, pointer progression, accumulation, and reductions. This exposes potentially useful structural information even when a candidate fails whole-kernel verification.

ConfigurationAvg. ER@10
Qwen3-14B3%
RvvCoder-SFT34%
GRPO + whole-kernel reward30%
Full RvvCoder59%

These reported results support the value of the evaluated RVV-grounded pipeline and reward configuration. In particular, the whole-kernel GRPO variant does not improve on SFT, underscoring that adding RL alone is insufficient.


W2. Fixed-budget generation vs. agentic optimization

Thank you for the insightful question. We view them as complementary research questions rather than interchangeable evaluation settings.

DimensionOur evaluationAgentic optimization
GoalLearn a strong generation policyIteratively improve a candidate
InferenceFixed-budget samplingGenerate–test–refine
FeedbackPrimarily training-timePrimarily inference-time
CapabilityISA-grounded direct generationInteractive debugging and optimization

Fixed-budget sampling measures whether the model has internalized the ISA, ABI, and VLEN constraints sufficiently to generate verified kernels without relying on repeated inference-time repairs.

An agentic optimizer instead measures how effectively a system exploits iterative compiler and execution feedback. A strong generation policy could provide a useful initialization for such an optimizer.


W3. Updated frontier baselines

Thank you for pointing this out. We agree that newer coding models would strengthen the evaluation. The original comparison reports 59% Avg. ER@10 for RvvCoder versus 13% for the evaluated GPT-5.5 baseline. This is evidence relative to the evaluated models, not a claim that GPT-5.5 remains the strongest currently available model.


Q1–Q2. Relationship among the three components

Thank you for pointing this out. We appreciate the opportunity to make the novelty and scope substantially clearer:

  • RvvCoT is an ISA-grounded supervision corpus containing rules, verified kernels, repairs, and performance reasoning. 
  • RvvBench is the evaluation suite measuring compilation, cross-VLEN correctness, and real-hardware efficiency. 
  • RvvCoder is the generation framework that learns from RvvCoT through supervised fine-tuning and execution-grounded reinforcement learning with DEV, and is evaluated using RvvBench.

Thus, RvvCoT provides supervision, RvvCoder learns the generation policy, and RvvBench measures its capabilities. We will distinguish the respective data construction, splits, and evaluation populations.

In summary, our contribution is an ISA-grounded, execution-verifiable framework for RVV assembly generation. The original results show substantial gains under the evaluated protocol, while the agentic comparison, newer baselines, and DEV-versus-dense-reward distinction remain important limitations.


Thank you again for identifying these important distinctions. We hope the clarified contribution, explicit implementation findings, and narrower interpretation help you reassess the work. We have carefully considered all your suggestions and we would be deeply grateful if you could kindly reconsider raising the score to 3 or above. Thank you very much!

Thank you for your response, but I remain unconvinced. My concern about newer models being able to do a lot more and the limit in the budget have not been adequately addressed. I am maintaining my score.

Reviewer_qo32OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RVVCoder, a framework that uses LLMs to generate high-performance RVV assembly kernels. The authors introduce RVVBench, which includes an ISA-aware training dataset RVVCoT and a benchmark covering 38 operators. RVVCoder first uses RVVCoT for SFT, followed by reinforcement learning with Decomposed Execution Verification and hardware feedback. The results show a substantial improvement in correctness over frontier LLMs, achieving 59% ER@10 compared to 13% for GPT-5.5.

Critical strengths
  1. RVV assembly generation is a relatively underexplored area. Generating efficient kernels while satisfying ISA constraints, ABI requirements, and functional correctness is a challenging and meaningful task for LLM-based code generation.

  2. DEV breaks down kernel correctness into smaller, executable checks, providing a practical way to address the sparse reward problem in reinforcement learning.

Critical weaknesses
  1. Hardware-in-the-loop optimization is one of the key motivations of RvvCoder. However, the ablation studies mainly focus on SFT, GRPO, and DEV. They do not clearly separate the benefits of real-hardware performance rewards from those of QEMU-based correctness feedback.

  2. The paper does not sufficiently discuss how well RVVCoder generalizes beyond its training distribution, e.g. unseen operator families or new hardware platforms.

Questions for authors
  1. Could the authors add an ablation that keeps SFT, GRPO, and DEV but removes the real-hardware performance reward? I would be interested to see how much hardware feedback actually contributes to the reported speedups.

  2. How were the 7 of 38 operators in Figure-9 selected? Are they representative of the full benchmark?

  3. DEV requires manually designed checks for different operator families. How much effort does it take to build these checks? Also, how strongly does progress on the individual DEV checks correlate with the final correctness of the generated kernel?

  4. Could DEV be applied to other code generation tasks, such as CUDA kernel generation? It would be interesting to know whether the approach is specific to RVV assembly or could work in a broader setting.

2 comments

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


Q1. Independent contribution of real-hardware rewards

Thank you for this question. RVVCoder separates two complementary objectives: generating correct RVV kernels and improving the runtime of verified kernels on physical hardware.

Feedback sourcePrimary roleWhat it establishes
Compilation and linkingISA/ABI feasibilityWhether a candidate can be built
QEMU executionFunctional verificationWhether it passes configured correctness tests
DEV diagnosticsSemantic partial progressWhich executable obligations are satisfied
Physical-board timingPerformance evaluationRuntime on a particular processor

This distinction is important because QEMU verification alone cannot establish physical performance improvements. Our paper reports physical measurements on two RVV platforms:

PlatformGeoMean speedupMaximum speedup
SpacemiT K1/X601.27×1.49×
Sophgo SG2044/C9201.17×1.31×

These results demonstrate the reported performance of verified generated kernels relative to the strongest applicable baselines on the evaluated workloads and platforms.


Q2. Selection of seven hardware-evaluation operators

Thank you for pointing this out. The physical performance evaluation includes seven workloads:

OperatorMain computational characteristic
SGEMMDense matrix computation and reuse
GEMVMatrix–vector computation
GEMM+ReLUFused computation
AttentionComposite neural-network computation
SoftmaxReduction and normalization
LayerNormStatistical reduction and normalization
SAXPYElementwise streaming computation

This subset covers several distinct computational patterns, including dense linear algebra, streaming operations, reductions, normalization, and fused workloads. Such diversity makes the subset useful for examining different RVV execution behaviors.


Q3. DEV construction cost and correlation with correctness

Thank you for this question. DEV's engineering advantage is that semantic obligations can be implemented as explicit executable checks, rather than relying solely on an end-to-end correctness verdict.

The retained supplemental implementation contains:

Operator familyDiagnostic checksWeighting
SGEMM6Normalized
GEMV4Normalized
Transpose3Normalized
Row reduction3Normalized
Total16Per-operator normalization

These checks probe different aspects of execution, such as output coverage, accumulation, addressing, and related operator-specific requirements.

We tested correct scalar references and incorrect controls. Scores exclude compilation reward shaping:

ControlSGEMMGEMVTransposeRow reductionFull-output pass fraction
Correct reference1.001.001.001.001.00
Zero-output0.200.200.250.200.00
NaN-output0.000.000.000.000.00

These controls reveal an important distinction between DEV and ordinary complete-output testing: a zero-output kernel may satisfy structural obligations, such as writing output elements, while failing numerical verification. DEV recognizes this limited progress, whereas complete-output testing assigns zero credit.


W2/Q4. Generalization and CUDA applicability

We distinguish functional portability, unseen-task generalization, hardware performance portability, and cross-domain transfer.

An important feature of the RVV generation task is its vector-length-agnostic requirement. The evaluation framework checks candidates under multiple VLEN configurations:

VLENVerification purpose
128 bitsShorter vector capacity
256 bitsIntermediate vector capacity
512 bitsLarger vector capacity

Correct handling requires appropriate strip-mining, pointer advancement, tail processing, and reduction semantics.

The general principle of DEV is not inherently limited to the RVV instruction set, for CUDA kernel generation, analogous diagnostic obligations might concern:

  • Thread/block indexing and output coverage
  • Memory access behavior
  • Reduction and accumulation semantics
  • Synchronization-sensitive computation
  • Numerical correctness of fused operations

However, CUDA introduces different execution and memory semantics, and the necessary checks would require target-specific design.


At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and we would be deeply grateful if you could kindly consider raising the score further to support our work. Thank you very much!

Thank you for the response. It addressed my questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.