- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents EVOKE, a framework for evolving native Ascend C kernels with LLMs. Instead of treating the whole kernel project as the unit of generation and optimization, it builds a task knowledge workspace, generates dependency ordered semantic genes pairing component designs with executable code, and searches over them with intent declared actions selected by a non stationary contextual bandit with delayed credit assignment. On 13 Ascend C operators, EVOKE solves all 13 tasks versus 9 for the whole project baseline, with a 1.34x geometric mean performance ratio on tasks solved by both.
Critical strengths
-
Well motivated. The paper identifies how whole project evolution entangles optimization decisions and proposes keeping the construction structure editable during search, an original and clearly articulated idea.
-
Principled design. The combination of progressive semantic gene construction, intent controlled search, and bandit scheduling with delayed credit is coherent and technically sound.
Critical weaknesses
-
Small, self built task set. The evaluation relies on 13 self constructed operators, far smaller than standard public benchmarks. With so few tasks, the main results have limited statistical power, and task selection bias cannot be ruled out.
-
Missing key baselines. The paper compares only against CANNBot, while several relevant systems including concurrent Ascend C generation methods and platform agnostic program evolution frameworks such as AlphaEvolve are neither included nor discussed as baselines. Without such comparisons, it is unclear whether the gains come from the proposed structure or simply from a weak baseline.
-
No evaluation on public benchmarks or other platforms. All experiments use a single self built Ascend setting, while the method's core components are tightly coupled to Ascend C. The claimed generality of the framework is therefore not supported by the evidence.
Questions for authors
Can EVOKE be applied to non-Ascend platforms, for example by evaluating it on KernelBench or similar public benchmarks?