- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a way to assist HLS pragma insertions/selections using bidirectional flow matching.
Critical strengths
The paper has achieved good improvements over a problem that was important a couple of years ago. (See below.)
Critical weaknesses
The paper's main weakness is that the problem has grown beyond pragmas. The challenge now is rewriting the entire code for better optimization, not just tweaking pragmas.
The works cited by the papers are before 2026. The latest versions of frontier LLMs are offering such amazing capabilities. For example, would equipping LLM-DSE with the latest and greatest LLM yield better results?
Another issue: there don't seem to be any tests on the quality of the individual components of the framework. For example, how good is the QoR predictor?
I think addressing these two main concerns of mine would be good.
Questions for authors
What if more recent LLMs are used for, say, LLM-DSE (2025 vintage)? The recent ones have improved so significantly.
3 comments
Nice results but I suspect the gap will be narrowed fairly soon. And I still think that pragma alone doesn't cut it. Sorry but I am maintaining my score.
Response to Reviewer vufG's follow-up comment
Thank you for the follow-up. We agree that code rewriting is where much of the future gain lies, and we see BiG-Flow as a component of that future rather than an alternative to it. Recent work on agentic HLS supports this view. Ding et al. (arXiv 2609.21157) compare agents that design at RTL with agents that design through HLS: with two pragmas, an array partition and a pipeline, an agent obtains 256-way parallelism that its hand-written RTL reaches only at 32-way, and with HLS a weaker model reaches 0.95 times the speedup of a stronger one, against 0.40 times at RTL. Zhang et al. (arXiv 2602.01401) argue that HLS remains essential for agentic hardware design and name inadequate performance feedback as a key limitation of current HLS tools. Agents that rewrite code will therefore keep relying on pragmas, and on fast feedback about what each design achieves.
The rewriting works themselves report what limits them today. Every candidate needs synthesis or simulation: Agent Factories synthesizes every variant of every sub-kernel, with a median of 5.8M tokens per kernel, and Ding et al. note that a simulation close to the time limit leaves little room for iterative exploration and propose HLS latency estimates during optimization as future work. Pragmas remain hard to set by reasoning alone, as HLSmith reports that general-purpose LLMs apply unrolling and memory banking inconsistently. What these flows lack is a fast, learned model of how pragmas map to QoR on code that no model has seen, which is what BiG-Flow provides.
We see three concrete ways to plug BiG-Flow into a rewriting agent. First, as the pragma stage after each rewrite: a restructured kernel is a new program, which BiG-Flow handles without retraining, and it proposes complete pragma designs for a QoR target before any synthesis, in place of the trial-and-error pragma tuning that ends the flows of C2HLSC, HLSRewriter, and HLSmith. Second, as the fast evaluator that Ding et al. call for: BiG-Flow predicts the QoR of a design in one network evaluation, so the agent can rank its rewrite candidates by the best predicted QoR of their pragma designs and synthesize only the most promising ones. Third, as a training signal: the same prediction gives a rewriting agent a reward for every candidate without a synthesis run per step. Each of these removes synthesis runs from the loop of the agent, which is where its cost lies.
The formulation of BiG-Flow also points to a longer research path. A parameterized code transformation, such as a tile size, a partition factor, or a loop order, is one more discrete choice at a site of the program graph, which is the object BiG-Flow generates. Extending the discrete flow from pragma sites to transformation sites would let one program-conditioned model propose rewrites and pragmas together for an unseen kernel and predict their QoR, and we see this as the most promising direction for future work. Stronger language models and BiG-Flow therefore compound rather than compete: the better an agent proposes rewrites, the more rewritten kernels need pragma designs and QoR estimates, and BiG-Flow supplies them at the cost of one network evaluation instead of one synthesis run.
Response to Reviewer vufG
The problem has grown beyond pragmas
We agree that LLMs are taking HLS beyond pragma tuning, and we surveyed this direction. It has made real progress. LLM pipelines now refactor C/C++ into synthesizable HLS code (C2HLSC, TODAES 2025; HLSRewriter, TODAES 2026), LLM agents solve the simple code-generation tasks of HLS-Eval (2025–2026), and agents restructure kernels for performance: HLSmith (2026) produces functionally correct designs for all 28 PolyBench kernels of HLSFactory and reaches up to 252× over the reference HLS kernels with frontier models, and Agent Factories (2026) reaches a mean 7× speedup with Claude Code agents.
Pragma design remains part of every one of these flows: C2HLSC ends with a separate pragma-identification stage, HLSRewriter with pragma-based PPA optimization, ChatHLS with a QoR-aware directive tuner, and HLSmith and Agent Factories set pipeline and unroll pragmas together with their code changes. It is also the part these systems solve by trial and error, for two reasons their authors report. First, they learn the QoR of a pragma choice only by synthesizing it: HLSmith diagnoses each new synthesis report before its next step, and Agent Factories synthesizes every variant of every sub-kernel, with a median of 5.8M tokens per kernel. Second, LLMs reason poorly about the hardware effect of pragmas: general-purpose LLMs "often fail to reason about the spatial architecture of the source code" and apply unrolling and memory banking inconsistently (HLSmith); fine-tuning on code text alone does not teach them "the architectural impact of pragma variations" (LIFT); and they "lack explicit program-structure awareness, limiting their ability to suggest effective pragmas" (PRISM).
BiG-Flow supplies exactly what is missing. It learns the mapping between pragma designs and QoR from synthesized designs, so for a kernel it has never seen, including code produced by these systems, it predicts the QoR of a pragma design with one network evaluation (per-kernel error 0.63 over three seeds, against 1.20 for HARP) and proposes complete designs for a QoR target before any synthesis. An agent can use it to propose complete pragma designs and estimate their QoR before synthesizing any of them. On the one comparison that isolates pragma design, LLM-DSE driven by the newest frontier models reaches at most 5.5× with 20 synthesis runs, against 8.6× for BiG-Flow (next answer). Pragma design also needs no functional verification, since pragmas never change what a kernel computes, which makes it the step that a learned model can automate at scale.
Newer LLMs for LLM-DSE
Appendix Table 22 already runs LLM-DSE online with GPT-4o, GPT-4.1, GPT-5 mini, and GPT-5. A larger model did not help there: GPT-5 reaches 2.9× and GPT-5 mini 5.2×, against 8.6× for BiG-Flow, the model of the paper and one of its three training seeds. We added the newest models available to us, released in September and October 2026, with the same protocol of 20 synthesis runs per kernel and online synthesis feedback. LLM-DSE sees the synthesis results of its earlier designs, while BiG-Flow chooses all 20 designs before any synthesis. The table reports BiG-Flow as the mean of its three training seeds:
The strongest model gains little over GPT-5 mini, and extended thinking adds nothing. With 20 runs, the agent can test only a few one-pragma changes per iteration, while BiG-Flow proposes complete designs for a kernel it has never seen.
Quality of the individual components
The QoR predictor is evaluated on its own in Section 5.2 and Table 1, on 22 kernels never seen in training. Over three seeds per model, its kernel-averaged error is 0.68 against 1.20 for HARP and 1.08 for HierarchicalMoE, and on a new split that holds out whole algorithm families it is 0.73 against 2.19 and 2.11 (responses to aM3x W2 and W4). Each other component has its own test: the program encoder in Table 9 and Figure 6, one network against two in Figure 5 and Table 13, legal decoding in Tables 13 and 14, and the generator in Section 5.3 and Table 10. The revision moves a summary of these ablations into the main text, using the added page.