acceptodds
Under review as a conference paper at ICLR 2027

AG-CoT: Verified Algorithmic Traces for LLM Program Synthesis on Clifford Circuits

Abstract

Scientific code generation can produce executable programs that fail to compute the intended scientific object. We study this problem in language-model synthesis of Clifford circuits, which prepare the stabilizer states used in quantum error correction and admit exact classical verification. In our target-conditioned framework, each target is given as compact signed stabilizer generators, and an exact verifier checks the generated OpenQASM circuits. We supervise models with Aaronson–Gottesman chain-of-thought (AG-CoT) traces checked by the verifier, and continue training on model generations that the verifier accepts. Across two independently trained model families (3B and 7B), AG-CoT supervision multiplies greedy-decode state-equivalence accuracy by four to six times over circuit-only baselines, and verifier-filtered continuation training adds a further consistent gain atop both. A complementary 32B study shows that supervised models achieve near-perfect syntax and Clifford validity while the strongest direct model reaches 6.14% state equivalence per target, rising to over 10% under verifier-guided selection with multiple candidates. These results show that algorithmic trace supervision gives a large, statistically significant gain in both model families and that verifier-filtered continuation adds a further repeated gain. The persistent gap between Clifford validity and state equivalence confirms that exact verification is necessary: a circuit can be syntactically and physically valid yet prepare the wrong quantum state.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
quantum circuit synthesis, large language models, exact semantic verification, Clifford circuits, chain-of-thought supervision, verifier-guided training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bfjwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses semantic correctness in LLM-based scientific code generation by focusing on Clifford circuit synthesis for quantum state preparation. The authors introduce a target-conditioned framework where targets are specified as compact signed stabilizer generators, verified using exact classical verifiers. The models are supervised using Aaronson-Gottesman chain-of-thought (AG-CoT) algorithmic traces and further trained on verifier-accepted completions. Controlled experiments across 3B, 7B, and 32B model families demonstrate substantial gains in state-equivalence accuracy compared to circuit-only baselines.

Critical strengths
  1. Supervising circuit generation with verified Aaronson-Gottesman reduction traces significantly improves exact quantum state equivalence over direct circuit-only fine-tuning across multiple model scales. 2. Formulating target states using compact signed stabilizer generators instead of full state vectors dramatically reduces token context length while maintaining exact classical verifiability.
Critical weaknesses
  1. Limitations of prior quantum LLM benchmarks and fine-tuning techniques are not sufficiently analyzed to show why general-purpose CoT or standard execution-based verification fail without domain-specific algorithmic trace alignment. 2. The overall methodology lacks a high-level algorithmic description or flowchart detailing the full data construction, inverse compilation, and verifier filtering loops to cleanly decouple novel contributions from existing components like Aaronson-Gottesman reduction. 3. Results in tables and text are presented too concisely without bulleted observational points, leaving key metrics like the gap between strict label matching and state equivalence under-discussed. 4. All empirical evaluations are restricted solely to Clifford random-circuit benchmarks derived from QCircuitBench, lacking evaluation on broader non-Clifford or realistic physical quantum error correction circuits.
Questions for authors
  1. In Section 3, tableau semantics are introduced, but it is not explained why existing code generation fine-tuning strategies cannot learn this representation directly. Can the paper clarify the domain limitations of existing approaches? 2. In Section 4, could the work provide a complete step-by-step top-level algorithm or pseudocode for the AG-CoT trace generation and inverse compilation pipeline, explicitly marking which modules use existing Qiskit routines versus novel verifier logic? 3. In Section 5, what are the exact compute requirements and tool flows (e.g., vLLM or TRL configurations) used during the RFT generation phase? 4. In Section 6.2 (Table 3 and Table 4), the performance gap between strict saved-label matching and full state equivalence is significant. Can the paper provide detailed observational points analyzing why label representation misalignment occurs so frequently despite valid execution? 5. In Section 6, evaluation is restricted to random Clifford targets. How does the AG-CoT approach perform when applied to structured quantum error correction benchmarks such as StabilizerBench?
4 comments
AuthorsOpenReview ↗

Response to Reviewer bfjw (1/3)

W1 + Q1. Motivation for algorithmic trace supervision

Reviewer concern. Limitations of prior quantum LLM benchmarks and fine-tuning techniques are not sufficiently analyzed to show why general-purpose CoT or standard execution-based verification fail without domain-specific algorithmic trace alignment.

In Section 3, tableau semantics are introduced, but it is not explained why existing code generation fine-tuning strategies cannot learn this representation directly. Can the paper clarify the domain limitations of existing approaches?

Response.

We thank the reviewer for the detailed questions, and for recognizing the gain over circuit-only training across model scales and the compact target representation.

Standard fine-tuning does learn this representation, to a limited degree. The paper measures three approaches on this task.

  1. Prompting. QCircuitBench reports 0.0000 fidelity on Clifford state preparation for every model it evaluates, including GPT-4o (§1, p2). The Qwen2.5-3B base model solves 0 of our 3,661 test targets (Table 1, p7).
  2. Standard fine-tuning on prompt and program pairs. Direct SFT solves 33 of the 3,661 targets with Qwen and 71 with Mistral, against 201 and 316 for AG-CoT (Table 1, p7). Its training pairs show the final circuit but omit the reduction steps that construct it from the target (§4, p4).
  3. Execution-based checking. Our check is execution-based. It executes the program and compares the prepared state with the target exactly (§3, pp3–4). Checking only that a program runs is not enough, because a program can run and still prepare the wrong state (Figure 1, p2). For the 32B Basic AG-CoT model, an executability check accepts 99.9% of the outputs and the exact state check accepts 5.37% (Table 3, p9). The Qwen and Mistral experiments use this exact check to build the training data, to select model outputs for continued training, and to score every result.

A general-purpose chain of thought has no procedure that generates it for each target and no exact check that can replay it. Each AG trace is produced by the same deterministic procedure as the teacher circuit, which the exact check accepts (§4, p5), and its steps are exact tableau updates that can be replayed and checked (§6.3, p10).

The three numbered points are now stated in §2 (p3).

AuthorsOpenReview ↗

Response to Reviewer bfjw (2/3)

W2 + Q2. Construction pipeline and reused components

Reviewer concern. The overall methodology lacks a high-level algorithmic description or flowchart detailing the full data construction, inverse compilation, and verifier filtering loops to cleanly decouple novel contributions from existing components like Aaronson-Gottesman reduction.

In Section 4, could the work provide a complete step-by-step top-level algorithm or pseudocode for the AG-CoT trace generation and inverse compilation pipeline, explicitly marking which modules use existing Qiskit routines versus novel verifier logic?

Response.

The full procedure is given below as one algorithm, and §4 now gives it in condensed form as Algorithm 1 (p6), which merges steps 1–2 and 6–7. [Qiskit] marks an existing Qiskit routine, [AG] the published Aaronson–Gottesman reduction, and [ours] logic written for this work. Figure 2 (p5) shows this pipeline as a flowchart, and Figures 6–7 in Appendix C (pp17–18) show the 32B variant.

Teacher construction for a target given by n signed generators (Qwen-3B and Mistral-7B)

  1. [Qiskit] Build the target state (StabilizerState.from_stabilizer_list).
  2. [ours] Row-reduce the generators to a canonical basis.
  3. [Qiskit] Complete the canonical basis to a full Clifford tableau, which adds n destabilizer rows.
  4. [AG] Reduce the tableau to the identity, one qubit at a time, with h, s, cx and swap operations and final sign corrections. Each tableau update is a Qiskit routine.
  5. [ours] After every operation, write the operation and the current destabilizer and stabilizer rows of that qubit into the trace.
  6. [Qiskit] Invert the recorded reduction to obtain the preparation circuit.
  7. [ours] Expand every gate into {h, s, cx}. [Qiskit] Serialize the circuit to OpenQASM 3.
  8. [ours] Accept the pair only if the serialized program parses, uses only {h, s, cx}, has n qubits, and prepares the target state. Parsing and the state comparison (StabilizerState.equiv) are Qiskit routines.
  9. [ours] Emit two completions for the same prompt, trace plus program for AG-CoT, and the program alone for Direct SFT.

Verifier-filtered continuation

  1. [vLLM] Sample one completion per training-pool prompt from the AG-CoT model.
  2. [ours] Keep a completion only if generation ended normally, it contains one trace block followed by one fenced program, and the program starts with OPENQASM 3.0; and passes the checks of step 8.
  3. [Transformers, PEFT] Continue training from the AG-CoT adapter on the kept completions, unchanged.

In the 32B study, steps 1–3 start from the Clifford of the QCircuitBench reference circuit, and step 8 requires saved-label equality (§4, p4).

The AG reduction and the Qiskit state routines are existing components. The contribution is what is built around them, a teacher derived from the target generators alone, the recorded reduction used as supervision against a circuit-only control with the same prompt and program, and one exact state check shared by teacher data, continuation data, and evaluation.

Q3. Generation tools and compute requirements

Reviewer concern. In Section 5, what are the exact compute requirements and tool flows (e.g., vLLM or TRL configurations) used during the RFT generation phase?

Response.

The continuation phase has three stages. The table gives the tool flow and the logged compute of the Mistral-7B run, on eight NVIDIA RTX A6000 GPUs.

StageTool flowCompute
Proposal generationvLLM 0.6.4.post1 with the AG-CoT LoRA adapter, BF16. 8,192 training-pool prompts in 8 shards of 1,024, one completion per prompt, batches of 16, temperature 0.8, top-p 0.95, seed 42, 8,192-token context with up to 8,000 new tokensOne GPU per shard, 3 h 23 min to 3 h 36 min per shard
VerificationQiskit 2.3.0 with the acceptance rule of §4 (p6), no repair. 983 completions accepted8 CPU processes, 25 min including the shard merge
Continuation trainingTransformers 4.52.4 Trainer with PEFT 0.19.1 on PyTorch 2.5.1, starting from the AG-CoT adapter. One epoch, learning rate 5e-6, global batch 32, BF168 GPUs, 31 updates in 4 min

The Qwen2.5-3B continuation uses the same sampler, the same sampling parameters, and the same verifier. It accepts 715 of 8,192 completions and trains for 23 updates on one GPU with the same learning rate and global batch size. TRL is used only for GRPO in the 32B study. Table 11 (p21) lists its version.

This table is now Table 28 in Appendix J.3 (p29).

AuthorsOpenReview ↗

Response to Reviewer bfjw (3/3)

W3 + Q4. Label matching, state equivalence, and presentation of results

Reviewer concern. Results in tables and text are presented too concisely without bulleted observational points, leaving key metrics like the gap between strict label matching and state equivalence under-discussed.

In Section 6.2 (Table 3 and Table 4), the performance gap between strict saved-label matching and full state equivalence is significant. Can the paper provide detailed observational points analyzing why label representation misalignment occurs so frequently despite valid execution?

Response.

The observations are listed below, and §6.2 now lists observations 1, 3 and 4 under Tables 3 and 4 (p9).

  1. Both checks start from a valid circuit. Both are applied only after the program parses and converts to a Clifford circuit, which holds for 99.6–99.9% of trained-model outputs (Table 3, p9). The gap between the two checks is therefore not an execution failure.
  2. Label matching compares one generator list, and state equivalence compares the state. Executing a circuit from the all-zero state gives one list of n signed generators. Label matching accepts the output only if this list equals the saved list, which is the one produced by the reference circuit, up to order. State equivalence, computed with the Qiskit routine StabilizerState.equiv, accepts it if the two lists describe the same state (§3, pp3–4).
  3. One state has many generator lists. The two-qubit program h q[0]; cx q[0], q[1]; yields {+XX, +ZZ}. Placing cx q[0], q[1]; in front of it prepares the same state and yields {+XX, −YY}. A model that reaches the target state through a gate sequence different from the reference can therefore end on a different list and fail label matching.
  4. About half of the correct states are affected. In Table 3, label matching and state equivalence give 2.62% and 5.37% for Basic AG-CoT, 2.31% and 4.83% for Step AG-CoT, 2.21% and 5.26% for GRPO-v2, and 3.13% and 6.14% for RFT-v1. Between 49% and 58% of the outputs that prepare the target state are thus rejected by label matching. With 64 candidates per target, coverage over the same 249,216 Basic AG-CoT candidates is 4.42% under label matching and 10.07% under state equivalence (Table 4, p9).

W4 + Q5. Evaluation scope and structured error-correction tasks

Reviewer concern. All empirical evaluations are restricted solely to Clifford random-circuit benchmarks derived from QCircuitBench, lacking evaluation on broader non-Clifford or realistic physical quantum error correction circuits.

In Section 6, evaluation is restricted to random Clifford targets. How does the AG-CoT approach perform when applied to structured quantum error correction benchmarks such as StabilizerBench?

Response.

The comparison needs an exact check on every output and a step-by-step derivation of a correct circuit for every target. Clifford state preparation provides both (§1, p1), which is why the experiments use random Clifford targets (§5, p6).

A code state fixed by n generators, such as a logical basis state, is a stabilizer state, so the same teacher construction and the same verifier apply to it. StabilizerBench specifies a code by fewer than n generators, so its targets are codespaces. Fixing a logical state gives a target that our pipeline can handle.

For non-stabilizer targets, §3 (p3) states what the extension requires, a representation for non-stabilizer states and a fidelity threshold fixed before evaluation.

I would like to thank the authors for the responses. In light of the rebuttal and other reviews, I keep my score.

Reviewer_65CJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper shows that LLM-based Clifford circuit synthesis can be improved (across various small- and mid-sized models) if exact verification and carefully chosen CoT traces are used for training. In and off itself, this is not uninteresting but also not a surprising or sensational result. Due to the understandable focus on Clifford circuits (which can be simulated classically) it is also a somewhat restricted result. Yes, there is a lot of current work on Clifford circuits so that this paper follows an ongoing trend. But, to a quantum computing researcher, the deluge of papers in this direction hardly reports truly noteworthy insights and this paper is no exception. The fundamental question to ask is therefore: Who would be genuinely interested in this paper and consider it to be essential for their own research? Any honest answer to this question will be very sobering …

Critical strengths

If model supervision via Aaronson–Gottesman chain-of-thought traces for verification and continued training of models accepted by a verifier are to be considered critical strengths, then so be it. I do not think that this makes the paper strong enough to be accepted into the ICLR program.

Critical weaknesses

As with most scientific texts written with or by AI, the information content of most sentences is overwhelming. As is the interspersion of the text with numerous mathematical expressions (with often unnecessary details) which would have better been rendered as stand alone equations. These are of course human concerns. AI readers will likely find is less difficult to read this paper. One furthermore recognizes a fixation on budget experiments which, by now, have become a hallmark of AI assisted papers. What do these contribute to representation learning for quantum computing? Indeed, the major critical weakness of this paper is that its intended audience is unclear. Is it quantum computing researchers? If so, then what are they supposed to take away from this work? Is it language model engineers? If so, then how does this paper position itself against current frontier models? I really don’t know what I am supposed to make of this work.

Questions for authors

see above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 65CJ (1/3)

Summary + Strengths + W1c + Q1. Scientific contribution, intended audience, and model scope

Reviewer concern. In and off itself, this is not uninteresting but also not a surprising or sensational result. Due to the understandable focus on Clifford circuits (which can be simulated classically) it is also a somewhat restricted result. Yes, there is a lot of current work on Clifford circuits so that this paper follows an ongoing trend. But, to a quantum computing researcher, the deluge of papers in this direction hardly reports truly noteworthy insights and this paper is no exception. The fundamental question to ask is therefore: Who would be genuinely interested in this paper and consider it to be essential for their own research? Any honest answer to this question will be very sobering …

If model supervision via Aaronson–Gottesman chain-of-thought traces for verification and continued training of models accepted by a verifier are to be considered critical strengths, then so be it. I do not think that this makes the paper strong enough to be accepted into the ICLR program.

Indeed, the major critical weakness of this paper is that its intended audience is unclear. Is it quantum computing researchers? If so, then what are they supposed to take away from this work? Is it language model engineers? If so, then how does this paper position itself against current frontier models? I really don’t know what I am supposed to make of this work.

see above.

Response.

We thank the reviewer for the review and for the direct question about who this work is for, which we answer first.

The paper addresses a concrete question for researchers who design supervision for scientific code generation. Does adding algorithmic construction steps improve target correctness when the final training answers are held fixed? Classical simulability is what makes Clifford state preparation a clean test of this question: it supplies a classical teacher and an exact check on each generated circuit. In Table 1, the two conditions share prompts, targets, and final teacher circuits within each model family. On the same 3,661 held-out targets, adding traces increases correct single outputs from 33 to 201 for Qwen and from 71 to 316 for Mistral.

Our contribution is a training pipeline that turns an exact circuit-synthesis procedure into paired derivation-and-program supervision. Sections 3–4 and Appendix C describe the following construction.

  1. Construct a teacher from the requested state. The input specifies a target state through signed stabilizer generators. The Aaronson–Gottesman procedure requires a full Clifford tableau, an operator representation containing additional auxiliary rows. For the Qwen and Mistral experiments, we canonicalize the target and choose a deterministic completion to this representation. This makes the teacher construction reproducible from the target alone.
  2. Generate the derivation and program together. We record the reduction of that representation to the identity. Reversing the recorded operations and inverting their gates yields the preparation circuit, which is compiled into the permitted gate set and emitted as OpenQASM. Thus, the derivation and program describe the same construction. The paired circuit-only control retains exactly the same prompt and final program while omitting the derivation.
  3. Check the scientific result and reuse that check in training. A pair is retained only if its program satisfies the task requirements and prepares the target state exactly. The state-equivalence check also selects model-generated completions for supervised continuation and evaluates held-out targets, allowing different programs to count as correct when they prepare the same state. The Qwen and Mistral test targets are disjoint from training and development targets at the state level.

This construction gives researchers a way to train on the process that produces a correct scientific program and compare it with training on that same program alone.

AuthorsOpenReview ↗

Response to Reviewer 65CJ (2/3)

Summary + Strengths + W1c + Q1 (continued)

Related quantum work predicts successive gates from a residual target operator (Feris et al., §IV). Our model receives a target state and generates a derivation followed by a complete program.

The Introduction now states that the comparisons train the same open model in different ways (p2), and Algorithm 1 gives the construction step by step (§4, p6). On positioning against frontier models: with prompting alone, every model evaluated in QCircuitBench, including GPT-4o, scores 0.0000 fidelity on Clifford state preparation (§1, p2). The paper therefore studies training, and its within-model design measures the effect of the training procedure.

We respectfully disagree that the paper reports no “truly noteworthy insights”. Since the review does not explain why our results are not noteworthy, we summarize the main findings:

  1. The gain holds with the final training answers fixed and with supervised tokens matched. Adding the construction steps gives four to six times as many correct programs in two model families (Table 1), and AG-CoT stays ahead in every seed of the token-matched Qwen comparison (§6.1, p8). The number of supervised tokens alone therefore does not explain the gain.
  2. The models write complete valid derivations for states they have not seen. Among correct final programs, the derivation is valid at every step in 228 of 316 Mistral outputs and 90 of 201 Qwen outputs.
  3. The written derivations first go wrong at the target statement. Among incorrect final programs, the earliest false statement is the restated target in 3,427 of 3,460 Qwen outputs and 3,339 of 3,345 Mistral outputs, before the first gate.

Findings 2 and 3 come from a replay of the archived AG-CoT outputs against exact state and gate-update checks, now reported in §6.3 (p10) and Appendix J.2 (Table 27, p29).

For quantum computing researchers who use or evaluate language models for circuit synthesis, the takeaways are where these models fail (finding 3) and how to score them: matching generator labels hides more than half of the correctly prepared targets (Table 4, p9).

If a specific claim or comparison remains unconvincing, please point us to it.

AuthorsOpenReview ↗

Response to Reviewer 65CJ (3/3)

W1b + Q1. Interpretation of the candidate-budget experiment

Reviewer concern. One furthermore recognizes a fixation on budget experiments which, by now, have become a hallmark of AI assisted papers. What do these contribute to representation learning for quantum computing?

Response.

Table 4 contributes two measurements about judging generated circuits: how much a scoring convention hides, and how much verifier selection adds. It belongs to the complementary 32B study. The main comparison in Table 1 uses one greedy output per target.

On the same 249,216 Basic AG-CoT candidates, at 64 candidates per target, saved-label matching gives 4.42% coverage and exact state equivalence gives 10.07% (Table 4). Different generator lists can specify the same state, so this gap is a penalty of the representation convention rather than a failure to prepare the state (§3, p4). In this pool, scoring by generator labels hides more than half of the correctly prepared targets.

Holding state equivalence fixed, the estimated one-candidate coverage is 4.65%, and the observed 64-candidate coverage is 10.07%. This quantifies what verifier selection recovers from additional candidates.

W1a + Q1. Readability and mathematical presentation

Reviewer concern. As with most scientific texts written with or by AI, the information content of most sentences is overwhelming. As is the interspersion of the text with numerous mathematical expressions (with often unnecessary details) which would have better been rendered as stand alone equations. These are of course human concerns. AI readers will likely find is less difficult to read this paper.

Response.

§3 now introduces the verifier with the two programs of Figure 1 and sets its two checks as one stand-alone equation (p4). The signed Pauli expansion and the tableau update are in Appendix A (p15). §4 now gives the construction as Algorithm 1 (p6), and §6.2 lists the observations on Tables 3 and 4 as bullets (p9).

The research, experiments, and claims are the authors’ own. As the AI Use Statement in the submitted manuscript discloses (p10), a language model assisted in polishing the language, and the authors verified all claims and reported values. The review ties the writing style and the budget experiments to AI use in three places. We ask that the paper be assessed on its content.

Thank you for the clarification.

Reviewer_bvp7OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains large language models to generate Clifford circuits using verified algorithmic worked examples, followed by rejection-sampling fine-tuning (RFT). The results suggest that worked examples substantially improve correctness over circuit-only training, although absolute success rates remain low and further gains from self-generated examples are small.

Critical strengths
  • Uses a well-defined task with efficient, exact verification, providing a controlled setting for experimenting with algorithmic learning techniques.

  • Demonstrates that training on worked algorithmic examples improves correctness across two model families.

  • Separately reports whether generated circuits are valid and whether they prepare the requested state.

Critical weaknesses
  • Reader gains limited insight into what the models actually learn, why they fail, or whether they can generalise what they have learned to substantially different problems.

  • Practical value is unclear since success rates remain low despite an efficient classical algorithm already solving the task.

  • Training comparisons do not isolate the benefit of algorithmic traces from the additional supervised tokens; robustness across training seeds is unclear.

  • Reproducibility seems difficult because details are scattered, with no clearly identifiable repository for the code, data and trained models.

Questions for authors
  • How do we know that the models are learning a reusable synthesis procedure, rather than say patterns specific to the training distribution?

  • Is there a code repository available to facilitate reproducibility?

  • What kind of analysis could help to explain why most outputs remain incorrect?

3 comments
AuthorsOpenReview ↗

Response to Reviewer bvp7 (1/2)

W2. Research contribution and practical value

Reviewer concern. Practical value is unclear since success rates remain low despite an efficient classical algorithm already solving the task.

Response.

We thank the reviewer for the careful review, and for noting the exact verification, the gain across two model families and the separate reporting of validity and state correctness.

The classical algorithm is the instrument of the study. It supplies the teacher, and an exact check scores every output, so we can ask whether supervising the construction steps makes the programs a model writes more correct, a question that applies to LLM program synthesis in general (§1, pp1–2).

With prompting alone, every model evaluated in QCircuitBench, including GPT-4o, scores 0.0000 fidelity on Clifford state preparation (§1, p2), and the Qwen2.5-3B base model solves 0 of our 3,661 test targets (Table 1, p7). On the same targets, circuit-only and AG-CoT supervision yield 33 versus 201 correct greedy outputs for Qwen2.5-3B-Instruct, and 71 versus 316 for Mistral-7B-Instruct-v0.3 (Table 1, p7). AG-CoT stays ahead when supervised tokens are matched (W3 below).

W3. Supervision tokens and training seeds

Reviewer concern. Training comparisons do not isolate the benefit of algorithmic traces from the additional supervised tokens; robustness across training seeds is unclear.

Response.

We have completed an additional Qwen2.5-3B comparison with three independent training seeds and a control matched in total supervised tokens. It is now in §6.1 (p8) and Appendix J.1 (Table 26, p28). The three conditions use a common filtered corpus of 35,039 paired examples. Within each seed they share the initialization, prompts, target states, and final teacher programs. One pass of circuit-only training contains 19,623,758 supervised tokens. AG-CoT contains 106,488,344, and the repeated circuit-only control is stopped at exactly that same token count.

Training conditionMean task success across three seeds
Circuit only, one pass0.63%
Circuit only, matched supervised tokens3.95%
AG-CoT, one pass5.14%

Every model is evaluated on the same 3,661 held-out targets with one greedy output per target. AG-CoT exceeds the token-matched control in each seed, by 45, 32, and 54 correct outputs. The advantage of trace supervision therefore holds at equal supervised tokens and in every training seed.

AuthorsOpenReview ↗

Response to Reviewer bvp7 (2/2)

W1 + Q1. Reusable synthesis and generalization

Reviewer concern. Reader gains limited insight into what the models actually learn, why they fail, or whether they can generalise what they have learned to substantially different problems.

How do we know that the models are learning a reusable synthesis procedure, rather than say patterns specific to the training distribution?

Response.

Three results show what the trained models learn.

  1. We tested the models on states they had not seen, at every size. The 3,661 test targets are mutually distinct states, generated after the Qwen checkpoints were frozen, and none shares its canonical identity with the 37,071 training and development targets (§5, p7). AG-CoT solves more of them than circuit-only training at every size from three to twelve qubits, including 40 versus 2 of the 841 twelve-qubit targets (Figure 4, p7, and Table 13, p23).
  2. We checked the written derivations step by step. We replayed the archived AG-CoT outputs against exact state and gate-update checks. Among correct final programs, the written derivation is valid at every step in 228 of 316 Mistral outputs and 90 of 201 Qwen outputs, and in 242 of 565 across three additional Qwen training seeds. In these cases the model wrote a full, valid reduction for a state it had not seen.
  3. We looked at how close the incorrect outputs are to the target. Among valid but incorrect Qwen outputs, the fraction missing at most two independent target constraints is 4.09% after circuit-only training and 7.77% after AG-CoT (Table 5, p10). In the 32B study, 112 of 961 errors at five to eight qubits miss at most two constraints, against 8.84 expected for random programs that preserve gate types and order (Appendix H, Table 18, p25).

W1 + Q3. Why most outputs remain incorrect

Reviewer concern. What kind of analysis could help to explain why most outputs remain incorrect?

Response.

We ran two analyses to find out where the outputs fail.

  1. We looked at what kind of error the incorrect outputs make. For Qwen AG-CoT, 3,425 of the 3,460 incorrect outputs are valid Clifford circuits with the requested gate set and qubit count that prepare a different state (Table 5, p10). The trained 32B models reach 99.6–99.9% syntax and Clifford validity (Table 3, p9), and Appendix I lists the failure categories (Table 21, p27).
  2. We looked for the first false statement in each written derivation. The replay above checks whether the written target generators represent the requested state, whether the completed tableau is valid, and whether the declared gate operations and rows agree and reduce to the identity. Alternative valid derivations are accepted. Among incorrect final programs, the earliest false statement is the restated target in 3,427 of 3,460 Qwen outputs and 3,339 of 3,345 Mistral outputs. Three additional Qwen training seeds, evaluated on the same 3,661 targets, give 10,344 of 10,418. The dominant failure therefore precedes the first gate.

The replay results are now in the error analysis in §6.3 (p10), with the counts per model in Appendix J.2 (Table 27, p29).

W4 + Q2. Reproducibility and artifact access

Reviewer concern. Reproducibility seems difficult because details are scattered, with no clearly identifiable repository for the code, data and trained models.

Is there a code repository available to facilitate reproducibility?

Response.

We will release the code, data splits, trained adapters, and evaluation scripts upon acceptance. Appendix F lists the required artifacts and evaluation procedure, and Appendix G.1 provides detailed Mistral settings.

Thank you for the additional experiments. The token-matched control and repeated training runs substantially address my concern about the comparison. I interpret them as showing additional circuit-only training accounts for much of the improvement, while trace supervision provides a smaller but consistent further benefit.

However, my central reservation remains. Given that an efficient classical algorithm already solves the task, the contribution depends on what this setting teaches us about algorithmic learning. The revised results establish a benefit from trace supervision, but I remain unclear what substantially new or transferable insight follows.

The finding that most failures begin with an incorrect restatement of the target is interesting. Do you have evidence that supplying the correct target at this stage improves subsequent synthesis? This would help distinguish identifying the first error from explaining the eventual failure. Valid derivations on unseen targets do not establish generalisation to substantially different problems.

Finally, release upon acceptance leaves reproducibility difficult to assess during review. Could you provide an anonymous repository containing the relevant code, data and evaluation scripts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.