- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CADET builds executable PCB routing demonstrations using native EDA operations, then uses them for BC and PPO training.
Critical strengths
I think the dataset is useful, and the replay checks are a good part of the work. The BC-only and full-pipeline results also show that these demonstrations can help with real-board routing.
Critical weaknesses
-
I'm not quite convinced by the explanation in Section 4.4. The authors describe demonstrations as the main source of the CP gain, then qualify this by saying the comparison covers the whole training procedure. That qualification still doesn't tell me where most of the gain comes from. Section 3.3 also changes the KL penalty and critic training, so I would be more cautious with the claim about demonstrations having a dominant effect.
-
The gains over the learning baselines look good, but I'm less sure how stable they are after retraining. The paper reports only one PPO training seed. Repeating evaluation on the trained policy, as described in Section 4.1, doesn't cover variation from training itself. I would like to see the main comparison repeated with different seeds under same settings.
Questions for authors
What evidence makes the authors think demonstrations are the main source of the CP improvement, given that the KL and critic settings also change?
2 comments
Hi, thank you for the response and the additional experiments. The four-seed results solved my concern about training stability. The fine-tuning ablations also show that the KL penalty and critic detachment do not explain the main gap over PPO. Section 4.4 reads much more clearly now. I can see what BC alone achieves and how much RL fine-tuning adds. As my 2 concerns have been resolved, I will raise my rating from 2 to 3.
Response to Reviewer bzJB
Thank you for the two constructive requests.
W1 / Q1. Source of the CP gain
We agree that §4.4 did not separate these. Starting from each seed's BC checkpoint, we reran RL fine-tuning with one setting or both turned off (D3-B test. CP: mean ± std over the 4 seeds).
All four CADET settings reached 63.3–67.2% CP, more than 25 pp above PPO (RouteDemos) (38.0%). So the two settings (KL penalty, critic detachment) are not the main source of the gain. The main source is BC on demonstrations followed by RL fine-tuning (43.1% with BC alone, 67.2% after fine-tuning) (Revised: §4.4, App. G).
Critic warm-up is needed because BC trains only the policy. Without it, the first PPO advantages would come from an untrained critic. The actor is frozen during warm-up, so RL starts from the BC policy unchanged (App. B), and warm-up counts toward the 200 fine-tuning iterations (Revised: App. I).
W2. Training seeds
We retrained the full pipeline with 4 training seeds (42, 43, 44, 45). CADET was trained with BC and RL fine-tuning, and each learned baseline with scratch PPO, and each seed's selected checkpoint was evaluated with the paper's protocol (4 runs) (Revised: Table 2, 3; std in Table 9).
CP (%). Submitted: a single training run. Revised: mean ± std over the 4 seeds.
On both D3-B test and D3-A, CADET@1's CP was higher than every PPO row, and the 95% CIs did not overlap (Revised: App. G; New: Table 11).
Your two requests let us analyse the source of the CP gain more rigorously. We would be glad to answer any further questions.