- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CSEM combines learned local energies with structural edits and continuous refinement to solve larger planning, scene, and trajectory problems than those used for training.
Critical strengths
-
Scale generalization is strong: five-bin packing reaches 52% versus 8% without edits, and length-100 trajectory validity reaches 72% versus 49%.
-
Search controls using the same energies and matched evaluation budgets help isolate the method’s contribution.
Critical weaknesses
-
The calibration claims need more evidence. BCE training and AUC do not establish calibrated probabilities on larger problems, and ranking differs from the stated energy objective.
-
Evaluation counts do not directly show runtime cost. Trajectory validity also measures self-consistency rather than recovery of the true trajectory.
Questions for authors
-
How reliable are the scores and stopping rule on larger problems? Please clarify how optimization and candidate ranking differ.
-
Can you report comparisons at the same wall-clock budget and position errors against the true trajectories?
2 comments
Thank you for the author's reply. Most of my concerns have been well resolved. I will maintain my score.
Thank you for your constructive feedback and positive comments. Following your suggestion, we have added calibration and stopping diagnostics, clarified optimization versus ranking, and reported matched-time comparisons with ground-truth trajectory errors. We address these points below.
Question 1. Score calibration, stopping reliability, and candidate ranking
Thanks for raising this question. We have now checked the learned energies and stopping rule on larger problems and added the results in Appendix B.6, Tables 17–19.
We find that reliability depends on the domain. On five-bin packing, 29 of 30 stopped runs return valid solutions (97%). On rearrangement with 12 objects and three boxes, this is 18 of 30 (60%); on the harder 20-object, five-box setting, it is 10 of 44 stopped runs (23%) at 1M evaluations. For eight-object stacking, we find a valid plan during search in 26 of 30 instances, but return one in 7 of 30, showing that ranking the candidates remains difficult at larger sizes.
We also clarify the distinction between optimization and ranking: we use time-conditioned energy gradients to refine continuous variables under a fixed graph, then use the sum of softplus-transformed clean factor energies to compare candidates across graphs. We stop when every factor’s predicted violation probability for the best candidate falls below the threshold.
Question 2. Matched wall-clock budgets and true-trajectory position errors
Thanks for this suggestion. We have now compared the methods under the same wall-clock budgets and measured position errors against the ground-truth trajectories. We report these results in Appendix D.3, Table 23:
We find that CSEM recovers positions more accurately at longer horizons and with sparse observations, while ALiBi has lower error at T = 20. For each trajectory, we average position errors over timesteps, then report the median across 100 fresh trajectories. We run all methods on four CPU cores and exclude model loading, warm-up, and grading from the timing.