- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents Grail-AD, a request-driven framework that combines controllable graphite-image generation with LLM-guided search for rectangle-packing algorithms. It also introduces a new dataset of graphite microscopy scans. The task is to fit rotatable rectangles into usable regions extracted from these images. Using three search frameworks (OpenEvolve, EOH, and MLES), the authors compare five ways of supplying training cases. The Fixed baseline trains on the same 10 real images. The other four add two extra images per evaluation, chosen at random or by request rules based on recent search results. The paper shows that the three request-based schemes outperform the Fixed baseline in 22 of 27 matched comparisons. The reported results are for the best candidate from each run, selected based on its score on the 100-image test set.
Critical strengths
The paper studies a practical problem where real data are expensive to collect, giving a clear reason to explore instance generation. The Random Extra control helps assess whether the request rules offer benefits beyond simply adding more training cases. Testing across three search frameworks also gives a broader view of how the approach performs. The graphite-image dataset and preprocessing pipeline could be useful resources for other researchers.
Critical weaknesses
- One main concern is that the gains may come largely from adding more training cases. The fair baseline is Random Extra, which adds the same number of random images. Random Extra beats Fixed in eight of nine comparisons while the three request-based schemes beat Random Extra in only 15 of 27. Real Request also beats Request in only two of nine comparisons. So most of the gain over Fixed appears without any requests at all. Does this mean more varied images help, not feedback-driven generation?
- What drives the requests? The paper uses newly “detected” images but what counts as detected? If this means successful packing, a new image could enter the request history even when an older program could already handle it, while fitting more rectangles into a previously detected image may not change the signal. This leaves a basic question about whether the requests follow improvements in the solver or mainly reflect which images were recently introduced.
- The training fitness is not well defined. Because the extra images change from one candidate to the next, candidates are compared on different cases, and this noise alone could keep the population diverse. That would be a simpler explanation for why the effect varies so much across frameworks, and it could be tested by showing the extra images to the LLM without counting them in fitness.
- Figure 1 attributes requests to the LLM, but Appendix D constructs them using a predefined rule. The abstract also says that useful cases are kept and unsolvable ones rejected, but it does not explain how the method decides.
- A long, narrow region and a compact region can have the same area and fit very different numbers of rectangles. The paper measures generator scale and request scale differently. I would want to see whether performance on generated cases is informative about performance on real ones. ACEvo (Duan et al.) and ASRO (Ke et al.) seem to already connect solver search with instance generation, so the learned graphite-image generator is an important part of what could distinguish this work.
- It is difficult to translate the packing scores into usable material. Each rectangle earns 100 points, while meeting the spacing condition adds only one. A program can therefore improve its score by placing more rectangles even when their spacing fails the stated condition. The paper should report packing counts with that spacing enforced.
- A strong conventional packing baseline is also needed to judge how good the proposed algorithms are.
Questions for authors
- How does Real Request compare with generating two fresh images per evaluation from the same model without using the search history? Does it offer an advantage over selecting from a fixed generated pool?
- What exactly counts as a “detected” image, and how does the request rule distinguish solver improvement from success on a newly introduced case? How do the extra-case scores affect candidate selection?
- How many rectangles can the algorithms pack when the stated spacing is enforced, and how do these counts compare with a strong conventional packing method?