acceptodds
Withdrawn from ICLR 2027

Grail-AD: Request-Driven Algorithm Evolution for 2D Graphite Material Packing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Growing large single-crystal graphite enables structural-superlubric devices, but converting grown crystals into products requires packing independently rotatable rectangular items into irregular usable regions while maintaining safe distances for laser cutting kerf. The physical crystals are expensive to acquire and their morphology is difficult to control. We release a dataset of large-scale single-crystal graphite flakes and propose Grail-AD, a closed-loop framework that connects a scale-controlled crystal image generator to a population of evolving packing algorithms. The generator combines pretrained feature alignment, scale control, and representation-based retrieval to supply diverse crystal images. The packing evolution employs an LLM-driven heuristic search with a composite objective combining geometrically valid packing count and a spacing reward — with an image-request mechanism through which algorithms query specific morphologies from the generator; cases that promote progress are retained while unsolvable ones are rejected, forming a task-directed data engine. Retrospective evaluation of 45 runs spanning three search frameworks, five image-supply schemes, and three repeats on a frozen independent 100-image test set shows that conditioned requests yield a higher retrospective maximum than matched Fixed runs in 22 of 27 comparisons. OpenEvolve improves in all nine matched comparisons; Real Request averages 34,638 versus 27,094 for Fixed, a 27.8% gain. MLES Request improves in all three matched comparisons, with a 10.7% mean gain. Because the reported score for each run is the retrospective maximum over its candidate pool, these results characterize candidate-discovery capability under a fixed search budget rather than unbiased deployment performance. This approach may extend to the product design and manufacturing of other two-dimensional crystal materials, including transferred heterostructures.

Primary area:
optimization
Keywords:
automated algorithm design, controllable image generation, learned evaluator

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_n7dXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Grail-AD, a request-driven framework that combines controllable graphite-image generation with LLM-guided search for rectangle-packing algorithms. It also introduces a new dataset of graphite microscopy scans. The task is to fit rotatable rectangles into usable regions extracted from these images. Using three search frameworks (OpenEvolve, EOH, and MLES), the authors compare five ways of supplying training cases. The Fixed baseline trains on the same 10 real images. The other four add two extra images per evaluation, chosen at random or by request rules based on recent search results. The paper shows that the three request-based schemes outperform the Fixed baseline in 22 of 27 matched comparisons. The reported results are for the best candidate from each run, selected based on its score on the 100-image test set.

Critical strengths

The paper studies a practical problem where real data are expensive to collect, giving a clear reason to explore instance generation. The Random Extra control helps assess whether the request rules offer benefits beyond simply adding more training cases. Testing across three search frameworks also gives a broader view of how the approach performs. The graphite-image dataset and preprocessing pipeline could be useful resources for other researchers.

Critical weaknesses
  1. One main concern is that the gains may come largely from adding more training cases. The fair baseline is Random Extra, which adds the same number of random images. Random Extra beats Fixed in eight of nine comparisons while the three request-based schemes beat Random Extra in only 15 of 27. Real Request also beats Request in only two of nine comparisons. So most of the gain over Fixed appears without any requests at all. Does this mean more varied images help, not feedback-driven generation?
  2. What drives the requests? The paper uses newly “detected” images but what counts as detected? If this means successful packing, a new image could enter the request history even when an older program could already handle it, while fitting more rectangles into a previously detected image may not change the signal. This leaves a basic question about whether the requests follow improvements in the solver or mainly reflect which images were recently introduced.
  3. The training fitness is not well defined. Because the extra images change from one candidate to the next, candidates are compared on different cases, and this noise alone could keep the population diverse. That would be a simpler explanation for why the effect varies so much across frameworks, and it could be tested by showing the extra images to the LLM without counting them in fitness.
  4. Figure 1 attributes requests to the LLM, but Appendix D constructs them using a predefined rule. The abstract also says that useful cases are kept and unsolvable ones rejected, but it does not explain how the method decides.
  5. A long, narrow region and a compact region can have the same area and fit very different numbers of rectangles. The paper measures generator scale and request scale differently. I would want to see whether performance on generated cases is informative about performance on real ones. ACEvo (Duan et al.) and ASRO (Ke et al.) seem to already connect solver search with instance generation, so the learned graphite-image generator is an important part of what could distinguish this work.
  6. It is difficult to translate the packing scores into usable material. Each rectangle earns 100 points, while meeting the spacing condition adds only one. A program can therefore improve its score by placing more rectangles even when their spacing fails the stated condition. The paper should report packing counts with that spacing enforced.
  7. A strong conventional packing baseline is also needed to judge how good the proposed algorithms are.
Questions for authors
  1. How does Real Request compare with generating two fresh images per evaluation from the same model without using the search history? Does it offer an advantage over selecting from a fixed generated pool?
  2. What exactly counts as a “detected” image, and how does the request rule distinguish solver improvement from success on a newly introduced case? How do the extra-case scores affect candidate selection?
  3. How many rectangles can the algorithms pack when the stated spacing is enforced, and how do these counts compare with a strong conventional packing method?
Reviewer_R5SKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Grail-AD, a framework for evolving packing algorithms for irregular usable regions in graphite microscopy images. It combines a scale-controlled image generator, image-to-mask processing, and existing LLM-based program search frameworks. Training cases are selected or generated using recent search outcomes. Experiments compare five image-supply schemes across three search frameworks and three repetitions, evaluating the resulting candidate programs on 100 real images. The reported results measure retrospectively selected candidate maxima instead of the test performance of independently selected final programs.

Critical strengths
  1. The paper connects algorithm evolution to a real material-utilization problem and provides graphite microscopy data and a processing pipeline. This makes the work potentially useful beyond a purely synthetic packing benchmark.
Critical weaknesses
  1. Random additional cases already improve over the fixed-data baseline in eight of nine comparisons. Conditional generation is not consistently superior to retrieval, and all conditioned schemes are outperformed by random additional cases under EOH. These results provide mixed evidence for the distinctive value of the proposed generation loop.
  2. Every candidate is evaluated on the same 100-image set, and the best score is selected retrospectively.This measures candidate-pool potential rather than the performance of a program selected without test access. Moreover, spatially disjoint regions from the same source image are permitted across splits, leaving generalization to independent physical samples less clearly established.
Questions for authors
  1. What evidence establishes the benefit of generating new cases over selecting existing cases, given that retrieval is competitive and random additional cases outperform all conditioned schemes under EOH?
  2. The dataset and controllable generator have potential practical value, but their independent contributions are insufficiently characterized. In particular, the dataset’s physical-sample diversity and the generator’s advantages over simpler conditional-generation or augmentation baselines remain unclear.
  3. How do the conclusions change when each run selects its final program using only training or separate validation data, followed by a single evaluation on a source-disjoint test set? Are test sources also excluded from generator training?
  4. If the minimum cutting distance is a manufacturing requirement, why is it treated as a soft reward in F=100N+S_safety? where rectangles violating the spacing requirement still contribute 100 points each? Do the reported improvements persist when spacing is enforced as a hard constraint and performance is measured by the number of safely cuttable rectangles?
Reviewer_72c1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies cutting rectangular products from single-crystal graphite flakes. The usable region of a flake is an image mask, and the task is to place as many 500×400 px rectangles as possible inside it, with free rotation and a 20 px spacing for the laser kerf (Sec. 2). The paper has three parts: (1) a microscopy dataset of 133 source images in two lighting modes, with a stitching and mask pipeline (Sec. 2, App. A); (2) a scale-conditioned StyleGAN2-ADA generator that produces synthetic flakes, which are turned into masks and used as packing instances (Sec. 3.1, App. B–C); (3) a "request" rule inside LLM-driven program evolution (OpenEvolve, EoH, MLES; Qwen as the proposer). Each candidate program is scored on ten fixed real instances plus two extra ones. The extra instances are chosen to match the scale (Request) or the scale and embedding (Embedding Request) of images recently "detected" by the search, taken from a frozen pool, or generated on the fly from the same request (Real Request) (Sec. 3.2, Alg. 2–6). The evaluation is a grid of 3 frameworks × 5 supply schemes × 3 seeds (45 runs, 50 rounds each). The reported number is the best test score of any candidate the run ever produced, on a frozen set of 100 real images. The three conditioned schemes beat the Fixed baseline in 22 of 27 matched pairs. A Random Extra control (two random extra images) beats Fixed in 8 of 9. Against Random Extra, the conditioned schemes win 15 of 27, and the direction depends on the framework (Sec. 4.4–4.5, Tables 5–6). The authors say clearly that the metric measures candidate discovery under a budget, not deployment performance.

Critical strengths
  • A real industrial problem with a real physical constraint, and an expensive dataset documented end to end: two lighting modes, stitching, mask extraction, and a test set kept spatially separate from the training regions.
  • The experimental grid is more careful than most LLM-evolution papers: full per-seed numbers, a shared initial program and round budget, candidate hashes, and a Random Extra control that separates "more images" from "requested images.
  • The authors are open about the limits of the metric and about the framework dependence of the result.
  • Scale control of the generator is checked quantitatively, including where it stops working outside the training range.
  • The supply schemes are given as pseudocode, so the structure of the experiments is easy to follow.
Critical weaknesses
  1. The headline gain is mostly "more training images", not "requested images". The abstract and introduction lead with Real Request vs Fixed under OpenEvolve (34,638 vs 27,094, a 27.8% gain) and never mention the Random Extra control. Against that control (Table 6), the conditioned schemes win 15 of 27 pairs, with an average gain of about 700 points on a score of about 31,000 — roughly 2%. Under OpenEvolve the gain over Random Extra is consistent (8 of 9). Under EoH it is negative in all 9 pairs. Under MLES, Request and Embedding Request win 3 of 3 each and Real Request 1 of 3. The abstract should report the comparison against Random Extra, since that is the paper's own control. There is also a gap in the design: Fixed scores each candidate on 10 images, all other arms on 12 (Alg. 2 vs Alg. 3–6), and Random Extra re-samples its two images at every evaluation. So the experiments cannot separate "two more images" from "images that change between evaluations". A Fixed-12 arm (twelve fixed images) would close this gap and costs only 9 runs.

  2. The main metric picks the best candidate using the test set. B_100 is the best test score over every candidate a run produced (Sec. 4.3). The authors say so, but it is the only metric in the paper. No number tells the reader what a program chosen without the test set would score, which is what matters for deployment. A "best of all candidates" number also grows with the number of candidates, and per-run candidate counts are not reported (only the total of 6,602, about 147 per run). Both fixes are cheap: Sec. 4.6 says per-image records are kept for every candidate, so the test score of the candidate with the best training fitness can be computed from the existing logs. Please add it, together with per-run candidate counts. (Oved et al. 2026 make the same point about best-of-k reporting in LLM evolutionary search.)

  3. Three seeds, no error bars, and one collapsed baseline run inflates the headline. OpenEvolve Fixed scores are 30,915 / 29,367 / 21,001 (Table 5). The third run is the widest outlier in the whole grid and is never discussed. Without it, the Real Request gain over Fixed is about 15%, not 27.8%. Real Request still wins every pair and its own spread across seeds is small (303 points), so the direction is solid; the size of the headline is not. OpenEvolve Fixed (27,094) is also far below EoH Fixed (31,584) and MLES Fixed (29,660), which suggests that part of the OpenEvolve gain comes from a weak baseline. For EoH vs Fixed, the differences are a few hundred points, inside the spread across seeds, so no conclusion is possible there. Please report per-seed differences and medians, explain the collapsed run, and add seeds for at least OpenEvolve.

  4. The generator is not shown to matter. Only Real Request uses the generator; Request and Embedding Request take images from a frozen pool. Real Request vs Request: slightly better under OpenEvolve (34,638 vs 34,273), worse under EoH and MLES (Table 5). So the component behind the title and the "task-directed data engine" framing is not shown to add anything over picking matching images from a pool. The paper also never says what the pool P contains (real patches or pre-generated images); either way, the generator's own value is untested. The generator evaluation (Sec. 4.2, Table 4) compares only to the authors' own earlier model, with 20 vs 20 images, and never checks whether generated masks look like real masks on packing-relevant quantities (usable-area fraction, component sizes, achievable rectangle count) beyond the scale proxy of Sec. 4.1.

  5. No absolute reference for packing quality. A score of about 34,600 means about 3.4 rectangles per 4096×4096 mask. The initial program's score is never stated; it can only be read off round 0 of Fig. 6/16 (about 15,700, i.e. about 1.6 rectangles per image). There is no hand-written baseline (for example, maximal-rectangles or bottom-left placement with a rotation sweep), no area-based upper bound per image, and no comparison to the cutting-with-defects literature, which is the closest classical version of this task (Afsharian et al. 2014; Gonçalves & Wäscher 2020; the ROADEF/EURO 2018 glass-cutting challenge, Libralesso & Fontan 2021; raster-based packing, Umetani & Murakami 2022). The paper cites irregular-packing surveys but not these. Without a baseline, the reader cannot tell whether the evolved programs are good at all.

  6. The core mechanism is not fully specified, and the story does not match the algorithm. "Detected", which drives every request rule (Sec. 3.2, App. D.1, Alg. 3–5), is never defined. K, the number of look-back rounds, is never given (and the symbol K is reused in Eq. 4 and Alg. 1 for other things). The abstract and Fig. 1 say the LLM requests morphologies and that "unsolvable" cases are rejected. Alg. 3–5 contain no LLM call and no rejection step; the request is a fixed rule (scale match, plus top-2 embedding similarity in two of the schemes). Contribution 3 calls the scheme "code-conditioned", but the request depends only on the round history (r, e, K), so every candidate in a round receives the same request. Sec. 3.2 says the extra images are "also evaluated and recorded"; Alg. 3–6 fold them into the training fitness. The Qwen model (name, size), temperature, prompts, and candidates per round are not given. OpenEvolve and MLES are used but not cited or described. There is no code link and no reproducibility statement.

  7. Positioning against closely related work is missing. Two gaps. First, LLM-era work that co-evolves problem instances together with LLM-evolved solvers already exists and is not cited: EALG/ACEvo (Duan et al. 2025), ASRO (Ke et al. 2026), MOSAIC (Gungordu et al. 2026), with the pre-LLM root in Smith-Miles & Bowly 2015. Those methods generate instances based on solver weakness; this paper generates them based on similarity to recent "detections". The paper should say what is new relative to them and, ideally, compare to a hardness-driven request variant. Second, the choice of search frameworks is not explained. OpenEvolve is an open-source AlphaEvolve-style system (MAP-Elites archive, island populations) and is where the gain is largest; EoH is a genetic algorithm, and there the gain disappears. The paper should state this and explain why an archive that keeps diverse programs benefits from requested cases while a plain GA does not. A discussion of whether the mechanism transfers to other systems of the same class (GigaEvo, ShinkaEvolve) and to single-incumbent loops such as AutoResearch (and population extensions like GEAR) would help; one extra framework would make the framework-dependence claim much stronger.

Questions for authors
  1. What does "detected" mean operationally? What value of K was used? Since the request depends only on the round history, is it correct that all candidates in one round receive the same two extra images?
  2. What is the frozen pool P: real patches, pre-generated images, or both? How many images, and what range of scales? If a run falls back to B = D0 at every round, does it become Random Extra?
  3. For each of the 45 runs, please report (a) the number of candidates produced and (b) the test score of the candidate with the best training fitness. Does the ranking of supply schemes survive under (b)?
  4. Would you add a Fixed-12 arm (twelve fixed images, never changing)? This separates the effect of two more images from the effect of images that change between evaluations.
  5. What happened in OpenEvolve Fixed repeat C (21,001)? Please add seeds for OpenEvolve and report per-seed differences and medians, not only means.
  6. Real Request beats Request only under OpenEvolve, and only slightly. In what setting does on-the-fly generation help — for example, when the requested scale is missing from the pool? Can you show one?
  7. Which Qwen model (name and size), temperature, prompt, and number of candidates per round were used? What was the LLM-call budget per run?
  8. OpenEvolve is a MAP-Elites/island system; EoH is a GA. What in the selection dynamics explains why requested cases help one and not the other? Does the effect transfer to another system of the same class (e.g. GigaEvo, ShinkaEvolve) or to a single-incumbent loop (AutoResearch-style)?
  9. What do the initial program and a simple hand-written baseline score on the 100 test images? What is an area-based upper bound on rectangles per image?
  10. Real source images are downsampled by 50% (App. A.5.1) and generated images are 500×500 and then upsampled (App. B.1). What is the exact resolution pipeline for the real test masks and for the generated masks, and are their boundaries comparable?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.