acceptodds
Under review as a conference paper at ICLR 2027

CellBench: Benchmarking Generalization in Fast Standard Cell Evaluation

Abstract

Standard cell libraries are collections of reusable building blocks for very large scale integration (VLSI) circuits. Developing and optimizing cell libraries involves exploring a large design space of pre-layout netlists with diverse size configurations, topologies, and functions. For each candidate netlist, accurate evaluation of power, performance, and area (PPA) requires time-consuming layout generation and simulation, creating a bottleneck in design exploration. Machine learning can alleviate this bottleneck through fast prediction, but generalization to unseen cell designs remains insufficiently evaluated. To address this evaluation gap, we introduce CellBench, the first open dataset and benchmark for fast standard cell evaluation, covering PPA and constrained routability prediction and evaluating generalization to unseen size configurations, topologies, and functions. Building upon the ASAP7 7 nm FinFET process design kit, we generate diverse pre-layout netlists through cell library extension, topology generation, and transistor size assignment, yielding 118,906 netlists for PPA prediction and 296,274 for constrained routability prediction. Evaluation of existing prediction methods shows declining overall performance from unseen size configurations to unseen topologies and functions. Further analysis shows that structural similarity to training circuits helps explain PPA prediction errors, whereas functional similarity alone does not consistently reflect prediction difficulty. The benchmark is available for anonymous review.

Primary area:
datasets and benchmarks
Keywords:
Standard Cell Library, Circuit Representation Learning, Electronic Design Automation, Datasets and Benchmarks, Graph Neural Network

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.80 over 5 reviews, above 85% of the others.
Reviewer_XuqTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CellBench benchmarks standard-cell PPA and constrained routability prediction from pre-layout netlists. It provides 118,906 PPA samples and 296,274 routability samples, with tests on unseen sizes, topologies, and functions. Three adapted baselines reveal generalization gaps, followed by analyses of functional and structural similarity.

Critical strengths
  1. The paper addresses a practical bottleneck in standard-cell optimization and provides a substantial dataset with structured generalization splits, creating a useful resource for evaluating early-stage prediction methods.
  2. The analysis goes beyond aggregate accuracy to examine functional and topological distances, within-function ranking, and uncertainty, offering useful insights into prediction difficulty.
Critical weaknesses
  1. Although Appendix D.3 includes useful controls within Test-3, comparisons across test sets still combine changes in topology, function composition, and data source. This limits how specifically the aggregate performance gaps can be interpreted.
Questions for authors
  1. Beyond the within-Test-3 controls, how sensitive is the Test-2/Test-3 performance gap to data source and circuit complexity?
  2. Do the retained routing-failure records reveal distinct failure categories? If available, a brief breakdown would help characterize the negative class.
4 comments
AuthorsOpenReview ↗

Response to Reviewer XuqT (Part 1/2)

Explanation to Weakness 1 and Answer to Question 1 (joint response)

Thank you for your detailed suggestions on comparisons across test sets.

CellBench evaluates generalization to progressively broader design changes, rather than the separate effects of sizing, topology, and function. Testing unseen functions also involves new topologies and sizing configurations that implement them, not just changing function labels. Function changes therefore cannot be fully separated from changes in topology and sizing. In graph learning terms, this setting tests generalization to unseen graph semantics (Boolean functions), which depend on graph topology and node attributes.

To better explain prediction error gaps across test sets, we added the following comparisons.

(1) Testing how function proportions and circuit size affect the Test-1/Test-2 error comparison. We used the original model predictions for the 42 functions shared by both test sets. We first calculated overall errors using the same function proportions, then compared samples with the same function, equal MOS counts, and similar total fin counts. Table 1 reports MAPE differences averaged over the five PPA metrics (Test-2 − Test-1).

Table 1: Test-1/Test-2 MAPE gaps after matching function proportions and circuit size (percentage points).

ModelOriginal gapSame function proportionsComparable functions and sizes
ASPDAC’24+0.51+0.07+0.43
DATE’25+1.04+0.91+0.66
DATE’26+0.99+0.88+0.67

Conclusion: Adjusting function proportions and circuit size changes the Test-1/Test-2 error gap, and the size of this change varies across models. Among samples with similar functions and sizes, Test-2 errors remain 0.43–0.67 percentage points higher than Test-1 errors for all three models.

(2) Testing how data source, circuit size, and topology similarity affect the Test-2/Test-3 error comparison. We kept the original models and first compared only ASAP7 cells. We then compared samples with equal input/output and MOS counts and similar total fin counts. Finally, we also required similar distances to the nearest topology in the training set. Table 2 reports the results.

Table 2: MAPE gaps under different comparison conditions (Test-3 − Test-2, percentage points).

ModelAll samplesASAP7 onlyComparable source and sizeAlso comparable topology distance
ASPDAC’24+1.96+1.63+1.94+0.93
DATE’25+2.87+1.79+2.30+0.94
DATE’26+2.94+1.75+2.13+1.04

Conclusion: Comparing only ASAP7 cells reduces the error gap for all three models. The gap increases again when comparing samples of similar size, then falls to 0.93–1.04 percentage points after accounting for topology similarity. Thus, the Test-2/Test-3 error comparison is affected by data source and topology similarity.

(3) Measuring the additional error for unseen topologies and functions. We fixed 644 test netlists from 38 functions across ASAP7 and TeMACLE and retrained the three models for two comparisons: (i) seen versus unseen topologies when the function is seen; and (ii) seen versus unseen functions when the topology is unseen in both cases. Training groups have the same sample counts, source composition, and MOS/fin distributions. Table 3 reports MAPE differences averaged over the five PPA metrics.

Table 3: Additional error for unseen topologies or functions with comparable source and size (percentage points).

ModelAdditional error: unseen topologyAdditional error: unseen function
ASPDAC’24+0.15+0.63
DATE’25+0.62+0.67
DATE’26+0.91+0.87

Conclusion: When the function is seen during training, prediction error is 0.15–0.91 percentage points higher for unseen topologies. When the topology is unseen in both cases, prediction error is 0.63–0.87 percentage points higher when the function is also unseen. This shows that including the corresponding topologies or functions in training helps reduce prediction error.

AuthorsOpenReview ↗

Response to Reviewer XuqT (Part 2/2)

Answer to Question 2

Thank you for this insightful question.

All 136,217 negative samples in CellBench-2 were reported as UNSAT by the original tool, meaning no feasible solution under the constraints of that run. Timeouts are not used as negative samples. We selected 668 historical failure samples, considering data source, MOS count, and total fin count. With transistor placement fixed, we relaxed groups of routing constraints to identify the groups involved in conflicts. Table 4 reports the constraint conflict distribution and the three models' correct identification rates: the proportion of these samples correctly predicted as unroutable.

Table 4: Constraint conflict distribution and correct classification rates for unroutable samples.

Constraint groupSamples (percentage)ASPDAC’24DATE’25DATE’26
Vertex mutual exclusion across nets668 (100.00%)75.22%82.78%84.66%
Metal spacing666 (99.70%)75.30%82.88%84.76%
Minimum metal area657 (98.35%)75.11%82.50%84.40%
Pin access234 (35.03%)64.96%75.43%77.14%
Path constraints214 (32.04%)63.79%76.87%79.21%
Via rules14 (2.10%)85.71%82.14%75.00%
Overall668 (100.00%)75.22%82.78%84.66%

Conclusion: Conflicts in most samples involve vertex exclusion, metal spacing, and minimum area constraints together. Some also involve pin access, path constraints, or via rules. The same design can involve multiple conflicts.

Thank you for the detailed response and additional experiments. The matched comparisons and retraining on a fixed test set clarify how data source, circuit size, and topology similarity affect the observed generalization gaps. The routing analysis also answers my second question by showing that failures often involve multiple overlapping constraint groups. These results address my main concerns, and I will keep my current rating of 3.

Thank you for your careful review and constructive feedback. We are glad that the additional analyses addressed your main concerns.

Reviewer_g2HLOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for evaluating the generalization of standard-cell performance prediction from pre-layout netlists. It provides datasets for post-layout PPA regression and constrained routability classification, constructed through logic-function extension, topology generation, and transistor sizing. The benchmark evaluates generalization to unseen size configurations, topologies, and functions. Experiments with three graph-based models reveal declining prediction performance across these settings. Further analysis suggests that topological similarity is more closely associated with prediction accuracy than functional similarity.

Critical strengths
  1. A valuable benchmark for standard-cell evaluation. CellBench provides a large dataset with post-layout PPA and routability labels derived from physical design tools. Its coverage of different logic functions and circuit topologies makes it a useful resource for evaluating learning-based prediction methods.

  2. A meaningful study of model generalization. The benchmark evaluates unseen size configurations, topologies, and functions rather than relying on random data splits. Results across three baselines reveal clear generalization challenges. The analysis of topology and functional distances also provides useful insights into prediction errors.

Critical weaknesses
  1. The generalization settings are not fully isolated. As noted in Section 5.3, Test-2 differs from the training set not only in topology but also in the distribution of logic functions. Test-3 further includes newly extended functions. These differences make it difficult to determine how much of the performance degradation is caused by unseen topologies or functions. A more controlled analysis would strengthen the conclusions.

  2. The practical value of the benchmark needs further validation. The experiments focus on prediction accuracy and ranking performance, but do not show whether these results lead to better cell selection or optimization. In particular, good overall ranking performance does not necessarily mean the model can identify the best candidates. A downstream optimization experiment would help demonstrate the benchmark's practical relevance.

Questions for authors
  1. Can the authors clarify how much of the performance degradation is caused by unseen topologies rather than changes in the function distribution? A comparison with matched function distributions would help support the conclusions about generalization.

  2. Can the authors evaluate how well the prediction models select promising cell designs? A simple top-k candidate selection experiment using the available PPA and routability labels would help demonstrate the practical value of the benchmark.

2 comments
AuthorsOpenReview ↗

Response to Reviewer g2HL (Part 1/2)

Explanation to Weakness 1 and Answer to Question 1 (joint response)

Thank you for suggesting more controlled comparisons to explain the generalization errors.

CellBench evaluates generalization to progressively broader design changes, rather than the separate effects of sizing, topology, and function. Testing unseen functions also involves new topologies and sizing configurations that implement them, not just changing function labels. Function changes therefore cannot be fully separated from changes in topology and sizing. In graph learning terms, this setting tests generalization to unseen graph semantics (Boolean functions), which depend on graph topology and node attributes.

To better explain the observed performance differences, we conducted three analyses.

(1) Testing whether function proportions and circuit size affect the Test-1/Test-2 error gap. We used the original model predictions for the 42 functions shared by both test sets. We first calculated overall errors using the same function proportions, then compared samples with the same function, equal MOS counts, and similar total fin counts. Table 1 reports the results.

Table 1: MAPE gaps under different comparison conditions (Test-2 − Test-1, percentage points).

ModelOriginal gapSame function proportionsComparable functions and sizes
ASPDAC’24+0.51+0.07+0.43
DATE’25+1.04+0.91+0.66
DATE’26+0.99+0.88+0.67

Conclusion: Adjusting function proportions and circuit size changes the Test-1/Test-2 error gap, showing that both affect the comparison. However, among samples with similar functions and sizes, prediction errors for unseen topologies in Test-2 remain 0.43–0.67 percentage points higher than in Test-1 for all three models.

(2) Measuring the additional error for unseen topologies and functions. We fixed 644 test netlists from 38 functions and retrained the models for two comparisons: (i) seen versus unseen topologies when the function is seen; and (ii) seen versus unseen functions when the topology is unseen in both cases. Training groups have the same sample counts, source composition, MOS counts, and device parameter distributions. Table 2 reports MAPE differences averaged over the five PPA metrics.

Table 2: Additional error for unseen topologies or functions on fixed test netlists (percentage points).

ModelAdditional error: unseen topologyAdditional error: unseen function
ASPDAC’24+0.15+0.63
DATE’25+0.62+0.67
DATE’26+0.91+0.87

Conclusion: When the function is seen during training, prediction error is 0.15–0.91 percentage points higher for unseen topologies. When the topology is unseen in both cases, prediction error is 0.63–0.87 percentage points higher when the function is also unseen. This shows that including the corresponding topologies or functions in training helps reduce prediction error.

(3) Testing how data source, circuit size, and topology similarity affect the Test-2/Test-3 error comparison. We kept the original models unchanged and first compared only ASAP7 cells. We then selected samples of similar size and further required similar topology distances to the training set.

Table 3: Mean MAPE gaps across five PPA metrics under different sample comparisons (Test-3 − Test-2, percentage points).

ComparisonASPDAC’24DATE’25DATE’26
All samples in the original Test-2 and Test-3+1.96+2.87+2.94
Only ASAP7 cells from both test sets+1.63+1.79+1.75
Within the above source, equal input/output and MOS counts and total fin counts in the same bin+1.94+2.30+2.13
Above, with distance to the nearest training topology also in the same bin+0.93+0.94+1.04

Conclusion: Comparing only ASAP7 cells reduces the error gap for all three models. The gap increases again when comparing samples of similar size, then falls to 0.93–1.04 percentage points after accounting for topology similarity. Thus, the Test-2/Test-3 error comparison is affected by data source and topology similarity.

AuthorsOpenReview ↗

Response to Reviewer g2HL (Part 2/2)

Explanation to Weakness 2 and Answer to Question 2 (joint response)

Thank you for your suggestion.

We added three experiments to test PPA selection under limited evaluation budgets, candidate selection using routability predictions, and circuit area optimization with TeMACLE. The first two use predictions from trained models and existing labels. The third runs layout generation for new cells and remaps the circuits.

(1) Comparing model-based and random cell selection under the same budget. We used existing predictions and PPA labels for 48 functions in Test-3. Within each function, we select the top k candidates by the predicted value of each PPA metric, then choose the one with the best true value. We compare this with random selection under the same budget. Table 4 reports mean regret (the percentage gap between the selected candidate's true value and the best value in the same function pool), averaged over functions and the five metrics.

Table 4: Mean regret (%) under different candidate evaluation budgets.

MethodTop-15% budget10% budget
ASPDAC’2414.03%0.65%0.33%
DATE’255.10%0.19%0.03%
DATE’263.69%0.16%0.07%
Random191.13%23.44%15.02%

Conclusion: All three models have lower mean regret than random selection under every budget, showing that models can select cells closer to the true optimum with a limited evaluation budget.

(2) Testing the effect of routability prediction on candidate selection. We added unroutable candidates to the pools for these 48 functions and compared three evaluation orders: ranking by predicted PPA, ranking by predicted routability probability, and evaluating candidates predicted to be routable first, followed by the remaining candidates, with each group ranked by predicted PPA.

Table 5: Candidate selection strategies when the pool includes unroutable candidates.

Each entry gives the rate of finding a valid candidate / the near optimal selection rate / mean regret when a valid candidate is found (%). Near optimal means within 1% of the pool optimum. Random reference with the same budget: 99.34/15.59/24.29.

ModelPPA rankingRoutability rankingRoutability-first
ASPDAC’2494.58/84.86/1.13100.00/33.89/26.1699.03/70.14/4.36
DATE’2598.06/95.14/0.16100.00/30.83/33.7399.72/80.28/2.25
DATE’2698.33/95.14/0.14100.00/33.33/34.50100.00/77.78/3.12

Conclusion: When the pool includes unroutable candidates, PPA-only ranking achieves a near optimal selection rate of 84.86%–95.14% across the three models, compared with 15.59% for random selection. Routability-first ranking increases the chance of finding a candidate whose layout can be generated, but reduces the near optimal selection rate. This shows a trade-off between prioritizing routable implementations and selecting the best PPA.

(3) Testing practical gains from model-assisted cell library extension. We integrated the three models into TeMACLE and tested 18 EPFL circuits. A uses predicted area to guide selection; A+R first removes candidates predicted to be unroutable, then uses predicted area.

Table 6: Model prediction performance, downstream area gains with TeMACLE, and speedups.

ModelTest-3 area MAPE ↓Test-3 routability accuracy ↑Mean area reduction: A / A+R ↑Speedup: A / A+R ↑
ASPDAC’248.919%75.39%5.124% / 5.231%2.16× / 5.93×
DATE’255.324%81.00%6.672% / 5.916%3.24× / 5.33×
DATE’265.320%81.09%6.615% / 6.473%1.69× / 2.18×

Conclusion: Compared with ASPDAC’24, DATE’25 and DATE’26 predict area more accurately and achieve larger circuit area gains in both TeMACLE settings. Relative to the original TeMACLE flow, speedups are 1.69–3.24× with area prediction alone and 2.18–5.93× with routability filtering.

Reviewer_MhNqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces CellBench for predicting post-layout PPA and constrained routability from standard-cell transistor netlists. It constructs datasets using function extension, topology generation, and transistor sizing under ASAP7, and evaluates three existing predictors on unseen sizes, topologies, and functions. The results show decreasing performance across these settings and a stronger association of prediction error with structural distance than functional distance.

Critical strengths
  • The benchmark addresses a relevant problem in standard-cell library development. Its separation of size, topology, and function generalization provides a useful evaluation structure.
  • The dataset construction involves substantial engineering effort, including topology search, physical implementation, and post-layout simulation. Evaluating both PPA and layout-generation outcomes is practically motivated.
Critical weaknesses
  • Practical utility is insufficiently validated. The paper does not establish whether the generated topology and sizing variants represent useful candidates in realistic library optimization, or whether better benchmark performance translates into better cell selection, reduced evaluation cost, or improved PPA.
  • The research contribution appears limited. While the topology generator and dataset construction are valuable, the main findings—performance degradation on unfamiliar designs and its association with structural distance—offer limited new insight. Stronger evidence of practical utility or actionable findings is needed to establish the benchmark’s significance.
Questions for authors
  • What evidence shows that the generated candidates resemble those considered in practical standard-cell optimization, rather than predominantly redundant or uncompetitive implementations?
  • Does better performance on CellBench translate into better outcomes in practical standard-cell optimization? Can the authors provide evidence that methods ranked higher on the benchmark also select better cells or reduce physical evaluation cost while preserving solution quality?
5 comments
AuthorsOpenReview ↗

Response to Reviewer MhNq (Part 1/3)

Explanation to Weakness 1 and Answer to Question 2 (joint response)

Thank you for your questions about candidate usefulness, selection cost, and optimization gains.

We provide additional evidence on candidate competitiveness, cell selection under limited budgets, actual physical evaluation cost, and downstream optimization. These experiments test whether the generated designs offer useful choices and whether prediction performance on the benchmark translates into practical gains.

(1) Generated candidates offer useful PPA trade-offs. Comparing candidates within each function, 9,951 candidates lie on the five metric Pareto front, accounting for 8.37% of CellBench-1. After combining them with ASAP7 library cells characterized under the same conditions, non-dominated candidates remain in all nine function subsets. Table 4 in Part 3 (Answer to Question 1) summarizes the results.

Conclusion: The generated candidates offer competitive PPA trade-offs and additional choices beyond the original library for standard cell optimization.

(2) Models improve cell selection under limited budgets. To test whether models select better cells than random selection under the same budget, we used existing predictions and PPA labels for 48 functions in Test-3, treating each of the five PPA metrics as a separate objective. Within each function, we select the top k candidates by the predicted target metric, then choose the one with the lowest true value. Table 1 reports mean regret (the percentage gap between the selected candidate's true value and the best value in the same function pool), averaged over functions and metrics. The random reference uses the same evaluation budget.

Table 1: Mean regret under different candidate evaluation budgets.

MethodTop-1 regret5% candidate budget regret10% candidate budget regret
ASPDAC’2414.03%0.65%0.33%
DATE’255.10%0.19%0.03%
DATE’263.69%0.16%0.07%
Random selection with the same budget191.13%23.44%15.02%

Conclusion: All three models have much lower mean regret than random selection, showing that model predictions improve selection quality under the same budget. DATE’25 and DATE’26 both have lower regret than ASPDAC’24, linking better benchmark performance to lower selection loss.

(3) Measured cell selection quality and cost. To test whether models can select high quality cells with fewer layout generation and post-layout simulation runs, we selected eight candidates each for NOR4 and p3_ef from Test-3. The objective is ADP = area × worst delay. For each model, we generate layouts and run post-layout simulation only for the two candidates with the best predicted ADP, then choose the one with the lower measured ADP. We compare this with random selection under the same budget and exhaustive evaluation of all eight candidates. Physical results are generated anew for each strategy, as reported in Table 2.

Table 2: Physical evaluation counts and ADP gaps from the exhaustive optimum for model-assisted selection.

StrategyEvaluations per functionNOR4 ADP gapp3_ef ADP gap
ASPDAC’2420.00%+11.97%
DATE’2520.00%0.00%
DATE’2620.00%0.00%
Random selection2+225.64%+7.49%
Independent exhaustive reference8ReferenceReference

Conclusion: DATE’25 and DATE’26 match the best ADP from an independent exhaustive evaluation of eight candidates using only two evaluations. This reduces layout generation and post-layout simulation runs by 75% and outperforms random selection under the same budget. Model-assisted selection can therefore find high quality implementations with fewer physical evaluations.

AuthorsOpenReview ↗

Response to Reviewer MhNq (Part 2/3)

Explanation to Weakness 1 and Answer to Question 2 (joint response, continued)

(4) Prediction models support downstream cell library optimization. To test whether benchmark prediction performance reflects practical optimization results, we integrated the three models into TeMACLE and compared final area gains and runtime on 18 EPFL circuits. We compared two settings: using predicted area alone to guide candidate selection, and first removing candidates predicted to fail layout generation, then using predicted area to guide selection.

Table 3: Model prediction performance, downstream area gains with TeMACLE, and speedups.

Speedups are relative to the original TeMACLE flow.

ModelTest-3 area MAPE ↓Test-3 routability accuracy ↑Area-only: mean area reduction ↑Area + routability: mean area reduction ↑Area-only: speedup ↑Area + routability: speedup ↑
ASPDAC’248.919%75.39%5.124%5.231%2.16×5.93×
DATE’255.324%81.00%6.672%5.916%3.24×5.33×
DATE’265.320%81.09%6.615%6.473%1.69×2.18×

Compared with ASPDAC’24, DATE’25 and DATE’26 predict area more accurately and achieve larger circuit area gains in both settings. Speedups are 1.69–3.24× with area prediction alone. With routability filtering, candidates predicted to be unroutable are removed, and speedups increase to 2.18–5.93×.

Conclusion: Model-assisted selection can find high quality implementations with fewer physical evaluations, and better benchmark performance can translate into better downstream optimization results.

Explanation to Weakness 2

Thank you for suggesting that we explain the benchmark's practical value more clearly. CellBench does more than report prediction errors on unseen designs. It provides candidates with different functions, topologies, and transistor sizing configurations. It also defines common tasks for PPA and constrained routability prediction, hierarchical data splits, and evaluation protocols.

To test whether prediction performance on CellBench translates into practical optimization gains, we added candidate selection and downstream TeMACLE experiments, reported in Tables 1–2 in Part 1 and Table 3 above. Compared with ASPDAC’24, DATE’25 and DATE’26 perform better on CellBench and also achieve lower candidate selection loss and larger downstream circuit area gains. This shows that CellBench can help identify prediction models that perform better in practical standard cell optimization.

AuthorsOpenReview ↗

Response to Reviewer MhNq (Part 3/3)

Answer to Question 1

(1) Candidates reflect design changes used in practical cell optimization. At the function level, we use 77 combinational logic functions from the ASAP7 library and 22 extension functions mined by TeMACLE from EPFL benchmark circuits. These support the practical tasks of optimizing existing cells and extending a cell library. At the topology level, we search for functionally equivalent implementations within a limited transistor count range. We use transistor removal and edge contraction to explore candidates with fewer transistors and different connections. Candidates undergo truth table validation, structural checks, and isomorphism-based deduplication. At the sizing level, we follow ASAP7 standard cell sizing practices, accounting for diffusion sharing and output stage drive requirements. CellBench-1 also uses an established physical implementation and simulation flow: AutoCellGen for layout generation, and the commercial tools Calibre and Spectre for design verification, parasitic extraction, and post-layout simulation to obtain PPA labels.

(2) Candidates are not mainly duplicate circuits. The 118,906 netlists in CellBench-1 contain 13,813 topologies. All netlists are distinct when both circuit connections and device configurations are considered. Even after merging parallel devices, 116,819 distinct circuits remain. The dataset thus contains a broad range of topology variants and their sizing configurations.

(3) The generated pool contains PPA competitive candidates. We first computed the five-metric Pareto front within each function in CellBench-1. We then selected nine functions and compared their candidates with 33 ASAP7 library cells characterized using the same flow. These functions include inverters, NAND/NOR gates, and AOI/OAI gates of different complexity. Table 4 reports the library comparison.

Table 4: PPA competitiveness of generated candidates compared with ASAP7 library cells.

The third column gives non-dominated candidates / total candidates.

Function groupLibrary referencesAll PPA candidates
Inverter (1 function)1023/25
NAND/NOR (2 functions)11159/526
AOI/OAI (6 functions)12922/6504

Conclusion: In CellBench-1, 9,951 candidates lie on their function's five metric Pareto front, accounting for 8.37% of all netlists. Non-dominated candidates remain in all nine function subsets after comparison with the original library, showing that the generated pool offers candidates with PPA competitive with original library cells.

Answer to Question 2

Thank you for this question. The experiments and conclusions are provided in Parts 1–2 (joint response to Weakness 1 and Question 2).

Reviewer_VCKTOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CellBench, a benchmark for predicting standard-cell power, performance, area (PPA), and constrained routability directly from pre-layout transistor netlists. Based on the ASAP7 technology, it provides approximately 119k netlists with post-layout PPA labels and 296k netlists with routability labels, together with evaluation splits for generalization to unseen transistor sizes, circuit topologies, and Boolean functions. The authors also propose a topology-generation procedure combining exact enumeration and heuristic search. Experiments with three graph-based prediction baselines demonstrate increasing generalization difficulty across the three settings, while additional analyses investigate the relationship between structural and functional similarity and prediction accuracy.

Critical strengths

Relevant and well-motivated benchmark contribution

CellBench addresses an important gap in ML-based electronic design automation by providing a large-scale, publicly available benchmark for predicting post-layout PPA and constrained routability directly from pre-layout standard-cell netlists. The explicit separation of generalization to unseen transistor sizes, topologies, and Boolean functions is particularly valuable for developing models that can support practical standard-cell library optimization.

Thorough experimental evaluation and reproducibility

The authors evaluate three established graph-based methods across multiple prediction targets, generalization settings, and random seeds. The additional analyses of functional and structural similarity provide useful insights into the limitations of existing approaches. The paper is also clearly written and documents its dataset construction, evaluation protocols, and experimental limitations in considerable detail, supporting reproducibility and future research.

Critical weaknesses

Limited validation of practical applicability

The benchmark evaluates prediction accuracy but does not demonstrate how these predictions translate into improved standard-cell selection or downstream PPA optimization. Although ranking metrics are reported, the main evaluation compares cells across different functions rather than directly measuring the quality of selecting among implementations of the same function. The appendix partially addresses this through within-function ranking, but no actual selection regret is evaluated. Moreover, constrained routability labels reflect the success of a particular layout-generation tool under potentially nonuniform computational budgets rather than general physical routability. All measurements are also restricted to one technology and characterization setting. Consequently, the practical generalizability and usefulness of the benchmark beyond the evaluated setup remain uncertain.

Questions for authors

1. Generalization splits and potential confounding (clarification)

How can the authors distinguish generalization to unseen Boolean functions from changes in circuit complexity, topology distributions, and data sources, particularly since all 22 TeMACLE-generated functions are assigned to Test-3?

2. Practical relevance for standard-cell optimization (additional evidence)

The benchmark is motivated by accelerating standard-cell library optimization, where the goal is to select high-quality implementations of the same Boolean function. While the appendix reports within-function ranking correlations, could the authors additionally evaluate the actual PPA regret when selecting the predicted-best implementation compared with the ground-truth best candidate? This should be possible using the existing predictions and labels and would provide stronger evidence that the benchmark metrics reflect practical design decisions.

6 comments
AuthorsOpenReview ↗

Response to Reviewer VCKT (Part 1/6)

Explanation to Weakness 1

Thank you for recognizing CellBench and for your suggestions on validating its practical use.

(1) Candidate design selection. We test whether models can maintain cell selection quality while reducing layout generation and post-layout simulation runs. The objective is to minimize the area–delay product (ADP). From CellBench Test-3, we selected eight candidates each for NOR4 (from the original library) and p3_ef (from TeMACLE). For each model, we generate layouts and run post-layout simulation only for the two candidates with the lowest predicted ADP, then select the one with the lowest measured ADP. Table 1 compares this approach with random selection under the same budget and an exhaustive evaluation of all eight candidates.

Table 1: Measured results of model-assisted candidate selection.

ADP is reported in µm²·ps. Parentheses show the gap from the exhaustive optimum.

StrategyEvaluations per functionNOR4: ADP (gap)p3_ef: ADP (gap)
ASPDAC’2424.5095 (0.00%)10.1594 (+11.97%)
DATE’2524.5095 (0.00%)9.0732 (0.00%)
DATE’2624.5095 (0.00%)9.0732 (0.00%)
Random selection214.6847 (+225.64%)9.7524 (+7.49%)
Independent exhaustive reference84.5095 (reference)9.0732 (reference)

Conclusion: For both functions, DATE’25 and DATE’26 match the best ADP from exhaustive evaluation of eight candidates using only two evaluations. This reduces layout generation and post-layout simulation runs by 75% and outperforms random selection under the same budget.

AuthorsOpenReview ↗

Response to Reviewer VCKT (Part 2/6)

Explanation to Weakness 1 (continued)

Downstream area optimization with TeMACLE. To test whether prediction models can help optimize real circuits, we integrated the three models into TeMACLE and compared final circuit area and runtime on 18 EPFL gate-level circuits. We compared two settings: guiding candidate cell selection using predicted area alone, and first removing candidates predicted to fail layout generation, then guiding selection using predicted area.

Table 2: Model prediction performance, downstream area gains with TeMACLE, and speedups.

Speedups are relative to the original TeMACLE flow.

ModelTest-3 area MAPE ↓Test-3 routability accuracy ↑Area-only: mean area reduction ↑Area + routability: mean area reduction ↑Area-only: speedup ↑Area + routability: speedup ↑
ASPDAC’248.919%75.39%5.124%5.231%2.16×5.93×
DATE’255.324%81.00%6.672%5.916%3.24×5.33×
DATE’265.320%81.09%6.615%6.473%1.69×2.18×

Conclusion: All three model-assisted flows reduce the mean final mapped circuit area, showing that prediction models can support practical cell library extension. Compared with ASPDAC’24, DATE’25 and DATE’26 predict area more accurately and achieve larger downstream area gains. Relative to the original TeMACLE flow, speedups are 1.69–3.24× with area prediction alone and 2.18–5.93× with routability filtering.

(2) Candidate selection quality within the same function. To directly evaluate cell selection quality within the same function, we compared model-based and random selection using existing predictions and PPA labels. For each PPA metric, the model selects the candidates with the best predicted values, while the random strategy selects the same number of candidates. Each strategy then chooses the candidate with the best true value from its selected set. Table 3 reports mean regret (the percentage gap between the selected candidate's true value and the best value in the same function pool).

Table 3: Within function selection under different candidate evaluation budgets (Test-3, mean regret, %).

MethodTop-15% candidate budget10% candidate budget
ASPDAC’2414.030.650.33
DATE’255.100.190.03
DATE’263.690.160.07
Random selection with the same budget191.1323.4415.02

Conclusion: All three models outperform random selection under all three budgets. With a 5% candidate evaluation budget, their mean regret is 0.16%–0.65%, compared with 23.44% for random selection. Thus, models can select implementations close to the best value of the target PPA metric within the same function candidate pool using a limited evaluation budget.

AuthorsOpenReview ↗

Response to Reviewer VCKT (Part 3/6)

Explanation to Weakness 1 (continued)

(3) Routability. We agree that routability depends on implementation conditions such as area constraints and cell architecture. In topology optimization and cell sizing, the practical task is to determine whether candidate designs can be physically implemented under given conditions. We therefore call this task constrained routability prediction: predicting whether a candidate netlist can produce a layout under specified tool settings and resource limits, to guide candidate selection during optimization. CellBench uses a fixed cell architecture and placement and routing configuration. Time limits vary with cell size and solver cost, but all candidate implementations of the same cell have the same time limit. This prevents budget differences from affecting comparisons between candidates and allows a fair comparison of their routability under the same conditions. This matches the practical need described above. Similarly, TOPCELL trains a GNN on NVCell 2 placement and routing results to predict candidate topology routability and guide optimization.

(4) A single technology and characterization setting. This work focuses on standard cell design at advanced nodes, where complex parasitic effects and strict layout rules make large-scale candidate exploration and PPA evaluation more challenging. We use ASAP7, an advanced node PDK widely used in EDA research. It has an established flow for layout generation, design verification, parasitic extraction, and post-layout simulation. Other published studies provide references to check the design and simulation flow and compare results, which also helps other researchers reproduce our work. For other advanced node PDKs, public references for these checks remain limited, making it difficult to verify the design and simulation flow. We therefore use ASAP7 to build a consistent, reproducible benchmark.

For characterization, CellBench targets early exploration in transistor sizing, topology optimization, and cell library extension. We use fixed typical characterization conditions to provide a common reference for candidate screening before final characterization under multiple conditions.

Reviewer_cbf4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CellBench, an open dataset and benchmarks for fast standard cell evaluation with both PPA and constrained routability lables.

Critical strengths
  • A large-scale open dataset and benchmark for fast standard cell evaluation.
  • A clear and scalable dataset expansion workflow through function extension, topology generation, and size configuration assignment.
Critical weaknesses
  • Limited logic functions.
  • No newly introduced and relatively larger scale benchmarks.
Questions for authors
  • The final PPA labels are computed with a specific P&R tool. How do the fixed tool based labels provide insights if the P&R algorithms are improved? Do the PPA prediction models need to be updated if the PPA output changes due to evolving P&R algorithms ?
1 comment
AuthorsOpenReview ↗

Response to Reviewer cbf4

Explanation to Weakness 1

Thank you for your comment on logic function coverage. CellBench is not limited to a few basic gates. It covers 77 common combinational logic functions from the ASAP7 PDK, including cells with 1–9 inputs and multiple outputs. It also includes 22 complex logic functions mined by TeMACLE from gate-level circuits, for a total of 99 combinational logic functions.

Explanation to Weakness 2

Thank you for your questions about benchmark novelty and circuit size. To our knowledge, CellBench is the first open dataset and benchmark for fast standard cell evaluation. Unlike FusionCell and EPiCell, which mainly explore layout variants of existing cells, we combine our topology generation algorithm with library extension and sizing to build pre-layout netlists with different logic functions, transistor topologies, and sizing configurations. CellBench supports the evaluation of both PPA and routability prediction.

CellBench-1 and CellBench-2 contain 118,906 and 296,274 distinct netlists, respectively. For comparison, the in-house datasets of FusionCell and EPiCell contain approximately 19,500 and 18,000 layout variants, respectively. FusionCell, EPiCell.

Complex cells are difficult to place and route reliably with our layout tool under the fixed physical constraints and compute budget. We cover as broad a range of cell sizes as these limits allow. Each netlist in CellBench-1 contains 2–28 MOS transistors. Each pre-layout netlist in CellBench-2 contains 2–48 MOS transistors.

Answer to Question 1

Thank you for asking us to clarify how tool updates affect the labels and models. AutoCellGen optimizes cell width and wirelength, allowing CellBench to provide PPA labels based on high quality physical implementations. To test the effect of changing placement and routing tools, we generated layouts for the same netlists from CellBench Test-3 using AutoCellGen and CPCELL, then obtained PPA with the same post-layout simulation flow.

Table 1: Mean relative PPA changes with CPCELL compared with AutoCellGen (%).

FlowWorst rise delayWorst fall delayLeakage powerInternal energyArea
CPCELL+1.499+0.746−0.005+1.417+1.954

To test the original models after the tool change, we kept the models trained on AutoCellGen labels unchanged and evaluated their predictions for the same netlists against labels from each tool. As shown in Table 2, the mean prediction errors of all three models changed little.

Table 2: Original model prediction errors against labels from the two placement and routing tools (MAPE, %).

Values to the left and right of each arrow are the MAPE against AutoCellGen and CPCELL labels, respectively.

MetricASPDAC’24DATE’25DATE’26
Worst rise delay21.98 → 21.757.11 → 7.388.98 → 8.19
Worst fall delay29.07 → 29.379.64 → 9.498.78 → 8.15
Leakage power9.60 → 9.605.30 → 5.305.89 → 5.90
Internal energy10.24 → 11.968.77 → 10.678.52 → 9.16
Area4.50 → 5.913.84 → 6.034.53 → 3.95
Mean over five metrics15.07 → 15.726.93 → 7.777.34 → 7.07

This work focuses on netlist-level changes in function, topology, and transistor sizing. We therefore use a fixed physical implementation flow to compare model predictions across candidate designs consistently. If a tool update substantially changes the PPA labels, the original models can be evaluated against labels from the new flow. The resulting prediction errors can then guide whether the models need to be updated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.