- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CellBench benchmarks standard-cell PPA and constrained routability prediction from pre-layout netlists. It provides 118,906 PPA samples and 296,274 routability samples, with tests on unseen sizes, topologies, and functions. Three adapted baselines reveal generalization gaps, followed by analyses of functional and structural similarity.
Critical strengths
- The paper addresses a practical bottleneck in standard-cell optimization and provides a substantial dataset with structured generalization splits, creating a useful resource for evaluating early-stage prediction methods.
- The analysis goes beyond aggregate accuracy to examine functional and topological distances, within-function ranking, and uncertainty, offering useful insights into prediction difficulty.
Critical weaknesses
- Although Appendix D.3 includes useful controls within Test-3, comparisons across test sets still combine changes in topology, function composition, and data source. This limits how specifically the aggregate performance gaps can be interpreted.
Questions for authors
- Beyond the within-Test-3 controls, how sensitive is the Test-2/Test-3 performance gap to data source and circuit complexity?
- Do the retained routing-failure records reveal distinct failure categories? If available, a brief breakdown would help characterize the negative class.
4 comments
Response to Reviewer XuqT (Part 2/2)
Answer to Question 2
Thank you for this insightful question.
All 136,217 negative samples in CellBench-2 were reported as UNSAT by the original tool, meaning no feasible solution under the constraints of that run. Timeouts are not used as negative samples. We selected 668 historical failure samples, considering data source, MOS count, and total fin count. With transistor placement fixed, we relaxed groups of routing constraints to identify the groups involved in conflicts. Table 4 reports the constraint conflict distribution and the three models' correct identification rates: the proportion of these samples correctly predicted as unroutable.
Table 4: Constraint conflict distribution and correct classification rates for unroutable samples.
| Constraint group | Samples (percentage) | ASPDAC’24 | DATE’25 | DATE’26 |
|---|---|---|---|---|
| Vertex mutual exclusion across nets | 668 (100.00%) | 75.22% | 82.78% | 84.66% |
| Metal spacing | 666 (99.70%) | 75.30% | 82.88% | 84.76% |
| Minimum metal area | 657 (98.35%) | 75.11% | 82.50% | 84.40% |
| Pin access | 234 (35.03%) | 64.96% | 75.43% | 77.14% |
| Path constraints | 214 (32.04%) | 63.79% | 76.87% | 79.21% |
| Via rules | 14 (2.10%) | 85.71% | 82.14% | 75.00% |
| Overall | 668 (100.00%) | 75.22% | 82.78% | 84.66% |
Conclusion: Conflicts in most samples involve vertex exclusion, metal spacing, and minimum area constraints together. Some also involve pin access, path constraints, or via rules. The same design can involve multiple conflicts.
Thank you for the detailed response and additional experiments. The matched comparisons and retraining on a fixed test set clarify how data source, circuit size, and topology similarity affect the observed generalization gaps. The routing analysis also answers my second question by showing that failures often involve multiple overlapping constraint groups. These results address my main concerns, and I will keep my current rating of 3.
Thank you for your careful review and constructive feedback. We are glad that the additional analyses addressed your main concerns.
Response to Reviewer XuqT (Part 1/2)
Explanation to Weakness 1 and Answer to Question 1 (joint response)
Thank you for your detailed suggestions on comparisons across test sets.
CellBench evaluates generalization to progressively broader design changes, rather than the separate effects of sizing, topology, and function. Testing unseen functions also involves new topologies and sizing configurations that implement them, not just changing function labels. Function changes therefore cannot be fully separated from changes in topology and sizing. In graph learning terms, this setting tests generalization to unseen graph semantics (Boolean functions), which depend on graph topology and node attributes.
To better explain prediction error gaps across test sets, we added the following comparisons.
(1) Testing how function proportions and circuit size affect the Test-1/Test-2 error comparison. We used the original model predictions for the 42 functions shared by both test sets. We first calculated overall errors using the same function proportions, then compared samples with the same function, equal MOS counts, and similar total fin counts. Table 1 reports MAPE differences averaged over the five PPA metrics (Test-2 − Test-1).
Conclusion: Adjusting function proportions and circuit size changes the Test-1/Test-2 error gap, and the size of this change varies across models. Among samples with similar functions and sizes, Test-2 errors remain 0.43–0.67 percentage points higher than Test-1 errors for all three models.
(2) Testing how data source, circuit size, and topology similarity affect the Test-2/Test-3 error comparison. We kept the original models and first compared only ASAP7 cells. We then compared samples with equal input/output and MOS counts and similar total fin counts. Finally, we also required similar distances to the nearest topology in the training set. Table 2 reports the results.
Conclusion: Comparing only ASAP7 cells reduces the error gap for all three models. The gap increases again when comparing samples of similar size, then falls to 0.93–1.04 percentage points after accounting for topology similarity. Thus, the Test-2/Test-3 error comparison is affected by data source and topology similarity.
(3) Measuring the additional error for unseen topologies and functions. We fixed 644 test netlists from 38 functions across ASAP7 and TeMACLE and retrained the three models for two comparisons: (i) seen versus unseen topologies when the function is seen; and (ii) seen versus unseen functions when the topology is unseen in both cases. Training groups have the same sample counts, source composition, and MOS/fin distributions. Table 3 reports MAPE differences averaged over the five PPA metrics.
Conclusion: When the function is seen during training, prediction error is 0.15–0.91 percentage points higher for unseen topologies. When the topology is unseen in both cases, prediction error is 0.63–0.87 percentage points higher when the function is also unseen. This shows that including the corresponding topologies or functions in training helps reduce prediction error.