- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
UIReq Bench evaluates whether generated UI code meets component and layout requirements. It contains 2861 annotated industrial designs and evaluates eight models on 421 designs. Separate tasks examine requirement inference and code generation with or without supplied requirements.
Critical strengths
The dataset captures implementation requirements that visual similarity scores can miss. The distinction between required and optional annotations also allows some flexibility in implementation. Supplying requirements improves code scores substantially while changing visual scores much less. This is a useful finding for evaluating UI generation.
Critical weaknesses
- The paper does not quantify agreement between the automatic evaluator and independent human judgments. It is unclear how often valid alternative implementations receive incorrect penalties.
- The evaluation comes from one commercial ecosystem and reports no repeated-generation results or confidence intervals. The stability of close model rankings is therefore unclear.
Questions for authors
How closely do automatic scores agree with expert judgments, including cases with valid alternative implementations? Are the main findings and model rankings stable across design collections and repeated generations?