acceptodds
Withdrawn from ICLR 2027

Towards Self-Adaptive GenUI Experiences: An Approach, Benchmark & Evaluation Suite

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

13 views

Abstract

Generative UI (GenUI) requires producing complete interactive experiences rather than static webpages: so the interfaces must remain functional, stateful, and usable across views and interactions. While frontier models generate visually plausible HTML, their outputs frequently fail under interaction, and these failures recur across tasks. We introduce SAGE-Create, a training-free framework that converts verified GUI repairs into reusable multimodal memory. Its Reflect–Refine–Remember loop infers a task specification, generates and executes candidate interfaces, and applies focused repairs that are admitted to memory only when before-and-after replay confirms that the failure is resolved and required behavior is preserved. Central to this loop is a multimodal memory that records how each rule fixed a concrete failure through code patches, screenshots, and interaction traces, together with a learning curriculum that governs rule creation, promotion, and retirement. To evaluate GenUI experiences, we introduce SAGE-Bench, 300 tasks spanning an intent-based taxonomy of 11 experience families and 48 subfamilies, and SAGE-Eval, which measures perceptual alignment, visual layout, and data integrity alongside specification fidelity and execution reliability. SAGE-Create achieves 74.12 on SAGE-Bench, surpassing the strongest baseline by 4.86 points and the no-memory variant by 2.08 points; its frozen memory transfers to other generators and improves WebGen-Bench performance from 54.56 to 57.94. Qualitative examples show clearer task structure and improved responsive layouts. Together, these results support reusable repair memory as a way to improve GenUI experiences across tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
GenUI experiances

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_ucf6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SAGE-CREATE, a framework for improving generative UI quality through task specifications, execution-based verification, and reusable multimodal repair memory. The system follows a Reflect–Refine–Remember workflow to reuse verified repairs across tasks. The authors also introduce SAGE-BENCH, comprising 300 tasks, and SAGE-EVAL, which assesses specification fidelity, execution reliability, and experience quality. The framework is evaluated through ablation studies, cross-model transfer, an external benchmark, and expert assessments.

Critical strengths
  1. The framework combines execution-based verification with reusable repair memory. Code patches, screenshots, and interaction traces provide supporting evidence for retrieving and validating previous repairs.

  2. The paper includes ablation studies, comparisons with alternative strategies, cross-model transfer, external benchmark evaluation, and expert assessments, providing evidence for examining the contributions and applicability of the proposed approach.

Critical weaknesses
  1. The approach relies heavily on accurate and complete task specifications. SAGE-BENCH provides structured contracts specifying components, states, views, and interactions, leaving its effectiveness under ambiguous or incomplete user requirements insufficiently evaluated. Moreover, repair acceptance relies on limited execution probes, which may miss functional regressions and propagate incorrect repair experiences across tasks. These issues raise concerns about reliability and generalization in realistic settings.

  2. The methodological novelty and practical benefits remain insufficiently justified. SAGE-CREATE combines established techniques, including execution-based verification, program repair, experience retrieval, and memory management. The experiments do not sufficiently establish the necessity of the more complex memory mechanisms over simpler alternatives. Furthermore, the paper motivates the approach through reduced user corrections but does not directly measure human intervention. The efficiency comparison also lacks matched backbone models and output quality, making the practical cost advantage difficult to assess.

Questions for authors
  1. How does SAGE-CREATE perform with only the original natural-language prompt, without a structured task contract? Can the authors evaluate requirement inference accuracy and the impact of incomplete specifications on task completion?

  2. Can the authors quantify incorrect repair acceptance, functional regressions, and negative transfer when execution probes fail to cover important interactions?

  3. Can the authors compare against simpler experience-retrieval and verified-repair approaches under matched generation and computation budgets to clarify the independent contributions of multimodal memory and curriculum management?

  4. Can the authors directly measure user correction effort and compare overall costs under matched backbone models and output quality, including memory construction and maintenance overhead?

Reviewer_vQZaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents SAGE-Create, a system that generates web UIs, tests their behavior, and stores successful repairs as lessons for later tasks. Each lesson includes a rule, code changes, screenshots, and interaction traces. A curriculum controls which lessons the system retains and reuses, without updating model weights. The authors also introduce SAGE-Bench and SAGE-Eval, with experiments on memory components, transfer to other generators and WebGen-Bench, and validation against expert assessments.

Critical strengths

I found the method practical, especially the part that checks whether a repair preserves existing behavior before storing it for future use. A change that fixes the layout could also hide a required control or lose a user's selection. The presented procedure addresses that problem and keeps the evidence behind each lesson. This is a useful design choice for a system that could otherwise repeat a harmful repair across tasks.

On the evaluation side, the authors compare against both no memory and a static checklist, test the frozen memory on an external benchmark, and include expert comparisons of generated interfaces. I particularly appreciate the latter two evaluations, since the evidence for improvement extends beyond the authors' own automated scores.

Critical weaknesses

The ablations show that memory generally helps, but I could not tell whether the multimodal design improves on text lessons managed by the same curriculum. Specifically, the text memory condition does not receive the curriculum that the full system uses (Table 3). Would text lessons learned from the same repairs work similarly with the same rules for lesson selection and retention? The static checklist is a useful control, but does not answer that question. In addition, prior work like ExpeL [1], ACE [2], and Dynamic Cheatsheet [3] already uses experience from earlier tasks to update reusable guidance without changing model weights. A comparison with an existing memory method, given the same repair experience and budget, would help explain what SAGE-Create uniquely contributes (I would also report variation across runs for the component comparisons).

I also could not follow how the system turns task requirements into executable tests for each generated interface. The author's appendices describe what the checks should establish, but leave the connection between those requirements and the generated code unclear. For example, how does a test identify which record should change after an edit and determine its correct value? An incorrect test could approve a faulty repair that then becomes guidance for later tasks. The expert study supports the quality of final outputs, but does not demonstrate how reliably the repair checks detect regressions. I would like to see a complete example with the code, tests, and expected outcomes, coupled w an independent check of accepted repairs.


References (introduced newly in the review):

[1] Zhao, Andrew, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. "ExpeL: LLM agents are experiential learners." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, pp. 19632-19642. 2024.

[2] Zhang, Qizheng, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton et al. "Agentic context engineering: Evolving contexts for self-improving language models." In International Conference on Learning Representations, vol. 2026, pp. 86069-86100. 2026

[3] Suzgun, Mirac, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. "Dynamic cheatsheet: Test-time learning with adaptive memory." In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7080-7106. 2026.

Questions for authors

Main questions (things to clarify/improve):

(1) Could the authors compare text lessons and multimodal lessons with the same curriculum, repair examples, and context budget, and report uncertainty for the difference? That comparison would affect overall assessment of the memory contribution. Also, how the proposed method differentiate from prior work (in similar theme), and how does an existing memory method perform with the same repair experience and verifier?

(2) How are the executable tests and expected outcomes constructed and connected to the generated code? Please provide a complete repair example with the tests and results before and after the change. Have accepted repairs been checked independently for regressions?

(3) How were the development tasks constructed, and how did the authors check for overlap with evaluation prompts and task requirements? Examples of the development tasks behind a few retrieved lessons would help clarify how much the tasks differ.

More details:

(A) What is the overall accuracy under the original WebGen-Bench evaluation, with and without memory? Please also report Data Management and Design Validation. Table 7 averages selected groups that overlap, so it is hard to tell whether the improvement holds across the full benchmark.

(B) Which overall scores for GPT-5 and Claude are correct? The weighted averages of the family scores agree with Table 5 vs. Table 2. Also, was the Claude model Sonnet or Opus (there seems to be some consistency issue)?

Reviewer_7zZYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SAGE, a framework for improving generative UI systems by learning from past repairs without updating model weights. The main idea is to turn verified UI fixes into reusable memory, so the model can avoid making similar mistakes on future tasks.

The proposed SAGE-Create framework follows a Reflect–Refine–Remember loop. It first identifies task requirements and retrieves relevant memories, then generates and tests candidate UIs in a browser. Repairs are stored only when verification confirms that they fix the target issue while preserving required behavior. A memory curriculum manages the creation, promotion, and retirement of these lessons.

The paper also introduces SAGE-Bench, a benchmark with 300 tasks across 11 experience families and 48 subfamilies, and SAGE-Eval, an evaluation suite covering specification fidelity, execution reliability, and experience quality.

Experiments report that SAGE-Create outperforms the tested baselines. Ablation studies show additional improvements from memory and its curriculum, and the stored memory can also benefit other models and improve performance on an external benchmark.

Critical strengths
  1. The paper applies cross-task repair memory to GenUI generation. Verified fixes from earlier tasks are stored and retrieved to guide later generation, helping address recurring interaction and layout issues that would otherwise require repeated corrections. A repair is admitted to memory only if it fixes the target failure while preserving previously passing checks. This helps avoid storing fixes that merely hide problems or introduce regressions.
  2. The ablation studies clearly separate the gains from the generation and verification pipeline from the additional improvements brought by memory.
Critical weaknesses
  1. Inconsistencies in Reported Scores and Model ID (Tables 2 and 5). The reported overall scores in Table 2 do not consistently match the task-weighted averages of the 11 experience-family scores. Table 5 reports the correct aggregates of 68.98 and 66.56 for GPT-5 and Claude 4.8, despite listing exactly the same per-family scores as Table 2. In addition, Table 2 labels the Claude model as Sonnet in the column header but Opus in the table note.
  2. Unclear construction of development tasks. The paper does not explain how the development tasks used to build memory are generated, or whether they share taxonomy leaves or the task planner with the test set. This leaves open the possibility of leakage.
  3. Limited novelty of the memory curriculum. Replay-verified admission, utility-based promotion and pruning, and helpful/harmful counters already appear in prior memory systems such as SEDM, ACE and ExpeL, none of which are discussed.
  4. Memory construction cost is not accounted for, and the “low token cost” claim is only partly supported. The cost of the 100 self-play iterations used to build the memory (Sec. 6) is never reported or amortized. At inference time, memory adds only about 6.5k tokens (89.0k → 95.5k, Table 4). However, the full pipeline uses 95.5k tokens per GUI, about 8 times as many tokens as a single draft (12.0k, Table 6), and it uses fewer tokens only than five-round refinement on a different model. The “low token cost” claim therefore holds for the memory component, but not for the system as a whole.
Questions for authors
  1. Could the authors clarify how the overall scores in Table 2 are calculated? The task-weighted averages appear to be 68.98 for GPT-5 and 66.56 for Claude 4.8, matching Table 5 but not Table 2. Also, was Claude Sonnet or Opus used in the experiments?
  2. How were the development tasks used to build memory generated? Do they share taxonomy leaves, task templates, or the same task planner with the evaluation set? How was overlap between development and evaluation tasks prevented?
  3. Could the authors clarify the key differences between the proposed memory curriculum and prior memory systems such as SEDM, ACE, and ExpeL? Which mechanisms are new beyond existing approaches to memory admission, promotion, and pruning?
  4. What is the total computational or token cost of the 100 self-play iterations used to build memory? How does the full pipeline compare with single-pass generation and iterative refinement under comparable model and compute budgets?
  5. Appendix D.5 states that defect detection and weight sensitivity are tested in Appendix E, but Appendix E only reports the human study. Could the authors provide the missing results?
Reviewer_r9GjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SAGE-CREATE, a framework for generating functional web interfaces without updating the generator’s weights. It translates a request into an experience contract, retrieves relevant repair lessons, generates candidate interfaces, and tests them in a browser. A targeted repair is accepted only when before-and-after replay shows that it resolves the observed failure while preserving previously working behavior. Accepted repairs become reusable lessons containing rules, code patches, screenshots, and interaction traces, with a curriculum governing their promotion and removal.

The authors also introduce SAGE-BENCH, comprising 300 tasks, and SAGE-EVAL, an 11-dimension evaluation suite. On SAGE-BENCH, the full system scores 74.12, compared with 72.04 without memory and 69.26 for the strongest direct-generation baseline. The paper additionally evaluates cross-model memory transfer, transfer to WebGen-Bench, and agreement between SAGE-EVAL and expert ratings.

Critical strengths

SAGE-CREATE filters lessons by applicability and negative evidence, ranks them using similarity, past outcomes, and recency, and limits the influence of new lessons until they have succeeded on distinct tasks.

The authors compare SAGE-CREATE with direct generation by multiple LLMs and with WebGen-Agent on 300 tasks. They also test whether frozen lessons help other generators and transfer to the external WebGen-Bench, and compare SAGE-EVAL with expert ratings. This breadth makes the reported benefits more informative than a result on the authors’ benchmark alone.

Critical weaknesses
  • Much of the improvement over direct generation is already present without cross-task memory: the same base model scores 67.50 with direct generation, 72.04 with the no-memory pipeline, and 74.12 with the full system. Since task-level testing and revision can themselves improve outputs, the paper should more clearly establish what is distinctive about its verification-and-repair loop relative to existing iterative refinement methods.

  • The ablations examine text lessons, multimodal evidence, and the memory curriculum in detail. Although there are some sampling and repair controls, the experience contract, browser verification, targeted repair, and candidate selection largely change together. This makes their individual contributions to the substantial no-memory gain difficult to assess.

  • The curriculum credits lessons based on subsequent patches that follow their rules (Appendix B.6), while the central motivation is to prevent errors during initial generation. A lesson that successfully prevents a defect may therefore produce no repair episode from which to receive credit. The paper does not establish whether repair success reliably predicts preventive utility. Reporting failure recurrence before refinement would help substantiate this central mechanism.

  • The multimodal ablation adds code patches, screenshots, interaction traces, and verification context simultaneously (Tab 3) Consequently, the reported improvement could arise from richer information or concrete repair examples, leaving the contribution of multimodality unclear. A comparison against equally informative text-and-code memory, together with separate ablations of visual and behavioral evidence, would help establish the specific value of the proposed multimodal design.

Questions for authors

What specifically distinguishes SAGE-CREATE’s task-level verification-and-repair loop from conventional generate–test–revise methods? Which existing comparisons establish this distinction under comparable browser feedback and revision budgets?

Could the authors isolate the effects of the experience contract, browser verification, targeted repair, and candidate selection under matched model and compute budgets?

How many distinct development tasks were used in the 100 self-play iterations, and how were they constructed? How was separation from the 300 SAGE-BENCH evaluation tasks enforced at the level of interface patterns, scenario templates, and specifications?

The paper states that lessons retrieved before generation can prevent failures, but lesson-level success appears to require a verified repair patch that follows the lesson. Are successful generations requiring no repair counted as lesson trials or uses? If not, how does the curriculum avoid under-crediting lessons whose main effect is preventive, especially when several lessons are retrieved together?

At the frozen checkpoint, how many lessons are candidate, active, cooled, and restricted? What are the distributions of distinct-task support, retrieval frequency, and verified use across lessons?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.