- Rating
- 4(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces WebLoop, a reinforcement learning framework for functional web generation that extends execution-based supervision to intermediate critique and subsequent refinement. A shared policy serves as the Generator, Critic, and Refiner, with the Critic trained using requirement-level diagnostic accuracy and downstream refinement gains. The framework is evaluated using Qwen3.5-9B and Qwen3.5-27B on two functional web generation benchmarks, WebRise and WebGen-Bench.
Critical strengths
- The paper addresses a clearly motivated and practically valuable problem that has received relatively limited systematic study: how to use execution feedback to optimize the intermediate Critic between generation and refinement in functional web generation. The supervision design is closely tied to the task. The paper also provides meaningful mechanism analyses and openly presents failures, distinguishing incorrect diagnoses, inaccurate fault localization, incomplete repair guidance, and Refiner-induced regressions, which helps clarify the system’s limitations.
- The experimental design is clear. In addition to comparisons with the original models and existing critique-learning methods, the paper includes generation-only training, refinement without critique, staged training, and joint training as controls. Furthermore, evaluations cover two model sizes, 9B and 27B, two functional web generation benchmarks, and five input modalities, providing some breadth to the findings.
Critical weaknesses
- Helpfulness depends on both feedback quality and the Refiner’s ability to implement the suggested changes. Figure 4(b) provides partial evidence by fixing the Generator/Refiner checkpoint and varying Critic checkpoints, but diagnostic accuracy, repair success rates, and regression rates for previously correct functionality are still missing on a fixed, independent set of drafts.
- Computational costs and key configurations are insufficiently documented. The paper does not adequately report actual training costs, inference latency, or gains under matched computational budgets. The rationale for selecting the warm-up duration and reward weight is also unclear.
Questions for authors
- Helpfulness is directly affected by the current Refiner’s capabilities. Have the authors considered fixing the Refiner during training or evaluation to compare the repair gains induced by different Critic outputs? If the same set of critiques is given to Refiners with different capabilities, do the Helpfulness rankings remain consistent?
- Why are the first 15 updates used to optimize only Discriminability, followed by the combined reward with ? Were these hyperparameters selected using a validation set or based on other empirical considerations? How would shortening or extending the diagnostic warm-up, or changing the relative reward weights, affect the final results?
- Do the test results in Tables 1–4 use execution rewards for candidate selection, resampling, or fallback? Please specify the sampling counts, selection rules, and token limits for each role, and clarify the apparent inconsistency between the ≥20k critique-length bucket in Figure 5 and the configuration in Appendix A.1. Please also report the performance gains under matched computational budgets and the additional computational costs.
7 comments
Response to Reviewer tKU7 (Part 2 / 4)
Q2: Why use a 15-step diagnostic warm-up and λ=0.2? How sensitive are the results to these choices?
A2: Our motivation for diagnostic warm-up is to establish the Critic’s ability to judge requirement satisfaction before optimizing downstream repair utility. Helpfulness depends on both the critique and the Refiner, so a failed repair does not necessarily indicate an incorrect diagnosis. We therefore begin with direct supervision from requirement-level execution labels, then introduce Helpfulness to encourage actionable feedback.
Based on our experience, we chose a 15-step warm-up and λ=0.2. All configurations use the same Qwen3.5-9B backbone, with Base denoting the original model. To assess sensitivity, we evaluated warm-up durations of 10, 15, and 20 steps and reward weights λ ∈ {0.1, 0.2, 0.4, 0.8} on both benchmarks.
R3-Table 2: Warm-up duration, with λ=0.2 fixed.
| Configuration / output | WebRise | WebGen‑Bench | ||||||
|---|---|---|---|---|---|---|---|---|
| T ↑ | Rₑ ↑ | Rᵢ ↑ | Overall ↑ | Yes (%) ↑ | Partial (%) ↑ | No (%) ↓ | Acc. ↑ | |
| Base | 23.15 | 45.29 | 22.11 | 30.18 | 18.39 | 10.20 | 67.08 | 23.49 |
| warm-up=10 (Gen) | 27.53 | 49.79 | 25.28 | 34.20 | 26.58 | 10.82 | 59.35 | 31.99 |
| warm-up=10 (Refine) | 33.02 | 54.61 | 30.19 | 39.27 | 27.98 | 12.83 | 56.41 | 34.39 |
| WebLoop (warm-up=15, Gen) | 31.72 | 55.38 | 30.03 | 39.05 | 28.75 | 10.82 | 56.88 | 34.16 |
| WebLoop (warm-up=15, Refine) | 34.55 | 57.89 | 31.98 | 41.48 | 31.68 | 14.37 | 48.38 | 38.87 |
| warm-up=20 (Gen) | 30.86 | 54.05 | 28.59 | 37.83 | 26.28 | 10.51 | 60.43 | 31.53 |
| warm-up=20 (Refine) | 33.68 | 57.10 | 30.92 | 40.57 | 31.53 | 11.44 | 52.86 | 37.25 |
R3-Table 3: Reward weight, with a 15-step warm-up fixed.
| Configuration / output | WebRise | WebGen‑Bench | ||||||
|---|---|---|---|---|---|---|---|---|
| T ↑ | Rₑ ↑ | Rᵢ ↑ | Overall ↑ | Yes (%) ↑ | Partial (%) ↑ | No (%) ↓ | Acc. ↑ | |
| Base | 23.15 | 45.29 | 22.11 | 30.18 | 18.39 | 10.20 | 67.08 | 23.49 |
| λ=0.1 (Gen) | 29.04 | 52.75 | 26.52 | 36.10 | 21.74 | 11.59 | 63.37 | 27.54 |
| λ=0.1 (Refine) | 32.37 | 55.76 | 29.76 | 39.29 | 27.98 | 12.67 | 56.57 | 34.31 |
| WebLoop (λ=0.2, Gen) | 31.72 | 55.38 | 30.03 | 39.05 | 28.75 | 10.82 | 56.88 | 34.16 |
| WebLoop (λ=0.2, Refine) | 34.55 | 57.89 | 31.98 | 41.48 | 31.68 | 14.37 | 48.38 | 38.87 |
| λ=0.4 (Gen) | 27.11 | 49.51 | 24.78 | 33.80 | 21.17 | 8.04 | 67.70 | 25.19 |
| λ=0.4 (Refine) | 30.17 | 51.93 | 27.35 | 36.48 | 29.06 | 11.75 | 53.63 | 34.93 |
| λ=0.8 (Gen) | 28.27 | 51.70 | 25.69 | 35.22 | 22.87 | 9.43 | 63.83 | 27.59 |
| λ=0.8 (Refine) | 32.73 | 55.67 | 29.27 | 39.23 | 32.30 | 11.90 | 51.78 | 38.25 |
The results show that the WebLoop configuration (15 steps, λ=0.2) achieves the highest initial-generation and final-refinement scores on both benchmarks among the tested settings. For warm-up duration, 15 steps improves final WebRise Overall and WebGen-Bench Acc. by 2.21/4.48 points over 10 steps and by 0.91/1.62 points over 20 steps. Similarly, λ=0.2 achieves the best final scores (41.48 Overall and 38.87 Acc.), while increasing or decreasing λ does not consistently improve performance.
Response to Reviewer tKU7 (Part 3 / 4)
Q2 (continued)
To further understand these differences, we evaluate the Critics from each configuration on the same held-out WebRise drafts with the WebLoop Refiner fixed, separating diagnostic accuracy from downstream repair and regression rates.
R3-Table 4: Critic warm-up sensitivity on fixed drafts (λ=0.2).
| Critic configuration | Violated-requirement accuracy ↑ | Satisfied-requirement accuracy ↑ | Repair rate ↑ | Regression rate ↓ |
|---|---|---|---|---|
| warm-up=10 | 45.67 | 64.79 | 13.00 | 8.92 |
| WebLoop (warm-up=15) | 48.67 | 61.50 | 11.67 | 8.92 |
| warm-up=20 | 39.33 | 62.77 | 12.00 | 9.86 |
R3-Table 5: Critic reward-weight sensitivity on fixed drafts (15-step warm-up).
| Critic configuration | Violated-requirement accuracy ↑ | Satisfied-requirement accuracy ↑ | Repair rate ↑ | Regression rate ↓ |
|---|---|---|---|---|
| λ=0.1 | 46.67 | 54.46 | 12.33 | 13.62 |
| WebLoop (λ=0.2) | 48.67 | 61.50 | 11.67 | 8.92 |
| λ=0.4 | 50.67 | 56.81 | 13.67 | 8.92 |
| λ=0.8 | 47.67 | 55.40 | 9.33 | 7.51 |
An insufficient warm-up can leave the Critic underprepared to identify unmet requirements, whereas an overly long warm-up can limit its learning of feedback that guides actual repairs. With drafts and the Refiner fixed, 10 steps yields higher repair than 15 steps (13.00% vs. 11.67%), consistent with earlier Helpfulness supervision encouraging actionable suggestions. Yet violated-requirement accuracy is lower (45.67% vs. 48.67%), suggesting underdeveloped diagnosis. At 20 steps, satisfied-requirement accuracy improves slightly over 15 steps (62.77% vs. 61.50%), but violated-requirement accuracy drops to 39.33% and repair remains below the 10-step result (12.00% vs. 13.00%). Longer diagnosis-only training therefore does not consistently improve defect identification or repair guidance.
With insufficient diagnostic weight (λ), the Critic may flag satisfied requirements as defects and prompt damaging edits; excessive weight can weaken its guidance for actual repairs. Reducing λ from 0.2 to 0.1 increases the relative influence of Helpfulness: repair rises from 11.67% to 12.33%, but both diagnostic accuracies decline and regression increases from 8.92% to 13.62%. At λ=0.4, violated-requirement accuracy and repair reach 50.67% and 13.67%, respectively, while satisfied-requirement accuracy falls to 56.81%, revealing more false alarms despite stronger repair guidance. Increasing λ to 0.8 reduces the emphasis on Helpfulness, weakening the incentive for actionable suggestions; repair falls to 9.33% and regression to 7.51%, indicating fewer repairs alongside less damage.
Response to Reviewer tKU7 (Part 1 / 4)
We sincerely thank the reviewer for the thorough assessment and valuable suggestions. We particularly appreciate your recognition of WebLoop's clear motivation and practical significance in addressing the relatively underexplored challenge of optimizing intermediate critique through execution feedback. We are also encouraged by your positive assessment of our task-aligned supervision design, insightful mechanism and failure analyses, and comprehensive evaluations across different model scales, benchmarks, and input modalities.
To address your concerns, we provide additional experimental evidence, further analyses, and clarifications below.
Q1: How much do different Critics help a fixed Refiner? Do their rankings change across Refiners?
A1: To assess whether Critic feedback remains effective across Refiners with different capabilities, we conducted a Critic × Refiner cross-pairing experiment on WebRise. Base is the original Qwen3.5-9B model; WebLoop-20 and WebLoop-45 use the same backbone after joint training. Using the same Base-generated drafts (initial Overall = 30.18), we provide saved critiques from the Base, WebLoop-20, and WebLoop-45 Critics, unchanged, to both the Base and WebLoop-45 Refiners. WebLoop-20 and WebLoop-45 denote checkpoints at training steps 20 and 45, respectively. This setup isolates the effect of Critic feedback within each fixed-Refiner setting.
As shown in R3-Table 1, the feedback utility ranking remains consistent across both Refiners (WebLoop-45 > WebLoop-20 > Base). Compared with the Base Critic, WebLoop-45 feedback improves Overall by 1.02 points with the Base Refiner and 1.90 points with the WebLoop-45 Refiner, demonstrating its effectiveness even without a jointly trained Refiner. Although the Refiner affects the magnitude of these gains (e.g., the WebLoop-45–WebLoop-20 gap increases from 0.02 to 0.90 points), the ranking remains unchanged. Together with the independent diagnostic evaluation in Q4, these findings support the learned Critic's contribution to both requirement-level diagnosis and downstream refinement across different Refiners.