- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces WebMix, a framework for adapting web agents to unseen websites without additional fine-tuning. WebMix first trains a bank of interface-specific LoRA adapters. A scout agent then explores a target website and generates an interface manual, which a learned router uses to predict mixing coefficients for composing the existing adapters. Experiments on TimeWarp and WebArena-Lite-v2 demonstrate improved performance on unseen domains and website versions compared to existing baselines.
Critical strengths
S1. Interesting Motivation and Approach
The paper addresses an important problem of adapting web agents to unseen interfaces without retraining. The proposed approach of combining interface exploration with adapter composition is intuitive and interesting.
S2. Clear Presentation
The paper is well-written, with a clear explanation of the proposed methodology and overall framework
Critical weaknesses
W1. Limited Practical Scalability and Comparison with General-Purpose Web Agents
While WebMix enables adaptation without target-side fine-tuning, it requires training and maintaining a separate LoRA adapter for each source interface. This raises concerns about scalability as the number of websites increases (e.g., training and storing over 1k lora adapters for each source website?). Also, although the paper compares against a unified LoRA baseline, it does not sufficiently investigate whether adapter composition offers practical advantages over stronger general-purpose web agents trained across diverse websites, such as those developed using VeriEnv or Weblica.
W2. Potential Target-Side Information Leakage and Unfair Baseline Comparisons
In the domain-transfer setting, the scout’s writer has access to reference procedures from target-domain training tasks, even though the final answers are removed. These procedures may provide substantial task-specific supervision, raising concerns about the claimed adaptation without target demonstrations. Moreover, the baselines do not appear to receive equivalent target-side information, making it difficult to disentangle the benefits of adapter composition from those of additional procedural knowledge.
Questions for authors
- Could the authors provide an ablation without target-domain reference procedures and compare against baselines using the same scout-generated manual?
- How does WebMix scale with more source interfaces, and what are its practical advantages over general-purpose web agents in terms of training, storage, and inference costs?
5 comments
Thank you for the response. I appreciate the authors' efforts to address my concerns, and some of them have been resolved.
However, I still have two concerns regarding W1.
-
The additional cost analysis helps clarify the computational overhaed. However, adding a new source interface still requires retraining the router. Although this is inexpensive, it introduces and additional training and maintenance step whenever the expansion happens.
-
My original concern was about the practical advantage of WebMix over general purpose web agents. The authors explain they are complementary, but this does not directly address my request for an empirical comparison (or joint evaluation). I believe this point is crucial, because it directly relates to the motivation.
Reply by the Authors
Thank you for following up on our rebuttal. We are glad that some of your concerns have been resolved! We respond to the remaining points below.
1. Router training and maintenance
We agree that our current approach still requires training a router, and we appreciate that you raised this issue. To remove this requirement, we evaluate a training-free variant, WebMix (prompted). In this version, we prompt the frozen web agent to select the adapter coefficients based on the scout's description of the target domain. This fully bypasses the router training and would only require an adapter bank. We evaluate WebMix (prompted) on TimeWarp domain transfer using the Qwen-3.5 4B model. The results turned out to be surprisingly good.
| Method | Wiki | News | Shop | Avg |
|---|---|---|---|---|
| WebMix (Router) | 24.73% | 56.06% | 25.93% | 33.75% |
| WebMix (Prompted) | 22.58% | 60.61% | 33.33% | 36.67% |
The prompted variant raises the average success rate from 33.75% to 36.67% (+2.92 points). It gains on News (+4.55) and Shop (+7.40) and drops slightly on Wiki (−2.15). We investigate this further by inspecting the weight coefficients produced by the trained router and the prompted agent.
LLM-prompted weights
| Held-out \ Expert | Wiki | News | Shop |
|---|---|---|---|
| Wiki | — | 0.89 | 0.11 |
| News | 0.74 | — | 0.26 |
| Shop | 0.58 | 0.42 | — |
Trained router
| Held-out \ Expert | Wiki | News | Shop |
|---|---|---|---|
| Wiki | — | 0.77 | 0.23 |
| News | 0.63 | — | 0.37 |
| Shop | 0.41 | 0.59 | — |
The weights are mostly consistent, showing that even without training, a prompted web agent can match the performance of a trained router. This is a very important finding, and we again thank the reviewer for this!
2. Empirical results against general-purpose agents
Thank you for clarifying this point. We compare WebMix, VeriEnv, and a joint training of WebMix with VeriEnv on TimeWarp domain transfer using Qwen-3.5 4B.
| Method | Wiki | News | Shop | Avg |
|---|---|---|---|---|
| VeriEnv | 21.51% | 43.94% | 27.16% | 29.58% |
| WebMix | 24.73% | 56.06% | 25.93% | 33.75% |
| VeriEnv + WebMix | 32.26% | 51.52% | 34.57% | 38.33% |
On average, WebMix outperforms VeriEnv, but combining both gives the best overall results at +4.58 pp above the base WebMix. The gains are higher on Wiki and Shop, as the synthetic VeriEnv domains are highly similar to TimeWarp's Wiki and Shop domains. Based on the results, combining WebMix with synthetic, scalable approaches such as VeriEnv is a promising way to improve generalization.
Thank you again for following up on our rebuttal! Your feedback is helping us shape our work into a much stronger paper.
We sincerely thank you for the thoughtful feedback and for recognizing our motivation, approach, and presentation. Your comments have helped us improve our work by analyzing scalability, practical advantages, and adding empirical results on procedure-free scouts.
W1/Q2. Scalability
Thank you for raising this! We have measured the parameter count, training, and inference costs for bank sizes K = 1–10, which goes beyond the previous maximum of five adapters (Table 5).
We have also added a new research question on scale and cost: What is the overhead in terms of parameters, compute, and inference as the adapter bank grows?
W1. Practical advantage over general-purpose web agents
VeriEnv (Chae et al., 2026) and Weblica (Kar et al., 2026) widen the training distribution by cloning or synthesizing websites at scale. However, each new interface then requires retraining on the full corpus. WebMix instead adapts at test time to unfamiliar interfaces by composing existing adapters, with no target demonstrations or target-side fine-tuning. Only the small router (~0.26M) needs retraining, which is much cheaper computationally. The two directions are actually complementary. A stronger general-purpose agent could serve as WebMix's backbone.
W2/Q1. Domain scout using target reference procedures and potential bias
Thank you for the careful observation! We agree that the current scout setup can provide additional task-solving guidance from the domain procedures. To isolate its effect, we removed the procedures and kept only the description of the target domain. The procedure-free baseline (n=0) performs on par with the other scout baselines and leads with the highest average of 34.17% (Table 3). This indicates that the router relies on the domain description instead of the procedure. We have added this discussion in the related works section.
We also compare against similar adapter composition baselines, such as uniform composition, MoLE, LoRAHub, and RRR (Table 4). WebMix, using the procedure-free setting, outperforms the other composition baselines.
We have added all the results and discussions to the revised manuscript and highlighted them in blue.
Thank you again for the review! Your feedback has helped us shape our work into a much stronger paper.