- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
GUIHub studies post-hoc reuse of independently trained GUI LoRA adapters. Adapters are profiled on a labeled global support set (GSS) of 768 episodes, and their centered behavioral responses are projected into a frozen eight-dimensional representation. An unlabeled local support set (LSS) specifies target demand: relevance distributions over the 64 nearest GSS episodes are computed separately for each support example and then averaged. Demand–capability scores determine the selected adapters and their weights. The default composition contains two adapters, using an available own/source adapter as an anchor or selecting the two highest-scoring candidates when no anchor is available. Their weighted output contributions are scaled by 1.5 on Qwen and 1.0 on Gemma, and the composition is reused across requests.
Evaluation uses teacher-forced next-action prediction on AndroidControl-derived mobile trajectories, with 24 in-hub and eight held-out synthetic semantic clients. IID/OOD and expansion experiments cover three backbones; mixed-client experiments cover two Qwen backbones; detailed ablations, diagnostics, and timing use Qwen3.5-4B. A single-client route uses ten selected LSS episodes, while support preparation accesses 40 distinct local instruction/screenshot inputs (Appendix A.2).
Critical strengths
-
The routing diagnostics provide direct evidence about candidate selection. All 24 adapters are evaluated on each of 32 targets using common singleton execution settings, including scale 1.5. Behavioral scores achieve a mean within-target Spearman correlation of 0.513, compared with 0.103 for the specified semantic scorer. Selected-set regret is 1.09 pp [0.60, 1.64] versus 3.07 pp [1.93, 4.33] (Table 15). These measurements do not use composition weights, so they support selection quality independently of how the selected adapters are combined. Figure 5(b) shows that regret reductions are uneven across targets, with many near zero and a smaller group contributing larger reductions. This qualifies the average selection benefit; it does not measure target-level gains in composed-model Step SR.
-
The framework addresses a useful reuse setting and has broad, well-documented offline evidence. GUIHub has the highest reported IID/OOD Step SR among the evaluated methods on all three backbones (Table 1). Paired-episode intervals exclude zero for all three IID comparisons with LoraRetriever and both Qwen OOD comparisons; Gemma OOD remains uncertain (Table 16). The expansion study evaluates seven pool sizes from 8 to 32 on the same targets, and Table 9 also tracks a fixed initially covered cohort. The paper clearly documents information access, prediction reuse, non-monotonic individual outcomes, and timing exclusions. Figures 1–2 make the profiling/adaptation/serving distinction clear, while Figures 6–7 describe heterogeneity in the constructed client distributions.
Critical weaknesses
- The advantage over directly using one's own adapter is not yet cleanly attributed, and the measured increment from combining two adapters is modest. Table 12 reports 64.98 IID Step SR for GUIHub versus 46.90 for Own-LoRA, but Appendix D.1 does not explicitly specify Own-LoRA's additional scale or its correspondence to GUIHub's execution path. The 63.43 IID score for behavioral top-1 in Table 14 changes adapter selection as well, so its 16.53 pp difference from Own-LoRA cannot estimate the effect of gamma. Figure 5(a), which pools different target–adapter pairs, does not resolve this ambiguity. The main baselines also use different composition operators, including factor fusion, so an explicit gamma is not sufficient to compare effective update magnitudes across methods.
Table 14 does provide a useful controlled adapter-count comparison. Within the anchor-free group at the same scale, moving from one to two adapters changes IID/OOD/mixed Step SR by approximately +0.62/−0.24/+0.35 pp. Thus, one selected adapter already achieves most of the accuracy in these conditions. These comparisons do not isolate the own-anchor/helper combination used in the main IID and mixed experiments, and paired intervals for the adapter-count differences are not reported. I would prioritize a common-path Own-LoRA control and uncertainty estimates for the existing singleton-versus-pair results. Cross-client selection is itself a useful form of adapter sharing; the issue is how much additional benefit can be attributed to simultaneous composition and how the larger Own-LoRA gap should be interpreted.
- The semantic ablation identifies an advantage of the implemented behavioral scorer, but does not fully separate the sources of the composed-model gain. Equation (11) uniformly averages semantic similarities over the LSS and an adapter source client's probes. GUIHub instead aggregates measured behavioral responses using target-dependent relevance over the shared GSS. Downstream selection, routing temperature, composition scale, and evaluation settings are fixed. The 1.02/5.18/3.86 pp IID/OOD/mixed differences therefore compare two complete scoring procedures, including their aggregation and score distributions.
This limitation should not erase the positive selection evidence: Table 15's rank-based diagnostics cannot be explained by a change in softmax weight concentration alone. Calibration could nevertheless affect the magnitude of the composed-model accuracy difference. The M = 64 versus 768 experiment makes truncation alone an unlikely explanation in the tested behavioral configuration; it does not test uniform versus soft relevance or all interactions with a different scorer. Reporting within-target score dispersion and selected-weight entropy, followed by an equal-weight composition check on the existing selected sets or a development-set calibration check, would provide a focused way to investigate this remaining ambiguity.
Additional Concerns
(a) The projection is an explicitly disclosed compression trade-off. Direct matching exceeds the projected variant by 0.21/0.76/0.26 pp in Table 13, without paired intervals for these small differences. Appendix D.2 already explains descriptor compression, shared storage, and the absence of measured indexing speedups. These results support a compact representation with similar observed accuracy; they do not show an accuracy benefit from the routing projection. Actual indexing or memory measurements at larger pool sizes would strengthen the deployment case. Direct matching retains the SVD used in GSS construction.
(b) Efficiency is established for the recorded workload with prebuilt assets. Figure 4 and Tables 10–11 show that GUIHub's advantage over LoRAHub comes from setup, while its recorded online mean is slightly higher (4.354 versus 4.102 s/action). Initial GSS selection requires 24 reference adapters to evaluate 1,200 candidate episodes; a new adapter is profiled on 768 selected probes. The costs of these offline operations are excluded, so their share of total cost and the amortization point remain unknown. Table 11 already reports target setup, including constructing and applying compositions. The useful missing evidence is offline construction/admission cost and its amortization over stated workloads, with the unpaired timing conditions kept explicit.
(c) LoGo's mixed-client evaluation does not replay the mixed request sequence. Appendix B.1 discloses reuse of single-client predictions, whereas Appendix A.4 describes routing state carried across episodes. This does not establish performance under a newly ordered mixed workload; the direction and size of any difference are unknown. A replay would strengthen the LoGo comparison. This concern does not by itself invalidate the comparison with LoraRetriever, whose prediction reuse is justified under identical inputs and settings.
(d) Expansion and transfer have specific experimental limits. The capability basis is fitted using 24 IID reference adapters before serving-pool expansion begins at eight, and admission is ordered by ascending mean GSS response. Figures 3 and 8 therefore show performance under a pre-established representation and a particular quality-ordered schedule. The curves are not uniformly monotonic, and Figure 8 shows both Qwen OOD groups starting below LoraRetriever at Hub8. These observations qualify subgroup performance without contradicting Section 4.5's pooled All32 Step SR claim. Table 9 also shows positive gains for targets whose own adapters are present throughout, supporting the paper's narrower claim that expansion gains are not solely due to own-adapter arrival. The evaluation still uses synthetic clients, a common training recipe, and teacher forcing; its intervals do not cover training variability or closed-loop recovery.
Questions for authors
-
Please specify Own-LoRA's scale and execution path and how gamma was selected. Can Own-LoRA be evaluated in the same execution path as Table 14 at gamma = 1.5, together with a GUIHub gamma = 1.0 control, stating which routes and weights are held fixed? Can paired IID/OOD intervals be reported for the existing one-versus-two-adapter comparison? These would clarify the large Own-LoRA gap and the incremental benefit of composition without requiring a full new factorial study.
-
Can you report within-target score dispersion and selected-weight entropy for the two scorers, and test either equal-weight composition on their existing selected sets or a comparable calibration procedure using development data? The goal is to distinguish the ranking advantage already supported by Table 15 from any additional effect of score scale on composition weights. A more aggregation-aligned semantic control would be helpful if its adapter–probe scoring rule is specified precisely.
The following would strengthen secondary comparisons and deployment interpretation.
-
Section 2 identifies LoRAuter as combining validation performance and task embeddings for per-query adapter routing. What prevents a comparison under compatible information and execution budgets, or can such a comparison be supplied? Explaining adaptation barriers would clarify the scope; an empirical comparison would provide additional performance evidence.
-
Can the excluded initial profiling and new-adapter admission costs be measured or estimated with a clearly stated method, and amortized over explicit numbers of targets and requests? Please distinguish one-time construction from per-adapter admission and the target setup already reported in Table 11.
-
Can LoGo's mixed workloads be replayed with explicit routing-state initialization and episode order, or can a smaller representative replay establish whether prediction reuse materially changes the comparison?
Suggestions For Future Work
- Evaluate closed-loop execution to measure error accumulation and recovery with a fixed target composition.
- Test real user partitions and adapters with heterogeneous quality, training budgets, and compatible ranks.
- Study alternative arrival orders and a representation initialized from only the initial pool, together with admission or eviction rules addressing the individual regressions in Table 9.
7 comments
Response to Reviewer 3Aum (2/5)
(W1 & Q1, continued) "Within the anchor-free group at the same scale, moving from one to two adapters changes IID/OOD/mixed Step SR by approximately +0.62/−0.24/+0.35 pp. … These comparisons do not isolate the own-anchor/helper combination used in the main IID and mixed experiments, and paired intervals for the adapter-count differences are not reported."
- Paired intervals for one versus two adapters. Appendix D.3 now reports them. Under anchor-free selection at the same scale, two adapters minus one gives +0.62 pp [−0.19, +1.46] on IID, −0.24 pp [−1.65, +1.13] on OOD, and +0.35 pp [−0.66, +1.29] on mixed targets. We agree that one behaviorally selected adapter already attains most of the accuracy, and we do not claim a stable additional gain from simultaneous composition.
- The own-anchor/helper combination. GUIHub's in-hub composition is the own adapter plus one helper, so the controlled comparison in our previous comment isolates exactly this combination: adding the helper raises IID Step SR at every scale, by up to +3.21 pp at γ = 2 ([+2.11, +4.32]) and by +2.07 pp at γ = 1 ([+1.17, +2.99]).
- Where the benefit comes from. As you note, cross-client selection is itself a useful form of adapter sharing, and our evidence indicates that this is where GUIHub's benefit comes from: without training an adapter, a held-out user obtains Step SR 0.95–5.80 pp above that of a supervised own adapter across scales, and a cross-client user obtains 1.68–4.84 pp above a single own adapter.
(W2) "Equation (11) uniformly averages semantic similarities over the LSS and an adapter source client's probes. GUIHub instead aggregates measured behavioral responses using target-dependent relevance over the shared GSS. … The 1.02/5.18/3.86 pp IID/OOD/mixed differences therefore compare two complete scoring procedures, including their aggregation and score distributions."
- Aggregation-aligned control. To isolate the evidence, we added the control you suggested, with its adapter–probe rule stated exactly (Appendix D.4, Table 16). It keeps GUIHub's relevance vector and the same aggregation over the shared GSS, and replaces each measured response with an indicator of whether probe comes from adapter 's source client:
Identical aggregation. The behavioral side is the Direct score , i.e., without the low-rank projection. The two scorers then differ only in whether an adapter is credited for its measured behavior or for the provenance of the probes. On the 768 singleton evaluations:
Targets Evidence Spearman with singleton Step SR Selected-set regret (pp) All 32 Measured behavior 0.532 [0.453, 0.607] 1.10 [0.64, 1.61] All 32 Probe provenance 0.150 [0.029, 0.273] 2.98 [1.73, 4.38] OOD 8 Measured behavior 0.554 [0.388, 0.682] 1.34 [0.46, 2.36] OOD 8 Probe provenance 0.008 [−0.178, 0.239] 6.08 [2.85, 9.38] Result. Across all 32 targets, the differences are −0.382 [−0.496, −0.270] in Spearman correlation and +1.88 pp [0.72, 3.24] in regret, and the selected sets differ for 30 of 32 targets. (For reference, the projected behavioral score in Table 15 has Spearman 0.513.) For the eight held-out targets, probe provenance is essentially uncorrelated with actual accuracy (Spearman 0.008), whereas measured behavior keeps 0.554 and reduces regret from 6.08 to 1.34 pp. With the aggregation held fixed, measured behavior is what drives selection quality.
Response to Reviewer 3Aum (3/5)
(W2 & Q2) "Can you report within-target score dispersion and selected-weight entropy for the two scorers, and test either equal-weight composition on their existing selected sets or a comparable calibration procedure using development data?"
We report the statistics you requested and ran the equal-weight check (Appendix D.4, Table 17).
Dispersion and entropy. Within-target score dispersion is similar for the behavioral and semantic scorers (standard deviation 0.033 vs. 0.039; IQR 0.046 vs. 0.042). Both give near-uniform weights over the selected pair (normalized entropy 0.95 vs. 0.90; mean maximum weight 0.61 vs. 0.65); the semantic weights are slightly more concentrated.
Equal-weight composition. Each scorer keeps its selected pair, and the final weights are fixed to 0.5/0.5 (Step SR, %):
Selected set Weights IID OOD Behavioral Softmax (GUIHub) 64.98 67.00 Behavioral Equal 64.85 67.10 Semantic Softmax 63.96 61.82 Semantic Equal 63.32 62.10 Result. With equal weights, the behavioral selected sets remain ahead by +1.53 pp [+0.68, +2.38] on IID and +4.99 pp [+3.33, +6.66] on OOD. Softmax weighting itself changes behavioral Step SR by +0.13 pp [−0.39, +0.65] and −0.10 pp [−0.71, +0.51]. For the semantic scorer it gives +0.65 pp [+0.10, +1.20] on IID and −0.29 pp [−1.06, +0.49] on OOD; the difference between the two scorers' weighting effects has intervals that include zero (−0.52 pp [−1.29, +0.24] on IID; +0.19 pp [−0.79, +1.14] on OOD).
Conclusion. The behavioral advantage remains when both scorers use the same weights, so it comes from selecting better adapters, not from score scale. Consistent with your reading of Table 15, we claim a ranking advantage and do not claim a stable additional benefit from softmax weighting.
(a) "These results support a compact representation with similar observed accuracy; they do not show an accuracy benefit from the routing projection."
We agree. Paired 95% intervals for Direct matching minus the projected score are +0.21 pp [−0.40, +0.81] on IID, +0.76 pp [−0.24, +1.78] on OOD, and +0.26 pp [−0.27, +0.84] on Mixed (All); all include zero (Appendix D.2).
- A compression choice. The projection is a compression choice rather than an accuracy component: it gives every adapter, including newly admitted ones, a fixed dimensional descriptor in a shared space (96:1 compression of the response profile) at similar observed accuracy.
Response to Reviewer 3Aum (1/5)
(W1 & Q1) "Table 12 reports 64.98 IID Step SR for GUIHub versus 46.90 for Own-LoRA, but Appendix D.1 does not explicitly specify Own-LoRA's additional scale or its correspondence to GUIHub's execution path. … Please specify Own-LoRA's scale and execution path and how gamma was selected. Can Own-LoRA be evaluated in the same execution path as Table 14 at gamma = 1.5, together with a GUIHub gamma = 1.0 control …?"