- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BrowSA targets the repeated processing of long DOMs, instructions, and interaction histories in browser agents. It combines structural prompt chunking, length-calibrated ranking, and KV-cache reordering. For ranking, it separates the extrema envelope into a signed center and a nonnegative width, calibrating only the width. For placement, it groups chunks by their worst-case calibrated score difference and reorders cached keys and values without changing logical positions. The full KV cache is retained, although decoding accesses only selected physical pages; valid neighboring tokens on those pages may also receive attention.
The experiments use Qwen3-32B on one NVIDIA B200, with task results on WebVoyager, the GAIA web subset, and Online-Mind2Web. The paper also reports decoding latency on recorded requests and ablates calibration and packing. Aggregate Pass@3 is competitive with or above Dense, and the measured decoding attention path, including scoring and selection, is up to 68% faster.
Critical strengths
S1. I find the width-only calibration particularly well motivated: it addresses length-dependent score inflation without discarding the signed center. Structural chunking and KV reordering address separate problems, namely fragmented boundaries and inefficient physical page access. The derivations in Appendices A.1 and A.5 help explain the method within their stated scope. The paper is also careful about which claims follow from score-difference bounds and which page-access benefits are empirical.
S2. The evaluation measures both completed browser tasks and system cost. BrowSA reports Pass@3 of 49.1–50.7% on WebVoyager versus 46.2% for Dense, 44.4–45.6% versus 37.8% on GAIA, and 40.3–40.7% versus 38.0% on Online-Mind2Web. At matched attended-token counts, Table 2 reports TPOT of 18.261 ms for BrowSA, 18.886 ms for Quest, and 33.019 ms for Dense. Table 3 reports about 10–12% fewer fetched pages with packing. I value the matched-token comparison because sparse attention alone does not guarantee cheaper memory access.
Critical weaknesses
W1. I cannot tell how much of the task-quality gain comes from structural boundaries. Every sparse variant in Table 4 retains them, while BrowSA differs from Quest in selection granularity, query-head pooling, and physical layout. LServe already separates fine-grained logical scoring from larger physical pages. A comparison with nonsemantic boundaries matched for chunk length, while holding scoring, pooling, packing, and protection fixed, would isolate the structural contribution more clearly.
I also have trouble connecting the calibration diagnostic to the serving results. Figure 5 uses Qwen3-VL-30B-A3B with A=1 and alpha=0.397; the serving experiments use Qwen3-32B with A=32 and alpha=0.25. The paper discloses this difference, but a smaller residual in task-averaged scores does not by itself show better ranking or attention preservation in the deployed setup. Finally, the main Pass@3 results lack paired task outcomes or uncertainty intervals. On 90 GAIA tasks, the 4.4- and 2.3-point packing gains amount to roughly four and two additional solved tasks, respectively. I cannot judge their stability from the aggregate rates alone.
W2. The latency results leave open what an interactive user would experience. Figure 7 uses batches of 15, 28, or 32 requests and a fixed 64-token generation length; prefix caching is disabled. Appendix A.4 reports about 29 ms of packing overhead per request and a break-even length of roughly 40–63 output tokens, measured at page size 64 and batch size 32. That threshold sits close to the 64-token replay length, and the paper does not report natural output lengths. The estimate also covers packing’s incremental cost, not BrowSA’s full per-call cost.
Access budgets are difficult to compare across experiments as well. Appendix A.4 applies the selection ratio before mandatory content is added, whereas the latency comparison counts protected pages within a page cap and may let mandatory entries exceed it. Equal nominal selection ratios may therefore mean different memory traffic. Table 2’s matched-token comparison is helpful, but does not settle this issue for the task-quality results. A batch-one replay using natural output lengths, with prefill, layout construction, KV copying, metadata installation, decoding, and fetched-page counts included, would make the practical latency claim easier to assess.
I understand the reported maximum 68% improvement to concern the decoding attention path, not complete task latency. My concern is whether the additional components pay off at the level of an interactive call.
Questions for authors
-
If page size, query pooling, scoring, protection, and packing are fixed, what happens to task success when structural boundaries are replaced by nonsemantic boundaries with a matched chunk-length distribution? Please clarify the admission rules in Table 1, Figure 6, and Table 4, and report fetched-page counts for this comparison.
-
At batch size one and with natural output lengths from the existing trajectories, how do full BrowSA, BrowSA without packing, and Quest compare on time to first token and total call latency? Does the 29 ms packing figure include layout construction, KV copying, and metadata installation? What proportion of requests exceeds the reported break-even length?
-
Can the three existing runs support single-run success rates and task-level paired intervals or tests? Please specify how success is evaluated, the randomization settings, and how website failures are handled. I am especially interested in whether the GAIA packing gains recur across runs.
-
How were the serving values A=32 and alpha=0.25 chosen, and were tuning tasks separate from evaluation tasks? A sensitivity check or ranking diagnostic using the serving model and semantic chunks would help connect Figure 5 to the main results.
8 comments
Author Rebuttal to the Reviewer zSwn (2/7)
Table 2. GAIA: pass rate (%) at two selection ratios, page size 32.
| Selection ratio | Dense | Quest | Block Sparse | LServe | BrowSA |
|---|---|---|---|---|---|
| 6.25% | 37.8 | 32.2 | 32.2 | 33.3 | 44.4 |
| 12.5% | 37.8 | 38.9 | 37.8 | 37.8 | 45.6 |
Table 3. Online-Mind2Web: pass rate (%) at two selection ratios, page size 32.
| Selection ratio | Dense | Quest | Block Sparse | LServe | BrowSA |
|---|---|---|---|---|---|
| 6.25% | 38.0 | 31.3 | 33.7 | 33.0 | 40.3 |
| 12.5% | 38.0 | 37.7 | 34.0 | 35.0 | 40.7 |
Table 4. WebVoyager: pass rate (%) at a 12.5% selection ratio and two page sizes.
| Page size | Dense | Quest | Block Sparse | LServe | BrowSA |
|---|---|---|---|---|---|
| 32 | 46.2 | 45.9 | 44.9 | 41.2 | 49.1 |
| 64 | 46.2 | 40.7 | 45.7 | 41.4 | 50.7 |
W1 (continued). I also have trouble connecting the calibration diagnostic to the serving results. Figure 5 uses Qwen3-VL-30B-A3B with A=1 and alpha=0.397; the serving experiments use Qwen3-32B with A=32 and alpha=0.25. The paper discloses this difference, but a smaller residual in task-averaged scores does not by itself show better ranking or attention preservation in the deployed setup.
Q4. How were the serving values A=32 and alpha=0.25 chosen, and were tuning tasks separate from evaluation tasks? A sensitivity check or ranking diagnostic using the serving model and semantic chunks would help connect Figure 5 to the main results.
Thank you for the comment. Figure 5 is an illustrative example of the calibration on a different model. The serving values A and α were selected on a held-out calibration set with the serving model (Qwen3-32B), and then verified with an end-to-end sweep on GAIA.
Selection on a held-out calibration set (Table 5). The held-out calibration set consists of prompts recorded from successful Dense-attention trajectories on tasks disjoint from the evaluation tasks. We replay each prompt through BrowSA with every (A, α) and measure how often BrowSA reproduces the action and target element chosen under Dense attention. This directly tests whether the calibrated score preserves the decisions of the deployed model under sparse attention, rather than a residual on task-averaged scores. Two patterns hold at both page sizes. First, α = 0.25 is clearly the best exponent: agreement is 54.5–57.5% at page size 32, against at most 51.3% for α = 0.125 or 0.5, and α = 0.5 collapses as A grows. Second, within α = 0.25 the result is robust to A: agreement varies by only 3 points from A = 1 to A = 32 and drops only at A ≥ 64.
End-to-end verification (Table 6). To confirm that this choice carries over to task success, we ran the full GAIA benchmark for every (A, α) at page size 64. The pass rates follow the same pattern: a plateau at α = 0.25 with A between 1 and 32 (mean Pass@1 16.7–21.5%, Pass@3 34.4–37.8%), rapid degradation at α = 0.5 as A grows (mean Pass@1 4.1% at A = 32), and lower success at A ≥ 64 for almost every α. The serving setting is the best or tied-best in agreement and in Pass@3.
Table 5. Offline agreement rate (%) between BrowSA and Dense attention on the action and target element, for each (A, α) at page size 32 and 64.
| A | page 32, α=0.125 | page 32, α=0.25 | page 32, α=0.5 | page 64, α=0.125 | page 64, α=0.25 | page 64, α=0.5 |
|---|---|---|---|---|---|---|
| 1 | 51.3 | 56.4 | 51.3 | 28.1 | 33.1 | 28.1 |
| 2 | 51.3 | 56.4 | 51.3 | 28.1 | 33.1 | 28.1 |
| 4 | 51.3 | 56.4 | 51.3 | 26.7 | 33.1 | 26.2 |
| 8 | 49.5 | 54.5 | 49.5 | 26.3 | 31.4 | 20.5 |
| 16 | 49.0 | 57.5 | 38.6 | 24.2 | 34.2 | 9.0 |
| 32 | 43.4 | 57.5 | 31.4 | 24.2 | 34.2 | 0.1 |
| 64 | 42.9 | 39.9 | 22.4 | 22.5 | 25.8 | 0.0 |
| 128 | 35.2 | 24.8 | 9.4 | 22.9 | 20.0 | 0.0 |
Author Rebuttal to the Reviewer zSwn (3/7)
Table 6. End-to-end sensitivity of A and α on GAIA at page size 64: individual-run success rate, mean Pass@1, and Pass@3 (%). The serving setting (A = 32, α = 0.25) is in bold.
| A | α | Run 1 | Run 2 | Run 3 | Mean Pass@1 | Pass@3 |
|---|---|---|---|---|---|---|
| 1 | 0.125 | 17.8 | 18.9 | 16.7 | 17.8 | 34.4 |
| 1 | 0.25 | 17.8 | 23.3 | 13.3 | 18.1 | 34.4 |
| 1 | 0.5 | 12.2 | 16.7 | 13.3 | 14.1 | 28.9 |
| 4 | 0.125 | 20.0 | 20.0 | 16.7 | 18.9 | 34.4 |
| 4 | 0.25 | 23.3 | 18.9 | 10.0 | 17.4 | 35.6 |
| 4 | 0.5 | 15.6 | 15.6 | 13.3 | 14.8 | 26.7 |
| 8 | 0.125 | 8.9 | 18.9 | 21.1 | 16.3 | 31.1 |
| 8 | 0.25 | 21.1 | 23.3 | 20.0 | 21.5 | 37.8 |
| 8 | 0.5 | 14.4 | 15.6 | 11.1 | 13.7 | 28.9 |
| 16 | 0.125 | 16.7 | 13.3 | 21.1 | 17.0 | 34.4 |
| 16 | 0.25 | 10.0 | 18.9 | 21.1 | 16.7 | 35.6 |
| 16 | 0.5 | 6.7 | 6.7 | 5.6 | 6.3 | 16.7 |
| 32 | 0.125 | 20.0 | 20.0 | 14.4 | 18.1 | 33.3 |
| 32 | 0.25 | 15.6 | 16.7 | 24.4 | 18.9 | 37.8 |
| 32 | 0.5 | 4.4 | 5.6 | 2.2 | 4.1 | 10.0 |
| 64 | 0.125 | 22.2 | 14.4 | 14.4 | 17.0 | 35.6 |
| 64 | 0.25 | 11.1 | 14.4 | 12.2 | 12.6 | 24.4 |
| 64 | 0.5 | 5.6 | 4.4 | 1.1 | 3.7 | 10.0 |
| 128 | 0.125 | 13.3 | 11.1 | 13.3 | 12.6 | 25.6 |
| 128 | 0.25 | 13.3 | 12.2 | 10.0 | 11.9 | 24.4 |
| 128 | 0.5 | 0.0 | 0.0 | 1.1 | 0.4 | 1.1 |
W1 (continued). Finally, the main Pass@3 results lack paired task outcomes or uncertainty intervals. On 90 GAIA tasks, the 4.4- and 2.3-point packing gains amount to roughly four and two additional solved tasks, respectively. I cannot judge their stability from the aggregate rates alone.
Q3. Can the three existing runs support single-run success rates and task-level paired intervals or tests? I am especially interested in whether the GAIA packing gains recur across runs.
Thank you for the comment. Tables 7 to 9 report the success rate of each individual run, the mean Pass@1 over the three runs, and Pass@3 for the Dense and BrowSA configurations in Table 1 and Figure 6 of the paper, and Tables 12 and 13 give the same breakdown for every row of the GAIA ablation (Table 1 above). We report Pass@3 as the primary metric because it measures the fraction of tasks the agent can solve within three attempts, which is the quantity that matters for browser agents: individual runs are subject to website nondeterminism, and deployed agents retry. Since Pass@3 does not average over runs, we also report mean Pass@1 as the per-attempt measure, with paired confidence intervals in Tables 10 and 11 below. The two metrics agree in direction: in all six configurations, a higher Pass@3 than Dense comes with a higher mean Pass@1.
Table 7. WebVoyager: individual-run success rate, mean Pass@1, and Pass@3 (%).
| Configuration | Run 1 | Run 2 | Run 3 | Mean Pass@1 | Pass@3 |
|---|---|---|---|---|---|
| Dense | 29.1 | 28.3 | 26.6 | 28.0 | 46.2 |
| BrowSA, page size 32, selection ratio 12.5% | 29.4 | 29.5 | 29.2 | 29.4 | 49.1 |
| BrowSA, page size 64, selection ratio 12.5% | 30.0 | 29.1 | 29.9 | 29.7 | 50.7 |
Table 8. GAIA: individual-run success rate, mean Pass@1, and Pass@3 (%).
| Configuration | Run 1 | Run 2 | Run 3 | Mean Pass@1 | Pass@3 |
|---|---|---|---|---|---|
| Dense | 15.6 | 21.1 | 18.9 | 18.5 | 37.8 |
| BrowSA, selection ratio 6.25% | 18.9 | 21.1 | 23.3 | 21.1 | 44.4 |
| BrowSA, selection ratio 12.5% | 31.1 | 22.2 | 24.4 | 25.9 | 45.6 |
Table 9. Online-Mind2Web: individual-run success rate, mean Pass@1, and Pass@3 (%).
| Configuration | Run 1 | Run 2 | Run 3 | Mean Pass@1 | Pass@3 |
|---|---|---|---|---|---|
| Dense | 24.0 | 21.0 | 22.3 | 22.4 | 38.0 |
| BrowSA, selection ratio 6.25% | 23.3 | 22.0 | 22.3 | 22.6 | 40.3 |
| BrowSA, selection ratio 12.5% | 22.3 | 22.3 | 24.7 | 23.1 | 40.7 |
Author Rebuttal to the Reviewer zSwn (1/7)
Thank you for taking the time to review our work and for providing thoughtful and detailed feedback. We have addressed your concerns and questions below.
Thank you for the comment. The admission rule is the same in Table 1, Figure 6, and Table 4 of the paper: chunks are ranked by score and admitted in that order, and the pages covering the admitted chunks are counted until the number of distinct pages reaches the selection ratio times the total number of pages in the request, so the fetched-page count is the selection ratio times the total page count.
To isolate the structural contribution, we augmented the ablation study to include semantic chunks alongside the BrowSA score and packing, introducing one component at a time (Table 1 below). Page size, query-head pooling, scoring, and protection are held fixed across the chunking variants, and packing is disabled in all of them. We compare semantic chunks with positional chunks at two lengths: 32 tokens, which match the page size, and 16 tokens, which match the upper bound of the semantic chunk length and serve as the granularity-matched control. Semantic chunks outperform positional chunks of either length at both selection ratios: 40.0% versus 37.8% (16-token) and 34.4% (32-token) at a 6.25% selection ratio, and 43.3% versus 40.0% (16-token) and 41.1% (32-token) at a 12.5% selection ratio. Granularity alone does not explain the gain: halving the positional chunk length helps at the 6.25% ratio but not at the 12.5% ratio, whereas semantic chunks improve at both. Adding packing on top gives the best pass rate at both ratios, 44.4% at 6.25% and 45.6% at 12.5%. We have added this ablation to Section 4.3 and Table 4 of the revised paper, with per-run success rates in Appendix A.8; the changes are marked in red.
Table 1. Ablation of semantic chunks, the BrowSA score, and KV packing on GAIA. Pass rate (%) in at least one of three runs at two selection ratios. All sparse rows use page size 32.
We also note that, although LServe introduces finer-grained logical block scoring, its layout has two limitations. First, it partitions tokens into blocks according to their sequence positions, which can split semantically coherent content across block boundaries. Second, it scores smaller blocks within each KV page but does not rearrange them. Consequently, selecting a page for a relevant block also brings less relevant blocks on that page into the attention computation. BrowSA addresses both limitations by forming semantically coherent chunks and packing chunks that are likely to be accessed together into the same or nearby KV pages. This improves the utilization of selected pages by reducing the amount of less relevant content included in attention computation. We also compare pass rates against LServe under the same selection ratio (Tables 2 to 4). BrowSA outperforms LServe in every setting on all three benchmarks: by 7.8 to 11.1 points on GAIA, 5.7 to 7.3 points on Online-Mind2Web, and 7.9 to 9.3 points on WebVoyager. On WebVoyager, LServe also falls below Dense and Block Sparse at both page sizes, which is consistent with the two limitations above.