acceptodds
Under review as a conference paper at ICLR 2027

BrowSA: Semantic Aware Sparse Attention for Web Agents with KV cache packing

Abstract

Browser agents re-read tens of thousands of tokens of serialized DOM at every step, yet only a small fraction of that context influences the next action. Sparse attention can skip the rest, but existing methods drag in irrelevant neighbors and/or put substantial pressure on the hardware. We observe that web pages, system prompts, and action histories already carry semantic boundaries through HTML and other structured formats. BrowSA exploits these boundaries twice. It selects among chunks defined by the prompt's semantic boundaries and ranks them with a score calibrated for unequal chunk lengths. A placement stage then packs these chunks into physical KV pages, so the units the scorer selects are the units the kernel fetches. BrowSA reduces decode attention latency by up to 68% over dense attention while matching or exceeding its task success on WebVoyager, GAIA, and Online-Mind2Web.

Primary area:
foundation or frontier models, including LLMs
Keywords:
web agent, sparse attention, LLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zSwnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BrowSA targets the repeated processing of long DOMs, instructions, and interaction histories in browser agents. It combines structural prompt chunking, length-calibrated ranking, and KV-cache reordering. For ranking, it separates the extrema envelope into a signed center and a nonnegative width, calibrating only the width. For placement, it groups chunks by their worst-case calibrated score difference and reorders cached keys and values without changing logical positions. The full KV cache is retained, although decoding accesses only selected physical pages; valid neighboring tokens on those pages may also receive attention.

The experiments use Qwen3-32B on one NVIDIA B200, with task results on WebVoyager, the GAIA web subset, and Online-Mind2Web. The paper also reports decoding latency on recorded requests and ablates calibration and packing. Aggregate Pass@3 is competitive with or above Dense, and the measured decoding attention path, including scoring and selection, is up to 68% faster.

Critical strengths

S1. I find the width-only calibration particularly well motivated: it addresses length-dependent score inflation without discarding the signed center. Structural chunking and KV reordering address separate problems, namely fragmented boundaries and inefficient physical page access. The derivations in Appendices A.1 and A.5 help explain the method within their stated scope. The paper is also careful about which claims follow from score-difference bounds and which page-access benefits are empirical.

S2. The evaluation measures both completed browser tasks and system cost. BrowSA reports Pass@3 of 49.1–50.7% on WebVoyager versus 46.2% for Dense, 44.4–45.6% versus 37.8% on GAIA, and 40.3–40.7% versus 38.0% on Online-Mind2Web. At matched attended-token counts, Table 2 reports TPOT of 18.261 ms for BrowSA, 18.886 ms for Quest, and 33.019 ms for Dense. Table 3 reports about 10–12% fewer fetched pages with packing. I value the matched-token comparison because sparse attention alone does not guarantee cheaper memory access.

Critical weaknesses

W1. I cannot tell how much of the task-quality gain comes from structural boundaries. Every sparse variant in Table 4 retains them, while BrowSA differs from Quest in selection granularity, query-head pooling, and physical layout. LServe already separates fine-grained logical scoring from larger physical pages. A comparison with nonsemantic boundaries matched for chunk length, while holding scoring, pooling, packing, and protection fixed, would isolate the structural contribution more clearly.

I also have trouble connecting the calibration diagnostic to the serving results. Figure 5 uses Qwen3-VL-30B-A3B with A=1 and alpha=0.397; the serving experiments use Qwen3-32B with A=32 and alpha=0.25. The paper discloses this difference, but a smaller residual in task-averaged scores does not by itself show better ranking or attention preservation in the deployed setup. Finally, the main Pass@3 results lack paired task outcomes or uncertainty intervals. On 90 GAIA tasks, the 4.4- and 2.3-point packing gains amount to roughly four and two additional solved tasks, respectively. I cannot judge their stability from the aggregate rates alone.

W2. The latency results leave open what an interactive user would experience. Figure 7 uses batches of 15, 28, or 32 requests and a fixed 64-token generation length; prefix caching is disabled. Appendix A.4 reports about 29 ms of packing overhead per request and a break-even length of roughly 40–63 output tokens, measured at page size 64 and batch size 32. That threshold sits close to the 64-token replay length, and the paper does not report natural output lengths. The estimate also covers packing’s incremental cost, not BrowSA’s full per-call cost.

Access budgets are difficult to compare across experiments as well. Appendix A.4 applies the selection ratio before mandatory content is added, whereas the latency comparison counts protected pages within a page cap and may let mandatory entries exceed it. Equal nominal selection ratios may therefore mean different memory traffic. Table 2’s matched-token comparison is helpful, but does not settle this issue for the task-quality results. A batch-one replay using natural output lengths, with prefill, layout construction, KV copying, metadata installation, decoding, and fetched-page counts included, would make the practical latency claim easier to assess.

I understand the reported maximum 68% improvement to concern the decoding attention path, not complete task latency. My concern is whether the additional components pay off at the level of an interactive call.

Questions for authors
  1. If page size, query pooling, scoring, protection, and packing are fixed, what happens to task success when structural boundaries are replaced by nonsemantic boundaries with a matched chunk-length distribution? Please clarify the admission rules in Table 1, Figure 6, and Table 4, and report fetched-page counts for this comparison.

  2. At batch size one and with natural output lengths from the existing trajectories, how do full BrowSA, BrowSA without packing, and Quest compare on time to first token and total call latency? Does the 29 ms packing figure include layout construction, KV copying, and metadata installation? What proportion of requests exceeds the reported break-even length?

  3. Can the three existing runs support single-run success rates and task-level paired intervals or tests? Please specify how success is evaluated, the randomization settings, and how website failures are handled. I am especially interested in whether the GAIA packing gains recur across runs.

  4. How were the serving values A=32 and alpha=0.25 chosen, and were tuning tasks separate from evaluation tasks? A sensitivity check or ranking diagnostic using the serving model and semantic chunks would help connect Figure 5 to the main results.

8 comments
AuthorsOpenReview ↗

Author Rebuttal to the Reviewer zSwn (1/7)

Thank you for taking the time to review our work and for providing thoughtful and detailed feedback. We have addressed your concerns and questions below.


W1. I cannot tell how much of the task-quality gain comes from structural boundaries. Every sparse variant in Table 4 retains them, while BrowSA differs from Quest in selection granularity, query-head pooling, and physical layout. A comparison with nonsemantic boundaries matched for chunk length, while holding scoring, pooling, packing, and protection fixed, would isolate the structural contribution more clearly.

Q1. If page size, query pooling, scoring, protection, and packing are fixed, what happens to task success when structural boundaries are replaced by nonsemantic boundaries with a matched chunk-length distribution? Please clarify the admission rules in Table 1, Figure 6, and Table 4, and report fetched-page counts for this comparison.

Thank you for the comment. The admission rule is the same in Table 1, Figure 6, and Table 4 of the paper: chunks are ranked by score and admitted in that order, and the pages covering the admitted chunks are counted until the number of distinct pages reaches the selection ratio times the total number of pages in the request, so the fetched-page count is the selection ratio times the total page count.

To isolate the structural contribution, we augmented the ablation study to include semantic chunks alongside the BrowSA score and packing, introducing one component at a time (Table 1 below). Page size, query-head pooling, scoring, and protection are held fixed across the chunking variants, and packing is disabled in all of them. We compare semantic chunks with positional chunks at two lengths: 32 tokens, which match the page size, and 16 tokens, which match the upper bound of the semantic chunk length and serve as the granularity-matched control. Semantic chunks outperform positional chunks of either length at both selection ratios: 40.0% versus 37.8% (16-token) and 34.4% (32-token) at a 6.25% selection ratio, and 43.3% versus 40.0% (16-token) and 41.1% (32-token) at a 12.5% selection ratio. Granularity alone does not explain the gain: halving the positional chunk length helps at the 6.25% ratio but not at the 12.5% ratio, whereas semantic chunks improve at both. Adding packing on top gives the best pass rate at both ratios, 44.4% at 6.25% and 45.6% at 12.5%. We have added this ablation to Section 4.3 and Table 4 of the revised paper, with per-run success rates in Appendix A.8; the changes are marked in red.

Table 1. Ablation of semantic chunks, the BrowSA score, and KV packing on GAIA. Pass rate (%) in at least one of three runs at two selection ratios. All sparse rows use page size 32.

Configuration6.25% selection ratio12.5% selection ratio
Semantic chunks (16 tokens upper bound), BrowSA score, packing, page size 3244.445.6
Semantic chunks (16 tokens upper bound), BrowSA score, no packing, page size 3240.043.3
Positional chunks (16 tokens), BrowSA score, no packing, page size 3237.840.0
Positional chunks (32 tokens), BrowSA score, no packing, page size 3234.441.1
Positional chunks (32 tokens), Quest score, no packing, page size 3232.238.9
Dense (100% KV)37.837.8

W1 (continued). LServe already separates fine-grained logical scoring from larger physical pages.

We also note that, although LServe introduces finer-grained logical block scoring, its layout has two limitations. First, it partitions tokens into blocks according to their sequence positions, which can split semantically coherent content across block boundaries. Second, it scores smaller blocks within each KV page but does not rearrange them. Consequently, selecting a page for a relevant block also brings less relevant blocks on that page into the attention computation.  BrowSA addresses both limitations by forming semantically coherent chunks and packing chunks that are likely to be accessed together into the same or nearby KV pages. This improves the utilization of selected pages by reducing the amount of less relevant content included in attention computation. We also compare pass rates against LServe under the same selection ratio (Tables 2 to 4). BrowSA outperforms LServe in every setting on all three benchmarks: by 7.8 to 11.1 points on GAIA, 5.7 to 7.3 points on Online-Mind2Web, and 7.9 to 9.3 points on WebVoyager. On WebVoyager, LServe also falls below Dense and Block Sparse at both page sizes, which is consistent with the two limitations above.

AuthorsOpenReview ↗

Author Rebuttal to the Reviewer zSwn (2/7)

Table 2. GAIA: pass rate (%) at two selection ratios, page size 32.

Selection ratioDenseQuestBlock SparseLServeBrowSA
6.25%37.832.232.233.344.4
12.5%37.838.937.837.845.6

Table 3. Online-Mind2Web: pass rate (%) at two selection ratios, page size 32.

Selection ratioDenseQuestBlock SparseLServeBrowSA
6.25%38.031.333.733.040.3
12.5%38.037.734.035.040.7

Table 4. WebVoyager: pass rate (%) at a 12.5% selection ratio and two page sizes.

Page sizeDenseQuestBlock SparseLServeBrowSA
3246.245.944.941.249.1
6446.240.745.741.450.7

W1 (continued). I also have trouble connecting the calibration diagnostic to the serving results. Figure 5 uses Qwen3-VL-30B-A3B with A=1 and alpha=0.397; the serving experiments use Qwen3-32B with A=32 and alpha=0.25. The paper discloses this difference, but a smaller residual in task-averaged scores does not by itself show better ranking or attention preservation in the deployed setup.

Q4. How were the serving values A=32 and alpha=0.25 chosen, and were tuning tasks separate from evaluation tasks? A sensitivity check or ranking diagnostic using the serving model and semantic chunks would help connect Figure 5 to the main results.

Thank you for the comment. Figure 5 is an illustrative example of the calibration on a different model. The serving values A and α were selected on a held-out calibration set with the serving model (Qwen3-32B), and then verified with an end-to-end sweep on GAIA.

Selection on a held-out calibration set (Table 5). The held-out calibration set consists of prompts recorded from successful Dense-attention trajectories on tasks disjoint from the evaluation tasks. We replay each prompt through BrowSA with every (A, α) and measure how often BrowSA reproduces the action and target element chosen under Dense attention. This directly tests whether the calibrated score preserves the decisions of the deployed model under sparse attention, rather than a residual on task-averaged scores. Two patterns hold at both page sizes. First, α = 0.25 is clearly the best exponent: agreement is 54.5–57.5% at page size 32, against at most 51.3% for α = 0.125 or 0.5, and α = 0.5 collapses as A grows. Second, within α = 0.25 the result is robust to A: agreement varies by only 3 points from A = 1 to A = 32 and drops only at A ≥ 64.

End-to-end verification (Table 6). To confirm that this choice carries over to task success, we ran the full GAIA benchmark for every (A, α) at page size 64. The pass rates follow the same pattern: a plateau at α = 0.25 with A between 1 and 32 (mean Pass@1 16.7–21.5%, Pass@3 34.4–37.8%), rapid degradation at α = 0.5 as A grows (mean Pass@1 4.1% at A = 32), and lower success at A ≥ 64 for almost every α. The serving setting is the best or tied-best in agreement and in Pass@3.

Table 5. Offline agreement rate (%) between BrowSA and Dense attention on the action and target element, for each (A, α) at page size 32 and 64.

Apage 32, α=0.125page 32, α=0.25page 32, α=0.5page 64, α=0.125page 64, α=0.25page 64, α=0.5
151.356.451.328.133.128.1
251.356.451.328.133.128.1
451.356.451.326.733.126.2
849.554.549.526.331.420.5
1649.057.538.624.234.29.0
3243.457.531.424.234.20.1
6442.939.922.422.525.80.0
12835.224.89.422.920.00.0
AuthorsOpenReview ↗

Author Rebuttal to the Reviewer zSwn (3/7)

Table 6. End-to-end sensitivity of A and α on GAIA at page size 64: individual-run success rate, mean Pass@1, and Pass@3 (%). The serving setting (A = 32, α = 0.25) is in bold.

AαRun 1Run 2Run 3Mean Pass@1Pass@3
10.12517.818.916.717.834.4
10.2517.823.313.318.134.4
10.512.216.713.314.128.9
40.12520.020.016.718.934.4
40.2523.318.910.017.435.6
40.515.615.613.314.826.7
80.1258.918.921.116.331.1
80.2521.123.320.021.537.8
80.514.415.611.113.728.9
160.12516.713.321.117.034.4
160.2510.018.921.116.735.6
160.56.76.75.66.316.7
320.12520.020.014.418.133.3
320.2515.616.724.418.937.8
320.54.45.62.24.110.0
640.12522.214.414.417.035.6
640.2511.114.412.212.624.4
640.55.64.41.13.710.0
1280.12513.311.113.312.625.6
1280.2513.312.210.011.924.4
1280.50.00.01.10.41.1

W1 (continued). Finally, the main Pass@3 results lack paired task outcomes or uncertainty intervals. On 90 GAIA tasks, the 4.4- and 2.3-point packing gains amount to roughly four and two additional solved tasks, respectively. I cannot judge their stability from the aggregate rates alone.

Q3. Can the three existing runs support single-run success rates and task-level paired intervals or tests? I am especially interested in whether the GAIA packing gains recur across runs.

Thank you for the comment. Tables 7 to 9 report the success rate of each individual run, the mean Pass@1 over the three runs, and Pass@3 for the Dense and BrowSA configurations in Table 1 and Figure 6 of the paper, and Tables 12 and 13 give the same breakdown for every row of the GAIA ablation (Table 1 above). We report Pass@3 as the primary metric because it measures the fraction of tasks the agent can solve within three attempts, which is the quantity that matters for browser agents: individual runs are subject to website nondeterminism, and deployed agents retry. Since Pass@3 does not average over runs, we also report mean Pass@1 as the per-attempt measure, with paired confidence intervals in Tables 10 and 11 below. The two metrics agree in direction: in all six configurations, a higher Pass@3 than Dense comes with a higher mean Pass@1.

Table 7. WebVoyager: individual-run success rate, mean Pass@1, and Pass@3 (%).

ConfigurationRun 1Run 2Run 3Mean Pass@1Pass@3
Dense29.128.326.628.046.2
BrowSA, page size 32, selection ratio 12.5%29.429.529.229.449.1
BrowSA, page size 64, selection ratio 12.5%30.029.129.929.750.7

Table 8. GAIA: individual-run success rate, mean Pass@1, and Pass@3 (%).

ConfigurationRun 1Run 2Run 3Mean Pass@1Pass@3
Dense15.621.118.918.537.8
BrowSA, selection ratio 6.25%18.921.123.321.144.4
BrowSA, selection ratio 12.5%31.122.224.425.945.6

Table 9. Online-Mind2Web: individual-run success rate, mean Pass@1, and Pass@3 (%).

ConfigurationRun 1Run 2Run 3Mean Pass@1Pass@3
Dense24.021.022.322.438.0
BrowSA, selection ratio 6.25%23.322.022.322.640.3
BrowSA, selection ratio 12.5%22.322.324.723.140.7
Reviewer_4187OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a sparse attention mechanism, BrowSA, for browser agents. BrowSA uses the semantic structure already present in HTML/DOM element and subtree boundaries to define sparse attention chunks instead of cutting the context at fixed token intervals. It then ranks these variable-length chunks using a length-calibrated score and packs them into physical KV pages to improve memory access efficiency. BrowSA is evaluated using Qwen3-32B on WebVoyager, a web subset of GAIA, and Online-Mind2Web. The authors report matching or exceeding dense-attention task success, measured using Pass@3, while reducing decode attention latency, including scoring and selection.

Critical strengths

1- Promising results on realistic tasks: The evaluation includes three external web benchmarks and reports both task success (accuracy) and inference latency (efficiency). WebVoyager success increases from 46.2% for dense attention to 49.1-50.7% (for Page size 32 and 64 respectively). Table 4 suggests that calibration has significant effect: removing it from the unpacked version reduces task success on GAIA by 8.9 percentage points at both given selection ratios. Table 3 also measures the intended physical effect of packing, with approximately 10-12% fewer attended pages compared to the natural prompt order on WebVoyager. These results support the practical value of the proposed approach.

2- A well-motivated use of webpage structure: HTML and DOM elements already contain useful structure and the paper uses these boundaries to group related information together. The scoring and packing components address practical difficulties that occur when these groups have different lengths and do not align with memory pages. These components fit together naturally and provide a decent motivation for the approach.

Critical weaknesses

1- The specific benefit of semantic chunking is not sufficiently clear: There are several ways that BrowSA is different than the baselines, such as using smaller chunks, changing how chunks are scored, and rearranging the KV cache. The ablations examine scoring and packing, but they do not directly compare semantic chunks with similarly sized fixed chunks under otherwise comparable conditions. Table 2 matches attended-token counts for the latency comparison, however, equivalent information is missing from the downstream task-success comparisons. This makes it difficult to determine how much of the improvement comes from following semantic boundaries.

2- The practical benefit of packing needs clearer evaluation: Packing introduces an additional cost at the beginning, while its reported improvement in overall decoding time is relatively modest. The explanation of when this cost is recovered is also difficult to follow, especially because setup cost is stated per request while decoding is measured in batches. The decoding latency experiments (Figure 7) generate exactly 64 tokens, which is close to the paper's claimed break-even point of 40-63 tokens. Reporting total request time, including packing, for output lengths representative of actual agent runs would make the practical benefit easier to assess.

Questions for authors

1- How much of the improvement comes specifically from following semantic boundaries? Does the advantage remain when comparing against similarly sized fixed chunks, with the same scoring and packing?

2- Could you clarify how packing affects total request time once its setup cost is included? Appendix A.4 mentions a cost per request, while decoding is measured in batches. I found the calculation of the 40-63 token 'turning' point difficult to follow. It would also help to see whether packing remains beneficial for the output lengths observed in actual downstream task runs.

3- Could you provide the individual-run success rates, or Pass@1 results, alongside Pass@3? To gather more information for assessing whether the improvements over dense attention are consistent.

Post-rebuttal update. Rating raised from 2 to 3; see my response to the authors.

6 comments
AuthorsOpenReview ↗

Author Rebuttal to the Reviewer 4187 (1/4)

Thank you for taking the time to review our work and for providing thoughtful and detailed feedback. We have addressed your concerns and questions below.


W1. The specific benefit of semantic chunking is not sufficiently clear: There are several ways that BrowSA is different than the baselines, such as using smaller chunks, changing how chunks are scored, and rearranging the KV cache. The ablations examine scoring and packing, but they do not directly compare semantic chunks with similarly sized fixed chunks under otherwise comparable conditions. Table 2 matches attended-token counts for the latency comparison, however, equivalent information is missing from the downstream task-success comparisons. This makes it difficult to determine how much of the improvement comes from following semantic boundaries.

Q1. How much of the improvement comes specifically from following semantic boundaries? Does the advantage remain when comparing against similarly sized fixed chunks, with the same scoring and packing?

Thank you for pointing this out. We agree that BrowSA differs from the baselines along three axes, namely chunk boundaries, chunk scoring, and KV layout, and that the ablation in the submitted paper holds the chunk boundaries fixed and varies only the score and the KV layout. To isolate the contribution of semantic boundaries, we added two variants that replace semantic chunks with positional chunks, while keeping the BrowSA score and the unpacked KV layout unchanged (Table 1 below): 32-token positional chunks, which match the page size of BrowSA, and 16-token positional chunks, which match the upper bound of the semantic chunk length and control for granularity. Semantic chunks outperform positional chunks of either length at both selection ratios: 40.0% versus 37.8% (16-token) and 34.4% (32-token) at a 6.25% selection ratio, and 43.3% versus 40.0% (16-token) and 41.1% (32-token) at a 12.5% selection ratio. Granularity alone does not explain the gain: halving the positional chunk length helps at the 6.25% ratio (34.4% to 37.8%) but not at the 12.5% ratio (41.1% to 40.0%), whereas semantic chunks improve at both. Adding packing on top of semantic chunks and the BrowSA score gives the best pass rate at both ratios, 44.4% at 6.25% and 45.6% at 12.5%. We have added this ablation to Section 4.3 and Table 4 of the revised paper, with per-run success rates in Appendix A.8; the changes are marked in red.

Table 1. Ablation of semantic chunks, the BrowSA score, and KV packing on GAIA.

Configuration6.25% selection ratio12.5% selection ratio
Semantic chunks (16 tokens upper bound), BrowSA score, packing, page size 3244.445.6
Semantic chunks (16 tokens upper bound), BrowSA score, no packing, page size 3240.043.3
Positional chunks (16 tokens), BrowSA score, no packing, page size 3237.840.0
Positional chunks (32 tokens), BrowSA score, no packing, page size 3234.441.1
Positional chunks (32 tokens), Quest score, no packing, page size 3232.238.9
Dense (100% KV)37.837.8
AuthorsOpenReview ↗

Author Rebuttal to the Reviewer 4187 (2/4)

W2. The practical benefit of packing needs clearer evaluation: Packing introduces an additional cost at the beginning, while its reported improvement in overall decoding time is relatively modest. The decoding latency experiments (Figure 7) generate exactly 64 tokens, which is close to the paper's claimed break-even point of 40-63 tokens. Reporting total request time, including packing, for output lengths representative of actual agent runs would make the practical benefit easier to assess.

Q2. Could you clarify how packing affects total request time once its setup cost is included? It would also help to see whether packing remains beneficial for the output lengths observed in actual downstream task runs.

Thank you for the careful reading. We answer the two parts of W2 and Q2 in turn. The 64-token generation length in Figure 7 is a sampling point for measuring TPOT, not the length of an agent step. Table 2 below gives the output lengths of Qwen3-32B under dense attention on the three benchmarks. Median output lengths are 273–301 tokens without reasoning and 741–798 tokens with reasoning, four to twenty times the 40–63-token break-even of Appendix A.4. That break-even compares packing with no packing when a fixed set of top-scoring tokens is selected (the top-token fraction in Section 4.2 of the revision), so that packing reduces the number of pages to fetch. In fact, every recorded output exceeds this break-even length: the shortest of the 28,720 non-reasoning outputs has 112 tokens and the shortest of the 26,367 reasoning outputs has 282 tokens, so 100% of requests are longer than 64 tokens. The distributions and measurement details are in Appendix A.7.

Table 2. Output length under Dense attention (Median / P90 / P99, in tokens). Each mode pools three runs over all tasks: 28,720 non-reasoning outputs and 26,367 reasoning outputs.

BenchmarkNon-reasoningReasoning
WebVoyager273 / 373 / 522754 / 1535 / 3515
GAIA-web301 / 403 / 553798 / 1543 / 3941
Online-Mind2Web277 / 375 / 548741 / 1359 / 3181

Tables 3 and 4 below replay full agent outputs at four representative input lengths (6k, 10k, 14k, and 25k tokens) and report TTFT, TPOT, and end-to-end latency (E2E) for Dense, BrowSA, BrowSA without packing, Quest, and Block Sparse in reasoning and non-reasoning mode. All sparse methods use the same 6.25% selection ratio. TTFT includes the full one-time cost of chunking, scoring metadata, and packing; E2E spans from the start of prefill to the last decoded token, so every overhead is included. BrowSA’s TTFT is 2–4% above Dense because of this one-time preparation, and 8–11% faster than Quest and Block Sparse in every setting.

Even with the setup cost included, BrowSA has a lower total request time than Dense, Quest, and Block Sparse in all 16 configurations: up to 52% faster than Dense, up to 11% faster than Quest, and up to 11% faster than Block Sparse. These tables are added as Appendix A.9 of the revised paper.

To isolate the effect of packing itself, the BrowSA w/o packing column in Tables 3 and 4 replays the same requests with packing disabled and everything else unchanged. We want to be precise about what packing does and does not provide. Under the page-capped selection ratio used here, both variants fetch the same number of pages by construction, so decode time is the same in this setting (TPOT differs by at most 0.36 ms in either direction across all 16 configurations); what differs is the content of the fetched pages. Packing adds 1.3–4.7% to TTFT, and end-to-end latency changes by −0.07 to +0.95 s (at most 3.6%). At a fixed page budget, packing is therefore close to latency-neutral; its benefit is that each fetched page carries more useful context. This shows up in two places. At a fixed set of selected tokens (Table 3 of the paper), packing places them on 10.1–12.1% fewer pages and cuts pure attention time by 9.3–14.1%. At a fixed page budget, the same pages hold more of the important chunks, which raises the pass rate from 40.0% to 44.4% at a 6.25% selection ratio and from 43.3% to 45.6% at 12.5% (Table 1 above). We have made this distinction explicit in Section 4.2 of the revised paper.

AuthorsOpenReview ↗

Author Rebuttal to the Reviewer 4187 (3/4)

Table 3. Reasoning mode: TTFT, TPOT, and end-to-end latency at natural output lengths

InputBMetricDenseBrowSABrowSA w/o packingQuestBlock Sparse
6k1TPOT (ms)19.3514.4314.2314.7015.00
TTFT (ms)353368358411408
E2E (s)13.6410.3610.2810.6110.78
6k32TPOT (ms)27.4216.4216.6618.1818.37
TTFT (ms)6,5656,7306,6157,5027,476
E2E (s)31.1624.1024.1126.7426.82
10k1TPOT (ms)22.9414.6714.5715.0215.33
TTFT (ms)715738705826826
E2E (s)19.8713.1513.0713.4913.77
10k16TPOT (ms)24.7016.1715.9717.3017.51
TTFT (ms)6,5226,6906,4607,5497,549
E2E (s)32.5625.9025.3428.3528.53
14k1TPOT (ms)26.7615.2214.8715.2815.58
TTFT (ms)1,1111,1461,1171,2781,276
E2E (s)37.6422.2021.7922.4322.81
14k16TPOT (ms)28.7016.3816.3917.5817.84
TTFT (ms)9,88610,16210,01611,38511,363
E2E (s)57.6241.6341.4845.4745.79
25k1TPOT (ms)36.3415.5415.6615.7516.14
TTFT (ms)2,7282,8152,7243,0673,069
E2E (s)35.4817.0717.1417.5417.87
25k8TPOT (ms)37.3316.6016.4517.4917.91
TTFT (ms)12,72913,11712,79714,35514,356
E2E (s)55.8438.1137.4340.9841.35

Table 4. Non-reasoning mode: TTFT, TPOT, and end-to-end latency at natural output lengths

InputBMetricDenseBrowSABrowSA w/o packingQuestBlock Sparse
6k1TPOT (ms)19.0013.8714.0514.6814.96
TTFT (ms)357369358408410
E2E (s)4.993.783.834.024.09
6k32TPOT (ms)26.8616.1316.4917.9518.18
TTFT (ms)6,5596,7116,6247,5227,497
E2E (s)18.8216.5616.5018.5318.54
10k1TPOT (ms)22.7014.3214.4714.8915.11
TTFT (ms)705728708818815
E2E (s)8.465.655.715.976.01
10k32TPOT (ms)35.1117.2317.2219.1419.51
TTFT (ms)12,33512,66612,35614,26714,148
E2E (s)35.3930.0629.4833.7233.61
14k1TPOT (ms)26.1814.8514.7815.1715.46
TTFT (ms)1,1061,1381,1141,2731,272
E2E (s)8.585.435.405.655.73
14k16TPOT (ms)28.1916.3416.3517.4517.70
TTFT (ms)10,02710,36410,04211,51211,517
E2E (s)26.5823.8523.2926.3126.37
25k1TPOT (ms)36.0415.4815.6615.8016.19
TTFT (ms)2,7592,8422,7233,0903,089
E2E (s)11.646.726.667.077.15
25k8TPOT (ms)36.9916.6916.4817.4017.84
TTFT (ms)12,98113,34512,80214,54914,558
E2E (s)31.6627.3526.4029.5929.71
Reviewer_WJ87OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a sparse attention system for browser agents by utilizing the webpages DOMs. It partitions the prompt into semantic chunks using existing structural boundaries, keeping the web pages related information. It also introduces a length calibration ranking score based on Quest to account for different chunk lengths. Third it place chunks with similary scoring new one another to reduce page reads.

Critical strengths
  1. The expriment only generate 64 tokens per request, and packing costs approximately 29ms per resuest and requires roughly 40-63 output tokens to amortize its overhead. For shorter calls, this additional cost may outweigh the decode saving. Evaluating the actural generation lengths and reporting total request latency as well as the TTFT would clarify when packing become worthwile.

  2. DOM-based chunking is well motivated for browser agents, but DOM boundaries do not always correspond to meaningful units for the task. Layout-oriented containers may group unrealted content or separate information that needs to be considered together.

Critical weaknesses
  1. Brower agents repeadly process long, structured prompts, making kv access an important inference bottleneck. The paper highlights how fixed-size attention blocks can split related webpage content and introduce unnecessary KV reads. DOM based chunking is a well-motivated design choice for this setting. The methods also considers how that selection should be implemented efficiently in memory. Connecting the application's structure with both attention selection and KV placement is a key methodological strength of the paper.

  2. The evalution covers ultiple relevent benchmarks and compares against both dense attention and other spare methods. The reported results show that the complete approch can reduce decoding cost while maintaining or improving task performance in the evaluated settings.

Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Author Rebuttal to the Reviewer WJ87 (1/2)

Thank you for taking the time to review our work and for providing insightful and supportive feedback. We have addressed your concerns and questions below.


W1. The expriment only generate 64 tokens per request, and packing costs approximately 29ms per resuest and requires roughly 40-63 output tokens to amortize its overhead. For shorter calls, this additional cost may outweigh the decode saving. Evaluating the actural generation lengths and reporting total request latency as well as the TTFT would clarify when packing become worthwile.

Thank you for your comment. We clarify that 64 tokens is the sampling point for measuring TPOT, not the length of an agent step. Table 1 below shows the output token length in both reasoning mode and non-reasoning mode of the Qwen3-32B model by using the dense attention. Median output lengths are 273–301 tokens without explicit reasoning and 741–798 tokens with reasoning enabled. We have added the corresponding distributions and measurement details to Appendix A.7 as well.

Table 1. Output length under Dense attention (Median / P90 / P99, in tokens). Each mode pools three runs over all tasks: 28,720 non-reasoning outputs and 26,367 reasoning outputs.

BenchmarkNon-reasoningReasoning
WebVoyager273 / 373 / 522754 / 1535 / 3515
GAIA-web301 / 403 / 553798 / 1543 / 3941
Online-Mind2Web277 / 375 / 548741 / 1359 / 3181

Tables 2 and 3 below replay full agent outputs at four representative input lengths (6k, 10k, 14k, and 25k tokens) and report TTFT, TPOT, and end-to-end latency (E2E) for Dense, BrowSA, Quest, and Block Sparse in both reasoning and non-reasoning mode. All sparse methods use the same 6.25% selection ratio. TTFT includes the full one-time cost of chunking, scoring metadata, and packing; E2E spans from the start of prefill to the last decoded token, so it includes every overhead. Relative to Dense, this one-time preparation raises TTFT by 2–4%, and BrowSA’s lower TPOT recovers the gap within the first 3–29 decoded tokens in every configuration, whereas the median output has 273–301 tokens without reasoning and 741–798 tokens with reasoning (Appendix A.7); even the shortest recorded outputs (112 tokens without reasoning and 282 with reasoning) are far past this point. Relative to Quest and Block Sparse there is nothing to recover: BrowSA’s TTFT is 8–11% lower and its TPOT is lower in every configuration. As a result, BrowSA is 10–52% faster than Dense, 1–11% faster than Quest, and 3–11% faster than Block Sparse end to end across all 16 configurations. Packing on its own is close to latency-neutral at this fixed page budget (TPOT within 0.36 ms of BrowSA without packing, TTFT higher by 1.3–4.7%); its benefit at a fixed budget is higher task success on GAIA, from 40.0% to 44.4% and from 43.3% to 45.6% (Table 4 below).

Table 2. Reasoning mode: TTFT, TPOT, and end-to-end latency at natural output lengths

InputBMetricDenseBrowSAQuestBlock Sparse
6k1TPOT (ms)19.3514.4314.7015.00
TTFT (ms)353368411408
E2E (s)13.6410.3610.6110.78
6k32TPOT (ms)27.4216.4218.1818.37
TTFT (ms)6,5656,7307,5027,476
E2E (s)31.1624.1026.7426.82
10k1TPOT (ms)22.9414.6715.0215.33
TTFT (ms)715738826826
E2E (s)19.8713.1513.4913.77
10k16TPOT (ms)24.7016.1717.3017.51
TTFT (ms)6,5226,6907,5497,549
E2E (s)32.5625.9028.3528.53
14k1TPOT (ms)26.7615.2215.2815.58
TTFT (ms)1,1111,1461,2781,276
E2E (s)37.6422.2022.4322.81
14k16TPOT (ms)28.7016.3817.5817.84
TTFT (ms)9,88610,16211,38511,363
E2E (s)57.6241.6345.4745.79
25k1TPOT (ms)36.3415.5415.7516.14
TTFT (ms)2,7282,8153,0673,069
E2E (s)35.4817.0717.5417.87
25k8TPOT (ms)37.3316.6017.4917.91
TTFT (ms)12,72913,11714,35514,356
E2E (s)55.8438.1140.9841.35
AuthorsOpenReview ↗

Author Rebuttal to the Reviewer WJ87 (2/2)

Table 3. Non-reasoning mode: TTFT, TPOT, and end-to-end latency at natural output lengths

InputBMetricDenseBrowSAQuestBlock Sparse
6k1TPOT (ms)19.0013.8714.6814.96
TTFT (ms)357369408410
E2E (s)4.993.784.024.09
6k32TPOT (ms)26.8616.1317.9518.18
TTFT (ms)6,5596,7117,5227,497
E2E (s)18.8216.5618.5318.54
10k1TPOT (ms)22.7014.3214.8915.11
TTFT (ms)705728818815
E2E (s)8.465.655.976.01
10k32TPOT (ms)35.1117.2319.1419.51
TTFT (ms)12,33512,66614,26714,148
E2E (s)35.3930.0633.7233.61
14k1TPOT (ms)26.1814.8515.1715.46
TTFT (ms)1,1061,1381,2731,272
E2E (s)8.585.435.655.73
14k16TPOT (ms)28.1916.3417.4517.70
TTFT (ms)10,02710,36411,51211,517
E2E (s)26.5823.8526.3126.37
25k1TPOT (ms)36.0415.4815.8016.19
TTFT (ms)2,7592,8423,0903,089
E2E (s)11.646.727.077.15
25k8TPOT (ms)36.9916.6917.4017.84
TTFT (ms)12,98113,34514,54914,558
E2E (s)31.6627.3529.5929.71

W2. DOM-based chunking is well motivated for browser agents, but DOM boundaries do not always correspond to meaningful units for the task. Layout-oriented containers may group unrealted content or separate information that needs to be considered together.

Thank you for the comment. BrowSA does not rely on DOM boundaries being perfect. The chunker enforces a maximum and a minimum chunk length: DOM elements above the upper bound are split, and elements below the lower bound are merged with their neighbors, so a layout container that groups unrelated content is broken into smaller chunks that are scored independently. Packing further limits the effect of imperfect boundaries: chunks that are likely to be selected together are placed in the same or adjacent KV pages, so selecting a page brings in less unrelated content. To quantify the contribution of semantic boundaries, we augmented the ablation study to include semantic chunks alongside the BrowSA score and packing, introducing one component at a time (Table 4). We compare semantic chunks with positional chunks at two lengths: 32 tokens, which match the page size of BrowSA, and 16 tokens, which match the upper bound of the semantic chunk length and control for granularity. Semantic chunks outperform positional chunks of either length at both selection ratios: 40.0% versus 37.8% (16-token) and 34.4% (32-token) at a 6.25% selection ratio, and 43.3% versus 40.0% (16-token) and 41.1% (32-token) at a 12.5% selection ratio. Granularity alone does not explain the gain: halving the positional chunk length helps at the 6.25% ratio but not at the 12.5% ratio, whereas semantic chunks improve at both. Adding packing on top gives the best pass rate at both ratios, 44.4% at 6.25% and 45.6% at 12.5%. We have added this ablation to Section 4.3 and Table 4 of the revised paper, with per-run success rates in Appendix A.8; the changes are marked in red.

Table 4. Ablation study of semantic chunks, BrowSA score and packing on the GAIA benchmark.

Configuration6.25% selection ratio12.5% selection ratio
Semantic chunks (16 tokens upper bound), BrowSA score, packing, page size 3244.445.6
Semantic chunks (16 tokens upper bound), BrowSA score, no packing, page size 3240.043.3
Positional chunks (16 tokens), BrowSA score, no packing, page size 3237.840.0
Positional chunks (32 tokens), BrowSA score, no packing, page size 3234.441.1
Positional chunks (32 tokens), Quest score, no packing, page size 3232.238.9
Dense (100% KV)37.837.8
AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer WJ87,

Thank you again for your time and thoughtful review. As the discussion period is approaching its end, we wanted to follow up to ensure that our additional experiments and clarifications have addressed all your concerns. We would greatly appreciate any further guidance or confirmation from your side.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to All Reviewers

We thank all reviewers for their careful reading and constructive feedback. This response adds the following experiments and clarifications; the parts that enter the revised paper are marked in red there.

  1. System performance at natural output lengths (Reviewers WJ87, 4187, zSwn). We report the output-length distributions of the agent in reasoning and non-reasoning mode on all three benchmarks, and replay complete agent outputs to measure TTFT, TPOT, and end-to-end latency for Dense, BrowSA, Quest, and Block Sparse at input lengths from 6k to 25k tokens and batch sizes from 1 to 32, with prefix caching disabled for all methods (Appendices A.7 and A.9). The responses to Reviewers 4187 and zSwn additionally report BrowSA without packing in the same setting: at a fixed page budget, packing is close to latency-neutral (TPOT within 0.36 ms, end-to-end latency within 3.6%), and its benefit appears as higher task success at the same budget and as 10–12% fewer fetched pages for a fixed set of selected tokens.
  2. Ablation of semantic chunking, the BrowSA score, and KV packing (Reviewers WJ87, 4187, zSwn). Starting from Quest on GAIA, we add each component one at a time at two selection ratios and report per-run success rates (Section 4.3, Table 4, Appendix A.8).
  3. Comparison against LServe (Reviewer zSwn). We add LServe as a baseline on GAIA, Online-Mind2Web, and WebVoyager at matched selection ratios and page sizes.
  4. Run-to-run variance, statistical significance, and evaluation protocol (Reviewers 4187, zSwn). We give the individual-run success rate, mean Pass@1, and Pass@3 for Dense and BrowSA on all three benchmarks and for every row of the GAIA ablation. We also report task-level paired comparisons of BrowSA against Dense, Quest, and Block Sparse (exact McNemar tests on Pass@3 and 95% paired bootstrap intervals on mean Pass@1): BrowSA matches or exceeds Dense in every setting, and significantly outperforms Quest or Block Sparse in 8 of 12 comparisons. Finally, we specify the judge, the randomization settings, and the handling of website failures.
  5. Sensitivity of the calibration parameters A and α (Reviewer zSwn). We report offline action agreement with Dense attention and end-to-end GAIA success over a grid of (A, α) at page sizes 32 and 64.

We believe these revisions strengthen the paper while keeping its core contributions unchanged. We thank the reviewers again for their time and feedback.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.