-
The task boundaries lack independent validation, making it difficult to determine whether the reported positives represent unnecessary data collection.
-
The context experiment changes page complexity and attack content together, so it cannot isolate attacker influence from ordinary broad context extraction.
-
The practical value of profile-aware scoring remains unclear. Generic scores rank observed positives better, while limited runtime coverage makes negative outcomes difficult to interpret.
Major Concerns
- The boundary definition is not validated well enough to support the central labels.
A deterministic profile assignment is reproducible, but it is not necessarily correct. The ordered regular-expression classifier assigns a single profile and defaults unmatched extensions to general copilot. The paper does not evaluate these assignments against independent human judgments, nor does it provide a complete profile-by-action-by-marker policy specifying which content is allowed under each exercised workflow.
The checks in Appendix A do not substitute for this validation. Agreement between metadata-only and metadata-plus-manifest classification measures stability of two related heuristics. Similar profile distributions across stores likewise do not establish that individual classifications or permitted-input boundaries are accurate.
- RQ3 does not isolate attacker influence from broader input collection.
High-complexity pages add hidden/contextual fields and attack-bait, while low-complexity pages contain task-essential surfaces. Comparing the same extension across these pages controls extension identity, but several treatment components still change together. More importantly, some markers that can make a request boundary-positive are introduced only in the high condition. An extension using the same broad extraction routine could therefore become newly positive simply because the page now contains an additional disallowed field.
The attack-bait result has a related ambiguity. Seeing instruction-like page text in an outbound request establishes transmission, not that the extension or its model followed those instructions. The discussion recognizes overbroad context assembly as a possible mechanism, but the causal framing around attacker influence remains stronger than the experiment supports.
- Dynamic coverage and the responsive-cohort selection are too weakly characterized.
Only 79 of 5,993 extensions with usable traces produce a remote marker-bearing event in the baseline. This is not evidence that the rest do nothing, as the paper acknowledges. However, the study does not independently measure how often its scripted actions actually reach the advertised feature. A usable trace may show that instrumentation worked while the extension remained blocked on onboarding, authentication, an unsupported page, or an unrecognized interaction.
The interaction ablation measures recovery relative to already observed positives. It does not estimate how many positives remain undetected outside that cohort. A stratified manual test of apparently negative extensions would make the negative outcomes much more interpretable.
RQ3 also needs its cohort size, exact enrichment criteria, per-browser/profile composition, and matched denominators for every condition. Percentages such as 25.00% for summarizers are hard to assess without counts. Repeated runs, uncertainty intervals, and analysis clustered by extension are needed because the same extension appears across page families and context levels. I agree that these results should not estimate Web prevalence, but even the within-cohort comparisons need a clearer experimental population.
- The static ablation does not demonstrate an improvement in triage.
Adding nonnegative rule weights raises the high/critical count from 1,303 to 2,492 and then 2,898. At a fixed score threshold, an increase is expected by construction. It does not show that generic analysis misses genuine privacy problems, or that the additional 1,595 packages are better review targets.
Appendix C.1 is more informative: generic scores achieve AUROC 0.7059 for remote-other outcomes, compared with 0.6520 for the full profile-aware score. Generic scores also perform best for boundary positives, with AUROC 0.7603. This does not invalidate using profiles to interpret a payload, but it leaves the proposed scoring system's practical benefit unestablished.
For the proposed review-queue use case, precision or yield at a fixed review budget would be more useful, alongside precision-recall curves and uncertainty. With only six baseline boundary-positive extensions, the stability of any ranking comparison is itself a concern. The paper should also avoid using "calibration" for a result that measures ranking discrimination rather than probability calibration.
- The methods and case evidence are not sufficiently specified for reproduction.
Appendix E provides rule names, weights, and counts, which is useful. It does not provide the exact detection patterns or establish whether a source and a model-facing sink merely coexist in a package or are connected by a feasible data flow. This distinction is essential for interpreting claims such as form-input-to-LLM coupling, especially in bundled libraries or dead code. The exact severity cutoffs are also not given in the supplied manuscript, despite the contribution statement saying they are exposed.
Technical and Presentation Issues
-
Define "remote-other-positive" before using it in Section 4.2. It is not one of the labels defined in Section 3.3. Also reconcile the 77 remote-other positives with the 48 high/critical and 26 non-high/critical positives, which sum to 74. If this is a restricted static/dynamic join, give its denominator and exclusions.
-
Report precision of the candidate-selection heuristic on a human-checked sample. The 5,151 packages with an endpoint or an LLM-specific code shape provide stronger evidence than metadata alone, but the remaining candidates still enter several LLM-extension analyses. Show whether the main results change on the stronger-evidence subset.