acceptodds
Under review as a conference paper at ICLR 2027

The Granular Gambit: Checkmating Behavior-based CAPTCHAs

Abstract

Behavior-based CAPTCHAs such as Google reCAPTCHA v3 enable frictionless verification by assigning risk scores from fine-grained user interaction traces. Prior studies, however, largely rely on coarse-grained analyses and fail to capture the atomic behavioral signals underlying v3’s scoring mechanism. This work examines whether agentic AI can synthesize human-like low-level interactions to obtain high trust scores and bypass reCAPTCHA v3. The problem is formulated as sequential decision-making over atomic behaviors, and Seqion, an LLM-driven agentic framework for fine-grained web interaction execution, is introduced. Experiments on a controlled reCAPTCHA v3 testbed, including registration, forum posting, and content reading tasks, show that agentic AI can consistently increase trust scores and achieve up to a 90% bypass success rate, even without domain-specific knowledge. Furthermore, evaluations on an external, real-world website reveal that Seqion achieves a mean trust score of 0.60, significantly outperforming the state-of-the-art baseline of 0.34 and closely approximating the genuine human average of 0.69. These results expose fundamental limitations of frictionless CAPTCHA systems under rapidly advancing agentic AI.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Behavior CAPTCHAs, Large Language Models, Agent

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GG2jOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLM-driven agents can bypass behavior-based CAPTCHAs such as Google reCAPTCHA v3. The authors propose Seqion, a multi-agent framework built on browser-use that plans task-oriented actions and injects human-like behaviors, including curved mouse trajectories, keystroke delays, and non-uniform scrolling. Experiments on a self-hosted testbed and an external website show that Seqion obtains higher trust scores than the RL-based baseline of Abstract. Its scores exceed those of human users on the testbed and approach them on the external website. The authors conclude that fine-grained behavioral synthesis by agentic AI can defeat behavior-based CAPTCHAs.

Critical strengths
  1. The topic is timely and practically important. Understanding how robust frictionless CAPTCHAs are against capable LLM agents matters to the security community.

  2. The evaluation includes an external real-world website with human comparison, not only a self-hosted testbed.

Critical weaknesses
  1. The central "granular" claim is not supported. In Table 5, Bézier trajectories score lower than straight lines on all four backbones. The main gain comes from having mouse movement at all, which is a coarse signal.

  2. The behavioral effects are mostly small, inconsistent, and possibly confounded. Gains vary in direction across backbones, and the largest occur on abnormally low baselines. Static factors rival or exceed most behavioral effects: changing the IP raises the mean score from 0.52 to 0.83 (+0.31), and adding cookies raises it from 0.19 to 0.51 (+0.32); in the single-behavior ablations (Tables 5–7), every gain larger than this starts from a baseline of 0.30 or below. Repetition penalties, IP history, and session duration are not controlled, and no significance tests are reported.

  3. The baselines do not isolate the claims. Abstract [1] is task-agnostic, has few reproduction details, and its results shift sharply between environments (0.69 vs 0.34). There is no comparison with trajectory-level methods (e.g., Akrout et al. [2]; BeCAPTCHA-Mouse [3]), and no non-LLM scripted baseline to show what the agentic components contribute.

[1] Tsingenopoulos et al. Captcha Me If You Can: Imitation Games with Reinforcement Learning. EuroS&P 2022.

Questions for authors

Q1. Are the mouse and keyboard events trusted (e.g., via CDP) or synthetic dispatchEvent events? Can the authors confirm that reCAPTCHA actually observes the simulated behaviors?

Q2. Can the authors rerun the behavioral ablations in a controlled environment: fixed IP pool, clean browser profile, randomized condition order, and a duration-matched control? Please report pass rates with significance tests.

Q3. How does Seqion compare with a non-LLM scripted baseline using the same behavior library, and with trajectory-level methods [2, 3]?

[2] Akrout et al. Hacking Google reCAPTCHA v3 Using Reinforcement Learning.

[3] Acien et al. BeCAPTCHA-Mouse: Synthetic Mouse Trajectories and Improved Bot Detection.

2 comments

W1 & Q1: Both mouse trajectories and keystrokes are executed via Tool Agent–generated JavaScript injected into the browser (e.g., dispatchEvent(new MouseEvent(...)) and input events with randomized delays). We confirm reCAPTCHA v3 observes these events because rrweb (which hooks standard DOM listeners like v3) captures the exact trajectories and intervals (Appendix K), and ablating these injected JS events immediately drops scores back to the 0.14–0.52 baseline (Table 5). We agree that Table 5 shows the primary gain comes from trajectory presence rather than curve complexity (with heuristic Bézier curves introducing wider curvature variance than human paths in Fig. 9d). Crucially, this reveals that reCAPTCHA v3 relies heavily on coarse spatiotemporal event presence and pacing rather than strictly enforcing isTrusted===true or complex trajectory kinematics.

W2 & Q2: First, single-behavior variations across tasks (Tables 5–7) reflect task-modality alignment (e.g., scroll helps on Blog Reading up to +0.39 rather than non-scrollable Signup), and gains are not limited to low baselines (on GPT-5-mini's 0.52 baseline, mouse movement reaches 0.70, and full Seqion reaches 0.78—a +0.54 gain over R1's 0.24 under a fixed IP without cookies, exceeding single static factors). Second, our experiments enforced strict environmental and temporal controls (to be documented in §4): except for the cookie ablation, browser profiles were fully reset before each run under a fixed IP and site key, with a 48-hour cooldown interval between experimental modules to prevent cross-condition repetition penalties, and identical on-page interaction time intervals across variants (the 111s vs. 251s in Table 3 reflects off-page LLM planning latency). Third, because trust scores are discrete ordinal values ({0.1, 0.3, 0.7, 0.9}) and consecutive trials across the 100-run sequence experience cumulative repetition effects, we applied the non-parametric Wilcoxon signed-rank test paired by trial index: Seqion achieves a 90% pass rate (score >= 0.7) on the testbed and 73% (vs. 21% for Abstract) on the external site, with Seqion's mean score (0.60) significantly exceeding Abstract (0.34) on the external site (Wilcoxon signed-rank p = 3.2e-12, rank-biserial r = 0.94).

W3 & Q3: While a non-LLM scripted baseline using the same behavior library performs similarly to R3 (0.71 mean score, Table 3) on known static selectors, it fails to generalize across arbitrary DOMs without manual selector rewriting and lacks task-aware topological ordering (R4 improves R3 from 0.71 to 0.78 under identical interaction pacing). This also explains why task-agnostic Abstract drops from 0.69 (testbed) to 0.34 (external e-commerce site). Furthermore, prior trajectory-level methods [2, 3] optimize isolated 2D pointer paths (e.g., for v2 checkboxes) without semantic task planning, typing rhythm, or scroll coordination; as Tables 5–7 show, v3 continuous verification requires matching the primary interaction modality of the task (mouse, keystroke, and scroll) rather than mouse curvature alone. We will add [2, 3] and this discussion to §2 and §4.

Thanks for the response. The rebuttal solves some of my concerns. My other concerns remain:

  1. rrweb capture does not show that v3 scores these events. Appendix L also generates no keydown/keyup events at all.
  2. No reruns were done, and the order of conditions within each module is undisclosed, with no randomization reported. Nominally comparable no-behavior settings give 0.52, 0.24 and 0.19 (Tables 5, 3 and 8), and there is no duration-matched control. The significance test covers Seqion vs. Abstract, not the ablations.

Therefore, I am inclined to keep my score.

Reviewer_PSLBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether LLM-driven browser agents can obtain high reCAPTCHA v3 scores by combining task-aware planning with human-like interaction patterns. The proposed framework, Seqion, integrates webpage parsing, strategy planning, and tool generation and execution to synthesize mouse movements, typing intervals, and scrolling behavior. Evaluation covers three tasks on a self-hosted website, four LLM backbones, and one external website. The paper reports trust-score improvements, component ablations, and comparisons of behavioral statistics against human users and an existing solver.

Critical strengths
  1. The paper addresses a consequential security question. As browser agents become more capable, evaluating their interaction with behavioral verification is increasingly relevant. This study complements work on solving explicit visual CAPTCHA challenges by examining whether task-completing agents can also generate interactions that receive high trust scores.

  2. The evaluation provides useful evidence beyond an aggregate success rate. The task-specific comparisons and component ablations help identify which parts of the system contribute to score improvements. Table 3 shows a substantial contribution from behavior generation and an additional benefit from planning. The external-site results also show a sizable improvement over the evaluated baseline, from a mean score of 0.34 to 0.60, while remaining below the human mean of 0.69. Together, these findings support acceptance as an informative empirical study of the evaluated configurations.

Critical weaknesses
  1. The evaluation lacks website diversity. The three main tasks come from the same self-hosted social website, and the additional evaluation covers only one external website. This leaves it unclear how Seqion performs across broader web environments, such as video platforms or e-commerce sites, where page structures, interaction patterns, and task workflows differ substantially. The current evaluation therefore provides limited evidence of the framework’s real-world generalizability.

  2. The contribution of semantic planning is not fully isolated. Table 3 shows that adding planning increases the mean score from 0.71 to 0.78, but also increases runtime from 111 to 251 seconds. A duration-matched comparison would help clarify whether the gain comes from task-aware planning or simply longer interaction time.

Questions for authors
  1. What type of website was used for the external evaluation, and what tasks did the agents perform there? Were the interaction patterns substantially different from those on the self-hosted social website? This would help assess how much evidence the external evaluation provides for cross-site generalization.

  2. In Table 3, adding planning improves the mean score from 0.71 to 0.78 while increasing runtime from 111 to 251 seconds. Do the authors have evidence that separates the benefit of task-aware planning from the effect of longer interaction time?

  3. Evaluate Seqion across more diverse website categories, such as video platforms and e-commerce sites, to examine whether its benefits extend to different page structures and interaction patterns.

1 comment

W1 & Q1 & Q3: The external website in §4.5 is an e-commerce shopping platform. Agents executed an end-to-end shopping workflow: locating the search bar, typing product keywords, scrolling through listings when necessary, moving the cursor to the target purchase button, and clicking to add to the cart. While low-level motor primitives remain a combination of mouse movement, typing, and scrolling, the page structure and conditional workflow differ from the static form-filling on the KLiK social testbed. Depending on whether the searched item already appears within the visible viewport, the agent dynamically decides whether wheel scrolling is needed before clicking. Thus, our evaluation spans both social media/content (signup, forum posting, blog reading) and e-commerce workflows. The task-agnostic Abstract baseline drops from 0.69 (social testbed) to 0.34 (e-commerce), whereas Seqion maintains 0.60 (vs. 0.69 human mean), demonstrating cross-site generalization.

W2 & Q2: In our experiments, we enforced identical inter-action time intervals across variants during browser execution to avoid duration confounding (we will add this detail to §4.4). Specifically, the runtime increase from 111s (R3) to 251s (R4) in Table 3 reflects end-to-end wall-clock time, dominated by off-page LLM inference for O(N^2) element-description matching and topological extraction (60.5k tokens in R4 vs. 18.0k in R3). Once the plan is generated, the Running Agent executes actions using the exact same per-step timing intervals in R3 and R4. Thus, the gain (0.71 to 0.78) stems from task-aware semantic ordering (e.g., avoiding clicking off-screen elements without scrolling, Appendix H) rather than longer interaction time.

Reviewer_7xUAOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents Seqion, a browser-use–based multi-agent pipeline (parse → categorize/plan → generate and run JavaScript interaction tools). It aims to raise reCAPTCHA v3 scores by synthesizing mouse, keystroke, and scroll behavior.

The evaluation has three parts:

  • Self-hosted testbed (KLiK PHP site): signup, forum reply, and blog-reading tasks, with 100 runs per condition across four LLM backbones.
  • Comparisons: a human condition and the "Abstract" RL baseline (Tsingenopoulos et al., 2022).
  • "External" site: Seqion reaches a mean score of 0.60, versus 0.34 for Abstract and 0.69 for humans.

The paper also reports KL divergences on rrweb-derived features. Headline claim: up to a 90% bypass rate at the ≥0.7 threshold.

Critical strengths
  • Timely, relevant question. Can LLM agents defeat invisible behavioral verification?

  • Useful factor decomposition. The breakdown into mouse, keystroke, scroll, cookie, and IP effects is informative. Arguably the most interesting finding, that static reputation signals dominate, cuts against the paper's own thesis and is worth noting.

Critical weaknesses
  1. The behavioral effects are likely confounded by environment and ordering.

    • The paper's own Appendix D shows static factors dominate. Changing IP raises the mean from 0.52 to 0.83, and cookies raise it from 0.19 to 0.51. Both effects are larger than most behavioral manipulations.

    • Appendix H reports a repetition penalty: scores fall from {0.7, 0.9} to {0.1, 0.3} over 100 repetitions. So the comparisons are only meaningful if conditions were randomized or interleaved, run on fresh browser profiles, and run on the same IP and site key.

    • Symptoms of confounding:

      • The no-behavior baseline varies wildly by backbone (GPT-5-mini 0.52 vs GPT-4.1-mini 0.14), even though the backbone shouldn't affect low-level signals.
      • Seqion "outperforms humans" (0.78 vs 0.68).
  2. The implementation doesn't match the behavioral claims.

    • The Appendix L tool code dispatches synthetic MouseEvents via dispatchEvent. These carry isTrusted=false and are trivially distinguishable from real input.
    • The Bézier mover is essentially deterministic: fixed control-point offsets (−80px), 14 steps, 320ms. That contradicts the "biological noise / Fitts's law" framing of Hypothesis 1.
    • It's unclear which actions go through browser-use/CDP, which would produce trusted events, and which go through injected JavaScript. This matters a lot for interpreting why scores change.
  3. The external evaluation and ethics statement contradict each other, and the "real-world" claim is overstated. The ethics statement says the Tool Agent's JavaScript was "never executed against live production systems." Yet §4.5 runs Seqion on an external site with "considerable web traffic."

Questions for authors
  • How were conditions ordered across the 100 runs? Were browser profiles, IP, and site key reset? Were human and bot runs interleaved?
  • Are interaction events trusted (CDP) or synthetic (dispatchEvent)? How are "keystrokes" generated?
  • What exactly ran on the external site? How does that square with the ethics statement? Was reCAPTCHA deployed?
  • What is EVAL in Algorithm 1? Does it query reCAPTCHA scores?
  • How many human participants were there, what protocol did they follow, and was there ethics approval?
1 comment

Q1: Experiments were conducted sequentially across modules (Mouse Movement, Keystroke Dynamics, Scroll Pattern, User Study, Practicality). Crucially, to eliminate cross-condition repetition penalties while keeping the IP and site key constant, we enforced a 48-hour cooldown window between experimental blocks. Except for the cookie ablation (Appendix D), browser profiles were fully reset (clearing cookies, cache, and storage) prior to each run under the same IP. The site key was intentionally kept constant to mirror real production deployments and preserve Google's backend baseline for the domain. Human runs were executed first (rather than interleaved) because reCAPTCHA v3 requires legitimate baseline traffic to calibrate site-specific risk scoring before evaluating subsequent bot sessions. Furthermore, in the no-behavior baseline, backbones still control execution pacing via browser-use; variations in LLM inference latency naturally altered inter-action dwell times across models.

W2 & Q2: Both mouse trajectories and keystrokes are executed via Tool Agent–generated JavaScript injected into the browser (e.g., dispatchEvent(new MouseEvent(...)) and input events with randomized delays, Appendix L). We agree that DOM-dispatched events carry isTrusted=false and use simplified geometric curves. Rather than invalidating our results, this exposes a critical empirical vulnerability: reCAPTCHA v3's scoring model relies heavily on coarse spatiotemporal event presence and pacing rather than strictly enforcing isTrusted===true or complex Fitts's Law kinematics. We will tone down Hypothesis 1 to accurately reflect this finding.

Q4: EVAL(t', s) in Algorithm 1 does not query live reCAPTCHA v3 scores. It is an offline heuristic check verifying that the generated JS code executes without syntax or DOM errors and produces valid kinematic event streams (e.g., non-zero trajectory points and typing intervals) before task execution.

W3 & Q3 & Q5 & Ethics: We apologize for the imprecise phrasing in the Ethics Statement ("never executed against live production systems"), which meant Seqion was never executed against unauthorized third-party websites. For §4.5, we obtained explicit consent from the external site administrator and deployed reCAPTCHA v3 in a passive, logging-only mode without affecting normal traffic (we will revise the Ethics Statement to remove this contradiction). For the user study, four adult volunteers naturally completed the tasks following a standard task briefing; under our institutional guidelines, collecting strictly anonymized cursor coordinates and timestamps via rrweb on a sandboxed page involves zero PII and minimal risk, qualifying as exempt from formal IRB review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.