ProxyEraseAgent: Blind Watermark Removal in the Wild

Invisible image watermark removal has received growing attention. Despite sub- stantial progress, existing attacks still face a tension between practicality and at- tack specificity. Attacks that exploit detector outputs, decoder responses, or paired watermarked and clean images can be tailored to the watermark decision bound- ary, but they require information that is rarely available in realistic dissemination scenarios. In contrast, attacks based on compression, geometric distortion, or image reconstruction are easy to deploy from a single watermarked image, but they are largely open-loop: they apply generic transformations without knowing whether the current image is moving toward watermark failure. Thus, the key challenge in single-image blind watermark removal is not merely how to trans- form the image, but how to obtain a useful direction for removal without access to the hidden decoder. To bridge this gap, we propose ProxyEraseAgent, an agent-driven framework that recovers attack specificity through proxy decoder responses. Publicly available watermarking schemes provide a natural external knowledge base of candidate decoders, where some are informative for a given unknown watermarked image. Our insight is that a decoder producing a stronger calibrated response to the query image is more likely to share a nearby decoding boundary with the hidden target mechanism. ProxyEraseAgent ranks knowledge- base decoders by calibrated response strength and uses the top ones as proxy boundary estimators. Their responses then guide a progressive search over hetero- geneous removal operations, including geometric distortion, JPEG compression, image reconstruction, and gradient-based perturbation, under perceptual-quality constraints. Experiments across 11 image watermarking systems show that Proxy- EraseAgent achieves an attack success rate of 94.8%, demonstrating the effective- ness of response-guided proxy retrieval and feedback-driven sequential planning for blind watermark removal