acceptodds
Under review as a conference paper at ICLR 2027

Draw What Matters: Decoupling Stroke Capacity from Sketch Complexity in Photo-to-Vector Generation

Abstract

A good sketch is an act of abstraction: it keeps what makes a subject recognizable and drops the rest. Most photo-to-vector methods, however, fix a stroke budget and render every stroke, so a single number sets both the of a sketch, how much it can express, and its , how much it must show. Too few strokes lose recognizable parts, too many add clutter, and the right number changes from one image to the next. We present AdaSketch, which decouples the two: the budget only bounds capacity, and each stroke has to earn its visibility. AdaSketch draws one closed contour that secures the silhouette, plus a pool of detail strokes with learnable opacity. Which details survive is decided by coverage rather than by count: salient interior points must be covered (faithfulness), each by as few strokes as possible (conciseness), and a continuous parameter sets how much image-relevant detail to keep. A progressive contour-to-detail schedule settles where to draw before deciding which strokes to keep. Under the same budget, AdaSketch outperforms five existing methods on all five semantic, structural and perceptual metrics, adapts the rendered complexity to both content and the chosen abstraction level, and extends to target-specific sketching in multi-object scenes.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
sketch generation, photo-to-sketch synthesis

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_mxN3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents AdaSketch, a photo-to-vector sketch generation method that aims to separate the initialized stroke capacity from the complexity of the rendered sketch. The method represents an object using a fixed closed contour and a set of detail strokes with learnable opacities. Foreground segmentation and CLIP attention maps guide stroke initialization, while coverage, uniqueness, and relevance objectives encourage the preservation of salient details and suppression of redundant strokes. A progressive contour-to-detail optimization strategy, combined with diffusion-based SDS guidance, jointly optimizes stroke geometry and visibility.

The method is evaluated against five existing approaches on 16,990 images from 85 object categories. The authors report improvements across five semantic, structural, and perceptual similarity metrics computed against XDoG edge references. Additional qualitative experiments demonstrate controllable abstraction under different stroke budgets and target-specific sketch generation in multi-object scenes.

Critical strengths

1. Reasonable motivation and structured optimization design.

The paper addresses a meaningful problem in vector sketch generation: different objects require different levels of detail, while fixed stroke budgets may lead to missing structures or redundant strokes. The contour-detail representation and progressive optimization strategy provide a reasonable framework for preserving global object structure while selectively adjusting internal details.

2. Promising visual results and relatively extensive evaluation.

The generated sketches generally preserve recognizable silhouettes and produce visually clean results. The paper evaluates the method against five baselines on a relatively large dataset of 16,990 images across 85 categories, reporting consistent improvements on the selected metrics. The qualitative examples also demonstrate controllable levels of detail and applicability to multi-object scenes.

Critical weaknesses

1. The novelty and effectiveness of the proposed adaptive stroke selection are not fully justified.

The claimed decoupling mainly relies on fixing the contour while dropping detail strokes, which is conceptually similar to the learnable stroke pruning in CLIPascene. In addition, the coverage and uniqueness losses do not explicitly reduce the number of strokes, and they may conflict with each other under soft coverage. The interaction between these two terms and their sensitivity to loss weights are not sufficiently analyzed. Controlled comparisons against sparsity-based baselines under matched settings, together with a trade-off analysis between semantic fidelity and stroke complexity, are needed to validate the method's effectiveness.

2. The reliability of the semantic guidance and the evaluation metrics requires further validation.

While the method uses foreground masks, CLIP attention, and sampled points to locate important structures, high spatial attention does not necessarily translate to stroke-level semantic contribution. The stability of these signals under imperfect segmentation and different random seeds is also left unexamined. Moreover, comparing against XDoG edges does not reliably measure semantic recognizability or the quality of visual abstraction. Additional experiments on guidance robustness and semantic recognizability would considerably strengthen the paper.

Questions for authors

Q1. Effectiveness of Coverage-based Selection
Could the authors compare coverage-based selection with sparsity-based selection (e.g., CLIPascene) under matched SDS guidance and initialization? A semantic fidelity–visible stroke count Pareto curve would help demonstrate its specific advantages if it is possible.

Q2. Reliability of Semantic Guidance
How sensitive are the selected strokes and their visibility to variations in CLIP attention, foreground masks, and guidance-point sampling? Is there evidence that the estimated spatial importance correlates with stroke-level semantic contribution?

Q3. Evaluation Robustness
Were cases with inaccurate or failed foreground segmentation or CLIP attention guidance included in the reported quantitative results? How were such failures handled, and could the authors provide a robustness analysis showing their impact on overall performance? Were comparable preprocessing and segmentation conditions used across baselines?

Reviewer_xs3LOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AdaSketch for photo-to-vector sketch generation. Unlike methods that render a fixed number of strokes, AdaSketch assigns learnable opacities to detail strokes and selects which ones to retain. The method combines a closed contour, a coverage-based loss, and a progressive optimization strategy. Experiments on the ControlSketch dataset show improvements over five baselines across five evaluation metrics.

Critical strengths
  1. The idea of separating stroke capacity from the number of rendered strokes is interesting. It addresses the problem that different images need different levels of detail.
  2. The contour-detail representation makes sense. Keeping the outer contour fixed while selecting interior strokes gives the method a clear way to preserve object shape.
  3. The coverage-based formulation is a nice part of the method. Instead of directly penalizing stroke count, it considers whether a stroke covers useful regions or overlaps with other strokes.
  4. The results in Table 1 are good. AdaSketch performs best on all five reported metrics, and the examples in Figure 3 generally look cleaner than those of the baselines.
  5. Figures 4 and 5 show that the method works with different abstraction levels and stroke budgets. The paper is also easy to follow.
Critical weaknesses
  1. Table 1 compares all methods using the same maximum budget of 62 segments, but AdaSketch does not necessarily render all of them. Since reducing redundant strokes is a central claim, the paper should report how many strokes are actually retained. A comparison at similar numbers of visible strokes would be more informative.
  2. The evaluation relies entirely on similarity to XDoG edge maps. This may favor sketches that closely reproduce object contours, but a good sketch does not necessarily need to match an edge map. The evaluation lacks any measure of recognizability or human preference.
  3. I would like to see a clearer ablation of the three losses in Section 3.3. The coverage loss, overlap penalty, and keep loss are central to the method, but their individual effects are not quantified in the main experiments. Figure 9 provides qualitative examples, which are useful but insufficient to disentangle their contributions.
  4. Section 4.1 mentions up to 3,000 optimization iterations per image on an A800 GPU. This seems expensive, especially compared with feed-forward methods such as SwiftSketch. The paper does not report actual generation time.
  5. Figure 4 shows several examples with different values of λkeep, but it is unclear how consistently this parameter controls the number of visible strokes across the full dataset. A simple plot of λkeep against the average retained stroke count would be useful.
Questions for authors

I lean toward acceptance. The main idea is interesting, and the method yields strong results in both quantitative and qualitative comparisons. I also like the coverage-based formulation for selecting strokes.

My main concern is that the experiments do not directly measure the reduction in stroke complexity, which is the paper's key motivation. The missing loss ablations and runtime comparison also leave some questions. Still, I think the method is reasonable, and the results are strong enough to support acceptance.

Questions For Authors:

  1. How many detail strokes are retained on average with N = 62? Can the authors compare against baselines with similar visible-stroke counts?
  2. What happens when Lcov, Lu, or Lkeep is removed individually?
  3. What is the average generation time per image, including preprocessing?
  4. Does the number of retained strokes change consistently with λkeep across the dataset?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.