- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to allocate a limited sampling budget for diffusion based zero shot classification. It introduces TAPS, a transferable policy that uses class permutation invariant features from denoising losses to predict which images need additional noise draws. The readouts and sampling schedule are trained on source data, then frozen and transferred across tasks and diffusion backbones while adapting priorities to observed losses. Experiments show that TAPS generally improves budgeted classification over uniform allocation and several adaptive baselines, although the transfer benefit depends on the receiving model and donor policy.
Critical strengths
-
The paper addresses a practically important bottleneck in diffusion based classification and formulates transfer as a complete policy problem. Freezing the readouts, normalization, and schedule while allowing priorities to adapt to receiving backbone observations is a meaningful distinction from methods that require target side refitting.
-
The experimental design includes several useful controls. Shared budgets, cached losses, paired bootstrap intervals, source subset refits, readout ablations, fixed schedule comparisons, and within image stability diagnostics make the claimed role of observation dependent allocation more credible than a comparison against uniform sampling alone.
Critical weaknesses
-
The transfer claim is stronger than the evidence in some parts of the paper. The main transfer study covers only two closely related Stable Diffusion backbones and eight image classification tasks, while the SDXL experiment is small and shows that the advantage over stronger controls depends on the donor. More evidence is needed to distinguish a generally transferable signal from a signal that transfers mainly between related diffusion models and preprocessing pipelines.
-
The policy depends on a source selected stage schedule that is frozen during transfer, and the schedule itself can contribute more than the readout. The paper reports that replacing the receiver selected schedule with a balanced schedule reduces AUC substantially, while the readout gain over a shared schedule is much smaller. This makes it difficult to determine whether the main improvement comes from transferable stability prediction or from source specific allocation of sampling depth. The relationship between finite reference disagreement and actual classification accuracy is also indirect, since the self supervision target is not semantic error.
Questions for authors
-
Can the authors provide transfer results across substantially different diffusion families or conditioning pipelines, rather than primarily between SD 1.5 and SD 2.0? In particular, please report whether the same frozen policy transfers across different text encoders, image resolutions, noise schedules, or prediction parameterizations, and quantify performance degradation as the receiver becomes less similar to the donor.
-
How much of the reported gain is attributable to the source selected schedule rather than the TAPS readout? Please include a fully controlled comparison in which transferred TAPS, transferred Quad, Margin, and uniform allocation all use the same schedule, as well as a comparison in which each policy uses a schedule selected independently on the receiver. Please report both macro AUC and per task results.
-
The training target is disagreement with a five draw reference prediction rather than semantic correctness. How sensitive are the results to the reference draw count and to the choice of disagreement target? Please evaluate alternative reference counts, such as three and ten draws, and report correlations between predicted risk, disagreement, and eventual classification error or accuracy improvement from an additional draw.
-
How does TAPS behave under different candidate set sizes and class imbalance? The feature dimension is designed to be independent of the number of classes for sufficiently large candidate sets, but the experiments appear to mix tasks with very different class counts. Please provide controlled results varying the number of candidate classes and clarify whether the same readout and schedule are used without task specific calibration.
-
What is the practical overhead of computing the feature map, applying the readouts, and reranking active images at every stage? Since the paper emphasizes efficient inference, please report wall clock latency and memory for TAPS, uniform allocation, and the strongest adapted baselines under realistic batch sizes, including cache construction and routing overhead. It would also be useful to show whether the ranking overhead remains negligible when the number of images or candidate classes is much larger.