acceptodds
Under review as a conference paper at ICLR 2027

Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning

Abstract

Generative zero-shot learning (ZSL) synthesizes visual features for unseen classes by learning a semantic-conditioned generator from seen classes. Since semantic conditions determine the knowledge transfer from seen to unseen classes, their effectiveness is critical for generating plausible features. However, existing class-level semantic prototypes lack semantic diversity to capture intra-class variations, while exhibiting inconsistent inter-class structures with the visual space due to the semantic–visual gap. To address these issues, we propose Adaptive Attribute Distribution and Visual Structure Alignment (AAVS), a unified semantic condition learning framework for generative ZSL. Specifically, the Adaptive Attribute Distribution (AAD) learns dimension-adaptive attribute distributions with discriminability-guided variation calibration to capture transferable semantic diversity, while Visual Structure Alignment (VSA) aligns the diverse attributes with refined visual prototypes to learn visual inter-class structures. By jointly modeling semantic diversity and visual structural consistency, AAVS provides more effective generation conditions for unseen classes to generate visual features. Extensive quantitative and qualitative evaluations on three benchmarks demonstrate that AAVS outperforms existing generative ZSL methods and validate the effectiveness of the learned generation conditions.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Generative zero-shot learning, Zero-Shot Learning, Generation Condition, Semantic-Visual Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gzPVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AAVS (Adaptive Attribute Distribution and Visual Structure Alignment), a semantic condition learning framework for generative Zero-Shot Learning (ZSL). It aims to address two limitations of standard class-level attributes: the lack of intra-class diversity and structural inconsistency with the visual space. The Adaptive Attribute Distribution (AAD) component captures semantic diversity by predicting attribute variance per dimension, calibrated by inter-class discriminability. Visual Structure Alignment (VSA) projects attributes into the visual space and aligns them with visual prototypes via a prototype alignment loss. The resulting semantic conditions are combined with a generator to synthesize visual features for unseen classes.

Critical strengths

The paper clearly identifies two core limitations of static class-level semantic conditions in Generative Zero-Shot Learning (ZSL): the lack of semantic diversity to capture intra-class variations, and the cross-domain structural inconsistency caused by the semantic-visual gap.

The proposed Adaptive Attribute Distribution (AAD) mechanism learns variation patterns entirely within the shared attribute space, constructing diverse semantic conditions without relying on visual features or instance-level annotations of unseen classes, which shows good logical consistency under strict ZSL settings.

Extensive testing was conducted on three standard benchmark datasets (AWA2, CUB, and SUN), achieving an average harmonic mean (H) of 72.4% in Generalized Zero-Shot Learning (GZSL), demonstrating the engineering effectiveness of the method.

Critical weaknesses

The core technical contribution is relatively thin. The AAD module merely introduces Gaussian distributions to sample attribute variations, while the VSA module relies on conventional Singular Value Decomposition (SVD) to remove dominant directions followed by simple contrastive alignment. This combination of techniques is highly foundational in the fields of feature enhancement and manifold alignment, lacking breakthrough theoretical depth or algorithmic innovation.

The AAD module forcefully assumes that attributes with higher discriminability have smaller intra-class variation variance, utilizing a negative correlation mapping via Equations 3 and 4. This strong assumption is not universally valid in the complex real visual world, as certain attributes serving as key classification criteria may still exhibit massive natural variations among different individuals within the same class. The paper lacks biological justification or rigorous mathematical proof for this mechanism, making it appear as a constructed heuristic.

At a time when generative models have highly evolved, the paper still selects the 2019 f-VAEGAN as the core generator. Although the authors mention the application of Diffusion Models in ZSL, they do not attempt to integrate AAVS with modern diffusion model backbones, nor do they provide in-depth comparisons with the latest frontier models based on Large Language Models or Diffusion Models, significantly weakening the paper's academic competitiveness and temporal relevance.

The VSA module refines prototypes by removing the dominant eigenvector pcomp_{com} of the seen-class prototype matrix. However, the visual distribution of unseen classes is highly likely to depend on these shared underlying structures that are treated as biases. Overly suppressing the common direction, as shown in the appendix experiments where a high α\alpha value leads to a drop in unseen class accuracy, could destroy the critical cross-domain transferable knowledge relied upon by zero-shot learning, and the paper fails to theoretically guarantee the safety of this operation for unseen classes.

Questions for authors

Can the authors provide more in-depth theoretical derivations or extensive empirical evidence to prove the rigor and universality of the assumption in Equations 3 and 4—that highly discriminative attributes must have minimal variability—across various fine-grained visual datasets such as CUB?

Given that f-VAEGAN is relatively outdated, if the conditional generation framework of AAVS is seamlessly migrated to current Diffusion Models, can AAD and VSA still independently bring significant performance gains independent of a stronger generator?

When using SVD to extract and eliminate the dominant common direction pcomp_{com}, how does the model distinguish between dataset-specific biases unique to seen classes and critical cross-domain shared fundamental visual features? How can it be mechanistically ensured that the pcomp_{com} calculated based on seen classes will not cause destructive compression on the feature space of unseen classes?

13 comments
AuthorsOpenReview ↗

Response to Reviewer gzPV [Part I]

Dear Reviewer gzPV,

Thank you for your detailed review. We appreciate your recognizing our motivation, the design of AAD, and the empirical effectiveness of AAVS.

We understand your main concerns regarding (1) the technical contributions of our AAD and VSA modules; (2) the justification and generality of discriminability-guided variation calibration; (3) whether migrating AAVS to diffusion models would yield further gains; (4) whether prototype refinement removes information needed for unseen-class transfer. We address these concerns below.

W1: "The core technical contribution is relatively thin. ..., lacking breakthrough theoretical depth or algorithmic innovation. ”

R1: Thank you for your question. We would like to clarify our contributions in two aspects:

(1) A sound and generalizable motivation.

Our motivation is that improving intra-class semantic diversity and inter-class structural consistency can lead to better semantic conditions for generative ZSL. We systematically analyze these two aspects, demonstrate their importance for reliable feature generation, and provide empirical evidence supporting the soundness and generalizability of this motivation.

Building on this understanding, we develop targeted methods to effectively leverage these properties and validate their effectiveness through extensive experiments. We believe that this work provides a valuable perspective on understanding and improving semantic conditions in generative ZSL, with the potential to inspire future research in this direction.

(2) An innovative methodological approach.

Although the resulting framework is relatively simple, realizing our motivation requires overcoming several challenging issues. In particular, it is challenging to effectively leverage intra-class semantic diversity and inter-class structural consistency while ensuring generalization to unseen classes. Previous methods have attempted to introduce instance-level semantic enhancement using visual features [R1, R2]. However, such semantic enhancement cannot be applied to unseen classes without visual features, limiting its applicability in zero-shot learning.

We find that improving intra-class diversity and inter-class structural consistency entirely within the shared semantic space enables natural generalization to unseen classes. Based on this insight, we design AAD to model attribute distributions and VSA to incorporate visual inter-class structures. This coherent design provides an innovative methodological perspective for generative ZSL.

Please see Part II for the remaining response.

Reference:

[R1] S. Chen, W. Hou, Z. Hong, et al. Evolving Semantic Prototype Improves Generative Zero-Shot Learning. Proceedings of the International Conference on Machine Learning (ICML), pp. 4611–4622, 2023.

[R2] W. Hou, S. Chen, S. Chen, et al. Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 23627–23637, 2024.

AuthorsOpenReview ↗

Response to Reviewer gzPV [Part II]

Dear Reviewer gzPV,

W2: "The AAD module forcefully assumes that attributes with higher discriminability have smaller intra-class variation variance, ... ”

and

Q1: "Can the authors provide more in-depth theoretical derivations or extensive empirical evidence to prove ... that highly discriminative attributes must have minimal variability”

R2: Thank you for raising this important question. We address this concern from the following perspectives:

(1) Clarifying the underlying motivation.

AAD does not assume that attribute discriminability is negatively correlated with the actual intra-class variability of real-world attributes. Our underlying motivation is that modeling intra-class variations can provide more diverse semantic conditions and thereby benefit generative ZSL.

(2) Interpreting the learned attribute variations.

As shown in Figure 3(a) of the paper (p. 8), we analyze the learned variances σ\sigma across attribute dimensions. We observe that attributes with higher discriminability tend to have smaller learned variances, whereas less discriminative attributes exhibit larger variances.

One possible interpretation is that the learned adaptive distributions reflect the role of different attributes in inter-class distinction. More discriminative attributes receive smaller perturbations to preserve their class-related semantics and maintain inter-class distinguishability. In contrast, less discriminative attributes receive larger perturbations to introduce semantic diversity and capture intra-class variations.

(3) Empirical validation of this interpretation.

To further examine this interpretation, we conduct additional comparative experiments using different attribute-discriminability-based calibration strategies. The results show that assigning larger perturbations to more discriminative attributes (Positive) degrades model performance, whereas our proposed negative-correlation calibration strategy achieves better results, as shown in Table R1.

Table R1. Comparison of different attribute perturbation calibration strategies on AWA2, CUB, and SUN. Acc, U, S, H and Avg. H are reported (%).

AWA2CUBSUN
StrategyAccUSHAccUSHAccUSHAvg. H
Uniformwj=1w_j=184.878.885.681.978.672.779.276.074.258.853.255.971.3
Positivewj=djw_j=d_j80.977.781.579.674.867.180.073.073.860.850.255.069.2 (↓2.1)
Negative (our)wj=1−djw_j=1 - d_j85.482.185.083.579.274.281.077.475.858.954.056.372.4 (↑1.1)

This suggests that excessively perturbing highly discriminative attributes may undermine class-related semantics. Restricting their perturbations while allowing larger variations in less discriminative attributes can better balance intra-class semantic diversity and inter-class distinguishability. These results support our interpretation of the learned intra-class variations from the perspective of inter-class discriminability.

Please see Part III for the remaining response.

AuthorsOpenReview ↗

Response to Reviewer gzPV [Part III]

Dear Reviewer gzPV,

W3: "At a time when generative models have highly evolved, the paper still selects the 2019 f-VAEGAN as the core generator. ... ”

and

Q2: "..., can AAD and VSA still independently bring significant performance gains independent of a stronger generator?”

R3: Thank you for raising this concern. We address it from two aspects.

(1) Reasons for adopting f-VAEGAN as the primary generator.

Our contribution focuses on addressing two fundamental issues in generative ZSL: insufficient intra-class semantic diversity and inconsistent inter-class structures. Accordingly, we propose AAD and VSA as targeted solutions.

We adopt f-VAEGAN as the primary generator to ensure fair comparisons with existing methods and directly validate our proposed solutions. It serves as the common generator for a large proportion of state-of-the-art generative ZSL approaches.

(2) Additional experiments with a diffusion-based generator.

To examine the compatibility of AAVS with stronger generative backbones, we integrate it into DDGAN [1], a diffusion-based generative model, and evaluate it on AWA2, CUB, and SUN. As shown in Table R2, AAVS can also improve the performance of diffusion-based generators, demonstrating its generalizability beyond f-VAEGAN.

Table R2. Evaluation of AAVS with Denoising Diffusion GAN (DDGAN) on AWA2, CUB, and SUN. Acc, U, S, H, and Avg. H are reported (%). Results are averaged over three random seeds and reported as mean ± standard deviation.

MethodAWA2CUBSUN
AccUSHAccUSHAccUSHAvg. H
DDGAN (Base)75.67±3.3575.67_{\pm3.35}66.63±1.1166.63_{\pm1.11}81.37±3.3381.37_{\pm3.33}73.20±0.7073.20_{\pm0.70}79.13±1.5079.13_{\pm1.50}71.57±1.0771.57_{\pm1.07}78.30±1.4778.30_{\pm1.47}74.80±0.1074.80_{\pm0.10}75.00±0.6175.00_{\pm0.61}60.87±0.5160.87_{\pm0.51}50.07±0.6150.07_{\pm0.61}54.93±0.2154.93_{\pm0.21}67.64±0.3067.64_{\pm0.30}
Base + VSA73.70±3.1273.70_{\pm3.12}67.93±3.5367.93_{\pm3.53}82.67±5.1782.67_{\pm5.17}74.43±0.1574.43_{\pm0.15}79.63±0.3279.63_{\pm0.32}72.63±1.8872.63_{\pm1.88}81.33±2.20\mathbf{81.33}_{\pm2.20}76.70±0.1076.70_{\pm0.10}74.27±0.7874.27_{\pm0.78}56.43±0.5556.43_{\pm0.55}53.70±0.61\mathbf{53.70}_{\pm0.61}55.07±0.5855.07_{\pm0.58}68.73±0.2668.73_{\pm0.26}
Base + AAD78.17±3.8778.17_{\pm3.87}74.70±2.0174.70_{\pm2.01}85.13±3.18\mathbf{85.13}_{\pm3.18}79.57±1.2979.57_{\pm1.29}79.87±0.8679.87_{\pm0.86}74.30±2.6574.30_{\pm2.65}79.70±2.7979.70_{\pm2.79}76.87±0.1576.87_{\pm0.15}76.40±0.56\mathbf{76.40}_{\pm0.56}63.00±1.66\mathbf{63.00}_{\pm1.66}50.77±0.8150.77_{\pm0.81}56.20±0.2056.20_{\pm0.20}70.88±0.4870.88_{\pm0.48}
Base + AAVS82.63±1.44\mathbf{82.63}_{\pm1.44}77.53±1.10\mathbf{77.53}_{\pm1.10}85.13±1.84\mathbf{85.13}_{\pm1.84}81.13±0.31\mathbf{81.13}_{\pm0.31}81.47±0.67\mathbf{81.47}_{\pm0.67}75.50±1.73\mathbf{75.50}_{\pm1.73}79.40±1.5579.40_{\pm1.55}77.37±0.46\mathbf{77.37}_{\pm0.46}75.77±0.5775.77_{\pm0.57}61.37±1.7261.37_{\pm1.72}52.30±0.8752.30_{\pm0.87}56.47±0.31\mathbf{56.47}_{\pm0.31}71.66±0.25\mathbf{71.66}_{\pm0.25}
Δ\Delta (vs. Base)+6.97+10.90+3.77+7.93+2.33+3.93+1.10+2.57+0.77+0.50+2.23+1.53+4.02

As shown, AAD and VSA independently improve the average H by 3.24% and 1.09%, respectively. Their combination yields a 4.02% gain, from 67.64% to 71.66%. These results demonstrate the effectiveness of both modules with diffusion-based generators and highlight AAVS's potential as a plug-in framework for enhancing diverse generative backbones.

The DDGAN-based framework is briefly introduced below. It consists of a generator GθG_\theta, a feature discriminator D0D_0, and a transition discriminator DtrD_{\mathrm{tr}}. DDGAN models the reverse diffusion process using conditional GANs rather than a Gaussian approximation. Given a noisy feature xtx_t, timestep tt, a semantic condition aca_c, and random noise ϵ\epsilon, the generator predicts a clean feature, from which a reverse transition is sampled:

x0^=Gθ(xt,t,ϵ;ac),xt−1^∼q(xt−1∣xt,x0^).\hat{x_0}=G_{\theta}(x_t, t,\epsilon; a_c),\qquad \hat{x_{t-1}}\sim q(x_{t-1}\mid x_t,\hat{x_0}).

DDGAN employs two discriminators with complementary roles: the feature discriminator D0D_0 distinguishes real clean features x0x_0 from generated features x0^\hat{x_0}; the transition discriminator DtrD_{tr} distinguishes real reverse transitions (xt−1,xt)(x_{t-1},x_t) from generated ones (xt−1^,xt)(\hat{x_{t-1}},x_t), encouraging realistic denoising transitions. Both discriminators are conditioned on the semantic condition aca_c. During inference, visual features are synthesized from Gaussian noise through successive reverse transitions conditioned on aca_c.

Revision Location: Revisions appear in Section 4.8 (p. 10), Appendix A.1 (p. 14), Table 4 (p. 10).

Please see Part IV for the remaining response.

Reference:

[R3] Z. Xiao, K. Kreis, and A. Vahdat. Tackling the Generative Learning Trilemma with Denoising Diffusion GANs. International Conference on Learning Representations (ICLR), 2022.

Reviewer_cN1UOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AAVS, a framework for improving semantic conditions in generative zero-shot learning. It addresses the limited diversity of fixed class attributes and their structural mismatch with visual features. Adaptive Attribute Distribution learns attribute-wise Gaussian variations, calibrated to preserve discriminative attributes, while Visual Structure Alignment maps these conditions toward refined seen-class visual prototypes. Integrated with f-VAEGAN, the learned conditions generate unseen-class features for classifier training. The effectiveness of the method is evaluated on AWA2, CUB, and SUN under ZSL and GZSL settings.

Critical strengths
  1. A useful feature of AAD is that it can generate varied conditions for unseen classes from their attributes alone. It therefore remains usable when unseen-class images are unavailable, which is essential to the zero-shot setting.
  2. The paper considers both diversity within a class and relationships between classes. I find this combination well motivated.
Critical weaknesses
  1. The learned variations are not sufficiently shown to preserve class semantics. In Section 3.2, Equation (4) reduces the variance of discriminative attributes but does not similarly restrict their learned mean shifts. The KL penalty encourages a standard Gaussian distribution rather than directly enforcing semantic validity. This leaves uncertainty about whether AAD learns meaningful within-class variation or simply useful conditioning noise. A comparison with matched fixed-noise perturbations and an analysis of semantic drift would help establish the claimed mechanism.
  2. The main comparisons do not clearly separate the contribution of AAVS from the visual representation. Section 4.2 uses pretrained ViT features and additional fine-tuning on CUB, while Table 1 does not establish that competing generative methods use the same features and training protocol. The component ablations are useful, but cannot resolve this uncertainty in the state-of-the-art comparison. This matters because the reported gains may partly reflect the backbone rather than the proposed semantic-condition learning.
Questions for authors
  1. Equation (4) calibrates attribute variances but leaves learned mean shifts unchanged. What prevents highly discriminative attributes from undergoing class-changing mean shifts? Please clarify how the KL constraint supports semantic preservation.
  2. Do any existing controls distinguish learned semantic variation from ordinary conditioning noise, such as fixed Gaussian perturbations with comparable magnitude? This would help determine whether the gains depend on adaptive attribute distributions rather than noise injection alone.
  3. Were the generative baselines in Table 1 evaluated using the same ViT features and CUB fine-tuning protocol as AAVS, or are their results taken from the original papers? Please identify which comparisons use matched visual representations.
  4. What validation procedure was used to select hyperparameters and fine-tune the CUB encoder? Please clarify how unseen test classes were excluded from model selection.
9 comments
AuthorsOpenReview ↗

Response to Reviewer cN1U [Part I]

Dear Reviewer cN1U,

Thank you for your thoughtful and detailed review. We appreciate your recognition of AAD’s ability to generate diverse conditions for unseen classes from their attributes alone, as well as your acknowledgment of our motivation to address both intra-class diversity and inter-class structural inconsistency.

We understand that your main concerns are: (1) whether the attribute variations learned from AAD preserve class semantics; (2) whether the learned adaptive attribute distributions offer benefits beyond fixed-noise perturbations; (3) whether the reported gains stem from AAVS or the visual representations; (4) how hyperparameters and the CUB encoder were selected without using unseen test classes. We address these concerns below.

W1: "The learned variations are not sufficiently shown to preserve class semantics. ...”

and

Q1: "...What prevents highly discriminative attributes from undergoing class-changing mean shifts? ..."

R1: Thank you for raising this important question, which highlights an ambiguity in our formulation.

(1) Discriminability-guided Variation Calibration.

In our implementation, discriminability calibration is applied to the entire sampled attribute variation, including both its mean and standard deviation. Accordingly, Equation (5) of the paper (p. 5) should be corrected as follows:

\tilde{\sigma}_c^j=(1-\bar d_j)\sigma_c^j.$$ Consequently, the attribute variation after discriminability calibration is sampled as: $\Delta \tilde{a}_c^j\sim\mathcal{N}(\tilde{\mu}_c^j,(\tilde{\sigma}_c^j)^2).$ Thus, variations in highly discriminative attributes are more strongly constrained, reducing the likelihood of mean shifts that could alter class semantics. **Furthermore, we conduct additional analyses to examine whether AAD preserves semantic consistency.** **(2) Semantic Consistency Regularization.** AAD is learned under the generation objective and regularized by the semantic consistency loss $\mathcal{L}_{SC}$. For attribute dimension $j$, the KL term is $$D_{\mathrm{KL}}\left(\mathcal N(\mu_c^j,(\sigma_c^j)^2)\,\|\,\mathcal N(0,1)\right)=\frac{1}{2}\left[(\mu_c^j)^2+(\sigma_c^j)^2-1-\log(\sigma_c^j)^2\right].$$ In particular, the quadratic term $(\mu_c^j)^2$ **penalizes large mean shifts**, while $\lambda$ further controls the magnitude of the variations. **Together, these constraints help prevent excessive deviations from the original class attributes.** Figure 8 of the paper (p. 17) presents the learned $\mu_c$ values across attribute dimensions on AWA2 and CUB. These values are generally centered around zero for both seen and unseen classes, **showing no obvious mean shifts**. However, this observation alone is insufficient to rule out semantic drift. We therefore further evaluate the semantic consistency of the sampled attributes. **(3) Empirical Evidence of Semantic Consistency.** For each class, we sample 256 attributes from its learned distribution and compare them with the original class attributes. The results demonstrate that **the sampled attributes maintain strong semantic consistency with the original class attributes for both seen and unseen classes**, as shown in **Table R1**. **Table R1. Semantic consistency on CUB and AWA2. Values of each metric are reported as estimates with 95% confidence intervals.** | Metric | CUB | | AWA2 | | | -------- | ----------- | ----------- | -------- | ------------ | | | Seen | Unseen | Seen | Unseen | | Spearman's $\rho$ | 0.994 (0.994, 0.995) | 0.994 (0.993, 0.995) | 0.999 (0.998, 0.999) | 0.996 (0.994, 0.999) | | Retention@5 | 0.883 (0.870, 0.894) | 0.885 (0.867, 0.903) | 0.959 (0.947, 0.971) | 0.945 (0.913, 0.974) | | $\mathrm{Retention}_{\mathrm{proto}}$ | 1.000 (1.000, 1.000) | 1.000 (1.000, 1.000) | 1.000 (1.000, 1.000) | 1.000 (1.000, 1.000) | As shown in **Table R1**, Spearman’s $\rho$ is at least 0.994, indicating that the global structure of inter-class similarities is largely preserved after sampling. Retention@5 ranges from 0.883 to 0.959, showing that most local class neighborhoods are retained, although some neighbors change. Moreover, $\mathrm{Retention}_{\mathrm{proto}}$ is 1.000 in every setting: **each sampled attribute is closest to its source class’s original prototype.** Together, these results provide empirical evidence that the sampled variations largely preserve class identity and inter-class structure while introducing attribute diversity. **Revision Location**: Revisions appear in **Equation (5) and corresponding discussion (p. 5)**, **Appendix A.5 (pp. 15-17)**, **Figure 8 (p. 17)**, and **Table 6 (p. 18)**. > ***Please see Part II for the remaining response.***
AuthorsOpenReview ↗

Response to Reviewer cN1U [Part II]

Dear Reviewer cN1U,

W2: "The main comparisons do not clearly separate the contribution of AAVS from the visual representation. ..."

and

Q3: "...Please identify which comparisons use matched visual representations."

R2: Thank you for raising this important concern.

(1) Matched Visual Representation Protocol.

To clarify the comparison protocol, we reproduce the baselines using publicly available implementations, with the same visual features and training protocol as AAVS. Specifically, the reproduced methods are marked with \textdaggerdbl\textdaggerdbl in Table 1 of the paper (p. 6). Their results are therefore directly comparable to those of AAVS under matched visual features.

For methods whose implementations are unavailable or cannot be reliably reproduced, we retain the results reported in their original papers. These methods are included for broader contextual comparison but are not considered strictly matched comparisons.

(2) Results under Matched Visual Representations.

Under matched visual representations, AAVS achieves the highest average H of 72.4%, outperforming the strongest reproduced baseline, ViFR, by 4.1%. The corresponding portion of the revised Table 1 is presented in Table R2.

Table R2. Comparison with state-of-the-art methods on AWA2, CUB, and SUN. Acc, U, S, H and Avg. H are reported (%). \textdaggerdbl\textdaggerdbl indicates self-collected methods using our visual features. Results of other methods are taken from their original papers.

AWA2CUBSUN
MethodVenueAccUSHAccUSHAccUSHAvg. H
Embedding-based\textit{Embedding-based}
SVIPICCV'2569.865.487.774.979.872.178.175.071.653.748.050.766.9
PSVMA+TPAMI'2579.274.286.479.878.871.877.874.674.561.549.454.869.7
SAGEAAAI'2683.778.187.282.482.175.177.576.377.263.752.257.472.0
Generative-based\textit{Generative-based}
f-VAEGAN$\textdaggerdbl$CVPR'1964.162.468.765.477.166.878.772.373.757.450.053.563.7
TF-VAEGAN$\textdaggerdbl$ECCV'2072.864.485.473.476.564.880.371.774.753.456.955.166.7
FREE$\textdaggerdbl$ICCV'2170.458.789.170.877.068.180.373.674.959.451.655.366.6
DMLACCV'24--62.282.370.9--57.181.667.2--39.652.745.961.3
VADSCVPR'2482.575.483.679.386.874.174.674.376.364.649.055.769.8
ViFR$\textdaggerdbl$IJCV'2580.066.983.374.281.470.081.475.376.959.452.155.568.3
AAVS$\textdaggerdbl$ (Ours)--85.482.185.083.579.274.281.077.475.858.954.056.372.4

Our AAVS also consistently outperforms the reproduced generative methods in terms of H across all three datasets, supporting the effectiveness of our proposed semantic condition learning.

Revision Location: Revisions appear in Table 1 (p. 6).

Please see Part III for the remaining response.

AuthorsOpenReview ↗

Response to Reviewer cN1U [Part III]

Dear Reviewer cN1U,

Q2: "Do any existing controls distinguish learned semantic variation from ordinary conditioning noise, such as fixed Gaussian perturbations with comparable magnitude? ..."

R3: Thank you for raising this insightful question. To distinguish learned semantic variations from ordinary conditioning noise, we conduct additional experiments using two types of perturbations on class attribute conditions: fixed noise perturbations and learned adaptive attribute distributions.

Our experiments reveal two key findings:

  • (1) Moderate perturbations can improve semantic diversity. Properly controlled noise perturbations introduce beneficial semantic diversity and improve performance, whereas excessive perturbations may cause semantic drift and degrade performance.
  • (2) Learned adaptive attribute distributions are more effective than fixed noise. They consistently outperform fixed Gaussian perturbations across all three datasets. Combining them with magnitude scaling and discriminability calibration further improves performance.

The results are summarized in Table R3.

Table R3. Comparison of attribute perturbations. H and Avg. H are reported (%). ϵ\epsilon denotes fixed Gaussian noise, λ\lambda controls perturbation magnitude, Δac\Delta a_c is the learned attribute variation, and Δa~c\Delta\widetilde{a}_c is its discriminability-calibrated version.

Attribute ConditionAWA2CUBSUN
HHHAvg. H
Class Attributeaca_c65.472.353.563.7
Fixed Noiseac+ϵa_c+\epsilon65.165.648.559.7 (↓4.0)
Perturbationsac+λ⋅ϵa_c+\lambda \cdot \epsilon73.273.553.066.6 (↑2.9)
Adaptive Attributeac+Δaca_c+\Delta a_c74.574.655.168.1 (↑4.4)
Distributionsac+λ⋅Δaca_c+\lambda \cdot \Delta a_c81.976.055.971.3 (↑7.6)
ac+λ⋅Δa~ca_c+\lambda \cdot \Delta\widetilde{a}_c83.577.456.372.4 (↑8.7)

(1) Fixed Noise Perturbations.

As shown in Table R3, directly adding fixed Gaussian noise ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,\mathbf{I}) reduces the average H from 63.7% to 59.7%. Introducing λ\lambda to control the magnitude of perturbations raises it to 66.6%, outperforming the original class attributes.

This indicates that appropriate perturbations can introduce beneficial semantic diversity, highlighting the importance of controlling perturbation magnitude. However, unconstrained noise may lead to semantic drift.

(2) Adaptive Attribute Distributions.

In comparison, the learned adaptive attribute distributions achieve an average H of 68.1%, outperforming fixed-noise perturbations on all three datasets. This demonstrates that learned semantic variations provide more effective conditioning than ordinary Gaussian noise.

Applying λ\lambda to control perturbation magnitude further increases the average H to 71.3%, while additional discriminability-guided calibration yields the best result of 72.4%. These results indicate that AAD's gains do not merely come from noise injection. Instead, AAD benefits from learning adaptive attribute distributions, with magnitude control and discriminability-aware calibration further enhancing their effectiveness.

Revision Location: Revisions appear in Section 4.6 (p. 9) and Table 3 (p. 9).

Please see Part IV for the remaining response.

Reviewer_Dxy4OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AAVS (Adaptive Attribute Distribution and Visual Structure Alignment), a unified semantic condition learning framework for ZSL. AAVS introduces:

  1. Adaptive Attribute Distribution (AAD): Learns dimension-adaptive attribute distributions calibrated by inter-class discriminability to inject transferable semantic diversity without requiring unseen visual features.

  2. Visual Structure Alignment (VSA): Projects attributes into the visual space and aligns them with refined visual prototypes (suppressing dominant common directions via SVD) to maintain visual-structural consistency.

Extensive evaluations across three benchmarks (AWA2, CUB, and SUN) show that AAVS achieves competitive results, yielding the highest average harmonic mean (HH) among compared methods.

Critical strengths
  1. Well-Motivated Problem Formulation: Identifying and explicitly targeting both intra-class semantic diversity and inter-class structural consistency tackles core failure modes in generative ZSL.

  2. Transferable Diversity Mechanism (AAD): Unlike existing visual-guided semantic enhancements that fail to generalize to unseen classes due to missing visual samples, AAD learns attribute-adaptive variation patterns purely from the shared attribute space, making it genuinely inductive to unseen classes.

  3. Principled Visual Structure Alignment (VSA): The use of SVD-based prototype refinement to suppress class-shared common directions before alignment prevents similarity compression and yields cleaner visual supervision.

  4. Thorough Evaluation and Ablations: The paper provides comprehensive ablations, qualitative visualizations and sensitivity analyses, validating the design choices.

Critical weaknesses
  1. Dependency on Hyperparameters across Diverse Datasets: The framework relies heavily on manual hyperparameter tuning—specifically the variation magnitude λ\lambda, loss weights λSC\lambda_{SC} and λPA\lambda_{PA}, and the prototype refinement strength α\alpha—which vary significantly across datasets (e.g., λ\lambda ranges from 0.010.01 to 0.50.5 across AWA2, CUB, and SUN). Discussing a heuristic or automated strategy for setting these parameters would improve the method's practical scalability.

  2. Computational and Optimization Overhead: AAVS introduces multiple sub-modules, including an Adaptive Attribute Encoder (E+E^+), a Structure Alignment Encoder (E∗E^*), SVD-based prototype refinement, and multi-objective optimization (Ltotal\mathcal{L}_{total}). Providing an explicit runtime/training-time complexity analysis or comparison with baseline generative models (like raw f-VAEGAN) would clarify the training overhead.

  3. Clarity on Discriminability Measure: The discriminability metric djd_j in Eq. (3) computes variance across seen classes. It remains a slight risk that seen-class discriminability might not faithfully reflect unseen-class discriminability if the attribute distribution shifts drastically between seen and unseen splits. A brief discussion on this assumption would strengthen the text.

Questions for authors

none

3 comments
AuthorsOpenReview ↗

Response to Reviewer Dxy4 [Part I]

Dear Reviewer Dxy4,

Thank you for your careful and constructive review. We appreciate your recognition of our motivation to address two important challenges: intra-class diversity and inter-class structural consistency—as well as your acknowledgment of the contributions of AAD and VSA and the supporting experimental evidence.

We understand that your main concerns are: (1) whether the hyperparameters can be selected adaptively; (2) how much training overhead the AAVS introduces; (3) whether the attribute discriminability remains consistent across seen and unseen classes. We provide detailed responses to these concerns below.

W1: "Discussing a heuristic or automated strategy for setting these parameters.”

R1: We explore adaptive strategies for selecting the variation magnitude λ\lambda and the prototype refinement strength α\alpha. These strategies achieve performance comparable to manual tuning and even improve H on SUN from 56.3% to 56.6%, as shown in Table R1 and Table R2.

Table R1. Adaptive selection of the attribute variation magnitude λ\lambda. Value denotes the selected parameter value; Acc and H (%) are reported.

AWA2CUBSUN
ValueAccHValueAccHValueAccHAvg. H
λ\lambda0.05085.483.50.10079.277.40.01075.856.372.4
λadaptive\lambda_{\mathit{adaptive}}0.08485.982.30.05278.576.30.03176.856.671.7
λadaptived\lambda^d_{\mathit{adaptive}}0.10384.981.90.05778.876.60.03476.856.671.7

Table R2. Adaptive selection of the prototype refinement strength α\alpha. Value denotes the selected parameter value; Acc and H (%) are reported.

AWA2CUBSUN
ValueAccHValueAccHValueAccHAvg. H
α\alpha0.585.483.50.579.277.40.575.856.372.4
αadaptive\alpha_{adaptive}0.685.083.00.679.077.20.675.856.372.2

Figure 5 (p. 10) and Figure 6 (p. 16) of the paper analyze the effects of λ\lambda and α\alpha, respectively. Figure 7 (p.16) also shows that performance remains stable across different values of λSC\lambda_{\mathrm{SC}} and λPA\lambda_{\mathrm{PA}}, so we use fixed values for them across datasets. We next describe our adaptive strategies in detail:

(1) Adaptive selection of the variation magnitude λ\lambda. λ\lambda controls the attribute variation magnitude in AAD, i.e., a~cj=acj+λ⋅Δacj.\tilde{a}^j_c = a_c^j + \lambda \cdot \Delta a_c^j. We explore two adaptive strategies based on inter-class semantic distances of seen-class attributes. For each seen class attribute ac∈RDa_c\in\mathbb R^D, we first compute its nearest-neighbor distance:

rc=min⁡k∈Ys,;k≠c∣ac−ak∣2,r_c=\min_{k\in\mathcal Y^s,;k \neq c}\\|a_c-a_k\\|_2,

and let rq10r_{\mathrm{q10}} be the 10th percentile of these nearest-neighbor distances, with a focus on more crowded classes where perturbations are more likely to cause class confusion. Using ϵ∼N(0,ID)\epsilon\sim\mathcal N(\mathbf 0,\mathbf I_D) as a reference residual, whose root-mean-square norm is D\sqrt D, we set

λadaptive=rq102D.\lambda_{\mathit{adaptive}}=\frac{r_{\mathrm{q10}}}{2\sqrt D}.

The half-distance sets a geometric reference: smaller displacements preserve aca_c as the nearest prototype under Euclidean distance. We further account for AAD’s discriminability calibration. With wj=1−djˉw_j=1-\bar{d_j}, the reference residual w⊙ϵw\odot\epsilon has root-mean-square norm ∑j=1Dwj2\sqrt{\sum_{j=1}^{D}w_j^2}, giving

λadaptived=rq102∑j=1Dwj2.\lambda_{\mathit{adaptive}}^{d} =\frac{r_{\mathrm{q10}}}{2\sqrt{\sum_{j=1}^{D}w_j^2}}.

This aligns the selected variation magnitude with the calibrated attribute scales.

(2) Adaptive selection of the prototype refinement strength α\alpha. α\alpha controls the prototype refinement strength, i.e., pc^=pc−α(pc⊤pcom)pcom.\hat{p_c}=p_c-\alpha(p_c^\top p_{\mathrm{com}})p_{\mathrm{com}}. We develop an adaptive strategy based on the preservation of inter-class structures among visual prototypes. Specifically, we measure globally and locally structural preservation using Spearman's ρ\rho (ρ(α)\rho(\alpha)) and Retention@5 (R5(α)R_5(\alpha)), respectively. We discretize α\alpha into 0,0.1,…,1.0\\{0,0.1,\ldots,1.0\\} and select

αadaptive=max⁡α∈0,0.1,…,1.0:ρ(α)≥0.99,  R5(α)≥0.95.\alpha_{\mathit{adaptive}}=\max\\{\alpha\in\\{0,0.1,\ldots,1.0\\}:\rho(\alpha)\geq0.99\\,\;R_5(\alpha)\geq0.95\\}.

This rule selects the strongest refinement that improves inter-class separation while retaining the original class structures.

Revision Location: Revisions appear in Section 4.7 (pp. 9-10), Appendices A.2–A.4 (pp. 14–15), and Figures 5 (p. 10), 6 (p. 16), and 7 (p. 16).

Please see Part II for the remaining response.

AuthorsOpenReview ↗

Response to Reviewer Dxy4 [Part II]

Dear Reviewer Dxy4,

W2: "Providing an explicit runtime/training-time complexity analysis or comparison with baseline generative models (like raw f-VAEGAN)...”

R2: Thank you for requesting a more explicit cost comparison. We measured f-VAEGAN and f-VAEGAN + AAVS on AWA2 using the same batch size and generator/discriminator update schedule, as shown in Table R3.

Table R3. Parameter count and runtime comparison.

ModelParameters (M)Attribute discriminability (s)SVD (s)Prototype refinement(s)Training time (s/epoch)Feature synthesis (s/epoch)Classification evaluation (s/epoch)
f-VAEGAN11.03———13.910.0150.894
f-VAEGAN + AAVS17.780.00370.1630.57229.170.0430.885

1) Parameter count. AAVS increases the number of trainable parameters from 11.03M to 17.78M, an increase of 6.75M (approximately 61%). Most of this increase comes from widening the generator and discriminator condition inputs to accommodate the 768-dimensional VSA embedding: their input layers add approximately 3.15M and 3.15M parameters, respectively. The AAD attribute mappings and VSA embedding network together add approximately 0.45M parameters. Thus, the additional modules themselves account for a small fraction of the parameter increase.

2) Runtime. Training time rises from 13.91 to 29.17 seconds per epoch. The expanded condition dimensions mainly increase the computational cost of the generator and discriminator. Discriminability computation and prototype refinement are performed only once before training, taking 0.0037 and 0.572 seconds, respectively. Feature synthesis, including semantic condition construction by AAD and VSA, takes only 0.043 seconds, indicating that the additional generation cost is modest.

W3: "It remains a slight risk that seen-class discriminability might not faithfully reflect unseen-class discriminability...”

R3: Thank you for highlighting this assumption. To assess whether attribute discriminability estimated from seen classes generalizes to unseen classes, we compute djd_j in Equation (4) (p. 5) separately for seen classes and unseen classes and compare the mean absolute error (MAE) between the two sets of estimates in Table R4.

Table R4. Seen–unseen differences in attribute discriminability.

NseenN_{seen}NunseenN_{unseen}NattributeN_{attribute}MAE
AWA24010850.09
CUB150503120.03
SUN645721020.01

These results show that attribute discriminability estimates for seen and unseen classes are similar across the three datasets, providing empirical support for using seen-class discriminability to calibrate variations for unseen classes. Figure 9 of the paper (p. 18) further shows similar dimension-wise patterns between seen and unseen classes on SUN.

This assumption builds on the shared attribute space underlying the standard ZSL formulation, where seen and unseen classes are represented using a common semantic vocabulary, enabling knowledge transfer through shared semantic attributes [R1, R2]. However, a shared semantic space does not guarantee identical attribute discriminability. Substantial shifts in attribute distributions may make seen-class discriminability less representative of unseen classes, thereby weakening the effectiveness of the calibration.

Revision Location: Revisions appear in Appendix A.6 (p. 17), Figure 9 (p. 18), and Table 7 (p. 19).

We sincerely thank you for your constructive feedback, which has helped improve our work. We welcome any further questions or discussion.

Reference:

[R1] C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to Detect Unseen Object Classes by Between-Class Attribute Transfer. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 951–958, 2009.

[R2] Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata. Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 9, pp. 2251–2265, 2019.

Dear Reviewer Dxy4,

Thanks for your time and effort in reviewing this paper. Your comments and suggestions are valuable to improve the quality of this paper. As the author-reviewer discussion period ends in about 3 days, we kindly request that you read the rebuttal and confirm whether your concerns are addressed. Thanks again for your time and help in improving the quality of this paper.

Regards from the authors.

Comments on the paper

1 comment

Author Final Remarks:

We thank all reviewers (Dxy4, cN1U, gzPV) for their careful reviews and constructive feedback.

Overall, the reviewers acknowledged the following strengths:

  1. Clear motivation. Addressing insufficient intra-class semantic diversity and semantic–visual structural inconsistency in generative ZSL. (Dxy4, cN1U, gzPV)

  2. Transferability to unseen classes. Constructing diverse conditions from attributes alone, without unseen-class images or instance-level annotations. (Dxy4, cN1U, gzPV)

  3. Effective visual structure supervision. Refining prototypes to alleviate similarity compression and improve alignment targets. (Dxy4)

  4. Extensive evaluation. Supporting effectiveness through benchmark comparisons, ablations, qualitative analyses, and sensitivity studies. (Dxy4, gzPV)

Meanwhile, following the reviewers’ suggestions, our main responses are as follows:

Responses to Dxy4

  1. Adaptive hyperparameter selection. We introduced seen-only strategies for selecting variation magnitude and refinement strength, achieving performance comparable to manual tuning while controlling excessive-refinement effects. (Tables R1–R2, Dxy4)

  2. Computational overhead. We reported parameter counts and runtime comparisons, explaining the additional costs and one-time preprocessing. (Table R3, Dxy4)

  3. Discriminability transferability. We compared seen- and unseen-class attribute discriminability, providing empirical support for transferring the calibration. (Table R4, Dxy4)

Responses to cN1U

  1. Semantic preservation. We clarified that calibration constrains both mean and standard deviation, and added prototype-retention and structural analyses supporting semantic consistency. (Table R1, cN1U)

  2. Fair comparisons. We reproduced baselines using identical visual features, confirming AAVS’s advantage under matched visual representations. (Initial Table R2; follow-up Table R1, cN1U)

  3. Benefits beyond noise injection. We compared fixed-noise and learned perturbations, both with and without VSA, validating the effectiveness of the learned distributions. (Initial Table R3; follow-up Tables R2–R3, cN1U)

  4. Model selection and stability. We clarified the seen-only CUB fine-tuning protocol and baseline settings, and reran experiments with adaptive parameters and multiple random seeds, showing that the performance advantage persists under the strict ZSL protocol. (Follow-up Table R1 and discussion, cN1U)

Responses to gzPV

  1. Methodological contribution. We clarified how AAVS differs from existing visually guided prototype enhancement methods, highlighting the novelty of jointly modeling semantic diversity and visual structural consistency for transferable condition learning. (Part I, gzPV)

  2. Justification for discriminability-guided calibration. We provided empirical evidence and clarified its role in preserving class semantics. Comparisons of negative-correlation, uniform, and positive-correlation calibration support our strategy within the proposed learnable perturbation framework. (Table R1, gzPV)

  3. Diffusion-based generation. We added DDGAN evaluations and component ablations, demonstrating the effectiveness of our method and its generalizability across generative frameworks. (Table R2, gzPV)

  4. Prototype refinement. We discussed its potential risks, introduced adaptive refinement-strength selection, and evaluated performance without refinement, confirming its auxiliary role in VSA. (Table R2, gzPV; manuscript Table 2)

Overall, we have addressed most reviewer concerns and believe these additional experiments and clarifications further strengthen our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.