acceptodds
Under review as a conference paper at ICLR 2027

COFAT: Correlation-Aware Query Design for Controllable Generation

Abstract

Conditional generative models should not only produce realistic samples but also reliably follow the requests they are given. Evaluating this requires checking four things: whether the requested attribute combination is *feasible*, whether the generated sample is *obedient* to it, and whether it remains *realistic* and *novel*. Existing controllability evaluations score obedience, realism, or novelty largely in isolation and take feasibility for granted. The dominant query design, One-Factor-at-a-Time with non-swept attributes fixed at their marginal means (OFAT-), ignores dependence between attributes. On correlated attribute spaces, it therefore issues requests that rarely or never occur in the data. Generators may then ignore the requested change, undermining the purpose of conditional generation. Our approach offers a new perspective by treating query design as part of controllability evaluation. We introduce a closed-form measure of query infeasibility and COFAT, a family of correlation-aware query designs that set non-swept attributes from their conditional rather than marginal distribution, within a framework that scores feasibility, obedience, realism, and novelty jointly. For a synthetic generator that is perfectly obedient on feasible requests, switching from OFAT- to COFAT raises measured obedience from to : the query, not the generator, can be the bottleneck. Across three tasks (lesion inpainting, lesion insertion, and molecule generation) and against seven baseline designs, COFAT raises obedience (e.g., for lesion intensity) and molecule validity (), reduces copying of training samples (improving the novelty score from to ), and keeps realism comparable. As the target range of the swept attribute is unchanged, these gains come from removing infeasible combinations, not from easing the target. Many reported controllability gaps may therefore be artifacts of query construction, which argues for treating query design as a first-class component of controllability evaluation.

Primary area:
generative models
Keywords:
Conditional Generative Model, Controllability evaluation, Correlated attributes, Obedience, Realism

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FAR3OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors present COFAT, a correlation-aware query design for controllability evaluation that removes the penalization of infeasible requests. When testing a specific attribute, COFAT sets the other attributes to their conditional mean given the training data distribution, instead of their marginal mean. The authors test it on tumor images, molecules, and a synthetic generator, showing that measured obedience and validity rise substantially when only the query is changed, claiming that many controllability gaps may be artifacts of query design rather than model failures.

Critical strengths
  • I think that treating the query as a source of measurement error is an interesting reframing of controllability evaluation. The mathematics of the paper appears solid, and the authors are honest about the limitations of their approach.
  • Experiments are run on 3 real tasks (lesion inpainting, lesion insertion and molecule generation) and compared against seven baseline designs, showing consistent gains over OFAT-µ across domains.
Critical weaknesses
  • The gains of COFAT over OFAT-µ are clear. However, compared to other correlation-aware designs, the results are close, and it is difficult to draw a firm conclusion without significance tests, repeated runs with different seeds, or confidence intervals.
  • The claim that "many reported controllability gaps may be artifacts of query construction" would require re-evaluating existing published controllability results. As it stands, the evidence comes from a narrow, controlled set of experiments and is too limited to support such a general claim.
Questions for authors
  • What is the computational cost of COFAT compared with the other designs? What is the runtime or complexity advantage (if any)? How large is the practical difference, and when should a practitioner prefer COFAT over KNN, whose results are close?
  • COFAT evaluates "change s as the data does". In which applications would rare but valid combinations be needed, and when should OFAT or COFAT be used? Is COFAT meant to replace OFAT or complement it?
1 comment
AuthorsOpenReview ↗

Summary of our actions (FAR3)

We thank the reviewer for finding the reframing of the query as a source of measurement error interesting, the mathematics solid and our discussion of limitations honest, and for noting the consistent gains over OFAT-μ\mu across three real tasks and seven baseline designs. Following the comments we:

  • added significance tests comparing COFAT with every correlation-aware design on all 10 attributes, paired 95% bootstrap confidence intervals for all metrics in all domains, and results over three training seeds (Comment 1; new App. A.1.11, Tables A and A2);
  • narrowed the claim that "many reported controllability gaps may be artifacts of query construction" to what our experiments show, and stated when the effect is expected (Comment 2);
  • measured the cost of building requests and generating outputs for every design, and added a practical guide for choosing between COFAT, KNN and COFAT-N (Question 1; new App. A.1.12, Table B);
  • clarified that COFAT complements OFAT rather than replacing it, and added a test with rare real requests (Question 2).

In short: COFAT's gain over the usual OFAT-μ\mu design is significant on all 10 attributes and in every training seed. Among the correlation-aware designs the differences are small and go both ways, so we do not claim that COFAT always obeys best. Its consistent advantages are a lower bias and practical ones: it needs only the correlation matrix, not the training records themselves, so it works when records are few or cannot be shared (e.g. clinical patient data); it stays well defined at and beyond the edges of the data percentiles; it has no parameter to tune; and it is as cheap as OFAT-μ\mu (Question 1, which also says when KNN is preferable).

A detailed response (response 1/2, response 2/2) and full experimental details (experiment 7, experiment 8) are in the follow-up comments below.

Reviewer_W56NOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a correlation-aware query design method for evaluating controllable generative models (COFAT). The authors argue that conventional evaluation sweeping one attribute at a time (OFAT) may produce attribute outside the data support when the attributes are highly correlated. To address this issue, COFAT adjusts the remaining attributes according to their linear correlations with the swept attribute, thereby minimizing the Mahalanobis radius of the query for a fixed target value. Besides, the authors also propose COFAT-S, a stochastic variant, and COFAT-N, a non-Gaussian variant which operates with the Gaussian copula transformation. The authors also defined four metrics for evaluating the conditional generation, including feasibility metrics, obedience metrics, realism metrics, and novelty metric. The authors evaluate COFAT on three real-data-tasks and one synthetic check. The experimental results show that COFAT improves obedience compared to OFAT in high correlation settings, while generally maintaining comparable realism.

Critical strengths
  1. The limitation of OFAT under correlated attributes is a real. The proposed COFAT provides a simple and intuitive solution by minimizing the Mahalanobis radius.

  2. The proposed controllability framework in Sec 3.3 is comprehensive, considering conditional feasibility, obedience, realism, and novelty jointly.

  3. The experiments cover diverse data-type, including medical image, molecular generation and synthetic data with correlation structure. The baselines also includes seven baseline query designs.

Critical weaknesses
  1. Despite the diverse data-type , the experimental evaluation is insufficient. In Limitations, the authors said only one generative model is evaluated for each task, with a fixed guidance scale. Different models or guidance scales could provide very different results. Moreover, the paper does not report results across multiple training seeds.

  2. COFAT only guarantees the minimization of the covariance-based Mahalanobis radius, rather than actual query infeasibility under the true data distribution. As shown in Table 2, COFAT achieves the minimum Mahalanobis radius ρˉ\bar \rho and zero ρˉc\bar \rho_c, but has a 100% empirical anomaly rate, while COFAT-N and KNN achieve 0%. This suggests that minimizing the Mahalanobis radius does not necessarily lead to queries with better empirical support.

Questions for authors

Please read the weakness.

And there are some typos:

  1. line 1226, a missing reference.

  2. line 53, feasibility been

  3. line 310, Here 2

Reviewer_9kCEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a correlation-aware query design method for evaluating conditional generative models. The authors argue that the existing One-Factor-at-a-Time method can produce unnormal attributes that conflicts with the pattern in the training data. The proposed method instead uses the training correlation matrix to adjust attribute combinations. Experiments on several tasks demonstrate the efficacy of the proposed method.

Critical strengths
  1. The problem is well defined and practically reverent.
  2. The proposed method is reasonable and simple to understand.
Critical weaknesses
  1. The paper defines query infeasibility using the Mahalanobis radius, but this metric measures statistical untypicality rather than actual infeasibility. Because it relies only on the mean and covariance, it cannot explicitly capture complicated structures. Consequently, an unsupported attribute combination may have a small radius, while a rare but valid combination may have a large one.
  2. The paper does not evaluate sensitivity to errors in the correlation matrix estimation, particularly under few-shot data or noisy measurements.
  3. The proposed may under-assess the generator’s ability to produce rare but meaningful combinations.
Questions for authors
  1. Could the authors more rigorously characterize the gap between the second-order statistical optimality and practical query feasibility, and clarify the assumptions under which the former provides meaningful guarantees about the latter?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to all Reviewers and Area Chair

We thank the reviewers for their careful and constructive reviews. They agree that the problem is real and practically relevant (9kCE, W56N), that COFAT is simple and intuitive (9kCE, W56N), that the mathematics is solid and the limitations honestly discussed (FAR3), and that the evaluation covers diverse data types and seven baseline designs (W56N, FAR3).

To address the weaknesses and questions raised, we conducted eight new experiments:

  • Experiment 1 (9kCE, W56N) — ρc\rho_c vs. An.%: We clarify the use of An.% alongside ρ\rho and evaluate how the two relate. They agree for 8 of 10 attributes; where they disagree, An.% flags the problem and COFAT-N fixes it.
  • Experiment 2 (9kCE) — Sensitivity to the estimated correlations: COFAT is robust to limited sample sizes and to noise in its estimated correlation matrix, and outperforms OFAT-μ\mu at every tested noise level.
  • Experiment 3 (9kCE, FAR3) — Rare but real requests: Generators follow rare requests nearly as well as typical ones, but produce less realistic outputs.
  • Experiments 4 and Experiment 5 (W56N, FAR3) — Training seeds and guidance scales: COFAT's gain over OFAT-μ\mu holds across additional training seeds and guidance scales, except for two shape attributes at high guidance.
  • Experiment 6 (W56N) — Second lesion generator: We trained and evaluated a second lesion generator (LeFusion). With LeFusion, COFAT beats OFAT-μ\mu on two of three attributes and ties on the third; LeFusion's unrealistic outputs make this supporting evidence only.
  • Experiment 7 (FAR3) — Statistical significance: We add 95% confidence intervals and Holm-corrected significance tests.
  • Experiment 8 (FAR3) — Computational cost: We report the cost of building requests for each design and of generating their outputs. COFAT is as cheap as OFAT-μ\mu, and building requests is negligible next to generation. We also add a practical guide to choosing a design: COFAT for few or private records and extreme targets; KNN or COFAT-N for curved, nonlinear relations.

Clarifications in the paper: (i) ρ\rho and ρc\rho_c are renamed statistical and conditional atypicality; (ii) COFAT tests typical requests only and complements OFAT; (iii) we state the assumptions under which a typical request is also a possible one; (iv) the Limitations section is extended; (v) all typos are corrected.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.