acceptodds
Under review as a conference paper at ICLR 2027

Prism: Class-Consistent Image Generation from Tabular Data

Abstract

Many real-world datasets are tabular. While recent open-domain foundation models condition image generation primarily on text, a common strategy for tabular data is to serialize rows into natural language and condition a text-to-image diffusion model. This clubs linguistically similar classes together even when they are visually distinct, giving incorrectly conditioned images. We propose PRISM, a tri-fold model, which maps tables directly to images without using intermediate mapping to text. We train a tabular conditioning encoder that converts each attribute into a dedicated cross-attention token for the diffusion model. We then fine-tune the model toward the target class, away from confusable alternatives using a GAN-style discriminative loss on generated images. At inference, we steer towards the correct class via classifier-free guidance maximizing target class alignment without any textual input. We evaluate PRISM on three diverse applications: medical imaging, fine-grained bird classification, and plant disease recognition. Evaluations reveal improvements of up to 116% in class consistency over the best language-based baseline. On the medical dataset, a board-certified dermatologist evaluated the generated images and judged most to be visually indistinguishable from real samples. https://anonymous.4open.science/r/PRISM-8FC2

Primary area:
generative models
Keywords:
Diffusion models, Conditional generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hyHNOpenReview ↗
Rating
4(+1)
Confidence
1
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces PRISM, a three-stage framework for class-consistent image generation directly from structured tabular metadata. It targets the observation that serializing tables into text makes linguistically similar but visually distinct classes collapse into indistinguishable conditioning vectors. Stage 1 trains a tabular conditioning encoder that maps each attribute to a dedicated cross-attention token, replacing CLIP embeddings, jointly with LoRA adapters on a frozen Stable Diffusion model under the standard denoising objective. Stage 2 freezes the encoder and applies a GAN-style similarity-aware separation loss, pulling generated images toward their target class prototype and away from metadata-similar confusable classes. Stage 3 uses classifier-free guidance at inference to strengthen class alignment. PRISM is evaluated on HAM10000, CUB-200-2011, and PlantVillage against frozen CLIP, fine-tuned CLIP, specialized CLIP, and non-language baselines. It achieves class consistency of 0.73, 0.96, and 0.93, improving over the best language baseline by up to 116%, with competitive FID and CLIPIQA.

Critical strengths

++The paper is well written.

++ The focus of this work is relatively novel, that is, categories that are linguistically similar under the conditions of text-serialized tables will collapse into indistinguishable condition vectors. Correspondingly, the subsequent fault analysis is relatively sufficient, and the corresponding improvement in experimental results is relatively obvious.

++The motivation of the model design is clear and definite. The subsequent experimental results further verify the effectiveness of this motivation.

Critical weaknesses

--This work is interesting, but I am not a researcher in this field. Therefore, I am particularly concerned about the ‌generalization performance‌ of this work, the ‌limitations of the current study‌, and its ‌future research directions‌. I would kindly request the authors to provide supplementary explanations and responses to these points.

Questions for authors

See Weakness

Reviewer_YECKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a framework for generating images directly from structured tabular data. The method consists of three components:

  1. Encoding each tabular attribute as an individual cross-attention token;
  2. Introducing a visual prototype-based discriminative loss that encourages generated images to approach the target class and move away from confusable classes;
  3. Applying classifier-free guidance at inference time to improve class consistency.

Experiments on skin lesion, bird species, and plant disease datasets show strong class consistency according to automated classifiers.

Critical strengths
  1. The problem is practically relevant, as it avoids converting structured data into text prompts.
  2. The method is clearly presented, and the attribute-wise tokens provide some interpretability and controllability.
  3. The experiments cover multiple domains and use two independent evaluators, DINOv2 and EfficientNet.
  4. The appendix and qualitative results are comprehensive, and the code is provided.
Critical weaknesses
  1. The source of the performance improvement is unclear. In Stage 2, PRISM uses class labels and real-image prototypes extracted by ConvNeXt to provide additional discriminative supervision. However, the main CLIP-based baselines do not appear to receive the same loss. Therefore, the improvement may primarily result from this additional class supervision rather than the proposed tabular encoding. The paper should:
    • Report the performance of PRISM using only Stage 1;
    • Apply the same Stage 2 loss to the main baselines;
    • Separately ablate the attribute-wise tokens, prototype loss, and hard-negative weighting.
  2. A sufficiently strong and fairly matched class-conditional baseline is missing. In these datasets, the most important tabular field is often the species or disease label. The task is therefore closely related to standard class-conditional generation. Existing non-language baselines, such as ACGAN and FineDiffusion, differ substantially from PRISM in architecture and model capacity. A stronger baseline using learned class embeddings with the same Stable Diffusion/LoRA architecture and training budget is needed.
  3. Classifier-based evaluation may overestimate semantic correctness. Stage 2 directly optimizes features from a visual classifier, while the main evaluation metrics are also produced by visual classifiers. Using different backbones partially alleviates this concern, but these classifiers may still rely on similar discriminative artifacts. In particular, on HAM10000, the dermatologist’s diagnostic accuracy is only 35.7% for generated images, compared with 57.1% for real images. This gap appears inconsistent with the relatively high automated class-consistency score.
  4. The generality of the “tabular-to-image” claim is not fully supported. The three datasets contain relatively small sets of attributes that are strongly correlated with the class labels. The paper does not sufficiently evaluate high-dimensional tables, high-cardinality attributes, missing or noisy values, or unseen attribute combinations. The current evidence more directly supports structured-metadata-conditioned generation than general tabular-to-image generation.
Questions for authors
  1. If all baselines are trained with the same prototype-based discriminative loss, does PRISM still significantly outperform them?
  2. How much improvement does attribute-wise encoding provide over a standard class-conditional Stable Diffusion model with learned class embeddings?
  3. Does HAM10000 use patient- or lesion-disjoint data splits? Were near-duplicate images and background leakage considered for PlantVillage?
  4. Can the model generalize to attribute combinations or categorical values not observed during training?
1 comment

Dear Reviewer,

As the discussion period is nearing its end, we wanted to kindly follow up on our rebuttal. We would greatly appreciate any feedback you may have on our responses and would be happy to clarify any remaining concerns.

Thank you again for your time and consideration!

Reviewer_2yXLOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PRISM, a framework for generating class-consistent images directly from tabular metadata without converting it into text. It encodes each attribute as a separate conditioning token and trains the encoder alongside LoRA adapters in a pretrained diffusion model. A second training stage uses a similarity-weighted prototype loss to reduce confusion between classes, followed by classifier-free guidance at inference. Experiments on bird species, plant diseases, and skin lesions show improved class consistency over most evaluated language-based baselines, while maintaining competitive image quality.

Critical strengths
  1. Generating images directly from structured metadata is useful for domains such as medical imaging, fine-grained species recognition, and plant disease analysis.
  2. A simple and lightweight design. Per-attribute conditioning tokens provide an intuitive way to represent structured inputs. Training mainly involves the tabular encoder and LoRA adapters, while most pretrained parameters remain frozen.
  3. Strong results across different domains. PRISM substantially improves class consistency over language-based baselines on birds and plant diseases. Additional analyses of diversity, memorization, missing attributes, and alternative evaluators strengthen the evaluation.
Critical weaknesses
  1. Evaluation is limited to an older backbone. All experiments use Stable Diffusion v1.5. It is therefore unclear whether the reported limitations of text conditioning also hold for stronger backbones. Evaluation on a more recent backbone would help establish whether PRISM’s advantages generalize beyond this specific setup.

  2. The benefit of per-attribute tokenization is not fully isolated. The single-token baseline in Appendix F uses only the class label and discards other attributes. This changes both the available information and the encoding structure. A fairer comparison would encode the same complete metadata using either a single vector or separate attribute tokens.

  3. Additional class supervision may explain part of the improvement. PRISM receives prototype-based class supervision during Stage 2, but it is unclear whether the language-based baselines receive the same refinement (this may lead to unfair comparison). Applying this loss to the baselines and reporting class consistency before and after Stage 2 would help establish whether language conditioning is the main bottleneck.

Questions for authors
  1. How much does Stage 2 improve class consistency? Do language-based baselines also benefit from the same prototype-based refinement?

  2. Do PRISM’s gains hold on a more recent backbone? This would help clarify whether the claimed language bottleneck extends beyond SD v1.5.

Comments on the paper

2 comments
Reviewer_hyHNOpenReview ↗

res.

Thank you very much for the authors' response. The concerns I raised have been fully addressed, and I would like to further raise my score accordingly.

Thank you for taking the time to review our response and for reconsidering your score. We sincerely appreciate your feedback and your thoughtful evaluation of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.