acceptodds
Withdrawn from ICLR 2027

Geometry-Conditioned Adaptation for CLIP-Based Cross-Domain Facial Beauty Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Facial beauty prediction (FBP) estimates ratings from particular annotator populations, yet imaging conditions and rating protocols vary across datasets. Landmark geometry encodes facial proportions and anatomical relationships without pixel-level texture or illumination, offering a complementary structural cue that may be less sensitive to changes in image appearance. This could improve robustness to domain shifts across datasets. Source-tuned CLIP provides transferable, language-aligned representations, but directly fusing image-derived landmarks may amplify noisy or redundant features and perturb the image–text alignment used for prompt-based scoring. We propose a geometry-conditioned tangent adapter that keeps the CLIP image embedding as the prediction anchor. A separately trained landmark–contour encoder supplies structural features that condition image-dependent corrections. With both encoders frozen, the adapter projects its correction onto the tangent plane, bounds its norm, and re-normalizes the image embedding before applying the original text prompts and scoring rule. Source-training statistics calibrate adapter inputs, and zero initialization recovers the frozen scorer before adapter training. We evaluate on LiveBeauty, MEBeauty, and SCUT-FBP5500 under three within-dataset and two cross-dataset protocols without target-domain adaptation. The full model improves all three correlation metrics over the reported state-of-the-art FPEM results across all five settings. For LiveBeauty-to-SCUT-FBP5500 transfer, the full system's relative SRCC, PLCC, and KRCC gains over FPEM are , , and , respectively. In the same setting, PLCC rises from for our source-tuned CLIP base to for the full model ( relative to that base). These results support bounded geometry-conditioned adaptation as a useful design for this CLIP-based FBP system without target-domain fine-tuning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
facial attractiveness prediction, vision--language models, facial landmark graphs, geometry-conditioned adaptation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uXbgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cross-dataset generalization in facial beauty assessment. It uses a frozen CLIP-based scorer. It adds a small correction module based on facial landmarks. The module starts with zero correction. It also limits the correction direction and magnitude. Experiments cover three datasets and the method outperforms FPEM on all three metrics. It also beats two methods that directly concatenate landmark features.

Critical strengths

The method only adjusts the image representation. It keeps the scoring rule unchanged. Direction and magnitude constraints prevent large changes.

The cross-dataset evaluation is strict, and the authors acknowledge key limitations, while they also reveal the main weaknesses of the paper.

Critical weaknesses

The key ablation missing the experiment that image correction without landmarks. The correction module already includes an image-only branch. The same constraints apply to this branch. The gains could come from constrained correction alone, not landmark information. The authors acknowledge this missing comparison. However, it is essential to validate the paper’s core claim of geometry-conditioned correction.

Other ablations are also missing:

  • removing the direction and magnitude constraints
  • testing each landmark feature

Most of the reported improvement over FPEM comes from the baseline difference, not the proposed correction.

Questions for authors

see Critical Weaknesses

Reviewer_2fsEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a geometry-conditioned adaptation framework for cross-domain facial beauty prediction using CLIP. The method combines a source-tuned CLIP image encoder with a dual-stream graph encoder that extracts facial landmark and contour representations. A lightweight adapter uses these geometric features to condition bounded corrections to the CLIP image embedding, while preserving the original text-based attractiveness scoring mechanism. Experiments are conducted on LiveBeauty, MEBeauty, and SCUT-FBP5500 under three within-dataset and two cross-dataset evaluation settings without target-domain adaptation. The proposed method reports improvements over existing approaches, including FPEM, across SRCC, PLCC, and KRCC. Additional experiments compare geometry-only prediction, feature fusion, and the proposed adaptation strategy.

Critical strengths
  • The proposed adapter provides a structured way to incorporate facial geometry while limiting changes to CLIP representations.
  • The method improves SRCC, PLCC, and KRCC across both cross-dataset evaluations without target-domain adaptation.
  • Combining landmarks, contours, and anatomical regions provides complementary structural information alongside CLIP image features.
  • The final adapter requires only about 77.8K trainable parameters while keeping both encoders frozen.
  • The architecture, mathematical formulation, and training procedure are generally clear and well explained.
Critical weaknesses
  1. The authors use the source-domain test partition for checkpoint selection, which compromises the reliability of the reported within-dataset results.
  2. The contributions of geometry conditioning, tangent projection, and norm bounding are not evaluated independently.
  3. Without a matched image-only adapter baseline, it is difficult to determine whether improvements come specifically from geometric information or additional adaptation.
  4. The method relies on facial landmarks, but robustness to detection errors, occlusion, and extreme head poses is not evaluated.
  5. Results are reported without confidence intervals or multiple-seed experiments, making it difficult to assess statistical significance.
  6. Facial attractiveness judgments are subjective and population-dependent, but performance across demographic groups is not investigated.
Questions for authors
  1. Why is the test partition used for checkpoint selection? Could the authors report results using a separate validation set?
  2. How does the full method compare with a matched image-only adapter without geometry conditioning?
  3. What are the individual contributions of tangent projection, norm bounding, and the landmark, contour, and region branches?
  4. How robust is the method to noisy or missing landmarks, facial occlusions, and pose variations?
  5. Could the authors provide confidence intervals or multiple-seed results to establish statistical significance?
  6. How does performance vary across demographic groups, particularly gender and ethnicity?
  7. How much of the cross-domain improvement comes from the source-tuned CLIP backbone versus the proposed adapter?
  8. Does the method generalize when training on MEBeauty or SCUT-FBP5500 and transferring to other datasets?
Reviewer_RxpQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on facial beauty prediction, focusing on cross-dataset generalization: training on one dataset without any target domain adaptation, and directly taking it to another dataset for testing. A pedestal is a CLIP that has been fine-tuned on the source dataset. Additionally, a facial key and contour encoder is trained separately to provide geometric features. A small adapter is then trained. It calculates a correction for the CLIP image embedding based on the geometric features.

Critical strengths

The adapter only has 77.8K parameters, and both encoders are frozen, resulting in minimal additional overhead. Table 4 shows that directly concatenating geometric features (Concat MLP) or performing probabilistic fusion can actually impair cross-domain performance. This shows that geometric information needs to be added in a constrained manner, which is of reference significance for this direction. The target domain data is not used throughout the evaluation process, and the settings are clean.

Critical weaknesses
  1. Text prompts use a fixed number of categories, which is essentially equivalent to a fixed 5-category linear classification header. The meaning of "language-aligned" may be more nominal.

  2. One of the core contributions of this paper is cross-domain FBP, but the cross-domain experiment only has two settings, both of which are trained from LiveBeauty. There are no cases where the opposite direction or other datasets are used as the source domain. The idea in cross-domain is that geometric structures are not affected by lighting, filtering, or shooting conditions, but this seems to have not been verified.

  3. Automatic scoring of appearance is inherently sensitive. Fig. 4(a) shows a recognizable face, with low scores such as 1.3 and 1.4 marked next to it, without mentioning whether it was approved. The article states that it is necessary to address differences in population distribution, but it does not analyze groupings by race or gender.

Questions for authors
  1. Would replacing prompt with a randomly initialized or learnable Class 5 linear header significantly reduce cross-domain performance? If they are similar, the motivation of "keeping the text and images aligned" is not very tenable;

  2. Can cross-domain experiments be evaluated by changing the source dataset? Furthermore, whether different datasets, under different conditions such as lighting, filtering, and cropping, will cause deviations in the extraction of geometric information and undermine the stability of the geometric structure requires experimental data to support this.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.