acceptodds
Under review as a conference paper at ICLR 2027

Depth-Graded Parameter-Efficient Adaptation of Foundation Models for Face Recognition

Abstract

Vision foundation models are an attractive starting point for training face recognition (FR) models, which often require large identity-labelled datasets. Current approaches to use foundation models for FR adapt them with low-rank adapters that use the same rank in every transformer block. We ask *where in the foundation model network needs capacity for FR adaptation*. After probing CLIP ViT-L/14 with high-rank adapters, we find that the rank of the learned adapter grows with model depth, while a uniform rank-16 adapter is saturated in deep blocks and under-used in shallow ones. Motivated by this, we propose *depth-graded rank allocation*, a search-free schedule that increases adapter rank linearly with depth. We pair it with rank-stabilised scaling and instantiate it with low-rank and weight-decomposed adaptation (called DG-LoRA and DG-DoRA, respectively). We study eight ViT-L foundation models in our experiments. We show that depth-graded ranks improve adapted ViT-L over uniform rank adapters with the same budget. Trained on WebFace4M our model with 14M trainable parameters reaches 95.1% TAR on IJB-C (FAR=), surpassing prior parameter-efficient adaptation of foundation models and a ViT-L trained from scratch on the same data.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Face Recognition, Foundation Models, Low-rank Adaptation, Vision Transofrmer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Z929OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is concerned with finetuning vision foundation models for the face recognition (FR) task. Following up from recent work on LoRA finetuning of these models they perform spectral analysis on LoRA learned weights and find that high rank LoRAs are effectively saturated in earlier layers while contributing more to the final layers. This leads to the proposition of increasing LoRA rank linearly by depth. Thei final model achieves SOTA performance on FR.

Critical strengths
  • The proposed approach offers benefits and can be used as a general solution for finetuning vision foundation models for FR.
  • It is sensible and applies existing knowledge, earlier layers work as generic pattern detectors while later layers extract task specific features, to design the method. As such early layers require little adaptation sicne they use the pretrained existing feature extractors while later layers require more extensive finetuning. It is also interesting this observation is verified in practice.
  • Ablations cover a significant part of the design space.
  • Literature review is comprehensive
Critical weaknesses
  • Training settings are nor consistent among tables. T2 uses PArtial Fc, T4 uses FC head, T5 trains output projection with Partial FC. A single method would make results more easily comparable.
  • How checkpoints are selected is not clear, a central claim is needed about which checkpoint is used which is consistent among datasets, for example the final checkpoint after N epochs. Proposed method is only tested on CLIP ViT-L/14 and ViT-B/16, despite other foundation models available.
  • For FR TAR@FAR=1e-4 is very lenient allowing 1 mistake every 10k comparisons, would be interesting to see more values reported for 1e-5 and 1e-6 too.
  • No tables contain any variance info. For saturated sets like these FR sets it is critical to see that reported metrics do not fall in the same region when accounting for variance.
  • Paper claims training efficiency is important, however, inference time for FR is much greater cost than training and all models have the same inference cost, as such the efficiency claim is very limited.
  • No domains where FR data are really scarse such as IR are tested.
  • No code is provided.
Questions for authors
  • What is the TAR of your model @FAR=1e-5 and 1e-6 against comparisons?
  • How do you select which checkpoint to test?
2 comments
AuthorsOpenReview ↗

Authors Reply to Reviewer Z929 [1/2]

We thank the reviewer for their time in reviewing our paper and for their insightful comments. Below, we tried our best to address the remaining concerns raised by the reviewer:

  • Reply to Weakness #1: We thank the reviewer for this suggestion. We reran the experiments and revised the paper by updating the results in Table 2 and 4. In the revised version, we ensure that tables 2, 4, and 5 are consistent (output projection trained, Partial FC head). After the revision, the conclusions remain valid.

  • Reply to Weakness #2 (checkpoint): We use the model at the last epoch for evaluation. We clarified this in the revised version of the paper.

  • Reply to Weakness #2 (other foundation models): In addition to the results reported in the paper for CLIP ViT-L/14 and ViT-B/16, in the revised version of the paper, we also provide results for adapting DINO and EVA-CLIP with depth-graded rank allocation. The following table shows the comparison of our adaptation of CLIP ViT-L/14, EVA02-CLIP ViT-L/14 and DINOv2 ViT-L/14 with DG-LoRA (r: 16 → 256) on CASIA-WebFace:

BackboneLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
CLIP ViT-L/1499.4095.7489.3891.8290.8793.4477.5182.22
EVA02-CLIP ViT-L/1499.3595.8187.9791.5391.2593.1875.1180.58
DINOv2 ViT-L/1499.5396.1192.6892.2891.3594.3976.8583.31

As can be seen, our method can be used with other foundation models as well. In particular, we observe that DINOv2 achieves the highest average verification accuracy (0.95 points above CLIP) and the highest TAR on IJB-C (83.31 against 82.22 for CLIP), while CLIP is the best on IJB-B (77.51 against 76.85 for DINOv2). EVA02-CLIP is 2.40/1.64 points below CLIP on IJB-B/C. We also report complete spectral probe analysis for these models in the appendix which confirms our findings.

  • Reply to Weakness #3: Following the reviewer's suggestion, we added a comparison with FRoundation at low FAR on IJB-B and IJB-C in the appendix of the revised version of the paper. The following table shows the comparison of our adaptation of CLIP ViT-L/14 with DG-LoRA and DG-DoRA (r: 16 → 256) on WebFace4M against FRoundation:
MethodAdaptationIJB-B @ 1e-6IJB-B @ 1e-5IJB-B @ 1e-4IJB-B @ 1e-3IJB-C @ 1e-6IJB-C @ 1e-5IJB-C @ 1e-4IJB-C @ 1e-3
FRoundationLoRA, r = 1640.9080.7790.0594.9578.3287.3592.7896.34
FMFace (ours)DG-LoRA, 16 → 25642.3386.5593.3296.3182.0190.9095.3097.66
FMFace (ours)DG-DoRA, 16 → 25640.3485.5093.0096.3480.8990.7495.0997.48

As the results show, our method outperforms FRoundation at all low FAR values on IJB-B and IJB-C datasets.

  • Reply to Weakness #4: Following the reviewer's suggestion, we added variance information to all tables for LFW, CFP-FP, AgeDB, CALFW, and CPLFW benchmarks. For comparison with baselines, we also reported TAR for low values of FAR (1e-6, 1e-5, 1e-4) on IJB-B and IJB-C in the appendix B of the revised version of the paper.

  • Reply to Weakness #5: We agree that when a face recognition model is deployed, its inference cost outweighs its training cost. However, we should note that our method does not change the inference cost. After training, the low-rank update can be merged into the weights. Hence, every adapted model in our tables has the architecture and inference cost of the CLIP ViT-L/14 image encoder. Our main claim is about accuracy at a fixed training cost. Depth-graded allocation improves IJB-B/C over uniform-rank adaptation and over other methods at the same inference cost. Training also costs the same as uniform adapters with same training budget, where the number of trainable parameters is the same, and unlike adaptive-rank methods such as SoRA, DyLoRA and PRILoRA, it needs no search or pruning. We should highlight that parameter-efficient adaptation still has practical value. Training 14M instead of 304M parameters needs no gradients or optimiser states for the frozen parametrs (304M). When inference cost is the constraint, a smaller encoder can also be used. Table 6 reports this trade-off for ViT-B/16, with 3.5 times fewer encoder parameters, which reaches 88.98/92.17 TAR at FAR = 1e-4 on IJB-B/C4. In the revised paper, we state explicitly that all compared adapted models have the same inference cost, and we have toned down the efficiency wording.

AuthorsOpenReview ↗

Authors Reply to Reviewer Z929 [2/2]

  • Reply to Weakness #6: We agree that heterogeneous face recognition, such as NIR-VIS, is an important data-scarce setting. Our study explores where capacity is needed when a foundation model is adapted to FR, and we evaluate it at two data scales on CASIA-WebFace and WebFace4M. Extending depth-graded allocation to cross-spectral FR with only a few hundred training identities can be studied in future work. However, following the reviewer's suggestion, we evaluated our method on near-infrared (NIR) to visible (VIS) face recognition using CASIA NIR-VIS 2.0 with the standard view-2 protocol with 10 folds. In each fold, the training set has about 8.7k NIR and VIS images of 357 identities. The test set matches about 6.2k NIR probe images against a VIS gallery with one image for each of 358 other identities. We report rank-1 identification accuracy and the verification rate (VR) at FAR = 0.1% and 0.01%, as mean ± std over the 10 folds. We consider two models trained on WebFace4M and then evaluate without any NIR training (zero-shot):
Training dataModelRank-1 (%)VR @ FAR=0.1%VR @ FAR=0.01%
WebFace4MrsLoRA98.53 ± 0.3396.85 ± 0.3888.16 ± 1.88
WebFace4MDG-LoRA (ours)99.40 ± 0.2198.53 ± 0.2193.54 ± 1.04

As an additional experiment, we consider the two models trained on WebFace4M and then fine-tune them on the NIR-VIS training set of each fold:

Training dataModelRank-1 (%)VR @ FAR=0.1%VR @ FAR=0.01%
WebFace4M → NIR-VIS trainrsLoRA99.95 ± 0.0399.90 ± 0.0599.20 ± 0.28
WebFace4M → NIR-VIS trainDG-LoRA (ours)99.96 ± 0.0299.96 ± 0.0299.55 ± 0.13

As the results show, the model adapted with depth-graded allocation outperforms the uniform-rank adaptation on both zero-shot and fine-tuned NIR-VIS face recognition. Further experiments on cross-spectral FR by adapting the foundation models can be studied in future work.

  • Reply to Weakness #7: As mentioned in the Reproducubulity Statement, we will release the code upon acceptance of the paper.

  • Reply to Question #1: Following the reviewer's suggestion, we added a comparison with FRoundation at low FAR on IJB-B and IJB-C in the revised version of the paper. The following table shows the comparison of our adaptation of CLIP ViT-L/14 with DG-LoRA and DG-DoRA (r: 16 → 256) on WebFace4M against FRoundation:
MethodAdaptationIJB-B @ 1e-6IJB-B @ 1e-5IJB-B @ 1e-4IJB-B @ 1e-3IJB-C @ 1e-6IJB-C @ 1e-5IJB-C @ 1e-4IJB-C @ 1e-3
FRoundationLoRA, r = 1640.9080.7790.0594.9578.3287.3592.7896.34
FMFace (ours)DG-LoRA, 16 → 25642.3386.5593.3296.3182.0190.9095.3097.66
FMFace (ours)DG-DoRA, 16 → 25640.3485.5093.0096.3480.8990.7495.0997.48

As the results show, our method outperforms FRoundation at all low FAR values on IJB-B and IJB-C datasets.

  • Reply to Question #2: We use the model at the last epoch for evaluation. We clarified this in the revised version of the paper.

We hope we could adequately address the reviewer's concerns. In case reviewer found our reply convincing, we appreciate it if the reviewer can increase their scores and rating. We are happy to continue the discussion if any part remains unclear.

Reviewer_Joo7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper probes high-rank LoRA adapters on CLIP ViT-L/14 trained on CASIA-WebFace. The rank needed to capture 90% of the update energy grows with depth: from 19 to 100 for the value projection, and from 3 to 52 for the query projection. Rank-16 adapters are nearly saturated in the deep blocks. The same pattern appears for DoRA, and for DINOv2 and EVA02.

Based on this, the paper lets the adapter rank grow linearly from 16 in the first block to 256 in the last, with rank-stabilised scaling, and applies this to LoRA and DoRA. It compares eight ViT-L foundation models, an equal-budget uniform rank, and several design choices.

On WebFace4M, DG-LoRA reaches 95.12% TAR at FAR=1e-4 on IJB-C with 14.2M trainable parameters. FRoundation's rank-16 LoRA reaches 93.40%, and a ViT-L trained from scratch reaches 94.79%.

Critical strengths

The spectral probe is a clean and convincing diagnostic. It is replicated across adapter types and backbones, and it gives practitioners a concrete reason to put adaptation capacity in the later blocks.

The WebFace4M result is strong for parameter-efficient face recognition: 93.10/95.12 TAR on IJB-B/C, with under 5% of the encoder's parameters trained. The RFW analysis shows gains on every demographic group, and it honestly reports that the skewed error ratio gets worse.

Critical weaknesses

The central claim, that depth-graded allocation beats uniform allocation at the same budget, is not supported by the paper's own comparison. In Table 2 on CASIA-WebFace, uniform rank 136 and the 16-to-256 schedule are essentially tied:

  • benchmark average: 93.42 vs 93.43;
  • IJB-B: 76.03 vs 76.76;
  • IJB-C: 80.80 vs 80.83.

These are single runs at the final checkpoint. The text describes this as an improvement, and it says depth grading is 1.1 points better on average than uniform rank 256. The table actually shows 93.43 vs 93.33, and rank 256 is better on both IJB-B and IJB-C.

The headline WebFace4M gain over FRoundation compares about 13.4M adapter parameters against about 1.6M for uniform rank 16, so it mixes budget with allocation. Several comparisons that would separate the two are missing: a uniform rank-136 run at WebFace4M scale, a reversed schedule at the same budget, and any comparison with adaptive allocation methods. PRILoRA, which the paper describes as already using an increasing rank, appears to be the closest prior schedule, and the difference from it should be stated clearly.

A smaller, related point is that the paper does not say whether the FRoundation and from-scratch numbers in Table 3 were reproduced under its own training recipe or taken from the original paper. The margin over training from scratch is small, 0.18 on IJB-B and 0.33 on IJB-C, so a difference in recipe could account for it.

Questions for authors
  1. Could uniform rank 136 and a reversed 256-to-16 schedule be reported on WebFace4M at the same budget? If that is too costly, CASIA results over at least three seeds would help.
  2. Were the FRoundation and from-scratch numbers in Tables 3 and 10 reproduced with the paper's recipe?
  3. The text says the output projection is frozen for Table 2, but the caption says it is trained. Which is correct?
  4. Some numbers in the text do not match the tables:
    • "1.1 points" in the Table 2 discussion, where the table gives 0.10;
    • "0.24 points" in the Table 8 discussion, where the table gives 93.44 vs 93.43;
    • the DFN-CLIP gaps of 2.24 and 38.18 in Sec. 4.2, which match Table 9 (adapters only) rather than Table 1, which gives 2.35 and 51.10.
  5. Why does training the output projection hurt OpenCLIP, MetaCLIP and DFN-CLIP so much in Table 9 (drops of 35.0, 29.0 and 9.9 on IJB-B)? Since Table 1 ranks the foundation models with the projection trained, could this choice be driving the ranking?
  6. How were the endpoints 16 and 256 chosen? The probe's 90% energy rank peaks around 100 for the value projection and 52 for the query projection.
  7. How does the proposed schedule differ from PRILoRA's, and how do the two compare in practice?
2 comments
AuthorsOpenReview ↗

Authors Reply to Reviewer Joo7 [1/2]

We thank the reviewer for their time in reviewing our paper and for their insightful comments. Below, we tried our best to address the remaining concerns raised by the reviewer:

  • Reply to Question #1: Following the reviewer's suggestion, we conducted a new experiment comparing uniform, reversed and depth-graded allocation of the rank budget on WebFace4M with rsDoRA. We consider low false acceptance rate (FAR) settings for IJB-B and IJB-C, which are more challenging:
AllocationRankIJB-B @ 1e-6IJB-B @ 1e-5IJB-B @ 1e-4IJB-C @ 1e-6IJB-C @ 1e-5IJB-C @ 1e-4
Uniform13636.5684.7392.6280.8689.9795.01
Reversed256 → 1637.8082.7592.4180.0089.0794.61
Depth-Graded16 → 25640.3485.5093.0080.8990.7495.09

As shown in the table, the depth-graded allocation outperforms the uniform and reversed allocation on different FAR settings for IJB-B and IJB-C. This result supports our claim that depth-graded allocation is more effective than uniform allocation for the same rank budget.

  • Reply to Question #2: In the initial submission, we reported numbers from FRoundation. Following the reviewers' concern, we used the released checkpoints from FRoundation and evaluated the baselines with our implementation and mentioned it in the revised paper. We should note that with this change, most of training and evaluation setup are identical, except for the rank allocation and loss function (we used ArcFace instead of CosFace). To further address the reviewer's concern, we trained our method with CosFace loss with exact same paramters (s = 64, m = 0.3) on WebFace4M and compared with FRoundation:
MethodAdaptationLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
Train from scratchFull training99.5595.5694.0094.3092.1295.1092.5194.53
FRoundationLoRA, r = 1699.6396.4092.9394.2593.4295.3390.0592.78
FMFace (ours)DG-LoRA, 16 → 25699.6797.3696.1895.3294.5096.6093.6195.58
FMFace (ours)DG-DoRA, 16 → 25699.7797.3096.0895.3094.7096.6392.9195.05

As shown in the table, our method outperforms FRoundation on all benchmarks. We added the results of training with CosFace loss to the appendix of the revised paper.

  • Reply to Question #3: We revised the paper and fixed the typo. The projection is trained in Table 2.

  • Reply to Question #4: We thank the reviewer for the feedback. We revised the paper and fixed all the typos and mismatches.

  • Reply to Question #5: With the projection trained, the verification accuracies of these three models are almost unchanged, and TAR on IJB-B/C at FAR=10−210^{-2} and 10−110^{-1} stays within 1.2 points of the frozen-projection model. But TAR at FAR=10−510^{-5} falls sharply, so the drop is mainly at low FAR values. At these FARs the threshold is set by the few highest-scoring impostor scores. Therefore, the drop means that a small number of impostor pairs receive higher similarity than most genuine pairs. This tail is a property of these encoders and is present even with the projection frozen. DFN-CLIP reaches only 3.08% TAR at FAR=10−510^{-5} on IJB-B, against 60.72% for CLIP, with frozen projection.

It is noteworthy that the projection training changes the middle of the ranking but not the best performing models. With the projection frozen for every model, CLIP and EVA02-CLIP remain the two best models and DINOv2 is third. DFN-CLIP remains last, so its low TAR at low FAR is not caused by training the projection. What changes is the order of OpenCLIP and MetaCLIP relative to the SigLIP models. With the projection frozen, OpenCLIP and MetaCLIP rank above SigLIP and SigLIP 2, while with trained projection, they rank below. We have added a discussion of this in the appendix A of the revised paper.

  • Reply to Question #6: Rank r = 16 is the uniform rank used in prior face recognition adaptation of foundation models (e.g., FRoundation). Our spectral probe analysis shows it is enough for shallow blocks. For the maximum rank, we should not set it to the peak r90r_{90}, for two reasons. Because, r90r_{90} leaves out the last 10% of the update energy, and in deep blocks that energy is spread over many more directions. In rsLoRA, for W_v in the last block, the rank holding 99% of the energy is 186 and the effective rank is 188. rLr_{L} = 256 is above the 99%-energy rank of every block, for both projections.
AuthorsOpenReview ↗

Authors Reply to Reviewer Joo7 [2/2]

  • Reply to Question #7: Following the reviewer's suggestion, we also trained with PRILoRA and several other adaptation techniques and added the results to Table 2 of the revised paper. We found that our method achieves competitive performance and outperforms PRILoRA on most benchmarks:
MethodRankParamsLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
DG-LoRA16 → 25614.16M99.4095.7489.3891.8290.8793.4477.5182.22
PRILoRA68 → 20414.16M99.2295.8488.9891.5791.3293.3977.5181.68

In Table 2 of the revised paper, we further compare our method with other adaptation techniques, including DyLoRA and SoRA, and showed that our method outperforms all of them on most benchmarks.


  • Reply to Weakness #1 (Uniform vs Depth-Graded): As mentioned in Reply to Question #1, following the reviewer's suggestion, we conducted a new experiment in response to question 1, comparing uniform and depth-graded allocation of the rank budget on WebFace4M. As shown in the table, the depth-graded allocation outperforms the uniform allocation on all different FAR settings for IJB-B and IJB-C. This result supports our claim that depth-graded allocation is more effective than uniform allocation for the same rank budget.
AllocationRankIJB-B @ 1e-6IJB-B @ 1e-5IJB-B @ 1e-4IJB-C @ 1e-6IJB-C @ 1e-5IJB-C @ 1e-4
Uniform13636.5684.7392.6280.8689.9795.01
Reversed256 → 1637.8082.7592.4180.0089.0794.61
Depth-Graded16 → 25640.3485.5093.0080.8990.7495.09
  • Reply to Weakness #2 (Comparison with FRoundation): For comparison with FRoundation, in the revised version of the paper we used the released checkpoints from FRoundation and evaluated the baselines with our implementation. Therefore, any different in the adaptation (eg number of trainable parameters, rank allocation, loss function) is due to the design of FRoundation. To investigate the effect of number of parameters, following the reviewer's suggestion, we have compared our depth-graded rank allocation with uniform allocation with the same number of parameters in Table 2 of the revised paper. In addition, as also mentioned in reply to Question 7, we have compared our method with other adaptation techniques, including DyLoRA, SoRA and PRILoRA, and showed that our method outperforms all of them on most benchmarks. The results for different adaptation techniques are shown in the Table 2 of the revised paper.

  • Reply to Weakness #3 (Comparison with other adaptation techniques): Following the reviewer's suggestion, we have compared our depth-graded rank allocation with other adaptation techniques, including DyLoRA, SoRA and PRILoRA. The results for different adaptation techniques are shown in the Table 2 of the revised paper and show that our method outperforms different adaptation techniques on most benchmarks. We also added discussions and compared with other adaptation techniques in the revised paper.

  • Reply to Weakness #3 (Evaluation of FRoundation): As mentioned in reply to Question 2, in the initial submission we reported numbers from FRoundation. Following the reviewers' concern, we used the released checkpoints from FRoundation and evaluated the baselines with our implementation and mentioned it in the revised paper. We should note that with this change, most of training and evaluation setup are identical, except for the rank allocation and loss function (we used ArcFace instead of CosFace). To further address the reviewer's concern, we trained our method with CosFace loss with exact same paramters (s = 64, m = 0.3) on WebFace4M and compared with FRoundation:

MethodAdaptationLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
Train from scratchFull training99.5595.5694.0094.3092.1295.1092.5194.53
FRoundationLoRA, r = 1699.6396.4092.9394.2593.4295.3390.0592.78
FMFace (ours)DG-LoRA, 16 → 25699.6797.3696.1895.3294.5096.6093.6195.58
FMFace (ours)DG-DoRA, 16 → 25699.7797.3096.0895.3094.7096.6392.9195.05

As shown in the table, our method outperforms FRoundation on all benchmarks. We added the results of training with CosFace loss to the appendix of the revised paper.

We hope we could adequately address the reviewer's concerns. In case reviewer found our reply convincing, we appreciate it if the reviewer can increase their scores and rating. We are happy to continue the discussion if any part remains unclear.

Reviewer_41ghOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This article focuses on adapting foundation models for face recognition (FR) and proposes depth-graded rank allocation (DG): through spectral rank probing, it is found that in ViT's LoRA/DoRA adapters, updates learned by deeper blocks require more effective rank, while uniform rank saturates in the deeper layers. Based on this, the authors allow the adapter rank to increase linearly from shallow to deep layers, and combined with rank-stabilised scaling, they obtain DG-LoRA and DG-DoRA, collectively referred to as FMFace. Experiments cover 8 ViT-L foundation models, mainly trained on CASIA-WebFace and WebFace4M using ArcFace/Partial FC, and evaluated on LFW, CFP-FP, AgeDB, CALFW, CPLFW, as well as IJB-B/IJB-C. On WebFace4M, DG-LoRA with approximately 14M trainable parameters achieves 96.53 Avg., 93.10 on IJB-B, and 95.12 on IJB-C, surpassing the uniform LoRA of FRoundation and ViT-L trained from scratch on the same data.

Critical strengths

Please identify one or two critical strengths of the paper that support acceptance. For each, explain briefly why it is consequential to your overall assessment. The goal is not to enumerate every positive point, but to identify and substantiate the considerations that most strongly determine your recommendation. Enter “None” if no substantive strength can be identified.

Empirical rank probing analysis is convincing. The authors directly measure the energy rank and effective rank of adapter updates at different depths, showing that the uniform rank is saturated in deeper layers but has spare capacity in shallower layers. This approach provides a more substantiated basis than purely heuristic rank scheduling and also offers a reusable analytical perspective for subsequent PEFT research.

The method is simple, easy to integrate, and achieves strong results on WebFace4M. DG-LoRA/DG-DoRA do not introduce pruning, search, or additional training overhead, only adding rank scheduling hyperparameters; with 14M trainable parameters on WebFace4M, they surpass FRoundation and ViT-L trained from scratch on the same data, demonstrating that adapting foundation models to FR has practical value.

Critical weaknesses

Please identify one or two critical weaknesses of the paper that support rejection. For each, explain briefly why it is consequential to your overall assessment. The goal is not to enumerate every negative point, but to identify and substantiate the considerations that most strongly determine your recommendation. Enter “None” if no material weakness can be identified.

The core benefits are not sufficiently isolated, and fair comparisons are lacking. The main claim of the paper is that depth-graded rank outperforms uniform rank. However, the comparison at equal budgets in Table 2 is only on CASIA-WebFace, with an average improvement of about 0.01 and an IJB-B improvement of about 0.73; there is no uniform rsLoRA/rsDoRA baseline with the same total rank/parameter budget on WebFace4M. Therefore, the substantial improvement on WebFace4M cannot rule out contributions from more parameters, rsLoRA scaling, or DoRA parameterization rather than depth-graded scheduling.

The novelty is limited, and there is a lack of comparison with existing rank allocation methods. Depth-increasing rank has similar ideas in NLP works such as PRILoRA; although this paper applies it to FR/ViT and supports it with a spectral probe, the method itself is linear interpolation of rank, so the incremental innovation is minor. The experiments also do not compare with rank allocation methods such as AdaLoRA, SoRA, DyLoRA, or PRILoRA under the same budget, making it difficult to support the strong claim of being a 'state-of-the-art parameter-efficient adaptation.'

Questions for authors
  1. Could you provide a comparison of uniform rsLoRA/rsDoRA and DG-LoRA/DG-DoRA with the same total rank/same parameter budget on WebFace4M?
  2. Is it possible to compare with rank allocation methods such as AdaLoRA, SoRA, DyLoRA, and PRILoRA under the same parameter budget?
  3. What are the specific values for uniform 136 and DG 16→256 on IJB-C in Table 2? How should the statement in the text '1.1 points better on average' be interpreted?
1 comment
AuthorsOpenReview ↗

Authors Reply to Reviewer 41gh

We thank the reviewer for their time in reviewing our paper and for their insightful comments. Below, we tried our best to address the remaining concerns raised by the reviewer:

  • Reply to Question #1: Following the reviewer's suggestion, we conducted a new experiment comparing uniform and depth-graded allocation of the rank budget on WebFace4M with rsDoRA:
AllocationRankAdapterParamsLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
Uniform136rsDoRA14.20M99.6897.3495.7594.7094.3396.3692.6295.01
Depth-Graded16 → 256rsDoRA14.20M99.6797.3395.7795.1794.5096.4993.0095.09

As shown in the table, the depth-graded allocation outperforms the uniform allocation on most benchmarks, including IJB-B and IJB-C. This result supports our claim that depth-graded allocation is more effective than uniform allocation for the same rank budget.

  • Reply to Question #2: Following the reviewer's suggestion, we compared our adaptation method with other adaptation techniques in the literature. The results are added to Table 2 of the revised paper. We found that our method outperforms other adaptation methods on most benchmarks.
AllocationRankAdapterParamsLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
Depth-Graded16 → 256rsLoRA14.16M99.4095.7489.3891.8290.8793.4477.5182.22
SoRA≤ 136rsLoRA14.16M99.3095.9489.7091.6391.2593.5776.0580.51
DyLoRA136rsLoRA14.16M99.3095.9689.4091.8591.4393.5976.9981.34
PRILoRA68 → 204rsLoRA14.16M99.2295.8488.9891.5791.3293.3977.5181.68
  • Reply to Question #3: As mentioned in reply to question 2, following the reviewer's suggestion, we updated Table 2 and added comparison with other adaptation methods in the literature. The average accuracy also refers to the average of verification accuracy for the five benchmarks (LFW, CFP-FP, AgeDB, CALFW, CPLFW) in the Table. We also revised the mentioned sentence in the paper.

  • Reply to Weakness #1 (Comparison on WebFace4M): As mentioned in Reply to Question #1, following the reviewer's suggestion, we conducted a new experiment in response to question 1, comparing uniform and depth-graded allocation of the rank budget on WebFace4M. As shown in the table, the depth-graded allocation outperforms the uniform allocation on most benchmarks, including IJB-B and IJB-C. This result supports our claim that depth-graded allocation is more effective than uniform allocation for the same rank budget.
AllocationRankAdapterParamsLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
Uniform136rsDoRA14.20M99.6897.3495.7594.7094.3396.3692.6295.01
Depth-Graded16 → 256rsDoRA14.20M99.6797.3395.7795.1794.5096.4993.0095.09
  • Reply to Weakness #2 (Comparison with other adaptation methods): As mentioned in reply to question 2, following the reviewer's suggestion, we updated Table 2 and added comparison with other adaptation methods in the literature.

We hope we could adequately address all the reviewer's concerns. In case reviewer found our reply convincing, we appreciate it if the reviewer can increase their scores and rating. We are happy to continue the discussion if any part remains unclear.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Authors General Response to Reviewers

We thank all reviewers for their time and valuable comments which helped us improve the quality of our paper. We carefully considered every point raised by the reviewers and revised the paper accordingly or provided point-by-point responses in our reply to each reviewer.

The summary of important changes in the revised version of the paper and reply to reviewers is as follows:

1. Comparison with different adaptation methods [Reviewer 41gh and Reviewer Joo7]. As a new experiement we compare our adaptation with other Table 2 now also includes SoRA, DyLoRA and PRILoRA at the same budget. DG-LoRA has the highest TAR on IJB-B and on IJB-C. On the verification benchmarks, all methods are competetive within 0.2 points on average. PRILoRA is the closest prior adaptation method and also increases rank with depth. It is initially designed for NLP and has also a pruning step, whereas our study is based on spectral analysis for FR and needs no pruning.

AllocationRankAdapterParamsLFWCFP-FPAgeDBCALFWCPLFWAvg.IJB-B @ 1e-4IJB-C @ 1e-4
Depth-Graded16 → 256rsLoRA14.16M99.4095.7489.3891.8290.8793.4477.5182.22
SoRA≤ 136rsLoRA14.16M99.3095.9489.7091.6391.2593.5776.0580.51
DyLoRA136rsLoRA14.16M99.3095.9689.4091.8591.4393.5976.9981.34
PRILoRA68 → 204rsLoRA14.16M99.2295.8488.9891.5791.3293.3977.5181.68

2. Same-budget comparison on WebFace4M [Reviewer Joo7 and Reviewer 41gh]. We trained a uniform (r = 136) and a reversed (256 → 16) rank allocation on WebFace4M with exactly the budget of DG-DoRA and the same training setup:

AllocationRankAvg.IJB-B @1e-5IJB-B @1e-4IJB-C @1e-5IJB-C @1e-4
Uniform13696.3684.7392.6289.9795.01
Reversed256 → 1696.3582.7592.4189.0794.61
Depth-graded16 → 25696.4985.5093.0090.7495.09

Depth-graded allocation has the highest TAR at every FAR from 1e-6 to 1e-4 on both IJB-B and IJB-C, and the reversed schedule is the lowest on IJB-C at every FAR. Most of the gain over FRoundation comes from giving the deep blocks much more capacity than rank 16, which the spectral probe shows they need. Depth grading allocates this capacity where the probe says it is needed.

3. Baselines and evaluations [Reviewer Joo7, Reviewer Z929]

  • FRoundation is now evaluated from their released checkpoints with our evaluation script (Table 3). As a new ablation on loss function, we also trained our models with CosFace, which is the same loss function used by FRoundation (Table 10).
  • Every run is evaluated at its final checkpoint (epoch-20), as stated in Sec. 4.1 of revised paper.
  • All tables in the revised paper report the verification benchmarks as mean ± std over the 10 folds. Appendix B reports TAR on IJB-B/C from FAR = 1e-6 to 1e-3.
  • The CASIA-WebFace ablations of depth grading (Tables 2, 4 and 5) now have one shared training setup, with the output projection trained and a Partial FC head. We re-ran Table 4 to make it consistent with Tables 2 and 5.

4. Additional experiments [Reviewer Z929]

  • DG-LoRA applied to DINOv2 and EVA02-CLIP (Table 11), with the spectral probe for both models (Appendix D).
  • Cross-spectral NIR-VIS face recognition on CASIA NIR-VIS 2.0, zero-shot and fine-tuned (reply to Reviewer Z929).

We hope we could adequately address the reviewers' concerns. We kindly invite all reviewers to further discussions if any part remains unclear.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.