acceptodds
Under review as a conference paper at ICLR 2027

RefRouteFR: Demand-Driven Sparse Reference-Token Routing for One-Step Face Restoration

Abstract

Reference-based face restoration uses high-quality images of the same identity to recover personalized details from degraded observations. However, reference images exhibit substantial input-dependent redundancy: only a subset of reference regions is useful for a given low-quality (LQ) input, whereas dense processing wastes computation and may introduce mismatched textures or attributes. We propose RefRouteFR, a one-step framework centered on demand-driven sparse reference-token routing. RefRouteFR exploits the native multimodal interface of a pretrained Diffusion Transformer to incorporate reference guidance and text control without requiring auxiliary reference-injection networks. A lightweight router estimates the restoration-dependent usefulness of reference tokens from the current multimodal context and forwards only the most informative subset through each Transformer block, while redundant tokens bypass expensive computation. To enable stable one-step restoration, a learned latent calibrator mitigates the mismatch between degraded LQ latents and the pretrained flow prior. Experiments on multiple datasets demonstrate state-of-the-art restoration performance, with RefRouteFR matching the quality of its dense counterpart while reducing inference time by approximately 40%.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Face Restoration, Diffusion Models, Sparse Token Routing, Multimodal Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XiPROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RefRouteFR, a one-step multi-reference face restoration method based on FLUX2. The main idea is to use a lightweight router in the Transformer’s single-stream blocks to select the Top-K important tokens from multiple reference images, reducing redundant computation while using a latent calibrator to improve restoration quality. Experiments show that this method can significantly cut down FLOPs and inference time while maintaining good restoration and identity consistency. The paper also includes ablation studies on token numbers, the number of references, and router design. ...

— —We have re-examined the paper and reconsidered our assessment in light of the authors’ responses and the other reviewers’ comments. The clarifications regarding the source of test prompts, the FFHQ-Ref evaluation protocol, and reference selection, together with the additional baseline comparisons, address our main concerns. Although the real-world evaluation remains limited in scale, we are inclined to raise our rating from 2 (Weak rejection) to 3 (Weak acceptance).

Critical strengths
  1. Effective computational efficiency optimization. Reference-token routing can significantly reduce computation while avoiding the introduction of an extra complex reference network, which is practically meaningful for multi-reference image scenarios.
  2. Ablation experiments are quite thorough. The authors systematically analyzed token budget, the number of reference images, and router design, providing some support for the method's effectiveness.
Critical weaknesses
  1. Experimental protocol is not clear enough and baselines are incomplete. The specific settings for FFHQ-Ref, the number of reference images, and the comparison conditions for different baselines are not sufficiently explained. Additionally, several recent directly related methods are missing for comparison.
  2. There are potential issues with prompt setup. During training, prompts generated from GT images are used, but the paper does not clearly specify what prompts are used during testing. If GT-derived prompts are also used during testing, there might be target information leakage; if generic prompts are used, there is an obvious train-test mismatch. Moreover, the real-world experiment scale is relatively small.
Questions for authors
  1. What prompts are specifically used during testing? Can you confirm that GT image information is completely excluded during testing?
  2. Please clarify which standard setting is used for FFHQ-Ref and how many reference images are used for each baseline.
  3. Could you include recent directly related methods and compare them under fully consistent reference/evaluation protocols?
3 comments
AuthorsOpenReview ↗

Response to W2/Q1 and W1/Q2: Test-Time Prompts and FFHQ-Ref Protocol

Thank you for recognizing the computational efficiency and the ablation studies. We clarify the information available at test time, the FFHQ-Ref evaluation protocol, and reference counts for each class of method. We also provide additional ASFFNet and ReFine comparisons and describe the scope of the real-world evaluation.

W2 / Q1: Test-time prompts and exclusion of GT information

Response. All main benchmark results use RAM-generated prompts derived solely from LQ images, without any GT information at inference. We adopt RAM to ensure a consistent and reproducible evaluation protocol, while Gemma can alternatively generate descriptions from LQ images and available references. During training, GT-derived prompts provide reliable semantic supervision for learning fine-grained facial attributes and text-controllable restoration. We also apply 10% text-prompt dropout to mitigate the train–test gap.

To assess prompt sensitivity, we additionally evaluate GT-derived prompts as an oracle setting. As shown in Table 1, the two prompt sources yield comparable performance, with modest perceptual improvements and nearly unchanged identity consistency under GT prompts.

Table 1. Effect of test-time prompt sources on CelebRef-HQ. LQ-derived prompts are generated using RAM for the main evaluation, while GT-derived prompts are used only for oracle analysis.

MethodPrompt SourcePSNR↑LPIPS↓FID↓IDS↑
RefRouteFRLQ-derived27.23830.154835.67200.8796
GT-derived27.21190.152034.31380.8806
RefRouteFR-FullLQ-derived27.18440.148635.52580.9021
GT-derived27.12880.146633.93760.9021

Revision. Appendix A.2 clarifies the distinction between HQ-derived training prompts and LQ-derived prompts used for the main benchmark results, which do not use GT information.

W1 / Q2: FFHQ-Ref protocol and reference counts

We evaluate on all 857 images from the official FFHQ-Ref test split using our model trained exclusively on CelebRef-HQ, without further fine-tuning. Following AdaRefSR [1], LQ inputs are generated using second-order degradation with ×4 downsampling. Consistent with the CelebRef-HQ evaluation protocol, all methods share identical pre-generated LQ inputs at 512×512 resolution. Single-reference methods use one reference, while multi-reference methods use up to four same-identity references, using all available references when fewer than four exist. Reference selections are fixed and shared across methods with the same reference count. All baselines are evaluated using their official implementations and publicly released pretrained weights.

Revision. The FFHQ-Ref evaluation protocol is clarified in Sections 4.1 and 4.2.

References

[1] Yuan Wang, Yuhao Wan, Siming Zheng, Bo Li, Qibin Hou, and Peng-Tao Jiang. Trust but verify: Adaptive conditioning for reference-based diffusion super-resolution via implicit reference correlation modeling. In The Fourteenth International Conference on Learning Representations, 2026.

AuthorsOpenReview ↗

Response to W1/Q3 and W2: Additional baselines and real-world evaluation

W1 / Q3: Recent related baselines

Response. We additionally evaluate ASFFNet [2] and ReFine [3] on CelebRef-HQ and FFHQ-Ref using their official implementations and released weights, following the evaluation protocol described in W1/Q2. Together with the existing baselines, these comparisons cover both representative earlier methods and recent advances in face restoration, as summarized in Table 2.

Table 2. Publication venues and years of representative baselines.

MethodASFFNetCodeFormerDMDNetReF-LDMRestorerID
PublicationCVPR'20NeurIPS'22TPAMI'23NeurIPS'24Preprint'24
MethodReFineInstantRestoreFaceMeRefSTARAdaRefSR
PublicationWACV'25SIGGRAPH'25AAAI'25AAAI'26ICLR'26

As shown in Table 3, both RefRouteFR and RefRouteFR-Full achieve better perceptual quality and identity consistency than these additional baselines on both benchmarks.

Table 3. Additional comparisons on CelebRef-HQ and FFHQ-Ref. Best and second-best results are highlighted in bold and underlined, respectively. Methods marked with * accept multiple reference candidates.

MethodPSNR↑SSIM↑LPIPS↓FID↓IDS↑NIQE↓MUSIQ↑CLIP-IQA↑
CelebRef-HQ
ASFFNet*26.42400.74510.217850.50720.80394.489272.03880.6162
ReFine26.80130.74660.215747.57410.79754.188572.62170.6003
RefRouteFR*27.23830.7489‾\underline{0.7489}0.1548‾\underline{0.1548}35.6720‾\underline{35.6720}0.8796‾\underline{0.8796}4.1439‾\underline{4.1439}74.1893‾\underline{74.1893}0.6200‾\underline{0.6200}
RefRouteFR-Full*27.1844‾\underline{27.1844}0.75160.148635.52580.90214.075174.59810.6236
FFHQ-Ref
ASFFNet*25.80090.71440.284039.54410.80394.417771.15030.6108
ReFine25.60800.71780.372349.49230.76025.995352.93580.4713
RefRouteFR*26.6149‾\underline{26.6149}0.7238‾\underline{0.7238}0.1918‾\underline{0.1918}24.8012‾\underline{24.8012}0.8945‾\underline{0.8945}4.1138‾\underline{4.1138}73.0259‾\underline{73.0259}0.6110‾\underline{0.6110}
RefRouteFR-Full*26.63220.72910.169923.94860.92583.905274.11580.6360

Revision. The ASFFNet and ReFine comparisons have been incorporated into Section 4.2 and Table 1.

W2: Real-world evaluation scale

Response. We acknowledge the limited scale of our real-world evaluation, which includes 50 naturally degraded LFW [4] images from 10 identities. Same-identity HQ references are collected from the web, with one or four references provided to single- and multi-reference methods, respectively. Large-scale public benchmarks for real-world reference-based face restoration remain limited, and existing studies often rely on relatively small, privately collected evaluation sets, such as the 60 real-world image pairs used in RefSTAR. We are currently collecting additional real-world samples and corresponding references to expand the scale and diversity of our evaluation.

Revision. Section 4.1 now specifies the scale and composition of the real-world evaluation set.

References

[2] Xiaoming Li, Wenyu Li, Dongwei Ren, Hongzhi Zhang, Meng Wang, and Wangmeng Zuo. Enhanced blind face restoration with multi-exemplar images and adaptive spatial feature fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2706–2715, 2020.

[3] Min Jin Chong, Dejia Xu, Yi Zhang, Zhangyang Wang, David Forsyth, Gurunandan Krishnan, Yicheng Wu, and Jian Wang. Copy or not? reference-based face image restoration with fine details. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2025.

[4] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007.

We sincerely thank the reviewer for carefully reconsidering our work and for the constructive feedback. We are glad that our clarifications and additional comparisons have addressed your main concerns. We greatly appreciate your time and thoughtful assessment.

Reviewer_AuBSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes RefRouteFR, a one-step and reference-based face restoration framework that leverages a pretrained FLUX2 native multimodal token interface to jointly process low-quality images, multiple high-quality reference images, and text prompts. The central idea is a demand-driven sparse token router. Each Transformer block selects only the most informative reference tokens conditioned on the full LQ–reference–text context, thereby reducing redundant computation while preserving identity-specific detail. To stabilize single-step restoration, a lightweight latent calibrator is introduced to correct the LQ latent toward the pretrained flow trajectory using a residual-guided KL regularization. Experiments on CelebRef-HQ, FFHQ-Ref, and a small real-world set show state-of-the-art identity and perceptual quality with approximately 40% lower inference time versus the dense counterpart.

Critical strengths

Full-context attention router enables demand-driven sparse reference routing.
The paper identifies a real bottleneck in reference-based face restoration,which is that not all reference regions are useful for a given low-quality input, and dense processing can introduce mismatched textures or attributes. By scoring reference patches against LQ self-context and text tokens, the router avoids compulsory reference usage and selects only the most informative tokens. It directly improves restoration reliability while reducing computation, and it fits naturally into the pretrained DiT token pipeline without adding heavy fusion branches. Residual-guided latent calibration makes one-step flow restoration practical.
Low-quality latents deviate from the pretrained flow prior, so using them directly for one-step generation is unstable. The proposed calibration aligns the LQ starting point with the pretrained flow trajectory, enabling a single forward pass to produce high-quality results. It removes the multi-step denoising latency that typically limits diffusion-based restoration, and the experiments show that the method achieves strong quality–efficiency trade-offs across multiple datasets.

Critical weaknesses

The method uses four references at test time. But for baselines that support multi-reference, it is not explicitly stated that they are also evaluated with multiple references under identical settings.

Questions for authors

1.The method uses four references at test time. But for baselines that support multi-reference, it is not explicitly stated that they are also evaluated with multiple references under identical settings. 2.How robust is routing under strong pose/expression mismatch or occlusion in references? Does the authors observe failure cases where the router repeatedly picks misleading regions? I encourage the authors to respond to the questions above, as doing so would greatly enhance the clarity and understanding of the proposed approach.

2 comments
AuthorsOpenReview ↗

Response to W1/Q1 and Q2: Evaluation Protocol and Reference Occlusion

Thank you for recognizing the demand-driven routing design and the efficiency of our one-step restoration framework. We clarify the reference inputs used by the compared methods and provide additional experiments evaluating robustness to reference occlusion and incorrect-identity references.

W1 / Q1: Fair multi-reference comparison

Response. All multi-reference baselines are evaluated with the same reference budget as RefRouteFR: four same-identity references on CelebRef-HQ and up to four on FFHQ-Ref, using all available references when fewer than four exist. Reference images are randomly selected and fixed before evaluation, with identical selections shared across methods using the same reference count. All methods also receive identical LQ inputs. Single-reference methods use one reference, while reference-free methods require none.

Revision. The evaluation protocol is clarified in Sections 4.1 and 4.2, and multi-reference methods are identified in Table 1.

Q2: Routing robustness

Response. Our results suggest that sparse reference-token routing reduces sensitivity to unreliable reference information, particularly under severe occlusion and high levels of identity mismatch. We evaluate these behaviors through two experiments on CelebRef-HQ using four reference images. RefRouteFR and RefRouteFR-Full receive identical perturbed references, while the LQ inputs and text prompts remain fixed.

Random reference occlusion. We randomly occlude each of the four reference images with a contiguous black block, varying the occluded area from 0% to 100%. As shown in Table 1, RefRouteFR maintains better restoration quality and identity consistency under severe occlusion. To examine routing behavior, we measure the routed-region occlusion ratio as the fraction of occluded pixels within the selected reference regions. This ratio remains below the input occlusion ratio under partial masking, suggesting that the router preferentially selects unoccluded reference content.

Table 1. Robustness to random reference occlusion on CelebRef-HQ. Better restoration results at each level are bold.

OcclusionMethodPSNR↑LPIPS↓FID↓IDS↑Routed-region occlusion (%)
0%RefRouteFR-Full27.18440.148635.52580.9021–
RefRouteFR27.23830.154835.67200.87960.0
25%RefRouteFR-Full27.10390.152636.31650.8854–
RefRouteFR27.20280.157136.55670.873212.1
50%RefRouteFR-Full27.06730.158636.61890.8637–
RefRouteFR27.18450.159236.54620.866833.6
75%RefRouteFR-Full27.01830.166337.19710.8502–
RefRouteFR27.17270.160336.99540.857952.9
100%RefRouteFR-Full26.93380.168739.61210.8495–
RefRouteFR27.10980.164138.21700.8542100.0
AuthorsOpenReview ↗

Response to Q2: Robustness to incorrect-identity references

Incorrect-identity references. As a complementary stress test, we progressively replace up to four correct references with images from different identities. As shown in Table 2, both methods experience performance degradation as incorrect references increase. However, RefRouteFR exhibits a smaller decline in identity consistency and achieves better identity preservation and perceptual quality when most references are incorrect. Qualitative examples in Figure B.4 of the revised manuscript show that RefRouteFR largely preserves identity-specific details despite incorrect references, although some identity variations may arise when most references belong to different identities.

Table 2. Robustness to incorrect-identity references on CelebRef-HQ. Better results at each level are bold.

Incorrect refs.MethodPSNR↑LPIPS↓FID↓IDS↑
0/4RefRouteFR-Full27.18440.148635.52580.9021
RefRouteFR27.23830.154835.67200.8796
1/4RefRouteFR-Full27.13270.150935.72640.8960
RefRouteFR27.21170.156736.07460.8709
2/4RefRouteFR-Full27.07680.154536.26810.8744
RefRouteFR27.19410.160037.13390.8617
3/4RefRouteFR-Full27.00220.162738.35040.8543
RefRouteFR27.16770.163338.24070.8565
4/4RefRouteFR-Full26.86760.171439.81110.8217
RefRouteFR27.14340.166038.60060.8498

Revision. Reference occlusion is analyzed in Section 4.5 and Figure 7. Additional robustness results are provided in Appendix B.6, including quantitative comparisons in Tables B.6 and B.7, incorrect-identity trends in Figure B.3, and qualitative examples in Figure B.4.

Reviewer_TditOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the RefRouteFR framework, which is based on a pre-trained FLUX2 DiT and achieves one-step face restoration in multi-reference images through Latent Calibration. To reduce the complex calculation of redundant tokens, RefRouteFR proposes Sparse Reference Token Routing to predict the importance of reference tokens and select important reference tokens to reduce the computational cost while ensuring the recovery level. Experimental results show that RefRouteFR achieves SOTA on CelebRef-HQ and FFHQ-Ref datasets. Ablation experiments also prove the contribution of Latent Calibration and Router.

Critical strengths
  1. The method in this paper achieves impressive results in the task of face restoration on multi-reference images, and achieves SOTA on multiple datasets.
  2. The Sparse Reference Token Routing method can reduce inference time by half while ensuring controllable inference performance loss.
Critical weaknesses
  1. The overall architecture of the paper is more of the design of task adaptation on the existing pre-training DiT, and Latent Calibration and one-step restoration are both adaptation optimization in engineering.
  2. The acceleration method of token routing is well established, even if the design of Full-Context Attention Router is innovative.
  3. Without Routing acceleration, the work's contribution lies in only the engineering adaptation optimization. The method achieves good performance. However, about this advantage is brought by pre-trained FLUX2 DiT or the proposed method, the paper does not make an experimental evaluation.
Questions for authors
  1. Excluding engineering optimizations such as latent calibration, how does the pure pre-trained FLUX2 DiT perform in the face restoration task under multi-reference images?
  2. Could you explain the base model and the number of parameters you are comparing? From the content of this paper, it is difficult to determine whether the superior performance of this method is contributed to the strong capability of pre-trained FLUX2 DiT or the superiority of modules proposed in this paper.
3 comments
AuthorsOpenReview ↗

Response to W1 and W2: Framework Contributions and Router Comparisons

We thank the reviewer for acknowledging the restoration performance and efficiency gains of RefRouteFR. We agree that the role of the pretrained backbone and the specific contributions of our modules require clearer separation. We therefore provide new same-backbone experiments, compare with adapted general-purpose routing strategies, and clarify the architectures and parameter accounting of the evaluated methods.

W1: Contribution beyond pretrained-DiT adaptation

Response. To distinguish the contributions of our framework from the pretrained DiT, we conduct two controlled comparisons. Under the same FLUX2-klein-9B backbone, our one-step framework outperforms conventional four-step fine-tuning in restoration quality and identity consistency, as detailed in W3/Q1. Compared with adapted general-purpose routers, our Full-Context Attention Router achieves better performance, while enabling more efficient scaling as the number of references increases, as shown in W2.

Revision. Same-backbone and adapted-router comparisons have been added in Appendices B.1 and B.3, respectively.

W2: Novelty of reference-token routing

Response. We agree that token routing is an established acceleration technique. Our contribution lies in restoration-demand-aware reference selection. Compared with adapted general-purpose routers, RefRouteFR achieves better LPIPS and FID while maintaining competitive identity consistency, as shown in Table 1. Its fixed reference-token budget also enables more efficient scaling as additional references are introduced.

To validate this distinction, we adapt DynamicViT-style [1] and DiffCR-style [2] routing to reference-token selection while preserving LQ and text tokens. Both baselines employ token-wise linear scoring. DynamicViT-style adopts a progressive keep-ratio schedule of (1.0, 0.6, 0.3, 0.1), whereas DiffCR-style retains a fixed budget of 512 reference tokens. In contrast, our Full-Context Attention Router jointly considers LQ self-context and text context when evaluating reference-token relevance.

Table 1. Four-reference routing comparison on CelebRef-HQ. Best and second-best values are bold and underlined.

RouterPSNR↑SSIM↑LPIPS↓FID↓IDS↑NIQE↓MUSIQ↑CLIP-IQA↑
DynamicViT-style26.82510.75480.1613‾\underline{0.1613}38.01710.8793‾\underline{0.8793}4.380873.35430.6069
DiffCR-style26.8559‾\underline{26.8559}0.74020.161737.3427‾\underline{37.3427}0.86884.2954‾\underline{4.2954}73.7689‾\underline{73.7689}0.6299
RefRouteFR27.23830.7489‾\underline{0.7489}0.154835.67200.87964.143974.18930.6200‾\underline{0.6200}

Beyond restoration quality, the fixed reference-token budget also improves scalability as more references are introduced. As shown in Table 2, RefRouteFR reduces four-reference inference time from 448 ms to 370 ms (17.4%) compared with DynamicViT-style, whose ratio-based selection retains more tokens as the reference count grows.

Table 2. End-to-end latency (ms) with increasing reference count.

Router1 ref.2 refs.3 refs.4 refs.
DynamicViT-style258315382448
RefRouteFR254296333370

Revision. Appendix B.3 provides comparisons with adapted general-purpose routers, with restoration quality reported in Table B.3 and inference latency in Table B.4.

References

[1] Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. DynamicViT: Efficient vision transformers with dynamic token sparsification. In Advances in Neural Information Processing Systems, 2021.

[2] Haoran You, Connelly Barnes, Yuqian Zhou, Yan Kang, Zhenbang Du, Wei Zhou, Lingzhi Zhang, Yotam Nitzan, Xiaoyang Liu, Zhe Lin, Eli Shechtman, Sohrab Amirghodsi, and Yingyan Celine Lin. Layer- and timestep-adaptive differentiable token compression ratios for efficient diffusion transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.

AuthorsOpenReview ↗

Response to W3/Q1 and Q2: Same-backbone results and parameter accounting

W3 / Q1: How much does the pretrained FLUX.2 backbone contribute?

Response. Building on the strong generative prior of FLUX2-klein-9B, our one-step restoration framework achieves better restoration fidelity and identity consistency than conventional four-step fine-tuning under the same backbone, while reducing the number of sampling steps from four to one, as shown in Table 3.

Table 3. Same-backbone comparison on CelebRef-HQ with four references. Best and second-best values are bold and underlined.

ConfigurationStepsPSNR↑LPIPS↓IDS↑FID↓
Pretrained FLUX.2 (frozen)411.72620.56540.553576.6106
Fine-tuned FLUX.2425.23300.21910.838655.3178
RefRouteFR-Full127.1844‾\underline{27.1844}0.14860.902135.5258
RefRouteFR127.23830.1548‾\underline{0.1548}0.8796‾\underline{0.8796}35.6720‾\underline{35.6720}

To evaluate the contribution of the restoration framework, we compare four configurations on identical CelebRef-HQ test samples with four same-identity references: frozen FLUX.2, conventional four-step fine-tuning, RefRouteFR-Full, and RefRouteFR. For the frozen model, we use the instruction “Remove the blur from the first image, using the remaining images as identity references” to specify the roles of the LQ input and reference images.

Revision. The same-backbone restoration comparison is provided in Appendix B.1 and Table B.1.

Q2: Compared backbones and parameter accounting

Response. RefRouteFR is built on FLUX2-klein-9B with a 9.079B-parameter Transformer backbone, while only 156.99M parameters (1.73% relative to the backbone) are trainable. We acknowledge that the strong pretrained generative prior contributes to restoration quality, and therefore provide controlled same-backbone experiments to distinguish its contribution from that of our proposed framework.

The trainable parameters comprise 71.44M LoRA parameters, a 10.03M latent calibrator, and a 75.52M Full-Context Attention Router. Table 4 summarizes the backbone architectures and parameter scales of representative methods.

Table 4. Backbone architectures and parameter scales of representative methods. Trainable parameters are additionally reported for RefRouteFR.

MethodBackboneParams
CodeFormerVQGAN + Transformer73.57M
DMDNetDictionary Network40.36M
RestorerIDSD 1.5∼860M
InstantRestoreSD-Turbo∼860M
FaceMeSDXL∼2.6B
AdaRefSRS3Diff / SD-Turbo2.679B
OursFLUX2-klein-9B9.079B(156.99M trainable)\begin{array}{r}\textbf{9.079B}\\\\\textbf{(156.99M trainable)}\end{array}

Despite its larger pretrained backbone, RefRouteFR achieves competitive inference latency through one-step restoration and sparse reference-token routing, as reported in Table 4 of the revised manuscript. Furthermore, the same-backbone comparison in Table 3 demonstrates the benefits of our restoration framework beyond the pretrained generative prior.

Revision. Component-level parameters and computational costs are reported in Appendix A.6 and Table A.2, with the same-backbone comparison provided in Appendix B.1.

We sincerely thank the reviewer for reconsidering our work and for the constructive feedback. It is encouraging that our additional experiments and clarifications have addressed your concerns, and we greatly appreciate your thoughtful assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Manuscript Revisions

We thank the Area Chair for taking the time to read and assess our rebuttal. To support this assessment, we summarize RefRouteFR's core contributions, our responses to the reviewers' main concerns, and the additional experiments and manuscript revisions.

Core Contributions of RefRouteFR

  • Demand-driven reference-token routing. Built on unified LQ, reference, and text tokens, the Full-Context Attention Router evaluates reference relevance against LQ self-context and text context, selecting a fixed budget of informative reference tokens for Transformer processing. This reduces redundant computation and enables efficient scaling to multiple references.

  • One-step restoration with latent calibration. A learned latent calibrator mitigates the mismatch between degraded LQ latents and the pretrained flow trajectory, enabling a single-step restoration update while preserving restoration fidelity and identity consistency.

Responses to Reviewers' Core Concerns

  • Contribution beyond pretraining and generic routing (Reviewer Tdit). Controlled same-backbone comparisons demonstrate that our one-step framework achieves better restoration quality than conventional four-step fine-tuning while reducing sampling from four steps to one. Comparisons with adapted generic routers further demonstrate the advantages of our Full-Context Attention Router in perceptual quality and computational efficiency.

  • Comparison fairness and routing robustness (Reviewer AuBS). We clarify the evaluation protocol to ensure consistent LQ inputs and fair reference selection across methods. Beyond standard comparisons, additional experiments under reference occlusion and identity mismatch demonstrate that sparse reference-token routing improves robustness to unreliable references, particularly under severe perturbations.

  • Test-time information and evaluation coverage (Reviewer XiPR). We clarify that all main results use LQ-derived prompts without test-time GT information, with additional prompt comparisons confirming consistent performance. We further clarify the evaluation protocols for FFHQ-Ref and the real-world test set, and include additional comparisons with ASFFNet and ReFine.

Additional Experiments and Revisions

  • Main manuscript. We incorporate additional baseline comparisons and reference-robustness analyses, while clarifying evaluation protocols and experimental settings.

  • Appendix. We provide controlled same-backbone comparisons, comparisons with adapted general-purpose routers in restoration quality and inference efficiency, and additional reference-robustness experiments.

We thank the Area Chair and all reviewers for their time and constructive feedback. Their comments have helped us clarify the methodology and strengthen the experimental evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.