acceptodds
Under review as a conference paper at ICLR 2027

RepFlow: Reciprocal Supervision Improves Generation and Representation in Flow Models

Abstract

Generative models learn visual structure through denoising, yet their internal states are entangled with both noise level and network depth, making it difficult to obtain a stable visual representation from the generator itself. We introduce RepFlow, which learns such a representation from the generator's evolving computation and uses it to guide generation. Specifically, a separate timestep-free encoder is trained, through a timestep-conditioned predictor, to recover generator states across depths and noise levels from masked clean images. By excluding the reference noise and masked-out content from the encoder's input, we encourage the encoder to distill visual information that is recoverable from the visible image context and predictive of generator states. The representation learned from the generator's evolving states is then fed back to guide and improve the generator, whose updated states provide supervision for further representation learning, forming a reciprocal learning process. This reciprocal process improves multi-step generation and representation quality, as measured by frozen linear probing on ImageNet with latent-space SiT and pixel-space JiT, without an externally pretrained representation teacher. Across the two unconditional settings, FID decreases by relative to native training, while linear-probe accuracy improves by 6.80–10.04 percentage points over the best searched raw generator features. The learned representation also serves as a distributional metric for one-step JiT post-training, extending its role from instance-level alignment to distribution-level supervision.

Primary area:
generative models
Keywords:
Generative model, Representation learning, Reciprocal Supervision, Flow Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_QfXvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RepFlow, a framework that jointly trains a generator and an encoder through reciprocal supervision. A timestep-independent encoder extracts visual representations from masked clean images, while a timestep-conditioned predictor uses these representations to predict the generator’s internal states. In turn, EMA of the encoder extracts representations from full clean images to guide the generator. Experiments in both SiT and JiT demonstrate the effectiveness of the approach.

Critical strengths
  1. The method jointly trains a generator and an encoder without relying on a pretrained teacher encoder. Assigning timestep conditioning to the predictor is a sensible design choice that allows the encoder to remain general.

  2. Training costs are clearly reported, and the method retains its advantage over the native baseline under matched training budgets.

  3. The paper is well written and easy to understand.

Critical weaknesses
  1. A straightforward alternative is to first train an MAE and then use it to guide generator training. The paper does not establish whether the proposed joint training approach offers advantages over this alternative in generation quality or overall training efficiency.

  2. Table 2 shows that the method achieves worse FID than REPA, which uses a pretrained visual encoder, while remaining competitive with SRA, which does not require one. Meanwhile, the learned encoder’s linear-probe accuracy is substantially lower than that of MAE. The paper should clarify the advantages of the proposed approach and the scenarios in which it would be preferable to these alternatives.

  3. The one-step post-training experiments in Table 3 start from a RepFlow-JiT-XL checkpoint that has already received supervision from the RepFlow encoder during joint training, whereas MAE was not involved in this process. This prior co-adaptation may favor the RepFlow encoder. An additional comparison starting from a natively trained JiT checkpoint would help isolate the effectiveness of the two encoders as distribution-matching metrics.

Questions for authors
  1. Please provides comparison between joint training and separate training. Using pretrained MAE is also accepetd.

  2. Provides fair comparison in one-step post-training experiments.

5 comments
AuthorsOpenReview ↗

## Response to Reviewer QfXv (W1 and Q1)

Thank you for suggesting the separately pretrained MAE comparison and the additional post-training controls. We report these below and clarify when joint generation and representation learning is useful.

W1 and Q1: Joint training versus a separately pretrained MAE

Following your suggestion, we use MAE ViT-L as the frozen teacher for REPA-style generator training. This directly compares separately pretrained MAE guidance with the representations learned jointly by RepFlow, using ViT-L encoders in the RepFlow-L comparison. All methods use unconditional ImageNet at 256 × 256 on eight H200 GPUs. Each budget equals Native's 200-epoch training time: approximately 33.3 hours for SiT and 22.1 hours for JiT. MAE's pretraining cost is excluded; RepFlow learns its encoder within the joint-training budget.

Table Q1. Unconditional FID-50K at matched wall-clock budgets and 200 epochs, including separately pretrained MAE guidance.

MethodSiT FID, ≈33.3 h ↓JiT FID, ≈22.1 h ↓SiT FID, 200 epochs ↓JiT FID, 200 epochs ↓
Native43.1544.2043.1544.20
REPA-MAE, ViT-L35.2635.8629.9134.94
SRA36.5245.9234.8243.71
Self-Flow34.1154.7827.6938.46
RepFlow-B29.7833.3127.3833.85
RepFlow-L31.5634.5227.8631.19

Both RepFlow variants outperform pretrained MAE guidance on both backbones at matched time and at 200 epochs. The joint-training budget includes learning the encoder.

Table Q2. Continued interaction after a shared warm-up. Unconditional SiT RepFlow-B starts from the same 50-epoch checkpoint and continues for another 50 epochs.

Training stateGenerator FID ↓Student ImageNet LP (%) ↑
Shared warm-up, 50 epochs61.4346.8
Freeze student; continue generator training41.19—
Freeze generator; continue student training—52.8
Continue updating both40.4153.9

Continuing to update both models improves FID by 0.78 and student linear-probe accuracy by 1.1 percentage points over the respective frozen-teacher controls.

AuthorsOpenReview ↗

## Response to Reviewer QfXv (W2)

W2: Advantages and intended use cases

RepFlow unifies generation and representation learning through reciprocal supervision. Generator states train a timestep-free image encoder, whose features then guide generator updates. The same training process produces a generator and a separately usable encoder without an externally pretrained representation teacher.

Its generation benefits hold across unconditional and class-conditional generation in both latent and pixel spaces, with 19.2–40.8% lower FID than Native (submitted paper, Sec. 4.2 and Table 2). Table Q1 adds comparisons at matched training time. The differences from the alternatives are:

  • REPA: DINOv2-based REPA achieves lower conditional FID in the submitted paper, but requires a pretrained representation teacher. RepFlow learns its teacher within generator training; the MAE-based comparison below evaluates the effectiveness of this learned supervision.
  • SRA: It remains competitive on conditional SiT, but gives weaker results than RepFlow on conditional JiT and both unconditional backbones. On unconditional JiT, SRA gives 45.92 FID at matched time versus Native's 44.20, and only a small gain at 200 epochs (43.71 versus 44.20).
  • Self-Flow: Its lower throughput limits progress at fixed training time. In our runs, throughput is 1.20/0.93 steps/s on SiT/JiT, compared with RepFlow-B's 1.82/2.70. It improves over Native at 200 epochs, while RepFlow gives the best results under both comparisons (Table Q1).

RepFlow is therefore useful when both a generator and a transferable encoder are needed, without separately pretraining a representation teacher.

Under the same pretraining budget of 800 epochs, RepFlow exceeds MAE by 24.21 and 20.46 percentage points on zero-shot SPair-71k correspondence (Table Q3). MAE has higher reported ImageNet linear-probe accuracy, while RepFlow has a clear advantage in spatial correspondence. Full RepFlow also improves over G2R-only on SPair-71k and Flowers102. Both evaluations use frozen encoders; SPair-71k requires no task-specific training, and Flowers102 uses a linear classifier.

Table Q3. Transfer of frozen representations. All SPair-71k encoders use the same 800-epoch pretraining budget; RepFlow uses SiT ViT-L student EMA checkpoints. Scores are percentages; pp denotes percentage points.

EncoderSPair-71k [email protected] ↑SPair-71k [email protected] ↑Flowers102 linear probe ↑
Public MAE ViT-L18.1411.14—
RepFlow-L, G2R-only41.8930.1776.17
RepFlow-L, full reciprocal supervision42.3531.6077.05
Full − G2R-only, percentage points+0.46+1.43+0.88
Full − MAE, percentage points+24.21+20.46—

The REPA-MAE comparison tests how effectively these representations supervise generator training. With ViT-L encoders in both methods, RepFlow-L reduces FID from 35.26 to 31.56 on SiT and from 35.86 to 34.52 on JiT at matched training time (Table Q1). At 200 epochs, FID falls from 29.91 to 27.86 on SiT and from 34.94 to 31.19 on JiT. These results favor jointly learned, evolving supervision over a fixed pretrained MAE teacher. The matched-time budget includes encoder learning.

The original one-step experiment in Table 3 of the submitted paper also shows that the encoder remains useful after it is frozen: from the same RepFlow-JiT initialization, it gives better FID, FDr6, and IS than MAE features. RepFlow's representation advantages therefore span spatial correspondence, supervision during generator training, and frozen-encoder reuse for post-training. We see this reuse across tasks and objectives as a promising direction for unified generation and representation learning.

AuthorsOpenReview ↗

## Response to Reviewer QfXv (W3 and Q2)

W3 and Q2: One-step post-training and prior co-adaptation

We added Native JiT and REPA-MAE JiT initializations, both post-trained with the same MAE-based FD loss. In the REPA-MAE control, MAE guides both generator pretraining and post-training, so the comparison includes prior interaction with the encoder in both complete training pipelines. Table Q4 combines these new controls with the two RepFlow-JiT results from Table 3 of the submitted paper.

Table Q4. New controls and original results for class-conditional one-step JiT post-training. All initializations use 400 pretraining epochs followed by 30 post-training epochs; evaluation uses NFE = 1 and CFG = 1. The final row is a calculated difference.

Generator initializationFrozen FD encoderResult sourceFID ↓FDr6 ↓IS ↑
Native JiTMAE ViT-LNew7.6415.06267.9
REPA-MAE JiTMAE ViT-LNew6.4913.82276.7
RepFlow-JiTMAE ViT-LSubmitted paper, Table 36.349.33286.1
RepFlow-JiTRepFlow ViT-LSubmitted paper, Table 33.148.06312.8
Full pipeline vs. REPA-MAE pipeline, change—Calculated difference−3.35 (−51.6%)−5.76+36.1

With MAE fixed as the FD encoder, RepFlow initialization gives the best results among the three initializations. Replacing MAE with the RepFlow encoder then improves all three metrics. The complete RepFlow pipeline reduces FID from 6.49 to 3.14 relative to REPA-MAE, with improvements in FDr6 and IS as well.

This compares a pipeline guided by MAE in both stages with reciprocal training followed by reuse of the learned encoder. FDr6 is evaluated in six feature spaces, none of which is RepFlow's FD encoder.

Reviewer_KdBaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RepFlow jointly trains a flow generator and a separate image encoder without an externally pretrained representation teacher. The encoder learns from masked clean images to predict generator features across several layers and noise levels. Its clean-image features then guide the generator through representation alignment, forming a feedback loop between the two models. On ImageNet, the method improves generation and linear-probe accuracy for both latent-space SiT and pixel-space JiT. Ablations examine the feedback path and prediction targets. The learned encoder is also reused for one-step JiT post-training, where it improves FID over a matched MAE-based loss.

Critical strengths
  1. The feedback ablation is convincing: enabling R2G improves both FID and student linear probing on SiT and JiT (Table 1). The pixel-target and single-depth controls provide further evidence for the proposed training loop.

  2. The learned representation remains useful after joint training. With the same one-step JiT initialization and 30-epoch budget, its FD loss reduces FID from 6.34 to 3.14 compared with MAE (Table 3), providing useful validation beyond linear probing.

Critical weaknesses
  1. Representation quality is evaluated only through ImageNet linear probing. Raw generator features and the pixel-target ablation are useful controls, but a comparison with a standard self-supervised encoder under matched training and evaluation would better establish the value of the learned representation.

  2. SRA is compared only in the conditional setting, although the largest gains are reported for unconditional generation. RepFlow also has slightly worse conditional SiT FID than SRA (6.99 vs. 6.77) and adds a large student. Its advantage over existing self-guidance at comparable cost remains unclear.

Questions for authors
  1. How does the student compare with a standard MAE ViT-L using the same training budget and linear-probe protocol? Do the gains over G2R-only also hold on a transfer dataset?

  2. Can you report unconditional SRA results on SiT and JiT at matched wall-clock budgets? Appendix C.5 matches RepFlow against Native, but does not establish this comparison.

2 comments
AuthorsOpenReview ↗

## Response to Reviewer KdBa (W1 and Q1)

Thank you for asking for a broader evaluation of the encoder and a cost-matched SRA comparison. We added transfer evaluations, a freezing control, and unconditional generation results on both backbones.

W1 and Q1: Standard MAE comparison and transfer beyond ImageNet

MAE is stronger on reported ImageNet linear probing, while RepFlow learns representations that are effective for spatial correspondence and generation. We compare RepFlow-L and MAE ViT-L under the same pretraining budget of 800 epochs on SPair-71k, using frozen encoders without task-specific training. We also evaluate transfer to Flowers102 with a linear classifier.

Table K1. Representation transfer. All SPair-71k encoders are pretrained for 800 epochs; RepFlow uses SiT ViT-L student EMA checkpoints. All scores are percentages; pp denotes percentage points.

EncoderSPair-71k [email protected] ↑SPair-71k [email protected] ↑Flowers102 linear probe ↑
Public MAE ViT-L18.1411.14—
RepFlow-L, G2R-only41.8930.1776.17
RepFlow-L, full reciprocal supervision42.3531.6077.05
Full vs. G2R-only, percentage points+0.46+1.43+0.88
Full vs. MAE, percentage points+24.21+20.46—

RepFlow exceeds MAE by 24.21 and 20.46 percentage points on SPair-71k without task-specific training. The gains over G2R-only also hold on both transfer datasets: full RepFlow improves both PCK metrics and Flowers102 accuracy.

The REPA-MAE comparison uses the same MAE baseline to assess how effectively the representations supervise generator training. With ViT-L encoders in both methods, RepFlow-L reduces unconditional FID from 35.26 to 31.56 on SiT and 35.86 to 34.52 on JiT at matched training time (Table K3). At 200 epochs, FID falls from 29.91 to 27.86 on SiT and from 34.94 to 31.19 on JiT. Both comparisons favor jointly learned, evolving supervision over fixed MAE features.

We also test whether continued reciprocal updates help after a shared warm-up:

Table K2. Unconditional SiT RepFlow-B: shared 50-epoch warm-up followed by 50 epochs of continuation.

Training stateGenerator FID ↓Student ImageNet LP (%) ↑
Shared warm-up, 50 epochs61.4346.8
Freeze student; continue generator training41.19—
Freeze generator; continue student training—52.8
Continue updating both40.4153.9

Updating both models improves FID by 0.78 and student linear-probe accuracy by 1.1 percentage points over the respective frozen-teacher controls.

Frozen-encoder reuse is supported by Table 3 of the submitted paper: starting from the same 400-epoch RepFlow-JiT initialization and using 30 epochs of class-conditional post-training at NFE = 1, replacing MAE with the RepFlow encoder reduces FID from 6.34 to 3.14 and improves FDr6 and IS. The new REPA-MAE comparison extends this evidence to supervision during generator training, while SPair-71k evaluates spatial correspondence. These are complementary strengths beyond ImageNet linear probing.

AuthorsOpenReview ↗

## Response to Reviewer KdBa ( W2 and Q2)

W2 and Q2: Generation benefits across settings and comparison with SRA

All methods use unconditional ImageNet at 256 × 256 on eight H200 GPUs. Each budget equals Native's 200-epoch training time: approximately 33.3 hours for SiT and 22.1 hours for JiT. RepFlow's budget includes its encoder; MAE's prior pretraining cost is excluded. We report equal-epoch results separately.

Table K3. Unconditional FID-50K: matched wall-clock budgets and 200-epoch endpoints.

MethodSiT FID, ≈33.3 h ↓JiT FID, ≈22.1 h ↓SiT FID, 200 ep ↓JiT FID, 200 ep ↓
Native43.1544.2043.1544.20
REPA-MAE, ViT-L35.2635.8629.9134.94
SRA36.5245.9234.8243.71
Self-Flow34.1154.7827.6938.46
RepFlow-B29.7833.3127.3833.85
RepFlow-L31.5634.5227.8631.19

RepFlow-B gives the best equal-time result on both backbones. Both RepFlow variants outperform SRA at matched training time and at the fixed-epoch endpoint.

SRA's gains vary across settings. In the submitted paper, it has slightly lower conditional SiT FID than RepFlow (6.77 versus 6.99), but higher conditional JiT FID (14.24 versus 12.65). On unconditional JiT, it gives 45.92 FID at matched time versus Native's 44.20, and only a small gain at 200 epochs (43.71 versus 44.20). RepFlow improves both conditioning regimes and both generation spaces (submitted paper, Sec. 4.2 and Table 2), while also providing the transferable encoder evaluated in Table K1.

Reviewer_ytzjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • RepFlow jointly trains a generator and a separate representation encoder.
  • G2R trains the encoder to predict multi-layer generator features from masked images. R2G then uses the encoder’s features to guide the generator.
  • Experiments on SiT and JiT show improvements in generation quality and linear probing. The learned encoder is also used for one-step generation post-training.
Critical strengths

The method introduces a separate encoder and connects two supervision directions, which works with both latent-space SiT and pixel-space JiT. The authors also report gains over naive training under matched wall-clock budgets. This suggests that the improvements are not solely due to additional training time. The learned encoder can also be used independently for representation extraction.

Critical weaknesses
  • Since Self-Flow also improves generation and representation learning without an external teacher, RepFlow should compare with Self-Flow.

  • The training cost is not low. RepFlow-L adds a 303M-parameter encoder and prediction modules. Reported training time increases by roughly 28%–59%. Yet it does not consistently outperform REPA or SRA. The paper does not establish that these additional costs offer better results under matched training budgets.

Questions for authors
  • Could the authors compare RepFlow-B, RepFlow-L, REPA, SRA, and Self-Flow using the same backbone and wall-clock budget?
  • Is continually updating the supervision necessary? Could the authors compare a frozen teacher with a continuously updated teacher after a shared warm-up stage?
4 comments
AuthorsOpenReview ↗

## Response to Reviewer ytzj (W1, W2, and Q1)

Thank you for the questions on training cost and continued supervision. We added comparisons with Self-Flow, REPA-MAE, and SRA under the same training-time budgets, a shared-warm-up freezing control, and representation transfer evaluations.

W1, W2, and Q1: Training cost and comparison with Self-Flow

All methods use unconditional ImageNet at 256 × 256 and eight H200 GPUs, with the generator backbone held fixed within each comparison. We set each budget to Native's 200-epoch training time: approximately 33.3 hours for SiT and 22.1 hours for JiT. RepFlow's budget covers both models; REPA-MAE uses an existing MAE ViT-L checkpoint whose pretraining cost is excluded.

For a fairer comparison, we use MAE ViT-L as the REPA teacher to evaluate against the same ViT student employed in our representation evaluation.

Table Y1. Unconditional FID-50K at the matched wall-clock budgets above. Throughput is measured in training steps per second.

MethodSiT FID ↓SiT steps/s ↑JiT FID ↓JiT steps/s ↑
Native43.152.0944.203.15
REPA-MAE, ViT-L35.261.7835.862.64
SRA36.521.7945.922.70
Self-Flow34.111.2054.780.93
RepFlow-B29.781.8233.312.70
RepFlow-L31.561.6334.522.12

RepFlow-B gives the lowest FID on both backbones within these budgets. RepFlow-L also outperforms all four baselines, although the smaller student gives better generation results at this training time.

Table Y2. Unconditional FID-50K at equal training epochs. Each cell is SiT / JiT.

Method50 epochs100 epochs150 epochs200 epochs
Native70.30 / 72.3053.09 / 54.5447.31 / 47.5643.15 / 44.20
REPA-MAE, ViT-L63.61 / 49.5846.09 / 39.1938.44 / 36.4629.91 / 34.94
SRA67.06 / 71.0048.14 / 54.2639.61 / 47.2234.82 / 43.71
Self-Flow64.12 / 63.5137.10 / 41.5131.54 / 39.4627.69 / 38.46
RepFlow-B61.43 / 60.1440.41 / 38.9532.76 / 34.8927.38 / 33.85
RepFlow-L60.30 / 59.8740.13 / 37.9232.10 / 32.5127.86 / 31.19

At 200 epochs, RepFlow-B gives the best SiT result and RepFlow-L the best JiT result. Self-Flow also improves over Native at equal epochs, but its lower throughput limits progress within the fixed time budget, especially on JiT: 0.93 steps/s versus RepFlow-B's 2.70.

The REPA-MAE comparison also tests the representations as supervision for generation. With ViT-L encoders in both methods, RepFlow-L gives lower FID on both backbones at matched time and at 200 epochs (Tables Y1–Y2). The jointly learned, evolving supervision is more effective here than a fixed pretrained MAE teacher.

These comparisons add training-cost controls to the results in the submitted paper (Sec. 4.2 and Table 2), where RepFlow-L reduces FID by 19.2–40.8% relative to Native across unconditional/class-conditional and latent/pixel generation.

AuthorsOpenReview ↗

## Response to Reviewer ytzj (Q2)

Q2: Continued supervision after a shared warm-up

All branches start from the same 50-epoch unconditional SiT RepFlow-B checkpoint and continue for another 50 epochs. We separately freeze the student supplying R2G supervision or the generator supplying G2R supervision.

Table Y3. Shared warm-up followed by frozen or continuously updated supervision.

Training stateGenerator FID ↓Student ImageNet LP (%) ↑
Shared warm-up, 50 epochs61.4346.8
Freeze student; continue generator training41.19—
Freeze generator; continue student training—52.8
Continue updating both40.4153.9

A frozen teacher still supports further learning. Continuing to update both models improves FID by 0.78 and student linear-probe accuracy by 1.1 percentage points over the respective frozen-teacher controls. We also added frozen-encoder evaluations on two transfer datasets:

Table Y4. Transfer of frozen representations. All SPair-71k encoders use the same 800-epoch pretraining budget; RepFlow uses SiT ViT-L student EMA checkpoints. SPair-71k is evaluated without task-specific training; Flowers102 uses linear probing. Scores are percentages.

EncoderSPair-71k [email protected] ↑SPair-71k [email protected] ↑Flowers102 linear probe ↑
Public MAE ViT-L18.1411.14—
RepFlow-L, G2R-only41.8930.1776.17
RepFlow-L, full reciprocal supervision42.3531.6077.05

Full RepFlow improves over G2R-only on both transfer datasets. On SPair-71k, it also exceeds MAE by 24.21 and 20.46 percentage points on the two PCK metrics.

Thank the authors for adding the wall-clock comparisons, including Self-Flow. These results address but of my concern about RepFlow’s training efficiency on ImageNet.

I wonder do the authors have any preliminary evidence that the gains carry over to text-to-image generation under same training budgets? (to show some scaling ability)

AuthorsOpenReview ↗

## Response to Reviewer ytzj

Thank you for the suggestion. We have now conducted text-to-image experiments on MS-COCO 2014, comparing native MMDiT with MMDiT + RepFlow. Both generators are trained from scratch for 150k updates with a batch size of 256 at 256×256 resolution, using the same generator architecture and sampling settings. We evaluate generation quality using FID-30K and image–text alignment using CLIP score.

Training updatesMMDiT FID ↓+ RepFlow FID ↓MMDiT CLIP ↑+ RepFlow CLIP ↑
50k15.314.928.528.6
60k12.011.628.828.9
70k10.39.929.029.1
80k9.39.029.129.2
90k8.78.429.229.3
100k8.17.829.329.3
110k7.87.529.329.4
120k7.67.329.329.4
130k7.57.329.329.4
140k7.37.229.429.5
150k7.37.129.429.5

RepFlow achieves lower FID at every evaluated checkpoint while matching or improving CLIP score. At 150k updates, it improves FID from 7.3 to 7.1 and CLIP score from 29.4 to 29.5. The advantage also becomes more evident in the final stage of training. From 140k to 150k updates, native MMDiT remains at an FID of 7.3, while RepFlow further improves from 7.2 to 7.1, widening the gap. This continued improvement motivates exploring RepFlow in future work with longer training schedules, larger text-to-image models, and larger datasets.

Building on these findings, we believe that scaling RepFlow to larger text-to-image models and datasets is a promising direction for future research. Further details are provided in Appendix C5.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.