acceptodds
Under review as a conference paper at ICLR 2027

GRAFT: Growing Agglomerative Foundation Models via Continual Teacher Distillation

Abstract

Vision foundation models such as DINOv2, SigLIP2, and MASt3R develop complementary capabilities from different pretraining objectives, yet their knowledge remains distributed across separate, specialized models. Multi-teacher knowledge distillation offers a path toward consolidating these capabilities into a single agglomerative backbone, but existing approaches assume a fixed set of teachers, and incorporating a new teacher requires repeating expensive joint distillation over the entire teacher set. We introduce GRAFT, a continual multi-teacher distillation framework that enables a unified backbone to progressively acquire capabilities from an open-ended sequence of foundation models. When a new teacher arrives, GRAFT treats the previously distilled model as a teacher for preserving learned capabilities, while the current student jointly learns from both the previous model and the incoming teacher. Furthermore, to reconcile the incompatible representation geometries of heterogeneous teachers, we introduce Teacher Specific Readout Tokens, which grant each teacher an independent read-out of the shared encoder, together with Geometry Agnostic Relational Loss that aligns a vision–language teacher by matching image–text similarity structures rather than raw feature values. We provide GRAFT model, which is a single, continually extensible backbone that unifies five domains, including image understanding, 2D dense prediction, 3D human pose estimation, 3D vision, and vision–language, delivering strong performance across all of them while acquiring each new capability at the cost of a single distillation rather than a full re-distillation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Knowledge distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_gpHCOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a new multi-teacher knowledge distillation framework called GRAFT, which is capable of collapsing capabilities of many different teacher networks into a single agglomerative backbone. The main novelty of the proposed GRAFT method, lies in its way of formulating multi-teacher knowledge distillation in a continual learning setting. The authors suggest to treat the previously distilled model as a teacher for the current student, keeping its learned capabilities while letting the student jointly learn from both the previous model and the new teacher. Experimental results reveal that this framework leads an efficient way of transferring the capabilities of a new teacher model to the agglomerative model in a progressive manner, without forgetting its earlier learned capabilities.

Critical strengths

S1. Formulating multi-teacher knowledge distillation framework in a continual learning is an interesting idea. It allows for transferring the capabilities of different foundation models to a much simpler agglomerative model in a progressive manner. Experiments demonstrate that with the suggested strategy, a single agglomerative model become capable of acquiring capabilities of a number of foundational models to a certain extent.

S2. In the continual learning setting of the proposed framework, the previously distilled model is treated as a teacher for the current student, which seems an efficient way of transferring new capabilities to the agglomerative model, while preserving its learned capabilities. It eliminates the need for repeating expensive joint distillation over the entire set of teacher foundation models.

S3. To harmonize the heterogeneous representation geometries as suggested by different teacher foundation models, the authors propose to utilize learnable teacher-specific readout tokens and a geometry-agnostic relational loss. Ablation studies show the effectiveness of these contributions, specifically on transferring capabilities of a vision–language teacher.

Critical weaknesses

W1. Since the competing multi-teacher knowledge distillation consider backbones and different set of teachers other than the ones GRAFT considers, the experimental results do not reflect a controlled comparison of the methods.

W2. Experiments demonstrate that transferring new capabilities from a new teacher does not fully preserve the previously learned capabilities that there could be some performance drops for some tasks.

W3. The paper lacks a discussion and comparison with the recently introduced C-RADIOv4 (Ranzinger et al., 2026) multi-teacher distillation method.

Questions for authors

Q1. The proposed GRAFT method enables a progressive learning mechanism of different capabilities. How is the performance of the agglomerative model affected when the order of the teacher models are changed in the continual learning setting?

Q2. For a more controlled comparison, would it be possible to train the competing multi-teacher distillation models by considering all the teacher foundations model that GRAFT is trained on?

3 comments

We thank the reviewer for the constructive suggestions. We have conducted additional experiments to examine controlled joint distillation, teacher-order sensitivity, and the applicability of GRAFT to an independently pre-trained agglomerative encoder.

W1 / Q2. Controlled comparison. We additionally trained a joint distillation baseline using the same four teachers (DINOv2, Multi-HMR, MASt3R, and SigLIP2), ViT-B student architecture, Teacher Specific Readout Tokens, and Geometry Agnostic Relational Loss as GRAFT.

MethodIN-1K ↑Depth ↓mIoU ↑F1 ↑PA-PVE ↓AUC ↑COCO T→I / I→T ↑Flickr T→I / I→T ↑
GRAFT S4 (sequential)81.20.44146.08978.181.851.9 / 69.076.2 / 88.4
GRAFT from DUNE81.10.43746.08975.382.452.3 / 69.776.8 / 89.7
Joint distillation81.60.43946.58971.783.053.2 / 71.177.0 / 89.9

GRAFT achieves competitive performance against joint distillation across the evaluated capabilities. The practical advantage is incremental efficiency: incorporating a new teacher into an existing DUNE encoder with GRAFT requires approximately 370 GPU hours, compared with 1,690 GPU hours for the reported four-teacher joint distillation.

W2. Retention of previous capabilities. We thank the reviewer for raising the important question of capability retention. A central objective of GRAFT is to mitigate the interference between acquiring a new heterogeneous capability and preserving previously consolidated representations.

Table 2 compares GRAFT with frozen-encoder adaptation, direct distillation, EWC, and component ablations. GRAFT achieves the strongest overall balance between acquiring new capabilities and retaining previous ones, with 81.2% ImageNet-1K accuracy and 46.0 ADE20K mIoU, compared with 77.1% / 40.2 for EWC and 82.6% / 35.4 for direct distillation. Moreover, its performance remains close to four-teacher joint distillation.

W3. C-RADIOv4 and cross-backbone applicability. We thank the reviewer for highlighting C-RADIOv4. We will include both C-RADIOv3 and C-RADIOv4 in our empirical comparison and clarify differences in teacher composition, model capacity, training protocols, and evaluation settings. Importantly, the smallest C-RADIOv4 variant, SO400M, contains 412M parameters, substantially more than the ViT-Base backbones (approximately 86M parameters) used by the agglomerative baselines in Table 1.

Furthermore, we demonstrate that even C-RADIO models could be extended by GRAFT method. To test this, we apply GRAFT to a pre-trained C-RADIOv3-B backbone (90M parameters) to incorporate 3D human mesh recovery (HMR) capabilities from the Multi-HMR teacher.

EncoderIN-1K CLS ↑ADE20K mIoU ↑HMR F1 ↑PA-PVE (mm) ↓PVE (mm) ↓
Multi-HMR (teacher)--95.036.957.5
C-RADIOv3-B82.6250.51---
C-RADIOv3-B + GRAFT (Multi-HMR)81.6450.1194.1457.590.9

We compare the original C-RADIOv3-B and our GRAFT extension, evaluating both the preservation of previously acquired capabilities and the acquisition of human reconstruction capabilities. The additional experiment improves human mesh recovery while largely retaining the encoder's original classification and segmentation performance. It further demonstrates that GRAFT can extend an independently pre-trained agglomerative model, rather than relying on a particular initialization or incoming teacher.

Q1. Teacher order. We additionally evaluated both arrival orders of DINOv2 and SigLIP2.

MethodIN-1K ↑mIoU ↑Depth RMSE ↓
DINOv2 only-45.50.420
DINOv2 → SigLIP281.246.00.450
SigLIP2 only82.639.80.535
SigLIP2 → DINOv281.545.20.469

The above table presents two separate experiments to assess sensitivity to teacher order. The first pair of rows (rows 1–2) shows starting with DINOv2 and adding SigLIP2, which gains classification ability while preserving dense segmentation and depth performance. The second pair (rows 3–4) reverses this sequence, starting with SigLIP2 and adding DINOv2. This results in substantial gains in segmentation and depth with minimal classification drop. Despite minor metric fluctuations between the two trajectories, both sequences achieve comparable overall performance. This suggests that GRAFT is not strongly sensitive to teacher order in the evaluated setting.

Thank you for your response.

The additional controlled experiments successfully address my concerns regarding a fair comparison and the ordering of the teachers. I also appreciate the inclusion of comparisons against C-RADIOv3 models, which makes the evaluation much more comprehensive.

However, I noticed that a revised version of the paper addressing the reviewers' concerns has not yet been uploaded. I am inclined to raise my rating, but I would like to see the updated manuscript before doing so.

Thanks for your constructive and timely feedback. We have revised the manuscript to address the comments and incorporated the additional experiments, analyses, and discussions as follows:

  1. Practical contribution: We strengthened the discussion of GRAFT's practical impact and computational efficiency in the Appendix A and highlighted this contribution in the Introduction.
  2. Joint distillation: We added a discussion comparing GRAFT with joint distillation in Section 4.2.
  3. Additional baselines: We included C-RADIOv4 and joint distillation results in Table 1, along with further discussion of C-RADIOv4 in the analysis accompanying Table 3.
  4. Generality of GRAFT: We added experiments extending C-RADIOv3 with new capabilities to demonstrate GRAFT's applicability to independently pre-trained agglomerative models (Appendix C).
  5. Teacher order sensitivity: We added experiments investigating the effect of teacher order on performance and knowledge retention (Appendix C).
  6. Data composition: We provided additional details on the training data composition for each distillation stage (Appendix D).

Please feel free to raise any further concerns. We would be happy to address them.

Reviewer_UXHeOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how an already-distilled vision foundation model can continue to absorb new teachers that appear only after its training has finished. To this end, the paper proposes GRAFT, which initializes the current student from the student of the previous round and freezes that previous student as an anchor, acquiring knowledge from the incoming teacher while keeping the outputs of previously learned tasks close to the anchor, so that old teachers need neither be stored nor re-run. To handle the heterogeneity among teachers, each teacher has its own learnable readout on top of the shared patch representation, and knowledge from the vision-language teacher is transferred through the pattern of image-text similarities rather than by matching embedding coordinates directly. In the experiments, four types of teachers are grafted one after another onto the same backbone, and the final model acquires new capabilities while retaining the old ones.

Critical strengths

1.The paper extends model aggregation from merging a fixed set of teachers to a setting where teachers arrive sequentially. By using previously trained students as proxies for historical teachers, it avoids the growth in the number of reference backbones as more teachers are integrated, making the problem itself clearly worth studying.

  1. Empirically, the paper sequentially integrates multiple heterogeneous capabilities, including dense 2D features, human mesh reconstruction, 3D geometry, and vision-language alignment, into a single ViT-B backbone. Ablation results show that adding TSRT and relational objectives substantially improves the trade-off between acquiring new capabilities and retaining existing representations.
Critical weaknesses

The overall novelty is moderate; the core contribution lies in extending continual distillation to heterogeneous teacher settings.

Questions for authors

1.What is the exact composition of the training dataset at each grafting stage? Do subsequent training rounds reuse images that belong to earlier teacher domains?

2.Could you provide a joint re-distillation baseline under the same teacher set, the same student model, and a comparable optimization budget, and report both the performance gap and the computational costs of the two approaches?

3.How sensitive is the method to the order in which teachers arrive, particularly when vision-language or 3D-geometry teachers are placed earlier in the sequence? When the previous student model is continuously used as an anchor, do approximation errors accumulate over longer sequences than those evaluated in the current experiments?

4 comments

W1. Novelty and contribution

We would like to highlight key aspects and contributions of GRAFT, supported by additional experiments.

1. Computational efficiency. GRAFT enables an existing agglomerative model to incorporate a new teacher without repeating joint distillation over the entire teacher ensemble. For example, extending DUNE with SigLIP2 requires only 370 GPU hours, compared with 1,690 GPU hours for joint distillation, saving approximately 1,320 GPU hours.

Model or training workloadGPU hours
PHi-S14,080
AM-RADIO17,408
DUNE1,372
Joint distillation of GRAFT teachers1,690
GRAFT (per stage)370

2. Generality of GRAFT. We additionally applied it to a pre-trained C-RADIOv3-B model, incorporating Multi-HMR as a new teacher.

EncoderIN-1K CLS ↑ADE20K mIoU ↑HMR F1 ↑PA-PVE (mm) ↓PVE (mm) ↓
Multi-HMR (teacher)--95.036.957.5
C-RADIOv3-B82.6250.51---
C-RADIOv3-B + GRAFT (Multi-HMR)81.6450.1194.1457.590.9

GRAFT successfully integrates strong human mesh recovery (HMR) capabilities while retaining the vast majority of the backbone’s original classification and segmentation performance. These results demonstrate that GRAFT generalizes well across distinct pre-trained agglomerative backbones, effectively mitigating catastrophic forgetting during capability expansion.

3. Differences from traditional continual learning. GRAFT extends continual distillation to heterogeneous foundation models with different representation spaces and downstream capabilities. Teacher Specific Readout Tokens and Geometry Agnostic Relational Loss enable new teacher representations to be integrated into a shared backbone while preserving previously acquired capabilities. As shown in Table 2, GRAFT achieves 81.2% ImageNet accuracy and 46.0 mIoU, compared with 77.1% and 40.2 for EWC. These results demonstrate the effectiveness of our design for continual heterogeneous representation aggregation.

Q1. Training data composition. Our training data are organized into three groups:

  • G1: DataComp-12M, ImageNet-19K, Mapillary Street-level Sequences, and GLDv2.
  • G2: BEDLAM, AGORA, CUFFS, and UBody.
  • G3: ARKitScenesV2, BlendedMVS, CO3Dv2, DL3DV, Habitat512, MegaDepthDense, Niantic, TartanAir, Unreal4K, VirtualKITTI, and WildRGB.

S1 uses G1, S2 uses G1:G2 at a 1:1 ratio, and S3/S4 use G1:G2:G3 at a 1:1:1 ratio. Sampling is balanced across active groups rather than individual datasets. Images from earlier groups continue to be sampled in subsequent stages, while previous teacher models are not revisited. Both the incoming teacher and the frozen previous student supervise the mixed training batches. We will clarify this protocol in the revised paper.

Q2. Joint distillation comparison. We additionally trained a joint distillation baseline using the same four teachers (DINOv2, Multi-HMR, MASt3R, and SigLIP2), ViT-B student architecture, Teacher Specific Readout Tokens, and Geometry Agnostic Relational Loss.

MethodIN-1K ↑Depth ↓mIoU ↑F1 ↑PA-PVE ↓AUC ↑COCO T→I / I→T ↑Flickr T→I / I→T ↑
GRAFT S4 (sequential)81.20.44146.08978.181.851.9 / 69.076.2 / 88.4
GRAFT from DUNE81.10.43746.08975.382.452.3 / 69.776.8 / 89.7
Joint distillation81.60.43946.58971.783.053.2 / 71.177.0 / 89.9

The sequential model achieves competitive performance across these capabilities. Joint distillation requires approximately 1,690 GPU hours, whereas incorporating an additional teacher into an existing backbone with GRAFT requires approximately 370 GPU hours. This highlights the computational advantage of incremental integration without repeating full joint distillation. The reported costs describe these respective training workloads rather than a comparison of equal total training budgets.

Q3. Teacher order and error accumulation. We conducted an additional experiment reversing the distillation order of DINOv2 and SigLIP2.

SettingIN-1K ↑mIoU ↑Depth RMSE ↓
DINOv2 onlyN/A45.50.420
DINOv2 → SigLIP281.246.00.450
SigLIP2 only82.639.80.535
SigLIP2 → DINOv281.545.20.469

The above table presents two separate experiments to assess sensitivity to teacher order. The first pair of rows (rows 1–2) shows starting with DINOv2 and adding SigLIP2, which gains classification ability while preserving dense segmentation and depth performance. The second pair (rows 3–4) reverses this sequence, starting with SigLIP2 and adding DINOv2. This results in substantial gains in segmentation and depth with minimal classification drop. Despite minor metric fluctuations between the two trajectories, both sequences achieve comparable overall performance. This suggests that GRAFT is not strongly sensitive to teacher order in the evaluated setting.

Reviewer_UXHeOpenReview ↗

The author has addressed our major concerns

The author has addressed our major concerns. No more questions.

We sincerely thank the reviewer for the constructive suggestions and for engaging in the discussion. We are pleased that our additional experiments and clarifications have addressed your major concerns.

We have incorporated these updates into the revised manuscript, including the controlled joint distillation comparison (Section 4.2 and Table 1), teacher order sensitivity and retention analysis (Appendix C), and detailed training data composition (Appendix D).

We greatly appreciate your thoughtful feedback, which has helped us strengthen the paper. We hope that the additional evidence and manuscript revisions will be reflected in your final assessment of our work.

Reviewer_tZePOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary
  • The paper considers a scenario where we want to continually train a single model to perform various CV tasks. The assumption is that we only have access to the current model and a new foundation model with new capabilities and we update the current model by distilling from the new one.
  • The authors propose to use two terms in the loss function: a standard distillation loss and an additional loss which prevents forgetting skills learned from previous models.
  • The results show that the approach works although some forgetting still happens and the final model does not outperform existing agglomerative baselines.
Critical strengths
  • The paper is easy to follow.
  • The method seems to deliver what it was designed to do.
Critical weaknesses
  • The core idea is well known in the continual learning literature.
  • The setup considered in the paper seems uncommon, in most cases one has access to old models if they are open-weights. So, a more realistic scenario compared to “grow a foundation model continually” could be “cheaply add a capability to an agglomerative model you didn’t train.” But if the paper is formulated that way, the contribution would seem even smaller.
  • If I understood correctly, the main GRAFT model (the last row in Table 1) was trained using DUNE as the starting point and DUNE was trained to distill from the three of the four models. If this is true, then the model presented as "ours" in Table 1 is trained with only one step, which is not a continual learning scenario.
Questions for authors

None

5 comments

We thank the reviewers for their constructive feedback. We would like to highlight key aspects and contributions of GRAFT, supported by additional experiments.

1. Computational efficiency. GRAFT enables an existing agglomerative model to incorporate a new teacher without repeating joint distillation over the entire teacher ensemble. For example, extending DUNE with SigLIP2 requires only 370 GPU hours, compared with 1,690 GPU hours for joint distillation, saving approximately 1,320 GPU hours.

Model or training workloadGPU hours
PHi-S14,080
AM-RADIO17,408
DUNE1,372
Joint distillation of GRAFT teachers1,690
GRAFT (per stage)370

2. Generality of GRAFT. We additionally applied it to a pre-trained C-RADIOv3-B model, incorporating Multi-HMR as a new teacher.

EncoderIN-1K CLS ↑ADE20K mIoU ↑HMR F1 ↑PA-PVE (mm) ↓PVE (mm) ↓
Multi-HMR (teacher)--95.036.957.5
C-RADIOv3-B82.6250.51---
C-RADIOv3-B + GRAFT (Multi-HMR)81.6450.1194.1457.590.9

GRAFT successfully integrates strong human mesh recovery (HMR) capabilities while retaining the vast majority of the backbone’s original classification and segmentation performance. These results demonstrate that GRAFT generalizes well across distinct pre-trained agglomerative backbones, effectively mitigating catastrophic forgetting during capability expansion.

W1. Method contribution.

GRAFT is designed for continual consolidation of heterogeneous foundation-model representations, rather than conventional task-level continual learning. At each stage, the frozen previously distilled student preserves accumulated capabilities, while an incoming teacher contributes a new representation space. Teacher Specific Readout Tokens provide independent readouts from the shared backbone, and Geometry Agnostic Relational Loss transfers image-text relations without enforcing direct feature-coordinate matching. The ablations in Table 2 support the value of these components beyond retention regularization alone. The new joint-distillation and C-RADIOv3 extension results further establish the practical efficiency and applicability of this setting.

W2. Practical motivation and contribution.

We thank the reviewer for highlighting two points that deserve clarification.

Why avoid revisiting old teachers when their weights are available? We do not assume that earlier open-weight models become inaccessible. Instead, not revisiting historical teachers is an intentional distillation protocol chosen to study a specific efficiency question: can an existing consolidated student absorb the next pre-trained foundation model without running the full historical teacher ensemble again? Public availability of model weights does not remove the computational cost of repeatedly evaluating them during joint re-distillation. The motivation is efficient aggregation, not a claim that old models are normally unobtainable.

Does this simply mean adding a capability to someone else's agglomerative model, and therefore a smaller contribution? Extending a third-party agglomerative backbone is one practical application, but it does not define or reduce the underlying research problem. Adding a downstream capability is not equivalent to growing a shared foundation representation. For example, one DINOv2 encoder already supports segmentation and depth through different downstream interfaces; an additional task head does not necessarily incorporate knowledge from another independently pre-trained foundation model. Agglomerative foundation models, introduced by prior work [1,2,3,4], aim to consolidate the complementary representations of multiple foundation models. Our contribution is to make that teacher integration continual under the no historical teacher revisit constraint, while managing interference in the shared backbone.

This distinction matters empirically. Table 2 shows that frozen-encoder adaptation retains dense prediction but acquires less of the incoming vision-language capability, whereas direct distillation improves acquisition but substantially degrades retention. GRAFT attains a stronger trade-off, using teacher-specific readouts and geometry-agnostic relational alignment to integrate heterogeneous knowledge. Moreover, Table 1 reports a fully sequential S1 to S4 teacher integration, so our setting is not limited to a single update of a third-party model.

We will clarify in the revision that continual, teacher-efficient aggregation of heterogeneous foundation representations is the methodological contribution; extending an existing agglomerative model is one of its applications.

W3. Sequential integration and generality across pretrained backbones.

Table 1 already reports a fully sequential GRAFT experiment: S1 (DINOv2), S2 (+ Multi-HMR), S3 (+ MASt3R), and S4 (+ SigLIP2). S4 is obtained through three successive grafting stages, with the previous student serving as the frozen reference at each stage. The final DUNE-initialized row is a separate experiment illustrating how GRAFT can extend an existing agglomerative model.

We further evaluated GRAFT on a different pretrained agglomerative backbone, C-RADIOv3-B, by incorporating Multi-HMR. Human reconstruction improves, while the original classification and segmentation capabilities remain strong. Together, these experiments demonstrate both multi-stage continual teacher integration and the ability to grow different pretrained VFM backbones. Our goal is to establish a general continual distillation approach for expanding VFM capabilities; DUNE is one initialization option, not a requirement. We will make this distinction clearer in the revision.

[1] Ranzinger, Mike, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. "Am-radio: Agglomerative vision foundation model reduce all domains into one." CVPR 2024.

[2] Shang, Jinghuan, Karl Schmeckpeper, Brandon B. May, Maria Vittoria Minniti, Tarik Kelestemur, David Watkins, and Laura Herlant. "Theia: Distilling diverse vision foundation models for robot learning." CoRL 2024.

[3] Sarıyıldız, Mert Bülent, Philippe Weinzaepfel, Thomas Lucas, Pau De Jorge, Diane Larlus, and Yannis Kalantidis. "Dune: Distilling a universal encoder from heterogeneous 2d and 3d teachers." CVPR 2025.

[4] Chaybouti, Sofian, Sanath Narayan, Yasser Dahou, Phúc H. Lê Khac, Ankit Singh, Ngoc Dung Huynh, Wamiq Reyaz Para, Hilde Kuehne, and Hakim Hacid. "SigLino: Efficient Multi-Teacher Distillation for Agglomerative Vision Foundation Models." CVPR 2026.

We would like to summarize the novelty and contributions of GRAFT as the rebuttal period approaches its conclusion. Our contributions lie in both methodological advances and practical efficiency.

Methodological contributions. We agree with the reviewer that using a previous model for knowledge retention is well established in continual learning. However, GRAFT addresses a distinct challenge: continually integrating heterogeneous vision foundation models into a unified encoder. Unlike conventional continual learning settings, the incoming teachers provide fundamentally different representations and capabilities, spanning semantic understanding, 2D and 3D perception, and vision-language alignment. To address the resulting representation conflicts, we introduce Teacher Specific Readout Tokens and a Geometry Agnostic Relational Loss. These components enable the student to accommodate heterogeneous teachers while preserving previously acquired capabilities. Their effectiveness is also supported by our ablation results in Table 2.

Practical contributions. Beyond methodological novelty, GRAFT provides an efficient way to extend existing vision foundation models without repeating expensive joint distillation. For example, extending DUNE with vision-language capabilities through joint distillation requires approximately 1,700 GPU hours, whereas GRAFT requires only 370 GPU hours. We further demonstrate that GRAFT is not limited to a particular initialization by successfully incorporating human pose estimation capabilities into C-RADIOv3.

We believe these contributions establish GRAFT as a practical and effective framework for continually extending agglomerative vision foundation models.

We have also revised the manuscript to address the comments and incorporated the additional experiments, analyses, and discussions as follows:

  1. Practical contribution: We strengthened the discussion of GRAFT's practical impact and computational efficiency in the Appendix A and highlighted this contribution in the Introduction.

  2. Joint distillation: We added a discussion comparing GRAFT with joint distillation in Section 4.2.

  3. Additional baselines: We included C-RADIOv4 and joint distillation results in Table 1, along with further discussion of C-RADIOv4 in the analysis accompanying Table 3.

  4. Generality of GRAFT: We added experiments extending C-RADIOv3 with new capabilities to demonstrate GRAFT's applicability to independently pre-trained agglomerative models (Appendix C).

  5. Teacher order sensitivity: We added experiments investigating the effect of teacher order on performance and knowledge retention (Appendix C).

  6. Data composition: We provided additional details on the training data composition for each distillation stage (Appendix D).

We would greatly appreciate the reviewer's thoughts on these clarifications and welcome further discussion, particularly regarding any remaining concerns of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.