acceptodds
Under review as a conference paper at ICLR 2027

PAFT: Auxiliary Bottlenecks Make Representations More Projectable

Abstract

Visual analysis of learned representations plays an important role in interpreting deep neural networks. Current practice predominantly focuses on applying post hoc dimensionality-reduction techniques, such as PCA, t-SNE, and UMAP. However, the strength and prevalence of these tools largely frame dimensionality reduction for visualization as a downstream stage, leaving the geometry learned during training unaddressed. In this work, we propose PAFT (Projection-Aware Fine-Tuning), a fine-tuning method built around an auxiliary branch consisting of a linear bottleneck and a classification head attached to the penultimate layer of a classifier. The classifier and the auxiliary branch are jointly trained with three additional terms in the training objective: prediction consistency between the main and auxiliary heads, together with CKA alignment and neighborhood preservation between the penultimate and bottleneck representations. During training, PAFT learns a linear projection from the penultimate representation to a low-dimensional bottleneck that preserves predictive and geometric structure, while simultaneously reshaping the high-dimensional representation itself so that its structure is more faithfully preserved in lower dimensions. At d = 2, this bottleneck also provides a directly inspectable two-dimensional view without requiring a separate post hoc projection. Across five image-classification datasets and three classifier backbones, we evaluate two-dimensional projections using 20 established projection quality measures. PAFT gives the strongest results, outperforming both the untreated baseline and rank-reduction alternatives overall, while maintaining comparable classification accuracy.

Primary area:
interpretability and explainable AI
Keywords:
deep learning, visualization, dimensionality reduction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HDoWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how to make learned representations of vision models more suitable for low-dimensional projection. The authors propose Projection-Aware Fine-Tuning (PAFT), which introduces an auxiliary branch consisting of a linear bottleneck and a classification head. In addition to the original classification objective, PAFT employs three regularization terms, including prediction consistency, CKA alignment, and neighborhood preservation, to jointly optimize the high-dimensional representations and their low-dimensional projections.

Critical strengths
  1. Simple and flexible framework. PAFT introduces a lightweight auxiliary bottleneck and classification head without requiring substantial modifications to the original classifier architecture.
Critical weaknesses
  1. Unclear motivation and practical significance. The paper does not sufficiently explain why a vision model needs to be explicitly optimized for projectability. If the purpose is to visualize and understand an existing model (Model A), applying PAFT modifies its learned representations and produces a different model (Model B). Consequently, the resulting visualization no longer directly reflects what Model A originally learned. Alternatively, if the goal is to train a new model whose representations are inherently easier to visualize, the authors should clarify what practical applications require this property and demonstrate its benefits beyond improved projection-quality metrics. Neither scenario is adequately justified, which raises concerns about the significance of the proposed research problem.

  2. Limited methodological novelty. The proposed method mainly combines three established techniques: knowledge distillation for prediction consistency, CKA for representation alignment, and the neighborhood-preservation objective from parametric t-SNE. Although jointly optimizing the high-dimensional representation and auxiliary bottleneck is a reasonable design, the paper does not introduce a fundamentally new mechanism for modeling or preserving high-dimensional and low-dimensional relationships. The methodological contribution therefore appears incremental.

  3. Questionable evaluation of projection fidelity. The projection-quality metrics use each method's own high-dimensional representation as the reference, while PAFT explicitly modifies this representation during training. Therefore, improved projection fidelity may result partly from simplifying or compressing the high-dimensional geometry rather than preserving more meaningful information from the original representation. The spectral analyses in Appendix I also demonstrate substantial changes in the representation's rank and variance distribution. Although classification accuracy is largely maintained, this does not necessarily imply that other useful information in the original representation is preserved. Consequently, the current evaluation does not sufficiently establish whether PAFT produces genuinely more informative visualizations.

  4. Insufficient comparisons with relevant methods. The experimental comparisons mainly involve standard fine-tuning and three rank-reduction alternatives. However, if PAFT is intended as a visualization or dimensionality-reduction approach, comparisons with existing learning-based visualization methods, such as parametric t-SNE, parametric UMAP, and other deep dimensionality-reduction approaches, are necessary. Conversely, if the contribution is a projection-aware representation-learning strategy, comparisons with existing geometry-preserving regularization and joint representation-learning methods would be more relevant. Although PCA, t-SNE, UMAP, and PaCMAP are included in the experiments, they are primarily used as post-hoc projection operators rather than competing learning-based approaches. It is also worth to compare other existing deep visialization methods, such as “Navigating the Effect of Parametrization for Dimensionality Reduction” (NIPS 2024), and "AKRMap: Adaptive Kernel Regression for Trustworthy Visualization of Cross-Modal Embeddings" (ICML 2025). The current comparisons are therefore insufficient to establish the advantage of PAFT over closely related methods.

  5. Limited validation of broader applicability. All experiments focus on supervised image classification. Although the geometric regularization terms do not directly require class labels, the overall framework relies on a supervised classification objective, while prediction consistency transfers label-related information through the main classifier's predictions. It remains unclear how the proposed approach can be adapted to unsupervised or self-supervised representation learning, where visualization is also commonly used. Furthermore, maintaining classification accuracy alone does not establish that the learned representations preserve other capabilities or information useful for downstream tasks. The applicability of PAFT beyond supervised classification therefore remains insufficiently validated.

Questions for authors

Effectiveness of the directly visualizable bottleneck. The paper emphasizes that setting the bottleneck dimension to d=2d=2 enables direct visualization without additional dimensionality reduction. However, Table 21 shows that the auxiliary classifier performs poorly on several datasets with many classes, including CIFAR-100, TinyImageNet, and CUB-200. How should these results be reconciled with the claim that the bottleneck preserves predictive information? To what extent can such a low-dimensional representation faithfully reflect the task-relevant structure learned by the original classifier?

6 comments
AuthorsOpenReview ↗

Response to Reviewer HDoW (1/3)

We thank the reviewer for the detailed assessment. We first address the question about the d=2 bottleneck, followed by the motivation, novelty, information retention, and comparison choices.

Q: Predictive information in the d=2 bottleneck. The d=2 bottleneck is a special configuration that provides a direct visualization; PAFT’s main goal is to make the high-dimensional representation more faithfully projectable. To assess how much task-relevant structure this view retains, the appropriate comparison is with other 2D visualizations, rather than expecting it to match the high-dimensional main classifier’s accuracy.

Information loss is a common limitation in dimensionality-reduction methods, and we do not claim to preserve all predictive information. Our claim is that the 2D bottleneck can preserve structure better than post-hoc PCA views. Since it is a linear map of the penultimate representation, PCA provides a natural comparison. Across the 15 backbone–dataset combinations, the d=2 view outperforms PCA of the untreated representation by +15.2/20 on average (Table 4). The low auxiliary accuracy partly reflects the limits of the linear auxiliary head rather than the view itself, since a linear classifier must divide the plane into 100–200 class regions using linear boundaries. On ResNet-50/CIFAR-100, for example, the auxiliary head reaches 9.4% (Table 21), whereas 15-NN accuracy in the same 2D view is 39.1%, compared with 15.8% to 23.2% across the PCA views in Table 2.

[Revision] We revised the phrase “preserves predictive and geometric structure” in the Abstract, Introduction, and Section 3 to state that the bottleneck is trained to preserve this structure, rather than implying a guarantee of complete preservation.

W1: Motivation and practical value. PAFT deliberately changes representation learning so that the resulting classifier, the model we would actually use, can be inspected more faithfully.

Training a model to make it easier to inspect has precedents. Tree regularization (Wu et al., 2018) trains neural networks to be more faithfully approximated by simple decision trees. More directly related to visualization, t-SimCNE (Böhm et al., 2023) jointly trains an image encoder and a 2D projection head. These works pursue different goals, but share the choice to improve inspection by changing how representations or models are learned.

For a visualization to help us understand a model, we need to trust that the patterns we see reflect its high-dimensional representation. Well-separated classes or clear clusters can make a plot easier to read, but they tell us little about the model if they arise mainly from the projection. This is why we focus on fidelity: preserving the model’s neighborhoods, distances, and class relationships makes the visualization a more reliable basis for interpretation.

Section 4.3 and Appendix F illustrate the practical value of this fidelity through radial structure and class confusion. Distances from a representation center underlie anomaly-detection methods such as Deep SVDD (Ruff et al., 2018), and feature-norm-based scores are used for out-of-distribution detection (Dhamija et al., 2018; Park et al., 2023). Our radial analysis shows that PAFT’s views better preserve samples’ distances from the overall representation center. The class-confusion analysis also shows that visible class relationships better reflect which classes the classifier confuses (Table 13). Together, these results help us read model diagnostics from the visualization more reliably.

[Revision] We added tree regularization and t-SimCNE to Related Work, and the references on center distances and feature norms to the radial-structure paragraph of Section 4.3.

W2: Methodological novelty. We agree that the individual terms are established, and we do not claim a new loss. Our contribution is to shape the classifier representation during fine-tuning so that it remains faithful under standard projections, and the evidence indicates that this joint shaping, together with how the terms are combined, drives the gains. The same auxiliary branch fitted to frozen representations performs better on PAFT representations than on untreated ones by +14.1/20 (Appendix H). Full PAFT outperforms the jointly trained neighborhood-only variant by +5.58/20 (Table 14) and joint parametric UMAP by +6.47/20 (new Appendix M), and the new stop-gradient ablations in Appendix G show that how the terms are coupled also matters.

[Revision] We added the stop-gradient ablations as a new paragraph at the end of Appendix G; the joint parametric UMAP comparison (new Appendix M) is detailed under W4.

AuthorsOpenReview ↗

Response to Reviewer HDoW (2/3)

W3: Compression and information retained in the representation. We agree that compression contributes to PAFT’s projection quality, but it does not fully account for PAFT’s advantage. The random-projection control in Appendix I (revised Table 18) isolates this effect. A random 2D map is not fitted to the data, so its gains come mainly from low rank: when the variance is concentrated in fewer directions, randomly chosen directions are more likely to capture it. Accordingly, these gains over the untreated baseline closely track the effective-rank ratio (Spearman ρ = −0.89). Compared directly with each rank-reduction alternative, PAFT’s advantage is larger under each of PCA, t-SNE, UMAP, and PaCMAP, which are fitted to the data, than under the random map: it rises from +4.97/20 to between +8.60 and +12.93 against the nuclear-norm penalty, and from +1.69 to between +7.20 and +11.07 against the top-d variance penalty. The narrow-penultimate baseline, whose selected widths are realistic penultimate-layer sizes for an ordinary model, compresses the spectral tail far more than PAFT (k99 ratio 0.03 vs. 0.88, Table 17) and is ahead of PAFT under the random map, yet PAFT is ahead of it on average under each fitted projection.

The spectral analysis also indicates that PAFT concentrates the representation in fewer leading dimensions without truncating the remaining ones. Relative to the untreated baseline, PAFT needs 0.43× as many principal components to explain 50% of the variance, but it still needs 0.88× as many components to explain 99% (Table 17).

To assess what PAFT preserves beyond accuracy, we compared PAFT with untreated models, using the variation between untreated models trained with different seeds as a reference (new Appendix K, Table 23). Mean prediction agreement is 0.889 between PAFT and untreated models, compared with 0.891 between untreated models. Their class-confusion structures have a mean Spearman correlation of 0.877, compared with 0.890 between untreated models. Untreated penultimate representations can also be linearly recovered from PAFT representations of a different seed on held-out data with mean R² = 0.594, compared with 0.589 from another untreated model. Despite the change in geometry, PAFT models therefore remain nearly as close to untreated models as two untreated training runs are to each other.

[Revision] We revised Table 18 to compare PAFT directly with each rank-reduction alternative, and added these analyses as the new Appendix K (Table 23).

AuthorsOpenReview ↗

Response to Reviewer HDoW (3/3)

W4: Comparisons with learned visualization and joint-learning methods. A useful distinction here is what each method learns. ParamRepulsor and AKRMap learn visualization mappings from fixed representations, whereas PAFT shapes the classifier representation itself. These methods can therefore serve as projection tools when comparing PAFT with other representation-learning methods, much as we use PCA, t-SNE, UMAP, and PaCMAP.

For this evaluation, we preferred the more widely used nonparametric methods. Huang et al. (2024) show that parametric methods tend to lose local structure relative to their nonparametric counterparts, and we observed the same on frozen untreated baseline representations: deep parametric t-SNE and UMAP score −2.53/20 and −1.47/20 against their nonparametric counterparts (new Appendix L, Table 24). We have now also trained learned projectors separately on PAFT and untreated representations; AKRMap requires a cross-modal metric, such as CLIPScore, as its regression target, which our single-modality classification setting does not provide. Each of the three performs better on PAFT representations, in all 15 backbone–dataset combinations: ParamRepulsor by +14.73/20, deep parametric t-SNE by +9.60/20, and deep parametric UMAP by +12.73/20 (Table 25).

In this context, the ablation experiment in Appendix H can be read as a variant of linear parametric t-SNE: it learns a linear projection of frozen representations using the t-SNE objective, with PAFT’s prediction-consistency and CKA terms added. Applying this procedure to PAFT representations gives a +14.1/20 advantage over applying it to untreated representations. We also tested the version with only the t-SNE objective on frozen PAFT and untreated baseline representations. Linear parametric t-SNE on PAFT representations outperforms its counterpart on untreated baseline representations by +15.13/20.

A similar connection can be made to joint learning. The neighborhood-only ablation in Appendix G can be viewed as a jointly trained linear parametric t-SNE variant, since classification loss and the neighborhood objective together update the backbone and its linear bottleneck. Full PAFT outperforms this variant by +5.58/20 (Table 14).

We have also considered other joint approaches. Parametric UMAP includes a joint-training variant aimed at improving classifier accuracy, in which a nonlinear UMAP head is trained jointly with the classifier. We adapted it to our setting (new Appendix M, Tables 26–28). Although it also shapes the representation during training, and improves on the untreated baseline by +9.30/20 while keeping main-head accuracy, PAFT’s penultimate representation outperforms its penultimate representation by +6.47/20, in 13 of 15 backbone–dataset combinations, and remains ahead even under nonparametric UMAP projection (+4.67/20). Comparing the low-dimensional outputs directly, PAFT’s linear bottleneck is ahead of the nonlinear UMAP head under t-SNE, UMAP, and PaCMAP (+7.00, +5.87, and +4.93/20); only under PCA is the UMAP head ahead (+3.67/20), since a nonlinear head can unfold clusters that a linear projection cannot. The UMAP head also adds up to 5.7% of the backbone’s parameters, compared with at most 0.62% for PAFT’s auxiliary branch. t-SimCNE takes a different direction: it learns a direct 2D view through augmentation-based contrastive learning, without a supervised classifier objective. Comparing it under our protocol would therefore require changing the training task, rather than simply replacing PAFT’s geometric regularization.

[Revision] We added ParamRepulsor (Huang et al., 2024) and AKRMap to Related Work, the parametric and learned-projector comparisons as the new Appendix L (Tables 24–25), the linear parametric t-SNE result as a new paragraph at the end of Appendix H, and the joint parametric UMAP comparison as the new Appendix M (Tables 26–28).

W5: Applicability beyond supervised classification. We agree that our experiments cover only supervised image classification. The two geometric terms, CKA alignment and neighborhood preservation, do not use labels, and removing the classifier-dependent prediction-consistency term reduces the win–loss difference by only 1.73/20 (Table 14). That variant is still fine-tuned with cross-entropy, however. In a self-supervised or label-free setting, we expect the classification loss to be replaced by the self-supervised objective, or by a distillation anchor to the pretrained model, with the geometric terms unchanged. We have not tested this. The predictive-behavior and representation-recovery analyses in our response to W3 also go beyond classification accuracy in assessing what PAFT preserves.

[Revision] We added this limitation to the Scope paragraph of the Limitations (Section 5).

Reviewer_X4hhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a new fine-tuning framework for classification networks. The idea is to train a second classification branch from the penultimate network activation, consisting of a linear layer to a low-dimensional space and a second classification head. Three additional losses encourage similar structure between the input and output of the linear layer and consistent classification results with the main classification head. The authors evaluate their method in terms of structure preservation in the low-dimensional bottleneck compared to general purpose dimensionality reduction methods and compared to rank reduction penalties. Finally, the authors ablate the importance of the three additional losses and disentangle the relevance of learning the projection to the bottleneck versus reshaping the penultimate representation via the additional losses.

Critical strengths
  • S1: The proposed method seems to improve interpretability of neural representations, which is an important topic.
  • S2: The experimental validation including different types of competitors and detailed ablations seems thorough.
Critical weaknesses
  • W1: The ablation study showed that the key loss term is the neighbor preservation, without which the win-loss difference drops significantly and nearly to the untreated baseline (Tab 14). As the authors state, the neighbor preservation is t-SNE's objective optimized for a linear encoder (possibly with affinities recomputed per batch). This casts some doubt on the novelty of the method. Especially since there are papers that explicitly investigate t-SNE (or UMAP) with linear encoders [a,b], but were not cited. The authors even explicitly distance their method from parametric t-SNE in lines 114-120, which seems misleading given the importance of the neighborhood preservation loss. Please note that the cited version of parametric t-SNE computes affinities per batch (while more recent implementations of parametric t-SNE indeed use the fixed whole data neighborhood graph). If the main difference between the propose neighborhood preservation loss and parametric t-SNE is that the neighborhood relations are recomputed differently, I think a more detailed empirical comparison to this approach would be needed. In addition, there are works like [c] that also combine a low-dimensional / visualization branch to a classification network to improve interpretability and might deserve a citation.

  • W2 (minor): I think mentioning how much the bottleneck dimension impacts the classification accuracy (Tab 19-20) would be useful in the main paper. Otherwise, Tab 2 makes d=2d=2 seem strictly better than higher dd.

References:

[a] Couplet, E., Lambert, P., Verleysen, M., Mulders, D., Lee, J. A., & De Bodt, C. (2023, September). Natively interpretable t-sne. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 107-123). Cham: Springer Nature Switzerland.
[b] Huang, H., Wang, Y., & Rudin, C. (2024). Navigating the effect of parametrization for dimensionality reduction. Advances in Neural Information Processing Systems, 37, 12977-13019.
[c] MENSAH, Samuel Ofosu, ROA, Camila, DJOUMESSI, Kerol, et al. Towards Interpretable Foundation Models for Retinal Fundus Images. arXiv preprint arXiv:2603.18846, 2026.

Questions for authors

Unfortunately, several aspects on the experimental setup remained unclear:

  • Q1: Are the affinities Π\Pi recomputed per batch or per epoch?
  • Q2: Why were the stop-gradients inserted in some auxilliary losses, but not others?
  • Q3: What is the number of PCs used for Table 2?
  • Q4: What is the classification-accuracy gate in line 348?

Other questions:

  • Q5: Did you run t-SNE & co on the improved PAFT representations? This might help disentangling the importance of reshaping the penultimate representation from learning the projection to the bottleneck.
5 comments
AuthorsOpenReview ↗

Response to Reviewer X4hh (1/2)

We thank the reviewer for the detailed comments and suggested references. We first answer the questions, beginning with Q5, which helps clarify the main evaluation, and then address the weaknesses in a second comment.

Q5: Applying t-SNE and other methods to PAFT representations. Yes, this is our main experiment. Tables 1 and 3 apply PCA, t-SNE, UMAP, and PaCMAP to each model’s penultimate representation, including PAFT’s, using the same protocol. These results measure how PAFT shapes the high-dimensional representation, rather than comparing its bottleneck with projections of untreated baseline representations. The d=2 bottleneck, a special case of PAFT that can be visualized directly, is evaluated separately (Table 4).

Q1: Affinity recomputation. Affinities are recomputed at every training minibatch from the current penultimate representations, using perplexity 30 and 32 bandwidth-search steps (Appendix A).

Q2: Stop-gradient placement. These choices were informed by early experiments. The main-head predictions and high-dimensional neighborhood distribution serve as targets for the auxiliary losses. Detaching these targets still allows the losses to update the backbone through p = Wf. CKA remains differentiable on both sides so that it also acts directly on the source representation.

In supporting ablations on ResNet-18, allowing gradients through the neighborhood target caused severe classification degradation: on SVHN, test accuracy fell to 19.6% at every coefficient, compared with 95.5% for the untreated baseline, and on STL-10 it fell to 60.9% (untreated: 94.7%). Additionally detaching CKA’s source argument changed main-head accuracy by at most 0.68 percentage points but reduced projection fidelity under PCA, with PAFT ahead by +8.69/20 on average.

[Revision] We added these stop-gradient ablations as a new paragraph at the end of Appendix G.

Q3: Number of principal components. Table 2 uses two-component PCA, except for the separate d=2 column, where the bottleneck is read directly.

[Revision] We clarified this in the caption of Table 2.

Q4: Classification-accuracy gate. This is a validation-based filter applied before configuration selection. A candidate below the untreated baseline is rejected if its relative accuracy drop is at least 1%, or if the drop is statistically significant. If no candidate passes, the method falls back to the untreated baseline. Appendix D specifies the test and reports a sensitivity analysis.

AuthorsOpenReview ↗

Response to Reviewer X4hh (2/2)

W1: Neighborhood preservation and parametric t-SNE. The −11.18 in Table 14 compares the ablated model with full PAFT, not with the untreated baseline. We computed the direct baseline comparisons from the existing results: against the untreated baseline, PAFT without neighborhood preservation achieves +8.17/20, and full PAFT achieves +12.58/20.

The reviewer is right that the cited parametric t-SNE computes affinities per batch. As van der Maaten (2009, Section 3.1) notes, however, “the subdivision of training data into batches was fixed”, so that the affinities could be precomputed once from the input data before training. This is the “fixed neighborhood structure” referred to in lines 114–120. In PAFT, affinities are recomputed at every step from the current, changing representation. We explicitly identify the neighborhood objective as parametric t-SNE in Section 3.1, and our contribution is not a new t-SNE objective or linear projection. The distinction we intended is that PAFT reshapes the classifier’s high-dimensional representation during training, whereas parametric t-SNE fits a projection to a fixed representation.

Appendix H helps examine this distinction. Its fitted branches can be viewed as linear parametric t-SNE with PAFT’s prediction-consistency and CKA terms added. Applying this procedure to frozen PAFT representations outperforms applying it to frozen untreated baseline representations by +14.1/20. As the empirical comparison the reviewer suggests, we also fit linear parametric t-SNE alone to frozen PAFT and untreated baseline representations, recomputing affinities in every minibatch as PAFT does. With the neighborhood computation identical, linear parametric t-SNE on PAFT representations outperforms its counterpart on untreated baseline representations by +15.13/20, so the difference lies in the representation rather than in how neighborhoods are computed. Because PAFT changes the representation itself, parametric projectors can also be applied on top of it: deep parametric t-SNE performs better on PAFT representations than on untreated ones as well (+9.60/20, new Appendix L). Joint training can also be examined directly. The neighborhood-only ablation in Table 14 is a jointly trained linear parametric t-SNE: the classification loss and the t-SNE objective together update the backbone and the linear bottleneck. Full PAFT outperforms it by +5.58/20, so the CKA and prediction-consistency terms add to joint training with the t-SNE objective.

[Revision] We added the comparisons against the untreated baseline as a new column in Table 14, the linear parametric t-SNE result as a new paragraph at the end of Appendix H, and expanded the Related Work discussion of learned projections, which now includes the suggested references [a–c] and the semi-supervised variant of parametric UMAP.

W2: Bottleneck dimension and classification accuracy. The main comparisons select d from {16, 64}; d=2 is a separate configuration. Across the existing sweep of d ∈ {2, 8, 16, 32, 64}, main-head test accuracy varies little with d: within each backbone–dataset combination, the difference between the highest and lowest accuracy across widths has a median of 0.25 percentage points and a maximum of 1.24.

Table 21 reports the accuracy of the auxiliary classifier operating on the bottleneck, not the main classifier operating on the high-dimensional penultimate representation. The lower auxiliary accuracy at d=2 therefore does not mean that the main classifier suffers the same loss.

A fair comparison of the class information retained in the d=2 view should be with other representations reduced to two dimensions, rather than with the high-dimensional main classifier. Since the bottleneck is a linear projection, PCA provides a natural comparison. Although the d=2 auxiliary classifier does not match the main classifier’s accuracy, the d=2 representation is actually strictly better on all four label-structure measures than every PCA view reported in Table 2 (ResNet-50/CIFAR-100). Thus, Table 2 shows an advantage over the compared 2D views, not over the high-dimensional classifier.

[Revision] We summarized the effect of the bottleneck width on main-head accuracy in a new sentence in Section 4.2 and added the full results as the new Table 22 in Appendix J.

Dear Reviewer X4hh,

As the discussion period comes to a close, we wanted to briefly check in to see if our responses and the updates to the manuscript addressed your questions and concerns. Please let us know if we can clarify anything further before the deadline, and thank you again for your helpful feedback and references!

Reviewer_ingkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Researchers usually look at a network's learned features by squashing them to 2D afterwards with PCA, t-SNE or UMAP. Nothing in training makes those 2D views faithful to the original high-dimensional space. PAFT (Projection-Aware Fine-Tuning) adds that goal to fine-tuning. The resulting representations keep more of their structure when projected to 2D, and classification accuracy stays about the same.

Critical strengths
  • The empirical evaluation is fairly extensive with 15 backbone dataset pairs × 4 projection methods × 20 measures × 10 seeds. Results use paired t-tests with multiple-comparison correction plus a 1% minimum effect, so small noisy improvements don't count.
  • The paper presents a simple idea that changes the framing: instead of training first and projecting afterwards, it trains the model to be projectable with very little overhead.
Critical weaknesses
  • PAFT makes the model easy to visualize by changing the model. Each 2D view is scored against that method's own features, and PAFT improves those scores partly by squeezing its features into fewer directions. So the plots are faithful to a reshaped model, not to the one you would normally train and want to understand. The paper only shows that accuracy survives this change, so it's unclear whether conclusions from PAFT's plots carry over to an ordinary model.
Questions for authors
  • Is PAFT a better way to look at a model, or a way to train a better-looking model? And what evidence shows that insights from its plots carry over to an ordinary model?
4 comments

We thank the reviewer for the thoughtful assessment and the precise question.

Q: A better way to look at a model, or a way to train a better-looking model? PAFT is the second approach: it is a method to train a model whose representation can be inspected more faithfully. However, “better-looking” is not what we measure or optimize. Each 2D view is scored by how well it preserves the model’s own high-dimensional neighborhoods and distances. PAFT’s largest gains are in local and global geometry (+5.7/7 and +4.6/5, Table 1), not in label separation, which is what would make a plot merely look cleaner.

Is the reshaping just compression? Compression contributes, but it does not fully account for PAFT’s superior projection quality. PAFT largely retains the spectral tail: 99% of the variance still requires 0.88× as many components as the baseline (Appendix I, Table 17). A random 2D map isolates the benefit of compression (its gain over the untreated baseline closely tracks the effective-rank ratio, Spearman ρ = −0.89). Compared directly with each rank-reduction alternative, PAFT’s advantage is larger under PCA, t-SNE, UMAP, and PaCMAP than under this random map (revised Table 18): it rises from +4.97/20 to between +8.60 and +12.93 against the nuclear-norm penalty, and from +1.69 to between +7.20 and +11.07 against the top-d variance penalty. The narrow-penultimate baseline, whose selected widths are realistic penultimate-layer sizes for an ordinary model, compresses the spectral tail far more (k99 ratio 0.03) and is ahead of PAFT under the random map, yet PAFT is ahead of it on average under each fitted projection.

[Revision] We revised Table 18 to compare PAFT directly with each rank-reduction alternative.

Do insights carry over to an ordinary model? We agree that PAFT’s plots describe the PAFT model, not an untreated one, and we do not claim otherwise. To quantify how far PAFT moves from an ordinary model, we used the variation between untreated models trained with different seeds as a reference (new Appendix K, Table 23). Comparing PAFT and untreated models trained with different seeds:

  • Mean prediction agreement is 0.889 between PAFT and untreated models, compared with 0.891 between untreated models; it falls within the range of untreated seed pairs in 14 of 15 backbone–dataset combinations.
  • The class-confusion structure remains largely that of an ordinary model: the Spearman correlation between the class-confusion structures of PAFT and untreated models is 0.877 on average, compared with 0.890 between two untreated models, and falls within the range of untreated seed pairs in 12 of 15 combinations.
  • Untreated penultimate representations can be linearly recovered from PAFT representations on held-out data with mean R² = 0.594, compared with 0.589 when recovering them from another untreated model, using the same seed offset in both comparisons.

In predictions, confusion structure, and linearly recoverable information, a PAFT model therefore remains nearly as close to an untreated model as two untreated training seeds are to each other. While the high-dimensional geometry is reshaped by design (so the plots depict the PAFT model), adopting PAFT does not alter how the model behaves in these respects. Moreover, class relationships visible in PAFT’s plots track its classifier’s confusions more faithfully than those in untreated plots, under all four projection methods (Table 13), and these confusions are largely shared with untreated models. Thus, while PAFT’s plots depict the PAFT model, the class-level relationships they reveal are largely shared with ordinary models.

[Revision] We added these analyses as the new Appendix K (Table 23) and summarized them in a new sentence at the end of Section 4.3.

Dear Reviewer ingk,

As the discussion period nears its close, we wanted to briefly check in to see if our response and the new analysis —showing that PAFT models agree with ordinary untreated models in predictions, class confusions, and linearly recoverable information as closely as two untreated seeds agree with each other—addressed your question. We would be happy to clarify any remaining points before the deadline, and we thank you again for your thoughtful review!

Reviewer_ingkOpenReview ↗

Reply to authors

I appreciate the authors' response, my main questions are mostly resolved.

Dear Reviewer ingk,

Thank you very much for reading our response and confirming that your questions are resolved—we really appreciate your thoughtful feedback on the paper!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment to AC and Reviewers

We thank the AC and the reviewers. We summarize here how we addressed the shared concerns; full answers are under each review.

1. Motivation and practical significance (HDoW). PAFT’s motivation is to change the framing of model inspection, which ingk notes as a strength: instead of training additional models for model inspection (e.g., deep parametric t-SNE or deep parametric UMAP), which are themselves learned networks, PAFT builds projectability into model training. The practical benefit of PAFT is that the visualization faithfully reflects the high-dimensional representation of the inspected model. PAFT’s views, for example, preserve distances from the representation center, a quantity commonly used in model diagnostics such as anomaly and out-of-distribution detection, much better: the Spearman correlation between high- and two-dimensional radius is 0.446, compared with 0.110 for the untreated baseline (Section 4.3). Similarly, class relationships visible in PAFT’s plots better reflect which classes the classifier confuses than those in untreated plots, under all four projection methods (Table 13). Related Work now also discusses earlier approaches that train models for inspectability, such as tree regularization and t-SimCNE.

2. PAFT changes the model (ingk, HDoW). This is by design: PAFT trains the classifier we would actually use so that it can be inspected more faithfully, and its plots describe that model, not an untreated one. The resulting model, however, remains close to an ordinary one. Using two untreated models trained with different seeds as the reference (new Appendix K), mean prediction agreement is 0.889 between PAFT and untreated models, compared with 0.891 between two untreated models; their class-confusion correlation is 0.877, compared with 0.890; and untreated representations are linearly recovered from PAFT representations with R² = 0.594, compared with 0.589 from another untreated model.

3. Gains from compression (ingk, HDoW). Compression contributes, but it does not fully account for PAFT’s advantage. A random 2D map isolates the benefit of compression. In direct comparisons with each rank-reduction alternative, PAFT’s advantage is larger under every fitted projection (PCA, t-SNE, UMAP, and PaCMAP) than under the random map (revised Table 18). The narrow-penultimate baseline, at realistic penultimate widths, compresses the spectral tail far more than PAFT (k99 ratio 0.03 vs. 0.88) and leads under the random map, yet trails PAFT on average under each fitted projection.

4. Novelty and relation to parametric t-SNE (X4hh, HDoW). The individual terms are established, and we do not claim a new loss. Our contribution is to shape the classifier representation during fine-tuning so that it remains faithful under standard projections, and stop-gradient ablations show that the way the terms are coupled matters (Appendix G). Our main evaluation therefore projects each model’s own penultimate representation, not the bottleneck (Tables 1 and 3). With the neighborhood computation identical, linear parametric t-SNE fitted to frozen PAFT representations outperforms the same procedure on untreated ones by +15.13/20 (Appendix H), so the gain lies in the representation. Full PAFT also outperforms a jointly trained neighborhood-only variant by +5.58/20, and even without the neighborhood term, PAFT improves on the untreated baseline by +8.17/20, compared with +12.58/20 for full PAFT (Table 14).

5. Comparisons with learned and joint methods (HDoW). The suggested methods (parametric t-SNE, parametric UMAP, ParamRepulsor, and AKRMap) learn maps from fixed representations, so rather than competing with PAFT, they can serve as tools to evaluate it. Fitted separately to frozen PAFT and untreated representations, deep parametric t-SNE, deep parametric UMAP, and ParamRepulsor each perform better on PAFT representations in all 15 backbone–dataset combinations, by +9.60 to +14.73/20 (new Appendix L); AKRMap requires a cross-modal metric that our setting does not provide. Parametric UMAP also has a semi-supervised variant that uses the UMAP loss to improve classification accuracy rather than to obtain faithful views. We adapted it to our evaluation (new Appendix M): PAFT’s representation outperforms it by +6.47/20, in 13 of 15 combinations, and remains ahead even under nonparametric UMAP projection (+4.67/20), despite a head of up to 5.7% of backbone parameters (PAFT: ≤0.62%).

6. Low auxiliary accuracy at d = 2 (X4hh, HDoW). The low accuracy belongs to the linear auxiliary head, not the main classifier: the difference between the highest and lowest main-head accuracy across bottleneck widths is at most 1.24 percentage points (new Table 22). As a 2D view, d = 2 outperforms PCA of the untreated representation by +15.2/20 (Table 4).

Revisions. New Appendices K–M, revised Table 18, new analyses in Appendices G and H and Table 14, expanded Related Work, and main-text clarifications.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.