acceptodds
Under review as a conference paper at ICLR 2027

FerVA: Faithful Expert Reconstruction and Visual Alignment

Abstract

A compact space can represent every expert update yet exclude their best shared reconstruction. Task-specific input correlations reorient expert contributions, so storing every update need not preserve that optimum. For fixed statistics and expert supports, we establish when an input restriction is lossless and identify the unique minimal faithful space. The restriction loss lower-bounds every merge confined to that input span under the same reconstruction objective. Faithful Expert Reconstruction and Visual Alignment (FerVA) turns this distinction into a two-component method: a static language solve retains full input features in compact output coordinates, while a supervised visual calibrator learns through an auxiliary decoder and transfers a valid visual mixture to the constructed model. In a controlled six-task UCIT study, recovering the omitted component raises final average score (FAA) from 69.54 to 76.04. At correction rank 16, objective-guided recovery reaches 75.54 versus 71.18–71.33 for three random bases. The separate complete method leads reproduced merged baselines in FAA across four UCIT/DCL–backbone settings, reaching 76.20 on UCIT–LLaVA, 4.05 points above the strongest comparator. These results link reconstruction-faithful coordinates to task value in multimodal consolidation.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Mitigating forgetting, Continual Learning, Model Merging

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_otDROpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates a specific issue in multimodal expert model merging: whether a low-dimensional space capable of accurately representing all expert updates can also preserve the optimal shared response reconstruction solution. The authors demonstrate that these two requirements are not equivalent, as task input correlations can cause the shared optimal solution to deviate from the joint input space of the expert updates.

Based on this observation, the proposed method, FerVA, retains full input coordinates on the language side while compressing the reconstruction problem via an output basis shared among experts; on the vision side, it learns mixture weights for a frozen vision projector, with the calibrator trained via an auxiliary decoder before being transferred to the final model. The paper provides a theoretical characterization of lossless output compression, reconstruction loss resulting from input constraints, and the minimal sufficient input space. Experiments include a controlled analysis of recovery directions and evaluations across two backbone models and two continual learning benchmarks. The full method achieves the highest final average scores across four settings; controlled experiments further demonstrate that selecting recovery directions based on the reconstruction objective is more effective than random recovery at the same rank.

Critical strengths

The theoretical issues are clearly defined, and the conclusions offer practical guidance. The paper distinguishes between the representability of expert updates and the representability of optimal shared-response solutions, while quantifying the exact objective gap and the minimal sufficient space resulting from input constraints. These findings inform the selection of low-dimensional merged spaces, preventing the erroneous assumption that a merge process is lossless simply because "all experts can be represented exactly."

Controlled recovery experiments effectively bridge theory and task performance. With fixed experts, input statistics, visual interfaces, and deployment gains, full input recovery improved FAA from 69.54 to 76.04; at a recovery rank of 16, target-guided recovery achieved 75.54, whereas three random bases yielded only 71.18–71.33. This demonstrates that the choice of recovery direction plays a substantive role, rather than merely increasing parameter capacity. Results across two backbones and two benchmarks further substantiate the method's practical utility.

Critical weaknesses

The incremental gains attributable specifically to the coordinate-based approach—distinct from existing full-input reconstruction methods—have not been fully isolated. Appendix C.1 clarifies that, given consistent statistics, task weights, and regularization, the language-side solution recovers the dense optimal reconstruction corresponding to RegMean; furthermore, MAny already employs a hybrid of weighted full-input activation reconstruction and prototype-driven visual projection. Consequently, while the experiment demonstrating a recovery of 6.50 points validates the paper's "coordinate insight," it does not directly explain the source of the performance improvement over MAny. Further differentiation is required to isolate the contribution of the coordinate-based implementation from other configuration changes—specifically by standardizing statistics, regularization, numerical precision, deployment gains, and visual interfaces. This distinction is crucial for establishing the substantive novelty of the proposed method.

The independent contribution of the final visual calibrator and the design of the auxiliary decoder lack direct empirical validation. Table 5 replaces the full visual module with a uniform mixture and removes both prototype routing and learned calibration; thus, it cannot isolate the benefits of the added calibrator. Although Table 6 provides a comparison using a fixed language model, it relies on an early configuration, meaning the ~0.32-point gain does not directly reflect the performance of the final FerVA. Nor does the paper directly compare "training the calibrator on the final decoder" versus "training via an auxiliary decoder followed by transfer." The conditional transfer bounds presented in the appendix cannot substitute for these experiments; consequently, it remains difficult to determine whether the additional visual training and its complex design are truly necessary.

Questions for authors

Alignment Comparison with Existing Methods: When using the same experts, source samples, statistic normalization, regularization, numerical precision, deployment gains, and visual interfaces, what are the FAA scores for FerVA and MAny, respectively? Which specific modifications contribute primarily to the performance gains shown in Table 1?

Independent Gains from Final Visual Calibration: If the language weights of the final FerVA model are fixed and only the prototype routing and the final learned calibrator are compared, what are their respective FAA scores and per-task performance?

Necessity of the Auxiliary Decoder: Under identical data and training budgets, how do the performance and cost of training the visual calibrator directly via the final decoder compare to training it via an auxiliary decoder followed by transfer? Is there direct evidence supporting the latter approach?

8 comments
AuthorsOpenReview ↗

Thank You for Your Time and Constructive Feedback

Thank you for recognizing the guidance provided by the faithful-space characterization and the support from the equal-rank recovery experiment for choosing recovery directions. To save you time, we have highlighted the key conclusions and important points in bold. The distinction you raise clarifies the paper's contributions: the theory characterizes which input directions should be retained and which output coordinates can be compressed losslessly, while the additional comparisons separately examine compact representation and implemented construction procedures. With the same direct solver, shared output coordinates reduce the right-hand side of the equations to 96 columns while matching the dense reference's scores on all six tasks at the reported precision. With source statistics, ridge regularization, the deployment coefficient, and the visual interface matched, FerVA retains a 3.41-point FAA advantage over the MAny implementation evaluated here. The former result supports fidelity under a compact representation, while the latter reports the empirical difference between implemented language-construction procedures. Below, we also use separate comparisons to assess learned calibration and auxiliary supervision and describe offline costs and deployment. All experiments are conducted on UCIT–LLaVA-1.5-7B. FAA denotes the final average score across the six tasks.

AuthorsOpenReview ↗

W1. What do the faithful space and its coordinate implementation each contribute?

Thank you for pointing out that the theoretical contribution of the faithful space should be distinguished from the effects of the numerical solution procedure. FerVA's central question is: which coordinates can be compressed without changing the reconstruction optimum, and which input directions must be retained? We adopt RegMean's activation-weighted reconstruction objective and characterize the minimal input space that is faithful for this family of expert updates, given the source statistics and the experts' admissible input directions. We also quantify the additional reconstruction error induced by restricting the input space. Source statistics can change the orientation of this minimal space, potentially taking it beyond the joint span of the experts' original input directions. FerVA therefore retains the full input space and compresses only the output side. Under the fixed-statistics and regularization conditions stated in the paper, this parameterization preserves the corresponding dense reconstruction optimum when the normal matrix is positive definite and the shared output basis is orthonormal and spans the output spaces of all expert updates.

Thank you also for asking about the task relevance of the faithful-space characterization. With the deployment coefficient fixed at 1 and prototype routing alone used in both conditions, full input outperforms restricted input on all six tasks and improves FAA by approximately 7.02 points. This comparison directly tests whether directions excluded by the input restriction have downstream value. The largest improvements are on CLEVR-Math and VizWiz-caption, at 16.00 and 12.02 points, respectively, showing substantial task benefits from retaining these directions in this experiment. The table reports the task-level results, with FAA and differences calculated from the listed scores before rounding.

TaskFull inputRestricted inputFull input minus restricted input (points)
ImageNet-R89.2781.77+7.50
ArxivQA91.4791.37+0.10
VizWiz-caption59.8047.78+12.02
IconQA73.9368.53+5.40
CLEVR-Math77.7361.73+16.00
Flickr30k57.8156.74+1.07
FAA75.0067.99Approx. +7.02

Thank you for further asking us to distinguish the space characterization from existing full-input methods. MAny already uses the full input space, so the comparison above addresses the cost of restricting that space. The output-side contribution is to preserve the corresponding dense reconstruction result in compact coordinates. With the same direct solver and prototype routing, shared coordinates reduce the right-hand side of the equations from thousands of columns to 96, while matching the dense reference's scores on all six tasks at the reported precision. The input-side loss characterization and output-side fidelity result jointly inform FerVA's design: retaining input directions with task value while compressing output coordinates that admit a lossless representation.

AuthorsOpenReview ↗

W2. Contribution of final visual calibration and value of the auxiliary decoder

Thank you for pointing out that learned calibration should be examined separately under the final language configuration. The new matched ablation shows that adding calibration to prototype routing increases FAA from 75.61 to 76.20, an improvement of approximately 0.60 points. Both conditions share the same reconstructed language weights, visual experts, prototype routing, and deployment coefficient, and use identical test samples and scoring protocols. Only whether calibration is enabled changes. All six tasks improve. Unlike removing the entire visual module, this comparison changes only the calibrator, directly measuring its independent contribution beyond the existing routing mechanism. The revised manuscript will replace the earlier-configuration calibration comparison with the matched ablation below. FAA and differences are calculated from the listed scores before rounding.

TaskPrototype routing onlyPrototype routing with calibrationDifference after adding calibration (points)
ImageNet-R88.3088.90+0.60
ArxivQA89.0389.37+0.34
VizWiz-caption66.0766.96+0.89
IconQA74.9075.73+0.83
CLEVR-Math78.4078.97+0.57
Flickr30k56.9557.29+0.34
FAA75.6176.20Approx. +0.60

Beyond the calibrator's contribution, thank you for suggesting that we directly test the auxiliary decoder's value during training. Auxiliary supervision performs better in all five paired training orders, with an average FAA improvement of approximately 0.49 points. Using five training-sample order seeds, we compare calibrators trained through the auxiliary decoder and then transferred with calibrators trained directly through the final decoder. Both are evaluated using the final decoder. In all five paired performance comparisons, the two supervision strategies use identical calibration data and training budgets. Improvements range from 0.30 to 0.59 points, showing that the advantage of auxiliary supervision recurs across the examined training orders and providing empirical support for this training strategy. Differences and summary statistics in the table are calculated from the listed FAA values.

Training-sample order seedAuxiliary-supervision FAADirect-supervision FAAAuxiliary minus direct supervision (points)
2026080476.2075.61+0.59
2026080576.1875.88+0.30
2026080676.2075.65+0.55
2026080776.1675.74+0.42
2026080876.1875.60+0.58
Mean ± sample standard deviation76.18 ± 0.0275.70 ± 0.12+0.49 ± 0.13
Reviewer_oNyHOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes an interesting method to merge multiple multimodal models while preserving their capabilities by designing two merge strategies, which handle language model parameters and vision representations. The comprehensive empirical results support the effectiveness of their method and generalization ability.

Critical strengths
  1. The method is novel and interesting: they theoretically analyze the conditions for keeping the optimal reconstruction way and design a language merging strategy avoiding information loss, complemented by task-calibrated visual composition.

  2. They conduct extensive experiments across multiple benchmark datasets and compare with many baselines, showing the improvements over existing baslines. The ablation studies demonstrate the effectiveness of components.

Critical weaknesses
  1. The current experimental results are not performed under various random seeds, which makes it difficult to assess the robustness of the method. The paper lacks ablations comparing prototype-only and calibration under same language weights. It would be better if authors complete the ablations.

  2. The method part focuses on optimal layer-wise reconstruction and cannot guarantee the optimal downstream performance, because the reconstruction error can accumulate across layers. They could conduct additional experiments to find out the relationship between the reconstruction error and downstream accuracy to better claim their contribution.

Questions for authors

NA

5 comments
AuthorsOpenReview ↗

Thank You for Your Time and Constructive Feedback

Thank you for recognizing the analysis of reconstruction fidelity conditions and the experiments across benchmarks, and for identifying three validation needs: robustness, the calibrator's independent contribution, and the task relevance of the reconstruction objective. To save you time, we have highlighted the key conclusions and important points in bold. We address these through source-sample resampling, calibration ablation with fixed language weights, and controlled recovery in the full model, respectively. Five resampling runs yield an FAA of 76.18 ± 0.01, and enabling calibration adds approximately 0.60 points. Along the five-level recovery path at γ=1.5, all four adjacent comparisons associate lower data reconstruction error with higher mean accuracy across the four accuracy-based tasks, with mean accuracy increasing from 78.07% at the restricted solution to 83.10% at the full solution. This whole-model evaluation directly connects per-module reconstruction analysis with downstream task performance and, together with the other two comparisons, provides additional empirical support under the examined settings. We report the individual results on UCIT–LLaVA-1.5-7B below. FAA denotes the final average score across the six tasks.

AuthorsOpenReview ↗

W1. Stability across random seeds and the calibrator's independent contribution

Thank you for suggesting repeated experiments to assess stability. Across five source-sample resampling runs, mean FAA is 76.18, with a sample standard deviation of 0.01 points and a range of 76.17–76.20. The randomized factor is the selection of source samples used to estimate language reconstruction statistics, so this experiment directly assesses sensitivity to source-sample selection. ArxivQA and IconQA exhibit the largest task-level variation, with maximum-to-minimum differences of 0.56 and 0.50 points, respectively. As the table shows, individual task scores vary with sampling, while the six-task average remains similar across these five repetitions, providing concrete evidence of stability with respect to source-sample selection. FAA and its summary statistics are computed from the six task scores below without intermediate rounding, with final values rounded to two decimal places using round-half-up.

Source-sample selection seedImageNet-RArxivQAVizWiz-captionIconQACLEVR-MathFlickr30kFAA
2026080489.1789.3366.6475.9378.8757.2376.20
2026080588.9389.1366.7475.6779.1757.4376.18
2026080688.8788.9766.8176.0379.3057.1976.20
2026080788.8388.9766.7776.0079.2757.1576.17
2026080889.2389.5366.4175.5379.2057.1776.18

Thank you also for suggesting that we isolate the calibrator's contribution using the same language model. With language weights fixed, enabling learned calibration increases FAA from 75.61 to 76.20, an improvement of approximately 0.60 points. This new matched ablation shares the same reconstructed language weights, visual experts, prototype routing, and deployment coefficient, uses identical test samples and scoring protocols, and changes only whether calibration is enabled. All six tasks improve, with the largest improvements on VizWiz-caption and IconQA, showing the additional contribution of learned calibration on top of the fixed language model and prototype routing. FAA and differences in the table are calculated from the listed scores before rounding.

TaskPrototype routing onlyPrototype routing with calibrationDifference after adding calibration (points)
ImageNet-R88.3088.90+0.60
ArxivQA89.0389.37+0.34
VizWiz-caption66.0766.96+0.89
IconQA74.9075.73+0.83
CLEVR-Math78.4078.97+0.57
Flickr30k56.9557.29+0.34
FAA75.6176.20Approx. +0.60
AuthorsOpenReview ↗

W2. Relationship between reconstruction error and whole-model downstream accuracy

Thank you for pointing out the need for a direct empirical connection between layerwise reconstruction quality and whole-model performance. In the five-level controlled recovery experiment at γ=1.5, reconstruction error and whole-model mean accuracy change in opposite directions in all four adjacent comparisons, and accuracy increases by 5.03 percentage points from the restricted to the full solution. We fix the experts, source statistics, regularization, prototype routing, and deployment coefficient, and interpolate between the restricted and full solutions at 0%, 25%, 50%, 75%, and 100% recovery, measuring reconstruction error and whole-model performance at each level. Each level uses the same 3,000 test samples per task. Because VizWiz-caption and Flickr30k use image-captioning metrics, the table separately reports the four accuracy-based tasks, with the mean of their unrounded accuracies in the final column.

Normalized data reconstruction loss difference (%)ImageNet-R (%)ArxivQA (%)IconQA (%)CLEVR-Math (%)Mean accuracy across four tasks (%)
100.0085.9791.7068.8065.8078.07
18.7588.7791.3372.8772.0781.26
−25.0089.8790.8074.0075.7082.59
−31.2589.7390.1374.8778.1383.22
0.0088.7089.3775.3778.9783.10

Table note: The rows correspond, in order, to the five recovery levels above. Data reconstruction error is aggregated over 224 modules after deployment scaling and excludes the regularization term. The first column is the difference between the error at each level and that of the full solution, normalized by the error difference between the restricted and full solutions. Negative values indicate lower error than the full solution, while 0 indicates the same error.

Thank you for encouraging us to ground the argument in full-model performance. At 75% recovery, data reconstruction error is lowest and mean accuracy across the four tasks is highest, at 83.22%. At 100% recovery, error increases and mean accuracy decreases by 0.12 percentage points. Together with the first three adjacent comparisons, this reversal supports the association between lower reconstruction error and higher whole-model mean accuracy along this recovery path. Individual tasks respond differently. For example, ArxivQA decreases as recovery increases, so the empirical relationship here concerns the four-task mean accuracy, with all task-level changes also reported in the table.

Reviewer_sTupOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies expert merging for multimodal continual instruction tuning. The authors show that a space that can represent every expert update may still exclude the optimal shared reconstruction under fixed input statistics, because task-specific input correlations can reorient the shared solution. They propose FerVA, which combines full-input language reconstruction in compact output coordinates with supervised calibration of the mixture weights over frozen visual projector outputs. Experiments on UCIT and DCL with LLaVA-1.5-7B and InternVL-Chat-7B report higher final average scores than the compared merged-model baselines across all four settings. Additional controlled recovery experiments support the task relevance of the omitted reconstruction component.

Critical strengths

1.The paper is well organized and clearly motivated. The two-feature example effectively illustrates why representing every expert update does not guarantee preserving their optimal shared reconstruction.

2.The evaluation covers two benchmarks and two backbones. The recovery experiment in Figure 4 provides evidence linking the theoretical argument to task performance.

Critical weaknesses

The recovery experiment in Figure 4 supports the reconstruction-space argument but does not isolate how much of the improvement over MAny comes from this change rather than other construction and tuning choices. The benefit of auxiliary-decoder calibration over direct calibration on the final decoder is not established. Table 5 evaluates the entire visual module, while Table 6 uses an earlier language configuration.

Questions for authors
  1. Beyond the shared evaluation protocol, were statistics collection, numerical precision, and gain tuning matched between MAny and FerVA?

  2. What are the full-input and restricted results at γ = 1, and what development criterion was used to select γ?

  3. Is there empirical evidence that calibrating through the auxiliary decoder is more effective than calibrating directly on the final decoder?

5 comments
AuthorsOpenReview ↗

Thank You for Your Time and Constructive Feedback

Thank you for recognizing the motivation, the two-feature example, and the evaluation across benchmarks, and for encouraging us to clarify the sources of performance improvement. To save you time, we have highlighted the key conclusions and important points in bold. The additional comparisons show that with source statistics, ridge regularization, the deployment coefficient, and the visual interface matched, FerVA retains a 3.41-point FAA advantage over the MAny implementation evaluated here. Ablations with fixed language weights and five paired training orders further assess the calibrator and its supervision strategy separately, yielding an approximately 0.60-point improvement from calibration and a mean 0.49-point improvement from auxiliary supervision. These comparisons separately assess the empirical benefits of language construction, learned calibration, and auxiliary supervision, providing more direct support for the full method's design. Below, we report the settings and results on UCIT–LLaVA-1.5-7B. FAA denotes the final average score across the six tasks.

AuthorsOpenReview ↗

W1. Sources of improvement in language construction and visual calibration

Thank you for recognizing that the recovery experiment supports the faithful-space analysis and for suggesting that we further distinguish the contributions of language construction, the calibrator, and its supervision strategy. Our additional comparisons address these three questions separately. With statistics, ridge regularization, and visual weights shared, FerVA retains a 3.41-point FAA advantage over the MAny implementation evaluated here, reflecting the performance difference between the two implemented language-construction procedures. Because MAny already uses the full input space, the approximately 7.02-point improvement from full rather than restricted input addresses the cost of input restriction in a separate comparison. On the visual side, enabling or disabling calibration with language weights fixed measures the calibrator's independent contribution. The five paired comparisons of auxiliary and direct supervision assess the supervision strategy, with auxiliary supervision improving FAA by 0.49 points on average.

Thank you for suggesting that we evaluate the calibrator itself separately from its supervision strategy. The new matched ablation shows that adding learned calibration to prototype routing increases FAA from 75.61 to 76.20, an improvement of approximately 0.60 points. To isolate the calibrator's contribution, both conditions share the same reconstructed language weights, visual experts, prototype routing, and deployment coefficient, and use identical test samples and scoring protocols. Only whether calibration is enabled changes. All six tasks improve, showing benefits across classification, question answering, and image captioning. The original Table 5 evaluates the entire visual module, whereas Table 6 examines calibration under an earlier language configuration. Your suggestion makes this comparison more complete, and we will replace the earlier comparison in Table 6 with the following matched ablation in the revised manuscript. FAA and differences are calculated from the listed scores before rounding.

TaskPrototype routing onlyPrototype routing with calibrationDifference after adding calibration (points)
ImageNet-R88.3088.90+0.60
ArxivQA89.0389.37+0.34
VizWiz-caption66.0766.96+0.89
IconQA74.9075.73+0.83
CLEVR-Math78.4078.97+0.57
Flickr30k56.9557.29+0.34
FAA75.6176.20Approx. +0.60
AuthorsOpenReview ↗

Q1. Are the comparison conditions matched for MAny and FerVA?

Thank you for pointing out that attribution requires matching statistics collection, numerical precision, and deployment conditions. With these conditions matched and ridge regularization aligned, FerVA retains a 3.41-point FAA advantage over the MAny implementation evaluated here. This rerun reuses the same backbone and six experts, shares the same statistical matrices obtained from 256 source samples per task, and keeps statistics normalization and task weights identical. Statistics collection, the core solve, the exported weights, and inference all use the same single-precision floating-point format. To isolate language construction, all three conditions also share the same prototype routing and the same trained calibrator weights, use a deployment coefficient of γ=1.5 for the language updates, and evaluate the same 3,000 samples on each of the six tasks. The FAA difference is 3.44 points with MAny's original regularization and 3.41 points after regularization is matched, indicating that regularization differences are not the main source of the improvement in this comparison. At the task level, improvements are concentrated in CLEVR-Math, IconQA, and VizWiz-caption, while ImageNet-R and ArxivQA decrease. The table reports both directions, with differences calculated from unrounded scores.

TaskMAnyMAny with matched ridge regularizationFerVAFerVA minus MAny with matched ridge regularization (points)
ImageNet-R89.7389.7388.90−0.83
ArxivQA92.2392.2389.37−2.87
VizWiz-caption61.0461.0866.96+5.88
IconQA69.6369.7075.73+6.03
CLEVR-Math67.2367.2778.97+11.70
Flickr30k56.7356.7257.29+0.57
FAA72.7772.7976.20+3.41

Thank you also for drawing attention to the baseline settings. The full-system comparison follows MAny's publicly reported expert fine-tuning settings for UCIT–LLaVA: a batch size of 16, a learning rate of 2×10⁻⁴, and 1 training epoch. The additional comparison above reuses the same experts and fixes the deployment coefficient to examine differences in language construction based on those experts.

Thank you for suggesting that we consider fair comparison and performance attribution together. With ridge regularization matched, both methods target the same regularized reconstruction objective for each module, but their numerical solution procedures differ. The MAny implementation evaluated here uses recursive pseudoinverse updates with a numerical-rank tolerance, whereas FerVA solves the per-module normal equations after aggregating statistics from all tasks. Before deployment scaling, FerVA achieves a lower common objective value in all 224 modules. Together with the 3.41-point FAA advantage in the table, this provides measurements at both the reconstruction and downstream-performance levels. To examine output representation separately, we additionally fix the same direct solver and prototype routing. Shared output coordinates reduce the right-hand side of the equations to 96 columns, while matching the dense reference's scores on all six tasks at the reported precision. The faithful-space analysis characterizes the conditions for retaining input directions and compressing output coordinates. The two comparisons respectively examine differences between implemented construction procedures and fidelity under a compact representation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Thank You for Your Time, Encouragement, and Insights

We sincerely thank all three reviewers for their careful reading and constructive comments. We thank sTup for the positive assessment of our motivation, the two-feature example, and the cross-benchmark evaluation. We thank oNyH for recognizing the theoretical analysis of reconstruction fidelity conditions and the design combining language reconstruction with visual calibration. We also thank otDR for recognizing the value of the minimal faithful space in guiding model merging and the equal-rank recovery experiment in supporting the selection of recovery directions.

Your feedback has helped us clarify our contributions and add experiments on fair comparisons, robustness, and component contributions. We have reported these results in our point-by-point responses, with key conclusions highlighted in bold.

We greatly value your insights and the opportunity to learn from further discussion. We understand that your time is limited and would be grateful for any further thoughts whenever your schedule permits. We would be happy to clarify any remaining questions. Thank you again for your time and thoughtful guidance.

AuthorsOpenReview ↗

Closing Thanks and Summary of the Discussion

Dear Area Chair and Reviewers,

We sincerely thank all 3 reviewers for their careful reading, conscientious assessment, and thoughtful engagement throughout the discussion, and the Area Chair for coordinating the review process. We greatly appreciate the time and care devoted to examining our work and suggesting ways to strengthen it.

We are grateful for the positive ratings from all 3 reviewers and sincerely appreciate their recognition of our work. In particular, we thank sTup for recognizing our motivation and benchmark evaluation, oNyH for recognizing the reconstruction-fidelity analysis and visual-calibration design, and otDR for recognizing the faithful-space characterization and equal-rank recovery experiment. Following our responses, oNyH stated that they had no further questions, and otDR stated that they had no other concerns.

The rebuttal provides targeted evidence for robustness and component contributions. With language weights fixed, learned calibration increases FAA from 75.61 to 76.20, improving all 6 tasks [oNyH, W1]. Auxiliary supervision improves FAA in all 5 equal-budget training-order pairs, averaging 0.49 points [sTup, Q3; otDR, Q3]. The 5-level recovery experiment at γ=1.5 further associates lower reconstruction error with higher mean accuracy across the 4 accuracy-based tasks in all 4 adjacent comparisons along the examined path, including the final reversal [oNyH, W2].

The additional controls also distinguish compact representation from implemented language construction. With the same direct solver and prototype routing, shared output coordinates reduce the equation right-hand side to 96 columns while matching the dense reference’s scores on all 6 tasks at the reported precision [otDR, W1]. Separately, matching source statistics, ridge regularization, the deployment coefficient, and trained visual-calibrator weights leaves a 3.41-point FAA advantage over the MAny implementation evaluated here [sTup, Q1; otDR, Q1]. These findings complement the faithful-space theory and clarify the empirical contribution of each comparison.

We hope to incorporate the additional evidence and clarifications into a future version, in accordance with the conference’s requirements. If a future version permits a 10-page main text rather than the original 9 pages, the additional space would help us integrate these findings and explanations more clearly. We would use that space to present the key supplementary results, including the matched calibration ablation, while keeping detailed protocols in the appendix. We sincerely thank the reviewers and Area Chair for their time, care, and constructive guidance. We have learned much from this discussion and will carry these lessons into our future research.

With sincere thanks,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.