acceptodds
Under review as a conference paper at ICLR 2027

Replacing Experts Without Retraining: Contract-Aware Composition of Sleep Models

Abstract

Many real-world AI applications require multiple models for different tasks within a larger system. The growing availability of pretrained and task-specific models makes it possible to assemble such systems from existing experts. While this reduces the amount of training during initial deployment, these systems should also evolve as stronger models, new architectures, and revised preprocessing become available. Ideally, replacing one expert in the system should not require retraining every other component, yet independently developed models may differ in their input and output contracts. We study this problem in medical sleep analysis, where related time-series tasks use data from the same patient but require different modalities and temporal resolutions. We propose an explicit model contract that specifies the required inputs and temporal meaning of a model’s predictions. We use these contracts to execute models over a common prediction envelope and to properly align class probabilities. We study four interacting polysomnography tasks with experts from three different model families. In our experiments, we score a total of 62.8 million windows from 1,820 recordings. Across all three families and four tasks, our contract-based alignment improves the F1 score by up to 0.2 points compared to naive alignment. More importantly, we show that zero-shot exchange of a strong expert from one family into another one can improve the performance of the system by up to 0.17 F1 score without impacting other tasks. To our knowledge, this is the first complete-night study of zero-shot replacement among heterogeneous, interacting sleep experts and the first systematic study towards model replacement in ML systems without fine-tuning or retraining.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series classification, foundational models, mixture of experts, ensemble, sleep

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_wn7sOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the composition of independently developed AI models into a joint system, addressing discrepancies in input and output contracts such as window size and resolution. The authors introduce contract-aware composition, instantiated on four expert models for sleep analysis (sleep staging, breathing-event classification, oxygen desaturation, and arousal detection). The outputs of these expert models are temporally aligned by physical time and refined using small linear correction heads that produce residual adjustments to their predictions. Experimental results show that contract-aware alignment outperforms naive tensor-based alignment. Furthermore, a stronger expert can be swapped into the composed system zero-shot, improving corresponding task performance without degrading the others.

Critical strengths

None

Critical weaknesses

The paper asserts a central problem without empirically validating its significance. Table 1 compares contract-based composition to a naive tensor-based baseline, but does not demonstrate that this naive approach reflects real-world practice or has caused real damage. At the same time, the experts used for demonstration and experiments are also selected and adapted by the authors, so the heterogeneity feels somewhat artificial.

This issue is compounded by the technical simplicity of the proposed mechanism. Since timestamp alignment and linear correction heads are off-the-shelf components, the paper’s contribution hinges almost entirely on the practical necessity of the problem, which remains unproven.

Questions for authors

Is there any concrete evidence (such as a real deployed system, a documented incident, or a industry survey) showing that the naive tensor-based alignment actually happens in practice and causes real damage? Providing such evidence would strengthen the motivation.

Reviewer_rPyXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose a framework for composing sleep model experts with different input requirements and output formats. With an explicit model contract, the pipeline achieve native execution and temporal alignment, while linear correction heads combine insights from several targets' predicted probabilities. The paper evaluates composition and expert replacement without retraining the remaining.

Critical strengths

Addresses a practical integration need that might be helpful for realworld project and practical application. Specifically, the observation that replacing the breathing expert improves desaturation detection without retraining, is an interesting discovery and worth deeper study in the future.

Critical weaknesses
  • The framework mainly concerns designing input/output wrappers, and linear probability fusion. Although practically useful, the paper does not establish a substantial research contribution beyond this plain integration.

  • The main comparisons provide limited evidence for the proposed contribution. Naive alignment can associate predictions with incorrect physical times, while the stronger rate only control often matches full contract alignment. Almost all observations from the experiments are expected with limited new insights

  • In general, I appreciate the effort behind this work and believe it could demonstrate its value in practical application. However, the methodological novelty and empirical insights demonstrated here are limited, and I do not find the overall research contribution sufficiently substantial for acceptance at ICLR.

Questions for authors

See above.

Reviewer_t6ZdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates model composition and replacement in heterogeneous multi-expert machine learning systems, aiming to enable expert updates without retraining other components. The authors propose a contract-aware composition framework that explicitly specifies each expert's input modalities, preprocessing procedures, temporal context, and output semantics. The framework aligns predictions from different experts based on physical timestamps and uses lightweight task-specific correction heads to integrate cross-expert predictions while preserving the independence of individual experts.

The authors conduct experiments on the Human Sleep Project (HSP 2.0) dataset, covering four tasks: sleep staging, arousal detection, respiratory event classification, and oxygen desaturation detection. Three expert model families—Sleepwalker, SleepFM, and OSF—are evaluated. Experimental results show that the proposed approach achieves better predictive performance than naive tensor-based alignment and enables stronger experts to replace existing ones without retraining other components, thereby improving performance on the corresponding tasks.

Critical strengths

This paper addresses model replacement and maintenance in heterogeneous expert systems, aiming to leverage cross-task information while preserving the independence of individual experts. This problem has practical significance, particularly for multi-task systems that require continuous updates to their model components.

Critical weaknesses

W1. The technical novelty is limited, and the necessity of the key components is insufficiently demonstrated.

The proposed framework mainly consists of input-output contracts, temporal alignment, concatenation of predicted class probabilities, and linear correction heads, offering relatively limited technical novelty. More importantly, the existing experiments do not adequately establish the independent contributions and necessity of these key components, weakening the paper's methodological contribution.

W2. The experimental validation of model replaceability is insufficient to support the paper's central claims.

The paper presents expert replacement without retraining as a major contribution, but the experiments are limited to replacing the sleep and breathing experts in SleepFM and OSF systems with their Sleepwalker counterparts. Furthermore, the observed performance improvements may primarily stem from the stronger predictive capabilities of the replacement experts themselves, rather than from the proposed composition mechanism.

Questions for authors

Q1. Table 1 shows that Joint Fine-tuning substantially outperforms training only the correction heads across all four SleepFM tasks, whereas Table 2 evaluates zero-shot expert replacement only in systems with frozen experts. Could the authors further evaluate expert replaceability after joint fine-tuning and analyze whether the model coupling introduced by joint optimization undermines the effectiveness of zero-shot replacement?

Q2. To what extent do the performance improvements in the expert replacement experiments result from the stronger predictive capabilities of the replacement experts themselves? Could the authors include a baseline that directly replaces independent experts without using cross-task correction heads to quantify the additional benefits of the proposed composition mechanism?

Q3. Could the authors further evaluate the robustness of the framework under other expert replacement scenarios, such as replacing arousal or desaturation experts, replacing multiple experts simultaneously, or using experts with different probability calibration characteristics? Such experiments would help substantiate the paper's central claims regarding model replaceability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.