acceptodds
Under review as a conference paper at ICLR 2027

Which Host Versions Should a Removable Branch Learn From?

Abstract

A branch trained beside a continuing host should be removable without changing the host's training. Optimization-Recoverable Augmentation () enforces this through complete-state isolation, with conditional guarantees and Qwen/BERT recovery tests. Which host versions should the branch observe before deployment on the final host? In one-block Qwen2.5-0.5B, reader-progress observation () exceeds terminal-only training by on GSM8K/SQuAD with only the version sequence changed. The advantage survives terminal learning-rate and checkpoint selection: on a separate estimation half of the historical holdout. Two prospective comparisons with latest-version observation give positive paired differences. The benefit is conditional on the branch's data. Independent branch orders attenuate the gain over latest-version observation. In a preregistered equal-size data control, exceeds terminal-only by when host and branch share a training stream, but trails by when host and branch train on group-disjoint data. A retained-seed extension separates order and data contrasts: independent ordering reduces the shared-data advantage, while SQuAD retains a positive gap. The tested full-model recipe also favors terminal-only training. These results establish a repeatable shared-stream observation effect, rather than a general advantage of earlier host versions, and show why observation policies must be evaluated jointly with branch data and the deployment host.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.