Endpoint Relations Extend Quantitative Readouts
Abstract
Reading familiar quantities is different from extending their scale. We study whether relations anchored to observed values can improve predictions beyond the range used to train a language-model readout. We learn a relation between distance from a source endpoint and nearby values, then use it to correct a global reader without new-range labels. Across four ordered tasks and 48 checkpoints, normalized absolute error decreases from 0.714 to 0.632, an 11.45% reduction, after selecting the global reader by its final prediction loss. The intact relation also outperforms controls that preserve the endpoint while disrupting the other pairings. Comparisons using the same features locate the useful ingredient: fitting the radius on all source values reduces error further to 0.529, and simple direct readers capture much of the gain. Controlled metric experiments then separate accessible quantity information from a distance that organizes it and a relation that continues across ranges. Together, these results show that quantitative extrapolation depends on the relation used to read a representation, and that endpoint relations can expose variation a global reader leaves unused.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.