Inducing Unseen Programs as Transformations in a Learned Representation

Program induction uses input–output examples to infer how an unknown pro- gram behaves on further inputs. We ask whether a shared representation of data can make this inference possible for programs absent from training. We intro- duce Joint-Embedding Program Inductive Architecture (JEPIA), which encodes a task’s input–output examples and fits a transformation between their embeddings. The fitted transformation predicts output embeddings for further inputs. Errors on these predictions train the shared encoder through the fitting procedure, so the representation is learned for the inference it must support. We instantiate JEPIA with rotations and reflections, which admit an analytical fit, and regularize the em- beddings against collapse. In modular addition, digit embeddings acquire a ring structure in numerical order, providing an interpretation of how the model pre- dicts held-out additions. On cube transformations, the model transfers from two- move training programs to programs generated by shorter and longer sequences. Additional context examples and broader exposure to training programs improve held-out prediction. These results support learning data representations that allow a common fitting procedure to infer new programs from examples.