acceptodds
Under review as a conference paper at ICLR 2027

A Model Zoo Is Not a List: Clone- and Reparameterization-Invariant Predictive Aggregation

Abstract

A model zoo is usually treated as a list whose rows receive prior mass. Saving a checkpoint twice, sampling a regularization path more densely, or renaming its coordinate changes the aggregate, although the encoded predictive family is unchanged. We remove this registry representation bias with representation-invariant measure ensembling (RIME), an output-only few-shot adapter that replaces row counts with quotient mass over exact predictive atoms and Hellinger arc mass along declared scalar paths. Under the declared family, path-order, embedding, and target-slice conventions, RIME is provably invariant to exact clone splitting and smooth monotone reparameterization; finite grid refinement is only approximately stable, with a bounded residual. We evaluate a 41-row registry on four benchmarks, two of them under pre-specified protocols. Duplicating one PACS branch sixteen times moves flat-row aggregation by up to 0.583 total variation, whereas RIME is invariant up to floating-point precision and cuts near-clone drift 440-fold. A NICO++ confirmation, planned before its download, reproduces both effects. Invariance is not paid for in accuracy: RIME has the lowest mean Brier of fifteen output-only methods. Against a matched family-balanced control it ties at 4 labels/class and gains 0.00069 at 16, 0.7% of the adaptation gain and unconfirmed under target-level clustering. The evidence supports representation stability at no measured predictive cost, not general baseline superiority: a zoo's bookkeeping can stop being a modeling decision. Code is available at https://anonymous.4open.science/r/rime-B03C.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
model zoo, ensembling, reparameterization invariance, Hellinger geometry, few-shot adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TTuJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper identifies a representation bias in model-zoo aggregation: duplicating checkpoints or changing the sampling density and parameterization of model families can alter ensemble predictions without meaningfully changing the underlying predictive family. The authors propose Representation-Invariant Measure Ensembling (RIME), which combines exact predictive quotienting with Hellinger arc-length weighting to construct a representation-invariant base measure. Cross-fitted Brier-loss evidence is then used to adapt the ensemble weights. The paper establishes theoretical results for exact clone-splitting and monotone reparameterization invariance, along with approximation bounds for finite-grid refinement. Experiments on PACS, NICO++, Office-Home, and ACSIncome demonstrate strong stability under registry transformations.

Critical strengths
  1. The paper identifies a subtle but meaningful limitation of conventional ensemble aggregation, where arbitrary registry choices can influence predictions. The proposed combination of predictive quotienting and Hellinger arc-length weighting provides a mathematically motivated solution, supported by explicit invariance guarantees.

  2. The experiments directly evaluate the proposed invariance properties through checkpoint duplication, near-cloning, and grid refinement. The results demonstrate substantial improvements in representation stability without meaningful degradation in predictive performance. I also appreciate the careful experimental analysis and transparent discussion of the limited predictive gains.

Critical weaknesses
  1. The invariance guarantees are conditional on a fixed family partition, path ordering, predictive embedding, and target-input slice. Changing these specifications can alter the resulting aggregation. While the authors acknowledge these limitations, they restrict the broader applicability of the representation-invariance claim.

  2. The strongest improvements arise from deliberately constructed registry perturbations, whereas predictive gains over the matched family-balanced baseline are very small and not robust under conservative target-level clustering. Additional evidence from naturally occurring model registries would help establish the practical significance of the proposed method.

Questions for authors
  1. How sensitive is RIME to different reasonable family partitions or path definitions? Can the authors provide guidance for specifying these structures in practical model zoos where the underlying families may not be explicitly known?

  2. How frequently do the identified registry biases meaningfully affect predictions in naturally constructed model zoos, without deliberately introducing clones or refining selected paths?

  3. What motivates Hellinger geometry over alternative invariant predictive embeddings beyond the stated analytical properties? Could the authors clarify when different geometric choices might lead to substantially different ensemble predictions?

4 comments
AuthorsOpenReview ↗

Response to Reviewer TTuJ (1/2)

Thank you for the careful review. We address each point below; all new results are in App. Z of the revision (marked in blue).

W1: "The invariance guarantees are conditional on a fixed family partition, path ordering, predictive embedding, and target-input slice..." Agreed: the guarantees are conditional, and the paper should say more precisely which guarantee depends on which declaration.

  • Some declaration is unavoidable. "Refinement" and "relabeling" are defined only relative to a path. Prop 2 shows that no rule that sees only row order and row count can be neutral to refinement. The declaration is therefore the minimal extra input; it is not an arbitrary add-on.
  • Each guarantee needs less than the full specification.
    • Exact-clone invariance needs only family membership and branch keys. In E1 it holds under all 10 declarations, including label-free automatic ones (max TV ≤ 4.3e-15).
    • Relabeling invariance additionally needs the path order.
    • Refinement stability additionally needs the path and the embedding.
    • The target slice enters only through the chords. App. B.2 gives a concentration rate. In App. V, prior-predictive TV falls from 1.7e-3 with 64 slice images to 3.2e-4 with 2048.
  • Changing the declaration itself is a modeling change; E1 (Q1) quantifies how much this matters in practice.

W2: "...predictive gains over the matched family-balanced baseline are very small and not robust under conservative target-level clustering. Additional evidence from naturally occurring model registries would help..." Agreed on the gains. Against FABLE, RIME ties at 4 labels/class and gains 0.00069 Brier at 16 labels/class, which is 0.7% of the adaptation gain. The interval crosses 0 under target clustering (Table 3). Our claim is "free, not profitable": RIME is insurance against registry bookkeeping. We added evidence from natural registries (E2, E5; details in Q2) and from registry edits with real refits:

  • E3 (App. Z.3) builds 2280 edited registries per method by cloning, one-sided refinement with real refits, and near-cloning.
    • Cloning the best candidate ×16 moves flat rows by up to 0.078 Brier in a cell, more than the mean adaptation gain (0.062 at 4 labels/class), and by up to 7.2 pp in accuracy. FABLE's worst cell over all edits moves 0.037 Brier and 2.4 pp.
    • Release flips: flat 9, FABLE 3, RIME 0.

Q1: "How sensitive is RIME to different reasonable family partitions or path definitions? Can the authors provide guidance...?" Sensitivity (E1, App. Z.2). We applied 10 declarations to the same fitted PACS registry: merged singletons, merged logistic-head paths, paths split in half, no paths, two label-free automatic declarations, and three random path orders as a control.

  • Stable:
    • the gain over the fallback (−0.097 to −0.10 Brier, five-seed, 16 labels/class);
    • exact-clone invariance;
    • no Brier-harmful unit after the release gate;
    • at most 4/300 release flips per budget.
  • Not stable: the small edge over FABLE. It appears only when a real path order is declared (−0.00054 to −0.00072 for D0, D1 and D3). It is exactly 0 with no paths declared and ≈ 0 with the automatic declarations. It turns slightly unfavorable with shuffled orders (up to +0.0015), which shows that the declared order is genuinely used.

Guidance (a new paragraph in the main text, details in App. Z.2):

  • Family. Use the training recipe or source that produced the models.
  • Path. Declare a single swept scalar whose order is known, such as a regularization strength, temperature or training epoch. Declare multi-dimensional sweeps per one-dimensional slice, or as unordered.
  • Unknown order. Declare the family unordered. This equals FABLE plus the exact quotient and keeps exact-clone invariance. A wrong order is worse than none.
  • Near-duplicates of a path member. Declare them on that path. This gives about 150× less drift than an unordered declaration (E4).
  • Unknown families. A label-free automatic declaration that clusters the deployed predictions keeps the fallback gain and clone invariance, although its clusters do not recover the registry families (ARI 0.18–0.22). It loses the edge over FABLE.
  • Timing. Fix all declarations before any outcome is seen.
AuthorsOpenReview ↗

Response to Reviewer TTuJ (2/2)

Q2: "How frequently do the identified registry biases meaningfully affect predictions in naturally constructed model zoos...?" Frequently at the prediction level, but rarely at the decision level. E2 (App. Z.6) audits unedited registries: 1080 PACS cells, plus ACSIncome, Office-Home and NICO++ (the latter with 2 of 5 bank seeds).

  • Natural exact collisions occur in every cell (PACS: 41 rows → 34–40 atoms).
  • Predictions (compared with RIME on PACS, 4 / 16 labels/class):
    • Per-cell TV exceeds 0.01 in 58% / 17% of cells for FABLE and 80% / 19% for flat rows (median TV for FABLE 0.011 / 0.0058).
    • The predicted class changes on about 1% / 0.4% of test examples.
  • Performance: mean ΔBrier stays within ±0.00078.
  • Decisions: releases differ in ≤ 4/540 cells per budget. In every such case RIME releases and the row rule abstains.
  • External registries: |mean ΔBrier| ≤ 0.00017, with no release changes (all methods abstain there).
  • Mechanism. FABLE and RIME differ only inside path families, where arc mass gives little weight to near-converged low-penalty endpoints. Flat rows instead change whole-family weights by row count: on PACS, a 6-row path family gets 1.6× RIME's family mass and a singleton family 0.28×.

E5 (App. Z.7) pools five NICO++ training seeds into a large real zoo of 101–111 checkpoints per registry. All base measures agree within 0.001 Brier on every architecture–target unit, because the evidence puts ≥ 0.98 posterior mass on the fallback. Yet the flat prior already carries a 1.6× domain imbalance (0.21 vs 0.13) from uneven checkpoint saving; FABLE and RIME give 0.18 per domain. The bias exists, but in these zoos it is swamped by evidence. It becomes large when the registry is edited (W2).

Q3: "What motivates Hellinger geometry over alternative invariant predictive embeddings...? ...when different geometric choices might lead to substantially different ensemble predictions?" Motivation beyond analytical convenience (App. Z.1). With the target slice QQ fixed, 4∥Ψ˙∥24\|\dot\Psi\|^2 is the Fisher metric of the joint law over (slice input, class). Two facts follow:

  • Uniqueness of the metric. By Chentsov's theorem and its conditional version (Montúfar, Rauh & Ay, 2014), this metric is the unique Riemannian metric, up to scale, that is invariant under homogeneous conditional embeddings. Examples: splitting a class into fixed-proportion sub-classes, or replicating the slice. The Hellinger chord is exactly invariant under these operations; probability-Euclidean and centered-log chords are not.
  • Jeffreys prior. The arc measure is the Jeffreys prior of the path (Remark after Thm 1).

Scope of this claim. The theorem fixes the metric, not the finite chord. All ff-divergences share the same local Fisher form, which is why root Jensen–Shannon stays closest to Hellinger (below). The theorem also says nothing about which geometry gives the lowest loss.

When geometries differ (E6, App. Z.8). The path speed weights the motion of class cc by pcp_c (Hellinger), pc2p_c^2 (probability-Euclidean) or 11 (centered-log), so geometries can disagree only on segments that move probabilities near the simplex boundary (likelihood ratio ≤ β/α\sqrt{\beta/\alpha} if p∈[α,β]p\in[\alpha,\beta]). We measured this on all 720 native PACS cells (6 seeds, 4 and 16 labels/class), with criteria fixed in advance:

  • Root-JS never matters (mass provably within 1.2× of Hellinger; max TV 0.0053). Euclidean barely matters (mean TV 0.0013, max 0.070; 1 of 720 cells substantially different).
  • Centered-log (with the paper's clipping) changes individual cells: mean TV 0.0079, max 0.16, predicted class changes on up to 4.1% of a cell's test examples. 37% / 17% of cells are substantially different (mean TV > 0.01, a release flip, > 1% argmax change, or |ΔBrier| > 0.005).
  • Where: it moves base mass from the strongly regularized start of the logistic, ridge and entropy paths (0.4–0.6× Hellinger) to their weakly regularized, near-boundary ends (C 10→100, λ 0.01→0, η 4→8: 1.6–2.5×).
  • Evidence gates it: per cell, centered-log's predictive TV is uncorrelated with its base-measure TV but tracks posterior mass on the reweighted families (Spearman 0.75 / 0.55).
  • Decisions: mean ΔBrier vs fallback differs from Hellinger's by < 0.001 in every seed and budget. Releases flip in 4 of 2160 comparisons, all knife-edge (Hellinger's log e-value 5.08–5.31 vs threshold 5.075).

We would be glad to answer any further questions.

Thank you for the clarifications and additional experiments, which substantially address my questions. While practical gains beyond representation stability remain limited, the rebuttal strengthens my assessment. I will maintain my rating of weak acceptance.

Thank you very much for taking the time to review our clarifications and additional experiments! We truly appreciate your constructive feedback throughout the process and are glad that our responses strengthened your positive assessment of our work.

Reviewer_PVsLOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes RIME, an output-only few-shot aggregation rule that combines predictive quotienting with Hellinger arc mass, followed by evidence-based reweighting. RIME is provably invariant to exact clone splitting and smooth monotone reparameterization, while achieving approximate stability under finite-grid refinement with a bounded residual error.

Critical strengths
  1. The paper defines a precise and worthwhile problem. It explains how registry representation can silently determine a prior.
  2. The paper provides good alignment between theory and experiments. The invariance properties also have corresponding stress tests, rather than being inferred from predictive accuracy.
  3. Strong reproducibility specification. The source code is available online, and the appendices provide detailed registry construction, label access, solver settings, sensitivity analyses, and failure cases.
Critical weaknesses
  1. The paper is overly dense and jargon-heavy, making it difficult for readers outside the field to follow. Many numbers are reported to more digits than their uncertainty justifies. A shorter main text built around one clear running example would help a lot.
Questions for authors
  1. How sensitive are practical conclusions to plausible alternative family declarations?
3 comments
AuthorsOpenReview ↗

Response to Reviewer PVsL

Thank you for the review and the concrete suggestions. We accept all of them.

W1: "The paper is overly dense and jargon-heavy, making it difficult for readers outside the field to follow." Agreed. The revision makes four changes (in blue in the PDF):

  • Running example (Figure 1, p. 4). A new Figure 1 and paragraph in Sec. 3 introduce the terms on a toy registry: a declared path, an unordered family of three sources, a singleton and the fallback. Saving one source four times and re-saving one path segment shifts flat-row mass from the singleton to the path (0.13 → 0.053) and doubles FABLE's mass on the copied source (0.10 → 0.20), while RIME's family and atom masses are unchanged.
  • Glossary. The appendix now opens with a glossary (Table 5, p. 15) of the terms the paper introduces.
  • Positioning table (Table 1, p. 3). A new table in the main text shows which registry transformations are handled by flat rows, family balancing (FABLE), hash deduplication, tolerance deduplication, the continuous Jeffreys prior, and RIME.
  • Shorter main text. Sec. 6.1 is condensed, the six-family results table now appears only in App. T, and all new results appear in App. Z with only short pointers in the main text.

W2: "Many numbers are reported to more digits than their uncertainty justifies." Agreed. The abstract, main text and main tables now use two significant digits. For example, 0.583 becomes 0.58, and 3.65×10−33.65\times10^{-3} becomes 3.7e-3.

Suggestion: "A shorter main text built around one clear running example would help a lot." The new figure in Sec. 3 is that running example.

Q1: "How sensitive are practical conclusions to plausible alternative family declarations?" We ran a new experiment on this (E1, App. Z.2). It applies 10 declarations to the same fitted PACS registry, on the development seed and on the five-seed campaign, at 4 and 16 labels/class. The declarations are:

  • D0: the paper's declaration;
  • D1: merge pairs of singleton families;
  • D2: merge the two logistic-head paths into one unordered family;
  • D3: split every path in half;
  • D4: declare no paths;
  • D5a/b: two label-free automatic declarations that cluster the deployed predictions;
  • D6: three random path orders, as a control.

What does not change:

  • The Brier gain over the pooled fallback stays between −0.097 and −0.10 (five-seed, 16 labels/class) and between −0.062 and −0.064 (4 labels/class).
  • Exact-clone invariance holds under every declaration (max TV ≤ 4.3e-15). FABLE drifts by 0.16–0.52 under the same clones.
  • No architecture–target unit becomes Brier-harmful after the release gate (Deploy).
  • Release decisions change in at most 4 of 300 cells per budget.

What changes: the small edge over FABLE, on which the paper makes no superiority claim. It comes entirely from the declared paths. At five seeds and 16 labels/class, the edge is:

  • −0.00069 (D0), −0.00072 (D1), −0.00054 (D3) and −0.00018 (D2);
  • exactly 0 when nothing is declared as a path (D4);
  • ≈ 0 for the automatic declarations (D5: −6.9e-5 and −5.1e-5, both intervals include 0);
  • slightly unfavorable under shuffled orders (D6: up to +0.0015).

The D6 result shows that the declared order is genuinely used.

Practical guidance. Declare a path only when its scalar order is genuinely known. Otherwise declare the family unordered; this keeps exact-clone invariance and matches FABLE's utility. A guidance paragraph in the main text and App. Z.2 spell this out.

We would be glad to answer any further questions.

Thanks authors for the response. It clarifies most of my questions. I maintain my initial evaluation score.

Thank you very much for taking the time to read our response—we are glad it helped clarify your questions and truly appreciate your constructive feedback throughout the review process!

Reviewer_BSPrOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

RIME aggregates a model zoo using a base measure that ignores registry bookkeeping. Exact-duplicate predictors are merged, and declared scalar paths get Hellinger arc mass over unlabeled target inputs. Cross-fitted Brier evidence then reweights this measure. The paper proves invariance to exact clones and to monotone relabeling, plus a grid-approximation bound. Empirically, clone and refinement drift drop sharply on PACS and NICO++, with no measurable predictive cost or gain against a family-balanced control.

Critical strengths

The proofs seem correct. The empirical reporting is unusually honest: pre-specified protocols, clustered re-analyses, and a reported failure case.

Critical weaknesses

Limited novelty.
The exact-clone result is matched by plain hash deduplication (Table 2). No predictive benefit?

Questions for authors

How does RIME differ from the Jeffreys prior on the declared path? When do branch keys change outputs relative to hash deduplication? How does RIME behave with off-path near-duplicates? Is there any setting with a large real zoo where the base measure materially changes performance?

2 comments
AuthorsOpenReview ↗

Response to Reviewer BSPr (1/2)

Thank you for the precise review. We agree with the facts behind the weaknesses and now state them explicitly in the paper.

W1: "Limited novelty." The ingredients are classical, and we thank you for pressing on the Jeffreys connection: on a declared path the continuous measure is the Jeffreys prior (Q1), and hash deduplication handles exact clones (W2). The contribution is to carry the Jeffreys idea from a smooth parametric model to a finite registry of saved prediction arrays, which needs what neither ingredient provides:

  • (i) a finite, derivative- and coordinate-free rule that uses only saved outputs and is exactly invariant to monotone relabeling at every resolution;
  • (ii) an exact quotient with evidence branch keys, which is both order-invariant and clone-neutral; keep-first and per-row hash dedup are each only one of these (Q2);
  • (iii) one base measure that mixes path, unordered, singleton and fallback atoms;
  • (iv) Prop 2 (no rule using only row order and count is refinement-neutral), the finite-grid bound (Prop 4), and systematic stress tests.

A new positioning table (Table 1) shows which of five registry transformations (exact clone, row permutation with heterogeneous evidence, near clone on a path, path refinement, off-path near-duplicate) flat rows, FABLE, hash dedup, tolerance dedup, continuous Jeffreys and RIME each handle.

W2: "The exact-clone result is matched by plain hash deduplication (Table 2)." Agreed. The Table 2 row "Equal family + exact quotient" is hash deduplication. It is now renamed to say so, and we added a tolerance-dedup (10−310^{-3}) row. Hash dedup does not cover three cases:

  • Near clones on a path. Mean TV is 3.7e-3 for hash dedup, 2.6e-3 for tolerance dedup and 4.5e-5 for RIME.
  • Path refinement and near clones (E3, App. Z.3). With one-sided refinement using real refits, RIME's mean |ΔBrier| is 11–27× smaller than FABLE+hash. For near clones it is at least ~100× smaller.
  • Atoms whose identical predictions carry different evidence (Q2).

In absolute terms these increments are small: FABLE+hash moves Brier by at most 8.0e-4 on average and 0.011 in the worst cell (RIME: 4.1e-5 and 6.4e-4), and it flips no release decision.

W3: "No predictive benefit?" Correct, and we claim none. Against FABLE, RIME ties at 4 labels/class and gains 0.00069 Brier at 16 labels/class. That is 0.7% of the adaptation gain, and the interval crosses 0 under target clustering (Table 3). The value is stability against how the registry was saved. In E3 (App. Z.3) we built 2280 edited registries per method by cloning, refining and near-cloning:

  • Cloning the best candidate ×16 moves flat rows by up to 0.078 Brier in a cell, more than the mean adaptation gain (0.062 at 4 labels/class). Accuracy moves by up to 7.2 pp. FABLE's worst cell over all edits moves 0.037 Brier and 2.4 pp. RIME's drift is floating-point zero for clones and at most 6.4e-4 Brier per cell otherwise.
  • Release flips: flat 9, FABLE 3, RIME 0.

Natural registries are covered in Q4.

Q1: "How does RIME differ from the Jeffreys prior on the declared path?" In the continuum, it does not. We now state this in a Remark after Thm 1, with the proof in App. Z.1. Read the path as the label model Y∣X∼pt(⋅∣X)Y\mid X\sim p_t(\cdot\mid X) with X∼QX\sim Q (the unlabeled slice), where pt>0p_t>0 is C1C^1. Since ∂tpt=∂tpt/(2pt)\partial_t\sqrt{p_t}=\partial_t p_t/(2\sqrt{p_t}), ∥Ψ˙(t)∥2=EQ∑c(∂tpt)2/(4pt)=14 EQEY∼pt[(∂tlog⁡pt(Y∣X))2]=I(t)/4\|\dot\Psi(t)\|^2=\mathbb E_Q\sum_c(\partial_t p_t)^2/(4p_t)=\tfrac14\,\mathbb E_Q\mathbb E_{Y\sim p_t}[(\partial_t\log p_t(Y\mid X))^2]=I(t)/4. So I(t)=2∥Ψ˙(t)∥\sqrt{I(t)}=2\|\dot\Psi(t)\|, and normalizing turns Eq. (2) into the Jeffreys prior of the path. Thm 1 is the one-dimensional case of this prior's classical reparameterization invariance; its finite-resolution content is the exact invariance of the discrete rule.

Two clarifications:

  • Only the base measure coincides. The update uses cross-fitted Brier evidence, not the Jeffreys model's likelihood.
  • The chord satisfies d=2sin⁡(DFR/4)d=2\sin(D_{FR}/4), where DFRD_{FR} is the Fisher–Rao distance. Using geodesic lengths would change chords only at relative order d2/24d^2/24.

What RIME adds is (i)–(iv) in W1; in particular:

  • Jeffreys needs ∂tpt\partial_t p_t; RIME needs only prediction arrays.
  • A Riemann sum of I\sqrt{I} needs coordinates and is invariant only in the limit.
  • Jeffreys does not say how to weigh path, unordered and singleton families against each other.
AuthorsOpenReview ↗

Response to Reviewer BSPr (2/2)

Q2: "When do branch keys change outputs relative to hash deduplication?" Only when byte-identical deployed predictions carry different OOF evidence, or when an atom is shared across families. If all rows of each atom carry the same evidence, as true saved copies do, RIME and hash dedup give identical outputs. Otherwise hash dedup must choose a rule, and each choice loses one property. We tested this in E3b (App. Z.4) on native PACS registries; 18/60 development cells at 4 labels/class contain such an atom.

  • Keep-first is clone-neutral but order-dependent. Reordering family blocks (path order preserved) changes the output in 38/72 cases (max TV 0.097, max |ΔBrier| 0.0056).
  • Splitting atom mass per row (the Table 2 control) is order-invariant but not clone-neutral; it was invariant in Table 2 only because the cloned atom there has homogeneous evidence. Cloning one row ×16 changes the output in 48/86 cases (max TV 0.015).
  • RIME (equal mass per distinct branch key) is invariant to both (≤ 5.6e-16).

In a two-row toy example, swapping the rows moves the hash-dedup weight of the shared atom from 0.96 to 0.77. The practical effect is small (no release flips in E3b). Branch keys are what make RIME a well-defined function of the registry as a multiset.

Q3: "How does RIME behave with off-path near-duplicates?" Like FABLE, and slightly worse; the exact results do not cover this case. App. Z.5 gives the bounds:

  • Inserting rows into an existing declared family moves the base measure only inside that family, with TV < πf=(1−π0)/∣F∣\pi_f=(1-\pi_0)/|\mathcal F| = 0.082.
  • Posterior TV is ≤ πfG+/(πfG++π0)\pi_fG^+/(\pi_fG^++\pi_0), where G+G^+ is the family's largest Gibbs factor.

Neither bound depends on the number of inserted rows. The cap comes from the family budget, so FABLE shares it. Flat rows have no cap: kq/(kq+C)→1kq/(kq+C)\to1 (Prop 1).

E4 inserts K∈{1,4,16}K\in\{1,4,16\} near copies into the unordered fixed-source family:

  • Base measure: TV is exactly πfK/(3+K)\pi_fK/(3+K) = 0.020, 0.047 and 0.069 for FABLE, hash dedup and RIME. For flat rows it is 0.26 at K=16K=16.
  • Predictive drift (best-source copies, K=16K=16, α=0.1\alpha=0.1, 4 labels/class, mean TV / mean |ΔBrier|): flat 0.013 / 4.3e-3, FABLE 0.0026 / 7.1e-4, RIME 0.0026 / 7.3e-4.

The practical lever is the declaration. With 16 near copies at αmax⁡=0.1\alpha_{\max}=0.1, RIME's mean TV is 4.3e-5 for near-clones declared on their path (E3) versus 0.0067 for ridge copies declared in an unordered family (E4), about 150× more. The anchors differ, so the ratio is indicative (App. Z.5).

Q4: "Is there any setting with a large real zoo where the base measure materially changes performance?" We found no such setting, and we do not claim one.

  • E5 (App. Z.7). We pooled five NICO++ training seeds into registries of 101–111 real checkpoints per architecture–target, saved unevenly across domains. We compared six base measures: flat, hash dedup, FABLE by domain or by trajectory, and RIME with or without epoch-trajectory paths. Under the paper's settings all six agree within 0.001 Brier on every architecture–target unit. The evidence puts ≥ 0.98 posterior mass on the fallback, and the gates abstain. The bias still exists: because of uneven checkpoint saving, the flat prior gives source domains 0.13 to 0.21 (1.6×), while FABLE and RIME give 0.18 each.
  • E2 (App. Z.6): 1080 natural PACS cells, no edits.
    • Relative to RIME at 4 labels/class, median per-cell TV is 0.011 for FABLE and 0.016 for flat rows. Mean ΔBrier stays within ±0.00078.
    • Release decisions differ in ≤ 4/540 cells per budget, always with RIME releasing and the row rule abstaining.
    • On ACSIncome, Office-Home and NICO++ (2 of 5 bank seeds), |mean ΔBrier| ≤ 0.00017 and no release decision changes (all methods abstain).

In short, the bias is frequent but small at the prediction level, rarely affects decisions in natural zoos, and becomes large only when the registry is edited (W3).

We would be glad to answer any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the reviewers for careful and constructive reviews. Reviewers found the proofs correct and the empirical reporting transparent; the open questions concern novelty relative to Jeffreys priors and hash deduplication, and practical significance. On a declared path, RIME's continuous measure is the Jeffreys prior of that predictive path. Plain hash deduplication does handle exact clones. Gains over the family-balanced control (FABLE) are small and not robust under target clustering. RIME's contribution is a finite, derivative-free rule built only from saved outputs. It combines an exact quotient with evidence branch keys, mixes unordered, singleton and path families, and comes with finite-grid bounds and stress tests. Together these give the targeted invariances at no measured predictive cost. In practice, RIME is invariance insurance rather than a source of accuracy.

New evidence

  • Jeffreys equivalence (App. Z.1). I(t)=4∥Ψ˙(t)∥2I(t)=4\|\dot\Psi(t)\|^2, so the normalized arc measure equals the Jeffreys prior of the declared path. We also add the conditional Chentsov theorem (Montúfar, Rauh & Ay, 2014) as motivation for the Fisher/Hellinger geometry.
  • Declaration sensitivity (E1, App. Z.2), 10 declarations on PACS. The gain over the fallback stays within −0.097 to −0.10 Brier (five-seed, 16 labels/class). Exact-clone invariance holds under every declaration (max TV ≤ 4.3e-15). No Deploy unit is Brier-harmful. The small edge over FABLE appears only when a real path order is declared: it is exactly 0 with all-unordered declarations and ≈0 with label-free automatic ones.
  • Registry edits (E3, App. Z.3), 2280 edited registries per method. Flat rows: worst cell ΔBrier 0.078 and Δacc 7.2 pp. Release flips: flat 9, FABLE 3, RIME 0.
  • Branch keys vs hash dedup (E3b, App. Z.4). On atoms with heterogeneous evidence, keep-first hash dedup depends on row order (38/72 permutations change the output). Splitting mass per row is not clone-neutral (48/86 clones change it). RIME is invariant to both.
  • Off-path near-duplicates (E4, App. Z.5). RIME ≈ FABLE here. Both are capped by the family mass (πf\pi_f = 0.082 for the base measure), independently of how many rows are inserted. With 16 inserted copies, flat rows reach a base-measure TV of 0.26. Declaring the copies on their path cuts drift about 150×.
  • Natural registries (E2, App. Z.6), 1080 PACS cells plus ACSIncome, Office-Home and NICO++, with no inserted edits. Every PACS cell contains natural exact collisions (41 rows → 34–40 atoms). Prediction-level differences from RIME are frequent but small (median TV 0.011 FABLE, 0.016 flat, 4 labels/class). They rarely reach decisions: at most 4/540 release disagreements per budget, all cases where RIME releases and the row rule abstains.
  • Large real zoo (E5, App. Z.7), NICO++, five training seeds pooled, 101–111 saved checkpoints per registry. All base measures agree within 0.001 Brier on every architecture–target unit. The flat prior already has a 1.6× domain imbalance caused by uneven checkpoint saving, but the evidence swamps it.
  • Geometry (E6, App. Z.8), 720 PACS cells. Root-JS and Euclidean barely change predictions. Centered-log changes individual cells (max TV 0.16), through the weakly regularized ends of logistic, ridge and entropy paths and only when the posterior sits there. Mean ΔBrier stays within 0.001 of Hellinger, and releases flip in 4/2160 knife-edge comparisons.

Revisions (changes in blue in the revised PDF)

  • Main text:
    • Sec. 6.1 condensed, and the six-family results table now appears only in App. T.
    • Remark after Thm 1 stating the Jeffreys equivalence (proof in App. Z.1).
    • Positioning table (new Table 1): which of five registry transformations flat rows, FABLE, hash dedup, tolerance dedup, continuous Jeffreys and RIME each handle.
    • Running-example figure in Sec. 3.
    • Declaration guidance paragraph.
    • Table 2: the quotient row is renamed to state that it is hash deduplication, and a tolerance-dedup row is added.
    • Short pointers to the new evidence.
    • Two significant digits in the abstract, main text and main tables.
  • Appendix:
    • A glossary at the start of the appendix.
    • New App. Z: Z.1 Jeffreys proof and Chentsov; Z.2 declaration sensitivity and guidance (E1); Z.3 registry-edit consequences (E3); Z.4 branch keys vs hash dedup (E3b); Z.5 off-path bound and stress test (E4); Z.6 natural registries (E2); Z.7 large checkpoint zoo (E5); Z.8 when geometries change predictions (E6).
  • Minor notational clarifications.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.