acceptodds
Under review as a conference paper at ICLR 2027

Forward-Only Learning Across Black-Box Interfaces: State Persistence and Pilot Recovery

10 views

Abstract

A changing black-box interface can present different coordinates to a downstream learner even when every realized transformation is invertible. We study how interface-state persistence and observability affect forward-only local learning without gradient propagation across the interface. Matched fixed and resampled channels share the same state distribution and paired upstream representations, separating per-state properties from consistency across calls. Experiments cover vanilla Forward-Forward, channel-wise competitive learning, supervised block-local learning, and an external DeeperForward ResNet. Matched Rademacher comparisons show that resampling invertible, norm-preserving states can cause substantial accuracy degradation. We introduce shared pilots that pass through the same hidden state as the data, making structured interface transformations observable from forward outputs. For invertible diagonal states and signed permutations, pilot constructions enable exact representation recovery under noiseless observations without upstream credit. On frozen DeeperForward representations, a single shared pilot restores downstream accuracy from 13.73% to 76.90%, matching the identity condition in all five paired seeds. These results distinguish per-state information preservation from downstream usability and identify state persistence or observability as key interface properties for forward-only learning.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Forward-Only Learning, Black-Box Interfaces, State Persistence, Representation Observability, Pilot Recovery, Interface Identifiability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sDXsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies forward-only learning across changing black-box interfaces, where downstream modules learn from upstream representations without gradient propagation through the interface. The paper investigates whether preserving information at each interface state is sufficient for downstream learning when the hidden interface transformation changes across calls. The authors provide exact recovery conditions for diagonal transformations and signed permutations, and evaluate the approach on multiple forward-only learners and representations. Experiments on CIFAR-10, Fashion-MNIST, and DeeperForward demonstrate substantial accuracy degradation under changing states and significant recovery using pilot-based canonicalization.

Critical strengths

1.The paper identifies an interesting and under-explored problem in forward-only learning. 2. The paper is overall well written. 3.The matched fixed and resampled channel comparison controls the state distribution and upstream representations, making the analysis more convincing.

Critical weaknesses

1.The proposed recovery guarantees are limited to specific structured transformations. It remains unclear whether the proposed approach can handle more general dense linear or nonlinear black-box interfaces. 2.The experimental evaluation mainly relies on synthetic interface transformations. 3.The practical scalability of pilot-based recovery is unclear. 4. The evaluation is limited to small-scale image classification benchmarks. The experiments mainly focus on MNIST, Fashion-MNIST, and CIFAR-10, and it remains unclear whether the observed interface-state effects generalize to larger-scale datasets, more complex tasks, or real-world modular learning scenarios. 5. The proposed recovery mechanism relies on prior knowledge of the interface transformation family and synchronized pilot-data states, which may not hold in realistic black-box systems.

Questions for authors
  1. How does the proposed pilot recovery mechanism extend to more general unknown transformations, such as dense linear mappings or nonlinear black-box interfaces?
  2. Have the authors considered evaluating naturally occurring interface changes, such as representation drift in distributed learning, split computing, or heterogeneous model collaboration scenarios?
  3. How does the pilot overhead scale with representation dimension and transformation complexity? Can the authors provide theoretical or empirical analysis of this trade-off?
3 comments
AuthorsOpenReview ↗

Response 1: W1, Q1, W2, Q2, W3 — Scope, Evidence and Overhead

Thank you for your comments on scalability, scope, and presentation. They prompted us to expand the overhead and precision analysis, clarify the interface requirements, and reorganize the paper to make the main comparisons easier to follow.

Revision overview

We separate structured guarantees from deployment evidence, quantify payload and precision costs, and move receiver controls to Figure 4.


W1 / WA1: Scope of the recovery guarantees

The revised claim is structured recovery under an explicit access contract, not inversion of an arbitrary black box. The constructions cover diagonal states and signed permutations when pilots and data share the state. Known family membership, nonzero diagonal entries where division is used, and noiseless observations are part of the exact guarantees. A1 separates the analytical dense extension from the nonlinear limitation.

The required knowledge is the transformation family, while the realized state remains hidden. For an invertible diagonal map, a known pilot with nonzero entries identifies each coordinate scale and division recovers the data coordinate. Signed permutations additionally require identifying coordinate correspondence, which is why the construction uses coded pilots. These guarantees concern the feature interface: the receiver must be able to insert the references and keep them synchronized with the data. Per-sample changes are compatible with B=1 when the whole pilot–sample group shares a state. An API that offers only raw-input calls, or changes state independently inside the group, does not meet this access contract.

Changes made: Sections 3–4; Appendix B.3–B.4; Discussion.


Q1 / A1: Dense and nonlinear extensions

A square dense linear map requires d spanning pilots to identify its matrix under one shared state. Recovery then requires an invertible matrix and is sensitive to conditioning. These requirements are materially different from the low-pilot structured constructions.

Changes made: Section 4; Appendix G.2.


W2 / WA2: What synthetic controls establish

The matched Rademacher comparison holds upstream representations fixed and uses a common generating family for persistent and resampled states. The original CwC fixed-state result conditions on one seed-17 realization; its training repeats do not average over multiple fixed states. The comparison evaluates persistence conditional on that retained state and the specified downstream learning procedure. The earlier stochastic sign/flip study is a separate channel-suite experiment. Receiver controls then ask whether the transformed observations remain useful under a different fitting policy. Figure 4 makes that distinction explicit. These are controlled scientific comparisons, not measurements of naturally occurring system drift.

Changes made: Section 3; Sections 5.3–5.4 and 5.8–5.9; Figure 4; Appendices A and C (state sampling and separate channel protocols); Discussion.


Q2 / A2: Naturally occurring representation changes

Natural representation drift has not been evaluated. We now state this directly and limit deployment claims to the implemented synthetic transaction contract. The new heads and LA1 calibrators use controlled CIFAR-10 features; they do not substitute for distributed-learning, split-computing or heterogeneous-model drift measurements. A natural application would need evidence that its transformations and reference access satisfy the stated conditions.

Changes made: Section 3; Sections 5.10–5.11; Discussion.


W3 / WA3: Payload and measured scalability

Let d be the representation dimension, m the pilot count, and B the number of data vectors sharing one state. Additional logical payload is md scalars per direction, or m/B relative to data. The table gives payload scaling and measured local latency at d=102,400.

Interface familyPilots mExtra payload B=1B=32B=256Measured B=1 latency (ms), baseline → recovery
Diagonal1100%3.125%0.391%0.12482 → 0.29825
Signed permutation181,800%56.250%7.031%0.32714 → 1.12532

Payload entries are analytical extra payload per direction, expressed as 100m/B percent. Latency is measured only at B=1. Each value is the median of ten synchronized 100-call block means, including local dispatch, state and recovery operations. The additional payload decreases as 1/B when data vectors share one state.

These implementation-specific measurements use shared hardware; network and tail latency remain unmeasured. A3 gives the construction costs and precision trade-off.

Changes made: Sections 5.6 and 5.11; Appendices G.2 and H.


AuthorsOpenReview ↗

Response 2: Q3, W4, W5, Presentation — Precision, Assumptions and Clarity

Q3 / A3: Dimension, complexity and finite precision

The costs below describe the specified constructions, not universal lower bounds for all adaptive methods.

FamilyPilots mIdentification/storageRecovery
Diagonal1O(d)O(Bd)
Signed permutation1+ceil(log₂d)O(d log d) identificationO(Bd)
Invertible dense lineardO(d²) storage; conventional O(d³) factorizationO(Bd²)

For unit signs/permutations, fixed nonadaptive pilots with peak bound A, shared state, noiseless data and closed coordinatewise pilot-error bounds, a single real-valued pilot requires ε<A/(2d−1). Bipolar {−A,+A} codes permit ε<A; the original {0,A} code requires ε<A/2. At d=102,400 the single-pilot threshold is A/204,799, whereas 18 bipolar pilots achieve the stated ε<A bound. Equal peak amplitude is not equal total energy.

The precision bound explains what the additional pilots buy. With d coordinates and unknown signs, a single pilot assigns 2d scalar signatures inside [−A,A]. Some adjacent pair is at most 2A/(2d−1) apart, so closed error intervals of radius ε require that separation to exceed 2ε. Bipolar codes instead separate every pair of signed signatures by 2A in at least one observed coordinate. They trade additional observations for a larger admissible pilot-error bound. The dense-linear row is costlier because spanning probes identify a full matrix rather than coded coordinate identities. These distinctions make pilot count, precision and payload explicit parts of the scalability comparison.

Changes made: Section 4.1; Appendix G.2 (construction costs and precision derivation).


W4 / WA4: Dataset and task scope

Evaluation remains limited to MNIST, Fashion-MNIST and CIFAR-10 classification. Three independent DeeperForward backbones strengthen repeatability on CIFAR-10; same-parent receivers and LA1 address alternative explanations within that setting. They do not establish broad dataset, task or natural modular-system generalization. The abstract and conclusion now state receiver- and interface-conditional findings.

Changes made: Abstract; Section 5.1; Sections 5.8 and 5.10; Discussion and Conclusion.


W5 / WA5: Known family and synchronized state

The transformation family must be known to choose the recovery construction; its realized state can remain hidden. Separately, references and data must share that state within one atomic group. Per-sample grouping can satisfy this requirement with B=1, but independent changes inside a group cannot. Neither assumption follows merely from the interface being black-box. We now present both as explicit capabilities and retain the mismatch failure rather than treating synchronization as automatic.

Changes made: Section 3; Section 5.11; Appendix D.2.


Presentation: Definitions and reading order

We define fixed/resampled states by their refresh rule and distinguish per-state invertibility from usability under state mixtures. Sections 3–5 separate pilots, receivers, recovery and deployment. Results follow persistence, invariants, receiver dependence, calibration and cost. Figure 4 presents the receiver comparison; Appendix G contains full protocols and historical studies. The zero-offset normalization result now requires T2(h)=c(h)T1(h), with c(h)>0 for every input, and explicitly uses N0(0)=0.

Changes made: Sections 3–4; Appendix B.1–B.2 (normalization assumptions and optimal-error interpretation); main Figure 4 and Appendix H.1 (reading order).


We hope the revised analysis and presentation make the scope and requirements of the approach clearer. Please let us know if there are other points we should clarify or improve; we would appreciate your further guidance and welcome continued discussion.

Considering the overall standards of this conference, I will maintain my score as is.

Reviewer_qpYQOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies forward-only local learning across black-box interfaces with hidden transformations that may remain fixed or change between calls. It distinguishes information preservation under an individual transformation from the usability of representations when the transformation state is unobserved and resampled. Matched fixed-versus-resampled experiments hold upstream representations constant and demonstrate substantial downstream accuracy losses under changing Rademacher states. It introduces shared feature-space pilots that experience the same hidden state as the data. It provides exact recovery constructions for invertible diagonal transformations and signed permutations under noiseless observations. Additional analysis addresses positive-radial normalization invariance and conditional concentration for repeated-query score averaging. Experiments cover vanilla Forward-Forward, channel-wise competitive learning, supervised block-local learning, and frozen representations from an external DeeperForward backbone. The evaluation includes pilot noise, pilot–data state mismatch, repeated-query inference, and resource costs.

Critical strengths

S1. Controlled evidence for the effect of interface-state persistence. The fixed and resampled conditions use matched upstream representations and the same stated transformation distribution, allowing a meaningful comparison of persistent and changing interface states. The effect appears across multiple learning rules and representations. On the external DeeperForward features, fixed Rademacher states achieve 59.55% accuracy, compared with 13.73% under resampling. This supports the central observation that per-state invertibility and norm preservation alone do not guarantee effective downstream learning. S2. Explicit recovery conditions with robustness and cost measurements. The paper gives transparent algebraic recovery procedures for diagonal states and signed permutations, clearly identifying the shared-state and noiseless-observation requirements. It also evaluates noisy pilots, pilot–data state mismatch, and communication and computational overhead. These measurements make the recovery mechanism and its limitations more concrete than an idealized accuracy demonstration alone.

Critical weaknesses
  1. The experiments do not sufficiently establish the necessity or advantage of pilot recovery over simple invariant representations. For a diagonal Rademacher state D, the observation satisfies |Dh| = |h|. If the interface representation is nonnegative, taking the absolute value recovers h exactly without a pilot. For signed representations, absolute-value or squared-feature inputs may still preserve substantial task information. The manuscript does not report these important baselines.
  2. This omission materially affects my assessment because the large accuracy drops may partly reflect the sensitivity of the chosen downstream models to coordinate signs. The current results establish that pilots work under the assumed protocol, but do not adequately establish their advantage over inexpensive pilot-free alternatives.
  3. The practical scope and incremental contribution of the recovery protocol remain insufficiently established. The protocol requires known feature-space pilots to be inserted into the interface and transformed under the same hidden state as the data. A black-box service accepting only raw inputs does not necessarily expose this capability. The paper would benefit from a concrete application or interface implementation showing that these assumptions arise naturally.
  4. In addition, exact recovery of the original representation makes identity-level downstream accuracy an expected outcome under coupled training. The ideal-pilot results therefore provide useful implementation validation, but offer limited additional evidence of effectiveness beyond the algebraic construction. A realistic interface experiment or a more challenging recovery setting would strengthen the contribution.
Questions for authors
  1. Clarification and additional evidence: Which evaluated interface representations are nonnegative, and how do absolute-value and squared-feature baselines perform under resampled Rademacher states? Strong results for these baselines would materially change the interpretation of the pilot-recovery gains.
  2. Clarification of practical applicability: Can the authors identify a concrete black-box interface that permits known feature-space pilots and guarantees a shared transformation state across pilots and data? Is this capability part of the intended deployment setting, or does it require additional interface cooperation?
  3. Clarification of experimental independence: Appendix C.2 indicates that the external DeeperForward conditions share one frozen checkpoint. Do the five seeds vary only the downstream readout and interface randomness, or also the upstream backbone? Evidence across independently trained backbones would strengthen the generality of the external validation.
2 comments
AuthorsOpenReview ↗

Response1: W1, Q1, W2 — Invariant Baselines and Receiver Controls

Thank you for pointing out the missing invariant baselines and the need to distinguish receiver sensitivity from information loss. Your comments led us to compare receivers on the same backbones, add experiments with independently trained backbones, and revise our interpretation of the recovery gains.

Revision overview

We added same-parent direct/suffix comparisons, full-vector CwC baselines and fixed-head learned calibration. Figure 4 now makes the central receiver comparison visible in the main text.


W1 / WA1: Simple invariants and recovery gains

At the nonnegative FF boundary, abs already recovers the representation: identity/pilot/abs accuracy is 82.028%, and square gives 82.000% over five runs. We now place this control alongside the same three independently trained DeeperForward parents:

ReceiverTrainable parametersClean / ideal (%)Abs (%)Square (%)Clean−abs (pp)
Frozen nonlinear suffix + affine head256,01077.20259.61370.17517.588
Direct affine head1,024,01072.67071.25368.3671.417

Both rows use the same three parent representations. Accuracy is reported in percent; gaps are in percentage points and computed before rounding. Suffix results average two readouts within each parent; direct results use one fixed initialization per parent. Strong magnitude prediction coexists with a larger recovery benefit through the frozen suffix. The comparison shows why recovery gains depend on the receiver.

These controls distinguish two questions: whether an observation retains useful task information, and whether it supplies the inputs expected by a specified prediction path. Under diagonal signs, |Dh|=|h| makes magnitudes available without pilots; at a nonnegative boundary it also recovers h exactly. At a signed boundary, a magnitude receiver can use retained information even when a particular continuation performs poorly on those inputs. A shared pilot instead restores the signed coordinates supplied to that continuation. Accordingly, the clean/ideal column represents recovered access, while abs is a pilot-free observation. A1 and the protocol table in Appendix C specify which components are refitted.

Changes made: Sections 3.1 and 5.8–5.9; Figure 4; protocol table in Appendix C; Table 8 and Appendices G.3–G.5.


Q1 / A1: Boundary signs, preprocessing and trained components

The FF boundary has 500 nonnegative coordinates. CwC uses a signed post-normalization boundary with 20,480 coordinates. DeeperForward transforms a signed 102,400-coordinate layer-8 boundary; its frozen suffix returns 25,600 coordinates to the trained final head. Signs were checked at the actual interfaces rather than inferred from a nearby ReLU.

Abs/square preprocessing is consistent during receiver fitting and testing. DeeperForward suffix studies freeze the backbone, suffix and buffers in evaluation mode; direct studies fit the boundary head without the suffix. Historical native CwC keeps train-mode BN, so fixed convolutional weights do not imply fixed normalization buffers. Its poor 9.65% abs run remains in G.5. The later CwC full-vector fits use representation-specific TRAIN scalers and auxiliary heads. Their validation images had been used upstream, and the test set was adaptively reused; these are not fresh-test confirmation results. The table in WA1 and Figure 4 separate these receiver policies.

Changes made: Protocol table in Appendix C; Sections 5.1 and 5.8–5.9; Appendices G.1 and G.3–G.5.


W2 / WA2: Receiver sensitivity versus information loss

The new controls support your explanation: receiver choice changes the observed recovery benefit. WA1 gives the matched-parent DeeperForward comparison. For CwC, direct MLP clean/abs/sorted accuracy is 85.652/82.338/35.118%; linear accuracy is 84.710/79.860/24.454%. The five-parent MLP clean−abs difference is 3.314 [2.308, 4.320] percentage points (97.5% paired t CI); abs−sorted is 47.220 points. Appendix G.1 gives both paired intervals and their two-comparison adjustment.

Abs removes signs while preserving coordinates; sorting removes coordinate correspondence as well. The large clean/sorted gap therefore cannot be assigned to sign loss alone. DeeperForward's direct head has 1,024,010 parameters versus the suffix readout's 256,010, and bypasses a nonlinear continuation. Direct clean and square also underperform their corresponding suffix results. We retain the conditional suffix advantage while explicitly narrowing the claim about what magnitude observations can support.

AuthorsOpenReview ↗

Response 2: W2 (continued), W3, W4, Q2, Q3 — Recovery, Access and Replication

W2 / WA2: Receiver sensitivity versus information loss (continued)

Abs and square contain the same coordinate magnitudes in exact arithmetic: squaring and the nonnegative square root convert between them. Affine heads on |h| and h² nevertheless define different function classes and operate on different numerical scales. Their accuracy difference therefore reflects how a fitted receiver uses the observation, rather than a difference in recoverable magnitude information. For CwC, each branch uses its own TRAIN-only scaler and the same validation-based selection rule, with no sorted vector reshaped into an image. The clean/abs/sorted controls evaluate sign removal and the additional removal of coordinate correspondence under these explicitly defined fitting rules.

Changes made: Sections 5.8–5.9; Figure 4; Tables 7–8; Appendices G.1 and G.3–G.4; Discussion.


W3 / WA3: Additional interface cooperation

Feature-space pilot insertion and atomic pilot–data grouping are additional interface capabilities. A raw-input-only API does not automatically provide either. Section 3 now makes them part of the access model, rather than assuming them from black-box access alone. Our controlled service implements these capabilities; A2 states its validation scale and the remaining deployment gap.

Changes made: Section 3; Section 5.11.


W4 / WA4: What exact recovery demonstrates

Identity-level prediction is expected when recovery exactly restores inputs and downstream training is coupled. We now label this an implementation-consistency reference, not independent evidence of predictive superiority. In the independent-backbone suffix experiment, the ideal arm uses the identity-feature cache after a TRAIN-only recovery check; it is not full-test online pilot transmission.

The substantive comparisons are now invariant receivers, matched-parent receiver changes and LA1 calibration with a fixed clean head. LA1 learned shared-pilot accuracy is near-clean, but only analytic recovery has exactly identical clean logits. These studies distinguish useful prediction, approximate learned calibration and exact representation recovery. The service and cost checks address a separate feasibility question.

Changes made: Section 4; Sections 5.8 and 5.10; Appendices G.3 and G.6.


Q2 / A2: Implemented service versus natural deployment

The implemented example is a cooperating feature-space service in a separate process. It applies a chosen hidden state atomically to references and data, including per-sample groups. All 12 supported checks reconstruct exactly using 16 TRAIN vectors per learner. This validates the programmed transaction contract, not a naturally occurring interface or full-dataset service accuracy.

Changes made: Section 3; Section 5.11; Discussion.


Q3 / A3: Independent backbone replication

The original five seeds shared one upstream checkpoint and varied downstream/interface randomness. The new study instead uses three independently initialized DeeperForward backbones, each trained for 150 epochs, with two readout seeds per parent and all five conditions.

Within each completed backbone, the five conditions share paired readout initializations and training orders. The backbone, nonlinear suffix and normalization buffers stay frozen in evaluation mode; only the final affine head is trained. We average the two readouts within each parent and then form one within-parent condition difference before computing across-parent statistics. Thus extra readout seeds measure downstream variability without becoming extra upstream replications. The direct-head follow-up reuses these same parent representations, which makes the receiver comparison independent of a change in backbone identity. Tables 8–10 show the aggregate, per-parent and direct-receiver results.

The resulting ideal/abs/square/resampled-sign accuracy is 77.202/59.613/70.175/13.407%. Pilot−abs is 17.588 [14.201, 20.975] percentage points and pilot−square is 7.027 [5.035, 9.018]. The 97.5% paired t intervals use independent, approximately normal parent differences (n=3, df=2), adjusted for these two prespecified contrasts. The same parents also supply the direct-head follow-up in WA1.

The final weights and feature-cache bindings are distinct. The three parents share one architecture and split; upstream training logged test performance each epoch, while final epochs were fixed. G.3–G.4 and the evidence package retain these training and cache records.

Changes made: Sections 5.1 and 5.8; Figure 4; Tables 8–10; Appendices G.3–G.4.


Thank you again for helping us improve these comparisons and their interpretation. We would welcome any further comments on the evidence or conclusions, and would be glad to clarify or strengthen any part that still needs attention.

Reviewer_8F6ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper study forward-only learning with changing black-box interface. It show even invertible transformation can hurt learning when state changing. Authors propose shared pilot to recover original feature. Experiments on MNIST, Fashion-MNIST and CIFAR-10. DeeperForward accuracy improve from 13.73% to 76.90% with pilot.

Critical strengths

Paper show clear difference between fixed and changing interface. Experiment improvement is large and consistent.

Pilot method simple and effective. Can recover feature without backward gradient. Theory also support this.

Critical weaknesses
  1. Strong assumption and limited real-world validation. The proposed pilot method highly depend on shared hidden state between pilot and data. But in real black-box system, such assumption maybe difficult to satisfy. For example, different API calls or hardware communication may have different hidden state. The paper mainly test synthetic sign flip and permutation. These are quite simple transformations. For more complex nonlinear or unknown channel, method may not work. Also Appendix show when state mismatch reach 50%, accuracy drop to around 9.52%. This make practical robustness still questionable. More realistic validation is needed.

  2. Limited methodological novelty and incomplete comparison. The pilot recovery idea is quite straightforward. For diagonal sign transformation, all-one pilot can directly identify signs. Then multiply back to recover feature. So accuracy matching identity is already expected from the mathematical construction. Similar idea already exist in channel estimation. The paper not fully explain what is new beyond

Questions for authors
  1. How authors guarantee pilot and data share exactly same hidden state in real black-box system? If interface state change for every sample, rather than every batch, can method still work? This assumption looks very important for whole method. Can authors provides more realistic experiments to support?

  2. The sign recovery with all-one pilot seems very direct. What is the main technical novelty compared with traditional pilot-based channel estimation? Can authors compare with learned feature alignment, sign-invariant representation, or other simple recovery baselines? This may help understand whether proposed method really provide new advantage.

  3. Current method mainly work for diagonal and signed permutation transformation. What happen if interface contains unknown dense mixing, nonlinear transformation, or combination of several transformations? Does authors have evidence the recovery still effective? Otherwise application scope seems quite narrow.

2 comments
AuthorsOpenReview ↗

###Response 1: W1, Q1, W2 — Interface Access and Contribution

Thank you for raising these questions about the contribution and interface assumptions. Your comments led us to clarify the access conditions and add learned calibration and invariant baselines, which help explain when shared pilots are useful.

Revision overview

We now organize the paper around matched state-persistence evidence, conditional structured recovery and receiver-dependent utility. Figure 4 brings the receiver controls into the main text, and LA1 adds learned calibration with a fixed clean head.


W1 / WA1: Shared-state access and robustness

Shared state is an explicit interface requirement, not a property guaranteed by an arbitrary black-box API. We implemented a separate-process feature-space service that applies one hidden state atomically to a pilot–data group. All 12 supported learner/family/grouping checks reconstruct exactly on 16 TRAIN vectors per learner. These checks establish an executable synthetic protocol at this scale.

The failure under state mismatch remains important: 50% test-time mismatch reduces accuracy to about 9.52%. Lower mismatch with retraining is a different experiment. We now separate protocol feasibility, prediction and deployment.

Changes made: Section 3; Section 5.11; Appendix D.2.


Q1 / A1: Per-sample changes and atomic grouping

Per-sample changes can be handled only if each sample and its pilots share one atomic state. Then B=1, so no batch amortization is available. Independent transformations within that group violate the guarantee. The service in WA1 checks sample-atomic grouping, but an existing raw-input-only service must expose the required feature insertion and grouping capabilities before this construction applies.

Changes made: Section 3; Section 5.11; Appendix G.2.


W2 / WA2: Contribution beyond the pilot estimator

The all-one estimator is an established pilot construction. The contribution is the joint analysis of three questions: how persistent versus resampled states affect matched local-learning pipelines; which explicit access and precision conditions permit structured recovery; and when recovery helps a specified receiver rather than being necessary for prediction.

The same-parent controls make the last distinction concrete. Frozen-suffix clean/abs accuracy is 77.202/59.613%, while direct affine clean/abs is 72.670/71.253%. Thus strong magnitude-based prediction coexists with a substantial conditional suffix benefit. Figure 4 puts this distinction in the main paper.

Changes made: Introduction; Section 4; Sections 5.8–5.9; Figure 4; Appendices G.1 and G.3–G.4.


AuthorsOpenReview ↗

Response 2: Q2, Q3 — Learned Calibration and Recovery Scope

Q2 / A2: Learned calibration and invariant receivers

LA1 compares learned calibration with the same frozen clean classifier. All six conditions share each parent's frozen backbone and clean direct head. Evaluation uses an independent state stream.

LA1 calibration with a frozen clean classifier

InputAccuracy (%)Normalized squared reconstruction error
Clean72.670 ± 0.2430
Resampled signs9.810 ± 0.1452.000250
Abs63.327 ± 0.8620.861423
Analytic shared pilot72.670 ± 0.2430
Learned, no pilot68.547 ± 0.1150.238217
Learned, shared pilot72.667 ± 0.2316.959061 × 10⁻⁶

Accuracy is mean ± sample SD across three independent parent–calibration runs. Reconstruction entries are means of the normalized squared errors reported in Appendix G.6. Both learned arms use 337 parameters and 3,000 optimization steps on paired clean/corrupted TRAIN targets. Shared pilots improve mean accuracy by 4.120 percentage points over the matched no-pilot calibrator.

The two learned arms use the same coordinate-shared 2→16→16→1 ReLU MLP and identical initialization. Each parent supplies 4,096 TRAIN images with 256 sampled coordinates per image. The second input is the observed sign pilot in one arm and zero in the other; class labels do not enter calibration. This pairs the architecture, clean-target access and fitting budget while varying access to the state reference. Because the classifier stays fixed, the resulting gain tests whether alignment preserves an existing prediction path. It complements the refitted invariant-receiver controls by answering a different operational question, rather than attributing the benefit to a new all-ones estimator.

Refitting the direct receiver to abs inputs gives 71.253% in the separate direct-head study. LA1 instead keeps the clean classifier fixed. Learned shared-pilot is near-clean in prediction, whereas only analytic recovery reproduces clean logits exactly. One calibrator seed per parent and access to clean targets limit the result. We independently verified all 18 classification counts from the saved logits and labels.

For CwC, direct MLP clean/abs/sorted accuracy is 85.652/82.338/35.118%. Abs removes signs while retaining coordinates; sorting also removes coordinate correspondence. Thus learned calibration and invariant receivers offer useful alternatives under their respective access and receiver policies.

Changes made: Sections 5.8–5.10; Figure 4; Table 13; Appendices G.1, G.4 and G.6.


Q3 / A3: Dense, nonlinear and composed transforms

Exact recovery is established for the stated structured families under a shared state. A square dense linear map can be identified by d independent pilots under that same state; inversion requires nonsingularity and is sensitive to conditioning. This is an analytical extension with different costs. Compositions are covered only when they remain in a supported family and preserve pilot–data synchronization.

Changes made: Section 4; Appendix G.2.


We hope these additions make the contribution and the role of shared pilots clearer. We would appreciate any further suggestions on the comparisons or analysis, and would be happy to address any remaining concerns and improve the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.