- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies forward-only learning across changing black-box interfaces, where downstream modules learn from upstream representations without gradient propagation through the interface. The paper investigates whether preserving information at each interface state is sufficient for downstream learning when the hidden interface transformation changes across calls. The authors provide exact recovery conditions for diagonal transformations and signed permutations, and evaluate the approach on multiple forward-only learners and representations. Experiments on CIFAR-10, Fashion-MNIST, and DeeperForward demonstrate substantial accuracy degradation under changing states and significant recovery using pilot-based canonicalization.
Critical strengths
1.The paper identifies an interesting and under-explored problem in forward-only learning. 2. The paper is overall well written. 3.The matched fixed and resampled channel comparison controls the state distribution and upstream representations, making the analysis more convincing.
Critical weaknesses
1.The proposed recovery guarantees are limited to specific structured transformations. It remains unclear whether the proposed approach can handle more general dense linear or nonlinear black-box interfaces. 2.The experimental evaluation mainly relies on synthetic interface transformations. 3.The practical scalability of pilot-based recovery is unclear. 4. The evaluation is limited to small-scale image classification benchmarks. The experiments mainly focus on MNIST, Fashion-MNIST, and CIFAR-10, and it remains unclear whether the observed interface-state effects generalize to larger-scale datasets, more complex tasks, or real-world modular learning scenarios. 5. The proposed recovery mechanism relies on prior knowledge of the interface transformation family and synchronized pilot-data states, which may not hold in realistic black-box systems.
Questions for authors
- How does the proposed pilot recovery mechanism extend to more general unknown transformations, such as dense linear mappings or nonlinear black-box interfaces?
- Have the authors considered evaluating naturally occurring interface changes, such as representation drift in distributed learning, split computing, or heterogeneous model collaboration scenarios?
- How does the pilot overhead scale with representation dimension and transformation complexity? Can the authors provide theoretical or empirical analysis of this trade-off?
3 comments
Response 2: Q3, W4, W5, Presentation — Precision, Assumptions and Clarity
Q3 / A3: Dimension, complexity and finite precision
The costs below describe the specified constructions, not universal lower bounds for all adaptive methods.
| Family | Pilots m | Identification/storage | Recovery |
|---|---|---|---|
| Diagonal | 1 | O(d) | O(Bd) |
| Signed permutation | 1+ceil(log₂d) | O(d log d) identification | O(Bd) |
| Invertible dense linear | d | O(d²) storage; conventional O(d³) factorization | O(Bd²) |
For unit signs/permutations, fixed nonadaptive pilots with peak bound A, shared state, noiseless data and closed coordinatewise pilot-error bounds, a single real-valued pilot requires ε<A/(2d−1). Bipolar {−A,+A} codes permit ε<A; the original {0,A} code requires ε<A/2. At d=102,400 the single-pilot threshold is A/204,799, whereas 18 bipolar pilots achieve the stated ε<A bound. Equal peak amplitude is not equal total energy.
The precision bound explains what the additional pilots buy. With d coordinates and unknown signs, a single pilot assigns 2d scalar signatures inside [−A,A]. Some adjacent pair is at most 2A/(2d−1) apart, so closed error intervals of radius ε require that separation to exceed 2ε. Bipolar codes instead separate every pair of signed signatures by 2A in at least one observed coordinate. They trade additional observations for a larger admissible pilot-error bound. The dense-linear row is costlier because spanning probes identify a full matrix rather than coded coordinate identities. These distinctions make pilot count, precision and payload explicit parts of the scalability comparison.
Changes made: Section 4.1; Appendix G.2 (construction costs and precision derivation).
W4 / WA4: Dataset and task scope
Evaluation remains limited to MNIST, Fashion-MNIST and CIFAR-10 classification. Three independent DeeperForward backbones strengthen repeatability on CIFAR-10; same-parent receivers and LA1 address alternative explanations within that setting. They do not establish broad dataset, task or natural modular-system generalization. The abstract and conclusion now state receiver- and interface-conditional findings.
Changes made: Abstract; Section 5.1; Sections 5.8 and 5.10; Discussion and Conclusion.
W5 / WA5: Known family and synchronized state
The transformation family must be known to choose the recovery construction; its realized state can remain hidden. Separately, references and data must share that state within one atomic group. Per-sample grouping can satisfy this requirement with B=1, but independent changes inside a group cannot. Neither assumption follows merely from the interface being black-box. We now present both as explicit capabilities and retain the mismatch failure rather than treating synchronization as automatic.
Changes made: Section 3; Section 5.11; Appendix D.2.
Presentation: Definitions and reading order
We define fixed/resampled states by their refresh rule and distinguish per-state invertibility from usability under state mixtures. Sections 3–5 separate pilots, receivers, recovery and deployment. Results follow persistence, invariants, receiver dependence, calibration and cost. Figure 4 presents the receiver comparison; Appendix G contains full protocols and historical studies. The zero-offset normalization result now requires T2(h)=c(h)T1(h), with c(h)>0 for every input, and explicitly uses N0(0)=0.
Changes made: Sections 3–4; Appendix B.1–B.2 (normalization assumptions and optimal-error interpretation); main Figure 4 and Appendix H.1 (reading order).
We hope the revised analysis and presentation make the scope and requirements of the approach clearer. Please let us know if there are other points we should clarify or improve; we would appreciate your further guidance and welcome continued discussion.
Considering the overall standards of this conference, I will maintain my score as is.
Response 1: W1, Q1, W2, Q2, W3 — Scope, Evidence and Overhead
Thank you for your comments on scalability, scope, and presentation. They prompted us to expand the overhead and precision analysis, clarify the interface requirements, and reorganize the paper to make the main comparisons easier to follow.
Revision overview
We separate structured guarantees from deployment evidence, quantify payload and precision costs, and move receiver controls to Figure 4.
W1 / WA1: Scope of the recovery guarantees
The revised claim is structured recovery under an explicit access contract, not inversion of an arbitrary black box. The constructions cover diagonal states and signed permutations when pilots and data share the state. Known family membership, nonzero diagonal entries where division is used, and noiseless observations are part of the exact guarantees. A1 separates the analytical dense extension from the nonlinear limitation.
The required knowledge is the transformation family, while the realized state remains hidden. For an invertible diagonal map, a known pilot with nonzero entries identifies each coordinate scale and division recovers the data coordinate. Signed permutations additionally require identifying coordinate correspondence, which is why the construction uses coded pilots. These guarantees concern the feature interface: the receiver must be able to insert the references and keep them synchronized with the data. Per-sample changes are compatible with B=1 when the whole pilot–sample group shares a state. An API that offers only raw-input calls, or changes state independently inside the group, does not meet this access contract.
Changes made: Sections 3–4; Appendix B.3–B.4; Discussion.
Q1 / A1: Dense and nonlinear extensions
A square dense linear map requires d spanning pilots to identify its matrix under one shared state. Recovery then requires an invertible matrix and is sensitive to conditioning. These requirements are materially different from the low-pilot structured constructions.
Changes made: Section 4; Appendix G.2.
W2 / WA2: What synthetic controls establish
The matched Rademacher comparison holds upstream representations fixed and uses a common generating family for persistent and resampled states. The original CwC fixed-state result conditions on one seed-17 realization; its training repeats do not average over multiple fixed states. The comparison evaluates persistence conditional on that retained state and the specified downstream learning procedure. The earlier stochastic sign/flip study is a separate channel-suite experiment. Receiver controls then ask whether the transformed observations remain useful under a different fitting policy. Figure 4 makes that distinction explicit. These are controlled scientific comparisons, not measurements of naturally occurring system drift.
Changes made: Section 3; Sections 5.3–5.4 and 5.8–5.9; Figure 4; Appendices A and C (state sampling and separate channel protocols); Discussion.
Q2 / A2: Naturally occurring representation changes
Natural representation drift has not been evaluated. We now state this directly and limit deployment claims to the implemented synthetic transaction contract. The new heads and LA1 calibrators use controlled CIFAR-10 features; they do not substitute for distributed-learning, split-computing or heterogeneous-model drift measurements. A natural application would need evidence that its transformations and reference access satisfy the stated conditions.
Changes made: Section 3; Sections 5.10–5.11; Discussion.
W3 / WA3: Payload and measured scalability
Let d be the representation dimension, m the pilot count, and B the number of data vectors sharing one state. Additional logical payload is md scalars per direction, or m/B relative to data. The table gives payload scaling and measured local latency at d=102,400.
Payload entries are analytical extra payload per direction, expressed as 100m/B percent. Latency is measured only at B=1. Each value is the median of ten synchronized 100-call block means, including local dispatch, state and recovery operations. The additional payload decreases as 1/B when data vectors share one state.
These implementation-specific measurements use shared hardware; network and tail latency remain unmeasured. A3 gives the construction costs and precision trade-off.
Changes made: Sections 5.6 and 5.11; Appendices G.2 and H.