acceptodds
Under review as a conference paper at ICLR 2027

When Goodness Forgets Shape: Information Loss in Forward-Forward Learning

Abstract

Forward-Forward (FF) learning replaces end-to-end backpropagation with local goodness objectives, but local supervision is only useful if the goodness function preserves the information needed for prediction. We identify a failure mode in spatial representations: mean-squared goodness is invariant to permutations of hidden spatial positions, making some shape-relevant distinctions invisible to the local objective. Under spurious correlations, this invariance can suppress shape evidence while allowing simpler nuisance cues to dominate. We study spatial goodness through controlled adaptations that expose spatial structure to local supervision while preserving block-local training and avoiding cross-block gradients. Capacity-matched and coordinate-randomization controls isolate stable spatial indexing as the relevant source of improvement. On Colored MNIST, spatial goodness increases reversed-color accuracy from 10.83% to 85.02%. In a controlled adaptation of Adaptive Spatial Goodness Encoding (ASGE), spatial local predictions achieve 77.48% reversed-color accuracy, whereas global-average fusion of the same trained representation reduces performance to 10.12%, showing that spatial evidence can be learned locally yet discarded during prediction. Experiments on Waterbirds extend the analysis to natural images and improve worst-group performance. These results show that robust local learning depends not only on what representations encode, but also on what local objectives and prediction rules preserve.

Primary area:
interpretability and explainable AI
Keywords:
Spatial Information Preservation, Shortcut Learning, Spurious Correlations, Goodness Functions, Spatial Readouts, Concept Auditing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nhaHOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This study explores the shape information missed in the recent proposed forward-forward network. Considering only one layer, traditional convolutional networks or transformers are translation equivariant. However, the original FF network drops the location information when applying mean square as a local goodness score. This study aims to retain the location information (either shift or rotation) to the local goodness by a) introducing pre-defined weight to replace the mean square MLP layer. b) utilizing the BiCovG(Jiang et al., 2026) to replace the global average readout for retaining spatial energy.

Therefore, the proposed shape FF network outperforms previous FF candidates on color binary MINST classification.

Critical strengths

The FF network is a newly rising study topic, which attracts increasing attention in the field. This study attempts to solve an important feature to the backbone, “permutation”(one can imagine translation or rotation also applies) invariance exhibited in the FF network.

Critical weaknesses

There exist several weaknesses in the current draft, the readability requires further improvement. As the title suggests “WHEN GOODNESS FORGETS SHAPE”, the shape information loss is clearly introduced in Fig 1(a) and Fig 2(a). It is intuitive that any global operations could drop the position information. One can also imagine, if we shift or rotate the example, Proposition 1 also holds. However, the term “color shifts” is used throughout the paper and also the experiment validation. All of those examples show how spatial info (H,W) changes, but it is not clear how the “shape” loses across channels(C). The overall introduction and experiment do not match the example story. Assuming this draft intends to solve the “spatial shifts” problem in FFnetwork. Then this problem can be formulated as the mean square “goodness” is translation invariant. The solution seems to be not very solid. Given Fig 2(c), my understanding is that if we would like to capture the diagonal visual feature, then we can design and apply a diagonal feature detector, weight W. If my understanding is correct, then this is equivalent to adding human crafted features to capture each of the shape patterns. We can assume any global operations, e.g. GAP in CNNs or transformers will drop the shape information, then we can introduce manually designed Gabor filters to capture vertical or diagonal visual features. Then this story falls into the old maybe obsolete topic, how to combine learned weights with human crafted vision features. This type of study has limited impacts(personally). Assuming we need to design a series of human crafted visual features as patches to the learned FF network. Then this falls more into the old story, how can one design such patterns. I can only find ONE EXAMPLE feature(diagonal in Fig 2 c), but it is necessary to show how to design such features, e.g., Local Binary Patterns or SIFT features, and especially what did this study use, is it ad-hoc? My understanding on this draft now is Sections 3.1, 3.2 and Fig 2 are all resolving Local goodness, Fig 1(a). How does Fig 1(b) relate to this draft? I guess section 3.3 is trying to solve the Fig1 (b) problem. Given the context, “A fusion rule may combine these logits or instead classify globally averaged block features”, I suppose this fusion head refers to the global average pooling layer used in the final prediction layer, e.g., for classification. Then this “problem” first seems to be a stretch in this study, some networks, regardless of CNNs or transformers, global operations(e.g., GAP) are used for the final classification or visualization, in which case, the spatial information is omitted by design. Otherwise, one can simply use a fully connected layer. That is, the loss of the spatial information is not a problem when choosing GAP, otherwise one can use other alternatives. Assuming all of our choices may lose the spatial information, the solution (using second order information to replace mean square?) is directly borrowed from (Jiang et al., 2026). This can hardly be introduced as a contribution in this draft.

Questions for authors

Please see the questions listed in the above section.

6 comments
AuthorsOpenReview ↗

# Response 1 (W1–W8): Overview and revision guide

Thank you for your close reading and for identifying important issues in the figures, terminology, and explanation of the learning process. Your questions prompted us to clarify the learned nature of the spatial heads, distinguish the analytical separator from handcrafted features, explain the difference between hidden-position permutations and input color shifts, and redesign Figure 1 to better connect local goodness, final prediction, and the fusion-interface analysis.

Overall revision

The revision connects local scoring, retained information, and fitted decisions. Sections 1 and 3 now explain this sequence. The spatial head learns jointly with each block; the analytical separator illustrates expressivity. Color reversal tests cue reliance while keeping digit position and orientation fixed. Table 1 identifies inherited CwComp, ASGE, and BiCovG components.

Added theory and experiments

Figure 2 and Appendix L add an input construction for a specified two-block graph. Interior points, disjoint receptive fields, and matching stride phases ensure identical norm summaries for all shared weights. A candidate-conditioned spatial affine score can separate the inputs.

New coordinate, common-readout, and equal-coefficient controls separate indexing, readout choice, and capacity. With 48 coefficients per score, the detached common-readout difference is +0.384 pp. Refits use our ASGE adaptation’s exact 48D fusion input. With features and other fitting conditions fixed, risk reweighting changes reversed balanced accuracy from 25.98% to 78.22%. Matched balanced accuracy changes from 91.07% to 78.14%. Scratch Waterbirds and FC/GAP tests examine training conditions and regularization.

Redrawn figures

Figure 1 connects local scores, readouts, and frozen affine fitting. Figure 2 shows the paired inputs, graph conditions, and separator. Figure 3 separates original FF/BP results from the new coordinate, common-readout, and equal-coefficient controls. Figure 4 moves cross-color similarity and retrieval into the main text. Figure 5 distinguishes feature fusion, logit fusion, and their inputs. Appendix Figures A1, A2, and A5 redraw readout intervention, frozen-state comparisons, and the correlation sweep.

How the continuations answer your comments

For clarity, we group your comments into eight topics, W1–W8. The Questions field refers back to the same comments.

  • Continuation 1 (2/4): W1–W3. We connect the study sequence, define the scores, and distinguish hidden permutations from input transformations and color shifts. Sections 1, 3.1–3.2, and 5.1, Figures 1–2, and Tables 1–2 and 4 provide these clarifications.
  • Continuation 2 (3/4): W4–W5. We distinguish learned heads, analytical examples, and inherited components, then explain the construction and control results. Section 3.2, Figures 2–3, Tables 1–2 and 5, and Appendices L–N and U give the definitions, attribution, and tests.
  • Continuation 3 (4/4): W6–W8. We define fusion, test retained information and FC alternatives, and explain how fitting and training conditions affect the findings. Sections 3.3 and 5.3–5.5, Figure 5, Tables 6–7 and A24, and Appendices O–P and R–S give the results and protocols.

If you have any further concerns or suggestions, we would sincerely welcome them. We would be grateful for the opportunity to address them and further improve the quality of the paper.


AuthorsOpenReview ↗

# Response 2 (W1–W3): Score definitions, hidden permutations, and color shifts

W1. Study scope, score definitions, and the connection between theory and experiments

Averaging removes explicit position indices. We study how this affects FF supervision and final prediction.

Norm goodness is a parameter-free mean of squared activations. The invariant MLP is a separate capacity control that reads channel energies. The spatial affine head reads raw activations and learns position-specific coefficients. Their different inputs and invariances separate score invariance from learned-head capacity. The spatial head changes what the local loss can access while retaining block-local gradients. CwComp, ASGE, and BiCovG comparisons use their specified components and prediction rules. Each result belongs to its stated FF adaptation and training protocol.

The tests address different steps of prediction. The input construction asks which spatial distinctions a local score can express. For the specified graph, two layouts have identical norm summaries, while a spatial score separates them. Color reversal asks which cues trained predictors use. Coordinate controls test spatial indexing during training. Frozen-interface fits test retained information and the decisions selected by fitting.

Sections 3.1–3.2 and Figure 1(a) define local losses, norm and spatial scores, and the input construction. They explain what each local objective can access and how the current block learns. Section 3.3 follows those representations into final prediction.

Changes made: Section 1 and Figure 1 connect the study sequence to Sections 5.2–5.3. Section 3.2, Tables 1–2 and 4, and Appendix U specify score inputs, parameter budgets, components, and adaptation choices. Figure 2 and Appendices L–O add the construction and controlled tests.


W2. Hidden permutations versus input translation or rotation

Proposition 1 states hidden spatial-permutation invariance: it permutes positions in an already formed hidden tensor. Input translation or rotation depends on the feature extractor. These transformations may change convolution responses, boundaries, and pooling phases. They therefore need a separate input-to-hidden argument. Our new two-block construction uses interior points, disjoint receptive fields, and matching stride phases. Under these conditions, the two input layouts induce permuted hidden responses. We distinguish this mathematical relationship from the color-correlation changes measured after training.

Changes made: Sections 1, 3.1, and 5.1 use these terms consistently. Figure 2(b) and Appendix L state the graph and input conditions.


W3. Color shifts, hidden channels, and spatial information

Color shift changes the relation between color and the label. Image position, orientation, and grayscale structure stay fixed. The binary target is digit ≥ 5. At the primary training correlation ρ = 0.9, color agrees with it with probabilities 0.9, 0.5, or 0.1. This tests cue reliance; the input construction separately tests whether scores distinguish spatial layouts.

K indexes learned hidden channels, while H and W index hidden positions. K differs from input RGB. Norm goodness averages squared activations across all three axes. Before reduction, convolution can encode layout through channel amplitudes. Color enters through RGB and can change those activations. Removing position indices therefore does not establish loss of all layout-dependent information.

Changes made: Section 3.1 defines the axes beside Equation 2 and explains channel amplitudes. Section 5.1 defines the target, RGB generator, and evaluation probabilities.


AuthorsOpenReview ↗

# Response 3 (W4–W5): Learned spatial heads, analytical examples, and controlled comparisons

W4. What is learned, what is analytical, and what comes from prior work?

Experimental training learns spatial coefficients jointly with the current convolutional block. Each block uses one affine score, g(h) = wᵀvec(h) + b, on raw hidden activations. The same score evaluates competing candidate-conditioned activations. Correct and incorrect candidate label planes produce h⁺ and h⁻. Equation 1 uses softplus(−[g(h⁺) − g(h⁻)]), favoring the correct candidate. The weight gradient depends on the activation difference between candidates at each position. Both block weights and score coefficients learn through the local loss; earlier blocks stay frozen. The candidate-shared bias cancels in the score margin. Thus, the coefficients are optimized across training examples rather than assigned through a catalogue of shape templates.

The old diagonal detector was an analytical weight choice and was never loaded during training. Figure 2 now shows horizontal and vertical point pairs under a specified graph. Their norm summaries agree, while a candidate-conditioned spatial score separates them. One pair suffices to distinguish the score families’ expressivity. It supplies neither experimental descriptors nor fixed training weights. Gabor filters (Daugman, 1985), LBP (Ojala et al., 2002), and SIFT (Lowe, 2004) do not enter this computation. Appendix U compares them with the learned rule. ASGE’s fixed Gaussian projection belongs to a separate adaptation.

Mean-squared goodness already uses squared activations. The primary learned spatial head is affine in raw h. Common-readout controls use h² for both scoring families. BiCovG’s multiscale energy, cross-channel probes, alignment, and logit fusion are attributed to Jiang et al. (2026). These components retain their identities in our two-block adaptation, where component ablations examine their roles. The new exact-fusion and risk-weight tests instead study the ASGE interface. These distinctions identify inherited components and the separate analyses conducted here.

ComponentHow obtainedRole
Analytical spatial separatorFixed illustrationShows expressivity; unused in training
Native spatial headLearned from local class supervisionScores raw h with block-local gradients
ASGE projectionFixed Gaussian component from prior workEncodes pooled spatial energies
Exact 48D auxiliary affine headFitted on frozen fusion inputsTests retained information and fitting risk

Changes made: Sections 2–3 and Tables 1–2 clarify inputs, joint updates, and attribution. Section 3.2 explains Equation 3. Figure 2(b,c) and Appendix L add the paired inputs, conditions, separator, and proof. Appendix N explains bias cancellation and restricted coefficients. Appendix U compares learned weights with cited classical descriptors. Table A6 retains component ablations; Appendix D.4 cites BiCovG beside its implementation.


W5. What do coordinate, common-readout, and equal-coefficient controls show?

Stable fixed coordinates retain roughly 89.6–89.9% reversed-color accuracy; fresh independent permutations give 21.53%. Fixed permutations change coordinate labels while preserving a stable index during training. At the function level, a fixed permutation can be absorbed into the affine coefficients. The construction establishes expressivity; stable-versus-fresh tests also depend on training. Fresh permutations also change stochastic loss estimation, which Appendix M examines separately. Common-readout tests keep the final classifier family fixed while changing the local score. With both scores restricted to 48 coefficients, the detached common-readout difference is +0.384 percentage points. These controls distinguish the roles of indexing, readout choice, and head capacity.

A separate raw-h control retains greedy training and native prediction, with 48 effective coefficients. Reversed-color accuracy changes from 11.720% to 14.732%, a +3.012-point paired difference. Its protocol differs from the h²/common-readout study.

Changes made: Figure 3, Table 5, and Appendices M–N now present these coordinate, readout, and coefficient controls.


Reviewer_EdTnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the phenomenon of shortcut learning in forward-forward (FF) learning. First, the paper shows that the commonly used mean squared goodness function is invariant to spatial permutations of hidden activations. Based on this observation, it studies FF learning through established shortcut learning protocols for color-based shortcut. The paper finds that FF learning exhibits similar shortcut learning behavior to convolutional neural networks (CNNs) and that spatial priors (e.g., indexed spatial head, spatial read-out) help reduce shortcut learning. On natural-image datasets, the paper further shows that spatially sensitive goodness functions improve worst-group accuracy and that applying established robustness methods to FF predictions can further improve performance.

Critical strengths
  • Studying shortcut learning in FF is interesting and a relevant research direction since it provides an alternative learning paradigm to backpropagation.
  • The findings that shortcut learning behavior observed for CNNs generalizes to FF learning is interesting.
Critical weaknesses

Novelty and contribution

The main observation that spatially invariant objectives and readouts can discard spatial information is generally well established in deep learning. The paper demonstrates that this observation holds for FF learning. I have the following comments to the three main contributions:

  • Contribution 1 - Proving spatial invariance for FF learning: The invariance of mean-squared goodness to spatial permutations seems like a straightforward mathematical property. While connecting this observation to shortcut reliance in FF is interesting, I question whether the mathematical argument itself constitutes a significant theoretical contribution.
  • Contribution 2 – Spatial information loss during final prediction: Demonstrating this phenomenon in FF is useful, but the general distinction between learned representations and final predictions is already well established (e.g., Kirichenko et al. [1]).
  • Contribution 3 – Natural-image experiments: This experiment extends the previous observations to natural images and apply established robustness methods to FF learning. It is certainly relevant to show the observations for another dataset than MNIST, but I feel like it is very close to contribution 2.

Clarity, presentation, and terminology

The paper uses terminology without sufficiently explaining what it means or how they relate to the experiments. I had some difficulties mapping certain terms to what was actually done in the paper. Further, some experimental setups were not thoroughly described. I will give some examples:

  • L262, “matched BP”: What exactly is matched here? The architecture, parameter count, or training setup?
  • L214, “controlled adaptations”: What has actually been changed compared to the original CwComp, ASGE, and BiCovG methods?
  • L328–334 / Table 2, spatial controls: Is “randomized coordinates” the same as “randomized spatial”? The invariant MLP and norm-pooled baselines are also not sufficiently described in the main experimental setup.
  • L371, “calibrated audit”: The term appears here for the first time, I was not sure what it related to.
  • L226, “blue-channel reference”: What exactly is this reference, I think I guessed from context making all colors blue, but it was not very clear to me.
  • Section 5.5, LocalFusion-ERM / LocalFusion-JTT / LocalFusion-GroupDRO: These methods are introduced without adequately explaining their implementation. The underlying ERM, JTT, and GroupDRO methods should also be explicitly introduced and cited here, rather than only briefly mentioning the corresponding work in Related Work without linking them to their method names.
  • Important experimental details are often only available in the appendix, it was sometimes hard to understand the experimental comparisons.

References

[1] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. In International Conference on Learning Representations, 2023.

Questions for authors

Please see weaknesses clarity, presentation, and terminology.

3 comments
AuthorsOpenReview ↗

# Response 1 (W1–W8): Overview, revisions, and contribution analysis

Thank you for your detailed comments on the paper’s contributions, novelty, and relation to prior work. Your discussion of last-layer retraining and the overlap among our contributions helped us sharpen the paper’s central message, better distinguish our analyses from existing methods, and strengthen the natural-image study with additional scratch-training and frozen-readout controls; we also clarified the adapted methods and moved key experimental details into the main text.

Overall revision

We group your unnumbered comments into eight response topics, W1–W8, for navigation. The revision connects local score expressivity, retained fusion information, and decisions selected by fitting. Sections 1–3 distinguish these questions and compare them with DFR and spatial FF systems. Tables 1–2 now place architectures, gradient paths, parameter budgets, and fitting protocols before the results.

Added theory and experiments

The paired-input construction extends the hidden-permutation identity to a specified two-block graph. It requires interior points, disjoint receptive fields, and matching stride phase. Norm scores match for all shared weights; a spatial affine score can separate the pair. Coordinate, common-readout, and equal-coefficient controls test learned predictions; with 48 coefficients per score, the detached common-readout difference is +0.384 pp.

Frozen affine refits recover useful prediction from the original 48D ASGE fusion input. A separate intervention fixes features and fitting conditions while changing color-risk weights. Scratch Waterbirds and frozen FC/GAP controls test initialization, selection, weighting, and regularization. The pretrained spatial advantage does not persist when pretraining and group-informed selection are jointly removed.

Figure changes

Figure 1 connects scoring, readout summaries, and frozen fitting. Figure 2 shows the input pair, graph, and separator. Figure 3 groups the controls. Figure 4 moves cross-color similarity and retrieval into the main text. Figure 5 distinguishes local, feature-fusion, and logit-fusion predictions. Appendix Figures A1, A2, and A5 redraw the readout intervention, frozen comparisons, and correlation sweep.

Response group 1: W1–W3

W1 addresses novelty and the input construction together. W2 explains the relation to DFR; W3 covers natural-image comparisons and established fitting methods. The contribution table maps prior work to our analyses. Sections 1–3 and 5.2–5.5 distinguish these questions and findings.

Response group 2: W4–W6

W4 combines terminology, setup, and BP comparisons. W5 explains all three adaptations. W6 combines coordinate operations with the invariant MLP and norm baseline. Sections 3/5.1–5.2, Tables 1–2/4, and Appendices D/M/N/U define these operations and budgets.

Response group 3: W7–W8

W7 explains the audit with its blue reference. W8 describes the common LocalFusion input and each fitting objective. Sections 4/5.1/5.4/5.5 and Appendices F/H define operations and citations; Table A10 separates group-information access.

Questions field

Your Questions field refers to the clarity comments, covered by grouped topics W4–W8. Response groups 2–3 answer all related questions.

If you have any further concerns or suggestions, we would sincerely welcome them. We would be grateful for the opportunity to address them and further improve the quality of the paper.


W1. Novelty and input construction

The contribution connects score expressivity, retained information, and fitted decisions in the studied FF systems. The analyses ask which distinctions scores miss, which information reaches fusion, and which decisions fitting selects.

Prior workAnalysis hereLocation
Spatial FF: ASGE and BiCovGInput construction; coordinate and common-readout controlsSections 3, 5.2; Appendices L–N
DFR; ASGE feature fusionAffine prediction at the adaptation’s exact 48D fusion inputSection 5.3; Appendix O
ERM, JTT, GroupDROFixed color-risk intervention; scratch FC/GAP weighting and penalty controlsSections 5.3, 5.5; Appendices P, R–S

The norm identity alone concerns hidden tensors. Convolution can encode input layout in channel amplitudes, so that identity does not establish an input-level failure. Our paired-input construction closes this gap for the specified two-block graph. Interior points, disjoint receptive fields, and matching stride phase make both layouts indistinguishable to norm summaries. This holds for every shared weight setting. A candidate-conditioned spatial affine score separates the pair for a constructed setting. This is an existence result; trained heads learn their weights without loading that construction.

Fixed coordinate relabeling retains strong reversed-color prediction, testing the role of consistent indexing.


AuthorsOpenReview ↗

# Response 2 (W1–W5): Contributions, DFR, natural-image comparisons, and experimental setup

W1 (continued)

Common-readout tests fix the final prediction family, but retain unequal coefficient counts and extra readout supervision. With 48 coefficients per score, the detached common-readout difference is +0.384 pp.

Changes made: Sections 1–2 distinguish our questions from prior work. Section 3, Figure 2, and Appendix L add the input construction. Sections 5.2–5.5 report the corresponding controls.


W2. DFR and the exact fusion input

DFR already studies standardized linear readouts and ColorMNIST (Kirichenko et al., ICLR 2023). Our analysis targets a failed FF classifier’s actual 48D fusion input, after propagation pooling and per-example RMS normalization. Native ASGE-adaptation fusion sums two affine heads. Affine refitting therefore retains its prediction family and restores no coordinates.

Already fitted affine heads reach 72.142% reversed balanced accuracy on frozen QMNIST evaluation, versus 10.001% for native fusion. QMNIST supplies evaluation digits, not fitting data. Thus, usable signal already reaches the native fusion interface. Those refits change both fitting colors and the scaler.

A separate intervention asks whether training risk alone selects different decisions from the same information. Both auxiliary affine fits share paired-color inputs, features, scaling, initialization, regularization, and convergence criteria. Positive L2 gives a unique minimizer; fitting uses no validation or epoch selection. Only color-risk weights change from .9 to .5. Reversed balanced accuracy changes from 25.98% to 78.22%; matched balanced accuracy changes from 91.07% to 78.14%.

Changes made: Section 2 expands the DFR comparison. Section 5.3, Table 6, and Appendices O–P separate the two analyses.


W3. Natural images and established fitting methods

Natural-image experiments test initialization, selection, and readout fitting. Scratch Waterbirds jointly removes pretraining and group-informed selection. Spatial/pooled worst-group accuracy is 8.19%/10.25%; the pretrained spatial advantage does not persist. Auxiliary heads then compare weighting and regularization while keeping those parent representations frozen. With group-balanced fitting, spatial GAP reaches 33.71% worst-group accuracy, versus 16.60% for FC.

ERM, JTT, and GroupDRO are established methods applied to one eight-logit interface. Their pretrained Waterbirds worst-group accuracies are 24.42%, 31.37%, and 51.43%. Their group-access requirements differ. The contribution is the comparison of fitting objectives and its findings; attribution remains with the original methods.

Changes made: Section 5.5 and Table 7 separate initialization, native prediction, and auxiliary fitting. Appendix H and Table A10 define the established objectives and group access. Appendices R–S add weighting and penalty controls.


W4. Terminology, experimental setup, and BP comparisons (L262)

Spatial goodness learns an affine function of raw activations jointly with the current block; earlier blocks stay frozen. Common-readout controls score squared activations. “Fusion input” means the final classifier’s features; an “auxiliary readout” is a new head fitted on fixed features.

Capacity/adaptation studies use 30k/5k train/test examples and 15 epochs per block; the original protocol is specified below. Waterbirds separates pretrained and scratch initialization. Tables 1–2 specify the principal local-training and readout protocols. Section 5.5 and Table 7 separate natural-image initialization and native/auxiliary prediction. Full LocalFusion objectives and selection rules remain in Appendix H; scratch-readout and penalty controls are detailed in Appendices R–S.

FF and BP share two convolution–ReLU–max-pooling blocks with 16/32 channels. FF adds a candidate-label plane to RGB; BP uses RGB alone. BP’s spatial head pools to 4×4; FF reads each block’s native grid. Norm/spatial FF have 5,232/9,938 parameters, versus 5,154/6,114 for GAP/spatial BP.

Both use 10k/2k train/test examples, batch size 256, and AdamW with learning rate .002 and weight decay .01. FF trains each block for 20 epochs with local losses; BP trains end to end for 20 epochs with cross-entropy. Separate controls restrict invariant and spatial scores to 48 coefficients each. The primary paired differences are +0.384 pp for the detached common readout and +3.012 pp for greedy/native raw-h prediction.

Changes made: Sections 3/5.1 and Tables 1–2 define operations, shared BP settings, and differing budgets. Equation 4 identifies the 48D input; Table 7 separates initialization and native/auxiliary prediction. Section 5.2 and Appendix N report equal-coefficient controls.


W5. CwComp, ASGE, and BiCovG adaptations (L214)

CwComp. We retain class-channel squared-energy competition and local cross-entropy. Ordinary convolution–ReLU–pooling–BatchNorm blocks replace CFSE.


AuthorsOpenReview ↗

# Response 3 (W5–W8): Method adaptations, spatial controls, audit calibration, and LocalFusion

W5: CwComp (continued)

Two blocks train greedily; the last block’s class-group energies predict.

ASGE. Fixed Gaussian spatial-energy projection and bias remain. Propagation uses average pooling then RMS normalization; fusion includes both block GAPs. Official ASGE v2 uses RMS pooling and excludes the first layer from Fusion. Its Last predictor trains on final GAP features; our final-local output uses the fixed energy projection.

BiCovG. Two detached blocks retain multiscale energy, cross-channel probes, one alignment, and logit fusion, with grids (2,4)/(1,2). Learned softmax-normalized weights combine detached local logits. Original Hybrid Goodness Blocks jointly optimize layers within each block; our graph is smaller.

Changes made: Table 1 and Appendix U map components and prediction rules; Figure 1(c) identifies ASGE’s input. Appendices D.1/D.2/D.4 cite Papachristodoulou et al. (2024), Gong et al. (2026), and Jiang et al. (2026), respectively.


W6. Coordinate controls and invariant baselines (L328–334)

The terms randomized coordinates and randomized spatial named the same control with fresh spatial permutations. Each example’s permutation is shared across channels and changes score inputs only. Unpermuted features propagate onward. Fixed-unmapped indexing retains one permutation and original initial weights; co-remapping also reorders those weights. Fresh-independent training draws positive/negative permutations independently; fresh-shared training and inference share their draw.

Norm goodness averages squared activations across channels and positions, without a learned head. The MLP learns a nonlinear function of channelwise mean-squared activations. Its ReLU heads are 16→131→1 and 32→69→1, with 4,706 head parameters and 9,938 overall. This matches spatial FF’s parameter count.

Changes made: Section 3.2 and Appendix M define coordinate indexing, candidate coupling, and unchanged propagation. Equation 2 defines norm goodness; Section 3.2 specifies MLP inputs, widths, and counts. Table 4 reports the comparison.


W7. Calibrated audit and blue reference (L371/L226)

The reference is the blue channel already present in each image. For grayscale digit s, RGB is (s, 0, 0.05s) or (0, s, 0.05s). Blue retains the digit at scale 0.05 as red/green assignment changes. Its class separation supplies the audit’s reference term.

The audit compares standardized color-associated class separation with a shape reference. Its 15 concepts are whole-image and quadrant RGB means. It subtracts the strongest blue separation from the strongest red/green separation. Seeds 100–119 yield 40 null scores under grayscale and independent colors. At α=0.05, the 39th ordered score sets threshold zero; triggering requires a larger score. Calibration precedes target seeds 0–4. The separate diversification experiment uses the earlier rule-selected raw-effect trigger.

Changes made: Sections 4/5.1/5.4 and Appendix F define the RGB generator, reference, and calibration before the results. Appendix F distinguishes the calibration and diversification triggers; Table A7 reports calibration.


W8. LocalFusion and its fitting objectives (Section 5.5)

Four frozen stages each output two logits. Training statistics standardize these eight values before a linear 8→2 fusion head. Fusion gradients update neither stages nor local heads. Fitting uses AdamW for 100 epochs, learning rate .01, weight decay .0001, and batch size 256. ERM, JTT, and GroupDRO change the objective; JTT also requires the preliminary fit described below.

ERM. ERM (Vapnik, 1991) minimizes mean training cross-entropy. Original fusion uses worst-group validation selection. The average-selected variant avoids validation groups when selecting fusion. Fully groupless experiments also select local stages by average accuracy. The loss is unchanged; information access differs.

JTT. JTT (Liu et al., 2021) upweights an initial model’s training errors. Our initial ERM head trains for 20 epochs. A fresh head trains for 100 epochs, with weight 20 on that fixed error set. Final fusion selection uses average validation accuracy. Training-group labels are unused.

GroupDRO. GroupDRO (Sagawa et al., 2020) adaptively emphasizes high-loss groups. Our fusion uses four group losses and group-weight step size 0.01. It requires training group labels and worst-group validation selection. The separate scratch-readout study uses static group-balanced logistic weights, which define a different objective.

Changes made: Section 5.5 summarizes the four-pair linear input, cites JTT/GroupDRO, and separates experimental conditions. Appendix H defines the objectives, standardization, detachment, and fitting schedules. Table A10 specifies selection and group access; Appendices H/R distinguish adaptive GroupDRO from static group balancing.


Reviewer_3Y9JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how the choice of goodness function in Forward-Forward (FF) learning affects the preservation of spatial information and robustness to spurious correlations. The authors identify a limitation of conventional mean-squared activation goodness,

gnorm(h)=1KHW∑k,u,vhk,u,v2,g_{\mathrm{norm}}(h)=\frac{1}{KHW}\sum_{k,u,v}h_{k,u,v}^{2},

which is invariant to permutations of spatial positions, i.e.,

gnorm(Pπh)=gnorm(h).g_{\mathrm{norm}}(P_\pi h)=g_{\mathrm{norm}}(h).

The authors argue that this invariance can prevent local goodness objectives from distinguishing certain spatial arrangements, potentially encouraging reliance on simpler spurious cues such as colour. To investigate this, they introduce spatially indexed goodness heads that retain access to spatial configurations while preserving block-local training without cross-block gradients. Capacity-matched invariant heads and coordinate-randomisation controls are used to investigate whether improvements arise from stable spatial indexing rather than increased parameter capacity.

On Coloured MNIST, spatial goodness improves reversed-colour accuracy from 10.83% to 85.02%. Experiments on Waterbirds show improvements in worst-group accuracy, although the resulting performance remains below conventional back-propagation baselines. Overall, the paper argues that robustness in local learning depends not only on the information encoded by intermediate representations but also on which distinctions are accessible to local objectives and preserved by downstream prediction rules.

Critical strengths

1. Investigation of spatial information preservation.

The paper's identification and empirical investigation of spatial-permutation invariance in conventional mean-squared goodness is note-worthy. The authors formalise this property through Proposition 1 and demonstrate that spatially indexed goodness heads can distinguish feature configurations that produce identical norm-based goodness scores.

The experimental design includes meaningful controls, including parameter-matched invariant heads and randomised-coordinate evaluations, to examine whether the improvements arise from stable spatial indexing rather than additional model capacity. The substantial improvement in reversed-colour accuracy on Coloured MNIST, from 10.83% to 85.02%, provides evidence that the choice of goodness reduction can significantly influence reliance on spurious cues.

2. Informative analysis of information preservation.

The distinction between information accessible to locally trained classifiers and information utilised by the final prediction mechanism is useful. Rather than evaluating only the locally learned representations, the authors investigate how subsequent feature pooling and prediction fusion affect robustness.

The ASGE experiments demonstrate a substantial difference between spatial-local and globally pooled predictions obtained from the same trained blocks. This comparison provides evidence that the final prediction rule can substantially influence whether spatially learned features contribute to robust decisions. The extension to Waterbirds, including worst-group evaluation, local-stage adaptation, and comparisons with ERM, JTT, and GroupDRO, broadens the analysis beyond synthetic colour-shift benchmarks.

Critical weaknesses

1. The theoretical invariance result does not sufficiently establish the claimed causal mechanism.

While Proposition 1 establishes that mean-squared goodness is invariant to spatial permutations of hidden activations:

gnorm(Pπh)=gnorm(h).g_{\mathrm{norm}}(P_\pi h)=g_{\mathrm{norm}}(h).

However, this does not imply that the combined feature extractor and goodness function cannot distinguish different spatial structures. Convolutional feature extraction may transform different input shapes into representations with different activation energies, even when the goodness function itself is spatially permutation-invariant.

Also, Table 1 shows that replacing global-average readouts with spatially indexed readouts produces similarly large robustness improvements under both FF and conventional back-propagation. This suggests that the observed gains may primarily reflect a general architectural or readout effect rather than a failure mode uniquely associated with local goodness supervision.

The paper would benefit from a more precise account of what the theoretical invariance establishes about the complete learned classifier, together with experiments that isolate goodness-specific effects from general spatial-readout effects. The novelty should also be positioned more clearly relative to existing spatially aware FF methods such as ASGE and BiCovG.

2. Claimed information loss is not fully supported. Also, natural-image robustness results are limited.

The ASGE experiments report a large reversed-colour accuracy gap between spatial-local prediction and global-average feature fusion which the authors interpret as evidence that global-average fusion discards spatial information learned by local heads. However, Appendix D.3 shows that separately trained classifiers using globally pooled representations can recover substantial reversed-colour accuracy, including 71.11% for a GAP-linear probe and 86.65% for a GAP-MLP when trained with label-independent colours. These findings indicate that the pooled representation retains useful predictive information and that the failure of the original fusion classifier may also arise from its training objective and reliance on the spurious colour correlation. Therefore, the reported performance gap cannot be attributed entirely to irreversible spatial information loss.

Additionally, the robustness improvements on natural images are substantially smaller than those on Colored MNIST. On Waterbirds, spatial local learning reaches only 24.55% worst-group accuracy, compared with 53.77% for BP-ERM, and performance under multiple correlated shortcuts remains very poor. The natural-image experiments also rely on ImageNet-pre-trained features and, in several configurations, group-aware validation.

Questions for authors

Q1. Can the authors provide evidence that spatial permutation invariance of the goodness function causes the observed shortcut reliance, rather than merely correlating with it? How do they distinguish this from the effects of spatially indexed readout architecture, given the similar behaviour of the BP baselines?

Q2. Appendix D.3 shows that globally pooled ASGE features can recover substantial reversed-colour accuracy with alternative readout training. How much of the original fusion performance collapse is attributable to spatial information removal, and how much to shortcut selection by the trained fusion classifier?

Q3. The randomised-coordinate control introduces independently sampled permutations during training. Could the resulting performance degradation partly arise from optimisation noise? Have the authors considered deterministic or fixed-permutation controls?

Q4. How would you reconcile the strong Coloured MNIST improvements with the substantially weaker Waterbirds and multiple-shortcut results? What evidence supports generalization of the proposed mechanism to natural-image learning without reliance on globally pretrained representations or group-aware validation?

Q5. How does the contribution differ from [A] Scodellaro et al. (2025), which investigates spatially extended label encodings and shortcut solutions in convolutional Forward-Forward networks?

[A] Training convolutional neural networks with the Forward–Forward Algorithm

5 comments
AuthorsOpenReview ↗

# Response 1 (W1–W2; Q1–Q5): Overview, revisions, and theoretical clarification

Thank you for your careful review and, in particular, for your questions about the theoretical interpretation and the actual fusion input. Your suggestions led us to add input-level constructions, fixed-permutation and readout controls, controlled risk-weighting experiments at the exact 48-dimensional fusion interface, and further natural-image analyses, which helped us connect local scoring, retained information, and final prediction more clearly.

Revision in brief

We clarify what local scores distinguish, what fusion inputs retain, and how fitting selects predictions. The introduction connects these questions. Tables 1–2 specify models, gradient scopes, training budgets, and selection rules. Added analyses include a conditional input-level proof, controlled readouts, risk reweighting, and scratch natural-image comparisons. W1–W2 and Q1–Q5 retain your numbering. Related weaknesses and questions share answers in the three response groups below.

Revised figures

Figure 1 connects local scoring, readout summaries, and the exact 48D fitting interface. Figure 2 adds paired inputs, the shared graph, and a spatial separator. Figure 3 groups coordinate, common-readout, and equal-coefficient controls. Figure 4 moves cross-color similarity and retrieval into the main text. Figure 5 compares local prediction with ASGE feature fusion and BiCovG logit fusion. Appendix Figures A1, A2, and A5 redraw readout interventions, frozen-state comparisons, and the correlation sweep.

Response group 1: W1; Q1; Q3

The constructed pair has identical norm scores for every shared-weight setting of the specified graph. The proof requires interior, nonoverlap, and stride-phase conditions. We connect it to BP, common-readout, native raw-activation, and permutation controls. One Q3 answer distinguishes fixed coordinates, loss averaging, and deterministic mean scores. Sections 3 and 5.2, Table 5, and Appendices L–N add these analyses. With 48 coefficients per score, the detached common-readout difference is +0.384 pp.

Response group 2: W1; W2; Q2; Q5

We clarify prior-work attribution and distinguish retained information from its use during fitting. One table separates 48D affine recovery from risk reweighting, including the matched-color tradeoff. Section 2, Section 5.3, Table 6, and Appendices O–P and U give the comparisons and distinct fitting protocols.

Response group 3: W2; Q4

We compare digit and natural-image results, then address BP, multiple shortcuts, pretraining, and group access. Native spatial worst-group accuracy remains below pooled worst-group accuracy when pretraining and group selection are jointly removed. Sections 5.5–5.6, Table 7, and Appendix R add scratch and frozen-readout comparisons, with label access and accuracy tradeoffs stated.

If you have any further concerns or suggestions, we would sincerely welcome them. We would be grateful for the opportunity to address them and further improve the quality of the paper.

W1 and Q1: What invariance establishes

Convolution can encode layout in channel amplitudes before scoring. Proposition 1 concerns the resulting hidden tensor. The added construction instead starts with horizontal/vertical two-point inputs and two shared convolution–ReLU–max-pooling blocks. Every affected receptive field lies inside the image; no second-block field contains both points. The displacement preserves stride phase. Every affected response therefore has a matching translated response.

For either candidate label and every shared-weight setting, both inputs have identical invariant summaries and norm scores. Any deterministic predictor restricted to these summaries has at least 50% error on the balanced pair. The corresponding GAP-BP graph shares this property. With explicit convolutional and spatial-head weights, candidate gating gives scores (1,−1) and (−1,1). Experimental heads learn their weights; this witness is not used for initialization. The construction establishes available score distinctions under the stated graph conditions. The training controls examine learning, without assigning the original FF gain solely to invariance.

Changes made: Section 3, Figure 2, and Appendix L now give the input graph, response identity, and explicit spatial separator. Section 5.2 and Figure 3 connect these results to the controls.


W1: BP effects and a common final readout

BP reversed-color accuracy rises from 11.45% to 79.28%, alongside FF’s 10.83% to 85.02%. Both benefit from spatial readout. Convolutional widths and pooling match; label conditioning, head grids, and training stages differ. BP therefore serves as an architectural reference.


AuthorsOpenReview ↗

# Response 2 (W1–W2; Q3, Q5): Readout controls, prior work, and retained fusion information

W1 (continued)

The common-readout study freezes each locally trained parent, then fits the same supervised readout family using independent-color data. Readout fitting and selection rules match. Local scores use squared activations with 48 invariant or 4,704 spatial coefficients. Spatial gains are 2.372 points with detached gradients and 1.008 with coupled gradients. Both optimize local losses; coupling allows the second loss to update the first block. With 48 effective coefficients per score, detached invariant/spatial accuracy is 93.264%/93.648%: a paired difference of 0.384 percentage points, with 95% CI [−0.080, 0.848].

A separate raw-activation score control retains greedy training and native prediction with 48 effective coefficients. Accuracy changes from 11.720% to 14.732%: a paired difference of +3.012 percentage points, with 95% CI [−0.070, 6.094]. Its training and prediction protocols differ from the common-readout study.

Changes made: Section 5.1 specifies architectures and budgets. Figure 3, Table 5, and Appendix N add common-readout, equal-coefficient, and native raw-activation comparisons.


Q3: Optimization noise, fixed coordinates, and deterministic scores

Fixed coordinates. One permutation is retained throughout training and evaluation. Accuracy is 89.876%, close to stable indexing at 89.612%. Only score inputs are reordered; propagated tensors keep their original order. This tests consistent indexing without repeated resampling. Co-remapped controls also permute initial head weights.

Loss averaging. R1 and R8 share starting model and optimizer states, data order, and learning-rate schedule. R8 averages eight independent permutation-pair losses before each update. Their expected local objective matches at a fixed state, while sampled updates differ. Reversed-color accuracy improves from 21.039% to 30.272%. Evaluation classifies each draw before averaging accuracies; it does not average logits.

Deterministic scores. Averaging the affine score over permutations gives a channel-weighted GAP score. Applying the nonlinear loss to that mean differs from averaging permutation losses. Deterministic mean-score training therefore changes the objective and score family as well as the estimator. It answers a different question from the R1/R8 noise comparison.

Changes made: Table 5, Table A20, and Appendix M add fixed, co-remapped, fresh-shared, loss-averaging, and mean-score controls. Their objectives and evaluation rules are stated separately.


W1: Relationship to ASGE and BiCovG

ASGE (Gong et al., 2026) introduced spatial-energy supervision and its prediction designs. We credit these components in our two-block adaptation. Our analysis follows its actual 48D fusion input after propagation pooling and normalization. Affine refits test retained information; a separate risk intervention tests its use during fitting.

BiCovG’s multiscale, cross-channel, alignment, and logit-fusion components come from Jiang et al. (2026). At the same checkpoint, fusion reaches 57.27% reversed-color accuracy and final-local prediction reaches 58.76%. Their similarity differs from ASGE’s feature-fusion outcome and motivates examining each prediction interface.

Changes made: Section 2, Table 1, and Appendix U specify attribution and adaptation choices. Figure 1(c) defines the interface. Figure 5 separates local prediction, fusion, and the independently trained single-scale variant.


Q5: Difference from Scodellaro et al.

Scodellaro et al. (2025) study spatially extended label encodings and shortcuts through injected label patterns. We study external color/background correlations and follow them into local scores and final prediction. The two shortcut sources can coexist. Our added coordinate controls, exact-interface refits, and fixed-risk tests examine external cues and their use during fitting.

Changes made: Section 2 and Table A26 now compare the shortcut sources and the prediction questions addressed by each work.


W2: Retained information and readout fitting

The earlier 71.11% linear and 86.65% MLP results use features before propagation pooling. The actual 48D fusion input instead follows propagation pooling, RMS normalization, and global averaging. Native fusion is itself affine in these coordinates. An affine refit therefore uses the same information and function family, without restoring a spatial map. On the original digit test, correlated/independent-color affine fits reach 14.40%/71.86% reversed-color accuracy; MLP fits reach 60.89%/88.28%. Both families recover useful prediction under independent-color fitting. These fits change fitting colors and their derived scalers together. We therefore added the fixed-scaler risk intervention below to isolate risk weighting within one affine family.


AuthorsOpenReview ↗

# Response 3 (W2; Q2, Q4): Risk weighting, information recovery, and natural-image results

W2 (continued)

Changes made: Figure 1(c), Section 5.3, and Appendix O define the exact interface and four refits. Appendix P fixes scaling for the risk comparison.


W2 and Q2: Risk selection versus information removal

Useful affine prediction remains at the exact 48D fusion input. Table R1 reports QMNIST results under two fitting protocols.

Table R1. Predictive information recovery and controlled risk weighting at the actual fusion interface.

Panel A: Information recoveryReversed-color BA
Native fusion10.00%
Independent-color affine refit72.14%
Improvement+62.14 pp
Panel B: Controlled risk weightingp = 0.9p = 0.5
Reversed-color BA25.98%78.22%
Matched-color BA91.07%78.14%
Independent-color BA58.52%78.18%

Panel A: already selected affine heads are evaluated on 50,000 additional QMNIST source IDs. Panel B: auxiliary affine heads use the same frozen features and paired-color digits. Standardization, initialization, regularization, and convergence conditions stay fixed. Only the color-agreement loss weight p changes. BA denotes balanced accuracy; pp denotes percentage points.

Panel B fits the same 49 affine-margin coefficients from zero, with fixed L2 regularization. The positive L2 coefficient makes each objective strongly convex, with a unique minimizer. Fitting uses common convergence criteria without epoch or validation selection. This isolates risk weighting within the specified affine problem. The retained input supports different predictions, with a matched-color tradeoff. Different baselines prevent a unique percentage decomposition of native fusion failure.

Changes made: Section 5.3, Table 6, and Appendices O–P separate recovery from risk weighting and report all color environments. Appendix T documents QMNIST’s writer overlap and study reuse.


W2 and Q4: Why natural-image gains differ

Colored MNIST isolates one controlled color–label relation. Waterbirds combines object recognition and background variation; the stress test adds patches. Spatial scores expose useful structure and nuisance patterns together. Their benefit therefore depends on initialization, predictor, and fitting policy. Scratch and frozen-readout comparisons test those conditions. Worst-group accuracy (WGA) is the minimum accuracy across evaluation groups.

Changes made: Sections 5.5–5.6 compare cue structures and fitting conditions. Table 7 separates pretrained, scratch, and auxiliary predictors.


W2: The BP-ERM gap and multiple shortcuts

Pretrained spatial-local WGA is 24.55%, compared with 53.77% for BP-ERM. Local training supervises each block through its own score; BP sends final-task feedback through the backbone. The gap compares complete feature-learning protocols. The added scratch and frozen-readout comparisons examine initialization and fitting while retaining the BP reference.

With background and patch cues together, spatial-local eight-group WGA is 5.35–6.16% across three environments. A spatial score can access the corner patch as well as object structure. Position access therefore leaves cue selection to the fitted learner. These results motivate examining both available spatial information and the risk used to fit predictions.

Changes made: Table 7 retains BP beside the local models. Section 5.5 summarizes initialization and selection; Appendix G gives the local gradient scope and selection procedure. Section 5.6 and Table A13 report all three background-and-patch environments.


W2 and Q4: Pretraining and access to group labels

We added Waterbirds training from random initialization. Training and average-validation selection use no background-group labels. Native spatial/pooled WGA is 8.19%/10.25%, a 2.06-point spatial deficit; average accuracy is 66.06%/61.79%. This protocol jointly removes pretraining and group-based selection.

On the same frozen spatial-parent FC features, class balancing raises WGA from 0% to 14.92%. Average accuracy changes from 77.84% to 57.69%. These auxiliary fits use ordinary labels and average validation. Group-balanced fitting additionally needs training background annotations and is reported separately. A separate pretrained JTT pipeline uses average validation throughout and reaches 28.50% WGA.

Changes made: Table 7 and Appendix R add spatial, pooled, and BP scratch parents and frozen FC/GAP readouts. Tables A10 and A24 state selection rules and class/group-label access.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.