- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This study explores the shape information missed in the recent proposed forward-forward network. Considering only one layer, traditional convolutional networks or transformers are translation equivariant. However, the original FF network drops the location information when applying mean square as a local goodness score. This study aims to retain the location information (either shift or rotation) to the local goodness by a) introducing pre-defined weight to replace the mean square MLP layer. b) utilizing the BiCovG(Jiang et al., 2026) to replace the global average readout for retaining spatial energy.
Therefore, the proposed shape FF network outperforms previous FF candidates on color binary MINST classification.
Critical strengths
The FF network is a newly rising study topic, which attracts increasing attention in the field. This study attempts to solve an important feature to the backbone, “permutation”(one can imagine translation or rotation also applies) invariance exhibited in the FF network.
Critical weaknesses
There exist several weaknesses in the current draft, the readability requires further improvement. As the title suggests “WHEN GOODNESS FORGETS SHAPE”, the shape information loss is clearly introduced in Fig 1(a) and Fig 2(a). It is intuitive that any global operations could drop the position information. One can also imagine, if we shift or rotate the example, Proposition 1 also holds. However, the term “color shifts” is used throughout the paper and also the experiment validation. All of those examples show how spatial info (H,W) changes, but it is not clear how the “shape” loses across channels(C). The overall introduction and experiment do not match the example story. Assuming this draft intends to solve the “spatial shifts” problem in FFnetwork. Then this problem can be formulated as the mean square “goodness” is translation invariant. The solution seems to be not very solid. Given Fig 2(c), my understanding is that if we would like to capture the diagonal visual feature, then we can design and apply a diagonal feature detector, weight W. If my understanding is correct, then this is equivalent to adding human crafted features to capture each of the shape patterns. We can assume any global operations, e.g. GAP in CNNs or transformers will drop the shape information, then we can introduce manually designed Gabor filters to capture vertical or diagonal visual features. Then this story falls into the old maybe obsolete topic, how to combine learned weights with human crafted vision features. This type of study has limited impacts(personally). Assuming we need to design a series of human crafted visual features as patches to the learned FF network. Then this falls more into the old story, how can one design such patterns. I can only find ONE EXAMPLE feature(diagonal in Fig 2 c), but it is necessary to show how to design such features, e.g., Local Binary Patterns or SIFT features, and especially what did this study use, is it ad-hoc? My understanding on this draft now is Sections 3.1, 3.2 and Fig 2 are all resolving Local goodness, Fig 1(a). How does Fig 1(b) relate to this draft? I guess section 3.3 is trying to solve the Fig1 (b) problem. Given the context, “A fusion rule may combine these logits or instead classify globally averaged block features”, I suppose this fusion head refers to the global average pooling layer used in the final prediction layer, e.g., for classification. Then this “problem” first seems to be a stretch in this study, some networks, regardless of CNNs or transformers, global operations(e.g., GAP) are used for the final classification or visualization, in which case, the spatial information is omitted by design. Otherwise, one can simply use a fully connected layer. That is, the loss of the spatial information is not a problem when choosing GAP, otherwise one can use other alternatives. Assuming all of our choices may lose the spatial information, the solution (using second order information to replace mean square?) is directly borrowed from (Jiang et al., 2026). This can hardly be introduced as a contribution in this draft.
Questions for authors
Please see the questions listed in the above section.
6 comments
# Response 2 (W1–W3): Score definitions, hidden permutations, and color shifts
W1. Study scope, score definitions, and the connection between theory and experiments
Averaging removes explicit position indices. We study how this affects FF supervision and final prediction.
Norm goodness is a parameter-free mean of squared activations. The invariant MLP is a separate capacity control that reads channel energies. The spatial affine head reads raw activations and learns position-specific coefficients. Their different inputs and invariances separate score invariance from learned-head capacity. The spatial head changes what the local loss can access while retaining block-local gradients. CwComp, ASGE, and BiCovG comparisons use their specified components and prediction rules. Each result belongs to its stated FF adaptation and training protocol.
The tests address different steps of prediction. The input construction asks which spatial distinctions a local score can express. For the specified graph, two layouts have identical norm summaries, while a spatial score separates them. Color reversal asks which cues trained predictors use. Coordinate controls test spatial indexing during training. Frozen-interface fits test retained information and the decisions selected by fitting.
Sections 3.1–3.2 and Figure 1(a) define local losses, norm and spatial scores, and the input construction. They explain what each local objective can access and how the current block learns. Section 3.3 follows those representations into final prediction.
Changes made: Section 1 and Figure 1 connect the study sequence to Sections 5.2–5.3. Section 3.2, Tables 1–2 and 4, and Appendix U specify score inputs, parameter budgets, components, and adaptation choices. Figure 2 and Appendices L–O add the construction and controlled tests.
W2. Hidden permutations versus input translation or rotation
Proposition 1 states hidden spatial-permutation invariance: it permutes positions in an already formed hidden tensor. Input translation or rotation depends on the feature extractor. These transformations may change convolution responses, boundaries, and pooling phases. They therefore need a separate input-to-hidden argument. Our new two-block construction uses interior points, disjoint receptive fields, and matching stride phases. Under these conditions, the two input layouts induce permuted hidden responses. We distinguish this mathematical relationship from the color-correlation changes measured after training.
Changes made: Sections 1, 3.1, and 5.1 use these terms consistently. Figure 2(b) and Appendix L state the graph and input conditions.
W3. Color shifts, hidden channels, and spatial information
Color shift changes the relation between color and the label. Image position, orientation, and grayscale structure stay fixed. The binary target is digit ≥ 5. At the primary training correlation ρ = 0.9, color agrees with it with probabilities 0.9, 0.5, or 0.1. This tests cue reliance; the input construction separately tests whether scores distinguish spatial layouts.
K indexes learned hidden channels, while H and W index hidden positions. K differs from input RGB. Norm goodness averages squared activations across all three axes. Before reduction, convolution can encode layout through channel amplitudes. Color enters through RGB and can change those activations. Removing position indices therefore does not establish loss of all layout-dependent information.
Changes made: Section 3.1 defines the axes beside Equation 2 and explains channel amplitudes. Section 5.1 defines the target, RGB generator, and evaluation probabilities.
# Response 3 (W4–W5): Learned spatial heads, analytical examples, and controlled comparisons
W4. What is learned, what is analytical, and what comes from prior work?
Experimental training learns spatial coefficients jointly with the current convolutional block. Each block uses one affine score, g(h) = wᵀvec(h) + b, on raw hidden activations. The same score evaluates competing candidate-conditioned activations. Correct and incorrect candidate label planes produce h⁺ and h⁻. Equation 1 uses softplus(−[g(h⁺) − g(h⁻)]), favoring the correct candidate. The weight gradient depends on the activation difference between candidates at each position. Both block weights and score coefficients learn through the local loss; earlier blocks stay frozen. The candidate-shared bias cancels in the score margin. Thus, the coefficients are optimized across training examples rather than assigned through a catalogue of shape templates.
The old diagonal detector was an analytical weight choice and was never loaded during training. Figure 2 now shows horizontal and vertical point pairs under a specified graph. Their norm summaries agree, while a candidate-conditioned spatial score separates them. One pair suffices to distinguish the score families’ expressivity. It supplies neither experimental descriptors nor fixed training weights. Gabor filters (Daugman, 1985), LBP (Ojala et al., 2002), and SIFT (Lowe, 2004) do not enter this computation. Appendix U compares them with the learned rule. ASGE’s fixed Gaussian projection belongs to a separate adaptation.
Mean-squared goodness already uses squared activations. The primary learned spatial head is affine in raw h. Common-readout controls use h² for both scoring families. BiCovG’s multiscale energy, cross-channel probes, alignment, and logit fusion are attributed to Jiang et al. (2026). These components retain their identities in our two-block adaptation, where component ablations examine their roles. The new exact-fusion and risk-weight tests instead study the ASGE interface. These distinctions identify inherited components and the separate analyses conducted here.
| Component | How obtained | Role |
|---|---|---|
| Analytical spatial separator | Fixed illustration | Shows expressivity; unused in training |
| Native spatial head | Learned from local class supervision | Scores raw h with block-local gradients |
| ASGE projection | Fixed Gaussian component from prior work | Encodes pooled spatial energies |
| Exact 48D auxiliary affine head | Fitted on frozen fusion inputs | Tests retained information and fitting risk |
Changes made: Sections 2–3 and Tables 1–2 clarify inputs, joint updates, and attribution. Section 3.2 explains Equation 3. Figure 2(b,c) and Appendix L add the paired inputs, conditions, separator, and proof. Appendix N explains bias cancellation and restricted coefficients. Appendix U compares learned weights with cited classical descriptors. Table A6 retains component ablations; Appendix D.4 cites BiCovG beside its implementation.
W5. What do coordinate, common-readout, and equal-coefficient controls show?
Stable fixed coordinates retain roughly 89.6–89.9% reversed-color accuracy; fresh independent permutations give 21.53%. Fixed permutations change coordinate labels while preserving a stable index during training. At the function level, a fixed permutation can be absorbed into the affine coefficients. The construction establishes expressivity; stable-versus-fresh tests also depend on training. Fresh permutations also change stochastic loss estimation, which Appendix M examines separately. Common-readout tests keep the final classifier family fixed while changing the local score. With both scores restricted to 48 coefficients, the detached common-readout difference is +0.384 percentage points. These controls distinguish the roles of indexing, readout choice, and head capacity.
A separate raw-h control retains greedy training and native prediction, with 48 effective coefficients. Reversed-color accuracy changes from 11.720% to 14.732%, a +3.012-point paired difference. Its protocol differs from the h²/common-readout study.
Changes made: Figure 3, Table 5, and Appendices M–N now present these coordinate, readout, and coefficient controls.
# Response 1 (W1–W8): Overview and revision guide
Thank you for your close reading and for identifying important issues in the figures, terminology, and explanation of the learning process. Your questions prompted us to clarify the learned nature of the spatial heads, distinguish the analytical separator from handcrafted features, explain the difference between hidden-position permutations and input color shifts, and redesign Figure 1 to better connect local goodness, final prediction, and the fusion-interface analysis.
Overall revision
The revision connects local scoring, retained information, and fitted decisions. Sections 1 and 3 now explain this sequence. The spatial head learns jointly with each block; the analytical separator illustrates expressivity. Color reversal tests cue reliance while keeping digit position and orientation fixed. Table 1 identifies inherited CwComp, ASGE, and BiCovG components.
Added theory and experiments
Figure 2 and Appendix L add an input construction for a specified two-block graph. Interior points, disjoint receptive fields, and matching stride phases ensure identical norm summaries for all shared weights. A candidate-conditioned spatial affine score can separate the inputs.
New coordinate, common-readout, and equal-coefficient controls separate indexing, readout choice, and capacity. With 48 coefficients per score, the detached common-readout difference is +0.384 pp. Refits use our ASGE adaptation’s exact 48D fusion input. With features and other fitting conditions fixed, risk reweighting changes reversed balanced accuracy from 25.98% to 78.22%. Matched balanced accuracy changes from 91.07% to 78.14%. Scratch Waterbirds and FC/GAP tests examine training conditions and regularization.
Redrawn figures
Figure 1 connects local scores, readouts, and frozen affine fitting. Figure 2 shows the paired inputs, graph conditions, and separator. Figure 3 separates original FF/BP results from the new coordinate, common-readout, and equal-coefficient controls. Figure 4 moves cross-color similarity and retrieval into the main text. Figure 5 distinguishes feature fusion, logit fusion, and their inputs. Appendix Figures A1, A2, and A5 redraw readout intervention, frozen-state comparisons, and the correlation sweep.
How the continuations answer your comments
For clarity, we group your comments into eight topics, W1–W8. The Questions field refers back to the same comments.
If you have any further concerns or suggestions, we would sincerely welcome them. We would be grateful for the opportunity to address them and further improve the quality of the paper.