acceptodds
Under review as a conference paper at ICLR 2027

From Parallelism to Specialisation: SSA-Guided Multi-Branch Implicit Neural Representations

Abstract

Although Multi-branch Implicit Neural Representations (INRs) increase the modelling capacity of the network, the parallel paths are not inherently guided to learn distinct components of the target signal. To address this limitation, we introduce a Singular Spectrum Analysis (SSA) guided multi-branch INR, which provides each branch with a specific signal component as the branch-specific target. For a two-dimensional signal, for example, an image, a separable two-dimensional SSA decomposes the target signal into 4 distinct components, which separately represent structure, horizontal-detail, vertical-detail, and joint-detail, to provide each branch with a designated target to achieve. Gated contributions of each branch are then additively combined to obtain the final signal reconstruction. In addition, we introduce a Particle Swarm Optimisation (PSO)-initialised, learnable coordinate scaling at the input to each branch to enable it to model the target effectively. Branch-wise pretraining against the respective SSA component ensures that all the parallel branches are properly supervised to attain the respective target. Experiments on Kodak images demonstrate that the proposed algorithm shows the best overall peak signal-to-noise ratio (PSNR) performance among state-of-the-art (SOTA) algorithms including COSMO-RC, INCODE, FR-INR, and SIREN. Moreover, ablation studies against unguided parallel INRs show that the guided model consistently outperforms the unguided baseline. Spatial alignment matrices, training dynamics, and Fourier-domain analysis suggest that the learned branches retain distinct and coherent roles. The same specialisation principle extends to any n-dimensional signal, including audio, which is a one-dimensional signal. Further ablations demonstrate that the main source of the improvement is branch specialisation, indicating that specialisation and explicit branch-wise task allocation are more important than branch parallelisation alone. The complete source code and implementation will be made publicly available upon acceptance.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Implicit neural representations, Singular spectrum analysis, Branch specialisation, Spectral bias

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_habcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a multi-branch implicit neural representation method based on singular spectrum analysis (SSA), aimed at improving image representation performance by explicitly assigning learning tasks to different branches. The authors first decompose an image into four components—structure, horizontal details, vertical details, and joint details—then use four SIREN networks to fit each component separately, and finally combine the outputs of each branch to reconstruct the original image. Additionally, the authors introduce strategies such as a gating mechanism, branch pretraining, and PSO initialization to optimize training, and evaluate the method’s performance on the Kodak dataset.

Critical strengths

The research motivation behind this paper is quite clear: addressing the challenge of naturally achieving effective task division in multi-branch networks, it utilizes SSA to provide explicit supervision objectives for different branches, enabling each branch to learn distinct signal components. This design offers a certain degree of intuitiveness and interpretability. Furthermore, the authors combine SSA decomposition, gating mechanisms, coordinate scaling, and joint optimization. By analyzing the performance of the method from multiple perspectives—including branch alignment, spectral analysis, and ablation experiments—they provide a basis for understanding the roles of each branch and leave room for exploring the application of this approach in other signal representation tasks.

Critical weaknesses
  1. Inconsistent training budgets and attribution issues The full method includes 500 rounds of independent pre-training, 500 rounds of joint training, and PSO search for each branch; the baseline only matches the computational effort of the joint training. The performance differences are confounded by the effects of pre-training, search initialization, supervision schemes, and computational overhead, making it impossible to attribute them solely to the benefits derived from branch division.
  2. Comparison of decomposition methods: Numerical values conflict with textual conclusions This is further substantiated in Table 5 and shows for the highest PSNR value (SSA) 48.61 dB, followed by the values for FFT and DWT of 48.29 dB and 48.31 dB, However, this contradicts the descriptions in the main text and Appendix A.8. Authors should use consistent benchmarks and experimental setups to ensure that the values and differences in the tables, as well as the conclusions in the main text, are consistent.
Questions for authors

After balancing the number of parameters with the total optimization budget and accounting for pretraining and PSO costs, how much of an advantage does the complete method retain over the unguided baseline? Which SSA result in Table 5 should be used as a reference? Which images and how many runs do the ablation experiments in Table 2 correspond to? Under the condition of retaining branch-assisted supervision but not assigning different spectral roles, can the additional benefits of SSA task division still be observed? Compared to existing decomposition-based supervision methods such as MINER, what is the most significant new contribution of this paper, and which experiments directly support this claim?

10 comments

[W] Inconsistent training budgets and attribution issues The full method includes 500 rounds of independent pre-training, 500 rounds of joint training, and PSO search for each branch; the baseline only matches the computational effort of the joint training. The performance differences are confounded by the effects of pre-training, search initialization, supervision schemes, and computational overhead, making it impossible to attribute them solely to the benefits derived from branch division.

[A] Thank you for raising this concern. It prompted us to make the training comparison fairer by accounting for the full fitting cost and to state more precisely what the ablation shows about the contribution of SSA supervision.

We compared the proposed method with an architecturally matched unguided four-branch INR under matched per-image fitting times on Kodak-24 (Table 6, Appendix A.7). The budget for the proposed method includes SSA target construction, PSO initialisation, branch pretraining, and joint training. Both methods averaged approximately 911 s per image. The proposed method achieved a mean PSNR of 48.31 dB compared with 46.36 dB for the unguided control, a gain of 1.95 dB, with higher PSNR on all 24 images.

In the cumulative ablation (Table 2), adding branch-specific SSA supervision during joint training increased PSNR from 45.50 to 47.01 dB on kodim20.png before branch pretraining and PSO were introduced. This result supports a contribution from SSA supervision. The matched-time gain measures the performance of the complete method.

We also reran the external baselines and the unguided parallel network for 1,000 epochs. The proposed method retained its PSNR advantage in these runs, and we will report the updated comparison in Figure 4.

[W] Comparison of decomposition methods: Numerical values conflict with textual conclusions This is further substantiated in Table 5 and shows for the highest PSNR value (SSA) 48.61 dB, followed by the values for FFT and DWT of 48.29 dB and 48.31 dB, However, this contradicts the descriptions in the main text and Appendix A.8. Authors should use consistent benchmarks and experimental setups to ensure that the values and differences in the tables, as well as the conclusions in the main text, are consistent.

[A] We would like to thank the reviewer for pointing this out. The absolute PSNR values in the original Table 5 placed SSA first on kodim20.png. The FFT and Haar DWT differences were computed against an incorrect SSA reference, which led to the conflicting interpretation in the text.

The revised comparison in Table 7 covers all 24 Kodak images. Each decomposition was fitted once per image at seed 42 using the same four-branch INR architecture and training schedule, including PSO scale initialisation, 500 epochs of branch pretraining, and 500 epochs of joint optimisation.

Table 7 uses SSA’s mean best-checkpoint full-image PSNR of 48.41 dB as the reference for Δ\DeltaPSNR. FFT, Haar DWT, and KLT were 0.31 dB, 0.34 dB, and 1.12 dB lower, respectively. Section 4.5 and Appendix A.8 now report these same differences and state that SSA achieved the highest mean PSNR among the four configurations evaluated on Kodak-24.

[Q] After balancing the number of parameters with the total optimization budget and accounting for pretraining and PSO costs, how much of an advantage does the complete method retain over the unguided baseline?

[A] We thank the reviewer for this question.

We added a matched-time comparison across all 24 Kodak images at seed 42 (section 4.4 and Appendix A.7, Table 6). The unguided control matched the proposed model in branch count, width, depth, and fusion. It used fixed unit coordinate scales and had 533,520 trainable parameters, compared with 533,528 for the complete method. The eight additional parameters are the learnable coordinate scales.

For each image, the unguided control received the complete method’s measured end-to-end fitting time, including uncached SSA target construction, PSO scale initialisation, branch pretraining, and joint optimisation with checkpoint evaluation. The mean fitting times were 911.27 s for the complete method and 911.19 s for the unguided control. The largest per-image difference was 0.57 s.

Under these matched budgets, the complete method achieved a mean best-checkpoint PSNR of 48.31 dB versus 46.36 dB for the unguided control. It retained a mean paired advantage of 1.95 dB and achieved higher PSNR on all 24 images.

Reviewer_ARgFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a Singular Spectrum Analysis (SSA) guided multi-branch INR in which each branch has a specific learning target. Details of the implementation include an additive combination of gated components, a PSO-initialised learnable coordinate scaling and a two-stage training process (branch-wise pretraining followed by joint optimization). The paper includes the theoretical formulation as well as some empirical studies on image and audio representation.

Critical strengths

The proposed idea is a clever approach to improve upon existing methods. The methodology was presented clearly. The categorisation of the related works makes the development of the field clear to follow and pinpoints the scope and motivation of this work.

Critical weaknesses

The empirical validation is limited to 24 images and a single audio file. The lack of scale (few images and only one audio) and diversity of experiments (no 3D tasks like NeRFs or video) makes it difficult to conclude that the significance of this paper is sufficient for publication.

The wall-clock time comparison to baselines’ runtime is limited to one run of one image and Appendix A.8 states that FFT and DWT outperform the proposed method. These details diminish the practical utility of adopting this method. Further, this information is all in the Appendices whereas the main paper only presents positive results - it would improve transparency if the tradeoffs were acknowledged in the main text.

There are also several minor formatting errors that when combined, reduce the professional standard of the paper's presentation:

  • \citet used instead of \citep for some works in Section 2.1
  • Figure 3 has poor cropping of the axis labels and the legend obstructs the data
  • No in-text references to Figures 2 and 4
  • Section 4.5 “table 2” missing capitalization for consistency and period to end sentence. This may not be a complete list but I strongly suggest editing and proof-reading.
Questions for authors

Appendix A.8 states that FFT and DWT outperform SSA, so why was SSA chosen for the proposed method? You correctly state that a single-image result does not establish a general ranking but no rigorous alternative is provided. Please also clarify why Table 5 seems to suggest the opposite with SSA being the top performer of these 4 methods.

Why do the PSNR values for kodim20 not match in Tables 4 and 5?

For the 24-image study, were multiple seeds tested? Reporting standard deviation over multiple seeds would significantly improve the robustness of the results.

11 comments

[W] The empirical validation is limited to 24 images and a single audio file. The lack of scale (few images and only one audio) and diversity of experiments (no 3D tasks like NeRFs or video) makes it difficult to conclude that the significance of this paper is sufficient for publication.

[A] Thank you for encouraging us to broaden the empirical evaluation. We added native-resolution image fitting on the seven DIV2K images for which the proposed method, INCODE, FR-INR, and SIREN all completed fitting. These images contain 2.74–3.77 megapixels each. The proposed method achieved the highest PSNR on all seven images, with a mean best-checkpoint PSNR of 33.50 dB and a mean paired gain of 5.87 dB over INCODE, the baseline with the highest mean PSNR. Section 4.4 summarises the results, and Appendix A.10 reports the per-image scores and fitting protocols. The COSMO-RC run on DIV2K was incomplete. Its Kodak-24 results appear in Fig. 4.

We also fitted a 3D occupancy field to the Lucy shape. The comparison figure shows the reconstructed surface alongside the ground truth and the COSMO-RC, INCODE, and SIREN reconstructions. We present the Lucy and audio results as single-instance demonstrations of 3D shape fitting and audio representation, respectively.

[W] The wall-clock time comparison to baselines’ runtime is limited to one run of one image and Appendix A.8 states that FFT and DWT outperform the proposed method. These details diminish the practical utility of adopting this method. Further, this information is all in the Appendices whereas the main paper only presents positive results - it would improve transparency if the tradeoffs were acknowledged in the main text.

[A] Thank you for asking us to quantify the fitting costs and discuss their practical implications in the main text. We added a computational profile across all 24 Kodak images and compared the SSA-guided INR with an unguided four-branch INR under matched per-image end-to-end fitting times (Section 4.4 and Appendix A.7, Tables 5 and 6). The mean fitting times were 911.27 and 911.19 s per image, respectively. The SSA-guided INR achieved a mean best-checkpoint PSNR of 48.31 dB versus 46.36 dB for the unguided INR, a mean paired gain of 1.95 dB, with higher PSNR on all 24 images. Section 4.5 also reports that uncached SSA target construction, PSO initialisation, and branch pretraining together accounted for 63.7% of the measured SSA-guided fitting time.

We resolved the reporting discrepancy by updating Appendix A.8 and Table 7 to describe the same Kodak-24 decomposition experiment. The SSA row uses the original seed-42 reconstruction fits reported in Table 3. The mean PSNRs were 48.41 dB for SSA, 48.10 dB for FFT, 48.07 dB for Haar DWT, and 47.29 dB for KLT. Section 4.5 now highlights the practical trade-off that Haar DWT came within 0.34 dB of SSA at a similar reported mean runtime of 364.5 versus 366.6 s per image.

[W] There are also several minor formatting errors that when combined, reduce the professional standard of the paper's presentation:

  • \citet used instead of \citep for some works in Section 2.1
  • Figure 3 has poor cropping of the axis labels and the legend obstructs the data
  • No in-text references to Figures 2 and 4
  • Section 4.5 “table 2” missing capitalization for consistency and period to end sentence. This may not be a complete list but I strongly suggest editing and proof-reading.

[A] We proofread the revised manuscript and corrected the citation style in Section 2.1, using \citep for citations that follow method names. In the joint-training plot, Figure 3(a) in the revision, we adjusted the margins to show the full axis labels and moved the legend above the axes. Section 4.3 now refers to Figure 2 when discussing branch specialisation, and Section 4.4 refers to Figure 4 when presenting the Kodak results. We also capitalised “Table 2” and added the missing full stop in Section 4.5.

We would like to thank the reviewer for pointing out these issues. Correcting them significantly improved the presentation quality of the paper.

Reviewer_3GUsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper focuses on improving the architecture of implicit neural representations (INRs) for image representation. More specifically, it addresses how to design and train multi-scale or multi-branch INRs. It proposes how to assign and guide each branch to represent a different component of an image decomposition-based features, such as Singular Spectrum Analysis (SSA). This ensures that each branch focuses on its own area and does not waste expressive capacity on information that other branches can represent. Through experiments, the paper shows that each branch captures different information and that the method performs better than the baselines. It also shows that branch-specific guidance clearly improves performance compared to not using it.

Critical strengths

The idea is well grounded in signal processing. As the paper assumes, having branch-specific targets seems to improve performance. Each branch can focus on what it does best, without wasting capacity on information that other branches can already represent.

Critical weaknesses

Weak Comparison Fig. 4 clearly shows that using a guided parallel INR (the proposed method) improves performance. However, it should also be compared with other multi-resolution and multi-branch and other advanced architectures, such as Neural Experiments, which the paper already cites. In addition, it is unclear how much of the improvement comes from the proposed method itself, rather than from having a larger and more complex architecture than the compared baselines.

Questions for authors
  • If time allows, would it be possible to add more comparable baselines with similar network sizes? This would make it much easier to see the improvement brought by the proposed branching and branch-specific guidance.
5 comments

[W] Weak Comparison Fig. 4 clearly shows that using a guided parallel INR (the proposed method) improves performance. However, it should also be compared with other multi-resolution and multi-branch and other advanced architectures, such as Neural Experiments, which the paper already cites. In addition, it is unclear how much of the improvement comes from the proposed method itself, rather than from having a larger and more complex architecture than the compared baselines.

[A] Thank you for the suggestion. We extended the native-resolution Kodak-24 comparison to include Neural Experts (Section 4.4 and Appendix A.9, Table 8). The SSA-guided INR achieved 48.41 dB mean PSNR compared with 42.27 dB for Neural Experts, a mean paired gain of 6.14 dB, with higher PSNR on all 24 images. Appendix A.9 also reports MINER’s results on its (256×256256\times256) crops.

To assess the effect of model capacity and fitting effort, Table 6 compares the full SSA-guided pipeline with an unguided INR of the same branch count, width, depth, and fusion mechanism. The models differ by only eight trainable scale parameters. We matched the end-to-end fitting time per image, accounting for SSA target construction, PSO initialisation, and branch pretraining. Under this protocol, the guided method retained a 1.95 dB mean PSNR advantage across Kodak-24.

[Q] If time allows, would it be possible to add more comparable baselines with similar network sizes? This would make it much easier to see the improvement brought by the proposed branching and branch-specific guidance.

[A] We would like to thank the reviewer for suggesting these additional experiments to make the comparison better.

Table 9 adds a four-expert Neural Experts model with 532,288 trainable parameters, within 0.23% of our model’s 533,528. On native-resolution kodim20.png, the two models achieved 44.13 dB and 48.31 dB, respectively, under their documented fitting schedules.

We also added a paired ablation in Section 4.5 to test branch-specific supervision directly. Both four-branch INRs received branch-level supervision and used the same network size, initial weights, batch order, and training budget. Replacing the shared branch target with distinct SSA targets increased PSNR by 1.34 dB on the tested image.

Thank you for your constructive feedback and for encouraging us to compare our method with a broader range of INR architectures, including models of comparable size.

We have addressed all of your comments in the revised manuscript, conducted all of the suggested experiments, and reported their results. The added comparisons help clarify the contributions of the proposed branching and branch-specific guidance to reconstruction performance.

Please let us know if you would like any further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.