- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper replaces the static latent transformation in an INR with a time conditioned vector field. A kinetic-energy penalty regularizes the dynamics. The paper reports improved reconstruction and parameter efficiency across INR families and data modalities. The authors claim that these improvements come from increased expressivity and gradient diversity,
Critical strengths
It makes sense to reuse a compact transformation across integration steps to save storage I like the diverse experimental setting. The paper includes FLOP-matched and training-time-matched FFNet comparisons, recurrent-update ablations, and comparisons with weight decay.
Critical weaknesses
I feel that the paper over states its novelty. Parameterizing latent derivatives and obtaining outputs through ODE integration are already existing Neural ODE constructions. Same goes for Kinetic regularization of these dynamics. The connection is hidden in Appendix C. I agree that applying these ideas effectively to INR compression could still be a useful contribution. However, as I see, with fixed Euler steps, Eq. (5) is exactly a time-conditioned residual network with shared weights.
Some important hyperparameters seems to be missing. The paper does not provide enough information such as default integration steps, batch sizes, initialization settings, or validation procedure. These details are important.
I believe that the empirical NTK procedure is underspecified. Can you please define information such as effective rank, eigenvalue thresholds used for condition numbers, coordinate sample size etc.? Further, eigenvalue distributions alone also do not establish improved learning of spatial high-frequency modes without examining the corresponding eigenvectors or target alignment.
Questions for authors
See above weaknesses
3 comments
Response to Reviewer Ck87 (Continue)
Some important hyperparameters seems to be missing. The paper does not provide enough information such as default integration steps, batch sizes, initialization settings, or validation procedure. These details are important.
We thank the reviewer for raising this point.
- Documentation: We have consolidated all training details into a dedicated Appendix E (complementing current Section 5) to enhance clarity and reproducibility.
- Open Source: We have linked the turbulence experiment code here https://anonymous.4open.science/r/DINR_Code-3EB2 and are cleaning up the remaining code and data to fully open-source everything.
Can you please define information such as effective rank, eigenvalue thresholds used for condition numbers, coordinate sample size etc.?
Thank you for pointing this out. We have updated Appendix F of the revised manuscript with the implementation details for our empirical Neural Tangent Kernel (NTK) evaluation pipeline. Below is a high-level overview of the NTK definition and its practical implementation on the turbulence dataset:
| Parameter / Metric | Implementation / Value | Description |
|---|---|---|
| Coordinate Sample () | ( uniform subgrid) | Sampled from the full field |
| Kernel Size | , symmetrized | |
| Normalization | Trace Normalization | |
| Computation Precision | Jacobian: float32, Eigensolver: float64 | Eigensolver via torch.linalg.eigvalsh |
| Effective Rank Formula | Participation Ratio | |
| Condition Number Threshold |
Further, eigenvalue distributions alone also do not establish improved learning of spatial high-frequency modes without examining the corresponding eigenvectors or target alignment.
We thank the reviewer for the suggestion. We therefore provide visualizations of NTK eigenvectors in Figure 5 of the revised manuscript. Moreover, we independently measure reconstruction error in spatial-frequency bands using full-resolution predictions and targets. Specifically, we apply an orthonormal 2D FFT to the prediction residual and sum its squared amplitudes within each band, dividing by the total number of pixels. The bands are defined by radial frequency , measured in cycles per original image pixel: low , middle , and high .
| Dataset | Frequency band | FFNet MSE | Dynamical FFNet MSE | Error reduction ↓ |
|---|---|---|---|---|
| Turbulence | Low | 3.480924 × 10⁻⁴ | 1.145660 × 10⁻⁴ | 67.09% |
| Turbulence | Middle | 4.405221 × 10⁻⁴ | 1.687602 × 10⁻⁴ | 61.69% |
| Turbulence | High | 2.461093 × 10⁻⁴ | 2.308004 × 10⁻⁴ | 6.22% |
| DIV2K | Low | 7.998935 × 10⁻⁴ | 1.306821 × 10⁻⁴ | 83.66% |
| DIV2K | Middle | 5.548542 × 10⁻⁴ | 6.120459 × 10⁻⁵ | 88.97% |
| DIV2K | High | 4.468282 × 10⁻³ | 1.691470 × 10⁻³ | 62.14% |
The results show DINR reduces error in every band in these two examples, including the high-frequency band. In the revised manuscript, the above table is included in Table 4 and the accompanying training curves is provided in Figure 5 of the revised manuscript.
Review Reminder
Dear Reviewer Ck87,
Thank you for your constructive feedback and thoughtful questions. We have carefully incorporated your suggestions into our revised manuscript (with key updates highlighted in red) and addressed each of your points in detail above. We invite you to review the updated paper and our responses, and we hope you will consider re-evaluating our work in light of these revisions.
Thank you for your time and service to the community!
Best regards, The Authors
Response to Reviewer Ck87
We acknowledge that parameterizing latent derivatives and obtaining outputs through ODE integration are established constructions in Neural ODEs. We therefore do not claim to introduce a fundamentally new architecture. Rather, our primary goal is to investigate the representational properties that continuous latent dynamics bring to implicit neural representations (INRs).
Specifically, our contribution is to establish latent dynamics as a source of representational capacity for compact INRs. By leveraging dynamical evolution, smaller network can achieve higher reconstruction fidelity than larger static counterparts, improving parameter efficiency and compression (Fig. 3), which are important objectives in implicit signal representation.
We further substantiate this contribution through extensive experiments, rigorous validation, and theoretical justification (Supported by a consensus of all three reviewers.). Rademacher complexity, which characterizes the expressivity and provides insights into generalization (Appendix A), and Neural Tangent Kernel (NTK) analysis, which examines gradient diversity and trainability (Appendix B). Both analyses are supported by our empirical results (Figs. 3 and 4).
In response to the reviewer's concern, we have revised the manuscript throughout to avoid overstating architectural novelty and clarify our intended contribution. Specifically, we have strengthened the positioning of DINR relative to Neural ODEs and related dynamical INR approaches also in the Introduction, and Method sections. We have also clarified that our contribution lies in investigating and establishing the representational role of dynamical evolution in compact implicit signal representations, rather than proposing a new dynamical architecture.
We agree with the reviewer that KE regularization is an established technique, and we adopt rather than propose it. We have revised the manuscript to avoid implying otherwise. Our contribution lies in demonstrating its benefits for implicit signal representation. While prior work has primarily employed KE regularization in continuous normalizing flows to reduce numerical integration costs, we investigate its role in controlling latent trajectory complexity and improving generalization in INRs. This benefit is demonstrated through reconstruction from noisy or limited observations (Appendix F) and further supported by our theoretical analysis (Appendix A).
We appreciate the reviewer's observation. We agree that the network formulation introduced in Eq. (5) corresponds to a standard Neural ODE with fixed-step Euler discretization. We have explicitly acknowledged this connection in the revised manuscript.
However, our contribution is not the underlying dynamical architecture itself. As the reviewer notes, our contribution lies in effectively applying this established formulation to INRs, where representational capacity and parameter efficiency are central objectives. To clarify this distinction and better contextualize our work, we have added a Related Work section (Section 2) to the main manuscript and revised the Introduction, Methodology, and Conclusion to explicitly establish the connection to Neural ODEs and articulate our distinct objectives.
We substantiate this through theoretical analyses, together with extensive empirical evaluations. Specifically, DINR consistently improves the fidelity-storage trade-off across four INR backbones (Fig. 3), while a FLOP-matched static INR does not close this performance gap (Fig. 7). Furthermore, time-conditioned updates significantly outperform time-independent updates (Table 3), highlighting the importance of explicitly modeling latent feature evolution.
We hope these revisions clarify that, while DINR builds on the established Neural ODE formulation, our work investigates a distinct application and representational objective, supported by theoretical analysis and empirical evidence. Our contribution is to establish and characterize the benefits of dynamical evolution for compact implicit signal representation and compression, rather than to propose a new dynamical architecture.