acceptodds
Under review as a conference paper at ICLR 2027

HerKry: A Hermite–Gaussian Hierarchy with Recurrent Krylov Trajectory Modeling for Robust Sparse-View CT Reconstruction

Abstract

Sparse-view computed tomography (SVCT) relies heavily on learned priors, leaving it vulnerable to clinical distribution shifts. While a coarse-to-fine hierarchy can regularize early reconstruction by restricting weakly constrained degrees of freedom, directly extending data consistency (DC) to coarse scales is nontrivial. Image and sinogram downsampling do not generally commute with fan-beam projection, introducing geometry-dependent inconsistencies. To resolve this, we propose Hermite-Krylov CT (HerKry), which models images via a continuous Hermite–Gaussian (HG) hierarchy shifting from sparse, high-order primitives to dense, lower-order ones. This parameterization decouples representation resolution from measurement scale, enabling geometry-consistent DC directly against native sinograms. Crucially, higher-order HG modes can absorb complex spatial deformations via linear coefficient combination. This allows DC to freeze nonlinear parameters and optimize exclusively in linear coefficient space via Hermite-Coefficient CGLS (HC-CGLS), slashing DC latency by over with better fidelity. A Krylov-Trajectory GRU (KT-GRU) further encodes intermediate solver iterates to propagate optimization dynamics across stages. Extensive evaluations across cross-organ transfer, varying noise, and extreme sparsity demonstrate superior robustness, outperforming competing methods by up to . The code is available at https://anonymous.4open.science/r/Code-for-HerKry-DB40https://anonymous.4open.science/r/Code-for-HerKry-DB40.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Sparse-View CT Reconstruction, Inverse Problems, Gaussian Representation, 2D Gaussian Splatting, Deep Unfolding Networks

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AK7fOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper identifies a gap in the current CT reconstruction literature where the operators for downsampling and projection to sinograms to not commute, leading to imperfect reconstructions. They proposed an ML framework based on Hermite-Gaussian bases and Krylov iterates to solve it. The results show very good performance and were validated against ~10 baselines.

Critical strengths

I liked the identification of the problem (non-commutative operators) and the solution to it, in particular the Hermite basis which leads to linear coefficients that are easy to optimize.

Critical weaknesses

No significant weaknesses that I could find.

Questions for authors
  1. How did you identify Hermite polynomials as the representation to use, any why? what are the key advantages compared to other representations. Any prior work applying these representations for images/medical scans, not necessarily CT?
  2. The recurrent modelling of Krylov trajectories with recurrent GRU, is that your contribution in this paper as well? has this been done before? would help to clarify what exactly is new and what has been done before
2 comments
AuthorsOpenReview ↗

Response to Reviewer AK7f

We sincerely thank Reviewer AK7f for the strong support and positive evaluation of our work. Below we address your two questions concisely.


1. Motivation, Advantages, and Prior Work of Hermite-Gaussian (HG) Representations "1. How did you identify Hermite polynomials as the representation to use, and why? what are the key advantages compared to other representations. Any prior work applying these representations for images/medical scans, not necessarily CT?"

Response:

  • Why Identified: Standard 2D Gaussians (d=0d=0) [1] lack directional structure when sparse, and optimizing their nonlinear coordinates (μi,Σi)(\mu_i, \Sigma_i) via Adam requires costly rendering cache rebuilds at every step. Taking the Taylor expansion of a displaced or stretched Gaussian reveals that its spatial derivatives can be approximated as Hermite polynomials multiplied by the Gaussian envelope [2] (Appendix C, Eqs. 20–24).
  • Key Advantages: (1) vs. Discrete Multi-Scale Bases (Pixel-Based): HG primitives are analytically continuous, projecting directly into native sinograms without fan-beam downsampling mismatches (DyAfine≠AcoarseDx\mathcal{D}_y\mathcal{A}^{\text{fine}} \neq \mathcal{A}^{\text{coarse}}\mathcal{D}_x). (2) vs. Standard Gaussians (d=0d=0): Higher-order HG modes absorb local translations and anisotropic deformations linearly under frozen geometry, slashing DC latency by >89% and boosting reconstruction performance (Table 2a, Table 4).
  • Prior Work in General and Medical Imaging: In 2D/3D image processing and medical scans, fixed-grid Gaussian-Hermite moments and discrete steered Hermite transforms have been applied to rotation-invariant image analysis [3], digital image watermarking [4], and cardiac CT optical flow estimation [5]. However, these prior works operate as fixed-window discrete filter banks on reconstructed pixel grids. Formulating learnable, spatially adaptive 2D anisotropic HG splatting to decouple tomographic scales and linearize physical DC is entirely new (now discussed and cited in Appendix A).

2. Novelty Boundary of Recurrent Krylov Trajectory Modeling (KT-GRU) "2. The recurrent modelling of Krylov trajectories with recurrent GRU, is that your contribution in this paper as well? has this been done before? would help to clarify what exactly is new and what has been done before"

Response:

Yes, this is our novel contribution. To clarify the exact boundary with prior work (updated in Sec. 3.4 & Appendix A):

  • Done Before: Classical inverse problem theory established that CGLS iterates exhibit semi-convergence across nested Krylov subspaces [6]. However, existing deep unfolding networks with inner CG solvers [7] treat the KK-step solver strictly as a black-box terminal mapping (a0↦aKa^0 \mapsto a^K), discarding all intermediate solver states, while recurrent unrolling networks [8] only apply RNNs across outer image blocks.
  • New in HerKry: To our knowledge, no prior work has extracted the intermediate inner-loop states of CGLS across both coefficient space and pixel space, and summarized their optimization trajectory via a GRU. Specifically, we recurrently encode the intermediate Hermite-coefficient trajectory {alk}k=1K\{a_l^k\}_{k=1}^K (alongside solver velocity Δalk\Delta a_l^k and acceleration Δ2alk\Delta^2 a_l^k) via KT-GRU and fuse it with sampled pixel-space residual back-projections UlU_l, capturing solver momentum to improve model performance (+0.70 dB OOD gain, Table 2b).

References

[1] Peng, L., Wu, A., Li, W., et al. Pixel to Gaussian: Ultra-fast continuous super-resolution with 2D Gaussian modeling. ICLR, 2026.

[2] Refregier, A. Shapelets—I. A method for image analysis. Monthly Notices of the Royal Astronomical Society, 338(1):35–47, 2003.

[3] Yang, B., Suk, T., Dai, M., & Flusser, J. 2D and 3D Image Analysis by Gaussian-Hermite Moments. In Moments and Moment Invariants: Theory and Applications, pp. 143–173, 2014.

[4] Sabbane, F., & Tairi, H. Watermarking approach based on Hermite transform and a sliding window algorithm. Multimedia Tools and Applications, 82(30):47635–47667, 2023.

[5] Mira, C., Moya-Albor, E., Escalante-Ramírez, B., et al. 3D Hermite Transform Optical Flow Estimation in Left Ventricle CT Sequences. Sensors, 20(3):595, 2020.

[6] Gazzola, S., & Sabaté Landman, M. Krylov methods for inverse problems: Surveying classical, and introducing new, algorithmic approaches. GAMM-Mitteilungen, 43(4):e202000017, 2020.

[7] Hu, M., & Xiao, G. MASCU-Net: Memory-augmented adaptive shrinkage cascading unfolding network for interpretable sparse-view CT reconstruction. Measurement Science and Technology, 37(4):045402, 2026.

[8] Zhang, Y., Chen, H., Xia, W., et al. LEARN++: Recurrent dual-domain reconstruction network for compressed sensing CT. IEEE TRPMS, 7(2):132–142, 2023.

AuthorsOpenReview ↗

Official Comment by Authors

We trust that these clarifications resolve your concerns. Should any part of our explanation remain ambiguous or warrant further revision, we warmly welcome your feedback and would be happy to discuss any remaining points.

Reviewer_yJNpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work identifies the problem of naively pairing coarse images with downsampled sinograms, as the geometry beceomes inaccurate. Therefore they propose a method thaat uses Hermite–Gaussian hierarchy to model the coarse to fine process by shifting from high-order to low-order. The network predicts HG primitive geometry and initial coefficients, while an inner CGLS solver refines the coefficients under fixed geometry to fit the measurements. Encoded solver trajectories guide subsequent neural reconstruction blocks. The proposed method shows better performance in reconstruction both in in-distribution and out-of-distribution scenarios.

Critical strengths

This work identified a problem where representation-based CT reconstruction would encounter. Experiments and results show the effectiveness.

The comparison is extensive.

Critical weaknesses

This work identifies the coarse-to-fine representation problem, but the proposed method at its core is actually an end-to-end supervised learning using the ground truth reconstructions.

The steps of the optimization is not very clearly described. For example I am not entirely clear how the Hermite–Gaussian representation measurement fitting is combined with supervised learning. Or how the gradient in CGLS plays along the neural branch.

This work combines Gaussian representation reconstruction with supervised learning. The Gaussian splating style reconstruction require no ground truth, and can just optmize to fit the measurement. This combination requires ground truth for training.

In addition, as it combines reconstruction with image-to-image supervised learning, the trained network would only work for that specific geometry used for reconstruction. I am not clear if the training is geometry specific or it trains with different geometries.

My feelings is that the paper lacks detailed experiments to understand why this Hermite–Gaussian modeling matters that much.

Questions for authors

It is not clear for me, why this Hermite–Gaussian representation should work? Since this is at its core a supervised learning, what if we don't make the coarse-to-fine process and directly use the fine representation?

What would be happen if you use the coarse represention from inaccurate geometry downsampling anyway? Would it really harm the performance? The mismtach percentage in Figure 1 b seems to be acceptable? And the coarse representation is not supposed to reconstruct perfect details anway.

What would be happen it the Hermite–Gaussian representation reconstruction is replaced by simply standard Gaussian splatting like R2Gaussian? Do authors have a explaination why doing Hermite–Gaussian modeling would perform better than standard Gaussian? Does this performance gain come from solely Hermite–Gaussian modeling or from more coefficients per primitive? Maybe a comparison under similar learnable coefficents would be interesting?

Why would this approach works better in OOD case than comparative methods? The Hermite–Gaussian modeling and supervised learning shalln't garanutee that? Is it because this method encorporate data consistency? It shall be separated to investigate if it comes from the data consistency encorporation or the Hermite–Gaussian modeling?

Why the approach would promote strict data consistency, although empirically it shows so in the results?

Minor: The code link doesn't work.

6 comments
AuthorsOpenReview ↗

Response to Reviewer yJNp

We thank Reviewer yJNp for the detailed feedback. Since several comments in Weaknesses and Questions are interconnected, we quote your exact sentences below and address them point by point.


1. Paradigm Clarification "This work identifies the coarse-to-fine representation problem, but the proposed method at its core is actually an end-to-end supervised learning using the ground truth reconstructions." "This work combines Gaussian representation reconstruction with supervised learning. The Gaussian splatting style reconstruction require no ground truth, and can just optimize to fit the measurement. This combination requires ground truth for training." "What would be happen if the Hermite–Gaussian representation reconstruction is replaced by simply standard Gaussian splatting like R2Gaussian?"

Response:

There is a conceptual distinction between per-scene 3D Gaussian Splatting (e.g., R²-Gaussian [1]) and feed-forward 2D Gaussian image representation.

Rather than optimizing a single scan from scratch, recent works use neural networks to predict 2D Gaussian primitives as a continuous, resolution-free spatial representation for super-resolution and tokenization (e.g., Image-GS [2], Pixel-to-Gaussian [3], GPSToken [4]). HerKry adopts this feed-forward representation paradigm to decouple representation density from physical sinogram resolution. We will revise the Introduction and Related Work to make this distinction explicit up front.

As for the comparison between HerKry and R²-Gaussian, both paradigms have distinct merits. Unsupervised per-scene optimization requires no paired dataset for pretraining, whereas our supervised unrolling leverages learned anatomical priors and avoids costly per-scan optimization during inference. We evaluated R²-Gaussian (30,000-step 3D optimization per patient) against HerKry (2D feed-forward inference) and FBP across 10 patient cases in the AAPM dataset on an RTX A6000 GPU:

SplitMethodParadigm16-View (PSNR ↑ / SSIM ↑)32-View (PSNR ↑ / SSIM ↑)64-View (PSNR ↑ / SSIM ↑)Mean Time / Slice ↓
Chest / Abdomen (ID)FBPAnalytical16.66 / 0.385220.12 / 0.479724.81 / 0.61600.06 ms
R²-GaussianPer-scene Unsupervised22.64 / 0.629026.75 / 0.751828.88 / 0.812612,099 ms
HerKry (Ours)Supervised Unrolling31.22 / 0.838733.58 / 0.879035.61 / 0.912856.13 ms
Cranial (OOD)FBPAnalytical21.69 / 0.506325.12 / 0.587530.96 / 0.70670.06 ms
R²-GaussianPer-scene Unsupervised29.52 / 0.867932.60 / 0.924134.85 / 0.960355,783 ms
HerKry (Ours)Supervised Unrolling39.53 / 0.966645.43 / 0.988851.16 / 0.996156.47 ms

Overall, learned HG priors allow HerKry to outperform R²-Gaussian across all sparsity levels and unseen cranial anatomy (+10.01–16.31 dB), while reconstructing slices in less than 60 ms (>200× faster) for real-time clinical deployment.


References

[1] Zha, R., Lin, T. J., Cai, Y., et al. R²-Gaussian: Rectifying radiative Gaussian splatting for tomographic reconstruction. Advances in Neural Information Processing Systems, 37:44907–44934, 2024.

[2] Zhang, Y., Li, B., Kuznetsov, A., et al. Image-GS: Content-adaptive image representation via 2D Gaussians. ACM SIGGRAPH Conference Papers, 2025. DOI: 10.1145/3721238.3730596.

[3] Peng, L., Wu, A., Li, W., et al. Pixel to Gaussian: Ultra-fast continuous super-resolution with 2D Gaussian modeling. ICLR, 2026.

[4] Zhang, Z., Wu, R., Sun, L., et al. GPSToken: Gaussian parameterized spatially-adaptive tokenization for image representation and generation. Advances in Neural Information Processing Systems, 38:12462–12493, 2025.

AuthorsOpenReview ↗

Response to Reviewer yJNp

2. How Differentiable 2D HG Splatting and CGLS Integrate with Supervised Learning "The steps of the optimization is not very clearly described. For example I am not entirely clear how the Hermite–Gaussian representation measurement fitting is combined with supervised learning. Or how the gradient in CGLS plays along the neural branch."

Response:

In HerKry, the 2D Hermite-Gaussian (HG) representation serves as an explicit, differentiable medium bridging the neural latent space and the physical measurement space:

  1. Differentiable Rendering: Similar to feed-forward 2D Gaussian networks [2–4], the 2D HG splatting operator Bl(Θl)\mathcal{B}_l(\Theta_l) is fully differentiable with respect to both geometric parameters Θl={μi,Σi}\Theta_l=\{\mu_i, \Sigma_i\} and coefficients ala_l.
  2. Differentiable Inner Solver: Within each block, the neural backbone predicts Θl\Theta_l and initial coefficients al0a_l^0. Because direct network predictions inevitably contain projection errors on observed views, we freeze Θl\Theta_l and run KK steps of HC-CGLS to linearly adjust ala_l to fit the sinogram ysy_s. Integrating a classical iterative solver inside an end-to-end network is the hallmark of Deep Unfolding Networks (DUNs) [5].
  3. Gradient Flow: Because Θl\Theta_l is fixed during HC-CGLS, all KK solver steps consist purely of differentiable linear matrix-vector multiplications (Pl=ABl(Θl)\mathcal{P}_l = \mathcal{A}\mathcal{B}_l(\Theta_l) and Pl⊤\mathcal{P}_l^\top). During training, end-to-end L1L_1 supervision gradients flow seamlessly back through the unrolled CGLS steps and splatting operator to train the neural backbone.

3. Scanner Geometry Specificity "In addition, as it combines reconstruction with image-to-image supervised learning, the trained network would only work for that specific geometry used for reconstruction. I am not clear if the training is geometry specific or it trains with different geometries."

Response:

We fully acknowledge that complete invariance to arbitrary scanning geometries is a unique strength of unsupervised per-scene optimization. Like virtually all learnable reconstruction and deep unfolding frameworks [6–8], HerKry is trained under a specific scanning geometry class (e.g., a given fan-beam source-to-detector configuration). However, in clinical practice, commercial helical CT scanners adhere to standardized acquisition geometries (as reflected in our AAPM [9] and real-sinogram LDCT Grand Challenge [10] benchmarks), making geometry-specific training standard and practical for clinical deployment.

Furthermore, HerKry achieves zero-shot generalization across view sparsities. As we jointly train HerKry across multiple sparse-view settings (16, 32, and 64 views) and A\mathcal{A} is injected modularly into HC-CGLS, a single trained model generalizes smoothly across 8 to 128 views without retraining (Appendix Fig. 9).


References

[6] Zhang, Yi, et al. "LEARN++: Recurrent dual-domain reconstruction network for compressed sensing CT." IEEE Transactions on Radiation and Plasma Medical Sciences 7.2 (2023): 132–142.

[7] Zhang, Jianjia, et al. "Trustworthy limited data CT reconstruction using progressive artifact image learning." IEEE Transactions on Image Processing 34 (2025): 1163–1178.

[8] Ma, Chenglong, et al. "Universal pre-training for generalizable incomplete-view CT reconstruction." Pattern Recognition 178 (2026): 113513.

[9] Moen, Taylor R., et al. "Low-dose CT image and projection dataset." Medical Physics 48.2 (2021): 902–911.

[10] McCollough, Cynthia H., et al. "Low-dose CT for the detection and classification of metastatic liver lesions: Results of the 2016 Low Dose CT Grand Challenge." Medical Physics 44.10 (2017): e339–e352.

AuthorsOpenReview ↗

Response to Reviewer yJNp

4. Why Hermite-Gaussian (HG) Modeling Matters "My feelings is that the paper lacks detailed experiments to understand why this Hermite–Gaussian modeling matters that much."

Response:

Our motivation for HG modeling follows a strict causal chain, each step backed by dedicated experiments in the paper:

  1. Why a Hierarchy? To regularize the vast null space in full-resolution reconstruction and improve OOD stability (validated in Table 1).

  2. Why Continuous Gaussians over Pixels? Discrete pixel-sinogram pyramids violate fan-beam geometry (DyAfine≠AcoarseDx\mathcal{D}_y\mathcal{A}^{\text{fine}} \neq \mathcal{A}^{\text{coarse}}\mathcal{D}_x), verified theoretically and empirically in Fig. 1b, Appendix B, Fig. 4, and Fig. 6.

  3. Why Hermite-Gaussian over Standard 2D Gaussians (d=0d=0)? Standard 2D Gaussians lack expressiveness when sparse, and updating their nonlinear coordinates via Adam is computationally prohibitive. HG modes can absorb local deformations linearly under frozen geometry.

    Experiments: The controlled deformation experiments (Appendix C, Fig. 5 & Fig. 7) prove that under fixed centers θi\theta_i, linear HG coefficients reduce translation fitting errors from >45% (d=0) to <2% (d=3), while substantially suppressing stretch and shear errors; The solver speed and fidelity experiments (Table 2a & Appendix Table 5) show that linear HC-CGLS slashes DC latency by 89.5%, while beating full-parameter Adam by +1.23 dB and beating 10K-step Adam at 2313 ms by +0.75 dB; The stage-wise ablation experiment (Appendix Table 4) demonstrates a +2.48 dB gain over d=0d=0 Gaussians at the sparse Stage 1.


5. Why Inaccurate Coarse Geometry Harms Reconstruction Despite a 1%–3% Global Mismatch "What would be happen if you use the coarse representation from inaccurate geometry downsampling anyway? Would it really harm the performance? The mismatch percentage in Figure 1 b seems to be acceptable? And the coarse representation is not supposed to reconstruct perfect details anyway."

Response:

While a coarse representation does not need fine details, it must avoid structural bias and iterative error accumulation:

  1. Fig. 1b reports the 1%–3% global normalized L2L_2 sinogram norm, which averages out severe localized discrepancies along high-contrast edges. As visualized in Appendix Fig. 6, even in a single-step noise-free full-view FBP reconstruction, this geometric mismatch produces localized boundary errors approaching 100 HU, which is substantial given that clinical soft-tissue or brain diagnostic windows span only 80–400 HU.
  2. In an iterative solver like CGLS, the forward and adjoint operators (A,A†)(\mathcal{A}, \mathcal{A}^\dagger) are applied repeatedly. Because tomographic inversion is ill-conditioned, small operator mismatches accumulate serially across iterations. To isolate this effect without neural network interference, we evaluated pure multi-step image-space CGLS on full-view coarse pyramids at scale k=8k=8 (32232^2 grid, matching HerKry's Stage 1 resolution) across all AAPM slices using detector-pooled sinograms:
SplitSolver StepSinogram Residual ↓PSNR (dB) ↑
Chest / Abdomen (ID)FBP (Step 0)2.29%29.28
CGLS Step 80.76%28.03
Cranial (OOD)FBP (Step 0)2.48%31.90
CGLS Step 80.79%30.54

Although CGLS steadily reduces the average sinogram residual, fitting mismatched measurements degrades image PSNR fidelity. Eliminating this bias via continuous HG projections is therefore essential for accurate subsequent refinement.

Reviewer_UMqHOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The authors identify that image and sinogram downsampling do not generally commute with fan-beam projection. To address this mismatch, they propose a progressive Hermite-Gaussian hierarchy that preserves structural details while keeping the data consistent with the original measurements. In addition, they propose HC-CGLS and KT-GRU to reduce computational overhead.

Critical strengths
  1. Clear motivation and methodological design. This work addresses a well-motivated issue, as the geometric mismatch is introduced by the naive image sinogram downsampling in coarse-to-fine reconstruction (i.e., Eq.1). The proposed Hermite-Gaussian hierarchy uses multiple basis functions to preserve structural detail while progressively refining the representation.
  2. The method achieves clear reconstruction improvements. For instance, the proposed method achieves impressive performance gains in OOD settings, which support the authors’ claim. The ablation studies are comprehensive. In particular, Tab.4 demonstrates the benefits of HG bases over vanilla Gaussians.
Critical weaknesses

The method section is somewhat difficult to follow, as it introduces the basis functions before clearly establishing the overall framework. Since Figure 2 already provides an overview, following its information flow and introducing each component step by step would make the presentation easier to understand.

Questions for authors
  1. Since higher-order HG introduces more coefficients, would the advantage persist against Gaussian with a matched coefficient budget?
2 comments
AuthorsOpenReview ↗

Response to Reviewer UMqH

We sincerely thank Reviewer UMqH for the positive assessment and insightful suggestions. Below we quote your comments in full and address them point by point.


1. Restructuring Section 3 to Follow the Information Flow of Figure 2 "The method section is somewhat difficult to follow, as it introduces the basis functions before clearly establishing the overall framework. Since Figure 2 already provides an overview, following its information flow and introducing each component step by step would make the presentation easier to understand."

Response:

We completely agree. In the revised manuscript, we have reorganized Section 3 to strictly mirror the step-by-step information flow of Figure 2:

  1. Overall Iterative Framework & State Propagation (Sec. 3.1): We first establish the unrolled multi-stage pipeline and order-density schedule (Fig. 2a), explicitly defining the input FBP initialization x0visx_0^{\text{vis}} and the three persistent states propagated across blocks: the visual state xl−1visx_{l-1}^{\text{vis}}, coefficient-space memory hl−1Kh_{l-1}^K, and pixel-space residual state Ul−1U_{l-1} (Fig. 2b).
  2. Neural Prior Synthesis and 2D Hermite-Gaussian Splatting (Sec. 3.2): Following the top branch of Fig. 2b, we first describe how the backbone predicts primitive geometries Θl\Theta_l and coefficients (Wl,al0)(W_l, a_l^0), and immediately introduce the mathematical formulation of the 2D anisotropic Hermite-Gaussian bases right where channel-wise 2D HG splatting Bl(Θl)\mathcal{B}_l(\Theta_l) occurs.
  3. Linear Coefficient Optimization via HC-CGLS (Sec. 3.3): Following the bottom branch of Fig. 2b, we explain how freezing Θl\Theta_l enables the composite forward operator Pl=ABl(Θl)\mathcal{P}_l = \mathcal{A}\mathcal{B}_l(\Theta_l) to solve physical data consistency linearly against native sinograms ysy_s.
  4. Krylov Trajectory Modeling (Sec. 3.4): Finally, following Fig. 2c, we detail how KT-GRU and spatial residual encoding summarize the intermediate CGLS iterates {alk}k=1K\{a_l^k\}_{k=1}^K into (hlK,Ul)(h_l^K, U_l) for the next block.

2. Comparison Against Standard Gaussians Under a Matched Coefficient Budget "1. Since higher-order HG introduces more coefficients, would the advantage persist against Gaussian with a matched coefficient budget?"

Response:

Yes, the advantage persists. Across our four stages, HerKry uses 322×16+642×9+1282×4+2562×1=184,32032^2{\times}16 + 64^2{\times}9 + 128^2{\times}4 + 256^2{\times}1 = \mathbf{184,320} linear coefficients. To match this budget with standard d=0d=0 Gaussians, we set Nl=2562N_l = 256^2 across all stages (262,144262,144 coefficients):

Model Variant (32-View AAPM)Spatial Schedule NlN_lBasis Order dld_lTotal Linear CoeffsID PSNR ↑\uparrowCranial OOD PSNR ↑\uparrow
Gaussian-0 (Matched NlN_l, Table 4)322→642→1282→256232^2 \to 64^2 \to 128^2 \to 256^20→0→0→00 \to 0 \to 0 \to 087,04037.27 dB44.88 dB
Gaussian-0 (Matched / Excess Coeffs)2562→2562→2562→2562256^2 \to 256^2 \to 256^2 \to 256^20→0→0→00 \to 0 \to 0 \to 0262,14437.30 dB43.51 dB
HerKry HG Hierarchy (Ours)322→642→1282→256232^2 \to 64^2 \to 128^2 \to 256^23→2→1→03 \to 2 \to 1 \to 0184,32037.58 dB45.63 dB

Why coefficient-matched d=0d=0 Gaussians underperform especially on OOD:

  1. Degeneration to Excessive Degrees-of-Freedom (DOF) Pixel Regime: Deploying 2562256^2 dense Gaussians on a 256×256256{\times}256 grid across all stages collapses the representation into the unconstrained, excessive-DOF regime illustrated in Fig. 1a. Without coarse spatial regularization, early blocks overfit sparse-view streak artifacts in the projection null space, performing even worse in generalization than the coarse-to-fine d=0d=0 schedule despite using 3×3\times more coefficients, and causing OOD PSNR to drop by 2.12 dB.
  2. Loss of Linear Deformation Expressiveness in HC-CGLS: Under frozen geometry Θl\Theta_l, d=0d=0 Gaussians act as static blobs that can only scale scalar amplitudes, whereas higher-order HG modes span local Taylor derivatives (Appendix C) to absorb translations in linear coefficient space. Without this property, HC-CGLS cannot fully leverage linear solving to rectify structural misalignments within a compact, low-latency subspace, instead degenerating into over-parameterized intensity fitting.
AuthorsOpenReview ↗

Official Comment by Authors

We trust that these clarifications resolve your concerns. Should any part of our explanation remain ambiguous or warrant further revision, we warmly welcome your feedback and would be happy to discuss any remaining points.

Comments on the paper

1 comment

Dear Area Chair,

We sincerely thank you and all three reviewers (AK7f, UMqH, yJNp) for the constructive feedback. All reviewers recognized the clear motivation of identifying the non-commutative geometric mismatch in multiscale CT and the strong empirical performance of HerKry. Below, we briefly summarize our core contributions and the targeted revisions made in the updated manuscript.

Core Highlights of HerKry

  • Problem & Hermite-Gaussian Representation: We formalize and empirically verify that discrete image/sinogram downsampling does not commute with divergent fan-beam projection (DyAfine≠AcoarseDx\mathcal{D}_y\mathcal{A}^{\mathrm{fine}} \neq \mathcal{A}^{\mathrm{coarse}}\mathcal{D}_x). HerKry resolves this via a continuous Hermite–Gaussian (HG) hierarchy that decouples representation resolution from measurement scale, enabling geometry-consistent data consistency (DC) directly against native sinograms across all stages.
  • Linearized DC & Trajectory Modeling: Exploiting the property that higher-order HG modes linearly span local Gaussian spatial derivatives, HerKry freezes primitive geometries during DC and solves a convex linear problem via Hermite-Coefficient Conjugate Gradient Least Square (HG-CGLS), slashing DC latency by 89.5% compared with full-parameter Adam. We further introduce KT-GRU to recurrently model inner-loop Krylov iterates across both coefficient and pixel spaces.
  • Robustness: Across AAPM and real-sinogram LDCT Grand Challenge benchmarks, HerKry achieves state-of-the-art fidelity and out-of-distribution (OOD) robustness, outperforming competing baselines by up to +4.17 dB on zero-shot cranial OOD transfer.

Summary of Revisions (Rebuttal Updates in Blue)

  1. Restructured Method & Clarified Boundaries (Addressing UMqH, yJNp, AK7f):
    • Reorganized Section 3 to strictly follow the step-by-step information flow of Figure 2, improving readability.
    • Explicitly clarified our supervised feed-forward 2D representation paradigm (vs. unsupervised per-scene 3DGS), end-to-end gradient flow through differentiable HG splatting and unrolled HC-CGLS (Sec. 1, 3.3), scanner geometry specificity with zero-shot view-sparsity generalization (Sec. 4.1), and the exact novelty boundaries of 2D anisotropic HG splatting and KT-GRU relative to prior Hermite-filter and black-box CG unfolding works (Sec. 3.4, Appendix A).
  2. New Targeted Experiments & Ablations (Addressing UMqH, yJNp):
    • Matched Coefficient Budget & Flat Fine-Only Ablation (New Table 3a, Sec. 4.3): Replacing the HG hierarchy with a flat fine-only d=0d=0 Gaussian schedule drops cranial OOD PSNR by 2.12 dB and nearly triples latency (163.90163.90 vs. 56.29 ms56.29\text{ ms}).
    • Decoupled OOD Mechanism Ablation (New Table 3b, Sec. 4.3): Removing HC-CGLS drops cranial OOD PSNR by 1.57 dB and more than doubles OOD sinogram error (0.29→0.610.29 \to 0.61), while pixel-grid CGLS without the HG hierarchy lags by 1.12 dB, confirming both components are indispensable.
    • Iterative Bias of Coarse Pyramids (New Table 4, Appendix B): Pure image-space CGLS on coarse detector-pooled pyramids (32232^2) degrades image PSNR by >1.2 dB across iterations despite lowering sinogram residuals.
    • Comparison with Per-Scene Unsupervised 3DGS (New Table 7, Appendix F): Across 10 AAPM patients, HerKry outperforms R2\text{R}^2-Gaussian by +6.73 to +16.31 dB while running >200$\times$ faster.
  3. Code Availability (Addressing yJNp): Refreshed and verified the anonymous code repository link.

We believe these revisions fully resolve all reviewer concerns and substantially strengthen the clarity and rigor of the paper. Thank you again for your time and consideration.

Once again, we sincerely thank you for your invaluable feedback and support in improving our work.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.