acceptodds
Under review as a conference paper at ICLR 2027

Learning Canonical Coordinate Systems for Deformable Organs: A Common Coordinate Framework for the Mouse Intestine

Abstract

Whole-organ tissue clearing and volumetric confocal microscopy have enabled cellular-resolution imaging of intact biological organs, generating multi-terabyte datasets that require new computational methods for standardized spatial analysis. Unlike the brain, the intestine exhibits substantial nonlinear deformation caused by tissue stretching, folding, and specimen preparation, preventing reliable anatomical correspondence across animals and limiting quantitative analysis of the enteric nervous system (ENS). Existing image registration approaches rely primarily on intensity matching or handcrafted landmarks and fail to preserve the hierarchical organization of intestinal tissue layers. Here, we present GutCCF, a learning- based Common Coordinate Framework (CCF) for the mouse intestine, formulating canonical coordinate learning as multi-task representation learning for deformable organs. We define the canonical space of the intestine as an intrinsic cylindrical parameterization C = U × V × W , where U , V , and W denote normalized longitudinal arc length, normalized circumferential position, and transmural depth (ordered tissue segment hierarchy), respectively. The CCF is a mapping Φ : Ω → C from a specimen-dependent image domain to a shared coordinate identity. Our multi-head 2.5D U-Net jointly predicts semantic tissue identity s(x) via cross-entropy and a continuous depth field d(x) via an RMSE objective. This depth-aware monotonic layer localization imposes an ordinal structure on the shared latent representation. Encoding each voxel’s position requires confidence within the transmural hierarchy, yielding features that remain anatomically ordered under arbitrary tissue deformation and enabling segmentation and coordinate assignments to reinforce one another. The model is further refined through an iterative human-in-the-loop protocol that incorporates expert corrections into training to improve boundary localization and structural consistency. GutCCF is trained and evaluated using more than 150 high-resolution confocal image volumes of optically cleared intestines, including jejunum, ileum, and colon, from 71 transgenic mice containing seven genetically defined enteric neuron subtypes. Image datasets are at 0.863 μm isotropic resolution, spanning specimen lengths of 0.5–30 mm and between 200 and 700 optical sections. The training dataset contains over 2487 expertly annotated image-mask pairs with hierarchical tissue annotations including longitudinal muscle, circular muscle, submucosa, glands, villi, and out-of-tissue regions. The resulting coordinate system assigns every voxel to a continuous canonical anatomical space, enabling standardized localization of neuronal morphology, tissue structures, and multimodal biological measurements. Experimental results demonstrate that jointly learning anatomical depth and semantic segmentation significantly improves tissue boundary localization, spatial consistency, and cross-sample correspondence compared with segmentation-only baselines, achieving 82.48% mean Dice, 70.74% mIoU, 81.85% recall, and 83.28% overall precision. The proposed framework produces a high-resolution three-dimensional reference representation of the mouse intestine and enteric nervous system and provides a scalable foundation for multimodal spatial atlases, computational anatomy, and organ-scale digital twins.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Enteric nervous system, Tissue segmentation, Gut atlas, Common coordinate framework, Deep learning, Spatial registration, gut twin

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_r8hUOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework for Gut segmentation based in a centerline parameterisation. The authors propose to learn a depth coordinate (measured as distance from the centerline) as a diffeomorphism-invariant ordered semantic segmentations where W is separated in fixed segments b1<b2<b3... and the network predicts the depth d.

Critical strengths

None

Critical weaknesses

The experimental section is a bit of a mess. It is unclear what methods are implemented, what is compared and under which conditions. Table 1 compares a few models but includes for eaxmple lines like "Consistency Loss" or "Uncertainty Loss" which are not introduced in the paper (indeed "Uncertainty Loss" is only used in Table 1). It is unclear why some models are reported with precision/recall, and some without. human-in-the-loop training is just reported as helpful but it is unclear which model, how many rounds, and whether all models shown have been trained with human-in-the-loop, equally.

I hoped the ablation study would resolve some of the issues, but it turns out that it is not a systematic check of removing components of a model to see their effects, but instead it seems to be an unsystematic comparison of different methods. I do not know why the numbers in Table 4 differ so significantly from Table 1, and if training settings are just short runs, then it is unclear to me how training settings in which all models perform that poorly should guide model-selection.

Beyond this, i have concerns about the methodology and novelty. U,V,W maps for gut coordinate systems by themselves are not novel. the authors propose a new way to encode the depth coordinate but they never disclose how the depth labels b_i are derived and computed and never discuss the results.

While RMSE on d is reported numerically, it is quite large and the authors report as the numbers of "their" model (DICE, precision etc) the "Cross-entropy + RMSE" model which performs slightly better in DICE score but is significantly worse in RMSE (0.22 vs 0.08 in the comparison model). This choice of final model is surprising, since the W-encoding is the core novelty and the authors decided to not even discuss the differences in RMSE as if the quality of the encoded depth variable does not matter at all. Note that RMSE ~0.29 is the baseline of predicting d=0.5, so the chosen model is only a little better than predicting the average.

Eq (5) is completely nebulous because it uses b_{s(x)} where s(x) is a centerline location in 3D space by eq (2), but b_i is indixed with an ordinal. Later on a new variable s is introduced which is the classification label, so i assume that this s is meant.

Questions for authors

Can you discuss the reported RMSE values and their importance to your work?

1 comment
AuthorsOpenReview ↗

Thank you to all reviewers for their thoughtful feedback and for helping improve the quality of our paper. In response to the reviewers, we have incorporated extensive additional experiments and made considerable revisions during the rebuttal period. We believe the manuscript has become substantially stronger as a result. We also want to assure the committee that we strictly followed all official ICLR guidelines throughout the rebuttal process.

Response to Reviewer (Critical Weakness): We thank the reviewer and have reorganized the experimental section to make the distinct components of this study (Tissue Segmentation, Statistical Analysis, and Registration/ Landmark Alignment) more clear and make all comparisons and training conditions explicit. In the revised manuscript, we have strived to use consistent terms to define our analyses and thank the reviewer for pointing out our inconsistencies. The revised tables 4 and 5 now separate architecture comparisons from controlled ablations, and all rows are evaluated using the same data split, preprocessing, checkpoint selection, and metric implementation. We also revised the previous HITL description.

Updated table 4 shows loss formulation and uncertainty weighting in the proposed 2.5D framework, and added table 5 depicts ablation of spatial dimensionality, auxiliary depth supervision, and HITL-based data refinement. We also resolved the previous discrepancy between Tables 1 and 2; identical models now report identical results across tables. Short-run experiments are no longer used for final model selection. We agree that the use of a (U,V,W) coordinate system itself is not novel. We have revised the manuscript to make this distinction explicit. In GutCCF, U and V are derived from the known imaging/acquisition geometry, while the methodological contribution is the anatomically supervised prediction of the continuous transmural coordinate W, followed by cross-specimen registration to construct a standardized mouse ileum CCF. We also added the previously missing description of the depth targets b_i. The targets are generated directly from expert anatomical masks by assigning the ordered serosa-to-lumen classes the normalized values b_i=(0,0.1,0.3,0.5,0.7,0.9,1.0). The depth head predicts a global normalized transmural position, W ∈ [0,1]. This auxiliary objective encourages the network to learn the ordered anatomical organization of the intestinal wall in addition to discrete semantic labels. The revised manuscript now also reports quantitative results for this coordinate prediction. The presented results provide direct evidence that the learned depth representation captures anatomically meaningful transmural position and supports registration to the population-derived CCF.We also clarify that the CE+RMSE objective does not mathematically guarantee monotonicity, continuity, or a non-folding coordinate map, instead anatomical ordering and boundary consistency are evaluated empirically.

During revision, we re-evaluated the held-out test set using a consistent evaluation pipeline and corrected Table 1. The previously reported Dice value of 0.8248 corresponded to the Macro-F1 score; the corrected test Dice is 0.6241. We also corrected the depth evaluation, with the proposed model achieving a Test Depth RMSE of 0.0806, substantially lower than the previously reported value and the ~0.29 constant-depth baseline noted by the reviewer. The revised table now clearly distinguishes Dice, Precision, Recall, Macro-F1, and Test Depth RMSE. We have updated table 2 (updated table 4) and added table 5 in the manuscript.

The reviewer is correct: Eq. (5) is intended to use the predicted tissue-class index, not the centerline arc-length variable used in Eq. (2). Reusing s(x) for these two quantities makes the current formulation unnecessarily ambiguous. We have corrected the notation throughout the manuscript by reserving s(x) exclusively for centerline arc length. Equation (5) has been rewritten. We have also revised the surrounding text before Eq. (5), and use this notation consistently in the model description and coordinate-construction sections. This is a notation error in the original manuscript rather than a change to the underlying formulation, and we appreciate the reviewer for identifying it.

Response to Reviewer (Questions For Authors): We thank the reviewer for this question. RMSE is a key metric in our work because it directly measures the accuracy of the predicted transmural coordinate (W), normalized from serosa (W=0) to the luminal/villus surface (W=1), while penalizing larger depth errors more strongly [updated manuscript, table 1, Line 433]. The selected CE+RMSE model achieves a validation depth RMSE of 0.0806, corresponding to approximately 8% of the normalized wall depth and substantially outperforming the (\sim0.29) constant-depth baseline. On 94 held-out test images, the class-derived depth analysis yields a mean per-image RMSE of 0.0809 and MAE of 0.0467. These results indicate that the predicted (W)-coordinate provides meaningful transmural localization and is therefore directly relevant to the accuracy of the proposed ileum CCF.

Reviewer_JjC5OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GutCCF aims to provide a common anatomical coordinate system for the mouse intestine from large confocal microscopy volumes. The proposed coordinates comprise normalized longitudinal arc length U, periodic circumferential position V, and layer-aware transmural depth W. A shared 2.5D U-Net predicts semantic tissue labels and a continuous depth field through cross-entropy and RMSE supervision. These outputs are combined with geometric processing, specimen metadata, and human-in-the-loop corrections to support tissue flattening and a proposed population reference representation.

The manuscript describes more than 150 volumes from 71 mice, with 2,487 annotated slices, and reports mean Dice of 0.8248, mIoU of 0.7074, and depth RMSE of 0.2253 for the selected model. It compares several segmentation architectures and training variants and shows a qualitative flattening example. The directly demonstrated result is a tissue segmentation/depth prediction pipeline; the stronger claim of validated cross-specimen canonical correspondence remains insufficiently established.

Critical strengths
  1. An important application with potentially valuable imaging and annotation resources. Standardized localization of enteric-neuron morphology in highly deformed intestinal tissue is a meaningful problem. The reported acquisition effort across 71 mice, multiple intestinal regions, labeling conditions, and two microscope platforms, together with expert layer annotations, could support a useful computational anatomy resource. The emphasis on weak background autofluorescence also identifies a concrete segmentation challenge that generic intensity matching may not resolve reliably.

  2. A plausible anatomy-aware segmentation strategy with a positive reported performance signal. Combining neighboring-slice context, shared semantic/depth representations, and expert corrections is sensible for layered tissue. Table 1 reports Dice of 0.8248 for the selected multi-head model versus 0.6487 for the 2D U-Net and 0.7277 for the 3D CNN. These results motivate investigating the approach further, although they do not isolate the contribution of depth supervision or establish canonical-coordinate accuracy.

Critical weaknesses
  1. The central canonical-correspondence contribution is neither operationally complete nor directly validated. Normalizing distance between each specimen's endpoints does not by itself identify the same anatomical longitudinal position across differently located or differently stretched segments. The angular coordinate in Eq. (4) requires a shared orientation, angular origin, and frame along the centerline; these are not defined, especially for tissue that has been cut open and mounted flat. Nonuniform longitudinal stretching and circumferential deformation are not removed merely by normalized arc length and local angle. The coarse-to-fine registration mentioned in Sec. 3.3 is not specified. The W construction is similarly ambiguous: Eq. (5) needs within-layer depth and canonical layer boundaries, while the network is also described as predicting global wall depth; target construction and the boundary values are absent. Cross-entropy plus pointwise RMSE can encourage ordered predictions but does not impose spatial monotonicity, continuity, or a nonfolding coordinate map. The paper says it measures layer-order violations, yet reports no corresponding numbers. Segmentation overlap and Figure 5's flattening visualization cannot establish cross-animal correspondence. The only reported downstream composition score, 0.513±0.249 with n=2, is not defined sufficiently and does not measure pointwise coordinate agreement. This gap directly affects the claimed framework and population atlas.

  2. The segmentation experiments do not establish which component is responsible for the improvement or whether it generalizes to independent animals. The selected method differs from the 2D U-Net in slice context, task heads, supervision, and potentially expert refinement. Table 2 does not provide an unambiguous, architecture-matched 2.5D segmentation-only control or a quantified HITL comparison; its “Retraining + CE loss” row still has a depth RMSE. The 2D scores also differ substantially between Tables 1 and 2 (0.6487 versus 0.4778) without an explained protocol difference. Table 1's focal-loss variant has much lower depth RMSE (0.0852 versus 0.2253), which is not reconciled with selecting the other model for coordinate accuracy. No mouse-level split counts, uncertainty across animals/seeds, or isolation of HITL correction data from the final test set are supplied; with many correlated slices and overlapping patches, this matters substantially. These omissions do not prove leakage, but prevent assessing independent generalization. Auxiliary geometric regression is already established in segmentation, for example in Navarro et al., Shape-Aware Complementary-Task Learning, 2019, so the anatomical-coordinate claim and controlled evidence must carry the methodological contribution rather than the presence of a second head alone.

Questions for authors
  1. Operational coordinate construction: Please specify the longitudinal anatomical anchor and specimen metadata used to align segments, how V's angular zero and cross-sectional frame are fixed across animals, how cut-open specimens are related to the cylindrical domain, and the full coarse-to-fine registration procedure. What direct landmark/correspondence errors or deformation-consistency results demonstrate that homologous locations receive comparable U/V/W values, rather than merely locally normalized coordinates?

  2. Depth target and ordering: How are continuous depth labels generated from the expert masks? Is the predicted depth within-layer or global, what are the canonical boundary values b_k, and how are background and absent/region-specific compartments handled? Please distinguish raw learned ordering from any subsequent denoising, and report measured layer-order violations, boundary accuracy, and coordinate continuity before and after that processing. The focal-loss versus selected-model RMSE difference also needs interpretation in the coordinate task.

  3. Independent evaluation and HITL: How many mice, volumes, slices, and patches are in each split, and is splitting performed at the mouse level before patch extraction? Which specimens and expert corrections enter each HITL round, and is the final test set untouched? Please provide animal-level uncertainty and per-region results, along with HITL annotation effort and improvement measured on a held-out set. Training/validation convergence alone does not establish these properties.

  4. Matched comparisons and scope of evidence: Please include the same 2.5D architecture trained without the depth objective, a matched auxiliary geometric-target baseline, and a comparison before/after HITL under the same data protocol. Explain all rows in Table 2 and its differing 2D baseline. Please also define the part-composition metric, what n=2 counts, and whether additional correspondence, layer-order, or population-atlas results were omitted. A new application can be valuable, but the current segmentation metrics do not substantiate the claimed general-purpose canonical framework.

1 comment
AuthorsOpenReview ↗

We thank the reviewer for the feedback. We clarified the coordinate framework, standardized model comparisons, revised the HITL description, removed the part-composition metric, and added direct validation of depth accuracy, layer ordering, deformation, and cross-specimen CCF registration.

  1. We thank the reviewer for this important comment and have revised the manuscript to more explicitly evaluate the canonical coordinate representation across specimens with diverse biological, staining, preparation, and imaging conditions. We clarify that U and V are initialized from the physical imaging/acquisition geometry, whereas W is predicted as the normalized transmural depth, providing a voxel-wise (U,V,W) assignment prior to registration. For flat-mounted tissue, the acquired specimen is treated as an unrolled tubular surface. Cross-specimen alignment is then established geometrically using the spatial distribution of villus center points as distributed anatomical landmarks, which anchor the specimen to the population-derived ileum CCF and compensate for local stretching and deformation. We have revised Sec. 3.3 of the methods to explicitly describe the registration procedure and the construction of the W targets. The network jointly predicts anatomical layer labels and global normalized wall depth. We stated as a limitation of the current formulation. Across the available ileum specimens, the mean boundary waviness was approximately 31.2 ± 5.0 µm, with greater geometric variability observed toward the submucosal and glandular boundaries. This is now explicitly stated in the Depth Prediction sections of the results. New analysis with updated results indicate that the predicted W-coordinate from unseen specimen is quantitatively accurate and anatomically monotonic.
  2. We thank the reviewer for requesting clarification and have revised the manuscript to make this clearer. The depth head predicts a global normalized transmural coordinate (W), rather than within-layer depth. Depth supervision is generated from the expert layer masks by assigning the ordered anatomical classes canonical normalized values (Supplementary S2). We now distinguish the raw model output from any subsequent registration/processing and quantify its anatomical consistency directly. Finally, the revised loss ablation removes the previously reported RMSE discrepancy: the resut supports selection of CE+RMSE for the final coordinate model [updated manuscript, table 2, line 486]. To quantify tissue deformation independently of the normalized coordinate system, we measure each anatomical boundary in absolute (Z)-space using the physical voxel resolution. For each (U,V) column, the serosal and luminal surfaces define the local tissue thickness, while the deviation of each layer boundary from its local fitted surface provides a physical measure of boundary waviness in µm. Under the revised unified evaluation protocol, the focal-loss model. The previously observed focal-loss/RMSE discrepancy was due to short runs and reporting error and is no longer present after re-evaluation under the same protocol.
  3. We thank the reviewer for this comment and have substantially revised the ablation analysis (updated manuscript, table 2 and table 3). We have revised the manuscript to be clear about how many datasets and tissue specimens were used for each specific analysis. All segmentation results are now evaluated on the same held out 94-image, 7-class (5 layers, out of tissue top, and bottom) test set, [Supplementary, table S1] resolving the previous inconsistency in the reported 2D U-Net scores and updated in the manuscript. The HITL refinement now refers to retraining the model on expert-curated data focusing on targeted correction of model outputs. Slices were pooled across all animals and partitioned into training, validation, and test sets at the individual-slice level by a fixed-seed (seed = 42) random shuffle. Finally, cross-specimen CCF consistency is evaluated independently on an unseen ileum specimen.. The specimen image is registered using landmark anchors as described in the new Registration/ Landmark Alignment section of the results in the manuscript, defining the applicability of our proposed CCF.
  4. We thank the reviewer for this important comment. We revised the ablation study to provide matched comparisons under the same seven-class data protocol and 94-image held-out test set, including the 2.5D architecture without depth supervision, the corresponding model with the auxiliary (W)-depth objective, and expert-curated retraining. We also added the auxiliary geometric-target comparison and clarified each row in Table 2. The previously inconsistent 2D baseline values resulted from different evaluation settings; all overlapping models have now been re-evaluated using the same data split, preprocessing, checkpoint selection, and metric implementation. We have removed the previously reported part-composition score (0.513 ± 0.249, n=2) as primary evidence for CCF validation. Instead, the revised manuscript evaluates the CCF using direct anatomical measures, including depth accuracy, layer-boundary error.
Reviewer_YPcNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GutCCF, a coordinate system for the mouse intestine defined by longitudinal position, circumferential position, and tissue-layer-based depth. The depth coordinate is derived from a dual-head 2.5D U-Net that jointly predicts six tissue layers and a continuous depth map. The model is trained on 2487 expert-annotated confocal slices from 71 mice and achieves a mean Dice score of 82.48% in comparisons with several segmentation baselines. However, the coordinate system itself is evaluated only qualitatively.

Critical strengths

The paper addresses an important, underexplored problem using a rare dataset of 71 animals, with expert annotations and very low-SNR tissue images. The layer-based depth coordinate is a sound approach for deformable organs.

Critical weaknesses
  1. The abstract part is too long.
  2. Tables 1 and 2 report inconsistent results. The 2D U-Net scores 0.6487 in Table 1 but 0.4778 in Table 2, while SwinUNETR scores 0.4726 versus 0.4025. Yet the Consistency Loss result is identical (0.3413), so a different test set alone does not clearly explain these difference. Table 2 also omits the base model it needs to compare. This makes the ablation difficult to interpret and leaves claim that it "supports the central hypothesis" unsupported.
  3. The contribution highlighted in the title is not evaluated. There is no assessment of the coordinate system or cross-animal discussion. The paper’s central claim therefore remains untested. It is better to narrow this claim.
Questions for authors

Please refer to weakness part.

1 comment
AuthorsOpenReview ↗

We thank the reviewer for the constructive feedback. We shortened the abstract, corrected and standardized the experimental results, reorganized the Results section, and added new analyses of the \(W\)-coordinate, tissue statistics, and registration of an unseen specimen to strengthen validation of the GutCCF.

Response to Reviewer (Questions For Authors):

  1. Thank you for your comment. Indeed, we agree that the abstract is too long. We updated the abstract as per reviewer’s suggestion.

  2. We thank the reviewer for identifying this inconsistency. The discrepancy arose from multiple analyses and errors in carrying the final analyses into the manuscript. We have updated Tables 1 and 2 (and added table 3) to better present our results. All our reported models were evaluated using the same held-out test partition (7 class, n= 94). We have additionally incorporated more evidence for evaluating our central CCF hypothesis. Reorganization of our results sections into Tissue Segmentation, Depth Prediction, Human-in-the-Loop Refinement, Statistical Assessment of Segmented Tissues, and Registration/ Landmark Alignment makes creation and evaluation of the GutCCF more clear and hopefully improved the readability of the manuscript.

  3. We thank the reviewer for this important comment. We recognize that the original manuscript did not provide enough evaluation of the GutCCF. We have extensively revised the manuscript to clarify the canonical coordinates and evaluate their consistency across specimens with varied staining, preparation, and imaging conditions. Specifically, new analysis within the Statistical Assessment of Segmented Tissues section and the new Registration/ Landmark Alignment section provides these evaluations. Specifically, we have included a Table S1 that assesses the W coordinate using model analyses. Likewise, we now provide registration of an unseen, neuron-labeled specimen to the standardized coordinates of the GutCCF. Therefore, rather than narrowing the claim as the reviewer suggested, we now feel that the newly provided analyses support the original claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.