acceptodds
Under review as a conference paper at ICLR 2027

Beyond Token Alignment: Human-Centered Posterior Learning for Multimodal Human Sensing

Abstract

Multimodal human sensing combines heterogeneous sensors such as RGB, depth, LiDAR, mmWave radar, WiFi CSI, and RFID for robust and privacy-preserving perception. Existing methods typically align or aggregate modality tokens in a shared embedding space. However, these tokens arise from distinct physical mechanisms and are differently entangled with environment factors, sensing noise, and uncertainty. We therefore argue that the common object across modalities is the underlying human state rather than token correspondence. Based on this insight, we propose HUPER, a Human-centered uncertainty-aware Posterior Estimation and Refinement framework. HUPER preserves sensor-specific inductive biases with modality-aware encoders, estimates a probabilistic human posterior for each modality, separates human content from environment factors, and forms a reliability-calibrated posterior consensus. It further corrects residual uncertainty through an uncertainty-conditioned vector field in the human state space. Experiments on MM-Fi and XRF55 demonstrate strong performance across pose estimation, activity recognition, missing modalities, and heterogeneous RF settings. On MM-Fi HPE, HUPER-Flow reduces full-modality and partial-average MPJPE over X-Fi by 22.7% and 15.9%, respectively. It improves full-modality MM-Fi HAR accuracy by 13.3 percentage points. On XRF55 HAR, it achieves 97.1% full-modality accuracy and improves the full- and dual-modality results over X-Fi by 7.3 and 13.7 percentage points. These results establish human-centered posterior learning as a more reliable alternative to direct token alignment.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal, Human sensing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qHniOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HUPER, a multimodal human sensing framework that represents each modality as a Gaussian posterior over a shared latent human state. It combines human/environment decomposition, reliability-weighted posterior fusion, and uncertainty-conditioned flow refinement. Experiments on MM-Fi and XRF55 report improvements in pose estimation, activity recognition, and missing-modality settings.

Critical strengths
  • The paper addresses a practical problem: fusing heterogeneous sensors under missing or unreliable observations.
  • Results show substantial improvements over X-Fi in several settings. Component, refinement-target, and synthetic-corruption analyses provide useful supporting evidence.
  • The paper is written in a good manner.
Critical weaknesses
  • While the author claims a human-centered posterior learning framework, the inherent human-state interpretation is insufficiently validated. The proposed state is a learned 256-dimensional latent. Cross-modal alignment, reconstruction, and orthogonality do not establish that it excludes environmental information. Existing ablations demonstrate component utility but do not isolate semantic disentanglement from additional capacity or regularization.
  • MM-Fi uses a random split, and no dedicated cross-subject or cross-environment results are reported. The authors are encouraged to provide more dedicated results in different data splits.
  • Octonet [1] is relevant literature, yet it is never cited, discussed, or compared. Actually, Octonet provides a rich source of modalities, activity classes, and 3D pose signals. The missing comparison with this dataset would weaken the contributions and scientific conclusions.
  • Several reported results are inconsistent. XRF55 dual-modality accuracy is 91.3% in Table 6 but 93.1% in Tables 3/7/8. MM-Fi full-modality DDPM MPJPE is 65.3 mm in Tables 1/5 but 70.8 mm in Table 4. Some “All Avg.” values also disagree with the stated cardinality-balanced averaging rule.
  • The benefit of flow refinement is not sufficiently isolated. The refinement target is derived from a stronger, full-modality representation. It remains unclear whether the improvement comes from the flow formulation or from this additional supervision.

[1] Yuan, Dongsheng, et al. "OctoNet: A large-scale multi-modal dataset for human activity understanding grounded in motion-captured 3D pose labels." Advances in Neural Information Processing Systems 38 (2026).

Questions for authors

See above.

8 comments
AuthorsOpenReview ↗

Response to Reviewer qHni: Semantic Validation, Generalization, and Flow Refinement

We sincerely thank the reviewer for the careful assessment.

W1. Human-state interpretation. Thank you for highlighting the need for direct semantic validation beyond task accuracy. We define “human-centered” operationally: cc is the only latent aligned across physical observations of the same event, fused across modalities, refined, and supervised by HAR/HPE and skeleton losses; sms_m is modality-specific and never fused. To distinguish this semantics from added capacity, we use linear probes and a parameter-matched control that removes sms_m but reallocates its parameters to the content branch:

RepresentationAction Acc.Environment Acc.Modality Acc.
Baseline feature fmf_m68.564.791.2
Human posterior μm\mu_m82.131.438.6
Environment latent sms_m27.686.394.1
Refined latent z~\tilde z89.425.824.7

For the human representations (μm\mu_m, z~\tilde z), higher action and lower environment/modality accuracy are desired. The capacity-matched model without sms_m reaches 83.0 All Avg., comparable to the 83.7 of the standard variant without sms_m (Table 8) and far below 88.2 for full HUPER, showing that the gain is not explained by parameter count alone.

W2. Cross-subject/environment generalization. We appreciate this important suggestion on evaluating generalization beyond the random split. Protocol-3 random split is used in the main tables to match X-Fi and prior baselines. We additionally evaluate unchanged models on MM-Fi cross-subject and cross-environment partitions:

SplitMethodHPE MPJPE↓HAR Acc.↑
Cross-subjectX-Fi104.861.7
Cross-subjectHUPER-Flow82.674.9
Cross-environmentX-Fi112.358.9
Cross-environmentHUPER-Flow88.472.1

W3. OctoNet. Thank you for pointing us to this highly relevant benchmark. OctoNet provides synchronized heterogeneous observations together with both activity labels and motion-captured 3D poses, which matches the setting HUPER is designed for: aligning WiFi, mmWave, and vision-like depth through a shared human posterior, then serving HAR and HPE from the same refined latent. We therefore evaluate HUPER on OctoNet using the overlapping WiFi, mmWave, and depth modalities. All four multimodal subsets below are obtained from one checkpoint, without changing the architecture or retraining a specialist for each mix.

ModalityHAR Acc.↑HPE MPJPE↓
Depth+mmWave93.3145.6
WiFi+Depth98.3141.5
WiFi+mmWave95.0148.4
WiFi+mmWave+Depth98.6141.4

Two regularities follow from this table. First, HAR is strong whenever WiFi is present (95.0–98.6). Second, HPE stays in a narrow band (141.4–148.4 mm) across all four subsets: WiFi+Depth (141.5) essentially matches the three-modality result (141.4), whereas WiFi+mmWave is the weakest pose setting (148.4). Depth therefore contributes more to geometric pose than to action identity, which is consistent with reliability-calibrated fusion: a modality is weighted by how informative it is for the current human-state estimate, rather than being concatenated uniformly. The three-modality subset is best on both tasks, and the dual-modality numbers show that the same human posterior remains usable when one sensor is removed. We have also cited OctoNet in the revised related work.

W4. Reporting consistency. We thank the reviewer for the careful numerical check and apologize for the ambiguity it revealed. The correct XRF55 dual accuracy is 91.3%, confirmed over five runs; 93.1% was a transcription error and has been corrected throughout. The DDPM values refer to different settings: 65.3 mm in Tables 1/5 is the separately optimized final HUPER-DDPM model, whereas 70.8 mm in Table 4 is the controlled residual-refiner replacement using the shared Flow ablation backbone and budget. We will label this distinction explicitly in the captions of Tables 4 and 5. All aggregate columns are regenerated from subset logs under one rule: within-cardinality means for Single/Dual, all incomplete multimodal subsets for Partial, and cardinality macro-average for All.

W5. Flow vs. stronger target. We appreciate this precise observation, which helps separate the benefit of the transport objective from that of stronger supervision. To isolate architecture from target supervision, all variants below use identical source samples, detached full-modality targets, training budget, and downstream head. The parameter-matched residual MLP receives exactly the same privileged target as Flow:

Refiner (same source/target)Full MPJPE↓All Avg.↓XRF55 Full↑
No refinement73.093.496.5
Residual MLP71.890.196.6
DDPM residual70.888.796.7
Flow w/o uncertainty68.684.297.0
HUPER-Flow64.779.697.1
AuthorsOpenReview ↗

Friendly follow-up

Thank you again for recognizing the practical importance of robust heterogeneous sensing, the substantial improvements over X-Fi, and the value of our component and corruption analyses. We hope the new semantic probes, capacity-matched control, cross-subject/environment experiments, OctoNet evaluation, corrected aggregates, and matched refinement comparison address your concerns. If you have any further questions, we would be pleased to discuss them and respond promptly.

I appreciate the authors’ detailed rebuttal and additional experiments. The linear probes provide useful evidence that the content and environment branches learn different information.

My central concern, however, remains the interpretation of the “human state,” which is fundamental to the paper’s contribution. It is unclear whether this state is intended to represent instantaneous pose, motion over time, identity and body shape, or more broadly any information useful for downstream tasks. HPE and HAR require information at different temporal scales, yet the paper does not specify which factors the shared representation is meant to retain or suppress.

The rebuttal explains the intended roles of cc and sms_m through their training objectives and use in fusion. This is an operational definition, but it does not establish that the learned variables consistently follow that semantic division. The new probes support partial separation on the evaluated data; they do not yet show whether it persists for unseen subjects or environments.

Since the human-state interpretation is central to the paper’s claims, I maintain my original assessment pending stronger justification and evidence.

Reviewer_a65GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses the problem of multi-modal human action recognition and 3D pose estimation, and proposes a method for integrating heterogeneous sensing signals from multiple sensors. More specifically, the proposed method first extracts modality-specific features and then projects them into two shared spaces, each corresponding to human semantics and environmental factors. Those shared representations are the basis for predicting the reliability of each modality and for refining representations weighted by reliability. Experimental evaluations with standard benchmarks (MM-Fi and XRF55) for multi-modal human sensing demonstrate that the proposed method outperformed simple baselines and X-Fi, one of the popular existing methods in multi-modal human sensing.

Critical strengths

S1. The problem dealt with in this paper is significant. Human activity recognition (using RGB camera images) is a well-established task in computer vision. More recently, the use of different types of sensors is becoming popular, such as IR light (depth cameras, LiDAR point clouds) and electromagnetic waves (WiFi CSI and millimeter-wave point clouds). However, every sensor has disadvantages for achieving accurate activity recognition. The multi-modal approach that accounts for the significance of each sensor is promising.

S2. The concept of the proposed method is reasonable. Since the task concerns human actions and poses, using human-centered intermediate representations is reasonable for integrating heterogeneous sensing signals. Also, the proposed method, which aims to account for the reliability of each sensing signal, is technically sound.

S3. The experimental evaluations demonstrate the effectiveness of the proposed method against a standard competitor in this research problem in human action recognition and 3D pose estimation.

Critical weaknesses

W1. I could not understand the essential difference between C (human-content variables) and S (environmental variables). I could understand the concept of those variables. However, the current description contained in the main paper does not differentiate C from S. More specifically, even if we switch the order of C and S, we can still understand the current description of the main paper without any contradictions. I assume that some components of the loss function presented in Equation (9) would characterize each variable. However, we cannot find any characterizations of variables in the main content. Given this, I have to say that the proposed method belongs to the same group as existing multi-modal fusion approaches. This point is critical to justifying the novelty of the proposed method, but the current paper fails to do so. The central argument should be presented in a self-contained manner within the main text, without relying on the appendix.

W2. Similar to the first concern, Equation (6) does not make sense for the reliability score of a modality. As a procedure, there is nothing wrong with calculating prediction reliability in this way. However, I could not find any other cues for believing the validity of this computation.

Questions for authors

Q1. I could not understand from the problem setting shown in Section 3.2 whether every training example requires all synchronized signals whose modalities belong to M, the set of candidate modalities.

[Post-rebuttal comments]

I appreciate the author feedback, which makes my understanding of this paper clear. However, I also understand that the current paper heavily relies on the appendix. The central issues of a paper should be addressed in the main text, not in the appendix. Furthermore, a paper should be self-contained within the main text, and structures that require the appendix to understand the content should be strictly avoided. I have to recommend this paper should be majorly revised and resubmitted to other related conferences to increase the impact of this work.

3 comments
AuthorsOpenReview ↗

Response to Reviewer a65G: Human–Environment Decomposition and Reliability

We sincerely thank the reviewer for the thoughtful assessment, for recognizing the significance of heterogeneous human sensing, and for highlighting places where the presentation can be made more self-contained.

W1. Human content vs. environment latent. Thank you for identifying that the asymmetric roles of cc and ss were not sufficiently explicit. First, they cannot be exchanged without changing the model. Only μm\mu_m is aligned across synchronized modalities, aggregated by Eq. (11), refined by Eqs. (14–17), and supervised through the HAR/HPE and skeleton heads. In contrast, sms_m is never aligned, fused, refined, or passed to a task head; it is used only with cmc_m for feature reconstruction and reliability estimation, while LorthL_{\rm orth} limits duplication. Exchanging them would therefore redirect both the supervision and the complete inference path, rather than merely rename two latent variables.

Second, this asymmetry distinguishes HUPER from conventional multimodal fusion. Existing fusion methods primarily combine modality tokens or features and leave the shared representation implicit. HUPER instead requires every modality to estimate a posterior over the same human variable cc, fuses only these human posteriors through reliability-calibrated precision, and excludes sms_m from consensus and downstream prediction. The common space is therefore constrained by cross-modal agreement and human-task/structure supervision, rather than being an unconstrained fused feature space.

The central mechanism is already specified in the main paper: Secs. 3.2–3.3 define qm(c∣xm)q_m(c\mid x_m), sms_m, their separate heads and reconstruction path; Secs. 3.5–3.6 use only cc for consensus and refinement. The Appendix expands the individual losses and derivations rather than introducing this distinction. We nevertheless agree that grouping the losses in Eq. (9) obscured the asymmetry. Secs. 3.2–3.3 now state this asymmetry explicitly, and we will further add a compact loss-to-variable mapping to the main text in the final version, while retaining the full algebra in the Appendix. The direct probes further support the intended specialization:

RepresentationAction Acc.Environment Acc.
Human posterior μm\mu_m82.131.4
Environment latent sms_m27.686.3

W2. Reliability score. We appreciate the opportunity to clarify the rationale behind Eq. (6). It combines two complementary signals. The prior −am-a_m down-weights a broad posterior, while gmρ([fm;μm;sm])g_m^\rho([f_m;\mu_m;s_m]) captures confident-but-wrong failures not represented by variance alone, such as occlusion or multipath. The agreement target in Eq. (53) gives ρm\rho_m a concrete training meaning: expected agreement with other synchronized observations of the same human state. At inference it remains unimodal, enabling arbitrary subsets. In Eq. (10), ρm\rho_m tempers expert precision, and Appendix A.4 proves that its normalized value is exactly the modality's sensitivity in the consensus. The ablation and real-error correlation support this design:

Reliability designSingle Avg.↑Dual Avg.↑Full↑ρ\rho-error Spearman↓
Precision only (ρm=1\rho_m=1)69.887.396.2-0.41
Learned ρm\rho_m (ours)76.291.397.1-0.69

Q1. Synchronized training data. Thank you for raising this important point about the training assumption. Complete synchronization is used to construct the pretraining teachers—cross-modal alignment, leave-one-modality-out reliability targets, and the clean full-modality Flow target—but it is not required by the model at inference. Every loss is defined over the actually available set A\mathcal A, so partially observed training examples remain valid. During downstream training we explicitly sample non-empty subsets; during inference Eq. (11) accepts any A⊆M\mathcal A\subseteq M, including one modality. We will explicitly distinguish these pretraining, subset-training, and inference conditions in Section 3.2 of the final version.

AuthorsOpenReview ↗

Friendly follow-up

We greatly appreciate your recognition of the significance of heterogeneous human sensing, the reasonableness of a human-centered representation, and the technical motivation for reliability-aware fusion. We hope our clarification of the asymmetric roles of human content and environment latents, together with the probe, ablation, and training-assumption analyses, addresses your concerns. We would be very happy to discuss any remaining questions or clarify any aspect of our response.

Thank you again for your thoughtful review and for recognizing the significance of heterogeneous human sensing, the reasonableness of our human-centered formulation, and the effectiveness of the proposed method. We hope that our revisions and clarifications have addressed your concerns regarding the human/environment decomposition, reliability design, and training assumptions. If no concerns remain, we would be grateful if you could consider updating your assessment. If any issue is still unresolved, we would be happy to clarify it promptly.

Reviewer_DgNhOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes HUPER, a human-centered uncertainty-aware posterior estimation and refinement framework for multimodal human sensing. It aligns heterogeneous sensor observations through a shared latent human representation rather than direct token fusion. The framework employs modality-specific encoders to extract sensor features then decompose them into Gaussian human content representations and environment latent variables, and estimates modality reliability. It then performs precision-weighted posterior aggregation to obtain a consensus human latent representation. An uncertainty-conditioned flow-matching module refines the consensus representation with a DDPM residual correction alternative. The resulting representations are used for downstream tasks. Experiments evaluates the framework across different modality combinations and settings, reporting improved performance over conventional fusion baselines on both human pose estimation and human activity recognition tasks.

Critical strengths
  • The proposed framework supports different modality combinations and multiple downstream tasks.
  • The evaluation shows improvements over baselines on various modality combinations.
Critical weaknesses
  • The proposed probabilistic framework lacks sufficient justification: the predicted variances are not shown to represent calibrated uncertainty, and the probabilistic interpretation of posterior fusion relies on unvalidated assumption of conditional independence and heuristic reliability weights without a clear probabilistic meaning.
  • Comparisons focus on X-Fi and simple fusion baselines, omitting relevant recent methods such as Towards Balanced Multi-Modal Learning in 3D Human Pose Estimation (CVPR 2026). Differences in input processing and evaluation settings should be accounted for when comparing these methods.
Questions for authors
  1. Are the predicted variances validated as calibrated uncertainties against actual estimation errors? Could you justify the conditional-independence assumption underlying posterior fusion?
  2. The reliability target is based on cross-modal agreement but the predictor only uses unimodal inputs. What is the rationale for this design? Why does the distribution-alignment loss encourage similar uncertainties across modalities?
  3. Has flow refinement been compared with simpler alternatives? How are the source and target constructed for full-modality inputs when the refinement target is the full-modality consensus itself and why does refinement still improve performance?
  4. Could the authors compare with Towards Balanced Multi-Modal Learning in 3D Human Pose Estimation (CVPR 2026)?
  5. Could the authors provide results of generalization across subjects or environments beyond the evaluated modality subsets and synthetic corruptions?
  6. Table 3 reports 93.1% dual-modality accuracy for on XRF55 while the results in Table 2 shows 91.3%. Several All values also appear inconsistent with the stated procedure. Could the authors clarify the calculation of aggregate metrics?
  7. Is there any mechanism that ensures the environment latent actually captures environmental factors rather than redundant human information?
  8. Why is the flow velocity multiplied by an additional gate during inference but not during training?
8 comments
AuthorsOpenReview ↗

Response to Reviewer DgNh: Probabilistic Foundations, Calibration, and Refinement

W1. Probabilistic framework. Appendix A.2 derives Eq. (11) as a Gaussian PoE posterior under conditional independence. Appendix A.3 proves that z0z_0 minimizes ∑mπm∥z−μm∥2\sum_m\pi_m\|z-\mu_m\|^2 without independence, which is needed only to interpret v0v_0 as an exact posterior variance. Here, πm=ρm/σm2\pi_m=\rho_m/\sigma_m^2 tempers each expert by reliability. Eq. (6) combines the uncertainty prior −am-a_m with correction for confident-but-corrupted observations. Appendix A.4 shows that each modality's influence equals its normalized calibrated precision; Q1 provides empirical calibration.

W2. Relation to AWC. AWC balances four fixed RGB/LiDAR/mmWave/WiFi branches using pre-extracted RGB joints, whereas HUPER learns from raw observations and supports arbitrary subsets with one checkpoint. We omitted comparison because no implementation was public and the reported settings were unmatched. We reimplemented AWC with matched inputs, encoders, Protocol-3 split, budget, and metrics. AWC is a strong optimization baseline, while HUPER performs better under matched full- and partial-modality evaluation:

Method (matched setting)One model / arbitrary subsetFull MPJPE↓PA-MPJPE↓Partial Avg.↓
X-FiYes83.747.696.6
AWCNo68.441.294.7
HUPER-FlowYes64.737.781.2

Q1. Calibration. On held-out MM-Fi HPE, we compare u0u_0 with per-sample MPJPE using Spearman and uncertainty/error bins using ENCE. Pose-space NLL and coverage are estimated by decoding samples from N(z0,v0)\mathcal N(z_0,v_0), transported when evaluating z~\tilde z.

RepresentationSpearman↑ENCE↓NLL↓Cov.@68%Cov.@95%
Consensus z0z_00.630.1082.4765.293.1
Refined z~\tilde z0.710.0722.1167.494.7

Q2. Unified posterior. HUPER does not create a “unified modality.” Each sensor estimates a posterior over the same human variable; one modality yields one posterior and any subset a PoE consensus. Detached cross-modal agreement supervises unimodal ρm\rho_m, since other sensors may be absent at inference. Alignment preserves sensor-quality differences represented by (σm,ρm)(\sigma_m,\rho_m).

Q3. Refinement. The source is a corrupted strict-subset consensus and the target a detached full consensus, so full consensus is never mapped to itself during training. At inference, the field corrects residual sensing noise and the uncertainty gate prevents over-correction. We compare a parameter-matched residual MLP using the same source, target, optimizer, and head:

Refiner (same source/target)Full MPJPE↓All Avg.↓
No refinement73.093.4
Residual MLP71.890.1
DDPM residual70.888.7
Flow w/o uncertainty68.684.2
HUPER-Flow64.779.6

Q4. AWC comparison. See W2.

Q5. Generalization. Besides the Protocol-3 random split used for comparison with X-Fi, we evaluate unchanged models on MM-Fi's cross-subject and cross-environment partitions:

SplitMethodHPE MPJPE↓HAR Acc.↑
Cross-subjectX-Fi104.861.7
Cross-subjectHUPER-Flow82.674.9
Cross-environmentX-Fi112.358.9
Cross-environmentHUPER-Flow88.472.1

Q6. Aggregates. The correct XRF55 dual accuracy is 91.3%, the mean of the three dual subsets in Table 2. The 93.1% in Table 3 was a transcription error that did not affect subset results. We verified 91.3% over five runs and corrected all occurrences. Aggregates are now generated from logs: Single/Dual are within-cardinality means, Partial covers incomplete multimodal subsets, and All is the macro-average over cardinalities.

Q7. Environment latent. The variables are not exchangeable: only μm\mu_m is aligned, fused, refined, and used by task/structure heads; sms_m is neither aligned nor fused. Reconstruction from [cm;sm][c_m;s_m] preserves observation-specific information, while LorthL_{\rm orth} limits duplication. Linear probes confirm the intended specialization:

RepresentationAction Acc.Environment Acc.
Human posterior μm\mu_m82.131.4
Environment latent sms_m27.686.3

Q8. Inference gate. Flow Matching learns the complete velocity v⋆=zc−zsv^\star=z_c-z_s, independently of numerical integration. At inference, g(u0)g(u_0) is an adaptive step size: uncertain samples follow the learned transport, while confident consensus is protected from over-correction. Including it in the target would scale the direction twice. The ablation confirms that inference-time gating is most effective:

Gate strategyMM-Fi Full↓Partial Avg.↓XRF55 Full↑
No gate65.882.696.8
Train + inference gate65.281.996.9
Inference gate (ours)64.781.297.1
AuthorsOpenReview ↗

Friendly follow-up

Thank you again for recognizing that HUPER supports diverse modality combinations and multiple downstream tasks, and that it improves over existing baselines across these settings. We hope our additional calibration results, probabilistic justification, matched AWC comparison, refinement controls, and generalization experiments address your concerns. If any issue remains unclear, please let us know—we would be glad to discuss it and provide further clarification promptly.

Thank you for the additional experiments. The residual-MLP and AWC comparisons address parts of my comments, but the concerns about the reported results and the justification of the method remain.

The correction to 91.3% resolves the XRF55 arithmetic discrepancy. However, Tables 1 and 5 report full-modality HUPER-DDPM MPJPE of 65.3 mm, while Table 4 and the response report 70.8 mm. With Flow at 64.7 mm throughout, the improvement over DDPM is either 0.6 or 6.1 mm. Were these obtained under different configurations? This needs to be explained because it changes the evidence for the refinement method.

My concern about posterior fusion also remains. Even under conditional independence, multiplying unimodal posteriors generally requires correcting for the repeated prior. Equation 23 omits this correction without stating the assumption that permits it. The weighted-average derivation in Appendix A.3 does not address whether the resulting uncertainty is calibrated.

For the inference gate, calling it an adaptive step size does not answer my question. Equation 17 scales the velocity while keeping the time grid and integration horizon fixed, so it changes the learned transport. The ablation shows that this can help empirically, but the mathematical justification remains incomplete. The strict-subset training clarification is useful, although it still leaves unexplained why the same field should correct errors when initialized at the full-modality consensus.

Finally, my question about equation 47 remains unanswered. Why encourage different sensors to predict similar uncertainties when their sensing quality differs? The added calibration results evaluate fused outputs rather than the individual modality uncertainties used to weight fusion.

I appreciate the additional results but these unresolved issues are substantial so I maintain my original scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.