acceptodds
Under review as a conference paper at ICLR 2027

Choice Heterogeneity as Structured Deviations from a Shared Value Anchor

Abstract

Action preferences can differ systematically across trajectories and over time. In clinical care, patients with similar observed physiological conditions may receive different treatments even when clinicians share the goal of improving patient outcomes. Behavioral prediction alone leaves these persistent differences unexplained against a common reference. We introduce a decomposition of sequential behavior that combines a shared action-value anchor with switching regimes characterized by interpretable action-cost profiles. A continuous-time latent-state model learns interpretable physiological phenotypes and their dynamics from irregular, partially observed records. It supports forecasting, constructs the anchor under a prespecified reward and continuation rule, and provides physiological context for clinical audit. With the anchor fixed, the regime layer learns action-dependent costs within a common feature dictionary, making treatment preferences comparable across regimes. Controlled synthetic experiments support regime and action-cost contrast recovery. In CKD–MBD and MIMIC–IV, the framework improves physiological forecasting and action prediction, while inferred regimes capture persistent treatment patterns. Clinical cases link these patterns to physiological trajectories and decompose anchor-relative departures into treatment-feature contributions, supporting longitudinal clinical audit.

Primary area:
learning on time series and dynamical systems
Keywords:
Sequential decision modeling, Latent regime switching, Interpretable cost decomposition

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yX9pOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a hierarchical (two-stage) model for actions in sequential data, e.g. treatment actions in clinical patient trajectories. Each level in the hierarchy is some kind of continuous time HMM, i.e. a latent variable model where the latent state can have discrete states that change continuously in time. The paper learns two such models on top of each other, such that the lower level is fixed while learning the higher level. The lower level is interpreted as a “shared action-value anchor” and the higher level as “behavioral regimes” deviating from the shared anchor.

Critical strengths

None.

Critical weaknesses

The modeling approach seems sensible although it is difficult to understand the model very well by reading the main text. However, there are very many existing hierarchical modeling approaches, some more sophisticated than the presented method (e.g. end-to-end), that have been applied to clinical and other kinds of sequential data, and hence I am missing methodological novelty in this work. The analysis of the real-world data is reasonable but not very thorough, and hence I don’t find that either convincing enough to support acceptance at a top ML conference.

Questions for authors

Suggestion: it would be useful to better define terms used in the main text, e.g. what are “continuous outcome-proxy variables” and how are they different from “measurements”?

2 comments
AuthorsOpenReview ↗

Response to Reviewer yX9p

Thank you for your review. We clarify the method, its novelty, and the key terms below.

1. Method and novelty

The method explains why patients in similar conditions receive different treatments, measured against one common clinical reference.

  • Stage I learns how a patient's physiology evolves from irregular lab records and scores each candidate treatment aa by its expected future lab outcomes under a prespecified reward. This score is the shared anchor Qanc(x,a)Q_{\mathrm{anc}}(x,a), fixed after Stage I.
  • Stage II explains departures from the anchor. Decisions follow a few persistent regimes that switch over time, and each regime adds its own cost on interpretable treatment features gg (e.g., continuing, adding, or switching drugs). Regime uu chooses actions by the softmax of Su(x,ξ,a)=Qanc(x,a)−wu⊤g(ξ,a).S_u(x,\xi,a)=Q_{\mathrm{anc}}(x,a)-w_u^\top g(\xi,a).

The novelty is this separation, not the hierarchy: a fixed, outcome-based reference plus regime costs on the same scale, so regimes are directly comparable. Prior work differs:

  • GLM-HMMs [1] give each regime its own policy, with no common reference.
  • IRL with switching rewards [2] and multi-modal constrained IRL [3] learn rewards or constraints from choices alone, so routine prescribing is absorbed into the reference; [3] also keeps one expert type per trajectory.
  • Hierarchical clinical HMMs [4] model heterogeneity in disease dynamics, not in treatment choices.

Fitting with the anchor is a real restriction: the anchored class q−Gq-\mathcal G equals G\mathcal G only if q∈Gq\in\mathcal G (Theorem 3), and anchoring during fitting, not post hoc, cuts cost RMSE from 0.547 to 0.125 (Table 7). A more flexible end-to-end model loses this:

ModelSynth. cost RMSE ↓CKD–MBD NLL ↓MIMIC–IV NLL ↓
Action-only IRL0.9800.9420.672
End-to-end0.4320.9310.649
Two-stage (ours)0.2570.9340.654

Takeaway: end-to-end training matches action NLL within 0.005 but raises cost RMSE by 68%; the action-only reference matches observed actions in 67.4% / 82.5% of decisions vs. 3.2% / 34.4% for ours, absorbing habits.

2. Depth of the analysis on real data

The analysis shows what changes in physiology and in treatment preference, and how this alters individual decisions:

  • Phenotypes (Figure 3): PTH-normal vs. low PTH states in CKD–MBD; more vs. less acute in MIMIC–IV.
  • Regimes (Figure 4): one CKD–MBD regime keeps a compact regimen, another extends it.
  • Decisions (Figure 2): continuity and drug identity explain choosing calcium carbonate over the anchor's sevelamer; acute-state effects explain norepinephrine over dobutamine.
  • Trajectories (Figure 7): the CKD phenotype stays stable from month 2 while treatments and regimes change.

Quantitative checks support this reliability: Stage I and II give the lowest forecasting and action NLL vs. six baselines and neural action models (Figures 5 and 8), robust to fixed classes, regime counts, anchor variants, features, and 40 refits per case (Tables 10, 16, 19–20, Appendix G.1).

3. Definitions of key terms

A measurement is a sparse lab snapshot; the outcome proxy is the model's continuous estimate that carries the trend between snapshots (Appendices C and F).

  • Measurement oℓo_\ell: a lab result (e.g., phosphorus) coded low / normal / high; sparse and irregular.
  • Outcome proxy λℓ(t)∈(0,1)\lambda_\ell(t)\in(0,1): the latent probability that lab ℓ\ell is normal at any time, with ddtλℓ=αℓ(z)(1−λℓ)−κℓ(z)λℓ\frac{d}{dt}\lambda_\ell=\alpha_\ell^{(z)}(1-\lambda_\ell)-\kappa_\ell^{(z)}\lambda_\ell, Pr⁡(oℓ=normal∣z,λ)=λℓ\Pr(o_\ell=\mathrm{normal}\mid z,\lambda)=\lambda_\ell.
  • Phenotype zz: a discrete physiological state (e.g., low PTH) switching in continuous time; it sets the proxy's rates.
  • Decision state xm=(bm,λm)x_m=(b_m,\lambda_m): the phenotype belief bmb_m and proxies at decision mm.
  • Anchor Qanc(x,a)Q_{\mathrm{anc}}(x,a): Stage I's score of action aa, shared by all clinicians.
  • Regime uu: a persistent treatment pattern of clinicians, switching over time.
  • Cost wu⊤g(ξ,a)w_u^\top g(\xi,a): regime uu's penalty on action aa's features, on the anchor's scale.

Removing the proxy worsens forecasting NLL in all four settings (Appendix J.2).

In the revised manuscript, Section 2 points to Table 2 (Appendix C); Appendix A extends related work, and Appendices J.2 and J.5 add ablations (Tables 14 and 18).

[1] Ashwood et al., "Mice alternate between discrete strategies during perceptual decision-making," Nat. Neurosci., 2022.

[2] Ke et al., "Inverse reinforcement learning with switching rewards and history dependency for characterizing animal behaviors," ICML, 2025.

[3] Qiao et al., "Multi-modal inverse constrained reinforcement learning from a mixture of demonstrations," NeurIPS, 2023.

[4] Severson et al., "Personalized input-output hidden Markov models for disease progression modeling," MLHC, 2020.

Thank you for your rebuttal. I have read it and we will discuss with the other reviewers. I still feel the methdological contribution is a bit limited for ICLR. For now, I keep my original score.

Reviewer_zErTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies persistent heterogeneity in sequential treatment decisions. The main idea is to separate treatment choice into a shared value anchor and regime-specific deviations from that anchor. The anchor represents a common outcome-based reference. The deviations are modeled as interpretable action costs that vary across persistent behavioral regimes.

The method has two stages. Stage I uses a continuous-time latent-state model to infer physiological states from irregular and incomplete clinical histories. It also constructs the shared action-value anchor using a prespecified reward and continuation rule. Stage II uses a switching regime model to explain treatment choices relative to this fixed anchor. Each regime has a different action-cost profile over a common feature dictionary.

The paper evaluates the method on synthetic data, a CKD-MBD cohort, and MIMIC-IV. The synthetic experiments evaluate recovery of regimes and action costs. The clinical experiments evaluate physiological forecasting, treatment prediction, regime structure, and case-level treatment audits.

Critical strengths
  1. The paper addresses an interesting and practically important form of heterogeneity in sequential decisions. Instead of explaining different treatment choices by assigning each group a different reward or policy, the paper fixes a common value reference and models persistent differences as structured deviations around that reference. This gives the regimes a common scale and makes their treatment preferences easier to compare.

  2. The paper makes a serious effort to connect prediction with interpretation. The continuous-time model handles irregular and partially observed clinical trajectories. The switching layer captures persistent treatment patterns rather than independent decision noise. The paper also includes synthetic recovery experiments, clinical forecasting, action prediction, anchor sensitivity, feature sensitivity, and several theoretical results on identification and stability.

Critical weaknesses
  1. The clinical interpretation of the inferred regimes and action costs remains difficult to validate. The clinical records capture only part of the information that can affect treatment. The CKD-MBD model mainly observes calcium, phosphorus, PTH, and medication history. The MIMIC-IV model uses six physiological variables, recent events, and treatment history. Many other factors may affect a treatment decision, including comorbidities, contraindications, treatment doses, additional laboratory results, provider preferences, resource availability, hospital protocols, and other information available to the clinician.

These omitted factors can be absorbed by the latent behavioral regimes. The model can therefore identify persistent patterns in observed choices without establishing what caused those patterns. This matters because resource constraints, policy requirements, and management burden are used to motivate the regime interpretation, but these factors are not directly observed or validated in the clinical experiments. The paper appropriately treats the clinical analysis as descriptive, but this also limits how strongly the learned costs can be interpreted.

  1. The meaning of the learned deviations depends heavily on the chosen anchor. The shared anchor is constructed from fitted physiological dynamics, a prespecified reward, and a continuation rule. It is not independently established as the clinically correct value function. The synthetic results make this issue especially clear. A perturbed anchor produces almost the same NLL and regime recovery but increases cost RMSE from 0.068 to 0.808. Thus good behavioral prediction does not imply correct recovery of the anchor-relative costs.

The clinical sensitivity results point in the same direction. Fitted actions remain almost unchanged under several alternative anchors, while the anchor-preferred action can change substantially. For CKD-MBD, for example, the one-step continuation anchor agrees with the primary anchor on only 57.0% of top actions, even though fitted-action agreement remains 99.3%. This makes the behavioral model robust as a predictor, but it does not by itself validate the clinical reference used for the audit.

Questions for authors
  1. How should the behavioral regimes be interpreted when important determinants of treatment are unobserved? Could the authors clarify which clinically relevant variables are unavailable in each dataset and whether provider, hospital, protocol, or operational variables could be incorporated to test whether the inferred regimes still persist?

  2. What evidence supports the chosen shared anchor as a clinically meaningful reference? The synthetic experiments show that a wrong anchor can preserve predictive fit and regime recovery while substantially changing recovered costs. Could the anchor be compared with an expert-defined or guideline-based reference, or otherwise validated independently of action prediction?

  3. How strong is the identification of the latent regimes in the clinical setting? Appendix B assumes regime-kernel identifiability and states that general identification with history-dependent contexts is outside the result. Could the authors clarify when this assumption should hold for the two clinical models and whether different latent decompositions could produce similar observed action probabilities?

2 comments
AuthorsOpenReview ↗

Response to Reviewer zErT

Thank you for recognizing that a shared value reference gives regimes a common scale and that the paper connects prediction with interpretation. We address each point below.

W1 & Q1. Interpreting regimes when treatment determinants are unobserved

The clinical analysis is descriptive; its value is qualitative. Regimes capture persistent choice patterns beyond the recorded context, including operational and institutional ones. With Stage I phenotypes, they separate physiological change from preference change; e.g., a CKD–MBD phenotype stays stable from month 2 while regimes change (Figure 7).

Since real data cannot validate this directly, we rely on synthetic experiments with known costs and regimes (Tables 4 and 6) and quantitative clinical checks; e.g., switching beats fixed classes, which absorb static hospital or physician factors, by 0.081 / 0.048 NLL (Table 16).

The regime cost features focus on what clinicians actually weigh in each setting (Table 9).

CKD–MBD lacks comorbidities, dialysis details, and continuous doses; MIMIC–IV lacks provider identifiers, bedside protocols, and infusion rates. Protocol and operational variables can enter the context ξ\xi directly, like the existing clinical-rule features and the previous regimen. We add: age, sex, and history of lab abnormalities in CKD–MBD (25 → 55 features), and night shift, weekend, admission year, prior ICU stay, and 14 organ-dysfunction indicators in MIMIC–IV (78 → 154 features).

Dataset (expanded ξ\xi)Single-regime NLL ↓Fixed-class NLL ↓Switching NLL ↓Profile cos. ↑Audit sign agr. (%) ↑
CKD–MBD1.2141.0050.9430.72398.6
MIMIC–IV0.7070.6690.6620.95298.7

Takeaway: switching still fits best and audit signs agree in ≥98.6% of decisions. Regimes are only weakly associated with the added variables (Cramér's V 0.04–0.21).

In the revised manuscript, Appendix J.8 adds the expanded-context refits (Table 21).

W2 & Q2. Evidence for the anchor as a clinical reference

We validate the anchor outside action prediction, as Table 4 shows action fit alone cannot. Its reward targets guideline lab ranges; three checks support it.

(1) Stage I forecasts held-out physiology best, beating six baselines on NLL, Brier score, and accuracy in all four settings (Figure 5).

(2) The anchor agrees with guidelines. We score actions from pre-action labs by KDIGO 2017 [1] and Surviving Sepsis 2021 [2] rules and compare with the anchor's top action.

(3) A guideline reference yields the same regimes when we refit Stage II with this score QguideQ_{\mathrm{guide}} as the anchor.

DatasetObserved = guideline (%)Anchor top = guideline (%)NLL: Qanc→QguideQ_{\mathrm{anc}}\to Q_{\mathrm{guide}}Profile cos. ↑Audit sign agr. (%) ↑
CKD–MBD49.690.50.934 → 0.9380.89599.2
MIMIC–IV90.075.90.654 → 0.6550.98699.4

Takeaway: the anchor follows KDIGO far more often than CKD–MBD prescriptions, while ICU clinicians already follow broad rules. The guideline score shifts NLL by ≤0.004 and reproduces the regime profiles and audit signs.

The one-step continuation anchor's 57.0% agreement reflects its short horizon, as CKD–MBD states persist 2.7–13.0 months; audits keep 90.1% of top-gap decisions with 98.4% sign agreement (Table 11).

In the revised manuscript, Appendix I.4 adds the guideline scoring and the QguideQ_{\mathrm{guide}} refit (Table 13).

Q3. Regime identification in the clinical models

When contexts do not depend on past actions, regimes are identified up to labels if the kernel is full rank and the regime action laws are linearly independent. Three consecutive actions then form a three-way mixture, unique by Kruskal's theorem [3]; the clinical kernel satisfies det⁡Π(Δ)=e−(K−1)Δ/τ>0\det\Pi(\Delta)=e^{-(K-1)\Delta/\tau}>0.

This does not formally cover our clinical contexts, which include the previous regimen (Appendix B), so we check it empirically:

  • the rank condition holds for 100.0% of CKD–MBD and 95.2% of MIMIC–IV held-out decisions;
  • fitted regimes reproduce across splits (cosine 0.820 / 0.973, Table 19) and 20 restarts (0.911 / 0.963).

Given the regimes, different decompositions match action probabilities only through two equivalences (Theorems 2 and 3):

  • coefficient shifts wu−wu′∈Nw_u-w'_u\in N, which change no action contrast;
  • anchor shifts, which fixing QancQ_{\mathrm{anc}} removes.

In the revised manuscript, Appendix B.3 proves Proposition 4 using four decisions.

[1] Ketteler et al., "Executive summary of the 2017 KDIGO CKD–MBD guideline update," Kidney Int., 2017.

[2] Evans et al., "Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock 2021," Crit. Care Med., 2021.

[3] Kruskal, "Three-way arrays: rank and uniqueness of trilinear decompositions," Linear Algebra Appl., 1977.

Thank you for the detailed response and additional experiments. The expanded-context analysis and guideline comparisons address much of my first two concerns.

Two points remain:

  1. The CKD-MBD regime-profile similarity falls to 0.723 with expanded covariates. Could the authors clarify how much the interpretation of individual regimes changes? Also, the guideline comparison supports the anchor as a clinical reference, but does not establish that its preferred treatments improve outcomes.
  2. The new identification result assumes contexts do not depend on previous actions, whereas the clinical models explicitly use treatment history. The empirical rank checks and stability results are useful, but do not establish identification in this setting. Could the authors clarify what identification guarantees remain valid for the implemented models?

These clarifications would help me assess the reliability of the clinical interpretations.

Reviewer_ywmcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper models persistent differences in sequential choices relative to a shared action-value anchor. A continuous-time latent-state model learns physiological dynamics from irregular clinical records and constructs the anchor under a specified reward and continuation rule. With this anchor fixed, a switching regime model learns action costs in a common feature dictionary. The paper characterizes conditional identifiability and the relationship between anchored and unanchored score classes. Synthetic experiments assess recovery, while CKD–MBD and MIMIC–IV experiments examine forecasting, action prediction, and retrospective clinical audits.

Critical strengths
  1. The modeling framework is well suited to irregular clinical data. The continuous-time latent-state model handles missing and unevenly sampled measurements, while the switching regime layer captures persistent changes in treatment behavior.

  2. The empirical controls address possible alternatives to regime switching. Patient-level splits, Stage-I cross-fitting, and pre-action predictions reduce opportunities for leakage. The fixed-trajectory-class baseline updates its class posterior as actions arrive, so it is a meaningful persistence baseline. In Table 11, switching improves NLL over fixed classes by 0.081 on CKD–MBD and 0.048 on MIMIC–IV. The CKD analysis restricted to decisions with a recorded prescription also supports the benefit beyond carried-forward treatment labels.

Critical weaknesses
  1. The clinical anchor is not causally justified. Stage I learns treatment-conditioned dynamics from observational records and rolls out alternative treatments to construct QancQ_{anc}, but provides no sequential ignorability, overlap, or unmeasured-confounding analysis. Consequently, an anchor-preferred action, and the inferred “cost” of departing from it, may reflect confounding by indication rather than treatment value.

  2. The interpretability theory assumes away the central identification problem. Appendix B.3 assumes that the regimes and their transition patterns can already be uniquely recovered from observed actions. The subsequent result therefore shows only that the costs are identifiable if the regimes are already identifiable, rather than proving identifiability of the complete proposed model.

  3. Moreover, in the synthetic experiment, changing the anchor leaves action prediction and regime recovery almost unchanged, but increases cost error from 0.068 to 0.808 (Table 3). Thus, the model can fit behavior well while providing an inaccurate explanation of why actions were chosen. Stronger validation is needed before these inferred costs can be considered clinically meaningful.

  4. Clinical preprocessing may artificially create persistence and predictability. In CKD–MBD, 24.9% of monthly “decisions” are carried-forward regimens. Because treatment continuation is a major model feature, this may mechanically strengthen the observed persistence. In MIMIC–IV, the analysis includes only hours containing treatment events, thereby omitting clinically meaningful no-treatment decisions. In addition, because sequences are aligned around a low-urine-output event, please clarify how the “hour index” feature is defined. If it represents position within the aligned window, it reveals proximity to a future event and may inflate action-prediction performance. The paper should show that regime profiles and audit explanations remain stable under a less artificial construction.

Questions for authors

Please address the points in weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ywmc (1/2)

Thank you for recognizing that the framework suits irregular clinical data and that our controls (patient-level splits, Stage I cross-fitting, the fixed-class baseline) make the switching gain meaningful. We address each point below.

W1. Causal justification of the clinical anchor

The anchor has a causal reading under three standard conditions, and the regime and audit findings are largely preserved under limited overlap and unmeasured confounding.

Stage I is a model-based g-formula: it rolls out the fitted treatment-conditioned dynamics from the filtered state xmx_m. It equals the interventional value E[⋅∣do(a),xm]\mathbb E[\cdot\mid\mathrm{do}(a),x_m] under sequential ignorability given the pre-action history Hm−H_m^-, positivity p(a∣Hm−)>0p(a\mid H_m^-)>0, and correct dynamics. Indication reflected in recorded labs and treatment history is thus adjusted for.

(1) To check overlap, we call an action supported when its propensity e^(a∣xm,ξm)≥t\hat e(a\mid x_m,\xi_m)\ge t and refit Stage II with the anchor restricted to supported actions.

Dataset (tt)Anchor action supported (%)NLL: primary → restrictedWeight cos. ↑Audit sign agr. (%) ↑
CKD–MBD (0.02)25.80.934 → 0.9360.92799.5
CKD–MBD (0.05)5.20.934 → 0.9360.88499.3
MIMIC–IV (0.02)65.00.654 → 0.6570.92398.0
MIMIC–IV (0.05)51.70.654 → 0.6560.92698.0

Takeaway: although anchor-preferred actions are often uncommon, restriction changes NLL by at most 0.003 and keeps audit signs at 98.0% or higher. The findings do not rely on extrapolation to unsupported actions.

(2) For unmeasured confounding, Theorem 3 bounds the cost error by the anchor error, ∣c^u−cu⋆∣ν≤εQ+2Ru/(β2p0A)\\|\hat c_u-c_u^\star\\|_\nu\le\varepsilon_Q+\sqrt{2R_u/(\beta^2p_0A)}, and an anchor error shared by all regimes cancels in their differences, C^u−C^v=Cu−Cv\hat C_u-\hat C_v=C_u-C_v, when it lies in the feature span. We add to the Table 6 setting a hidden persistent severity of strength γ\gamma that both worsens dynamics and raises treatment intensity.

γ\gammaAnchor Q RMSE ↓Cost RMSE ↓Regime-difference RMSE ↓ARI ↑
00.2550.2570.1880.451
0.50.3320.2700.2060.414
1.00.3460.3280.2340.427
2.00.7260.6660.4030.479

Takeaway: cost error tracks anchor error as Theorem 3 predicts, regime differences grow least, and ARI shows no systematic decline. Confounding biases absolute costs but retains the regime structure. Consistently, 20% noise in Stage I values keeps cost-sign agreement at 96.1% or higher on real data (Tables 11 and 20).

In the revised manuscript, Appendix I.2 adds the propensity diagnostics and refits with the restricted anchor (Table 12), and Appendix D.6 the hidden-confounder simulation (Table 8).

W2. Identifiability of the complete model

We prove that the complete model is identifiable under a checkable rank condition when contexts do not depend on past actions, in two steps.

(1) Three consecutive actions form a three-way mixture over regimes. Since the kernel is full rank, det⁡Π(Δ)=e−(K−1)Δ/τ>0\det\Pi(\Delta)=e^{-(K-1)\Delta/\tau}>0, Kruskal's theorem [1] makes this mixture unique up to label permutation whenever the regime action laws are linearly independent, a standard condition for hidden Markov models. Two overlapping triples align labels across four decisions (Proposition 4), giving Assumption 1.

(2) The softmax is invertible, so Theorem 2 recovers each cost contrast from the identified eue_u: log⁡eu(a)eu(a′)=β[Qanc(a)−Qanc(a′)−(Cu(a)−Cu(a′))]\log\frac{e_u(a)}{e_u(a')}=\beta[Q_{\mathrm{anc}}(a)-Q_{\mathrm{anc}}(a')-(C_u(a)-C_u(a'))].

Empirically, the rank condition holds for 100.0% of CKD–MBD and 95.2% of MIMIC–IV held-out decisions.

In the revised manuscript, Appendix B.3 adds Proposition 4, its proof via Kruskal's theorem, and the held-out rank check.

[1] Kruskal, "Three-way arrays: rank and uniqueness of trilinear decompositions," Linear Algebra and its Applications, 1977.

AuthorsOpenReview ↗

Response to Reviewer ywmc (2/2)

W3. Can the model fit behavior well while explaining it wrongly?

Our explanations do not rely on fit alone: the regime differences we interpret remain far more accurate than absolute costs under anchor error, and the anchor is validated outside the fit.

Fit alone cannot certify absolute costs, as Table 4 is designed to show and as in inverse reinforcement learning [1]: an anchor error δq\delta q shifts every cost equally, Qanc−Cu=(Qanc+δq)−(Cu+δq)Q_{\mathrm{anc}}-C_u=(Q_{\mathrm{anc}}+\delta q)-(C_u+\delta q), leaving the fit unchanged.

δq\delta q cancels in regime differences: their RMSE moves only from 0.104 to 0.149, against 0.808 for absolute costs.

The anchor is validated outside the fit: it recovers costs in synthetic data (RMSE 0.257 vs. 0.711 without an anchor, Table 6), and the clinical anchor follows KDIGO rules in 90.5% of CKD–MBD decisions vs. 49.6% for observed prescriptions.

In the revised manuscript, Appendix I.4 adds the KDIGO and Surviving Sepsis guideline comparison (Table 13).

[1] Ng & Russell, "Algorithms for inverse reinforcement learning," ICML, 2000.

W4. Do the preprocessing choices create persistence and predictability?

The preprocessing does not drive the results: under three stricter constructions, switching still beats fixed classes and the audit conclusions hold.

We first clarify the current construction:

1. Hour index (MIMIC–IV): it counts hours since the sequence's first event and cannot inflate our predictions, because none of the 78 Stage II cost features (Table 9) or Stage I inputs uses it; only the neural baselines do. Appendix G's description of it as part of the context ξ\xi is imprecise, and we will make it precise.

2. Carried-forward regimens (CKD–MBD, 24.9% of decisions): they cannot favor switching by themselves, since fixed classes receive identical inputs, and on recorded prescriptions alone, switching still predicts treatment changes better (NLL 3.020 vs. 3.115, Table 16).

3. No-treatment hours (MIMIC–IV): the primary analysis omits them, so we now add them as an explicit action below.

We then check profiles and audits on recorded CKD–MBD prescriptions (NLL from Table 16), and refit Stage II on all MIMIC–IV hours with an explicit no-treatment action and on MIMIC–IV hours before the low-urine onset.

ConstructionSwitching NLL ↓Fixed-class NLL ↓Profile cos. ↑Audit sign agr. (%) ↑
CKD–MBD, primary0.9341.0151.000100.0
CKD–MBD, recorded prescriptions only1.1911.2610.92499.4
MIMIC–IV, primary0.6540.7021.000100.0
MIMIC–IV, with no-treatment hours0.7210.7490.82694.7
MIMIC–IV, pre-onset hours only0.8090.8130.94598.1

Takeaway: switching beats fixed classes in every construction, and audit signs agree in at least 94.7% of decisions. The regimes reflect persistent treatment patterns rather than preprocessing.

In the revised manuscript, Appendix J.4 adds the refits under the three alternative constructions (Table 17).

Ethics review.

To preserve anonymity during double-blind review, we withhold ethics and data governance details that would identify the institution of the CKD–MBD cohort, and will provide them in the camera-ready version if the paper is accepted.

Thanks for the authors response and additional empirical results. It clears out some of my reservation on the empirical evaluation, however, my concerns on how to establish causality and the values and contribution of the theoretical results remain. I will keep my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.