acceptodds
Under review as a conference paper at ICLR 2027

Where Latents Belong and How They Flow: Joint Identification of Structural Roles

Abstract

Latent factors in dynamical systems can play different roles across observations and time. For example, an idea in a group meeting may be expressed by only some participants, inherit information from earlier discussions, and shape what is said next. We formalize this notion as the structural role of a latent factor, describing where it is expressed, whether it receives influence from the past, and whether it transmits influence to the future. The main challenge is correspondence, since the three structures may be identified under different permutations, while a structural role is defined only when they are aligned to the same latent coordinate. We show that, under suitable structural and variability conditions, all three roles are jointly identifiable under a common permutation at each time point. Guided by this theory, we develop RoleTrace, a framework that traces structural roles in a shared latent coordinate system. Synthetic experiments validate structural recovery, while experiments on multiparty meetings examine the identified roles in real sequential interactions.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Identifiability, Causal Representation Learning, Temporal Representation Learning, Nonlinear ICA, Grouped Observations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pK7EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

This paper addresses the problem of identifying the structural roles of latent factors in dynamical systems. A central difficulty is the correspondence problem: the relevant structural components can be recovered only up to an arbitrary permutation of the latent coordinates, precluding unambiguous role assignment without an explicit alignment procedure. The authors establish the joint identifiability of the three roles up to a shared permutation at each time step, contingent on specified structural assumptions and variability conditions. They subsequently propose ROLETRACE, a method for role estimation that leverages Jacobian and cross-Hessian derivatives of an encoder–decoder architecture, and evaluate its performance on both synthetic benchmarks and the AMI meeting corpus.

Critical strengths
  1. The paper tackles a well-motivated and comparatively underexplored question in the representation learning literature: how to systematically characterize the structural role of each latent factor in terms of where it is expressed.

  2. This study presents a principled formalization of structural role identification in grouped sequential data, wherein each latent coordinate is systematically characterized along three orthogonal dimensions: group-specific expression, incoming temporal influence, and outgoing temporal influence. This tripartite decomposition furnishes a unified, interpretable framework for elucidating the contribution of latent factors to group dynamics.

  3. The authors develop ROLETRACE, a theory-guided framework that estimates structural roles ensuring that the estimator directly instantiates the theoretical definitions. The framework is validated on both controlled synthetic settings and the real-world AMI meeting corpus, demonstrating that the identified roles carry meaningful structure in naturalistic sequential interactions.

Critical weaknesses
  1. The empirical validation on real-world data is limited to a single dataset — the AMI meeting corpus — which raises concerns about the generalizability of the identified roles across different domains and interaction types. Moreover, the linkage between the theoretical identifiability guarantees and the empirically observed behavior on real-world data is not adequately substantiated, rendering it uncertain whether the practical performance of ROLETRACE is in fact supported by the proposed theoretical framework.

  2. The empirical comparison is underwhelming given the framework’s complexity: the gains over baselines don’t convincingly justify ROLETRACE’s theoretical and computational overhead. Notably, the quantitative comparisons are confined to the appendix rather than integrated into the main text, which may indicate that the observed performance differences are not sufficiently robust or consequential to warrant emphasis as a principal result.

  3. The paper lacks any discussion of computational complexity, which is a notable omission given the derivative-based role estimation procedures. Without a clear characterization of computational efficiency, it is difficult to assess whether the proposed framework is practically viable.

(minor) Deferring all related work discussion to the appendix is a notable structural weakness, as it prevents readers from clearly situating the proposed framework within the broader literature without leaving the main body.

Questions for authors
  1. Can the authors provide additional evidence or analysis demonstrating that the synthetic data generation process adequately reflects the structural properties of the real-world settings?

  2. The proposed framework is evaluated primarily on small-scale synthetic data. Can the authors provide experiments on larger synthetic datasets to assess whether ROLETRACE scales gracefully as the number of latent factors, group size, or sequence length increases?

  3. As shown in Table 4, the performance gains of ROLETRACE over traditional baselines are not particularly striking. Given that classical methods were likely not subject to extensive hyperparameter tuning, is it possible that a well-tuned baseline could match or even outperform the proposed method? How robust are the reported results to this concern?

5 comments

Thank you for sharing your thoughtful feedback. We appreciate the opportunity to address your concerns below.


W1. Real-world generalizability and connection to theory.

A1. We appreciate the reviewer's careful consideration of our real-world evaluation and its theoretical grounding. We would like to clarify that our synthetic and real-world experiments serve complementary purposes: synthetic experiments directly evaluate structural recovery under known ground truth, while AMI examines whether the estimated roles provide useful information in real sequential interactions.

The connection to our theory follows naturally from the grouped temporal structure of meetings. Participants define meaningful observation groups (L430–431, L460-463), while conversational variation and temporal interactions motivate the variability and influence conditions. Faithfulness and minimality further characterize when these structural influences are distinguishable. Although AMI does not provide ground-truth latent roles to directly verify identifiability, our framework applies to a broader class of grouped sequential systems (L35-39), with weaker identification requirements (L224-L226, L270-277, Appendix A.5) and extensions to higher-order temporal dependencies (Appendix B). We hope this clarifies how the AMI study supports the practical relevance of our framework and complements the theoretical and synthetic results. Thank you for encouraging us to elaborate on this important aspect.


W2. Quantitative comparisons.

A2. Thanks for your thoughtful feedback. We would like to clarify that the main quantitative comparisons are already presented in Tables 1 and 2 of the main text. Table 1 directly evaluates structural-role recovery, where RoleTrace achieves 0.854 Joint accuracy versus 0.708 for the closest baselines, an improvement of 14.6 percentage points. Table 2 compares AMI next-speaker prediction at matched latent budgets using the same representation and downstream classifier. Appendix Table 4 (In the latest revised version, it is Table 10) provides supplementary comparisons with conventional predictors using different input dimensions, while Table 2 offers the more direct, matched-budget comparison, where the learned representation, latent budget, and downstream classifier are shared. This isolates the contribution of structural role selection. We appreciate the opportunity to clarify these points.


W3. Computational complexity.

A3. We sincerely appreciate your great suggestion. For latent dimension NN and observation dimension dxd_x, the decoder Jacobian contains O(dxN)O(d_xN) entries, while each transition cross-Hessian contains O(N2)O(N^2) entries. Our implementation computes only the required mixed derivative block through automatic differentiation. For the conditional Gaussian transition, the same block can also be expressed through first-order network Jacobians.


W4. Related Work.

A4. Thank you for this constructive suggestion on the paper's organization. We compared the most relevant literature in the introduction (L52-79). In the revision, we have included this section in the main text (Section 6).


Q1. Is there additional evidence or analysis demonstrating that the synthetic data generation process reflects the structural properties of real-world settings?

A5. We appreciate this thoughtful question about the realism of our synthetic setting. Our synthetic DGP is designed to capture the structural properties underlying grouped sequential interactions (L1465-1503). Specifically,

(a) Different observation groups express different, potentially overlapping subsets of latent factors through nonlinear mappings, corresponding to participants expressing shared or distinct information.
(b) Sparse directed transitions characterize incoming and outgoing temporal influence.
(c) Nonlinear, state-dependent means and variances capture varying temporal dynamics.

Moreover, the mechanisms are randomly generated and exhibit substantial structural diversity. Among the 16 retained mechanisms, all have distinct group-incidence patterns and labeled delayed graphs, spanning 14 directed graph isomorphism classes. Each mechanism contains 3–5 latents shared across multiple groups and 1–3 coordinates with overlapping incoming or outgoing activity.

We have also examined the structural assumptions of these synthetic mechanisms. Further quantitative diagnostics on the fitted models are provided in our response to oQyj's Q1, which we hope will be helpful here as well.

Q2. How does RoleTrace scale with the number of latent factors, group size, or sequence length?

A6. Thank you for emphasizing the value of evaluating larger systems. We would first like to highlight that Figure 6 already evaluates sample-size scaling from n=50n=50 to 32003200 training sequences. To further investigate scalability, we conducted additional experiments varying latent dimension NN, group dimension dd, and sequence length TT, with eight independently sampled mechanisms per setting. For brevity, we present the results of RoleTrace with the two strongest baselines, with the full comparisons provided in Figure 9 of the revised manuscript.

  • (a) Scaling Latent Dimension. We evaluate the settings from N=4N=4 to N=20N=20:

    NNRoleTraceNo-refitThoughtComm-adapted
    40.844 ± 0.0660.719 ± 0.1200.812 ± 0.091
    60.812 ± 0.0580.667 ± 0.1090.771 ± 0.070
    80.906 ± 0.0390.844 ± 0.0610.828 ± 0.058
    100.775 ± 0.0410.650 ± 0.0570.750 ± 0.065
    120.781 ± 0.0440.802 ± 0.0440.750 ± 0.039
    140.741 ± 0.0380.661 ± 0.0440.625 ± 0.052
    160.766 ± 0.0520.664 ± 0.0620.672 ± 0.054
    180.799 ± 0.0440.771 ± 0.0450.750 ± 0.038
    200.744 ± 0.0260.694 ± 0.0410.706 ± 0.037
  • (b) Scaling observation structures. We further vary the group dimension dd:

    ddRoleTraceNo-refitThoughtComm
    60.812 ± 0.0580.667 ± 0.1090.771 ± 0.070
    90.792 ± 0.0760.708 ± 0.0820.708 ± 0.093
    120.708 ± 0.0760.625 ± 0.0930.729 ± 0.070
    150.708 ± 0.0610.688 ± 0.0660.604 ± 0.070
  • (c) Scaling Sequence Length. We additionally extend the sequence length from T=5T=5 to T=20T=20:

    TTRoleTraceNo-refitThoughtComm
    50.812 ± 0.0580.667 ± 0.1090.771 ± 0.070
    100.896 ± 0.0440.875 ± 0.0610.854 ± 0.058
    150.938 ± 0.0440.917 ± 0.0450.875 ± 0.061
    200.938 ± 0.0440.917 ± 0.0630.917 ± 0.045

Overall, RoleTrace achieves the highest joint accuracy in most settings. We hope these additional experiments help address your concerns and provide a clearer picture of RoleTrace's scalability. Thanks again and we sincerely appreciate the opportunity to better understand how RoleTrace scales in practice.


Q3. Could better hyperparameter tuning allow baselines Table 4 to match or outperform RoleTrace? How robust are the reported results to this concern?

A7. Thank you for highlighting the importance of baseline tuning. We would like to clarify that the baselines were indeed tuned, as confirmed by our experiment records. In particular,

(a) SVM: We searched C∈0.1,1,10C\in{0.1,1,10} with an RBF kernel.
(b) XGBoost: We searched tree depths 3,6{3,6}, with 300 estimators, learning rate 0.05, and row/column subsampling of 0.8.
(c) PCA and TS2Vec: Their downstream predictors were selected from three logistic regression and four MLP configurations.

Moreover, all results in Table 4 are evaluated across five team-disjoint folds and five random seeds, following the same evaluation protocol. We have made these details clearer in the revised version (L1911-1915). Thanks again for the careful attention to our baselines.


We sincerely appreciate your thoughtful suggestions and engagement with our work! In light of your feedback, we have expanded the discussion of computational complexity (L345–348), added related work to the main text (Section 6), included larger-scale synthetic experiments (Figure 9, L1598–1603), and provided further experimental details (L1911–1915) in the revised manuscript. Thank you again and we would be happy to discuss any further questions.

Dear Reviewer pK7E,

Thank you again for your thoughtful review and constructive suggestions. We would like to briefly follow up and share a short summary of the main clarifications and revisions:

  • Theory and real-world relevance: Clarified the complementary purposes of synthetic and AMI experiments, and explained how our structural assumptions relate to real-world interactions.
  • Scalability: Conducted additional experiments varying latent dimension, group dimension, and sequence length (Figure 9, L1598–1603).
  • Computational complexity and related work: Added complexity analysis (L345–348) and added the related work discussion into the main text (Section 6).
  • Experimental comparisons: Clarified the main quantitative results (Tables 1–2) and provided further details on baseline tuning and evaluation fairness (L1911–1915).

We would be grateful for any further thoughts you might have after reviewing our responses. Please feel free to let us know if anything needs clarification. Thank you again for your time and consideration!

Best regards,


Authors

Reviewer_wsdnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors study how hidden factors in a time series play different roles: which groups of observations express them, whether they receive influence from the past, and whether they pass influence to the future. Each of these roles can be recovered separately, but only up to a different shuffling of the hidden factors, so combining them can mix up which role belongs to which factor. The authors prove that, under certain conditions, all three roles can be aligned to the same hidden factor at each time step. Based on this theory, they propose RoleTrace, a method that learns a latent representation, estimates the three roles from its derivatives, and refines the coordinates so that all three roles line up. The authors test it on synthetic data and on recorded multiparty meetings, where they use the recovered roles for next-speaker prediction.

Critical strengths
  1. The authors clearly motivate the problem. In Figure 1, the meeting example shows why recovering each role separately is not enough, and why the roles need to be aligned to the same factor.
  2. The theory builds up step by step, and after each theorem the authors explain the assumptions in plain words. In Section 3.3, the authors also claim their joint identification result doesn’t need the path-connectedness assumption that prior work relies on.
  3. In Figure 3, the authors show why joint alignment matters. Aligning on only one role leaves many factors with the wrong combined roles, while aligning on all three recovers most of them correctly.
Critical weaknesses
  1. The synthetic experiments are very small, with only a handful of latent factors, groups, and time steps. The paper gives no evidence that the method scales to larger systems.
  2. In Table 1, the authors evaluate on only 16 small synthetic systems, so the uncertainty in joint accuracy is fairly large relative to the gap between RoleTrace and the closest baselines.
  3. In Figure 4, the authors test sensitivity to the assumptions using the true derivatives from the known generator. This tests the theory, not the actual estimator, so we don’t know how RoleTrace behaves when the assumptions weaken and it has to learn everything from data.
  4. The real-world evaluation is weak. The abstract says the meeting experiments examine the identified roles, but there is no ground truth for roles and the authors never look at what the roles mean in the meetings. Instead, in Table 2, they only use the outgoing role for next-speaker prediction. The gains are small, they disappear as the latent budget grows, and the experiment doesn’t use the other two roles at all.
  5. The assumptions are strong and hard to verify on real data. The theory requires an invertible mapping between hidden factors and observations, enough variability, faithfulness, minimality, and influence only from the previous time step. The authors never discuss whether these plausibly hold for meetings.
Questions for authors
  1. How do you choose the thresholds that turn the role scores into binary roles, and how sensitive are the results to them?
  2. The theory allows a general nonlinear reparameterization, but the refinement step only searches over rotations with greedy pairwise updates. Is there any guarantee that this step finds the right alignment?
  3. How expensive and stable is estimating second derivatives of a learned transition density as the number of hidden factors grows?
  4. How did you adapt ThoughtComm and InterLatent to this setting, and is the comparison fair to them?
  5. Your theory identifies, for each time step, whether each factor is expressed in each group and whether it receives or sends influence, under a permutation that can change from one time step to the next. Can you identify which factor influences which, or follow the same factor across time steps? If not, how would someone use these roles to understand how an idea flows through a meeting?
5 comments

Thank you for your detailed review and valuable suggestions. We appreciate the opportunity to clarify several aspects of our work and respond to your feedback below.


W1. Scalability to larger systems.

A1. Thank you for highlighting the importance of scalability. We conducted additional experiments at more latent dimension NN, group dimension dd, and sequence length TT. Each setting uses eight independent mechanisms and 3,200 training sequences. For brevity, we present the results of RoleTrace alongside the two strongest baselines, with the full comparisons provided in Figure 9 of the revised manuscript.

  • (a) Scaling Latent Dimension. We evaluate the settings from N=4N=4 to N=20N=20:

    NNRoleTraceNo-refitThoughtComm-adapted
    40.844 ± 0.0660.719 ± 0.1200.812 ± 0.091
    60.812 ± 0.0580.667 ± 0.1090.771 ± 0.070
    80.906 ± 0.0390.844 ± 0.0610.828 ± 0.058
    100.775 ± 0.0410.650 ± 0.0570.750 ± 0.065
    120.781 ± 0.0440.802 ± 0.0440.750 ± 0.039
    140.741 ± 0.0380.661 ± 0.0440.625 ± 0.052
    160.766 ± 0.0520.664 ± 0.0620.672 ± 0.054
    180.799 ± 0.0440.771 ± 0.0450.750 ± 0.038
    200.744 ± 0.0260.694 ± 0.0410.706 ± 0.037
  • (b) Scaling observation structures. We further vary the group dimension dd:

    ddRoleTraceNo-refitThoughtComm
    60.812 ± 0.0580.667 ± 0.1090.771 ± 0.070
    90.792 ± 0.0760.708 ± 0.0820.708 ± 0.093
    120.708 ± 0.0760.625 ± 0.0930.729 ± 0.070
    150.708 ± 0.0610.688 ± 0.0660.604 ± 0.070
  • (c) Scaling Sequence Length. We additionally extend the sequence length from T=5T=5 to T=20T=20:

    TTRoleTraceNo-refitThoughtComm
    50.812 ± 0.0580.667 ± 0.1090.771 ± 0.070
    100.896 ± 0.0440.875 ± 0.0610.854 ± 0.058
    150.938 ± 0.0440.917 ± 0.0450.875 ± 0.061
    200.938 ± 0.0440.917 ± 0.0630.917 ± 0.045

Overall, in the aspect of scaling latent dimension, group dimension, and sequence length, RoleTrace achieves the highest joint accuracy in most settings. We would also like to highlight that Figure 6 already examines sample-size scaling over n=50n=50 to 32003200 training sequences, showing substantial improvements in structural recovery as more data become available. Thanks again and we hope the additional results provide further insight into the scalability of RoleTrace.


W2. Statistical reliability of the synthetic results.

A2. We appreciate your careful attention to the statistical reliability of our results. We expanded the evaluation from 16 to 48 independent mechanisms (mean ± SE).

MethodJoint accuracyRoleTrace gain [95% CI]
RoleTrace0.819 ± 0.031-
No-refit0.719 ± 0.034+0.101 [0.066, 0.139]
ThoughtComm0.715 ± 0.033+0.104 [0.062, 0.149]

Paired confidence intervals use 10,000 bootstrap resamples of complete mechanisms. The average joint gain remains about 10 percentage points over both baselines, with both confidence intervals above zero. These results provide strong statistical evidence for the improvement in joint structural recovery.


W3. Sensitivity to identification assumptions with learned representations.

A3. Thank you for highlighting this point. Figure 4 deliberately uses ground-truth derivatives to isolate the effects of weakening the identification assumptions, avoiding confounding from representation learning and transition estimation. To further examine the practical estimator, we conducted additional experiments using the complete RoleTrace pipeline under the same three degradation paths. The parameter γ\gamma ranges from 1 (favorable conditions) to 0 (degenerate boundary).

Sensitivity pathγ=1.00\gamma=1.00γ=0.75\gamma=0.75γ=0.50\gamma=0.50γ=0.25\gamma=0.25γ=0.00\gamma=0.00
Group conditioning1.000 ± 0.0000.700 ± 0.0500.675 ± 0.0530.700 ± 0.0500.575 ± 0.038
Temporal span1.000 ± 0.0001.000 ± 0.0000.950 ± 0.0330.925 ± 0.0380.750 ± 0.000
Faithfulness1.000 ± 0.0001.000 ± 0.0001.000 ± 0.0000.575 ± 0.0530.500 ± 0.000

The results are qualitatively consistent with the sensitivity patterns examined in Figure 4. We appreciate this suggestion, which allows us to complement the analysis with direct evidence from the learned estimator. We have included the corresponding results and discussion in the revised manuscript (Table 5, L1747–1753).

W4. Real-world evaluation and interpretation of structural roles.

A4. We appreciate the reviewer's close attention to our real-world evaluation. Our synthetic and AMI experiments serve complementary purposes: synthetic experiments directly validate structural structural recovery against known ground truth, while AMI examines whether the recovered structural information is useful in real sequential interactions. Next-speaker prediction is a well-established task in multi-party dialogue and provides a natural downstream evaluation of outgoing influence C\mathbb C. Since C\mathbb C characterizes how current latent factors influence future states, selecting outgoing-active coordinates directly connects our structural formulation to this predictive task. The other two roles, A\mathbb A and B\mathbb B, characterize group expression and incoming influence, respectively, and their joint recovery is evaluated in our synthetic experiments. These results demonstrate the practical value of the identified outgoing role for information selection under limited representation budgets. We sincerely hope this helps clarify the motivation and scope of our AMI evaluation.


W5. Assumptions and real-world applicability.

A5. We appreciate your thoughtful concerns about the assumptions underlying our results. Our assumptions follow established principles in nonlinear ICA and causal representation learning (L245-247), while relaxing several assumptions:

(a) We allow dx>Nd_x>N and do not require individual observation groups to be invertible (Theorem 3.1).
(b) Our temporal minimality concerns incoming/outgoing active coordinates, without requiring edge-wise minimality (L224–226).
(c) Our joint identification does not require path-connectedness (L270–277).

For the case where time lag = 1, this is common in temporal formulation to present the main identification results (L119-120). Appendix B.1 also discusses the situation of higher-order temporal dependencies.

For meetings, participants naturally define observation groups (L429-431, L460-463), while conversational variation and temporal interactions motivate the variability and influence assumptions. Faithfulness and minimality formalize when these structural influences are distinguishable.


Q1. How are binary-role thresholds chosen, and how sensitive are the results?

A6. Thank you for this helpful and practical suggestion. RoleTrace uses unsupervised two-means clustering on normalized RMS scores in log space, applied separately to each observation group in A\mathbb A and to B\mathbb B / C\mathbb C (L1512–1515). The midpoint of the two cluster centers determines a data-adaptive threshold, without requiring ground-truth role labels or prespecified support sizes. Unlabeled held-out scores are also included in threshold estimation.

We further evaluated threshold sensitivity from 0.5 to 1.5 across 48 independently sampled mechanisms.

MultiplierJoint accuracyA\mathbb A F1B\mathbb B F1C\mathbb C F1
0.50.733 ± 0.0340.958 ± 0.0060.924 ± 0.0130.907 ± 0.015
0.60.795 ± 0.0280.962 ± 0.0060.941 ± 0.0130.951 ± 0.011
0.70.840 ± 0.0260.966 ± 0.0060.959 ± 0.0110.970 ± 0.008
0.80.847 ± 0.0250.964 ± 0.0060.967 ± 0.0100.976 ± 0.008
0.90.830 ± 0.0280.959 ± 0.0070.970 ± 0.0090.972 ± 0.009
1.0 (default)0.819 ± 0.0310.951 ± 0.0100.973 ± 0.0090.971 ± 0.009
1.10.799 ± 0.0370.943 ± 0.0130.973 ± 0.0090.971 ± 0.009
1.20.778 ± 0.0420.931 ± 0.0160.975 ± 0.0090.971 ± 0.012
1.30.760 ± 0.0460.926 ± 0.0170.978 ± 0.0090.969 ± 0.013
1.40.743 ± 0.0500.921 ± 0.0180.973 ± 0.0100.962 ± 0.016
1.50.729 ± 0.0520.914 ± 0.0200.971 ± 0.0090.962 ± 0.016

The results show a stable region around the default threshold, and the marginal A\mathbb A / B\mathbb B / C\mathbb C F1 scores also remain high throughout this range. Following your suggestion, we have added the threshold sensitivity results and corresponding discussion in the revised manuscript (Table 3, L1605–1616). We are grateful for your attention to this detail.


Q2. Why does refinement use orthogonal transformations when the theory considers general nonlinear reparameterizations?

A7. Thank you for raising this important question. The orthogonal constraint applies only to coordinate refinement, after learning the representation through a nonlinear encoder–decoder. It does not restrict the original observation map. Orthogonal transformations preserve latent geometry and reconstruction, while the same QtQ_t simultaneously transforms the decoder Jacobian and the incoming/outgoing cross-Hessians, directly reflecting the common alignment in Lemma 3.4 (L315–319). Thanks again for giving us the opportunity to elaborate.

Q3. How expensive and stable is estimating second derivatives of a learned transition density as the number of hidden factors grows?

A8. Thanks for pointing out this insightful consideration. An important feature of RoleTrace is that it computes only the required mixed cross-Hessian block, which has a particularly tractable structure under our conditional diagonal Gaussian model (L1505–1508). Thus, the cross-Hessian can be expressed using first-order network Jacobians, with O(N2)O(N^2) entries. To further verify computational cost and stability, we conducted additional experiments across different latent dimensions:

NNReadout (ms)Tensor peak (MiB)Hessian error
40.456 ± 0.0020.1770.199 ± 0.008
121.054 ± 0.0040.4500.272 ± 0.005
201.682 ± 0.0131.0210.293 ± 0.005

Readout times are measured for 256 transitions on a single CPU thread. The computed Hessians agree with the analytic formula within 2.86×10−62.86\times10^{-6} in FP32. More importantly, despite increasing Hessian estimation error, B\mathbb B / C\mathbb C F1 remains 1.000 across all 144 transition fits in ground-truth latent coordinates. Jacobi refinement also maintains orthogonality errors below 1.3×10−141.3\times10^{-14}. These results support numerical stability across dimensions and we greatly value your attention to this issue.


Q4. How did you adapt ThoughtComm and InterLatent to this setting, and is the comparison fair to them?

A9. We appreciate your thoughtful emphasis on fair experimental comparisons. We adapt the core estimation principles of ThoughtComm and InterLatent to a common structural role recovery task, as described in Appendix D.1 (L1521–1523). Specifically:

ThoughtComm: Following its sparsity principle, we perform Jacobi refinement using only the decoder Jacobian objective on a learned representation. We then evaluate A\mathbb A / B\mathbb B / C\mathbb C in the resulting coordinates using our common derivative readout.
InterLatent: Following its joint representation and structural learning principle, we jointly optimize reconstruction, transition likelihood, and A\mathbb A / B\mathbb B / C\mathbb C sparsity penalties using a shared encoder–decoder. The original InterLatent employs a sequential VAE with a temporal prior and includes both decoder and temporal Jacobian regularization. Our adaptation aligns these objectives with the structural role task.

We further verified the fairness of the shared experimental protocol across 128 paired training records: all methods use the same model initializations, data splits, network configurations, training settings, and evaluation procedures. ThoughtComm even reuses the same pretrained representation and initial transition. We believe these comparisons are fair within the shared task and experimental setting. The baselines are explicitly adapted to our identification target, and we hope this clarification helps address your concerns.


Q5. Can the method identify specific causal edges or track the same factor across time? How can the recovered roles help understand meetings?

A10. Thank you for this particularly perceptive question, which touches on a distinctive aspect of our theory. Our central contribution is the joint identification of each latent coordinate's structural signature. Identifying specific factor-to-factor edges is a finer target, studied by previous methods under additional structural conditions (L73-78). Persistent cross-time identity also requires stronger temporal alignment. Appendix A.5 already discusses the distinction between common alignment and global permutation invariance.

It is worth mentioning that the identified roles have clear practical meaning. For example, in a four-participant meeting, a signature ((1,1,0,0),1,1)((1,1,0,0),1,1) indicates a latent factor expressed by the first two participants that both receives past influence and affects future states. Such roles support comparisons of participants' structural participation, changes in role composition across meeting stages, and selection of outgoing-active factors. These group-level summaries are permutation-invariant, so they remain meaningful without tracking a fixed latent identity across time.


We are truly grateful for your thoughtful feedback and engagement throughout the review. In light of your valuable suggestions, we have added experiments on binary-role thresholding (Table 3, L1605–1616), sensitivity (Table 5, L1747–1753), and scalability (Figure 9, L1598-1603), along with further discussions of theoretical assumptions and algorithmic details (L245–249, L318–319, L345-348). Thank you again for your time and consideration! We would be happy to discuss any further questions.

Reviewer_oQyjOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper formalizes a latent variable's structural role in grouped sequential data through three binary support views: the observation groups in which it is expressed, whether it receives delayed influence, and whether it transmits delayed influence. The theoretical contribution identifies group-expression support from decoder Jacobians and incoming and outgoing support from transition cross-Hessians. Because all three constrain the same time-indexed reparameterization, the paper argues that their supports can be aligned under one common permutation. ROLETRACE operationalizes this idea by learning an encoder-decoder and transition density, then applying an orthogonal coordinate refinement that jointly sparsifies the three derivative scores. On 16 synthetic mechanisms, ROLETRACE reports 0.854 joint role accuracy at 3,200 sequences and outperforms several adapted baselines. On 118 AMI meetings, coordinates ranked by outgoing-role score give modestly higher next-speaker accuracy at tight latent budgets. The work is conceptually interesting, but the current evidence does not adequately connect the identifiability theorem to the proposed finite-sample estimator or establish the real-world claim.

Critical strengths
  1. The paper isolates a meaningful correspondence problem that is easy to miss when observation and temporal supports are identified separately. The common-permutation formulation is precise, and the progression from grouped Jacobian support to row and column support of the transition cross-Hessian is coherent. Full proofs and extensions to higher-order and instantaneous dependencies make the theoretical scope unusually transparent.
  2. The synthetic evaluation is structured around recovery, alignment, and sensitivity rather than only downstream accuracy. It uses a single evaluation permutation for all roles, includes ablations for observation-only and temporal-only refinement, and explicitly degrades the span and faithfulness conditions. The AMI experiment also uses team-level folds and keeps next-speaker labels out of representation training.
Critical weaknesses
  1. The theorem and estimator are not yet connected. Identification assumes observational equivalence, general diffeomorphic coordinate changes, global span conditions, faithfulness, and support minimality. ROLETRACE instead learns an approximate autoencoder and diagonal-Gaussian transition, then searches only over orthogonal transformations with fixed sparsity weights and two-means thresholding. No consistency result or diagnostic shows that these restrictions recover the theorem's object. The main synthetic generator is closely matched to the fitted transition family, so it does not resolve this gap.
  2. The real-data evidence is too weak for the applicability claim. The AMI representation is trained to predict future latent states and linguistic content, after which a transition-derived score is evaluated on another future-prediction task. The strongest gains are small: about one accuracy point at k=4 and nearly zero over the density selector at k=8 to 16. Uncertainty is reported across seeds rather than held-out teams, the original five-team test partition is unused, and there is no matched learned bottleneck or ablation removing the future-prediction losses.
Questions for authors
  1. Can you provide either a finite-sample result or controlled experiments under non-orthogonal mixing, transition-family misspecification, and imperfect reconstruction to show when orthogonal joint sparsification recovers the support object identified by the theorem? Please also report diagnostics for the span, faithfulness, and minimality assumptions on the fitted models.
  2. Can the AMI claim be reevaluated on the untouched five-team test split with fold- or team-level confidence intervals, an equally sized generic predictive bottleneck, and ablations that remove future-state and future-content training? This would separate structural-role recovery from selecting features already trained to encode future information.
4 comments

We sincerely appreciate your constructive review. We have carefully considered your comments and provide our point-by-point responses below.


W1. Connection between the theory and estimator.

A1. Thank you for raising this important question. We would like to clarify that RoleTrace is directly guided by our identification theory, as in Section 4:

(a) Structural estimation: Theorem 3.1 identifies A\mathbb A through decoder Jacobian supports, while Theorem 3.2 and Corollary 3.3 identify C\mathbb C and B\mathbb B through transition cross-Hessian column/row supports. Eqs. (11)–(12) directly estimate these objects.
(b) Common alignment: Lemma 3.4 and Theorem 3.5 motivate jointly refining all three structures under the same coordinate transformation QtQ_t in Eq. (13).
(c) Minimality: The theoretical support minimality conditions directly motivate the sparsity surrogate.

Regarding the synthetic generator, the conditional Gaussian family provides a setting with known ground truth to evaluate our central goal of structural recovery and common alignment. The mechanisms are randomly generated, and the latent coordinates, nonlinear observation maps, and structural supports remain unknown to the estimator. Additional robustness results under alternative transition distributions and model misspecification are presented in our response to Q1.


W2. Real-world evidence and statistical reliability.

A2. We appreciate your careful examination of our empirical results. We would like to emphasize that AMI evaluates the downstream utility of the recovered structural roles, complementing the direct structural recovery experiments on synthetic data. In particular, Table 2 compares all selectors under matched latent budgets using the same representation and downstream classifier, ensuring that the future-prediction training is shared across methods. While the gains narrow at larger budgets, the improvements under tighter budgets highlight the value of structural role selection.

We would also like to clarify that our experiments already use team-disjoint five-fold cross-validation (Appendix D.2, L1828–1833). Among 30 teams, each fold uses 20 for training, 4 for selection, and 6 for held-out evaluation. Besides, while seeds and held-out teams capture different aspects of uncertainty, for every seed, accuracy in each fold is already evaluated on teams held out from both training and model selection. We report variability across ten seeds to assess the stability of these held-out results under training randomness. Regarding the ablation, we appreciate the reviewer's valuable suggestions and kindly refer it to our detailed response to Q2. We hope these clarifications provide a better understanding of our empirical findings.

Q1. How does the theory relate to the practical estimator under nonorthogonal mixing, transition-family misspecification, and imperfect reconstruction? What assumption diagnostics are available?

A3. Thank you for these insightful and concrete suggestions. To further examine the connection between identification theory and practical estimation, we conducted three additional experiments. All experiments use 8 paired mechanisms with 3,200 training sequences. For brevity, we compare RoleTrace with the two strongest baselines here, with full comparisons are provided in Tables 6–8 of the revised manuscript.

  • (a) Non-orthogonal mixing. We apply invertible transformations to the learned latents and compose their inverses with the decoder. Rescale and Dense introduce increasing non-orthogonality, with condition numbers κ=2,4\kappa=2,4. | Mixing | RoleTrace | No-refit | ThoughtComm | |---|---|---|---| | Identity | 0.688 ± 0.097 | 0.625 ± 0.103 | 0.583 ± 0.094 | | Orthogonal | 0.688 ± 0.097 | 0.604 ± 0.113 | 0.542 ± 0.112 | | Rescale 2 | 0.521 ± 0.086 | 0.375 ± 0.098 | 0.458 ± 0.088 | | Rescale 4 | 0.438 ± 0.070 | 0.396 ± 0.083 | 0.312 ± 0.086 | | Dense 2 | 0.521 ± 0.091 | 0.417 ± 0.063 | 0.417 ± 0.055 | | Dense 4 | 0.438 ± 0.083 | 0.333 ± 0.045 | 0.375 ± 0.042 |

    Recovery decreases under stronger non-orthogonal mixing, while RoleTrace achieves the highest mean Joint accuracy in all six settings.

  • (b) Transition-family misspecification. To evaluate robustness to transition-model misspecification, we further tested Gaussian mixtures, Student-t, Logistic, Gumbel, and generalized Gaussian innovations. | Innovation distribution | RoleTrace | No-refit | ThoughtComm | |---|---|---|---| | Gaussian | 0.688 ± 0.097 | 0.625 ± 0.103 | 0.583 ± 0.094 | | Gaussian mixture 0.75 | 0.625 ± 0.136 | 0.604 ± 0.126 | 0.583 ± 0.104 | | Gaussian mixture 1.5 | 0.625 ± 0.112 | 0.562 ± 0.113 | 0.542 ± 0.098 | | Student-t (8) | 0.708 ± 0.103 | 0.646 ± 0.124 | 0.583 ± 0.083 | | Logistic | 0.708 ± 0.088 | 0.604 ± 0.118 | 0.604 ± 0.099 | | Centered Gumbel | 0.750 ± 0.077 | 0.688 ± 0.091 | 0.646 ± 0.097 | | Generalized Gaussian (β=4\beta=4) | 0.771 ± 0.083 | 0.667 ± 0.100 | 0.625 ± 0.082 |

    RoleTrace consistently outperforms both baselines across all distributions, with Joint accuracy ranging from 0.625 to 0.771.

  • (c) Imperfect reconstruction. We evaluate checkpoints from epochs 10, 50, and 150 of the same autoencoder training trajectories. | AE checkpoint | RoleTrace | No-refit | ThoughtComm | Reconstruction MSE | |---|---|---|---|---| | Epoch 10 | 0.792 ± 0.088 | 0.667 ± 0.094 | 0.667 ± 0.109 | 0.00872 | | Epoch 50 | 0.729 ± 0.089 | 0.646 ± 0.097 | 0.646 ± 0.091 | 0.000507 | | Epoch 150 | 0.688 ± 0.097 | 0.625 ± 0.103 | 0.583 ± 0.094 | 0.000227 |

    RoleTrace maintains competitive recovery with imperfect reconstruction. Interestingly, lower reconstruction error does not necessarily imply better structural recovery.

  • (d) Assumption diagnostics. We evaluated span, faithfulness, and minimality on the fitted models. | Diagnostic | Result | |---|---| | Group Jacobian full-rank rate | 100% (24/24 groups) | | Outgoing temporal span (rank/support dimension) | 7/36 | | Incoming temporal span (rank/support dimension) | 36/36 | | Incoming edge/non-edge AUROC | 0.954 ± 0.032 | | Outgoing edge/non-edge AUROC | 0.953 ± 0.032 | | Minimality violation rate (A/B/C) | 37.5% / 12.5% / 25.0% |

    (d.1) Span: All group Jacobians have full column rank on their estimated active supports. The outgoing span of the fitted Gaussian is limited to N+1=7N+1=7 dimensions at N=6N=6, while the true transition density achieves full empirical span over its edge directions. (d.2) Faithfulness: The edge/non-edge AUROCs above 0.95 indicate strong separation of temporal influence signals. (d.3) Minimality: We directly compare the thresholded support counts with their true counterparts. The observed violation rates quantify the gap between finite-sample sparsity regularization and exact support minimality.

Thank you again for these insightful questions and we hope this clarification and the additional evidence are helpful.

Q2. Can the AMI claim be reevaluated using the untouched test split, team-level confidence intervals, a generic predictive bottleneck, and ablations of future-prediction losses?

A4. We appreciate your valuable suggestions for strengthening the evaluation. We would like to clarify our existing evaluation protocol and share additional ablation results.

  • (a) Held-out evaluation. As in response to W2, the experiments already use cross validation over five folds, with teams kept separate across training, selection, and evaluation (Appendix D.2, L1828–1833). For every seed, accuracy is evaluated on teams excluded from both training and model selection, and each of the 30 teams serves as an evaluation team exactly once.

  • (b) Generic predictive bottleneck. Our existing evaluation already compares prediction accuracy across different latent budgets (k=4,8,12,16,24k=4,8,12,16,24), with all methods having equally sized representations from the same learned latent space and using the same downstream classifier. This allows us to directly assess the contribution of structural role selection under a common predictive setting.

  • (c) Future-prediction loss ablations. Following the suggestion, we conducted additional experiments removing the future-content and future-state training objectives, individually and jointly. | Representation training | k=4 | k=8 | k=12 | k=16 | k=24 | |---|---:|---:|---:|---:|---:| | Full | 46.635 ± 0.815 | 49.749 ± 0.363 | 50.717 ± 0.273 | 51.051 ± 0.217 | 51.464 ± 0.141 | | Without future content loss | 46.255 ± 1.255 | 49.629 ± 0.628 | 50.452 ± 0.357 | 50.866 ± 0.229 | 51.417 ± 0.140 | | Without future state loss | 45.887 ± 0.997 | 49.358 ± 0.350 | 50.426 ± 0.183 | 50.990 ± 0.256 | 51.302 ± 0.173 | | Reconstruction only | 45.769 ± 0.918 | 49.381 ± 0.336 | 50.299 ± 0.288 | 50.773 ± 0.252 | 51.254 ± 0.131 |

    Both future prediction objectives contribute to the learned representation, with the full model achieving the highest accuracy at every budget. Meanwhile, Table 2 evaluates the additional contribution of structural coordinate selection, with all methods using the same learned representation and downstream classifier.


Thank you again for your time and thoughtful feedback! Following your valuable suggestions, we have further clarified the connection between theory and estimation (L340–345), added experiments on non-orthogonal mixing, transition-family misspecification, and imperfect reconstruction (Tables 6–8, L1764–1813), and included additional AMI ablation results (Table 9, L1877–1882) in the revised manuscript. We hope our responses address your concerns and would be happy to discuss any remaining questions.

Dear Reviewer oQyj,

Thank you again for your careful review and insightful feedback. For convenience, we would like to briefly summarize our responses and additional analyses below:

  • Theory and estimation: Clarified how our identification results directly guide RoleTrace's design (L340–345).
  • Robustness experiments: Conducted three additional studies on non-orthogonal mixing, transition-family misspecification, and imperfect reconstruction (Tables 6–8, L1764–1813).
  • Assumption diagnostics: Evaluated span, faithfulness, and minimality directly on the fitted models (our response to Q1).
  • AMI evaluation: Clarified the existing team-disjoint five-fold evaluation and matched-budget comparisons, and added new ablations of future-prediction objectives (Table 9, L1877–1882).

As the discussion period draws to a close, we wanted to kindly check in in case you have any remaining questions or concerns. We sincerely appreciate your time and would be more than happy to discuss further.

Best regards,


Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.