acceptodds
Under review as a conference paper at ICLR 2027

Colosseum of Mechanisms: A Unified Framework for Unsupervised Multi-Mechanism Learning via Quasi-Gradient Descent

Abstract

An unsupervised multi-mechanism model has mechanisms and observations. Fitting it optimally means an intractable search over assignments. The \`divide-and-conquer\' (DaC) approach assumes a segmentation form and searches only the assignments that form expresses: tractable, but broken when the assumption is incorrect. We propose the Colosseum of Mechanisms (CoM), a \`conquer-then-divide\' framework: it fits the parameters first, then reads off the segmentation. At every observation, CoM stages a competition among all mechanisms, then aggregates only the winners. CoM assumes no segmentation form and therefore accommodates any. Optimizing the CoM objective solves the combinatorial search without traversing those assignments. An algebraic identity makes this possible. This identity assumes nothing about the noise, nor the correctness of the supplied mechanism forms. But the combinatorial difficulty does not disappear; it reappears as nonconvexity and nonsmoothness in . Our Quasi-Gradient Descent algorithm guarantees -Clarke stationarity in iterations, exactly what standard gradient descent achieves on a single smooth function. Even the simplest CoM case is NP-hard to minimize. We therefore give a necessary condition for global optimality. In DaC's best case, CoM matches its accuracy about 1000 faster. In a non-ideal DaC case, CoM's accuracy and speed are unchanged, while DaC loses eleven accuracy points and runs slower still. On U.S. GNP data, CoM halves the out-of-sample forecast error of a published benchmark. CoM also reconstructs entangled high-frequency dynamics in EEG.

Primary area:
optimization
Keywords:
Multi-Mechanism Learning, Non-convex Non-smooth Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qCWkOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a Colosseum of Mechanisms (CoM) framework, which is a conquer-then-divide approach. They introduce Quasi-Gradient Descent algorithm that is doing non-convex optimization to fit the parameters of the model, given that the forms of JJ (possible functions) are specified. They that their method is faster than competitors and is better than ‘divide-and-conquer’ (DaC) approach unless the segmentation form is assumed correctly for the ‘divide-and-conquer’ (DaC) approach

Critical strengths
  1. Strong empirical improvements and diverse evaluations - section 5.1 evaluates a toy setup, section 5.2 evaluates the forecasting setup, section 5.3 provides a real data evaluation example (EEG)
  2. Rigorous theoretical background: Appendix provided step-by-step proves and the paper also clearly references textbook facts for a more naive reader
  3. Decently clear writing: for example, I really liked Section 2 with the toy explanation of the problem and solution - it is very nice and clear
  4. Reproducibility: Code is attached for submission
Critical weaknesses
  1. Need of a simpler into and occasional lack of notations:
  • A paper systematically skips rigorously telling what letter means what (for example what is RR in eq. 3 - is it region-wise loss objective? what is PP in the same equation? is it a specific assignment or data partition? same could be pointed through a majority of equations throughout the text).
  • The paper introduces some terms / jargons without explaining them properly or explaining them too late - for example, Clarke subdifferential is mentioned several times before being introduced even on a high level. The abstract also mentions "multi-mechanism model" and "segmentation form", which are field-domain jargons, which a naive reader might be unaware of.
  • Some simple schematics could improve the readability - for example, in section 5.2 it is not completely obvious without looking in the appendix that the current rule for yty_t is decided by the previous predictions with dd steps window, a small schematics could help to understand it
  1. Sensitivity analysis is not provided: - non-convex optimization methods, performance remains sensitive to initializations but no analysis on this is provided. Some practical recommendations on initializations and amount of needed refits would also help the practitioners.
  2. Empirical EEG analysis could be improved: - see questions 1-3.
  3. All of the evaluations are based on the linear forms: I am interested in evaluations where candidate mechanisms are non-linear in their parameters, since there the non-convex optimization might become even harder and sensitivity to initialization even stronger, so its interesting how well CoM could still perform
Questions for authors
  1. In section 5.3 - was J=2J = 2 or not? It could have been more straightforward in the writing as well that JJ is a set of seizure / no-seizure modes and NN is the EEG rhythms bands
  2. Why do you report RMSE in section 5.3 if in fact you are not doing forecasting but state classification? Could you please report the classification numbers there and also add some baselines (like supervised classification based on extracted features from the random forest)? I would also inspire you to do the forecasting setup for the EEG data (same way as you did section 5.2) since its predicting seizures is a more relevant real life problem than classifying them post-hoc.
  3. Could you please provide error-bars for table 3?
  4. The noise in section 5.1 is not really heteroscedastic - it is homoroscedastic depending on the branch. Do you think the differences in noise amplitudes could help CoM to distinguish the regimes? Could you please provide the experiment with the same noise level and with more complex heteroscedastic noise, when noise depends on x value? What about different signal-to-noise ratios? How depended it CoM on the SNR?
  5. Could you please show how the GT classification looks like for Fig2? Also for CoM in both Fig1 and Fig2 there is a mix of blue and red points on the intersection of the lines - could you please comment why does it happen and how could this be solved / imporved?

Clarity:
6. what does TAR stays here for?

Reviewer_wSZ2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes Colosseum of Mechanisms (CoM) for learning multiple parametric mechanisms without assuming a segmentation rule. Its objective, ℓ(θ)=∑imin⁡jℓˉj(θj;xi,yi)\ell(\theta)=\sum_i\min_j\bar\ell_j(\theta_j;x_i,y_i), equals the partition-based objective minimized over all assignments. The paper derives Clarke subdifferentials and a Quasi-GD algorithm with an O(ε−2)\mathcal{O}(\varepsilon^{-2}) stationarity guarantee under smooth component losses. It also proposes a refitting-based necessary condition for global optimality. Experiments cover synthetic regression and switching models, GNP forecasting, and EEG reconstruction, reporting competitive accuracy and substantial speedups.

Critical strengths
  1. Useful convergence analysis. The exact Clarke sum rule (Theorem 3.4) and smooth majorant support an explicit O(ε−2)\mathcal{O}(\varepsilon^{-2}) stationarity bound (Theorem 3.7; Corollary 3.8) for the nonsmooth, nonconvex objective.
  2. Unified formulation. Profiling out assignments allows the same objective to handle different switching patterns. The oblique-split example (Sec. 5.1, Table 2) illustrates the benefit of avoiding a fixed threshold assumption.
Critical weaknesses
  1. The experiments do not isolate the proposed algorithm. In the supplied optimization loops, updating min_loss immediately before checking current_loss >= min_loss triggers an OLS refit attempt every iteration, rather than only when progress stalls as the README describes. Successful refits overwrite the parameters before a cluster-normalized gradient step (e.g., SBR_compare_vertical_form.R, L269–320). This is a hybrid algorithm, not standalone Algorithm 1. On the supplied vertical-split replicate with seed 22, assignment/refitting alone reaches the saved final loss of approximately 13820.34. Removing OLS while retaining the script's gradient step causes divergence, but that step differs from Algorithm 1 and does not establish failure under its step-size assumptions.

    Table 2 lacks a standalone assignment/refitting baseline, leaving the source of the reported speed and accuracy unclear. Relevant precedents include Späth (1979), Clusterwise linear regression and Manwani and Sastry, K-plane regression, Sec. II. The paper already cites both; the issue is the missing comparison. Remark 3.10 also misleadingly associates Späth's heuristic with brute-force enumeration. Resolving this central empirical gap requires separating faithful Quasi-GD, refitting alone, and the implemented hybrid.

  2. The local-optimum example and forecasting protocol do not establish the claimed conclusions. Theorem 4.1 is a valid necessary condition, but its ability to reject genuine suboptimal local minima is overstated for convex fixed-partition losses. Since ℓ(θ)≤R(θ;P∗)\ell(\theta) \leq R(\theta; P^{\ast}), with equality at θ∗\theta^{\ast}, any local minimum θ∗\theta^{\ast} of ℓ\ell is also a local minimum of R(⋅;P∗)R(\cdot; P^{\ast}). If this fixed-partition objective is convex, that minimum is global; if its minimizer is unique, refitting returns θ∗\theta^{\ast} (Appendix G). Moreover, evaluating Table 1's “Local Optimum 1” using CoM_bad_inital.Rmd (L321–323) gives no ties and a gradient norm of approximately 132,854, contradicting the stated stopping condition (L355–356). The example and interpretation need correction.

    Sec. 5.2 reports test MSE 0.0095 versus 0.0224, but does not explain how one forecast is selected before observing yty_t. If evaluation uses min⁡j(yt−y^t,j)2\min_j(y_t-\hat y_{t,j})^2, it uses the test target to select the regime and cannot establish causal forecasting performance. This remains unverified: the supplied supplement lacks the GNP and EEG scripts despite the reproducibility statement (p. 10). An explicit forecasting rule using only information available before tt, with supporting code, is needed to validate Table 3.

Questions for authors
  1. for W2: How is the single GNP forecast formed before observing yty_t? Does Table 3 use target-dependent minimum residuals or a rule based only on past observations? Where are the GNP and EEG scripts?
  2. for W1: Is refitting every iteration intentional, and which implementation produced Table 2? Is there existing evidence separating standalone Algorithm 1 from refitting alone and the hybrid?
  3. for W2: What stationarity measure and tolerance accepted Table 1's “Local Optimum 1”? How should the diagnostic's claimed scope be reconciled with the convex fixed-partition argument above?
Reviewer_mo2jOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Colosseum of Mechanisms (CoM), a unified framework for unsupervised multi-mechanism learning without explicitly specifying a segmentation rule. The framework minimizes a sum of pointwise minimum losses across candidate mechanisms, and the authors establish its equivalence to optimization over all possible assignments. To optimize this nonsmooth and nonconvex objective, they introduce Quasi-Gradient Descent (Quasi-GD), derive a Clarke subgradient-based update, and establish an O(ε⁻²) stationarity guarantee. They also provide a necessary condition for global optimality based on region-wise re-estimation.

The method is evaluated on synthetic structural-break regression, threshold autoregression, and ICA-related tasks, as well as real-world GNP forecasting and EEG reconstruction. The reported results suggest competitive fitting accuracy and substantial computational advantages over the selected segmentation-based baselines.

Critical strengths
  1. The paper provides a coherent optimization perspective for multi-mechanism learning without prescribing a segmentation structure. The majorization-based analysis of the nonsmooth objective is potentially useful, particularly for understanding the convergence behavior of hard-assignment learning methods.

  2. The experiments cover structural breaks, threshold autoregression, ICA-related models, and real-world signals. The results indicate that the proposed objective can fit data under different switching patterns without explicitly modeling their segmentation rules. This demonstrates the potential applicability of the framework, although the evidence does not yet establish the superiority of the proposed optimization algorithm.

Critical weaknesses
  1. The proposed objective closely resembles existing hard-assignment methods, such as K-means and K-plane regression. Quasi-GD reduces to standard hard-assignment gradient descent when each observation has a unique best-fitting mechanism, and its distinct advantages remain unclear.

  2. Theorem C.1 (Eq. 52) does not hold under its stated assumptions, while Appendix F.3 derives an iteration complexity inconsistent with Theorem 3.7. The claimed overall linear computational complexity is therefore insufficiently justified.

  3. The experiments primarily compare against segmentation-based methods while omitting closely related approaches such as hard-assignment gradient descent, alternating minimization, and K-plane regression. Thus, the results do not establish the specific advantages of Quasi-GD.

  4. The complex TAR implementation initializes model parameters using ground-truth coefficients, and its delay and noise configurations differ from those reported in the paper. These issues undermine the validity and reproducibility of the parameter recovery results.

  5. The GNP experiment does not explain how mechanisms are selected during forecasting without accessing the true target values. Moreover, the supplementary code lacks the GNP and EEG experiments, preventing independent verification of the reported results.

Questions for authors

See Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.