acceptodds
Under review as a conference paper at ICLR 2027

WHEN SHARING HURTS: A PRE-REGISTERED REFUTATION OF FIXED HIERARCHICAL CREDIT SHARING IN NON-STATIONARY STRUCTURED BANDITS

Abstract

Non-stationary environments challenge learning systems because information that is useful in one regime may become misleading after the underlying process changes. Hierarchical mechanisms offer a natural way to share information across related decisions, but such sharing can also propagate stale or locally inappropriate evidence. We study this tension through a controlled comparison of fixed hierarchical routing and edge-local adaptation under explicitly defined non-stationary regimes. Rather than assuming that hierarchical sharing improves adaptation, we formulate and pre-register a falsifiable hypothesis and evaluate it under a fixed confirmatory protocol. The resulting evidence refutes the preregistered superiority claim: under the tested conditions, the fixed hierarchical routing update does not provide the expected advantage and can perform worse than the edge-local alternative. Additional diagnostic analyses examine how this behavior varies across regimes and experimental conditions, while separating confirmatory evidence from post-hoc interpretation. These results do not establish that hierarchical representations are intrinsically harmful; instead, they identify a concrete failure mode of a particular fixed credit-sharing mechanism under non-stationarity. More broadly, the study illustrates how seemingly beneficial information sharing can become detrimental when adaptation occurs across changing environments, and provides a reproducible framework for testing such claims in adaptive learning systems.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
non-stationary learning, hierarchical learning, credit assignment, adaptive systems, online learning, graph-based learning, concept drift, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_t148OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper compares a fixed hierarchical credit-sharing mechanism with an edge-only learner in a synthetic non-stationary bandit problem. Across 97 paired scenarios, the hierarchical mechanism has 42.1% longer mean recovery time in the primary setting and fails to establish non-inferiority under purely local disruptions. A subsequent audit reveals substantially different action distributions before the disruption, limiting attribution of the result to hierarchical sharing.

Critical strengths
  1. Careful experimental protocol. The reported preregistration, paired scenarios, prespecified outcome, and separation of confirmatory and exploratory analyses are useful safeguards against selective reporting.

  2. Transparent reporting. The authors openly report unfavorable results and acknowledge the limitations of their comparison, including the inability to isolate the effect of hierarchical representation.

Critical weaknesses
  1. Limited scientific informativeness. The methods already behave substantially differently before the disruption, with the hierarchical policy close to uniform in the reported warm-up diagnostics. Without calibration and controlled ablations, the study offers little explanation of whether the recovery disadvantage arises from sharing or other implementation choices. The importance of refuting this particular mechanism is also insufficiently motivated.

  2. Insufficient specification and validation. The manuscript omits essential algorithm updates, hyperparameters, and environment-generation details. Evaluation is restricted to one small synthetic graph and a fixed configuration, with insufficient sensitivity analysis or substantive reporting of the additional baselines mentioned in the paper.

Questions for authors

as above.

Reviewer_WKznOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether a hierarchical learning mechanism that shares information through node-level and edge-level parameters can improve adaptation in a non-stationary structured bandit problem. The authors compare the hierarchical mechanism with an edge-based alternative in a synthetic routing environment with 27 possible paths. Their experiments show that the hierarchical mechanism requires a longer recovery time under the main experimental configuration. Further analysis reveals substantial differences in the action distributions induced by the two methods.

The paper uses a controlled simulation protocol and statistical analysis to document these observations. However, the study does not introduce a new learning algorithm with theoretical guarantees, and its empirical findings are limited to a particular mechanism and a narrowly specified environment.

Critical strengths

The paper makes an effort to conduct a systematic empirical comparison. The use of paired experimental scenarios helps reduce variability arising from environmental randomness, and the authors provide uncertainty estimates for their reported results. The additional analysis of action-selection behavior is also useful for understanding the differences between the two implementations.

These aspects provide some credibility to the reported numerical observations. However, I do not consider them sufficient to compensate for the limited novelty and explanatory power of the study.

Critical weaknesses

1. Insufficient novelty and limited scientific insight.

My primary concern is that the paper does not establish a sufficiently substantial research contribution. The central observation—that sharing information across related decisions can be detrimental when the assumed relationships are inaccurate or the learning mechanisms are not properly calibrated—is already well understood in structured estimation and online learning. The reported outcome therefore does not, by itself, provide a particularly surprising or generalizable insight.

The paper does not propose a fundamentally new learning principle, establish theoretical guarantees, or identify general conditions under which the hierarchical mechanism should outperform or underperform alternative methods. The main contribution is instead an empirical comparison of two specific implementations.

The additive decomposition of edge scores into node-level and edge-specific components is perhaps the most concrete modeling element in the paper. However, its novelty and general applicability are insufficiently established. Such a decomposition resembles conventional additive-effect models, and its usefulness depends on whether the underlying reward structure can be meaningfully represented through shared node contributions and local edge effects.

The paper does not explain why this particular representation is appropriate beyond the chosen routing environment, nor does it investigate how the mechanism behaves under alternative dependency structures, more complex interactions, or different graph topologies.

Consequently, even this modeling component remains insufficiently developed as a general methodological contribution.

2. The experimental evidence is too limited to explain the observed performance differences.

A second major concern is that the experimental design does not isolate the effect of information sharing from other differences between the learning mechanisms.

According to Section 4.2, the compared methods differ not only in their parameter representations but also in their updating procedures, score construction, and parameter constraints. The subsequent analysis confirms that these differences lead to substantially different action-selection behavior.

For example, Table 2 reports normalized path entropies of 0.629 and 0.969 for the edge-based and hierarchical methods, respectively. Their maximum path-selection probabilities also differ considerably (0.468 versus 0.076). These discrepancies indicate that the algorithms operate under very different effective exploration behaviors.

As a result, the reported difference in recovery time cannot be attributed specifically to the proposed information-sharing structure. It may instead reflect differences in score scaling, parameter updates, or the resulting exploration patterns.

Furthermore, the experiments rely on a single small graph family, a fixed disruption magnitude, and a limited set of algorithm configurations. There is no systematic component-wise analysis that separates the effects of node parameters, edge parameters, regularization, and action selection. Nor is there sufficient evidence showing that the conclusions persist across substantially different structured decision problems.

The observed performance difference may therefore be specific to the implementations and configurations being compared.

More importantly, the paper does not establish a general mechanism explaining when hierarchical information sharing is useful, when it becomes harmful, or how its effectiveness depends on the underlying problem structure.

Without such an explanation, the findings remain primarily descriptive rather than providing a substantial advance in our understanding of structured online learning.

Questions for authors

1. What is the specific methodological novelty of the node–edge parameterization?

The hierarchical mechanism represents each edge score as the sum of node-level and edge-specific effects. How does this formulation differ substantively from existing additive or hierarchical parameter models? What properties of the underlying reward structure justify this representation, and under what conditions should it be expected to generalize to other structured decision environments?

2. Can the observed performance difference be attributed specifically to hierarchical information sharing?

The two methods exhibit substantially different action-selection distributions. Could the authors provide controlled comparisons that separate the effects of parameter sharing from score scaling, update rules, regularization, and exploration behavior? In particular, does the reported disadvantage persist when these other components are made comparable?

These clarifications would materially affect my assessment because they concern whether the paper establishes a general scientific insight rather than documenting the behavior of two particular implementations.

Reviewer_VRJJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper was trying to address the problem of sharing information across environments.

Critical strengths

The paper had not strengths

Critical weaknesses

This was a very hard to read and not understandable paper. It does not articulate in an understandable way the problem it is trying to solve and was not able to understand the contribution. At best the contribution is a small controlled synthetic experiment trying to address a question that is not sufficiently well motivated, or whose motivation and real world relevance is not stated in an understandable way.

Questions for authors

Why is your experiment practically relevant or representative of some real world problem.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.