acceptodds
Under review as a conference paper at ICLR 2027

Connectivity, Distractors, and the Speed of Learning

Abstract

In deep learning, neural network architectures are typically fixed at the start of training, allocating each unit a predetermined set of inputs. Since that set is fixed independently of what the unit learns, some of its input connections may end up being useless, carrying no information about the unit's target. We study how these distractor connections affect the speed of learning. Drawing from theory in the linear setting, we characterize how updating the weights of such connections slows learning, then investigate the extent to which this phenomenon affects neural networks. To measure this, we construct a problem in which good connectivity is known in advance, allowing good and bad connectivity patterns to be compared directly. We find that bad connectivity slows learning and is especially costly in non-stationary problems, where learning speed determines performance. Scaling network width in this regime cannot solve the problem, and beyond a point worsens rather than improves performance. Learning connectivity has been pursued mainly for parsimony; our results suggest that it could also lead to faster learning, further motivating the use of methods that learn connectivity.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
continual learning, deep learning, representation learning, distractors, connectivity, credit assignment

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_KQRgOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether distractor connections slow neural-network learning. Motivated by classical linear LMS analysis, the authors construct multi-MNIST, where independent classification problems are combined and useful connectivity is known in advance. Experiments compare dense and block-sparse networks under stationary and changing label mappings. The central empirical finding is that dense connectivity impairs adaptation in the non-stationary setting, and increasing network width does not eliminate this gap within the tested range.

Critical strengths
  1. Studying connectivity from the perspective of learning speed, beyond parameter efficiency, is worthwhile
  2. The controlled construction makes relevant and irrelevant connections identifiable, enabling informative comparisons.
Critical weaknesses
  1. The practical relevance of the setting remains limited. All experiments use constructed independent subproblems with known ideal connectivity and, in the non-stationary setting, abrupt label permutations. Real tasks often involve correlated inputs and shared representations, where useful and distractor connections are less clearly separated.
  2. The theoretical justification is limited to the linear setting. Section 2 motivates the argument through classical LMS analysis, but its extension to nonlinear networks with evolving representations is primarily heuristic.
  3. The experiments do not fully isolate the proposed mechanism. In Section 4.2, enabling additional connections changes optimization geometry and potentially stability, even though initial predictions are preserved. The transition experiment does not rule out those optimization effects.
  4. Empirical results are conducted on the MLP architecture and MNIST dataset only, which limits the generality of conclusions.
Questions for authors
  1. Does the distractor effect persist when subproblems share useful features?
  2. Does the principal connectivity gap persist with tuned alternative optimizers?
  3. Why are the step sizes chosen on a very small scale? Does increasing those step sizes validate the argument of distractors slow down learning?
4 comments

Thank you for your review!

W1, W4, Q1: Your main concern seems to be how far our results generalize to other problems (W1), architectures, and datasets (W4). We address these together.

We too want to understand how large a role distractor connections play across problems. But isolating their cost requires either knowing which connections are distractors or having an algorithm that reliably removes them, neither of which is available for general problems. The methods we test, SET and DEEP R, barely find the structure even in multi-MNIST, where the problem is as clean as it gets (Section 4.4). Had we tested real problems with these methods, we might have falsely concluded that distractor connections are unimportant simply because the algorithms failed to remove them. This is why we constructed a problem where good connectivity is known in advance, allowing us to study its effects directly.

However, we can still study which types of problems would be most affected. Our experiments vary properties of the problem, specifically the number of independent predictions (Figure 3) and the frequency of non-stationarity (Figure 5, added in the revision). We find that distractors become increasingly costly when a network must learn more independent things and adapt to more frequent changes. Together with the mechanism in Section 2, these results explain when and why unnecessary connectivity slows learning, beyond this particular problem. The mechanism is also not specific to abrupt changes, as the linear analysis in Section 2 considers gradually drifting targets.

Our broader motivation is to improve algorithms that learn many predictions and/or policies from a single datastream in parallel, as in Sutton et al., 2011, Veeriah et al., 2018, and Modayil & Abbas, 2023. As your review points out, some of the things a network learns may benefit from shared representations, while others require different information. This distinction is present in multi-MNIST (Q1), as the ten outputs within each classification benefit from shared features but not from the features of other subproblems. We deliberately chose classification instead of regression because it naturally has this property.

This motivation also informs how we think about your point on architectures (W4). The mechanism we propose is not specific to any particular architecture, but different architectures may contain different numbers of distractor connections. CNNs, for example, may avoid some unnecessary connections by exploiting known spatial structure. Understanding which existing architectures best avoid distractor connections is an interesting question, but it is not the one motivating our work. Our goal is to understand and ultimately address the problem when useful connectivity cannot be specified in advance. This is particularly relevant to agents that pose their own learning targets (e.g. Veeriah et al., 2019), where the designer cannot anticipate which features each prediction will need. In such settings, we cannot rely on inductive biases to determine architectures; the agent must learn which connections are useful for itself.

We realize this motivation should be clearer in the paper and will revise the introduction accordingly before the discussion deadline.

W2: We agree. Our aim is not to theoretically characterize the complete non-linear setting, nor do we claim to. The linear analysis is there to explain the mechanism and make predictions, and the experiments are there to test them in networks.

W3: The changes in optimization caused by unnecessary connections are precisely what we are investigating, so we are not sure which alternative mechanism you have in mind. We account for differences in step-size sensitivity by tuning the step-size independently for each method, and the performance gap persists without divergence. We would welcome clarification of any specific concerns about optimization geometry.

Q2: We are running Adam and RMSProp on the Figure 1 experiment and will report the results here when they are completed. As we discuss in Section 5, we expect momentum to reduce the gap somewhat because it averages out some of the noise, but not to remove it.

Q3: Step-sizes are independently tuned through sweeps for every method and setting, so increasing the step-sizes for any method in the paper would result in worse performance. The best step-sizes for dense networks are smaller than those for block-sparse networks, which is consistent with our explanation that distractors increase the cost of larger updates, resulting in smaller optimal step-sizes.

We hope these revisions and responses address your concerns and would appreciate it if you would reconsider your score. We are happy to address any remaining questions.

AuthorsOpenReview ↗

Update on Experiments

We have completed the optimizer experiments promised in response to Q2 and added them to Appendix D. We tune the step-size separately for each network and optimizer. Adam and RMSProp perform worse than SGD for both dense and block-sparse networks.

We also test batch sizes from 1 to 32 with a fixed sample budget. The connectivity gap persists across these batch sizes.

Appendix D contains further discussion on why neither of these modifications help.

Thank you for the response. It partially addressed my concerns, in particular, it clarified the motivation for the controlled benchmark and the choice of step-sizes. Several concerns remain:

  1. I understand the risk of falsely concluding that distractor connections are unimportant in real problems. However, even in the controlled setting with multi-MNIST, within n samples, there might be some samples that share useful features (or even they have the same label). So, in that case, do distractors remain harmful?
  2. Regarding the transition experiment, preserving predictions at the transition does not isolate the proposed noise mechanism. In particular, adding trainable connections also changes the optimization dynamics and available gradient directions. Tuning step-sizes does not rule out those effects.
  3. I appreciate the authors for running additional experiments with other optimizers. In Figure 6, training a dense network with RMSProp shows better performance than block-sparse trained with RMSProp even though the performance is bad in both cases (around 50%). This somewhat contradicts the claim in this experiment.

Please clarify if I misunderstood something.

1. We can see two possible meanings to your question, and I’ll try to answer both. Our first interpretation of your question was that different images within a sample could benefit from the same feature detector (i.e. a detector for the number 8). CNN-style weight sharing allows the same detector to be applied separately to different images because they generate multiple activations with a single set of weights, but our networks do not use weight sharing. In our case, even if there is neuron that detects when an image is an 8, it would have to send that signal to all of its outgoing connections, which would be problematic given that images for each subproblem are sampled independently. We therefore expect the benefit of this kind of sharing to be limited in our setting. More likely, we think your question is referring to the case where multiple subproblems genuinely benefit from the same feature activations. For example, suppose two of three subproblems always use identical images. Those two could share useful features, but connections between them and the independent third subproblem would still be distractors. There would be fewer distractors than if all three subproblems were independent, so our mechanism predicts a lower overall cost from distractors. You could imagine a spectrum where with all problems independent there would be the highest number of distractors, which would cause the most issues. And if all of the subproblems were identical there would be almost no distractor connections, so they would cause no issues. The main point is that useful sharing and harmful distractors can coexist. Sharing can reduce the number of distractors, but connections carrying irrelevant information can still cause issues. We expect the common case to fall somewhere between complete sharing and complete independence, with the cost depending on how many distractor connections remain.

2. We agree that our previous claims were too strong and not specific enough, and we have weakened this a little in the newest revision (line 269). The transition experiment shows that adding distractor connections prevents the network from maintaining its previous performance, even when it starts from the block-sparse solution. Thus, the gap cannot be explained solely by a failure to find that solution. However, we agree that that does not generally tell us everything about how the relevant optimization dynamics that affect learning.

3. We expect distractor connections to cause problems with other SGD-based optimizers as well, but that does not mean a network with fewer distractors will always perform better, since other optimizer effects can also influence performance. We do not know why RMSProp performs so poorly in both networks, or why block-sparse is affected more. The decline after an initial peak in block-sparse might suggest loss of plasticity, but at this point that is just a hypothesis. I wish we could offer a more satisfying answer, but understanding the interactions other optimizers add goes beyond the scope of what we explored in our experiments.

Reviewer_mo5ZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies distractor connections: connections that carry no information useful to the receiving unit, but exist because the architecture is fixed before training (e.g., fully connected layers). The authors argue that because these connections keep receiving credit and being updated even though their optimal weight is zero, they inject noise and slow learning. They motivate this with established results on LMS learning for a single linear unit, where distractor inputs shrink the optimal step-size. They then introduce multi-MNIST, a problem composed of independent subproblems whose ideal connectivity is known by construction, allowing them to compare dense networks against block-sparse networks with ground-truth connectivity. In a stationary setting, dense networks match block-sparse ones once their width is matched (while having more parameters). In a non-stationary setting, where one subproblem’s labels are periodically permuted, dense networks never catch up and degrade beyond a certain size. Additional experiments add zero-initialized dense connections to a trained block-sparse network (dense-transition) and block backward gradients through them (forward-only variant), attributing the slowdown roughly equally to forward noise and backward interference. The gap grows sharply with the number of subproblems. Finally, two connectivity-learning methods (SET and DEEP R) are shown to close less than half of the gap to block-sparse connectivity.

Critical strengths

The paper is well written and easy to follow, and uses established theoretical results to motivate the problem and the experiments. The experimental setup is intuitive and well designed for the main question, exposing the right knobs (model capacity and task complexity) and allowing for a clean comparison against known ground-truth connectivity. Comparing stationary and non-stationary versions of the same problem is a useful control: in the stationary case the dense network eventually matches block-sparse, showing it has sufficient capacity, so its deficit in the non-stationary case can be attributed to learning speed.

Critical weaknesses

Novelty: While I like the setup and the experiments, I don’t find the results that surprising or novel. The lottery ticket line of work already shows that sparse subnetworks with the right connectivity exist and perform as well as dense networks. In particular, the original paper (The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks) reports that winning tickets learn faster than the original dense network, not just match its accuracy. Faster learning is therefore expected once you know the redundancy exists and can identify it. Other works point in the same direction, even if framed differently. In [1], the authors show that redistributing a fixed parameter budget across more neurons with disjoint sparse supports substantially improves performance, most strongly when the split follows the ground-truth structure of the problem, but even with random splits. This is the same axis as the paper’s equal-parameter comparison in Section 4.1, where block-sparse networks have n× more hidden units than dense ones, and [1] attributes the gain to reduced interference between features that share neurons, a mechanism close to the backward-interference explanation given here. Consistent with [1]’s random-split result, the static sparse baseline in Figure 5 also clearly outperforms the dense network. Both lines of work consider stationary settings, so the main extension here is to the non-stationary case, but based on the above, the finding that bad connectivity slows learning there I feel is expected.

Also, the non-stationary benchmark comes from work on loss of plasticity (Elsayed & Mahmood, 2023), and the paper would benefit from discussing how its findings relate to that literature.

The paper identifies the problem but doesn’t make progress on solving it: The hard and interesting problem, and what I would find most valuable, is how to learn good sparse connectivity, which the authors seem to agree with. In this direction, they test only two simple structure-learning methods, which improve over static sparse connectivity but still lag well behind block-sparse. Several directions the paper itself proposes are left untested:

  • RigL and SNFS are discussed as likely to add fewer distractors, but neither is evaluated.
  • Step-size optimization methods (IDBD, Autostep) are described as “well-suited to the problem,” but none is tested, even though they are cheap to run on two-layer networks.

Setup: All experiments use two-layer MLPs trained with single-sample SGD on a single problem family with an artificial form of non-stationarity (label permutation). It is unclear how the findings transfer to settings with batching, adaptive optimizers, normalization, or more natural non-stationarity.

Overall, for the paper to be a strong contribution, I think it needs to take concrete steps toward the problem it identifies, since the identification itself is relatively established.

[1] Expand Neurons, Not Parameters (https://arxiv.org/abs/2510.04500)

Questions for authors

See weaknesses.

3 comments

Thank you for your review! We have split the review into two comments because your review is thorough, and we want to properly address the points you bring up, especially on novelty.

We agree that prior work shows connectivity can affect learning speed. What remains unclear is why unnecessary connectivity slows learning, how severely it can limit continual adaptation despite ample capacity, and how to isolate these effects in neural networks. These are our three contributions. We restate and provide context on each below.

1. We provide a more complete explanation of why unnecessary connectivity slows learning. The Lottery Ticket Hypothesis demonstrates that certain sparse networks learn faster than dense ones, but does not establish why. The reviewer also points us to [1], which explains how sharing neurons between features causes interference, directly relevant to the backward interference we study. We appreciate this connection and will incorporate it into our discussion. However, interference is only part of our explanation. We also identify the noise that distractor connections inject into the forward pass, and experimentally separate these two effects in Figure 2, finding that both contribute substantially to slower learning. We develop this explanation by connecting classical theory of misadjustment in linear learning (Widrow et al., 1976) to modern neural networks, where unnecessary connections can also interfere with learning in earlier layers. Bringing these mechanisms together and demonstrating their effects experimentally is a novel contribution.

2. We establish that unnecessary connectivity can severely limit continual adaptation, in ways not apparent from prior work. We agree with the reviewer that several findings in [1] are consistent with ours, including its explanation for why even random sparse connectivity can outperform dense connectivity. However, the papers address different questions and settings, and there is an important difference our their findings. In Figures 2 and 4 of [1], the benefits of sparse connectivity diminish as network width increases. In our non-stationary setting, however, increasing width does not overcome bad connectivity. Our Figure 3 shows that dense networks cannot match good connectivity at any width tested, and beyond a point, adding more neurons actually makes performance worse. This is not an obvious result, particularly given the substantial literature showing that wider or larger networks often learn faster or more efficiently in stationary settings (e.g. Ardalani et al., 2018; Sankararaman et al., 2020; Kaplan et al., 2020). Those findings make increasing capacity a natural way to improve learning, but do not establish what happens when the problem continually changes. Our results show that poor connectivity can impose a limitation that additional capacity cannot overcome, and can even make worse. This is particularly important for continual learning, where learning speed remains a persistent determinant of performance rather than becoming irrelevant after convergence.

3. We introduce a problem that makes these effects directly measurable. On general problems, identifying which connections are distractors is difficult, making it hard to isolate their cost. Multi-MNIST addresses this by providing known good connectivity against which other connectivity patterns can be compared. It also provides a clear reference for evaluating connectivity-learning algorithms. For example, although SET and DEEP R improve performance, we find that they recover remarkably little of the underlying structure. Without knowing the correct connectivity, we would not realize how much room these methods still have to improve. Loss of plasticity: We considered this possibility, but Figure 1 suggests that neither network exhibits loss of plasticity. We have added a sentence to Section 4.1 of the revision to clarify this.

Loss of plasticity: We considered this possibility, but Figure 1 suggests that neither network exhibits loss of plasticity. We have added a sentence to Section 4.1 of the revision to clarify this.

We agree that learning good connectivity is an important next step, but believe that understanding the problem and establishing a way to measure progress are valuable contributions in their own right.

W2 (Progress toward solving the problem): We chose SET and DEEP R because they are simple, effective, and scalable methods for learning sparse connectivity. RigL and SNFS may introduce fewer distractor connections, but both rely on dense-gradient information, making them less suitable for the extremely large, sparse networks that motivate our work.

IDBD and Autostep were developed for linear learning and cannot be applied directly to our nonlinear networks. Developing effective per-weight step-size adaptation for neural networks is itself an open research question. When we describe step-size adaptation as "well-suited," we mean the general approach of controlling how much individual weights change, not that these specific algorithms are directly applicable.

We have also explored several simple extensions to SET, with encouraging preliminary results. One variant closes 88% of the accuracy gap between dense and block-sparse connectivity on our non-stationary problem. We initially left this work out to keep the paper focused on understanding the problem rather than proposing a new algorithm. However, given this feedback, we are conducting a more thorough evaluation and plan to include the results in an appendix. We will follow up once the evaluation is complete.

W3 (Setup): We address generality in more detail in our response to Reviewer 3 (W1, W4, Q1). In brief, multi-MNIST allows us to isolate the cost of distractors, while varying the number of independent predictions (Figure 3) and frequency of non-stationarity (Figure 5) helps identify the types of problems where connectivity matters most. Different architectures may avoid some distractors by exploiting known structure, but our interest is in settings where that structure is unknown. Our linear analysis also considers gradually drifting targets, beyond abrupt permutations.

We are additionally testing batching, Adam, and RMSProp. Preliminary results show these changes hurt rather than help. We will include them in the next revision. We have not tested normalization, but do not expect it to eliminate the noisy credit assignment to distractor connections.

We hope our discussion clarifies the novelty of our contributions relative to prior work, and that the additional experiments will help address your remaining concerns. We would appreciate it if you would reconsider your score, and are happy to address any follow-up questions.

AuthorsOpenReview ↗

Update on Experiments

We have completed the additional experiments promised in our response.

Section 4.4 now includes Continual SET, which tracks connection utility over time instead of pruning by current weight magnitude. This improves accuracy from 62% to 78.2%, closing 87% of the gap between the dense and block-sparse network performance. Appendix B provides a description of the method.

Appendix D now reports the optimizer and batch-size experiments. We test Adam and RMSProp, both of which perform worse than SGD on this problem. We try larger batch sizes of up 32 and find that any increase to the batch size decreases performance. We discuss why neither of these changes help in the the Appendix.

Reviewer_Um4hOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper focuses on distractor connections: connections that carry no useful information for the receiving unit and may lead to slower learning. The authors examine this in a controlled setup, comparing a network with good connectivity (block-sparse) to a dense network that contains useless connections. The results show a performance gap between the two cases, especially in non-stationary problems, along with a lower-optimal step size. The authors also address alternative explanations for the gap such as network capacity and loss landscape.

Critical strengths

The dense transition result in section 4.2 shows that adding zero-initialized connections degrades the performance, this strengthens the claim that the added useless connections are responsible for the performance gap and addresses the alternative explanation of the loss space difference.

Critical weaknesses
  1. The main claim that distractor connections slow learning is only partly supported. The results show that dense connectivity gives worse performance compared to block-sparse, but learning speed was not measured directly but inferred from the accuracy measure and from the optimal step sizes. For example, what happens to the network within permutations? How long does it take the network to recover in each case? How were the number of steps between permutations chosen? Were other options tested for comparison?

  2. It’s not clear how relevant the results are to more complex architectures and to real-world data. Multi-MNIST is an ideal case for the question, as it’s a simple case where each subproblem is largely independent and many connections are purely distractors by design. Do the results also hold for data with more shared structure?

Questions for authors
  1. What is the relevance of figure 1, as it shows the performance gap is due to capacity and not distractor connections?

  2. Figure 2 shows the accuracy gap only in the non-stationary case, but the authors claim that dense connectivity has an effect in both settings. Can the authors show additional support for this?

  3. In Figure 3, what happens if you add a dense transition control that starts with the block-sparse optimal step-size of 2^-8, then lower it to 2^-11 when the dense connections are added?

  4. Some experimental details are missing for sections 4.1-4.3 (like network size and step-size). Could the authors clarify the missing hyperparameters and specify how they were chosen and how they affect the results? For example: how was the step-size chosen for each network size in figure 2? Can the authors explicitly state the size of each network in figure 3?

2 comments

Thank you for your review! We have heavily revised Section 4, replaced Figures 1 & 2 with a new figure, and added Appendix B to address your questions and concerns. Responding to your weaknesses and questions in order:

W1: W1 could be read as questioning whether learning is actually slower, whether distractor connections are the cause, or both, so we address both.

The revised Figure 1 shows full learning curves on the non-stationary problem for block-sparse and dense networks with matched hidden unit counts. It includes a zoomed view of the learning curves within individual permutation intervals, so the reader can see the slower learning for themselves. We hope this presentation will be clearer in demonstrating the difference in learning speed.

You ask how long it takes each network to recover. Within an interval neither network fully recovers, so there is no clear recovery time to report. What we mean by learning speed is more general. Both networks will reach the same accuracy if given enough time, so if one does worse when the problem keeps changing, the only explanation left is that it learns slower. We separate this from the question of why it learns slower.

Section 2 establishes that distractor connections slow learning in the linear case and explains why the same tradeoff carries over to networks. The aim of the empirical work is to isolate and measure the effect there. We believe we rule out other effects (loss of plasticity with the revised Figure 1, and a difference in loss landscape with the dense-transition experiment), but we would be happy to test other ideas if you have something in mind.

The permutation interval controls how much learning speed matters. If the interval is long, both networks have time to converge and reach nearly the same accuracy. If it is short, the gap widens. We chose frequent permutations because they allow us to most clearly demonstrate the problem. We added an experiment in Appendix B that sweeps the interval from 200 to 6400 steps, and the results confirm this relationship.

W2: Distractor connections always slow learning, and our scaling results show that the cost grows with how many there are when the problem is non-stationary. So the question for real problems is not whether distractor connections hurt but how many of them a problem has. That depends on the problem. A single task on stationary data probably has few. A system that makes many predictions from the same observations and has to keep updating them probably has many (e.g. Sutton et al., 2011; Veeriah et al., 2018; Modayil & Abbas, 2023). These are the settings we eventually want to apply this work to, and it should matter most when the agent poses its own prediction targets (e.g. Veeriah et al., 2019), since then no designer can know which inputs each prediction will need.

The issue is that even there, measuring the cost requires knowing which connections are the distractors, and no existing method finds them. The closest methods we have, like SET and DEEP R, barely find the structure even in multi-MNIST, where the problem is as clean as it gets (Section 4). Had we approached real problems with these algorithms, we would have falsely concluded that distractor connections are not a problem. That is why we built a problem where the distractors are known in advance. Building such a problem is the only way we know of to study this relationship, which we suspect is why it has gone underappreciated, and we pose the problem design itself as a worthy contribution. Showing what algorithms can do on real problems once they overcome this is the next step, and this paper lays the groundwork for it.

Q1: Agreed that it was clunky. The revised Figure 1 replaces it.

Q2: That sentence relied on Figure 2 of the submission and has been removed along with it.

Q3: We ran this and found that the network still degrades to the same accuracy as all other dense-transition methods. The plot is here. It supports the same conclusion as Figure 3 of the submission so we have not added it to the paper.

Q4: Step-size values and network size details have been added throughout Section 4. The networks in Figure 3 of the submission each have 256 hidden units. Step-sizes are swept over powers of two and selected individually for every method and every point in every figure. We expand the edges of the sweep until an increase or decrease results in worse performance to ensure we find the best value. We revised a sentence in the first paragraph of Section 4 to clarify this.

We hope these changes address your concerns about soundness and would appreciate it if you would reconsider your score. We are happy to address any remaining concerns.

I appreciate the explanations and additional experiments, which address some of my original concerns. However, my main concern from the original review remains unresolved: the generality of the findings.

The current experiments use a single dataset, architecture, and type of non-stationarity. I understand the reasoning for using controlled synthetic problems where good connectivity is known, but I still think additional synthetic settings with different properties are important for building confidence that the conclusions are not specific to Multi-MNIST. For example, a synthetic case where there is more shared structure across subproblems.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.