acceptodds
Under review as a conference paper at ICLR 2027

Length-Independent State Tracking Under a Parallel Scan

Abstract

Learning robust and scalable finite-state tracking is fundamental to sequence processing. While linear recurrent neural networks (RNNs), linear attention, and state space models enable scalable parallel training through affine recurrences, their theoretical expressivity guarantees assume idealized arithmetic and do not extend to finite precision, where the parallel scan that makes them fast is itself a source of perturbation. We formalize finite-state tracking at finite precision and characterize *length independence*: tracking that stays correct at every sequence length, at a precision cost that does not grow with the length. We show that length-independent state tracking requires two competing dynamics within a single map: *contraction* to suppress numerical perturbations and *separation* to keep distinct states apart. We prove that affine recurrences, which offer a single rate at each step to serve both roles, realize at most *definite* automata at finite precision. Instead of treating scan compatibility as a restriction on the update map, we reinterpret it as a computational budget and introduce the *Neural Finite-State Machine* (NFSM): a non-affine, scan-compatible RNN built for length-independent finite-state tracking. On synthetic benchmarks spanning abelian and non-abelian groups, non-invertible monoids, and textual state-tracking tasks, affine baselines fail on every non-definite task, most of them within a few hundred steps. A single NFSM layer instead learns the exact transition tables of every algebraic task, which certifies correctness beyond the tested lengths, and a stack of NFSMs keeps perfect accuracy on the textual tasks at every tested length.

Primary area:
learning on time series and dynamical systems
Keywords:
state tracking, recurrent neural networks, parallel scan, finite precision, length generalization, finite automata, state space models, linear RNNs, transformation monoids, attractor dynamics, automata extraction, sequence processing

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_GtGAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces a novel approach to sequence modeling called "neural finite state machine" (NSFM) that can be parallized via a parallel scan. In contrast to Linear RNNS, the associative mathematical object is a n×nn \times n square matrix of "state transitions" for all possible nn states of that head. The paper rigorously derives that the transition matrix construct can be parallel-scanned.

Critical strengths
  • The idea of parallelizing such a transition matrix is novel.
  • The paper makes a rigorous derivation with good formalizations and proofs.
Critical weaknesses
  • The main weaknesses of the paper lie in the poor experimental validation. The model was only tested on small synthetic tasks for which it was specifically designed for (long-sequence state tracking). There is no larger scale study that would prove the scalability of the method in terms of performance, speed, compute or memory requirements.
  • The main paper is missing a section about training stability. I assume getting the gumbel softmax right is not straight-forward and might require some tricks to get it to run.
Questions for authors
  • could you please add a table to compare performance, speed, compute and memory requirements across the methods you applied?
3 comments

We thank the reviewer for the review and answer the specific points below. The general comment describes the new experiments and lists the changes in the revised PDF.

Experimental validation and scale. Section 7.2 adds two experiments on real data, described in the general comment. The Mamba-2 experiment tests the theory on released models of up to 780M parameters, and on keyword spotting an NFSM trained on streams of a few seconds keeps its accuracy on streams of more than two hours. Moving the NFSM, or restorative recurrences in general, to large-scale experiments is ongoing work, and it needs substantial computational resources that we cannot afford during the discussion period.

Training stability. We did not observe training instability in our runs. Appendix H.3 describes the training procedure. The Gumbel estimator uses one temperature, ν=0.5\nu=0.5, and the loss has one extra term that keeps the sampling exploratory. We used these settings on every task, without tuning. To measure how sensitive training is, we drew 15 configurations at random on keyword spotting, with learning rate from 10−410^{-4} to 3⋅10−33\cdot10^{-3}, batch size from 16 to 128, sampling temperature from 0.25 to 2, and weight decay from 10−610^{-6} to 10−210^{-2}. Every configuration learns the task and none loses accuracy as the streams grow (Figure 8, accuracies in the general comment), and Section 7.2 now states this result. The four configurations drawn near a learning rate of 10−410^{-4} stay lower, possibly because they need more training steps.

Table comparing performance, speed, compute and memory. Thank you for the suggestion. We added Table 8, which gives, for every model on S4S_4, the number of parameters, the failing length, and the time per training step and peak memory of one layer at L=212L=2^{12} and L=217L=2^{17}. Figure 7 gives the full curves against the sequence length and the state size. Sections 5 and 6 give the theoretical cost of the NFSM, a scan of depth O(log⁡L)O(\log L) whose merges take O(d)O(d) gathers. The measured numbers depend on the hardware and on dedicated kernels, which some baselines have and our implementation does not yet. Our response to reviewer Gkds discusses such a kernel.

We would be glad to answer further questions and, if time allows, to add measurements to Table 8 if the reviewer finds some missing.

Thank you for the additional experiments and clarifications regarding runtime. I decided to keep my score. I think it's a very interesting approach and theory, however it seems not be suitable yet for larger scale setting like language modeling.

We thank the reviewer for the follow-up and for finding the approach and the theory interesting.

We agree that large-scale language modeling is beyond the scope of this paper, and the NFSM was not designed for it here. The paper is first a general theoretical contribution. Theorem 4.1 characterizes what any affine recurrence (linear RNNs, linear attention, state space models) can track at finite precision. The NFSM is a constructive answer: it shows that this limit can be lifted while keeping scan compatibility, through a restoring organ in the sense of von Neumann.

The first theoretical half of the paper is tested on pretrained language models. In Section 7.2, released Mamba-2 checkpoints of up to 780M parameters, which we did not train, collapse on RULER as Theorem 4.1 predicts. Bounding their decay away from one, without retraining, restores retrieval with up to 128k tokens of text before the needle.

The same picture holds at 2.7B parameters. Beyond its 2k training context, Mamba-2 collapses, for the released checkpoint and after fine-tuning: the loss on PG19 books explodes, and on RULER needle 1 the passkey is lost at 16k. In ongoing work, guided by the analysis of this paper, we added a restoring organ to Mamba-2 while preserving Mamba-2 training and inference throughput. Converted from the same checkpoint and fine-tuned on 7B tokens, it keeps the PG19 loss flat up to 32k and retrieves the passkey at every tested length, without a significant drop in accuracy on common-sense reasoning benchmarks, when evaluated within the training context. This model is a separate contribution, with its own architecture, conversion recipe and evaluation at scale, which we cannot present and validate in full within this paper; we mention it here only because it bears on the scale question raised by the reviewer.

2.7B, trained at 2kZero-shot avg. (%)Passkey 4kPasskey 8kPasskey 16kPasskey 32k
Mamba-2, released63.21009900
Mamba-2, fine-tuned64.51009200
Hysteretic recurrence, fine-tuned64.3100100100100

Passkey: RULER needle 1, accuracy in %, 100 examples per length. Zero-shot: mean accuracy on LAMBADA, HellaSwag, PIQA, ARC-e, ARC-c and WinoGrande. Both fine-tuned models use the same recipe and token budget; one seed.

These results indicate that both the failure identified by Theorem 4.1 and the effect of adding a restoring organ carry over to language models at this scale.

Scaling the NFSM itself to language modeling is the natural next step. It depends on the fused kernel discussed with reviewer Gkds, and we will state this limitation explicitly in the final version.

Thank you again for your time and the constructive review.

Reviewer_GkdsOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Think of finite-precision arithmetic as a bounded perturbation of idealized arithmetic. If the dynamics of a sequential computation aren't contractive, then those perturbations can accumulate. However, that contraction smears true differences among states.

Ideally, one would have a contractive basin for each state. An affine map (as used in linear RNNs, linear attention, etc.) can't accomplish such multistability. Theorem 4.1 shows how the historical window of such models is effectively truncated to fixed length.

The Neural Finite State Machine is strictly more expressive. It is parameterized by dd. It maintains dd basin vectors, each of dd dimensions. The state is updated using argmax operations which are essentially contractions to the different basins.

In synthetic state tracking tasks, this model shows significantly more long-range state-tracking capability than SSMs.

Critical strengths
  • The clarity and depth of the presentation exceeds that of the typical ICLR paper. Table 3 in the Appendix is a nice summary of the literature. The writing is overall fine but I think its clarity stems more from the fundamental clarity of the research idea. There are multiple helpful figures. Figure 1 looks nice although I personally found the underlying concept easy to grasp without it.

  • The motivating observation about contraction and separation are quite natural; so is the proposed fix. The theory is helpful.

Critical weaknesses
  • I don't think it is correct to say that parallelization using scans is either necessary or sufficient for modern hardware. Scans are typically not especially tensor-core friendly. How well can the NFSM use tensor cores? The appendix indicates its “block itself is not compute-bound but rather bandwidth-bound” - that is not good.

  • The experiment is quite limited. First, it's on synthetic data. More importantly, the state is given as labeled data, at every time sequence, to the model. In practical sequence learning tasks, "state" is just an abstract conceptualization of modern internals which have to be learned in a more indirect way. This paper would be greatly improved if more realistic experiments were conducted.

I am particularly concerned about two aspects of the model. First, it involves nondifferentiable argmax, which may hinder learning tasks where the state has to be learned indirectly. (I did see the parts about Gumbel max, which don’t resolve this concern.) Second, while the state update is sufficient to enable length-independent tracking, that does not mean the update is comparatively expressive at shorter sequence lengths.

Questions for authors
  • The paper "Stable Recurrent Models" (Miller and Hardt, ICLR 2019) isn't cited, but seems relevant to understanding the novelty of Theorem 4.1. (It seems overall supportive of this paper).

This is well-conducted research. I would ideally raise my score to a clear accept if the above criticisms are accounted for.

5 comments

We thank the reviewer for the positive assessment and answer the specific points below. The general comment describes the new experiments and lists the changes in the revised PDF.

Scans, tensor cores and the bandwidth-bound block. The reviewer is right, and our wording was too strong. We wanted to contrast the NFSM with classical nonlinear RNNs, which cannot be parallelized over the sequence, yet scan compatibility alone does not meet the needs of current hardware. We will soften this in the final version.

We do expect the NFSM to use tensor cores well. Most of its FLOPs go into the input projection that produces the logits, a dense matrix product over all tokens at once that does not involve the recurrence. The recurrence itself is cheap. The carried state is an integer index of ⌈log⁡2d⌉\lceil\log_2 d\rceil bits per head, composing two tables takes O(d)O(d) gathers on small integers, and every merge is exact. The bandwidth cost comes from our current implementation, which writes the d2d^2 logits of every step to GPU memory. A fused kernel could compute the logits, take the column argmax and compose the tables in on-chip memory, writing back only indices. The backward pass needs only the column used at each step (dd values), which can be recomputed instead of stored, and the dictionary parameterization of Appendix G.6 would cut the traffic further. We are currently writing such an optimized kernel.

Synthetic data and state labels. We agree. Section 7.2 now has two experiments on real data, described in the general comment, and the state is never given in either of them. In keyword spotting, only the final prediction is supervised, so the model has to learn indirectly what is a state, how to relate them to whether a digit was heard and which one.

Argmax and indirect supervision. In keyword spotting, the NFSM is trained with the Gumbel estimator through four layers, from a single label per sequence, at every stage of the length curriculum. In the random hyperparameter search of Figure 8 (general comment), every configuration learns the task. Accuracy changes smoothly with the hyperparameters, which is good news for tuning, and the learning rate matters most.

Expressivity at short lengths. The reviewer is right that length independence says nothing about short sequences. On one-second streams in keyword spotting, the NFSM reaches about 93% with untuned default hyperparameters, against 95 to 97% for minGRU$^-,Mamba, Mamba^-$ and DeltaProduct, and the random search found configurations that do better than the default. At these lengths, the NFSM is therefore close to the baselines, and we cannot yet tell whether the remaining gap comes from training or from the model. We thank the reviewer for this question, which also made us think about hybrid models. Affine recurrences are very good at extracting features from signals and are easy to optimize, so a natural design combines affine layers for short-range signal processing with NFSM layers, or other restorative recurrences, that keep the performance robust along the length. We leave this for future work.

Miller and Hardt (2019). Thank you for the pointer, which we now cite in the related work and the discussion. In exact arithmetic, they show that a contracting recurrent network is well approximated by its truncation to the last kk inputs, for inference and for training. Branch (ii) of Theorem 4.1 reaches the same conclusion for affine recurrences at finite precision. It is indeed supportive of our results.

We would be glad to discuss any of these points further.

Reviewer_GkdsOpenReview ↗

Response

Thanks for your thoughtful and informative reply.

W1: Hardware efficiency. Ah, so the logit matrix θx\theta_x would be expanded on-chip. Can you provide details on the learned map which expands θx\theta_x? These details are needed to assess arithmetic intensity and the complexity of the ensuing fused kernel.

W2: Real data experiments. Thank you, I think this is an improvement over state supervision. There are plenty of more standard "off the shelf" long-context / state-tracking benchmarks: what motivated this particular setup?

W3: Expressivity. Yes, I agree that expressivity at shorter contexts can be handled by other models in a hybrid setup. It would have been nice to see such an experiment, but not necessary.

Edit: the answer to these may simply be "we still have to completely figure out a fused kernel, and training a hybrid model that does well on more standard benchmarks is not practical until then." That is fine --- there are useful ideas here, and I don't think research papers need to be production ready. In that case, it'd be helpful to describe current limitations / provide "todo" details for researchers interested in picking up that work.

Thank you for the follow-up. GPU kernels are not our primary area of expertise, so if we misunderstand something below, or if you see a better design, we would be grateful to hear it.

W1. The map and the fused kernel. The map that produces the logits is a single linear layer (of course, the input of this layer is the output of the previous one that is laredy doing non-linear transformation per token, or through time if this is another recurrent layer). Each NFSM layer has one weight matrix Wθ∈Rm×DW_\theta\in\mathbb{R}^{m\times D} and a bias, with D=∑idi2D=\sum_i d_i^2, and the logits of a token are utWθ+bu_tW_\theta+b, split into one di×did_i\times d_i block per head. The table of a head is the column-wise argmax of its block. The dictionary of Appendix G.6 is not used in the experiments. In keyword spotting, m=128m=128 and a layer has 64 heads of size 4, so D=1024D=1024.

Since the reply, we have written a first possible version of fused kernels for this layer (JAX Pallas, Triton backend), for training, inference and token-by-token decoding. The kernels match the reference implementation exactly in the forward pass, and the gradients agree to about 2⋅10−32\cdot10^{-3} relative error because of the bf16 operands. A trained keyword-spotting model gives the same accuracy within half a point with either implementation.

This is evidence that the kernel is performing the correct computation; however, we are having trouble figuring out how to evaluate it: We are not sure which metrics to report or how to make a fair comparison. We have access to both A5000 and A100 GPUs. What do you think ? We could try to do it quite quickly. We agree on the fact that this is not strictly necessary but it would allow easier scaling toward large models or experiments.

We would really welcome any insight or suggestion on these points.

W2. Choice of the setup. The main difficulty was to find a state-tracking benchmark that does not require a pretrained language model, and that has enough training data to avoid strong overfitting, so that the results do not hinge on optimization or hyperparameter tuning. Streamed keyword spotting combines real audio signals and real recorded noise with a simple task. If the you knows other datasets of this kind, we would be glad to have pointers for future work.

W3. Hybrids. Thank you. For such hybrids, we think it will be important to make sure that the affine layers decay enough, so that perturbations do not pile up in their state as in the pretrained Mamba-2 experiment.

Thank you for your time and the great discussion.

Reviewer_2aQnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether recurrent models can track a finite set of states correctly for arbitrarily long sequences despite small numerical errors. Under its worst-case error model, it shows that affine recurrences can only robustly solve tasks whose state depends on a bounded number of recent inputs. The proposed Neural Finite-State Machine instead learns discrete transition tables that can be evaluated in parallel. It performs well on synthetic state-tracking tasks, including algebraic state-tracking problems and tracking tasks with shuffled objects.

Critical strengths
  • The theory explains a meaningful tradeoff between a model that removes numerical errors and one that forgets the state it is tracking. The paper identifies which tasks affine recurrences can handle under this requirement. This gives a clearer account of their limitations than simply showing that accuracy drops on longer sequences.

  • The authors check the learned transitions exhaustively on the algebraic tasks. This is strong evidence, as it establishes that the learned machine will keep making the correct transitions at any length.

  • The results are reproduced across five seeds.

Critical weaknesses
  • The paper is hard to parse and feels like technical jargon. The writing needs significant cleanup.

  • The paper should distinguish its error model more clearly from actual finite-precision computation. The theorem requires correctness for any sequence of bounded errors, while rounding errors follow specific rules. Appendix D also notes that some affine models can carry out the required operations exactly.

  • The experiments give the model considerable information about the task structure. The number and sizes of the heads are chosen using knowledge of the target problem. These choices are reasonable for testing the mechanism, but it is unclear how well the method works when that information is unavailable. This also matters for scalability.

Questions for authors

Please see the weaknesses above. Apart from those:

  • Could you clarify the scope of the impossibility claim in the main text?
  • If the same trained baseline is evaluated at higher precision, how much does its failure length change?
  • How much do the results depend on the task-specific head sizes and intermediate state labels?
2 comments

We thank the reviewer for the detailed review and answer the specific points below. The general comment describes the new experiments and lists the changes in the revised PDF.

W1. Writing. We have reworked the writing throughout. If some passages remain hard to parse, we would be grateful if the reviewer could point to them, and we will revise them during the discussion if time allows.

W2 and Q1. Error model and scope of the impossibility claim. We agree that the main text should state this clearly. A new paragraph after Theorem 4.1 now says that the theorem is a worst-case statement. No affine recurrence is guaranteed to be correct at every length under every perturbation of size η\eta, although a single run, which follows one η\eta-trajectory, may be correct. Requiring the worst case makes the guarantee robust, since it then holds whatever the pattern and the source of the errors, as long as each step stays within η\eta. This matters for two reasons:

  • The rounding errors of a run depend on the number format and on the order of the operations, so they are not a property of the model. A parallel scan and a sequential loop that compute the same layer already follow different trajectories from the first steps (Figure 3). A model trained and tested on one machine may later run on other hardware, and a guarantee over every perturbation covers all of them.
  • Rounding is only one source of perturbation. A trained model only approximates an exact realization, so as long as its state stays bounded, its runs are η\eta-trajectories of that realization, even in exact arithmetic (Appendix E).

As the reviewer notes, some affine models carry out the required operations exactly when the arithmetic is exact (η=0\eta=0). Theorem 4.1 holds for every η>0\eta>0, however small, so a finer format may delay the failure of an affine recurrence but does not remove it.

Q2. The same baseline at higher precision. We are running this experiment, evaluating the trained baselines at higher and at lower precision, and will post the results here if time allows, or otherwise add them to the final version. Figure 4 already covers the ideal case without learning error, where the exact ρ=1\rho=1 realization of Z7\mathbb{Z}_7 fails at a finite length in every format, fp32 included. On most tasks, the trained baselines fail within a few hundred steps (Table 1), so we expect the error of the learned approximation to matter more than the number format.

W3 and Q3. Dependence on the head sizes and on the labels. Thank you for this question. The new Appendix H.6 (Figure 5) draws 1 to 5 heads of size 2 to 16 at random, without looking at the task. We call a configuration representable when, by the Myhill–Nerode theorem, heads of these sizes can hold the states of the minimal automaton of the task and carry its transitions. This label is assigned after training, only to color the figure. As the theory predicts, every configuration that is not representable fails. Every representable one solves S3S_3, Z2\mathbb{Z}_2 and FF, and 3 of 10 solve S4S_4, so learning may depend on the optimization hyperparameters, which we did not tune.

On the labels, the synthetic tasks supervise the state at every position. Keyword spotting (Section 7.2) removes them: the state is never given, and only the final prediction is supervised. We agree that choosing the number and sizes of the heads is an open problem on real tasks. We expect many very small heads to work best in most cases, and this choice also scales best, since the cost of a head grows with the square of its size. Keyword spotting uses this design without tuning, with four layers of 64 heads and 4 attractor indices per head. The NFSM keeps its accuracy on streams of more than two hours (Figure 2A), and every configuration of a random hyperparameter search learns the task (Figure 8).

Scalability. Sections 5 and 6 give the theoretical cost of a head, a scan of depth O(log⁡L)O(\log L) whose merges take O(d)O(d) gathers on d⌈log⁡2d⌉d\lceil\log_2 d\rceil bits. The new Table 8 reports measured times and memory. These numbers depend on the hardware and on dedicated kernels, which some baselines have and our implementation does not yet. Our response to reviewer Gkds discusses such a kernel for the NFSM.

We would be glad to discuss any of these points further.

Thanks a lot for the clarifications and for the thoughtful rebuttal! The rebuttal address my concerns and I'll retain my positive score of 3.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Official comment to all reviewers

We thank the three reviewers for their careful and constructive reviews. This comment describes the new experiments and the changes to the paper, and the individual responses answer each review point by point. We will add the new code to the anonymized repository soon.

Revised PDF. We have uploaded a revised version of the paper, so that the new experiments can be read in context. The changes, listed below with the place where each one appears, answer points raised in the reviews. If a reviewer finds a change unwelcome, or prefers the submitted version of a passage, we would be grateful to hear it and will revise it accordingly.

New baselines. We added minGRU$^-$ and DeltaProduct to Table 1, and Appendix H.5 varies the number of Householder steps of DeltaProduct. Both new baselines fail as Theorem 4.1 predicts, which extends the evidence to more affine recurrences. PD-SSM is still training on keyword spotting. Its runs are slow, so we will add it once they finish, and we preferred to post this response now so that the discussion can start early.

Experiments on real data (Section 7.2, Figure 2, Appendix I). Several reviews note that the synthetic tasks supervise the state at every position. In the two new experiments, the state is never given, and each panel of Figure 2 shows the pattern Theorem 4.1 predicts next to the measured results.

  1. Streamed keyword spotting. The model listens to background noise in which one spoken digit may appear, and predicts the digit or "none" at the end of the stream. Only this final prediction is supervised. The NFSM is trained on streams of a few seconds and keeps its accuracy on test streams of more than two hours, whether the extra noise comes before or after the digit. Mamba$^-$ and minGRU$^-$ behave as kk-definite systems. At every tested length, noise before the digit leaves them unaffected, while enough noise after it pushes the digit out of their memory. DeltaProduct and AUSSM collapse, and their accuracy falls with the total stream length wherever the digit sits.
  2. Pretrained Mamba-2 on RULER. This experiment tests the theory on released checkpoints of 130M, 370M and 780M parameters, which we did not train. Needle-in-a-haystack evaluations usually vary the total context and the relative depth of the needle. We vary separately the text before the needle and the distance between the needle and the question. A kk-definite model then depends only on this distance, while a collapsing model fails once the total context grows too long. The released models collapse. Their decay is too weak for the perturbations, which accumulate in the state. They fail past a fixed total context, even with the needle right before the question. Once the decay of every head is bounded away from one, without retraining, they become kk-definite and retrieve the needle whenever it lies within a fixed distance of the question, for up to 128k tokens of text before the needle, with a very small impact in WikiText-2 perplexity.

Training stability (Appendix I.1, Figure 8, one sentence in Section 7.2). On keyword spotting, we drew 15 random configurations of learning rate, batch size, sampling temperature and weight decay, each trained with one seed. All 15 learn the task, and none loses accuracy as the streams grow. With a learning rate of at least 2.5⋅10−42.5\cdot10^{-4}, they reach 84 to 94% on the test streams of Figure 8, against 93.4% for the default configuration, and the four drawn near 10−410^{-4} reach 69 to 74%. Everywhere else in the paper, we used default hyperparameters without any search, so the reported numbers show typical behavior rather than the best performance. The only exception is AUSSM on keyword spotting, trained at a learning rate of 10−410^{-4} because it does not learn the task at 10−310^{-3}.

Head sizes (Appendix H.6, Figure 5). In Table 1, the head sizes follow the structure of each task. The new experiment trains the NFSM with head sizes drawn at random on S3S_3, S4S_4, Z2\mathbb{Z}_2 and FF. Every configuration the theory rules out fails, as predicted. Our response to reviewer 2aQn gives the details.

Cost and performance (Table 8). The new Table 8 gives the parameters, failing length, time per training step and memory of every model.

Text. A new paragraph after Theorem 4.1 states that it is a worst-case result. Because the guarantee covers every perturbation, it stays valid when the hardware, the number format or the order of the operations changes, and it also covers part of the learning error (Appendices D and E). We now cite Miller and Hardt (2019) and have reworked the writing throughout.

We welcome further questions and will run more checks during the discussion if time allows.

(Differences with the submitted version. As we read the ICLR guidelines, OpenReview generates automatically a comparison between the revised and the submitted PDF, but we cannot see it from the authors' side or from the public page. If it is not available to the reviewers and is useful, we can send a PDF that marks every change on request.)

Edit November 13th: Dear reviewers, we updated the code that can still be found following the original anonymized link: https://dd8wgy467d-hash.github.io/reviewanonymous/ ; It may require you to refresh the cache from your browser. Best, the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.