acceptodds
Under review as a conference paper at ICLR 2027

Action-Conditioned Predictive Learning with Causal Self-Attention: Identification, Initialization, and Planning

Abstract

We study whether causal self-attention trained on action-conditioned outcomes can learn predictive models that support near-optimal decisions without optimal-action supervision. For a one-sparse logistic model on fixed binary history features, we characterize identification by positive feature-disagreement probabilities. Under this condition, population gradient flow and gradient descent from uniform attention converge, with prediction bounds uniform over all feature combinations. A logarithmic-size design attains the identification lower bound, with finite-sample guarantees from independent exploratory episodes. For a complementary multi-head model, we show that uniform-attention initialization yields tangent features of degree at most two and makes population training stationary for pure higher-order parity targets. A target-independent nonzero score initialization yields positive expected kernel eigenvalues, with interaction-order expansions and explicit bounds at every nonzero scale. These target-eigenvalue guarantees extend to finite-width population and empirical training, including bounded perturbations of paired initialization. A separate finite reachable-set analysis treats general full-support outcome distributions under representation and coverage conditions. Exact planning with the Bayes predictive model is Bayes-optimal, and uniform predictive error yields explicit value-suboptimality bounds. Controlled experiments examine sparse prediction, kernel stability, coverage, and higher-order learning, and corroborate the predicted initialization spectrum.

Primary area:
learning theory
Keywords:
Causal self-attention, neural tangent kernels, model-based planning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_2kfzOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies action-conditioned predictive learning with causal self-attention. The authors develop identification and convergence guarantees for sparse binary models, analyzing how attention initialization affects higher-order interaction learning. They also draw connections between predictive accuracy and optimal actions through an external planner.

Critical strengths
  • The analysis of how initialization affects the ability to learn higher-order interactions provides some insight into how the attention mechanism benefits the model.

  • The experiments examine several relevant factors, including model width, initialization, and data coverage.

Critical weaknesses
  • My main concern is that the theoretical results apply to several separate, highly simplified settings, and their connection is not strong enough. For example, the sparse model assumes that the relevant information is already provided as fixed binary features, with only one relevant feature per query. The higher-order learning results do not carry over to the multistep planning guarantee. It is therefore difficult to see how these results together address the broader problem of learning predictive models for decision-making.

  • The theoretical requirements are much larger than the models used in the experiments. Appendix U.4 gives sufficient widths of roughly 2.5 million and 75 million head pairs, while the experiments use only 16 to 256.

  • Parts of the manuscript appear inconsistent with the actual results. For example, Appendix P claims contributions involving a “fixed-token-pair finite inverse,” “cache-level reconstruction,” and intervention-specific identification, but I could not find corresponding definitions or results in the paper.

Questions for authors
  • Can the authors show an experiment where the model learns from sampled trajectories and then successfully plans using its predictions?

  • How does the proposed initialization compare with standard random initialization under the same training budget?

  • Can the theoretical bounds be tightened enough to help explain the experimental results?

Reviewer_T4ekOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when action-conditioned outcome prediction with causal self-attention supports decision-making. It establishes identification and optimization guarantees for a one-sparse model, analyzes how score initialization changes the learnability of higher-order interactions, and derives finite-width and finite-sample learning guarantees. Separate uniform-error results connect prediction to external planning. Six synthetic experiments examine the theoretical mechanisms.

Critical strengths

The strongest contribution is the initialization analysis within the same softmax architecture. The paper identifies an exact higher-order obstruction at zero scores, derives the nonzero-initialization Walsh spectrum, and extends target-eigenvalue guarantees to finite-width training with noisy samples. This provides a concrete account of how initialization changes learnability. The sparse analysis also goes beyond expressivity: an explicit identification condition supports optimization, generalization across feature combinations, and guarantees from independent exploratory episodes. These are substantive learning results.

Critical weaknesses

My main concern is that the strongest learning result and the planning guarantees apply to different regimes. The higher-order initialization analysis controls average prediction error under a fixed input distribution; it does not provide the uniform control needed for the multistep planning guarantee. The results that do support planning instead assume either a one-sparse response on predetermined binary features or coverage of every reachable context together with a full-rank fixed representation. These assumptions leave representation learning and the acquisition of adequate exploration data outside the analysis. The number of reachable histories can also grow rapidly with the horizon, while planning is delegated to exact external dynamic programming. The paper states several of these limitations, but they materially restrict its broader significance for decision learning. It establishes precise results in complementary settings, without yet showing that the initialization insight improves planning in a setting requiring richer history dependence.

In addition, the practical scope of the initialization remedy remains uncertain. Exact stationarity at zero scores relies on paired zero-output initialization, uniform Boolean inputs, and population gradients. This does not establish the same failure under finite-sample or stochastic training, and the finite-sample experiments do not provide a matched zero-versus-nonzero comparison. Moreover, positive higher-order eigenvalues establish learnability without ensuring favorable resource requirements: the resulting width, sample, and time guarantees can become very large, potentially exponential at growing interaction order. The experiments illustrate escape from stationarity more convincingly than efficient higher-order learning. These limitations qualify the practical interpretation of the spectrum, although they do not invalidate the theoretical contribution.

In addition, Appendix P (p. 36) attributes finite-inverse, cache-reconstruction, and intervention-specific dimension/identifiability results to this paper. I could not locate the corresponding constructions or theorems.

Questions for authors

Please address the concerns in Weaknesses, particularly whether the zero-score disadvantage persists under matched finite-sample training.

Reviewer_aWRROpenReview ↗
Rating
1(−1)
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies whether causal self-attention trained on action-conditioned outcomes, without optimal-action labels, can support planning. It analyzes (a) identification in a sparse prediction model, (b) convergence on a finite set of contexts, and (c) the effect of initialization on learning higher-order dependencies, and then translates uniform prediction error into a bound on planning sub-optimality. 6 synthetic experiments illustrate these results, including a two-stage planning example.

Critical strengths

Initialization analysis (Prop. 13, Thm. 14). The paper gives a concrete distinction within a single architecture: with zero attention scores, parity targets of degree ≥3\geq 3 are exact stationary points of population training at any width, while a target-independent random score initialization removes this obstruction with explicitly quantified eigenvalue behavior.

Critical weaknesses

1. The paper does not deliver the contribution its title and abstract describe.

  • Sections 3 and 4 analyze prediction over a fixed binary feature vector or a finite context set. Histories and actions enter only as indices; the causal mask only lets one query token attend to the P feature tokens (Prop. 18).
  • The decision-making setting enters only through the simulation lemma in Section 5, and the paper notes (p. 7) that the Section 4.1 result does not supply the uniform guarantee needed there.
  • Sections 3, 4 and 5 apply established techniques; the paper itself locates its distinctions in "training distributions and initialization" (p. 2).

The initialization result is genuine but concerns a one-layer attention model on uniform binary inputs with parity targets. As submitted, the framing and the results are two different papers.

2. The experiments do not test the guarantees under their own conditions.

  • The Section 4 theorems require widths on the order of 10^6 to 10^7; Experiments 3 and 4 use 256 and 64 head pairs, and the appendix states these runs do not test the finite-width theorems (U.4–U.5).
  • The sample condition of Thm. 22 holds in 3 of 12 settings of Experiment 2.
  • In Experiment 5 the learned target coefficient is about 0.0125 against a target amplitude of 0.2; U.6 does not state whether the conditions of Thm. 15 are met.
  • In Experiment 6 the bound H(H+1)ε ≈ 0.93 exceeds the value gap of 0.085, so the theorem is uninformative for that instance.

Apart from Experiment 1 and three settings of Experiment 2, the experiments are either outside the assumptions or set up so that the bound is not binding, so they offer little evidence about how informative the guarantees are.

Questions for authors
  1. Which results change if the causal mask is removed and inputs are treated as fixed feature vectors?
  2. What does Section 3 establish that does not follow from Wang et al. (2024b) or Marion et al. (2025)?
  3. How do the results relate to supervised pretraining for in-context decision-making (Lee et al. 2023; Lin et al. 2023), which also obtain Bayes-optimal behaviour?
  4. Appendix P lists "cache-level reconstruction" and "intervention-specific dimension and identifiability results" as contributions; I could not find these in the paper. Where do they appear?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.