acceptodds
Under review as a conference paper at ICLR 2027

When Learners Shape Rewards: Learning Dynamics in Trading Populations

Abstract

Market simulation is valuable for understanding collective market behavior and evaluating trading strategies under interaction. Existing work emphasizes how traders interact or how well individual agents perform. Less clear is how collective behavior and the benefits of learning change as more participants continually revise their decision rules. We introduce FactorMarket Lab (FML), coupling factor-based stock predictors with a multi-stock order-book market. Market interaction generates new training examples that, once their labels become available, enter subsequent model updates. Traders share an initial model within each family but independently sample available data and update separate copies. Historical experiments demonstrate learning benefits, and interactive measurements show behavioral differentiation from a shared start. We vary peer learning density and update frequency, comparing the same focal cohort with and without learning. Across the tested populations, behavioral differentiation varies with learning density and update timing, with distinct responses across models. The gain from a trader's own updates also depends on how its peers learn: more widespread or frequent learning is not uniformly better. In mixed-model populations, conditions favoring collective outcomes need not maximize focal learning gains. These findings motivate evaluating continual learning as a population process, not only as an individual capability.

Primary area:
learning on time series and dynamical systems
Keywords:
Who learns and how often shapes market behavior and the value of individual learning.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ztKXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to learn to simulate the stock market by having traders models as agents that learn through interactions. In doing so it combines factor-based stock predictors with a multi-stock order-book market. A few studies of population behaviors has been conducted to show the traits of proposed method learning stock dynamics.

Critical strengths

None

Critical weaknesses

There are a few issues in this manuscript that prevents an acceptance in its current form.

The biggest issue is the lack of discussion and comparison of recent LLM based multi-agent stock market simulation methods, which both consider order book structure for studying the dynamics. To name a few: Gao et al., 2024 https://arxiv.org/abs/2406.19966, Papadakis et al., 2025 https://arxiv.org/abs/2507.09255 , Hashimoto et al., 2026 https://link.springer.com/article/10.1007/s42001-026-00465-4 . Such omission largely limits the contribution of this manuscript. Even though this manuscript choose to focus learned / fine-tuned models rather than context engineering more powerful models, however the validity of such choice remains unclear for the claimed problem to solve (studying stock dynamics), making the whole narrative under justified, and technically missing a wide body of works.

There are also concerns regarding the methods and experiments. For example, no noise / stochasticity in market has been modeled. Also, the LLM used in this experiments contains only contains only Qwen3.5-2B, which is are not sufficient. Furthermore, as the author mentions, "FML contains 32 research traders, 20 synthetic stocks, and 32 features, over 120 decision days" which is insufficient for a thorough stock market behavior analysis.

For presentation, the formalization in Sec 3 is hard to parse. It feels like formalization only for formalization's seek. A wide range of detail are also missing, likely explain the difficulties in assessing the merit of this work.

Questions for authors

None

2 comments

1. Missing market-simulation literature and choice of agents

Thank you for identifying these relevant studies. We will add a dedicated comparison discussing ABIDES, ASFM, StockSim, FCLAgent (PAMS), TwinMarket, and FinEvo:

System / agentOrder matchingCross-sectional factor scoringAgent heterogeneityContinual learningShared training pool
ABIDES (original)✓×✓××
ASFM✓×✓××
StockSim✓×✓××
FCLAgent (PAMS)✓×✓××
TwinMarket✓×✓××
FinEvo✓×✓××
FML✓✓✓✓✓

✓ denotes a capability present in the evaluated setup; × denotes its absence there rather than an architectural restriction. Continual learning refers to deployment-time parameter updates during market interaction; a shared training pool supplies training samples to multiple learners (distinct from market feeds, social communication, or prompt histories). StockSim reflects its order-level mode; FCLAgent is Hashimoto et al.'s trader in PAMS; FinEvo reflects its implemented market-interaction setup, where parameters remain fixed during simulation.

Why trainable parameter-updating models rather than context-engineered LLMs? Prompt- and memory-based LLM agents are well-suited for studying narrative reasoning or behavioral framing, but they conflate linguistic drift with learning and do not allow controlled attribution over update cadence, minibatch budget, or training-pool composition. Our research question specifically asks how the value of parameter-level continual learning changes when counterparties also learn. Trainable cross-sectional scoring controllers (MLP, Transformer, and Qwen+Head) built atop ABIDES enable exact common-start focal/peer switches, matured-label construction, and matched historical/personal/public sample pools—isolating interactive learning dynamics that context engineering alone cannot cleanly separate.

Stochasticity. FML includes random market, sector, and stock-level shocks, as well as random background order flow. Learning agents also sample minibatches independently. We will clarify these sources of randomness and report the seed settings.

Presentation. We will reorganize Section 3 around one decision day to show how traders make predictions, place orders, and later learn from market outcomes. We will introduce each equation alongside the step it explains.

2. Evaluation horizon (day 120 vs. day 512)

We compared the same runs at two time points. The interaction (D−C)−(B−A)(D-C)-(B-A) measures how much peer learning changes the gain from the focal traders’ own updates. Values are means across five seeds, in percentage points (pp).

TradersDayFocal gain with frozen peers (B−AB-A)Focal gain with learning peers (D−CD-C)Interaction
321200.1880.024−0.164
32512−0.742−0.4120.33
64120−0.071−0.0370.034
64512−1.091−0.3680.723
  • Focal learning gains were lower at day 512. With 32 traders, the early gains turned negative under both peer conditions. With 64 traders, the gains were already negative at day 120 and fell further by day 512.
  • The interaction was larger at day 512 in both populations. With 32 traders, peer learning reduced the early benefit of focal updates by 0.164 pp but softened their later penalty by 0.330 pp. With 64 traders, peer learning softened the penalty at both time points, by 0.034 pp at day 120 and 0.723 pp at day 512.

We will report results for each 128-day window in the revision to show how the interaction develops between these time points.

3. Update interval and share of learning peers (τ×ρ\tau \times \rho)

The table reports mean focal learning gains at day 512 for 32 traders, averaged across five seeds, in percentage points (pp). Rows show the update interval (τ\tau), and columns show the share of peers that learn (ρ\rho). Bold marks the highest mean gain for each peer share. The final column reports the interaction: focal gain at ρ=1\rho=1 minus focal gain at ρ=0\rho=0.

Update interval (τ\tau)ρ=0\rho=0ρ=0.2\rho=0.2ρ=0.4\rho=0.4ρ=0.6\rho=0.6ρ=0.8\rho=0.8ρ=1\rho=1Interaction
1 day−0.716−0.521−0.487−0.968−0.746−0.124+0.592
5 days−0.742−0.729−0.601−0.573−0.691−0.412+0.330
10 days−0.668−0.714−0.877−0.395−0.177−0.325+0.343
Spread across intervals0.0740.2080.3900.5730.5690.288—
  • The ranking of update intervals depended on how many peers learned. At 40% peer learning, daily updates outperformed 10-day updates by 0.390 pp. At 60%, the ranking reversed: 10-day updates outperformed daily updates by 0.573 pp. Daily updates led again at 100%.
  • The peer-learning interaction was positive at all three intervals. Full peer learning narrowed the penalty from focal updates by 0.592 pp for daily updates, 0.330 pp for 5-day updates, and 0.343 pp for 10-day updates. Mean focal gains remained negative in all 18 settings.
Reviewer_jwmuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces FML, a market simulator where traders update their models using historical and market-generated data. It studies how the share of learning traders and their update frequency affect trading behavior and the gains from continued learning.

Critical strengths
  • The paper studies an important question: how the value of continued learning depends on whether and how other market participants learn.
  • The experiments cover three model families, different update schedules, and mixed-model populations.
Critical weaknesses
  1. The feedback loop itself is not new and the discussion of related work could be strengthened:

(1) Early artificial stock market studies already study how traders affect one another’s learning:

  • [1] LeBaron, Blake. "Short-memory traders and their impact on group learning in financial markets." Proceedings of the National Academy of Sciences 99.suppl_3 (2002): 7201-7206.

(2) Work on multiplayer performative prediction also studies learning when multiple decision-makers change the data distribution.

(3) self-consuming generative models are also relevant, since they retrain on model-generated data. (this is a broader connection, not the same setting)

What new mechanism or empirical finding does FML add beyond these lines of work?

  1. Proposition 1 is a general variance decomposition. It would also hold for identical model copies trained on the same fixed historical dataset with independent minibatch sampling. From identical parameter and optimizer states, independent updates with nonzero variance already produce positive expected dispersion. Neither market interaction nor newly generated data is needed.

What does Proposition 1 explain about market feedback that it would not explain on a fixed historical dataset?

  1. The current comparisons combine changes in model updates, trading, prices, and training data. They do not show what adding new market-generated samples contributes. A key missing control is continued training on historical data alone. This should use the same initial models, update schedule, and training budget as training on historical plus new data. Without this comparison, it is unclear how much of the observed effect comes from new training data rather than continued optimization on historical data.
Questions for authors

Please refer to the questions in the Weaknesses

1 comment

Relation to prior learning-feedback research

We agree that feedback is well established in prior work. These studies examine how learning proceeds and what outcomes it produces when agents’ actions affect what happens next. FML builds on this work by treating other traders’ learning as part of each trader’s market environment.

StudyMain question
LeBaron (2002)Can traders who rely on short histories disrupt learning across the market?
Narang et al. (2023)When players’ decisions change the data distribution, under what conditions can learning reach an equilibrium?
Alemohammad et al. (2024)How does training on earlier models’ outputs affect the quality and diversity of later outputs?
FMLWhen other traders learn and change the market environment, how does that affect a trader’s gain from updating its own model?

We measure this gain by comparing the same focal traders’ net returns with and without updates, first when peers keep their models fixed (B−A) and then when peers also learn (D−C). Having more peers learn or update more often does not consistently increase this gain. This suggests evaluating a trader’s update strategy across different peer learning conditions.

Proposition 1

We agree. Proposition 1 does not explain market feedback. It only explains why independent random updates can lead to different parameters in initially identical models, even when they train on the same fixed dataset. We will restate it as a lemma and remove the market-feedback interpretation. Our evidence for market feedback comes from the subsequent group experiments, where we vary which traders learn and what market data they can access.

Historical-only continued-training baseline

We added a baseline in which focal traders continue training on historical data alone (H). Across all three conditions, focal traders start from the same models, update every five days, and use the same training budget. Within each peer condition, we vary which new observations focal traders can use.

The table reports mean net returns for the 8 focal traders in a market of 32 traders, averaged over five seeds at day 120. Δ is the difference from H only under the same peer condition, in percentage points.

Focal training dataFrozen peers (B), returnΔ (pp)Learning peers (D), returnΔ (pp)
H only−0.324%0−0.320%0
H + samples from stocks the trader traded−0.276%0.048−0.274%0.046
H + public-market samples−0.200%0.124−0.165%0.155

These results illustrate how FML models information sharing through access to public market observations. In both peer conditions, adding new observations from a trader’s own stocks increased the average focal return relative to H only. Access to the broader public pool increased it further. Average returns remained negative, so the improvement took the form of smaller losses.

Reviewer_GTEDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores how learning shapes the collective behaviors of trading agents. Through simulations of agents with different learning settings, the authors find some interesting observations. For example, they discover that learning can produce both differentiation and convergence.

Critical strengths
  • Interesting topic. Learning dynamics and collective behaviors are important topics for behavioral science. And the experiments designed by the authors reveal several interesting observations. If these observations can also be validated on humans, it would be more influential and inspiring.

  • Clear experimental designs. The authors have properly designed the comparative experiments, which support the central claim of the paper.

Critical weaknesses
  • Unclear significance and implications. The connection between supervised parameter updates and human/institutional adaptation is not sufficiently justified. It remains unknown what transferable insights provide beyond the particular simulated market.
  • Insufficient explanations of mechanisms. The current experiments cannot adequately explain why the observed patterns arise.
Questions for authors
  • What are the implications on human behavior, real-world markets, or agent designs?
  • Would the results change if the training data were presented in a different format, for example, using natural-language data for the Qwen model?
1 comment

Significance and implications

Thank you for this important question. FML studies a market in which traders can learn over time. Each trader makes predictions, submits orders, observes the results, and updates its model. This process provides a simplified representation of periodic model retraining in quantitative trading. It does not fully represent human learning, but it allows us to study several basic aspects of adaptation, such as when traders update and what data they use.

Our findings have three main implications:

  • Agent design: Updating more often or using more data does not always improve returns when other traders are also learning. The update schedule, training data, and behavior of other agents should therefore be considered together.
  • Agent evaluation: An agent that performs well against fixed opponents may behave differently when its opponents also learn. FML can be used as a benchmark for testing agents in such learning environments.
  • Market research: FML provides a controlled environment for studying questions such as crowding, timing advantages, and information sharing. The results can help form hypotheses for later tests with real-market data or human participants.

Natural-language inputs

We further compared two output and update methods for Qwen 2B. Both methods use the same numerical factors converted into text and the same 1,285-day historical window.

Qwen 2B output routeInitial trainingOnline updatesFrozen returnOnline return
Numerical HeadLoRA + HeadHead only-1.60%0.06447
Formatted numerical textLoRALoRA only-2.41%-2.80%

The Numerical Head achieves a clear gain after online updates, while direct numerical-text generation does not. This result supports our current design for stable online learning.

However, the two methods also update different parameters, so this experiment does not isolate the effect of output format alone. In addition, both methods use numerical factors rather than news or other natural-language data.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.