acceptodds
Under review as a conference paper at ICLR 2027

Reverie: An Inference-Time Cognitive Architecture for Continuous LLM Deliberation

Abstract

LLMs are reactive: a prompt arrives, the model computes, and cognition halts until the next prompt. Existing multi-agent LLM systems inherit this constraint: they coordinate multiple calls per response but still respond only when prompted. We introduce Reverie, an inference-time architecture in which a set of specialised agents runs continuously on a shared workspace where ideas accumulate, decay, and compete for influence. An executive composes a response when the user speaks, or on its own when accumulated internal activity crosses a threshold; between turns, the system keeps thinking. On a downstream task of research assistance over open-ended scientific questions, five independent human raters, blinded to system identity, preferred Reverie's conversations over Mixture-of-Agents on pairs (; mixed-effects CI –, ), with every rater preferring Reverie. At matched compute, Reverie also produces more conceptually distinct thoughts per turn than Mixture-of-Agents and more than Self-Consistency (Vendi Score, Bonferroni-significant), at roughly lower cloud-equivalent cost per session than Mixture-of Agents. On nine verifiable benchmarks (factual QA, math, code) Reverie is statistically tied with Mixture-of-Agents on eight and worse on rare-fact retrieval (PopQA), so the cost of continuous deliberation on convergent tasks is small.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM agents, multi-agent systems, continuous deliberation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YArBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents Reverie, a multi-agent LLM framework that aims to enable continuous reasoning beyond the usual prompt-response setting. The framework consists of eleven specialized agents that share information through a persistent blackboard. These agents generate thoughts, evaluate ideas, maintain memories and track emotional states. The system also includes an initiative mechanism that allows it to respond without receiving a new user message.

I appreciate the authors releasing their source code for reproducibility. But unfortunately I noticed several inconsistencies between the code and the paper. Some of these directly affect the main claims which makes me less confident in the reported results.

Critical strengths
  1. I like the motivation behind maintaining a continuous cognitive process across user interactions. Most multi-agent frameworks are organized around individual prompts. So IMO exploring persistent internal deliberation is a good direction to work on. The combination of a shared blackboard, specialized agents and autonomous initiative gives the work a distinct architectural focus.
  2. The authors evaluate the framework from several perspectives including diversity, coherence, human preference, naturalness and verifiable-task performance. I liked their experiments with different thinker-model sizes. These help provide a broader picture of the frameworks behavior.
Critical weaknesses
  1. The paper argues that persistent reasoning across turns is responsible for the improvements. But the BareThinkers ablation removes several components at once (shared workspace, continuous reasoning, feedback between agents etc.). I understand the authors tried to match the number of generated thoughts but this does not necessarily mean the systems receive the same amount of computation or information. So it is difficult to tell which part of the architecture actually drives the improvements. A compute-matched baseline that retains agent interaction but removes persistence would make the central claim more convincing.

  2. Out of curiosity, I checked a few source code files of this work. Unfortunately, I found several major inconsistencies between the code and the paper. For example, one of the central claims in main paper and Appendix is that the executive does not receive multi-turn conversation history and that cross-turn information is carried entirely through the persistent cognitive substrate. But in the released executive.py (lines 120–127),

I can see this line: " conversation = self.blackboard.conversation_history[-3:] "

So the executive appears to receive recent conversation history directly, right?

Similarly, bare_thinkers_batch.py (lines 103–110) also passes conversation history to the executive despite Appendix H describing BareThinkers as having no multi-turn history. So if I am not wrong, this directly contradicts the explanation given in the paper. If the executive can already access previous conversation messages, it becomes difficult to determine how much of the reported 31.4% reduction in cross-turn drift (Section 6.4) actually comes from the cognitive substrate. This is a serious concern because it affects the interpretation of one of the paper's main findings. A comparison where both systems receive exactly the same conversation history would make the claim more convincing(but before that I would expect a clear justification of the issue i mentioned).

  1. Similarly the paper describes Reverie as a continuously operating(asynchronous multi-agent system with a nominal two-second tick interval). But in society.py (lines 67–70)

the thinker agents appear to be executed sequentially:

shuffled = thinkers[:]
random.shuffle(shuffled)
for thinker in shuffled:
    await thinker.run_tick(self._tick)

If I correctly understand this implementation, each thinker must finish its LLM call before the next thinker can execute. So the actual duration of a tick could be much longer than two seconds depending on the inference time of the underlying model. This could be another issue because several mechanisms in Reverie depend on tick counts (such as attention decay, thought generation and the executive waiting period). If tick duration varies substantially, it is unclear whether the reported timing behavior corresponds to the architecture described in the paper. Could the authors please explain this issue?

  1. I dont think generating more diverse thoughts necessarily leads to better responses. Some of these thoughts could be irrelevant or less useful. Although the human preference study provides some supporting evidence, I would have liked to see a more direct analysis of whether higher thought diversity actually correlates with response quality or human preference.

  2. The idea of reasoning between user interactions has already been explored in Sleep-time Compute [1]. Although Reverie uses specialized agents and a shared blackboard,, I think the authors should clarify what additional benefits their approach offers over existing background reasoning methods.

Minor:

  • In Figure 1 & 2, the light gray text is difficult to read. Please use a darker color for better visibility.

  • The paper has some inconsistent citation formatting. For instance, in lines 115–123 “Self-Consistency Wang et al., 2023” should be “Self-Consistency (Wang et al., 2023).” The paragraph is also quite dense and difficult to read. I suggest shortening the paragraph and fixing the citation formatting.

[1] Lin et al., "Sleep-time Compute: Beyond Inference Scaling at Test-time," 2025. https://arxiv.org/abs/2504.13171

Questions for authors
  1. Please refer to Weaknesses where I have discussed my main concerns in detail. I am also happy to be corrected if I have misunderstood any of the points raised in my weaknesses.
Reviewer_Pv6sOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Reverie, a continuous multi-agent deliberation architecture that maintains a shared cognitive workspace between user turns, improving thought diversity and human preference on open-ended tasks.

Critical strengths
  • Moving from Reactive Agents to Continuous Deliberation is a valuable direction worth exploring for me.

  • I find the distinction between thought-level and response-level evaluation particularly interesting.

Critical weaknesses

I'm not an expert in this area, so my comments may be of limited value. But I will try my best to review the paper from a broader AI research perspective.

  • The ablations do not clearly distinguish the benefits of continuous deliberation from those of shared memory, inter-agent communication, and increased thought accumulation.

  • While the paper emphasizes continuous cognition and autonomous initiative, most experiments focus on diversity metrics rather than meaningful autonomous progress or long-horizon task performance.

Questions for authors

See weakness

Reviewer_Bui8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Reverie, a multi-agent LLM system that keeps running between user turns. Eleven agents share a blackboard and run on a 2-second tick loop. Four persona "thinkers" (analytical, creative, skeptical, empathetic; Qwen2.5-32B) post thoughts that decay over time. A critic scores the thoughts, and rule-based modules handle perception, censoring, episodic memory, emotion, and an "initiative pressure" score. A Gemini-2.5-Flash executive answers when the user speaks, or on its own when initiative pressure passes a threshold. Before answering, it waits about 40 s for thoughts to build up. Evaluation covers four experiments: (1) diversity of thoughts and responses over 40 topics × 12 turns with a simulated user; (2) a blinded 5-rater preference study against Mixture-of-Agents (MoA) on 60 open-ended questions; (3) nine verifiable benchmarks; (4) a hyperparameter and role sweep. Headline results: +15.8% within-turn Vendi over MoA, a 67% human preference over MoA, ties on 8/9 benchmarks, and a loss on PopQA.

Critical strengths

A useful ablation and honest reporting. The BareThinkers ablation keeps the same four persona prompts, model, token cap, and executive, but removes the shared blackboard. Within-turn Vendi drops by 22.5%, and the result holds under a second embedding model. This shows the diversity gain comes from agents seeing each other's thoughts, not only from the persona prompts. The paper also reports its negative results openly (the PopQA loss, the naturalness tie with MoA, and the memory/emotion ablation that loses significance after correction).

Critical weaknesses
  1. The main claim, that thinking between turns helps, is never tested. The paper's selling point is that the system keeps thinking between turns and can speak on its own. But autonomous mode appears only in two demos, and most of the 11 components show no measurable effect: the initiative threshold, cooldown, critic, and censor change no metric, and memory/emotion is not significant after correction. The only knobs that matter (decay rate and the 40 s wait) just control how many thoughts pile up before answering. No ablation keeps the blackboard but clears it between turns, so the gains could come simply from four agents seeing each other's thoughts plus extra time.

  2. The comparisons with MoA are unfair. In the diversity experiment, MoA uses Claude and Gemini models while Reverie uses Qwen, so the +15.8% Vendi gap mixes the architecture with different models. In the human study, raters compared 200-word Gemini summaries of whole 12-turn conversations, not actual answers, and rater agreement is close to chance (Fleiss κ = 0.043). The 67% preference may reflect conversations that wander more widely and summarize better, not better answers.

  3. The efficiency claims are not supported. The "3× cheaper than MoA" result comes from pricing Reverie's Qwen calls at low cloud rates, while Reverie actually makes about 3× more LLM calls per session (about 175 vs. about 60). The token counts in Table 2 leave out input tokens, even though every Reverie call reads accumulated blackboard context while MoA's proposers see only the question. The cost of keeping the system running between turns is never measured; one demo ran for 6 hours on a single user message.

Questions for authors
  1. Can you add an ablation that keeps the blackboard within a turn but clears it between turns and turns off wandering? Without it, the claim that thinking between turns helps is not directly supported.

  2. Can you report total tokens (input + output) and LLM calls per session for all conditions, and rerun the Experiment 1 comparison with MoA using the same Qwen2.5-32B proposers? This would show whether the diversity and cost advantages come from the architecture or from model and pricing differences.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.