acceptodds
Withdrawn from ICLR 2027

Universe of Thoughts: A Computational Framework for Creative Reasoning in Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advances in Large Language Model (LLM) reasoning have improved conventional problem solving, but creative reasoning remains comparatively underexplored. Inspired by cognitive science, we formalize combinational, exploratory, and transformational creativity as executable computational operators over structured problem and solution spaces, specifying how each mode combines, explores, or transforms those spaces, respectively. Combinational reasoning transfers ideas across domains to form unfamiliar combinations; exploratory reasoning searches for new solutions within an existing conceptual space; and transformational reasoning modifies the rules or constraints that define that space. This formalization yields distinct algorithmic procedures, which we directly instantiate in Universe of Thoughts (UoT), an LLM reasoning framework. Existing creativity benchmarks emphasize either open-ended ideation or highly constrained problem solving. We therefore introduce three novel creative-reasoning tasks requiring concrete solutions in low-constraint settings. Across 10 generations per method and task, T-UoT with GPT-4o performs strongest on the low-constraint, high-objective-specificity Bridge and Electricity tasks, while C-UoT shows its strongest relative performance on the low-constraint, lower-objective-specificity Society task. In addition, we evaluate UoT on HypoArena, an independent scientific hypothesis-generation benchmark with 100 tasks across biomedical, machine-learning, and social-science domains. With Qwen3-14B, Exploratory UoT ranks first among seven reasoning methods, achieving a 32.7% pairwise win rate compared with 25.5% for the next-best method. Our results suggest distinct performance patterns across task structures: T-UoT is strongest in low-constraint, high-specificity settings, E-UoT in more constrained, high-specificity settings, and C-UoT in low-constraint, lower-specificity settings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Creative Reasoning, Computational Creativity, Agentic Reasoning, Scientific Hypothesis Generation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Yo4nOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces Universe of Thoughts (UoT) framework by operationalizing Boden's three modes of creativity, namely, combinational, exploratory and transformational creativity into three LLM search procedures: C-UoT, E-UoT and T-UoT. C-UoT combines solutions through analogy, E-UoT explores the search space for unexplored solutions and T-UoT transforms the underlying rules of the problem to find new solutions.

For evaluation, the authors consider a setup characterized by two properties: degree of constraint and objective specificity. As this allows a 2x2 task regime, they choose at least one task per quadrant where tasks for two quadrants are chosen from the HypoArena benchmark and tasks for the other two are introduced by the authors. They compare their method to other reasoning strategies such as CoT, ToT, and EGoT using some open-source and closed-source models. They score generations using both humans and LLMs. They show that different reasoning methods perform best on those tasks that are more suitable for it in how it is operationalized with respect to the 2x2 task regime.

Critical strengths

[S1] The paper is well-written, clear and appropriately structured.

[S2] The proposed framework is theoretically grounded in computational creativity research.

[S3] The authors evaluate their methods on both their proposed tasks and also on an external benchmark and compare to both frontier LLMs and other similar reasoning methods.

Critical weaknesses

[W1] There seems to be some coupling and task-specific behaviour between the different search procedures and it is not clear why this is the case. T-UoT algorithm uses a CREATIVESEARCH procedure once a transformation is applied, but this procedure is instantiated to E-UoT in the novel creative reasoning tasks and C-UoT in the HypoArena task. This is in contrast to the main claim that the three modes are distinct and it is not clear whether T-UoT's success on the novel reasoning tasks can be attributed to the transformation alone.

[W2] Some of the reported creativity scores are close to each other and there are no confidence intervals, significance tests or inter-rater reliability results among the ten human raters.

[W3] The separation of the utility and feasibility is not well supported. Utility or usefulness aspect of creativity generally covers all those dimensions of value for an artifact. Implausible or unqualified solution can not really be useful, so these dimensions are coupled.

[W4] The corpus of known solutions is too small (3-4 solutions per problem) to provide a meaningful novelty measure.

[W5] While the 2x2 task regime seems to nicely map onto the types of creativity, it is not clear how those tasks satisfy different degrees of constraints and objective specificity? How do we determine that Society task is low constraint and low specificity and Bridge and Electricity are high-specificity? Is it even possible to quantify these dimensions?

[W6] The novelty of the introduced reasoning tasks is questionable. These are just a few problems that can be at most part of a broader Design Solutions Task (Distefano et al. 2025).

[W7] While the work is theoretically grounded in the computational creativity literature (Boden's framework), it does not discuss any other works that have (at least theoretically) operationalized these types of creativity. A notable one is Geraint A. Wiggins 2006, "A preliminary framework for description, analysisand comparison of creative systems". I think the paper should discuss how their framework relates to or differs from this work.

Questions for authors

See weaknesses above.

Reviewer_Qm7DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Universe of Thoughts (UoT), an LLM reasoning framework inspired by theories of creativity in cognitive science. The authors operationalize creative reasoning as three search procedures: combinational reasoning (C-UoT), which transfers mechanisms from other domains; exploratory reasoning (E-UoT), which expands the set of candidate solutions under existing rules; and transformational reasoning (T-UoT), which modifies some rules or assumptions to open up a new solution space.

The paper evaluates UoT on three newly designed creative problem-solving tasks and the HypoArena scientific hypothesis-generation benchmark, comparing it with methods such as OnePrompt, CoT, ToT, and EGoT. On the three new tasks, results with GPT-4o show that T-UoT scores relatively well on Bridge and Electricity, while C-UoT has a relative advantage on Society. On HypoArena, E-UoT ranks first with Qwen3-14B. The paper also includes ablations and failure-case analyses to examine the roles and limitations of the different search mechanisms.

Critical strengths
  1. Decomposing creativity into distinct search modes is an insightful perspective. Combinational, exploratory, and transformational reasoning involve different search targets and operations, providing a framework for studying how LLMs can generate solutions that are both novel and useful.

  2. The paper compares different creative search strategies within a common framework. UoT instantiates the three modes as concrete reasoning procedures, making it possible to compare them with baselines such as OnePrompt, CoT, ToT, and EGoT.

  3. The evaluation covers different types of tasks. The paper introduces low-constraint creative problem-solving tasks and also evaluates UoT on evidence-grounded scientific hypothesis-generation tasks from HypoArena, using both human and automatic evaluations.

  4. The paper includes mechanism ablations and failure analyses. It analyzes failure modes such as losing analogical mechanisms during synthesis and failing to fully reason through the consequences of rule transformations. It also reports interventions targeting these issues, helping to explain the sources of performance differences.

Critical weaknesses
  1. The source of the performance gains has not been isolated with budget-matched controls.
    Appendices B and C provide UoT pseudocode, some default parameters, and selection rules. Appendix C also states that E-UoT and ToT use the same voting interface. Table 18 reports inference costs on a 10-task HypoArena subset: E-UoT uses an average of 36.7k tokens, compared with 27.1k for ToT. These details help clarify the implementation and its cost, but they do not establish that the methods were compared under the same inference budget or candidate count. It therefore remains unclear how much of the observed gains comes from the C-/E-/T-UoT search strategies themselves and how much comes from the candidate-selection effect of generating and filtering more candidates. I encourage the authors to add budget-matched controls—for example, have CoT or ToT generate and select candidates under the same token budget and candidate count—to assess more directly the incremental contribution of UoT’s search strategies.

  2. The number of newly designed tasks is limited, and the relationship between task structure and method performance requires further validation.
    The paper introduces three tasks—Bridge, Electricity, and Society—and explains the performance of different UoT modes in terms of constraint level and objective specificity. However, each task structure is represented by only a small number of tasks, making it difficult to determine whether the results support general conclusions about which creative search mode is suitable for which type of problem. The operational definitions of constraint level and objective specificity, as well as the assignment of tasks to these categories, also require further validation. I encourage the authors to add task instances from more domains and test whether the observed trends are robust rather than driven by the characteristics of individual tasks.

  3. HypoArena does not directly measure creativity, and the current evidence is insufficient to establish robust cross-model patterns.
    Appendix E.3 explicitly states that HypoArena is not a direct measure of creativity. Its primary scoring dimensions concern contextual grounding and evidential justification, although the Inferential Insight dimension partially rewards non-obvious insights that are supported by the context. Thus, HypoArena results can indicate performance in scientific hypothesis generation under that benchmark’s rubric, but they cannot directly establish that a particular UoT mode is more creative. I encourage the authors to distinguish scientific hypothesis quality from creativity and qualify their interpretation of the HypoArena results accordingly.

    The paper also reports results with two backbones, Qwen3-14B and GLM-4-9B. E-UoT ranks first with the former, while OnePrompt ranks first with the latter and E-UoT is the best-performing structured method. The authors report this reversal and acknowledge backbone dependence. The concern is that results from only two backbones are insufficient to establish general rules about the applicability of each search mode. The authors should present cross-model conclusions cautiously and, where possible, evaluate a broader range of model families.

    For the three newly designed tasks, the paper reports that each method generates 10 answers and provides mean scores across generations, but it does not report variances, confidence intervals, or significance analyses. I encourage the authors to report uncertainty and avoid interpreting small differences in point estimates as stable advantages.

Questions for authors

Please see the Critical Weaknesses section.

Reviewer_pNqyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper maps three kinds of proposed creativity onto three prompting pipelines called "Universe of Thoughts". It evaluates them on three design tasks and on 100 HypoArena hypothesis-generation tasks. The authors argue these modes' performance depends on how constrained and how the task is.

Critical strengths

The paper is a straightforward attempt to put Boden’s modes in one paper. The failure analysis, which determines where each pipeline performs poorly, is a clear idea.

Critical weaknesses
  • The “formalization” is notation for a sequence of prompts. It doesn’t constrain or predict anything, and is close to existing analogical prompting and earlier work on Tree of Thoughts

  • The new-task evidence is minimal

  • The novelty score is embedding distance from 3–4 author-chosen outputs

  • The constraint by specificity regime story is fit after the fact to three tasks and three domains

  • One-shot prompting wins outright with GLM-4-9B

Questions for authors

None.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.