acceptodds
Under review as a conference paper at ICLR 2027

Exploring Consciousness in LLMs through IIT-Inspired Reward Learning

Abstract

Consciousness-like processing has been hypothesized to play a role in achieving Artificial General Intelligence (AGI). However, despite numerous studies seeking to assess consciousness in LLMs from the perspectives of various theories of consciousness, comparatively little attention has been devoted to evaluating models after explicitly training them to enhance consciousness-related properties, largely due to the inherent complexity of consciousness. Here, we adopt Integrated Information Theory(IIT) as a quantitative framework for assessing consciousness in a system. This allows maximization of such metrics through Reinforcement Learning(RL). In this theory, a system is viewed as causal evolution of states, and the theory provides a formal, axiom-based mathematical framework for quantifying consciousness. Inspired by its postulates, we formulate three novel reward functions to measure some or all properties associated with consciousness. Our results show that the intrinsic information () reward enables substantially more concise reasoning, reducing response length by up to 70% while maintaining comparable accuracy and improving reasoning faithfulness, consistent with the Information Postulate. On the other hand, training with the -reward, an IIT-based measure that incorporates all IIT postulates, did not yield strong empirical evidence for consciousness. Nevertheless, the proposed framework provides a basis for more concrete empirical investigation of the open question of whether LLMs can exhibit consciousness.

Primary area:
reinforcement learning
Keywords:
Consciousness, LLM, IIT

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UkuROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores whether rewards derived from Integrated Information Theory (IIT) can enhance consciousness-related properties in LLMs through reinforcement learning. The method applies PCA and binarization to token representations, constructs transition probability matrices, and computes three rewards based on intrinsic information IintI_{\mathrm{int}}, maximal system integrated information Φs∗\Phi_s^*, and global Φ\Phi. Each reward is separately combined with an accuracy reward to fine-tune DeepSeek-R1-Distill-Qwen-7B and Qwen3-8B using GRPO. Evaluation covers general reasoning, consciousness-related behavioral tasks, and reasoning faithfulness. The IintI_{\mathrm{int}} reward produces shorter responses and higher faithfulness scores than the baselines, while accuracy changes vary across models and tasks. The more complete IIT rewards do not consistently increase during training, and the behavioral evaluations do not establish consistent improvements in consciousness-related capabilities.

Critical strengths
  1. The paper extends earlier IIT analysis to model training. This provides a concrete way to test how optimizing these measures changes LLM behavior. The IintI_{\text{int}} reward clearly reduces response length on both base models without an explicit length penalty. This gives the method practical value for producing shorter answers.

  2. The experiments cover both general reasoning and consciousness-related tasks. Reporting accuracy alongside response length helps readers judge whether shorter answers preserve task performance.

Critical weaknesses
  1. The paper does not establish that PCA and binarization preserve the causal structure required by IIT. This weakens the interpretation of the computed rewards as measures of consciousness. The equal weighting of accuracy and IIT rewards is insufficiently justified, and their interaction receives little analysis. The weak results of the more complex rewards could therefore reflect the training settings as well as the reward design.

  2. Figures 3 and 4 report very different ranges for IintI_{\text{int}} without explaining differences in data, scaling, or averaging. This makes the reward values difficult to interpret and verify. Table 2 shows different effects across the two base models, but the authors provide little explanation. It remains unclear when each reward improves accuracy, reduces length, or sacrifices one for the other.

Questions for authors
  1. What justifies interpreting the reduced token representations and their observed state transitions as the causal structure required by IIT?

  2. Why do the IintI_{\text{int}} values in Figures 3 and 4 have such different ranges? What data, normalization, and averaging were used for each figure?

  3. How do you explain the different effects in Table 2, particularly the accuracy gain and longer responses from Φ\Phi on Qwen3, compared with its accuracy decline on R1-Distill?

  4. How were reward scales handled when combining each IIT reward with accuracy? Was equal weighting evaluated against other choices, and could this balance explain the weak reward growth in Figure 4?

1 comment

We thank the reviewer for the helpful feedback.

Response to Q1: IIT Mapping and Causal Interpretation

We agree that PCA, binarization, and observationally estimated state transitions do not, by themselves, establish preservation of the interventional causal structure required by IIT. Our objective is not to claim an exact causal reconstruction of the LLM according to IIT, but rather to develop a computationally tractable, IIT-inspired operationalization that enables response-level reward computation and reinforcement learning, as explicitly reflected in the paper's title.

IIT's Information Postulate emphasizes that experience is specific rather than generic (Lines 168–169; Appendix B.3). Accordingly, IintI_{\mathrm{int}} favors states with selective and informative transitions. Our token-level analysis (Appendix B.2) supports this principle, while optimizing IintI_{\mathrm{int}} yields more concise reasoning, comparable accuracy, and improved faithfulness.

Despite the computational approximations introduced in LLM-Map-IIT, these observed behaviors are consistent with the information-specificity principle underlying IIT's Information Postulate. This empirical consistency suggests that our approximate framework retains practical value in operationalizing IIT-inspired information properties for LLM optimization.

Response to Q2: Reward Scale Differences Between Figures 3 and 4

The difference in the reported ranges of IintI_{\mathrm{int}} between Figures 3 and 4 arises from the application of the reward transformation function f(x)f(x).

Specifically, Figure 3 reports the raw reward values before applying f(x)f(x), whereas Figure 4 presents the transformed reward values after applying f(x)f(x), where:

f(x)=1−e−αxf(x) = 1 - e^{-\alpha x}

Importantly, applying the same transformation to the reward values in Figure 3 changes the Pearson correlation coefficient only slightly, from −0.63-0.63 to −0.65-0.65, without altering the qualitative findings or the conclusions drawn from the analysis.

Response to Q3: Model-Dependent Effects of the Φ\Phi Reward

Regarding accuracy, the Φ\Phi-based model improves the average performance on consciousness benchmarks by less than 2% over the Base model for Qwen3-8B, while exhibiting a decrease of less than 1% for DeepSeek-R1-Distill-Qwen-7B. This model-dependent behavior may be related to the limited learnability of the Φ\Phi reward, as discussed in Section 3.4 (Lines 458–468), where Φ\Phi does not exhibit consistent improvement during reinforcement learning.

To address the reviewer's concerns raised in Q3 and Q4 and further investigate whether the weak growth of the Φ\Phi reward in Figure 4 reflects limitations in its learnability, we conducted additional experiments evaluating the predictive learnability of the IIT-based rewards.

Specifically, for each prompt in a set of prompts, we generate more than 128 responses and compute the corresponding IIT-based reward for each response. The responses are then ranked according to their reward values, with the top 10% labeled High Reward and the bottom 10% labeled Low Reward. Using the resulting dataset, we train Logistic Regression and Multi-Layer Perceptron classifiers to distinguish between high- and low-reward responses and evaluate their predictive performance on a held-out test set. Across both reference models, the Φ\Phi reward achieves AUROC values of approximately 0.45–0.55, close to random discrimination, whereas the IintI_{\mathrm{int}} reward consistently achieves AUROC values above 0.70.

These findings suggest that the limited improvement of Φ\Phi may be associated with the difficulty of learning its reward signal, rather than solely with the choice of reward weighting. In contrast, the higher learnability of IintI_{\mathrm{int}} is consistent with its more stable improvement during training.

Importantly, as highlighted in Lines 467–468, the learnability of Φ\Phi reward is identified as a key research question. As noted in Lines 483–485, the proposed framework provides a foundation for systematically investigating this question in future work.

Response to Q4: Reward Scaling and Equal Weighting

Since the raw IIT-based rewards are not necessarily bounded between 0 and 1, we apply a nonlinear normalization function before combining them with the accuracy reward:

f(x)=1−e−αxf(x) = 1 - e^{-\alpha x}

The scaling coefficient α\alpha is determined based on the median raw reward computed over a diverse set of LLM-generated responses. Specifically, we use:

  • Φ\Phi: α=0.09\alpha = 0.09
  • Φs∗\Phi_s^*: α=2.36\alpha = 2.36
  • IintI_{\mathrm{int}}: α=0.79\alpha = 0.79

This transformation is intended to bring the reward components onto more comparable scales, making equal weighting a reasonable initial choice.

Reviewer_M94wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is an ambitious and genuinely valuable contribution: the authors tackle a hard, timely question—whether IIT-inspired rewards can induce consciousness-like properties in LLMs—and do so with an unusually broad evaluation spanning reasoning, consciousness, faithfulness, and causal benchmarks, while honestly reporting null results for the main Φ\Phi-based rewards. The practical finding that an information-theoretic reward yields up to 70% length reduction with preserved accuracy and improved faithfulness is a real asset, and the systematic ablation of the LLM-map-IIT pipeline as a hyperparameter space shows careful experimental thinking. That said, the paper still has several flaws: the mathematical presentation contains numerous correctness issues (e.g., inconsistent definitions of Φ\Phi and Φ∗\Phi^*, questionable derivations in the IIT formalism, and at least one exponential-complexity claim that is correct only in a loose asymptotic sense), and several reproducibility gaps remain—including missing details on hyperparameter searches, ambiguous dataset splits, and incomplete documentation of the training pipeline. The related-work section would also benefit from tighter integration, as some citations are verified but not clearly connected to the paper's own claims, and the prose occasionally obscures rather than clarifies the technical contributions.

Critical strengths

This paper makes a serious, well-executed contribution to an ambitious research agenda, and several aspects of the work stand out as particularly commendable.

  • Honest reporting of null results. The authors transparently report that the consciousness-specific rewards did not produce positive evidence: the abstract states that training with the Φ\Phi-reward “did not yield strong empirical evidence for consciousness” (p1), and the main text likewise notes that Φ\Phi and Φs∗\Phi^*_s fluctuated or decreased during training. This candor is rare and valuable in a field where null results are often underreported, and it strengthens the credibility of the positive findings that are reported.

  • Broad and multi-benchmark evaluation. The evaluation is unusually comprehensive: five reasoning benchmarks, ten consciousness benchmarks comprising more than 11,000 samples (p7), seven faithfulness datasets with 7,000 samples, CausalBench, plus component and pipeline ablations, training dynamics, and test-time scaling. This breadth gives the reader confidence that the conclusions are not artifacts of a single benchmark or dataset.

  • Practically useful length-reduction result. The intrinsic information (IintI_{\mathrm{int}}) reward is shown to “enable substantially more concise reasoning” (p1) while preserving accuracy and improving faithfulness. The clean component ablation demonstrates that both informativeness and selectivity contribute to this effect, making the result actionable for practitioners who need shorter, faithful reasoning traces.

  • Systematic treatment of the LLM-map-IIT pipeline as hyperparameters. The pipeline is not treated as a black box but is ablated along multiple axes: layer selection strategy, state dimensionality, thresholding method, and TPM construction mode. For example, Figure 9's caption reports the chosen configuration (Layer Selection Strategy = SOME, IIT State Dimensionality = 3, TPM Construction Mode = PROMPT) (p24), and Section E.2 shows that the Average thresholding method outperforms Median (p23). This level of detail makes the pipeline reproducible and helps future work understand which choices matter.

  • Reproducibility infrastructure. The authors state that “the complete source code, experimental outputs, and information required to access the datasets are provided in the accompanying GitHub repository” (p27), and compute details are documented in Section F. This is exactly the kind of infrastructure that enables independent verification and follow-up work.

  • Verified component ablation numbers. The component ablation in Table 6 is consistent with the prose in Section E.1: the reported highest accuracy on GSM8K (89.26%) and Math500 (91.16%) (p21) matches the table exactly. This internal consistency is a sign of careful experimental reporting.

  • Correct exponential-complexity claim. The statement that computing ϕs\phi_s requires searching over possible partitions (p18) is correct: the number of partitions of an nn-element set is the Bell number BnB_n, which grows faster than exponential. This justifies the need for the approximate methods used in the paper.

Critical weaknesses
  • Major — Reframe the contributions around demonstrated effects. The statement on p.2 that “We present a framework for inducing partial or complete properties associated with a conscious LLM” exceeds what the current evidence supports. Only the IintI_{int} variant shows positive effects on response length and faithfulness, while the Φ\Phi and Φ∗\Phi^\ast results are null. We suggest tempering the claim to something like: “We present a framework for inducing specific behavioral properties (reduced response length, improved reasoning faithfulness) in LLMs, with positive evidence for the Iint\mathrm{I}_{\mathrm{int}} variant; the Φ\Phi and Φ∗\Phi^\ast variants did not yield significant effects in our experiments.” This more accurately reflects the findings and avoids overclaiming.

  • Minor — Unbundle the Abstract’s headline sentence. The sentence “reducing response length by up to 70% while maintaining comparable accuracy and improving reasoning faithfulness” (p.1, Abstract) combines three distinct results into one claim. We suggest separating them into explicit, benchmark-specific statements, e.g., “reducing response length by up to 70% on CausalBench, maintaining comparable accuracy on reasoning benchmarks, and improving reasoning faithfulness on NSG.” This will make the contributions easier to verify and cite.

  • Major — Acknowledge ALP’s larger length reduction on CausalBench. The claim on p.6 that “Together with ALP, OurIint consistently achieves the largest reductions in response length across the evaluated benchmarks” is contradicted by the numbers on p.21: for Qwen3-8B on CausalBench, ALP reduces response length from 2542→2402542 \rightarrow 240 (a 90.6%90.6\% reduction), while OurIint reduces it from 2542→7682542 \rightarrow 768 (a 70%70\% reduction). We suggest correcting this claim to acknowledge ALP’s larger reduction on this benchmark, and qualifying the “largest reductions” statement to the specific benchmarks or settings where it holds.

  • Major — Correct the Section G claim about deception. The statement “although no increase in deception was observed” (p.24, Section G) is contradicted by Table 2(b): for Qwen3-8B, Deception rises from 69.7469.74 (Base) to 71.8771.87 (OurIint) and 73.5473.54 (Our$\Phi$). We suggest revising this sentence to accurately report the observed increase, and discussing possible interpretations and implications for the framework.

  • Minor — Qualify “requires no external data or auxiliary models.” The claim on p.3 that the framework “requires no external data or auxiliary models” should be scoped to the reward computation itself. Training uses OpenThoughts-114k-Math, the hybrid reward includes an accuracy term, and the faithfulness evaluation uses an auxiliary observer model (Qwen2.5-7B-Instruct). We suggest rephrasing to something like: “The reward computation itself requires no external data or auxiliary models, although training and evaluation do rely on standard datasets and an observer model.” This will prevent misinterpretation.

  • The paper is positioned at the intersection of IIT-based LLM consciousness assessment, RL-based length control, and reasoning faithfulness. We suggest making the dual home of the contribution explicit: the positive result (length reduction) belongs to the length-control literature, while the framing belongs to the consciousness literature. A sentence in the introduction or discussion acknowledging this explicitly would help readers evaluate the novelty in both communities (p1).

  • The core novelty claim — that this is the first work to train for consciousness-related properties — is defensible. Li (2025) and Shin et al. (2025) assess rather than train, and the Iint\mathcal{I}_{\mathrm{int}} reward is indeed a novel training objective. However, its mathematical form plog⁡(p/q)p \log(p/q) is standard IIT. The novelty is genuine but narrower than the framing suggests; we recommend tempering “first to train” to “first, to our knowledge, to train with an IIT-derived reward” (p1, p3).

  • Relatedly, the selectivity/informativeness decomposition of the Iint\mathcal{I}_{\mathrm{int}} reward is the standard IIT form, but it is not attributed to its IIT source; the only pointer is the Albantakis et al. (2023) citation for “further details.” Please attribute the decomposition explicitly to the IIT framework (e.g., Tononi, 2004; Albantakis et al., 2023) at the point where the reward is introduced (p3).

  • The duplicate Li citations should be merged: Li (2025a) and Li (2025b) are the same paper. Only “Natural Language Processing Journal” 12:100163 exists (DOI 10.1016/j.nlp.2025.100163), and the two entries differ only in journal name, quotes, and a hyphenation artifact. Please consolidate them into a single reference (References, p11).

  • Tononi (2004) should be removed from the list of LLM evaluations. It is the IIT theory paper (BMC Neuroscience 5:42) and contains no LLM evaluation; cite it only for the IIT framework, not as evidence about attributing consciousness to LLMs (p1).

  • The Butlin et al. (2023) attribution should be corrected. The verified abstract of arXiv:2308.08708 is a survey/assessment report and does not state that RL can induce consciousness-related capabilities. Please cite primary RL + agency/embodiment work instead for the claim that “RL can also induce consciousness-related capabilities, such as agency and embodiment” (p2).

  • Dehaene et al. should be cited once: the 2021 entry is a book-chapter reprint of the 2017 Science article “What is consciousness, and could machines have it?” The two entries are duplicate content; please keep one and cite it consistently (p10).

  • The Lee & Tong (2025) bibliography entry should be reformatted: the title currently appears after the arXiv ID, and the “arXiv preprint” prefix is missing. The paper itself exists and supports the claim, so this is purely a formatting fix (p11).

  • Consider citing “Quantifying Consciousness in Transformer Architectures: A Comprehensive Framework Using Integrated Information Theory and ϕ∗\phi^\ast Approximation Methods” (Ali, 2025, preprint) as the closest competing IIT-ϕ∗\phi^\ast measurement framework. The preprint quality is low, but it is the nearest methodological competitor, and acknowledging it would preempt concerns about coverage (A.2 or Section 2.3).

  • Please cite Chen et al. (2025) also for the premise that self-consciousness representations “can be acquired through targeted fine-tuning.” Currently the paper cites it only for the C0–C1–C2 methodology; the training-enhancement premise at p1/p2 is exactly where this additional citation belongs.

  • Please state whether the ALP baseline numbers are re-implemented or copied from Su & Cardie (2025), and report the source values and training details. The current provenance gap makes the comparison unverifiable (p6).

  • Please add published reference values for the base models on the evaluation benchmarks. Without them, the Base row cannot be checked against the published models’ performance, and the missing decoding settings make external comparison impossible (p6).

  • Please qualify the ALP faithfulness contrast. The statement “without comparable improvements in faithfulness” is currently absolute, but Table 3 shows ALP improves NSG over Base (8.03 vs. 7.3); only relative to OurIint (10.1) is it worse. We suggest rewording to “without the same level of faithfulness improvement as OurIint” (p2).

  • Undefined P′P' in Eq. (4). The symbol P′P' appears in Eq. (4) but is never defined elsewhere. If the intended quantity is IIT’s selectivity factor, the numerator should be PconstraintP_{\text{constraint}}, not an undefined P′P' rendered as P′/1P'/1. Please define P′P' explicitly or replace it with the correct probability.

  • Zero-probability handling for log-ratios. Both Eqs. (4) and (5) involve logarithms of ratios of probabilities. With per-prompt TPMs over 32 states (for IintI_{\text{int}}) or 16 states (for Φ\Phi), zero entries are guaranteed. Please specify the smoothing or clipping procedure used for log⁡(Pconstraint/Punconstraint)\log(P_{\text{constraint}}/P_{\text{unconstraint}}) and log⁡(Punpartitioned/Ppartitioned)\log(P_{\text{unpartitioned}}/P_{\text{partitioned}}); otherwise the quantities are undefined on the very data you analyze.

  • Constructive definition of the partitioned TPM in Eq. (5). The statement “the resulting TPM factorizes according to the imposed partition” is the entire specification. To make Φ\Phi computable from the paper, please provide the explicit cut/noise-injection formula, including how the partition is applied to the TPM and how the unpartitioned and partitioned TPMs are compared.

  • Reconcile C.1 with Eq. (4). C.1 states that a token receives positive reward when its intrinsic information is strictly greater than zero. But the log-ratio in Eq. (4) is negative whenever Pconstraint<PunconstraintP_{\text{constraint}} < P_{\text{unconstraint}}. Please state whether the reward is thresholded, clipped at zero, or allowed to be negative, and adjust C.1 accordingly.

  • Reconcile C.2 with the state definition. C.2 claims that intrinsic information excludes external inputs, yet the state is defined as “the contextualized representation of a response token after attending to the prompt.” Since the state is a function of the prompt, it depends on an external input. Please clarify what “external” means in this context, or revise C.2 to match the actual state definition.

  • Causal interpretation of a passively estimated TPM. The postulates require causal power, but the TPM is estimated from passive token transitions, and the statement “no state is externally pinned” rules out the interventions that IIT 4.0 requires. Please justify why a passively estimated TPM can support the causal claims made in the paper, or soften the language accordingly.

  • Zero rows in Eq. (1). The estimator Tij=Nij/∑kNikT_{ij} = N_{ij}/\sum_k N_{ik} is undefined for states with no outgoing transitions (e.g., a state appearing only as the last token). Please describe how zero rows are handled, e.g., by adding pseudocounts, assigning a uniform distribution, or excluding such states.

  • State space S\mathcal{S} for Eq. (1). The paper mentions a state space of size 2L×d2^{L \times d} but never names the reduced state set actually used after PCA. Please define S\mathcal{S} explicitly, including the number of discrete states and how the PCA-reduced dimensions map to states.

  • Disambiguate yy and y^\hat{y}. In Eq. (2), yy is the gold answer and y^\hat{y} is the model output; in Eq. (3), y=(y1,…,yT)y = (y_1, \ldots, y_T) denotes the generated response. Please use distinct symbols for the gold answer and the generated response, and define the answer-extraction procedure that produces y^\hat{y}.

  • Subscript in rconc(yt)r_{\text{conc}}(y_t). The subscript “conc” is used without explanation. Please define it (presumably “consciousness-related”) and state explicitly that Eq. (3) is instantiated with rconc∈{rIint,rΦs∗,rΦ}r_{\text{conc}} \in \{r_{\text{Iint}}, r_{\Phi^*_s}, r_\Phi\}.

  • Unify notation for the core reward. The same object is called IintI_{\text{int}}, iic/eii_{c/e}, iicauseii_{\text{cause}}, iieffectii_{\text{effect}}, iicandii_{\text{cand}}, iieii_e, and rconc(yt)r_{\text{conc}}(y_t). Please adopt a single notation throughout, or provide a table mapping the variants.

  • Formal relationships among ϕs\phi_s, ϕ∗\phi^\ast, Φs∗\Phi^\ast_s, and Φ\Phi. The hierarchy is never stated. Please define each symbol and its relation to the others. In particular, Table 1’s “Our$\phi^\ast_s$” (lowercase phi) conflicts with the text’s “Our$\Phi^\ast_s$” (uppercase Phi); please unify.

  • Scale of reward terms. The paper states “equal weighting” of Racc∈{0,1}R_{\text{acc}} \in \{0,1\} and the unbounded log-ratio RconcR_{\text{conc}}. Equal weights do not imply equal influence because the scales differ. Please analyze the relative scale, or normalize the terms before combining.

  • Substantiate the O(T3)O(T^3) complexity claim. No derivation is given, and it is unclear whether the cost applies to TPM estimation, cause/effect search, or reward computation. Please specify the operation and provide a brief derivation.

  • Rephrase B.2 in terms of probabilities. A large transition count NcxN_{cx} does not imply a large probability Ncx/∑kNckN_{cx}/\sum_k N_{ck} if the row total is also large. Please rephrase B.2 to refer to probabilities rather than counts.

  • Fix the axiom count. The paper says “IIT consists of five postulates” (p2) but later lists “The five axioms plus the zeroth axiom are: Existence, Intrinsicality, Information, Integration, Exclusion, Composition” (p4), which is six items and never identifies the zeroth. IIT 4.0 has six axioms; please correct the count and identify the zeroth axiom explicitly.

  • NSG denominator edge case in Eq. (6). If Accwithout exp=1\text{Acc}_{\text{without exp}} = 1, the denominator is zero. Please state the metric’s range (including negative values when the explanation hurts) and specify the handling of the degenerate case.

  • Verify NSG standard deviations in Table 3. Values of ±0.01\pm 0.01 across five independent runs on means of 7–31 are implausibly small and suggest the runs are not independent. Please verify the computation or report the actual variance.

  • Justify the uniform unconstrained distribution in Eq. (4). The paper states that the unconstrained probability is the average under a uniform distribution, but does not explain why a uniform distribution is appropriate for LLM token states. Please provide a justification or cite a source.

  • Stationarity assumption. Using a single first-order TPM estimated from token transitions assumes stationarity. Please state this assumption explicitly and test it, e.g., by comparing TPMs estimated from different segments of the responses.

  • Correct the B.1 weakening factor. Incrementing Nij∗N_{ij^*} by 1 changes another transition from Tij=Nij/NiT_{ij} = N_{ij}/N_i to Tij(1−1/(Ni+1))T_{ij}(1 - 1/(N_i+1)); the relative factor is 1−1/(Ni+1)1 - 1/(N_i+1) (where NiN_i is the row count), not 1/(n+1)1/(n+1) (sequence position). Please correct the formula and the surrounding text.

  • Re-derive Appendix B.1 for the actual optimized object. The argument analyzes the running average rii(0:n)r_{ii}(0:n) within one response, but GRPO optimizes the group-relative advantage of the response-level reward Rconc(y)=1T∑t=1Trconc(yt)R_{\text{conc}}(y) = \frac{1}{T}\sum_{t=1}^T r_{\text{conc}}(y_t). The “terminate early” conclusion does not follow from the running-average analysis. Please re-derive the claim for the actual objective.

  • Complete the B.1 “maximum value” argument. The reward is the sum of cause and effect intrinsic information, but the claim that a new state gives the maximum reward proportional to log⁡∣S∣\log |\mathcal{S}| only establishes the effect side; the cause side is dropped. Please complete the argument or state the assumptions under which the cause side is bounded.

  • State run counts per experiment. Tables 2/3/4/6 say “five independent runs,” Figure 4 says “two independent runs,” Section 3.4 describes four runs, and Table 1 states no run count. Please standardize and report the run count for every experiment.

  • Use a single symbol for the trained reward. Table 1’s header “Our ϕs∗\phi^\ast_s” (lowercase phi) conflicts with the text’s “Our$\Phi^\ast_s$” (uppercase Phi). Please unify the notation.

  • Verify Table 1(b) against the PDF. The Qwen3-8B block appears to show 29 values where 30 are expected (HumanEval row for Our Φs∗\Phi^\ast_s and Our Φ\Phi missing). Please verify the table against the data and correct any transcription errors.

  • Verify Table 1’s std block. The extraction shows duplicated sequences (e.g., “±0.53/±8, ±0.90/±49, ±2.11/±137, ±1.61/±59” appears twice). Please check the table for accidental duplication.

  • Correct Figure 8’s example. The OurIint explanation describes the reference patient (15–19mm, upper-left quadrant) while the prompt asks about the counterfactual patient (25–29mm, lower-left quadrant), labels RECURRENCE, and concludes “points towards a lower risk of recurrence” — contradicting its own label. Please verify against the PDF for transcription artifacts and correct the example.

  • Fix misspelled equation subscripts. Eq. (4) uses PconstraintP_{\text{constraint}} and PunconstraintP_{\text{unconstraint}}, while the prose defines “constrained probability” and “unconstrained probability.” Please align the spelling (e.g., PconstrainedP_{\text{constrained}} and PunconstrainedP_{\text{unconstrained}}).

  • Map figure-caption TPM mode names to E.2. The figure captions use “Trajectory, Prompt, Batch” but E.2 defines these modes only in prose. Please state the mapping explicitly, e.g., “Trajectory = using only the response corresponding to the same prompt.”

  • Name the TPM construction mode used in the main experiments. The text says “a TPM is constructed from model-generated responses for each prompt,” but E.2 lists three modes (Trajectory/Prompt/Batch), and the main-experiment mode is never explicitly identified. Please name the mode used in the main experiments.

  • Specify the PCA fit population and variance explained. The paper states only the number of components (5 for Iint, 4 for Φ\Phi). Please specify whether PCA is fit per-prompt, per-batch, or globally, and report the variance explained by the retained components.

  • Specify the population for binarization mean. The paper mentions “mean-based thresholding” but never states whether the mean is computed per-response, per-prompt, or globally. Please specify the population over which the mean is taken.

  • Weaken or substantiate “consistent with the Information Postulate.” The B.1 derivation is flawed and the r=−0.63r = -0.63 correlation is under-specified, so the causal claim is not established. Please either provide a rigorous derivation or soften the claim to a suggestive observation.

  • The B.1 saturation claim is empirical, not mathematical. The statement “the state space becomes increasingly saturated” depends on the number of responses per prompt, not just sequence length, when using a per-prompt TPM. Please rephrase as an empirical observation and support it with data.

  • State the TPM timing in B.1. The reward for token tt depends on the TPM built from the complete response, so the narrative of the TPM “containing a single value of 1” during generation is only coherent if the TPM is built incrementally, which contradicts the stated construction. Please clarify the timing: is the TPM fixed before generation, or updated online?

  • Specify the correlation basis. The statement "the Iint reward exhibits a strong inverse correlation with response length (r=−0.63r = -0.63)" (p7) is not verifiable as written. Please specify the computation basis: across tokens, responses, training steps, and for which model. For example, state whether rr is computed over all generated responses at a fixed step, over per-step means, or over individual tokens, and report the corresponding nn and pp value.

  • Define or remove settings_76. The compute section (p24, Section F) refers to "Training for settings_76" without defining this term. Please either define the configuration it denotes or replace it with an explicit description of the setting.

  • Define the C0 level. The C0–C1–C2 framework of Dehaene et al. (2017) is cited (p7), but only C1 and C2 are described. Please add a one-sentence definition of C0 so the framework is self-contained.

  • Report core training hyperparameters. The statement "The main hyperparameters and experimental settings are reported in the paper" (p27) is not yet accurate. Please report, for every trained model: learning rate, batch size, optimizer, GRPO group size (the paper mentions "the model generates a group of candidate responses", p5, but not the group size), number of epochs, random seeds, and decoding settings (temperature, top-pp, max length). This is the single most important addition for reproducibility.

  • Describe the hyperparameter-tuning protocol. No tuning protocol is reported for any method, including the baselines (e.g., Su & Cardie, p6). Please describe how hyperparameters were selected, the search space, and the selection criterion (e.g., validation performance). If no tuning was performed, state that explicitly.

  • Report train/validation/test split ratios. The only mention of a split is "computed as averages over the validation dataset across different training steps" (p19, D.1). Please report the exact split ratios for every dataset, and state whether the same split was used across all models and baselines.

  • Report dataset versions and preprocessing. For each dataset, including OpenThoughts-114k-Math (Guha et al., 2025) (p5), please report the exact version or commit, and describe all preprocessing/filtering steps (e.g., deduplication, length filtering, format conversion).

  • Report benchmark sizes. Only the consciousness (>11,000) and faithfulness (7,000) dataset sizes are given. Please report the number of samples for each of the reasoning benchmarks as well.

  • Report software and model versions. The hardware and runtime are documented (p24), but library versions and pretrained-model versions/commits are missing. Please list the versions of the training framework, tokenizer, and all libraries, and the exact model identifiers (e.g., HuggingFace commit hashes) for Qwen3-8B, DeepSeek, and Qwen2.5-7B-Instruct.

  • Add significance tests or effect sizes. Results are averaged over five independent runs (p8), yet no significance tests or effect sizes are reported anywhere. The NSG standard deviations of ±0.01\pm 0.01 are implausibly small for five runs; please report per-run results, confidence intervals, or at least a paired test (e.g., Wilcoxon signed-rank) for the headline comparisons.

  • Describe the cross-validation scheme. No cross-validation scheme is described. If any form of cross-validation was used, please specify the number of folds and the assignment procedure; if not, state that a fixed split was used.

  • State the test-set protocol. The test-set protocol is never described. Please state explicitly how the test set was constructed, whether it was used only once, and how the reported numbers were aggregated over it.

  • Overall reproducibility blocker. In its current form, a competent peer cannot reproduce the central result from the paper alone. The blocking items are: missing training hyperparameters, underspecified reward functions, the ambiguous TPM mode, the missing answer-extraction procedure, the unspecified PCA/binarization populations, the unknown ALP provenance, and the absence of significance testing. Addressing the points above will resolve these blockers.

  • Qualify "comparable accuracy". The abstract claims "maintaining comparable accuracy" (p1). On Qwen3-8B reasoning Avg, OurIint is 77.3877.38 vs. Base 79.4779.47 (−2.09-2.09 points), and on consciousness Mean 60.8660.86 vs. 61.8161.81 (−0.95-0.95). Please acknowledge this non-trivial drop explicitly and discuss whether it is within the expected trade-off.

  • Specify the model for the harm claim. Section G (p24) states "the language model shows a slight increase in the harm metric under consciousness evaluation." This holds only for Qwen3-8B Our$\Phi$ (75.0375.03 vs. Base 72.3972.39); DeepSeek Harm decreases for all trained models. Please specify the model and variant in the claim.

  • Correct the length-reduction figure. The claim of "approximately 30% and 60%" (p7) is inaccurate for the Qwen3-8B consciousness-eval Mean row: (190−84)/190=55.8%(190-84)/190 = 55.8\%, not 60%60\%. The DeepSeek 30%30\% figure checks out. Please correct the percentage or the underlying numbers.

  • Train or defer the Exclusion-postulate reward. The reward ϕ∗\phi^* is introduced as "an alternative reward signal" (p19, C.5), but only Iint, Φs∗\Phi^\ast_s, and Φ\Phi are trained. Please either train ϕ∗\phi^\ast and report results, or explicitly mark it as future work.

  • Acknowledge the faithfulness domain gap. Models trained on math are evaluated for faithfulness on seven tabular datasets (Heart Disease, Pima Diabetes, etc.) with a different observer model (Qwen2.5-7B-Instruct, p20). Please acknowledge this domain shift explicitly and discuss its potential impact on the faithfulness conclusions.

Questions for authors
  1. Could you clarify whether the ALP baseline numbers in Table 3 are reproduced by you or taken directly from Su & Cardie (2025), and if reproduced, provide the exact source values, training details, and any hyper‑parameter settings used for a fair comparison?

  2. Please list the full training hyper‑parameters for each model (learning rate, batch size, optimizer, number of epochs, GRPO group size, random seeds) and the decoding settings (temperature, top‑p, max length) employed in your experiments, as these are currently missing and impede reproducibility.

  3. In Eq. (4) the symbol P′P' is undefined and the log‑ratio involves probabilities that can be zero; could you specify what P′P' denotes, and describe the smoothing or clipping strategy you use to handle zero‑probability entries in both Eqs. (4) and (5)?

  4. The paper mentions three TPM construction modes (Trajectory, Prompt, Batch) but does not state which mode is used in the main experiments; could you identify the mode applied for the reported results and explain whether the TPM is built before generation or updated incrementally during token generation?

  5. Regarding the reported correlation r=−0.63r=-0.63 between the IintI_{\text{int}} reward and response length, please indicate the exact aggregation level (e.g., per token, per response, per training step), the sample size nn, and the corresponding pp-value so that this claim can be verified.

  6. The standard deviations reported for NSG in Table 3 (±0.01) appear unrealistically small for five independent runs; could you provide the per‑run NSG values, confidence intervals, or appropriate significance tests to substantiate the reported improvements?

3 comments

We thank the reviewer for the insightful comments and constructive suggestions.

Q1

The ALP results in Table 3 are based on our own implementation of Su & Cardie (2025), not copied from their paper. Following their method, the adaptive penalty coefficient is updated as:

\qquad \lambda_0=\eta=10^{-3}.$$ Here, $A_{\mathrm{ref}}$ is computed separately for each base model before training, and $A_t$ is the accuracy at step $t$. All other training hyperparameters and evaluation settings are identical across methods, ensuring a fair comparison. ### **Q2** - **Learning rate:** 3e-6 - **Learning-rate scheduler:** Cosine - **Training steps:** 1,200 - **Batch size:** 8 or 36, depending on the reference model - **Temperature:** 0.7 or 1.0 - **Top-p:** 1.0 - **Maximum sequence length:** 5,000–15,000 tokens, depending on the model and benchmark ### **Q3** In Eq. (4), P' denotes the **backward probability** obtained by applying Bayes' rule to the forward transition probabilities in the cause TPM. This probability is used as the constrained probability in the selectivity factor on the effect side. Regarding zero probabilities in Eqs. (4) and (5), reward computation is restricted to states corresponding to generated response tokens. For each evaluated token, at least one valid cause and effect transition exists. Furthermore, cause/effect states with no transition relationship to the current state are excluded from the computation. ### **Q4: TPM Construction** In the main experiments, we use the **prompt-based TPM construction mode**, as stated in Line 260. The TPM is constructed **after the complete response has been generated**, as illustrated in Figure 2 (Lines 132–133). ### **Q5** The reported Pearson correlation was computed at the **per-response level**, using **104 generated responses** from the final training iteration. The results show a statistically significant negative correlation between the intrinsic-information reward and response length: $$r = -0.63, \qquad p = 7.11 \times 10^{-13}$$ ### **Q6** The unusually small STDs resulted from a **scaling inconsistency**: NSG means were reported as percentages, while STDs remained on a 0–1 scale. We corrected this by **multiplying the STDs by 100**: The **mean results and conclusions remain unchanged**. ### **Responses to Weaknesses** For clarity, our responses are numbered according to the order of weaknesses listed by the reviewer, starting with Weakness 1. **1.** Major — We agree that the term "inducing" may overstate our contribution. As clarified in the Abstract, Section 3.4, and Lines 468–469, we do not claim that our framework induces consciousness in LLMs. **3.** Major — Across different models and benchmarks, $I_{\mathrm{int}}$ achieves larger length reductions in some settings, while ALP performs better in others. However, both consistently outperform the remaining methods in length reduction, which motivated our use of the phrase “Together with ALP.” **4.** Major — Our intention was to highlight the absence of a consistent increase in deception, given the decreases observed for DeepSeek-R1 and the modest increases for Qwen3-8B. **7.** The Abstract (Lines 14–16) states "comparatively little attention," while the Introduction (Lines 50–53) explicitly uses "to the best of our knowledge. **24.** In our LLM-Map-IIT framework, we adopt an observational TPM as a practical approximation due to the computational challenges of constructing the interventional TPM required by IIT. Consequently, the resulting rewards should be interpreted as IIT-inspired measures rather than validated measures of consciousness, as reflected in the paper's title. **31.** We normalize the raw IIT-based rewards using: $$f(x) = 1 - e^{-\alpha x}$$ where $\alpha$ is determined from the median raw reward across diverse LLM responses, with values of 0.09 for $\Phi$, 2.36 for $\Phi_s^*$, and 0.79 for $I_{\mathrm{int}}$. **52.** IIT's **Information Postulate** emphasizes specificity over genericity (Lines 168–169; Appendix B.3). Accordingly, $I_{\mathrm{int}}$ favors selective and informative transitions, as supported by our token-level analysis (Appendix B.2). The resulting **concise reasoning, comparable accuracy, and improved faithfulness** are consistent with this principle. ### **Remaining Concerns** The authors have defensible responses to the remaining concerns, which are not detailed due to the rebuttal's length constraints. These concerns fall into four categories: 1. **IIT-related Challenges:** Computational, technical, and implementation limitations. 2. **Reproducibility:** Additional methodological and experimental details. 3. **Related Work:** Clarifications regarding relevant literature and prior studies. 4. **Minor Corrections:** Mathematical notation, formatting, and a few typographical errors.
AuthorsOpenReview ↗

Response to Remaining Reviewer Concerns

6. : As reflected in the title and abstract, our primary focus is IIT-based LLM consciousness assessment and reward learning. Response-length reduction and improved faithfulness are empirical outcomes of the proposed framework rather than its primary objectives.

8. : Appendix C provides the computational details, while Albantakis et al. (2023) is cited as a comprehensive IIT reference that explicitly presents the general selectivity/informativeness decomposition. We have not claimed this decomposition as our contribution anywhere in the paper.

9. : The duplicate citations have been merged into a single reference.

11. : Although Butlin et al. (2023) do not explicitly make this claim, they argue that agency and embodiment are considered necessary for consciousness and that typical RL systems satisfy substantive criteria for agency(pp. 37–39). Our statement was an interpretation based on these two arguments rather than a direct claim from the cited work.

14.: We agree that Ali (2025) is relevant to the related work and will include it in the revised manuscript. However, its inclusion does not affect the novelty or contributions of our work.

15.: We agree that Chen et al. (2025) provides relevant evidence that self-consciousness-related representations can be acquired through targeted fine-tuning. We will incorporate this citation into the Introduction to strengthen the motivation for our IIT-inspired reward learning approach.

16. :The ALP baseline (Su & Cardie, 2025) was independently re-implemented and trained by us;

18. : We clarify that "without comparable improvements in faithfulness" refers to ALP's smaller gains relative to OurIint, not the absence of improvement over Base.

19. : In Eq. (4), (P') denotes the backward probability obtained by applying Bayes' rule to the forward transition probabilities in the cause TPM. This probability is used as the constrained probability in the selectivity factor on the effect side. We will explicitly define (P') and clarify its computation in the revised manuscript.

20.: In our TPM construction, consecutive tokens in each generated response define observed state transitions. The IIT-based rewards are computed only for states corresponding to tokens present in the generated response, rather than over all theoretically possible states. Consequently, unobserved states are excluded from reward computation

21.: The cut operation follows the standard IIT 4.0 partitioning procedure rather than a newly proposed formulation. Its computational details are provided in Albantakis et al. (2023)

23. : By “external,” we mean that intrinsic information is computed based on the system's internal states, not that external inputs have no influence on these states.

25.: Reward computation is restricted to tokens for which valid cause–effect states can be identified; tokens without such states are excluded. Since the TPM is constructed from consecutive token transitions, each included state has at least one observed outgoing transition, ensuring a nonzero row sum and avoiding division by zero in Eq. (1).

26.: The reduced state space is defined as S={0,1}k\mathcal{S}=\{0,1\}^{k}, with ∣S∣=2k|\mathcal{S}|=2^k, where kk is the PCA-reduced dimensionality. The base 2 arises from binarizing each reduced dimension into either 0 or 1.

28.: Equation (3) already defines rconcr_{\mathrm{conc}} as the token-level reward. Explicitly listing the three reward functions at this stage may introduce ambiguity, as they are explained later in the section.

30.: The necessary clarifications regarding the definitions and relationships among the measures are provided in Lines 241–246. We have also resolved the inconsistencies in uppercase and lowercase notation throughout the manuscript.

32.: The explanation in Lines 189–193, together with Figure 2(a), appears to sufficiently illustrate the computational complexity challenge.

33.: Our discussion in Appendix B.2 assumes that the other transition probabilities remain approximately unchanged

35. : NSG can take negative values when explanations reduce accuracy. Although a zero denominator is theoretically possible, it is highly unlikely in practice.

37.: As clarified in Lines 926–927, the unconstrained probability on the effect side is computed as the average probability of a given next state across all possible current states, assuming a uniform distribution over those states.

38.: In our framework, the TPM is constructed after the LLM has generated its complete response, using the observed state transitions across the entire token sequence. Thus, the TPM represents an aggregate transition model rather than a matrix updated during token generation. Nevertheless, we acknowledge that this approach implicitly assumes stationary transition probabilities across the response.

AuthorsOpenReview ↗

Response to Remaining Reviewer Concerns

42. : All benchmark evaluations were conducted over five independent runs, whereas Figure 4 reports two independent training runs for each reward function and reference model.

44.: Table 1(b) contains all 30 expected values in the original PDF, and no corrections were found necessary.

45.: We have carefully verified the standard deviation values in Table 1 and confirmed that they are correct, with no errors or accidental duplication.

48.: To improve readability and clarity, we have explicitly aligned the TPM mode names with their corresponding definitions in Appendix E.2, as suggested by the reviewer.

49.: The Prompt mode used in the main experiments is specified in Line 260.

50.: As illustrated in Figure 2(a), PCA is fitted to the concatenated hidden-layer representations of response tokens

51.: The mean is computed over all token representations used to construct the TPM. This mean is then used as the threshold for binarization.

53.: Since the TPM has a fixed state space, increasing the response length can introduce previously unobserved states, thereby reducing TPM sparsity. Mathematically, the number of observed states is non-decreasing as more tokens are processed. However, the rate of state-space saturation also depends on the number of responses per prompt and their diversity.

54.: As illustrated in Figure 2(a), the TPM is constructed after the complete response is generated and remains fixed during token-level reward computation. The discussion in Appendix B.1 was intended as a conceptual analysis of how additional tokens affect transition statistics, not as an online TPM update mechanism.

57.: To maintain conciseness and avoid expanding the already lengthy manuscript, we omitted additional background details, including the definition of C0.

58 and 59: The key hyperparameters are reported in the manuscript, while the remaining hyperparameters are available in the source code for reproducibility.

60.: Fairness was maintained in dataset splitting, evaluation, and benchmark selection across all models and baselines. These procedures can be verified through the publicly available source code.

61.: All datasets, including OpenThoughts-114k-Math, are publicly available on Hugging Face or through their official source repositories. Their versions and preprocessing procedures, including filtering and format conversion, are reproducible from the corresponding source code.

63.: The details of the pretrained models and software library versions are available in our publicly accessible source code and Hugging Face repositories.

65.: The main benchmarks (Table 1) were evaluated using fixed in-domain and out-of-domain datasets. Further evaluation details are available in our source code.

66.: The complete source code provides details of all benchmarks, including the training and test data splits and their construction.

67.: As stated in the Reproducibility Statement, the paper and supplementary materials provide experimental settings and implementation details, while the accompanying GitHub and Hugging Face repositories provide the source code, experimental outputs, dataset access information, and trained models to support reproducibility.

68.: We intentionally used "comparable accuracy" rather than "maintaining accuracy" to acknowledge modest performance differences.

71.:We selected IIT-based measures that allow relatively simple and computationally tractable reward formulations.

72.: We acknowledge the domain shift between training and faithfulness evaluation. Some evaluation tasks, such as Annual Income, may indirectly benefit from mathematical reasoning skills, although the connection is weaker for other datasets. Furthermore, using a separate observer model is an inherent assumption of the NSG evaluation framework

Reviewer_CFykOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adopts Integrated Information Theory (IIT) as a quantitative framework for assessing consciousness in a system. The paper proposes a framework that maps LLM hidden representations into a low-dimensional discrete-state system and constructs transition probability matrices from generated token sequences. Experiments on DeepSeek-R1-Distill-Qwen-7B and Qwen3-8B show that intrinsic-information-based rewards can reduce response length while maintaining comparable reasoning accuracy.

Critical strengths
  1. Interesting to connect theories of consciousness with language-model optimization. The paper explores an underexplored research direction by applying concepts from integrated information theory to reinforcement learning objectives for LLMs.
  2. The proposed intrinsic-information reward reduces response length across several benchmarks while preserving task performance.
  3. The evaluation covers a wide range of datasets.
Critical weaknesses
  1. The authors construct discrete states by applying PCA and binary threshold to selected hidden representations, then estimate transition probabilities from generated token sequences. However, statistical transitions between observed representations do not necessarily establish the interventional causal relationships required by IIT.
  2. The effect of the intrinsic-information reward may primarily reflect implicit length regularization. Appendix B.1 shows that the intrinsic-information reward naturally favors shorter sequences because the marginal reward from generating additional tokens tends to decrease. Does the observed improvements primarily result from implicit length regularization instead of learning IIT-related properties? Could the authors compare against length-matched baselines to determine whether the improvements in reasoning faithfulness are independently of response-length reduction?
  3. The proposed method seems to just optimize a statistical measure computed from LLM output trajectories, not a direct measure of consciousness.
Questions for authors

See weakness.

1 comment

We thank the reviewer for the insightful comments and constructive suggestions.

W1 & W3: Observational TPM and Interpretation of IIT-Based Rewards

In our LLM-Map-IIT framework, each autoregressive generation step is represented as a discrete state derived from contextualized token representations. Given the autoregressive nature of LLMs, constructing the interventional TPM required by IIT remains computationally challenging. We therefore adopt an observational TPM as a practical approximation, acknowledging that statistical transitions do not necessarily establish the causal relationships required by IIT.

In our main experiments, the sequence-segmentation hyperparameter (line 216) is set to prompt, allowing TPM estimation from token transitions within each prompt and potentially capturing more diverse observed states. However, this does not constitute an interventional TPM.

Each step of LLM-Map-IIT is motivated by IIT principles, while practical approximations, including observational TPMs and PCA-based dimensionality reduction, address computational constraints arising from exponentially growing state spaces. The subsequent rewards follow the mathematical formulations of their corresponding IIT quantities. Nevertheless, due to these approximations, the resulting rewards, including Φ\Phi, should be interpreted as IIT-inspired measures rather than validated measures of consciousness, as explicitly reflected in the paper's title.

Importantly, IIT's Information Postulate states that experience is specific rather than generic, distinguishing a particular state from alternative possible states (Lines 168–169; Appendix B.3). Motivated by this principle, IintI_{\mathrm{int}} favors states with selective and informative transition relationships. Our token-level analysis (Appendix B.2) shows that selected reasoning-relevant tokens receive higher intrinsic-information rewards. Furthermore, optimizing IintI_{\mathrm{int}} leads to more concise reasoning while maintaining comparable accuracy and improving faithfulness.

Despite the computational approximations introduced in LLM-Map-IIT, these observed behaviors are consistent with the information-specificity principle underlying IIT's Information Postulate. This empirical consistency suggests that our approximate framework retains practical value in operationalizing IIT-inspired information properties for LLM optimization, although it does not establish preservation of the causal structure required by IIT.

W2: Effect of Response Length on Faithfulness

Following the reviewer's valuable suggestion, we investigated whether faithfulness improvements primarily stem from shorter responses by comparing IintI_{\mathrm{int}} against length-matched ALP baselines. Specifically, we introduced ALP+, which extends ALP with an additional target-length regularization term:

RALP+(x,y)=−0.5[log⁡(L(y)+1Ltarget(x)+1)]2R_{\mathrm{ALP+}}(x,y) = -0.5 \left[\log\left(\frac{L(y)+1}{L_{\mathrm{target}}(x)+1}\right)\right]^2

Here, L(y)L(y) is the generated response length, and Ltarget(x)L_{\mathrm{target}}(x) is the corresponding target length obtained from IintI_{\mathrm{int}}. This regularizer penalizes both shorter and longer deviations from the target.

We trained ALP+ using DeepSeek-R1-Distill-Qwen-7B as the reference model. Across all benchmarks in Table 1, ALP+ reduced the average response length from 2,162 (ALP) to 2,077, closely matching the 2,053 achieved by IintI_{\mathrm{int}}. This confirms that ALP+ successfully approximates the average response length of IintI_{\mathrm{int}}.

However, across five independent runs, ALP+'s faithfulness (NSG) improved only modestly from 8.03 to 8.8, remaining substantially below the 10.1 achieved by IintI_{\mathrm{int}}, despite their nearly identical average response lengths.

Overall, these findings suggest that the faithfulness improvements of IintI_{\mathrm{int}} cannot be explained solely by response-length reduction, indicating benefits beyond explicit length regularization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.