acceptodds
Under review as a conference paper at ICLR 2027

Perceptual Reality Transformer: What Must an Illustration Preserve?

Abstract

Text-to-image models can illustrate a perceptual account, but preserving its objects does not preserve how they were experienced. We introduce Perceptual Report Translation (PRT), a source-linked workflow and atlas, and examine the path from reports to representations to illustrations. Across 3,145 Ganzflicker records, narrative encoders predict complex content well (AUROC 0.887–0.894), while separate imagery/context responses improve access to vividness. A held-out retrieval audit shows that nearest narratives transfer content more reliably than vividness, persistence or affect. In 1,344 synthetic comparisons per encoder, counterbalanced wording and supervised probes separate similarity failure from recoverable distinctions. A 24-report image pilot finds high visible-detail coverage without a demonstrated structured-prompt advantage. Removing colour reverses all 45 eligible judge answers, yet none of the 69 original questions directly tests memory, uncertainty or analogy status. The atlas makes these gaps inspectable by retaining accounts, questionnaire responses, evidence and illustrations together. The results identify concrete requirements for perceptual communication: retain separate reporting channels, evaluate the deployed readout, and distinguish visible-content accuracy from preservation of the speaker's meaning.

Primary area:
applications to neuroscience & cognitive science
Keywords:
perceptual experience, representation learning, vision-language models, compositionality, generative models, computational cognitive science, human-AI communication

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LFQPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Not being a neuroscientist or psychologists I have a very hard time understanding what is done. The authors study the Ganzflicker records which I assume is a highly valuable resource in their field but not known to me. From what I understand the records are accounts of patients describing images from their memory and this is translated into a image to discuss with the doctor. Observation is that in the current dataset a lot of information that is in the records is not fully considered in the score that is given to an image. The paper is mostly considering questions like what information should be preserved in the image, whether structured prompts are useful or whether it is better to look for similar records.

Critical strengths
  • The work might be relevant from a psychology / neuroscientist point of view. Especially if this dataset is used as standard reference while not being optimal.
  • Careful study of different characteristics of the dataset and where it could be improved.
Critical weaknesses
  • Although it is submitted to the track applications in neuroscience and cognitive science I find it hard to see the match to ICLR.
  • The results of the study are only relevant for the dataset but it remains unclear how this would extend to the general scope of people describing images from memory (e.g. for retrieval), or better ways to go from prompt to image.
  • Text is not really accessible for the regular ICLR audience.
Questions for authors
  • What are the implications of your work beyond the Ganzflicker dataset?
Reviewer_wynCOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces Perceptual Report Translation (PRT), a workflow and atlas linking perceptual narratives, questionnaire responses, source-grounded annotations, and generated illustrations. The authors perform three studies to examine different stages of this process. These include prediction and nearest-neighbour transfer of reported qualities across 3,145 Ganzflicker records, synthetic diagnostics of qualification preservation in frozen text embeddings, and a 24-report comparison of direct, evidence-prose, and structured prompts for image generation. While the central problem is interesting, I think the claims are hard to follow and there are several issues/questions that need addressing.

Critical strengths

S1 - The central question is quite interesting. "how can an illustration preserve both what someone describes and how they describe experiencing it?" is interesting, and I think the authors should spend more time elaborating this (see W1).

S2 - I think that comparing structured PRT with evidence-matched prose is a useful control. It separates the effect of prompt structure from the effect of supplying additional annotations and instructions. However, I also think the experiment does not establish a benefit from structure.

S3 - The authors are transparent about the limits of their evidence.

Critical weaknesses

W1 - My biggest issue with the paper is how it is written. It is difficult to follow because it presents many analysis without a clear hierarchy of what matters most. In simple terms, the paper presents several experiments and individual results, but I find it hard to come away with a takeaway message after reading each section. It is difficult to state its main empirical takeaway or explain what I learned about perceptual communication. The paper has good experiments and an interesting problem, and would greatly benefit from a narrower argument with a clear statement of what it establishes.

W2 - Same as my previous point, the paper does not spend enough time establishing its central problem. The examples are intuitive, but the intended reader and communication task remain unclear. In particular, showing that an image alone cannot distinguish a memory from something currently seen does not establish that readers misunderstand an image presented alongside the original account. The paper needs clearer evidence of when this problem arises and why ordinary text and captions are insufficient.

W3 - The paper argues that claim status must be preserved, but I don't think there are any experiments that measures whether any condition communicates it correctly (the authors acknowledge that none of the 69 questions directly tests this).

W4 - The narrative prompt emphasizes visible content, so the it is expected that the content is more predictable than affect or persistence. Using frozen sentence encoders with logistic heads does not, by itself, establish a new representation-learning contribution. These encoders are also not used in the illustration pipeline. The connection between studies 1 and 3 is therefore mainly a design argument for the atlas, rather than an experimentally demonstrated link, according to my understanding. The paper does not show that the representation failures identified in study 1 lead to illustration errors, or that PRT reduces them.

Questions for authors

Q1. What is the single main empirical takeaway of the paper, and which experiment provides the strongest evidence for it?

Q2. Who is the intended reader of a PRT illustration, and what evidence shows that existing account-plus-image or caption-based presentations cause misunderstanding?

Q3. Since none of the evaluation questions directly tests claim status, what evidence supports the usefulness of PRT’s status tags for communicating the speaker’s meaning?

Reviewer_WJMDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Perceptual Report Translation for translating first-person perceptual experiences into visual illustrations while preserving not only visible content but also the speaker's qualifications, uncertainty, and context. It argues that an image can accurately depict the objects mentioned in a report while fundamentally misrepresenting how those objects were experienced. To deal with this, The paper develops an empirical framework using Ganzflicker experience reports. It first evaluates whether different text representations can predict independently recorded questionnaire responses. It also investigates whether sentence-embedding similarity actually preserves the meaning of qualifications such as negation, certainty, etc. It further evaluates image generation using selected reports and prompting conditions. The paper's broader contribution is an interactive perceptual-experience atlas, wihch argues that perceptual communication should preserve separate reporting channels and make every interpretation inspectable,

Critical strengths

The paper addresses a meaningful gap between visual generation and the subjective experiences. Its distinction between depicting what someone describes and preserving how they experienced it is refreshingly novel. Studies are also well-designed and exhaustive and the analyses accompanied are sensible. Study 2 is particularly interesting. The demonstration that cosine similarity can fail to distinguish opposite qualifications while supervised probes recover some of them is a valuable observation. It highlights a fundamental limitation of using embedding similarity as a proxy for semantic equivalence, especially when subtle distinctions such as negation, uncertainty, or agency matter.

Critical weaknesses

The relationship between the application and the actual dataset requires further justification. Also, The image-generation evaluation does not establish that PRT improves perceptual fidelity. The synthetic experiments are limited in semantic and linguistic diversity.

Questions for authors

Why not employ human evaluation as the primary measure of perceptual fidelity, as I think it will provide a significantly stronger claim?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.