acceptodds
Under review as a conference paper at ICLR 2027

Privacy of the Mind: How the Law Should Respond to Emotional Processing Through AI Chatbots

Abstract

People increasingly use AI chatbots not only to retrieve information or complete tasks, but to process emotions, reflect on personal experiences, and think through intimate problems. These interactions externalize cognitive and emotional processes that have traditionally occurred within the mind or through socially protected relationships such as diaries, friendships, and therapy. Yet existing approaches to AI privacy largely treat these interactions as exchanges of sensitive data, overlooking the role of the AI system in the cognitive-emotional process itself. We examine whether this phenomenon reflects a salient form of privacy in the age of AI that ought to be recognized and legally protected. We propose privacy of the mind as a distinct legal concept grounded in historical U.S. legal traditions protecting freedom of thought, cognitive liberty, and mental health integrity. We examine how AI-mediated conversation externalizes inner voice and outsources inner work, and identify design features that can invite further emotional disclosure, foster dependency, and create an appearance of confidentiality. These features make the interaction bidirectional: users expose increasingly intimate aspects of their inner lives to AI systems, while those systems simultaneously participate in shaping the cognitive and emotional processes through which users understand themselves. We then situate privacy of the mind within a century-long legal evolution protecting the boundaries of human thought—from First Amendment protections of cognition, to Fifth Amendment protections against self-incrimination, and to protections arising in mental health law. We argue that privacy of the mind in the AI era cannot be secured through data governance alone; it requires privacy to shape interaction design, system behavior, and the legal frameworks governing AI-mediated emotional processing.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.