Reviewers: Two anonymous reviewers

Editorial Assistant: Elena Benini

Why do we treat Alexa or ChatGPT like a helpful coworker – or even a friend? This article explores how talking to conversational AI shapes the beliefs we form about machines.

Have you ever found yourself saying “thank you” to Alexa, asking ChatGPT “How are you today?”, or getting annoyed when your virtual assistant just does not “get” you? If so, you’re not alone – and you might be revealing more about yourself than you think. Recent advances in artificial intelligence (AI) have brought about a quiet revolution: We no longer just click and type our way through machines, we talk to them using natural language. Whether it’s voice assistants like Alexa and Siri or primarily text-based systems like ChatGPT, Gemini, and Claude, people are increasingly interacting with AI in ways that resemble interactions with other people.

From Theory of Mind to Theory of Machine

Psychologists have long studied how we think about other people’s minds. We use our so-called Theory of Mind to infer others’ thoughts, intentions, and feelings, often by talking with them and observing how they behave [1]. Conversational AI now invites a similar mode of interaction: We ask questions, receive answers, and adjust our expectations. Researchers have argued that people also form a kind of Theory of Machine: beliefs about how intelligent systems work, what information they use, what they can and cannot do, and how they might behave in the future [2]. What’s striking in the case of large language models (LLMs) – the AI systems behind tools like ChatGPT – is how quickly and effortlessly such beliefs can form. The same cognitive channel we use for conversations with humans becomes available for interactions with natural-language chatbots. A few natural-sounding responses delivered through a chat window-like interface can boost the AI’s impact on people [3], and conversational systems can be especially context-sensitive, socially engaging, and persuasive [4]. In other words, conversational interfaces are more than a usability feature – their use reveals how humans relate to intelligent systems.

But there’s a catch. Our social instincts can blur the line between artificial and human intelligence. We might assume that AI “understands” us in a human-like way when in fact, human and artificial intelligence operate on fundamentally different principles. Research shows that while people are good at things like identifying causal relationships, reasoning morally, or adapting flexibly to new environments, algorithms often struggle with these tasks [5], [6]. On the flip side, AI can detect subtle patterns in massive datasets or juggle thousands of variables at once – things humans usually struggle with [7]. These differences make the human tendency to reason socially about AI both fascinating and fraught.

Why Theory of Machine matters

Understanding how people relate to conversational AI is not just an academic exercise. It has real-world consequences for trust, decision-making, education, and more. For example, if people overestimate a chatbot’s understanding, they may rely on it too heavily in sensitive domains like mental health or legal advice. On the flip side, overly skeptical users might underuse instrumental AI tools, for instance, by ignoring helpful feedback from AI-based language learning apps or by refusing to trust AI-assisted medical triage systems that could speed up care in crowded emergency rooms. Designers of AI systems therefore face a tricky balancing act: making tools that feel intuitive and helpful without creating false impressions of empathy, authority, or morality.
Much of the behavioral research thus far has focused on whether people trust algorithms more or less than they trust other humans [8], [9]. The so-called Theory of Machine 2.0 treats these comparisons between human-human interactions and human-AI interactions as useful but incomplete [10]. In everyday life, users also compare one AI system with another. Just as different people bring different minds to a conversation, different AIs embody different capabilities: some are pattern recognizers, others text generators; some offer explanations or ask follow-up questions, others can search for relevant external information themselves. Essentially, users’ beliefs about different AI systems might reflect these peculiarities, ultimately affecting their behavior toward specific systems. This current disconnect between research and the real world is what makes studying Theory of Machine so important.

Conversational user interfaces can reveal important insights about people’s Theory of Machine

To explore people’s Theory of Machine, we investigated how they interact with LLMs in conversational contexts [3]. In this study, participants solved numerical estimation tasks with access to ChatGPT’s pre-generated advice. They either saw only a numerical recommendation, received an additional verbal explanation by default, or could actively request one. Interactive explanations are important because they provide natural-language cues about the AI’s (alleged) rationale, offer additional context for evaluating advice, and may help users adjust their expectations about future interactions. In our study, both explanations and the opportunity to interact increased how much people weighted the chatbot’s advice. This finding shows that conversational features can influence advice uptake. Notably, past work on so-called algorithm aversion shows that people avoid AI advice after seeing it err [11]. Likewise, we observed that participants gave less weight to chatbot advice when the accompanying explanations provided context indicating that the advice might be of lower quality. This suggests that people are sensitive to cues in the interaction and adjust their reliance accordingly – precisely the kinds of experiences from which beliefs about AI may develop [10]. These beliefs may not be technically accurate (e.g., most people do not know how LLMs are trained), but they can still shape behavior in important ways.

Human-AI interaction is not just shaped by what the AI can do, but also by what users believe it can do. In our study, users weighted advice from the AI system more than their own initial estimate [3]. Compared with classic advice-taking studies, where people often favor their own judgments [12], this shows that AI advice can receive substantial weight. In the case of conversational AI, the dynamics seem to hinge on how a system is framed (e.g., as personalized, competent, or transparent), how users can interact with it, and which AI system they think they are using – not simply whether the source is human or machine [2], [8], [9], [11]. In summary, our findings suggest that conversational interfaces make AI more usable by tapping into familiar routines for asking questions and evaluating answers. In this sense, conversation is not merely a technical feature, but a psychological affordance – it makes AI feel like something we can question, compare, and reason with.

The limits of Theory of Machine

As helpful as it is to understand AI through the lens of social reasoning, this approach also has its limits. After all, algorithms do not just mimic human behavior but process information in ways that are fundamentally different from us. Older rule-based systems follow strict guidelines set by their human designers [13]. In that sense, they reflect a kind of human logic. In contrast, today’s data-driven systems, such as ChatGPT, learn patterns from massive amounts of data rather than being told exactly what to do. Their “thinking” is shaped by data, not deliberate rules. This means that while humans often generalize and reason abstractly, AI tends to rely on very specific patterns and correlations it has observed before. As a result, conversational AI may sound reflective but does not automatically monitor where information came from, whether it was repeated many times, whether a prompt hides important alternatives, or whether a general pattern applies to a specific case [14]. Future social AI would thus need not only to react to social information but also to monitor its own limits and adjust its behavior accordingly [15].
Our relationships with AI are shaped not just by how we see machines, but also by how machines “see” us. Many systems build internal models of users based on past behavior – personalizing recommendations or responses to match our preferences. While we may not fully understand how these systems work, we often feel that some AI tools “know” us better than others [16]. Importantly, this feeling does not require technical knowledge either. It can stem from nothing more than repeated experiences with a system that sometimes gets it right and sometimes gets it wrong [17]. When AI behaves in ways that reflect our own past choices, it becomes not just human-like, but eerily self-like. In those moments, drawing on social reasoning can be understandable, even though the resulting Theory of Machine remains an approximation. 
We should not forget that humans and AI remain different kinds of “thinkers.” AI judgment often involves so many variables and hidden layers that even the engineers who built it often cannot explain why it made a certain decision. In that sense, treating AI like a person is not just a shortcut; it’s a coping strategy. Faced with complex, opaque systems, we reach for the most familiar frame of reference we have: humans. And much of the time, this coping strategy may work surprisingly well. But it can also lead us to miss differences between one AI system and another.

Final takeaways

Talking to machines may seem like science fiction turned mundane. But beneath the surface, it reveals a profound psychological reality: When machines speak our language, we respond in kind. We form expectations, compare systems, and rely on familiar routines – and in doing so, reveal the deeply social architecture of our own minds. This opens the door to new psychological asymmetries: Users may sometimes judge machines by human benchmarks while overlooking the algorithmic peculiarities that distinguish one AI system from another. If we want to truly understand the psychology of human-AI interaction, we need to move beyond simplistic comparisons between humans and AI. Only then can we appreciate artificial cognition – its limits, strengths, and the ways in which it reshapes our social reasoning.

Bibliography

[1]    F. Cuzzolin, A. Morelli, B. Cîrstea, and B. J. Sahakian, “Knowing me, knowing you: Theory of mind in AI,” Psychological Medicine, vol. 50, no. 7, pp. 1057–1061, 2020, doi: 10.1017/S0033291720000835.
[2]    J. M. Logg, “The psychology of big data: Developing a ‘Theory of Machine’ to examine perceptions of algorithms,” in The psychology of technology: Social science research in the age of big data, S. C. Matz, Ed., Washington: American Psychological Association, 2022, pp. 349–378. doi: 10.1037/0000290-011.
[3]    T. R. Rebholz, A. Koop, and M. Hütter, “Conversational user interfaces: Explanations and interactivity positively influence advice taking from generative artificial intelligence,” Technology, Mind, and Behavior, vol. 5, no. 4, 2024, doi: 10.1037/tmb0000136.
[4]    F. Salvi, M. Horta Ribeiro, R. Gallotti, and R. West, “On the conversational persuasiveness of GPT-4,” Nat Hum Behav, vol. 9, no. 8, pp. 1645–1653, May 2025, doi: 10.1038/s41562-025-02194-6.
[5]    M. B. Fazi, “Can a machine think (anything new)? Automation beyond simulation,” AI & Society, vol. 34, no. 4, pp. 813–824, 2019, doi: 10.1007/s00146-018-0821-0.
[6]    M. B. Fazi, “Beyond human: Deep learning, explainability and representation,” Theory, Culture, & Society, vol. 38, no. 7–8, pp. 55–77, 2021, doi: 10.1177/0263276420966386.
[7]    I. Rahwan et al., “Machine behaviour,” Nature, vol. 568, no. 7753, pp. 477–486, 2019, doi: 10.1038/s41586-019-1138-y.
[8]    J. W. Burton, M. Stein, and T. B. Jensen, “A systematic review of algorithm aversion in augmented decision making,” Journal of Behavioral Decision Making, vol. 33, no. 2, pp. 220–239, 2020, doi: 10.1002/bdm.2155.
[9]    H. Mahmud, A. K. M. N. Islam, S. I. Ahmed, and K. Smolander, “What influences algorithmic decision-making? A systematic literature review on algorithm aversion,” Technological Forecasting and Social Change, vol. 175, p. 121390, 2022, doi: 10.1016/j.techfore.2021.121390.
[10]    T. R. Rebholz, “Lay beliefs about artificial versus artificial intelligence: Rethinking theory of machine,” Collabra: Psychology, vol. 12, no. 1, 2026, doi: 10.1525/collabra.155671.
[11]    B. J. Dietvorst, J. P. Simmons, and C. Massey, “Algorithm aversion: People erroneously avoid algorithms after seeing them err,” Journal of Experimental Psychology: General, vol. 144, no. 1, pp. 114–126, 2015, doi: 10.1037/xge0000033.
[12]    I. Yaniv and E. Kleinberger, “Advice taking in decision making: Egocentric discounting and reputation formation,” Organizational Behavior and Human Decision Processes, vol. 83, no. 2, pp. 260–281, 2000, doi: 10.1006/obhd.2000.2909.
[13]    A. Barredo-Arrieta et al., “Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,” Information Fusion, vol. 58, pp. 82–115, 2020, doi: 10.1016/j.inffus.2019.12.012.
[14]    F. Scholten, T. R. Rebholz, and M. Hütter, “Metacognitive myopia in large language models,” 2026, arXiv. doi: 10.48550/arXiv.2408.05568.
[15]    L. Cominelli, D. Mazzei, and D. E. Rossi, “SEAI: Social emotional artificial intelligence based on Damasio’s theory of mind,” Frontiers in Robotics and AI, vol. 5, p. 6, 2018, doi: 10.3389/frobt.2018.00006.
[16]    T. Araujo, N. Helberger, S. Kruikemeier, and C. H. Vreese, “In AI we trust? Perceptions about automated decision-making by artificial intelligence,” AI & Society, vol. 35, no. 3, pp. 611–623, 2020, doi: 10.1007/s00146-019-00931-w.
[17]    F. Dudhwala and L. Björklund Larsen, “Recalibration in counting and accounting practices: Dealing with algorithmic output in public and private,” Big Data & Society, vol. 6, no. 2, pp. 1–12, 2019, doi: 10.1177/2053951719858751.

Image sources

Image 1–4: Created by the author.