Reviewers: Two anonymous reviewers

Editorial Assistant: Maren Giersiepen

Artificial Intelligence (AI) is revolutionizing healthcare, from enhancing radiology diagnoses to supporting mental health treatment. But while it promises efficiency and accuracy, its integration raises questions of trust, bias, and decision-making dynamics. Will AI become an indispensable medical partner, or will barriers hinder its adoption? This article explores AI’s expanding role, its impact on healthcare professionals, and the road ahead.

Artificial intelligence (AI) has become an integral part of modern life, often operating in the background without us even noticing. From virtual assistants like Siri and Alexa, which use AI to understand and respond to our commands, to more complex systems, AI powers a wide range of technologies we interact with on a daily basis. In a broader sense, AI refers to computational systems designed to perform tasks that usually require human intelligence, such as pattern recognition, decision-making, and problem-solving [1]. While AI's use is more and more part of everyday life, its impact is particularly significant in healthcare, where it's increasingly used for diagnostics, treatment recommendations, and administrative tasks, offering the potential to improve both efficiency and accuracy [2], [3]. However, integrating AI into healthcare is not without challenges, particularly in balancing trust, acceptance, and skepticism. This article delves into how AI influences decision-making in healthcare, with a focus on radiology and mental healthcare.

The expanding role of AI in healthcare

AI is rapidly transforming healthcare, offering a potentially powerful way to manage the huge flood of medical data that hospitals generate every second. From patient vitals and lab results to insurance details and treatment records, the healthcare system produces more information than any human could reasonably keep up with – at least, not without an unlimited supply of coffee. And this is where AI algorithms come in. Designed to process vast amounts of data with speed and precision, these AI systems have the potential to help practitioners detect illness patterns, refine diagnoses, and predict patient outcomes more accurately [3]. This includes a supercharged assistant that can sift through enormous datasets in seconds, spotting patterns and reducing human error. 

Radiology was one of the first areas where AI applications were explored, with early discussions about the possibility of AI eventually replacing radiologists altogether [4], [5]. Spoiler alert: That hasn’t happened (yet?). Radiology is particularly well-suited for AI because it relies heavily on imaging techniques, providing clear and structured data that algorithms can easily analyze [6]. This makes it a prime area for AI integration, with around 230 AI-based tools already available on the European market (AIforRadiology.com). For example, one of these tools is designed to assist radiologists in interpreting mammograms by detecting potentially suspicious areas indicative of breast cancer, such as masses, distortions, and asymmetries [7]. So, under ideal conditions, AI systems can match or even surpass human performance in detecting diseases from X-rays, CTs, and MRIs [8], [9], [10].

Mental healthcare, in contrast, is still in the early stages of AI adoption. Unlike radiology, which is built around standardized imaging techniques, mental healthcare involves more subjectivity. Diagnosis relies considerably on patients’ self-observation or practitioner’s observations, and appropriate treatment options depend on a range of different therapeutic approaches. This makes it trickier to develop algorithms for this field and to integrate them in a meaningful way, which is one reason why practitioners’ experience with AI in this field is currently rather low [11]. However, AI is beginning to find its place, with application areas ranging from diagnostic tools and treatment support systems to feedback mechanisms for therapists and administrative management systems that reduce the administrative burden on mental health professionals [11]. As of 2024, only seven AI applications in the mental healthcare domain have been cleared by regulatory authorities, indicating they have met safety and effectiveness standards [12]. Despite this small number, the surge in patents in this field suggests a growing trend in AI-driven mental healthcare technologies [13]. For instance, one cleared AI-enabled tool recommends personalized antidepressant treatments based on a patient’s clinical history and genetic data. It generates a report predicting treatment response, including metabolism, potential side effects, and optimal dosage [12].

The slower adoption of AI in mental healthcare compared to radiology or other medical specialties isn’t solely due to the complexity of the field but also about trust and acceptance. While it might be helpful to let AI spot a lung tumor on a scan, the idea of relying on it for something as personal and sensitive as one’s fears, emotions, or psychological state might feel unsettling. After all, mental health care involves deep human connections, and it’s not always easy to imagine an algorithm being a part of that process.

Factors influencing the acceptance of AI

Trust is one key factor for people to accept and adopt AI technology [14]. In psychology, it is often defined as the willingness to rely on another party and accept some level of risk, based on the expectation of reliable behavior [15]. In the context of AI, this translates to a user’s belief that a system will act correctly and reliably, prompting acceptance of uncertainty and potential risk even when its processes are not fully observable. Alongside trust, the functionality of AI tools themselves is crucial in determining how widely they’re adopted.

Technology acceptance models like the Unified Theory of Acceptance and Use of Technology (UTAUT) are often used in research and offer valuable insights into the factors that drive the adoption of new (AI) technologies [16]. It emphasizes the importance of four main factors: (1) performance expectancy refers to the belief that technology will improve productivity. A psychotherapist might be more likely to adopt an AI-enabled documentation tool if they believe it will save time by automatically recording session details. (2) Similarly, effort expectancy relates to how easy a technology is to use – when it’s user-friendly, it’s much more likely to gain traction. A psychotherapist would probably value a simple, intuitive system that reduces their workload without requiring complex training. (3) Social influence also plays a significant role: the opinions of colleagues can be a deciding factor. If a psychotherapist’s peers rave about how helpful AI is in managing patient data, the psychotherapist might be more inclined to give it a try. (4) Facilitating conditions, like training and support, ensure that users feel confident in using AI effectively. A therapist might recognize the availability of training and technical support as important, as these resources build confidence in successfully using the technology and receiving help when needed [16], [17].

However, it’s not only functional factors that matter. How well an AI aligns with the specific clinical setting, workflows, professional roles, and decision environments in which it is embedded also plays a crucial role. User characteristics are important as well: Research indicates that, among mental health practitioners, individual factors such as learning intention, ethical knowledge, and affinity for technology influence their willingness to use AI [11]. From the patient's perspective, there seems to be an overall preference for receiving health advice from human doctors rather than AI for their own health [18], [19]. Interestingly, this preference disappears when evaluating healthcare decisions for others, suggesting that psychological distance may play a role – people tend to feel more detached when assessing another person’s situation [19]. Other key factors affecting the choice between AI-enabled technologies and human doctors include cost, expertise, and privacy considerations [18].

Having explored the general interaction with and acceptance of AI, let’s now shift our focus to how AI specifically impacts decision-making on a deeper level, where its influence on the way choices are made and the underlying dynamics become even more critical.

AI and (medical) decision-making

As already mentioned, the development of AI technologies is accelerating at a rapid pace, but the acceptance and use of these technologies have been riddled with uncertainty – like a high-speed train that practitioners are still deciding whether to board.

Some studies show that people rely more on recommendations coming from an AI compared to a human, a phenomenon known as algorithm appreciation [20]. In these cases, AI might be seen as a highly efficient assistant, much like a supercomputer, compared to a tired doctor who’s been on a 12-hour shift.

On the flip side, other studies show the opposite effect: algorithm aversion – meaning that people are hesitant to trust AI recommendations, especially when compared to human ones. This typically happens when individuals see AI make a mistake and lose confidence in it entirely – even if, on average, AI outperforms humans [21]. It’s a bit like watching a GPS make one wrong turn and then deciding you’d rather trust your friend’s questionable sense of direction for the rest of the journey. Psychological factors play a significant role in this: Humans tend to favor intuition and are generally more forgiving of mistakes made by other people than by machines [22]. For example, if a doctor misdiagnoses a patient, we might think, “They did their best, they’re only human.” However, if an AI system makes a similar mistake, people might be quick to judge that the technology is not trustworthy.

Overall, AI is beginning to affect clinical decision-making, though its impact is inconsistent and context-dependent. And while it’s mostly a good thing – for instance in radiology, where it can enhance diagnostic accuracy, especially for practitioners who aren’t imaging experts [23], let’s not get ahead of ourselves – there are a few bumps on the road.

Challenges in decision-making with AI

And here comes the plot twist: AI isn’t perfect, and that imperfection affects not only medical decision-making but also the broader integration of AI in healthcare.

Automation bias: When AI overshadows human judgement

Relying too much on AI systems can lead to something called automation bias, where practitioners follow the AI’s recommendations without really questioning them and taking other relevant information into account, even when the AI may be clearly off-track [24]. It’s like trusting your GPS to lead you down a shortcut, only to end up stuck in traffic – because even the smartest system can miss the bigger picture sometimes. This is particularly a risk in high-pressure situations where there isn’t enough time, information, or cognitive capacity to make a fully informed decision [25]. In these moments, we often lean on simple decision strategies or heuristics, which can be effective but also lead to biased or flawed judgments [26]. However, the opposite of automation bias (excessive distrust) can also occur, especially among domain experts, who may overly distrust AI due to overconfidence, reluctance to take advice, or the high stakes of clinical decisions [29]. Rather than increasing or decreasing trust in AI, a key challenge is achieving calibrated trust, meaning that people trust an AI in line with what it can and cannot actually do. This calibration is difficult to achieve because cognitive biases, systematic patterns of error in judgment that occur when processing complex information, can occur [26].

Bias in AI: When data becomes a double-edged sword

As mentioned before, AI technologies can process vast amounts of data at rapid speed and consequently improve medical outcomes. However, there is a twist to it. AI algorithms rely on the data they are trained with, and if that data is biased – due to factors like underrepresentation of certain demographics or societal inequalities – the AI may perpetuate these biases, potentially leading to skewed or unfair outcomes. For example, an algorithm used in healthcare disproportionately recommended fewer resources for Black patients, due to biases in the training data [27]. Hence, addressing these biases requires careful attention to data diversity and fairness to ensure AI systems support equitable healthcare for all.

The COVID-19 pandemic exposed another weakness: An AI algorithm designed to assist in predicting patient mortality risks experienced performance degradation during the pandemic because the sudden shift in data affected the model’s performance [28]. Close model surveillance and carefully re-training the model with updated data is needed to overcome such challenges.

Future outlook and conclusion

As AI is set to play a big role in healthcare, with potential in diagnostics, treatment planning, and personalized medicine, unlocking its full capabilities will require overcoming skepticism, tackling ethical challenges head-on, and building trust in the right places.  The last one is often described as calibrated trust, one key principle for the future of AI in medicine. It means trusting AI when it regularly delivers good results but not turning off our own judgment and following it blindly. And just as mentioned earlier, a helpful comparison for understanding trust is checking your GPS: most of the time it guides you the right way, but you still look out the window to make sure it matches what you see. Calibrated trust helps avoid both extremes – expecting too much from AI or dismissing it too quickly. And because AI is currently more like a promising new intern, it also requires constant adjustments and, as a second key principle, human oversight. Even the most impressive AI still needs a trained professional keeping an eye on it. Just like an intern who can be incredibly helpful but still needs guidance, AI benefits from professionals who notice when something seems off and can step in when necessary. In other words, AI can support tasks, but humans (still) stay in the driver’s seat.  So, the future of AI in healthcare isn’t just about technological breakthroughs; it’s also about building well-calibrated trust, maintaining human oversight, and keeping patient care at the heart of everything we do. Striking the right balance between AI-driven efficiency and the invaluable expertise of healthcare professionals will be essential in shaping a healthcare system that truly benefits everyone – patients and practitioners alike.

Bibliography

[1]    E. Glikson and A. W. Woolley, “Human Trust in Artificial Intelligence: Review of Empirical Research,” Acad. Manag. Ann., vol. 14, no. 2, pp. 627–660, Jul. 2020, Juli 2020, doi: 10.5465/annals.2018.0057.
[2]    E. Lermer et al., „Patient:innen und KI: Eine Frage der Perspektive bei der Bewertung von KI bei medizinischen Online-Diensten [Patients and AI: A Matter of Perspective in the Evaluation of AI in Medical Online Services]“, in Künstliche Intelligenz im Healthcare-Sektor [Artificial Intelligence in the Healthcare Sector], Frankfurt University of Applied Sciences, 2024, pp. 74–76. Accessed: Mar. 31, 2025. [Online]. Available: https://doi.org/10.48718/1cw9-3c06
[3]    E. J. Topol, “High-performance medicine: the convergence of human and artificial intelligence,” Nat. Med., vol. 25, no. 1, pp. 44–56, Jan. 2019, doi: 10.1038/s41591-018-0300-7.
[4]    D. Pinto dos Santos et al., “Medical students’ attitude towards artificial intelligence: a multicentre survey,” Eur. Radiol., vol. 29, no. 4, pp. 1640–1646, Apr. 2019, doi: 10.1007/s00330-018-5601-1.
[5]    L. Yang, I. C. Ene, R. Arabi Belaghi, D. Koff, N. Stein, and P. (Lina) Santaguida, “Stakeholders’ perspectives on the future of artificial intelligence in radiology: a scoping review,” Eur. Radiol., vol. 32, no. 3, pp. 1477–1495, Mar. 2022, doi: 10.1007/s00330-021-08214-z.
[6]    A. Hosny, C. Parmar, J. Quackenbush, L. H. Schwartz, and H. J. W. L. Aerts, “Artificial intelligence in radiology,” Nat. Rev. Cancer, vol. 18, no. 8, pp. 500–510, Aug. 2018, doi: 10.1038/s41568-018-0016-5.
[7]    J. Gjesvik, N. Moshina, C. I. Lee, D. L. Miglioretti, and S. Hofvind, “Artificial Intelligence Algorithm for Subclinical Breast Cancer Detection,” JAMA Netw. Open, vol. 7, no. 10, p. e2437402, Oct. 2024, doi: 10.1001/jamanetworkopen.2024.37402.
[8]    N. Agarwal, A. Moehring, P. Rajpurkar, and T. Salz, “Combining human expertise with artificial intelligence: Experimental evidence from radiology,” Natl. Bur. Econ. Res., Working Paper no. 31422, Jul. 2023, revised Mar. 2024, doi: 10.3386/w31422. 
[9]    P. Lakhani and B. Sundaram, “Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks,” Radiology, vol. 284, no. 2, pp. 574–582, Aug. 2017, doi: 10.1148/radiol.2017162326.
[10]    K. Lång et al., “Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomized, controlled, non-inferiority, single-blinded, screening accuracy study,” Lancet Oncol., vol. 24, no. 8, pp. 936–944, Aug. 2023, doi: 10.1016/S1470-2045(23)00298-X.
[11]    J. Cecil, A.-K. Kleine, E. Lermer, and S. Gaube, “Mental health practitioners’ perceptions and adoption intentions of AI-enabled technologies: an international mixed-methods study,” BMC Health Serv. Res., vol. 25, art. no. 556, 2025, doi: 10.1186/s12913-025-12715-8.
[12]    A.-K. Kleine, E. Kokje, P. Hummelsberger, E. Lermer, I. Schaffernak, and S. Gaube, “AI-enabled clinical decision support tools for mental healthcare: A product review,” Artif. Intell. Med., vol. 160, p. 103052, Feb. 2025, doi: 10.1016/j.artmed.2024.103052.
[13]    A.-K. Kleine, E. Lermer, J. Cecil, A. Heinrich, and S. Gaube, “Advancing mental health care with AI-enabled precision psychiatry tools: A patent review,” Comput. Hum. Behav. Rep., vol. 12, p. 100322, Dec. 2023, doi: 10.1016/j.chbr.2023.100322.
[14]    S. Afroogh, A. Akbari, E. Malone, M. Kargar, and H. Alambeigi, “Trust in AI: progress, challenges, and future directions,” Humanit. Soc. Sci. Commun., vol. 11, no. 1, pp. 1–30, Nov. 2024, doi: 10.1057/s41599-024-04044-8.
[15]    Mayer, R. C., Davis, J. H., & David Schoorman, F. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709-734, doi: 10.2307/258792.
[16]    V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User Acceptance of Information Technology: Toward a Unified View,” MIS Q., vol. 27, no. 3, p. 425, 2003, doi: 10.2307/30036540.
[17]    J. Cecil, E. Kokje, S. Gaube, E. Lermer, and A.-K. Kleine, „Einflussfaktoren auf die Nutzung KI-gestützter Technologien in der psychischen Gesundheitsversorgung [Factors influencing the use of AI-based technologies in mental health care]“, in Künstliche Intelligenz im Healthcare-Sektor [Artificial Intelligence in the Healthcare Sector], 2024, pp. 83–88. Accessed: Mar. 31, 2025. [Online]. Available 
[18]    S. Gaube, I. Biebl, M. K. M. Engelmann, A.-K. Kleine, and E. Lermer, “Comparing preferences for skin cancer screening: AI-enabled app vs dermatologist,” Soc. Sci. Med., vol. 349, p. 116871, May 2024, doi: 10.1016/j.socscimed.2024.116871.
[19]    M. F. C. Hudecek, E. Lermer, S. Gaube, J. Cecil, S. F. Heiss, and F. Batz, “Fine for others but not for me: The role of perspective in patients’ perception of artificial intelligence in online medical platforms,” Comput. Hum. Behav. Artif. Hum., vol. 2, no. 1, 100046, Jan. 2024, doi: 10.1016/j.chbah.2024.100046.
[20]    J. M. Logg, J. A. Minson, and D. A. Moore, “Algorithm appreciation: People prefer algorithmic to human judgment,” Organ. Behav. Hum. Decis. Process., vol. 151, pp. 90-103, Mar. 2019,, doi: 10.1016/j.obhdp.2018.12.005.
[21]    B. J. Dietvorst, J. P. Simmons, and C. Massey, “Algorithm aversion: People erroneously avoid algorithms after seeing them err.,” J. Exp. Psychol. Gen., vol. 144, no. 1, pp. 114–126, 2015, doi: 10.1037/xge0000033.
[22]    J. B. Lyons, K. Hobbs, S. Rogers, and S. H. Clouse, “Responsible (use of) AI,” Front. Neuroergonomics, vol. 4, p. 1201777, Nov. 2023, doi: 10.3389/fnrgo.2023.1201777.
[23]    S. Gaube et al., “Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays,” Sci. Rep., vol. 13, no. 1, Art. no. 1, Jan. 2023, doi: 10.1038/s41598-023-28633-w.
[24]    D. Lyell and E. Coiera, “Automation bias and verification complexity: a systematic review,” J. Am. Med. Inform. Assoc., vol. 24, no. 2, pp. 423–431, Mar. 2017, doi: 10.1093/jamia/ocw105.
[25]    J. Cecil, E. Lermer, M. F. C. Hudecek, J. Sauer, and S. Gaube, “Explainability does not mitigate the negative impact of incorrect AI advice in a personnel selection task,” Sci. Rep., vol. 14, no. 1, p. 9736, Apr. 2024, doi: 10.1038/s41598-024-60220-5.
[26]    D. Kahneman, Thinking, Fast and Slow, New York, NY, USA: Farrar, Straus and Giroux, 2011.
[27]    Z. Obermeyer, B. Powers, C. Vogeli, and S. Mullainathan, “Dissecting racial bias in an algorithm used to manage the health of populations,” Science, vol. 366, no. 6464, pp. 447-453, Oct. 2019, doi: 10.1126/science.aax2342.
[28]    D. I. Andonov et al., “Impact of the COVID-19 pandemic on the performance of machine learning algorithms for predicting perioperative mortality,” BMC Med. Inform. Decis. Mak., vol. 23, no. 1, p. 67, Apr. 2023, doi: 10.1186/s12911-023-02151-1.
[29]    V. Sivaraman, L. A. Bukowski, J. Levin, J. M. Kahn, and A. Perer, “Ignore, trust, or negotiate: Understanding clinician acceptance of AI-based treatment recommendations in health care,” in Proc. 2023 CHI Conf. Human Factors Comput. Syst. (CHI ’23), Hamburg, Germany, Apr. 2023, pp. 1–18, doi: 10.1145/3544548.3581075.

Image sources

Image 1: https://de.freepik.com/fotos-premium/ki-in-der-medizinischen-behandlung…
Image 2: https://de.freepik.com/fotos-premium/medizinischer-ct-oder-mri-oder-pet…
Image 3: https://de.freepik.com/fotos-premium/mittelabschnitt-eines-mannes-der-e…