Eliminate embedded bias in algorithms to ensure current bias is not exacerbated.

1. Justice and fairness in healthcare AI require equitable distribution of medical resources and unbiased decision-making. These principles encompass “distributive justice” (fair resource allocation) and “procedural justice” (fair decision-making) [25]. AI systems must avoid reinforcing biases that could disadvantage certain patient groups. Biases also intersect with Social Determinants of Health (SDOH), as algorithms trained on non-representative data can lead to unequal access, lower-quality care, and misdiagnosis in marginalized populations. Incorporating SDOH in data collection and algorithm development supports both distributive and procedural justice.
2. In a literature review of 45 sources, justice and fairness were the ethical issues of highest concern in 24 of the articles, arising in concert with themes such as bias, discrimination, and equality [26]. Strategies noted among the literature took both an algorithmic and data perspective, suggesting that developers purify algorithms of decision support tools, manage fairness constraints and distribution, guarantee responsible data collection, and encourage the cooperation of stakeholders in AI development.
   1. a. A widely used healthcare algorithm assessing overall health status assigned equal risk levels to Black and white patients, despite Black patients being significantly sicker. The algorithm used healthcare costs as a proxy for medical need, introducing implicit racial bias, as less is typically spent on Black patients. Adjusting for this disparity would increase care for Black patients from 17.7% to 46.5% [27].
3. Medical data typically reflects historical trends of discrimination through underrepresented minority groups in research and biased disease labels, thus mandating thoughtful design of AI to avoid exclusion. ChatGPT has been shown to exacerbate discriminatory biases by advising patients with identical symptoms, but distinct demographics differently [28]. Insured patients were advised to seek emergency care, but some uninsured patients were referred to community clinics.
4. Marginalized groups lack access and power to voice their concerns in AI decision-making or implementation. Unequal concentration of power and resource imbalance also raise concerns of justice, as only certain hospitals may have access to life-saving AI. Ethical solutions demand government support in the form of subsidization of AI for underfunded hospitals or regulatory frameworks designed to prioritize need, or open-source AI initiatives to increase access.

Trustworthiness: Establish representative data to train and test an AI model while ensuring transparency [29].

1. Trustworthy AI systems in healthcare are grounded in transparency, explainability, and interpretability, as these qualities collectively ensure that models are safe, reliable, and fair across diverse patient populations. For healthcare providers to trust AI-based insights, they must be confident that these insights are generated from “representative data” that covers relevant demographic and clinical variations.
   1. a. One study examined the influence of multiple factors on patient trust of AI systems and devices in healthcare, finding that younger respondents (18–30) were more concerned with AI ease of use, accountability, risk, and social influence (their perception of AI being affected by those around them), while elder respondents (60+) were concerned with the practicality of AI and less sensitive to issues of risk and data privacy. The development of trustworthy AI should thus consider the expectations of all patients and include avenues for feedback and criticism during the process [30].
2. Machine learning is trapped in a circular paradox: AI applications have a “large need for data to keep learning and improving, hence becoming safer, hence more trustworthy” [31]. For models to be considered trustworthy, they must be well-trained with a representative dataset. Rare health conditions lack extensive amounts of data, thereby making the software less trustworthy in its results and explanations. However, big data can be worrisome in terms of reliability and complexity, thus reducing the model’s trustworthiness and, by extension, transparency.

Transparency: Address the problem of explainability within AI models by providing the ability to verify results and in-program qualities.

1. Transparency is a key principle of AI ethics, especially in healthcare, where trust and accountability are critical. It encompasses multiple dimensions: “data transparency” (clarity on data sources and representativeness), “algorithmic transparency” (insights into model structure and assumptions), “process transparency” (disclosure of development steps, including human interventions), and “outcome transparency” (explanation of how results are generated) [32]. Related concepts include “explainability,” the ability to describe how an AI model reaches conclusions, and “interpretability,” the extent to which humans can understand cause-and-effect relationships within a model. Together, these aspects help mitigate AI’s “black-box” nature.
2. The “black-box” problem in AI limits interpretability, making it difficult for developers to predict or explain model decisions due to complex internal architectures. This challenge is especially critical in healthcare, where solutions must be understandable to both caregivers and patients. AI-generated explanations are often inaccurate or misleading, as they are typically post hoc. Developers, healthcare organizations, and clinicians must ensure AI tools are trustworthy, user-friendly, and human-centric. Key considerations include sample size limitations, inappropriate statistical significance, and self-serving biases like “data shopping” [29]. These guidelines help address concerns related to data and process transparency.
3. Patients must know and understand the process behind healthcare decisions made by AI, and require care givers to explain to patients the limitations and reason for such AI-driven decisions [33].

Acquire informed consent to use patient data and ensure anonymity.

1. Patient consent and confidentiality are fundamental ethical concerns in healthcare, especially as AI relies on large datasets. “Patient consent” upholds autonomy by allowing individuals control over their health data, while “confidentiality” prevents unauthorized access, fostering trust. AI presents unique challenges, as its need for diverse data can conflict with privacy rights. Without strong consent mechanisms and confidentiality protections, AI-driven healthcare risks violating privacy, undermining trust, and weakening ethical standards.
2. An inherent point of conflict exists in the decision of whether to prioritize comprehensive datasets for AI models, or guaranteeing consent of patients even if confidentiality is already ensured. Of course, data leaks are still possible thus preserving the importance of informed consent. In relation to issues of trustworthiness and transparency, increasing the rate of consent is an important area for improvement in the field.
3. A study on patient perspectives regarding informed consent in AI-driven diagnosis found that when physicians consult AI instead of human radiologists, patients place greater importance on AI use. This challenges the assumption that additional explanations are unnecessary in lower-risk or widely used settings or when AI outperforms physicians. Demographic factors such as gender, age, and income significantly influenced perceptions, highlighting the need for a personalized approach to disclosure [34].

Respect the privacy rights of users and third parties.

1. Patient trust and autonomy are affected by their right to privacy. Most “mobile disorder detection systems” risk data hacking as they use mobile devices to acquire signals, transfer, analyze, and forward the results to users in a stored database [26]. The same is true for online systems.

Remind patients of their ability to opt-out at any time and empower them to exercise autonomy in choice.

1. To respect patient autonomy, experts suggest clinicians discuss the topic in terms of trust, shared decision-making, and legal responsibilities of clinicians; a uniform understanding of issues involved in patient–clinician relationships; and ensure transparency regarding the use of AI to convey to patients that human-judgment takes priority over AI systems [26].
2. While consent may be acquired at the time of use, many AI models use this data in an ongoing fashion to continue updating. Thus, an additional challenge arises in considering whether patient consent should be frequently in discussion. Patients should have the right to opt-out at any time, but this may impact the model, which has already applied the data to learning algorithms.

Properly delegate and accept responsibility for transparent and ethical conduct.

1. Accountability is a key ethical concern in healthcare AI, defining responsibility for AI-driven decisions in patient care. Unlike traditional medicine, where clinicians are accountable, AI involves multiple stakeholders, including developers, providers, and institutions. Errors or unsafe recommendations complicate accountability, especially when AI systems are opaque or lack documentation. Without clear accountability, patient safety risks increase, and trust erodes. Robust frameworks are essential to ensure stakeholders prioritize ethical conduct and patient well-being.
2. Interactions between AI model developers, organizational leaders, and healthcare providers create risk in that they may not be inclined to take responsibility for errors. AI developers may fear monetary consequences over ethical considerations, while medical professionals may inadvertently place patients more at risk due to a subconscious feeling of immunity from the AI system. There is a clear misalignment of risk and return that requires ethical consideration.
3. Conflicting advice given by AI and medical experts will prove difficult for healthcare providers, as models often lack a measure of certainty. Further, the tendency of humans to default to decisions generated by machines as opposed to conflicting data (commission errors) or human decisions (omission errors, or the nonobservance of AI failures) is known as automation bias. This phenomenon involves blindly and earnestly accepting AI results, raising concerns of human culpability [35].
4. The role of organizations and healthcare institutions in terms of monitoring AI, assessing AI implementation, and oversight need to be included in this measure of accountability. Organizations promote certain regulations under which developers and providers are bound by. Another challenge relates to whether groups must disclose if they are using AI as part of shared decision-making.

Ensure AI complements the role of primary caregivers.

1. AI can provide real-time insights, streamline diagnostic processes, and offer data-driven recommendations that assist caregivers in critical, complex situations [29]. By enhancing diagnostic accuracy and efficiency, AI allows caregivers to focus on the personal and emotional aspects of patient care, where human empathy and judgment are irreplaceable.
2. To align with the goals of patient-centered care, AI tools must be adaptable, ensuring recommendations are consistent with individual patient needs and preferences. This requires transparency in how AI systems generate insights, allowing clinicians to interpret AI outputs in a way that respects each patient’s unique context and circumstances.
3. Researchers comparing algorithmic and human diagnostic and treatment decisions found that patients are “more resistant” to the former due to concerns that the model can’t account for patient individuality, are less likely to use healthcare, and perceive negative utility. It is recommended that AI is personalized and deployed alongside a clinician to reduce resistance and the feeling of “uniqueness neglect” [36]. AI outputs should be treated as supportive tools rather than definitive instructions, empowering caregivers to make informed, context-aware choices that best serve their patients.

AI should communicate with empathy and equity to patients.

1. AI systems must be developed to support equitable healthcare delivery, which requires careful attention to socioeconomic, gender, and ethnic factors that can affect patient care. Research has shown that disparities in treatment, such as black women receiving less care when reporting pain [37], highlight the potential for AI systems to inadvertently perpetuate biases. Ensuring that AI is trained on representative data can help minimize biased responses and improve fairness across patient populations.
2. AI systems must also be designed to communicate in a way that respects patients’ emotional and psychological needs. This requires embedding a sense of empathy within AI interactions to create responses that are human-centered and sensitive to patients’ vulnerabilities. The goal is for AI to offer support in a manner that feels respectful and considerate, rather than impersonal or dismissive, fostering patient comfort and trust.
3. Finally, AI-driven insights and recommendations should provide clear, understandable explanations that help patients feel informed and reassured. By delivering transparent and granular explanations of diagnoses or treatment plans, AI can empower patients to make informed healthcare decisions and feel more involved in their care. This human-centric approach reinforces empathy and equity, helping to mitigate biases and promote fairness in healthcare.

Sustainability, Human Centeredness, Inclusiveness, Fairness, Transparency (SHIFT).

A thematic analysis of recent literature [39] identified key subthemes in AI ethics across 253 articles, with their corresponding frequencies: responsible local leadership (14), social sustainability (22), embedding humanness in AI (20), the role of health professionals in public trust (32), interdisciplinary collaboration for artificial wisdom (6), inclusive AI governance (19), mitigating algorithmic and data bias (89), data representation and equality (22), health disparities in low-resource settings (22), privacy protection (54), explainability of AI models (56), legislative safeguards for confidentiality (16), user empowerment (6), and informed consent for data use (58). Fig 2 illustrates the distribution of these concerns in responsible AI implementation in healthcare.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Frequency of key subthemes in AI ethics literature.

Algorithmic bias, informed consent, explainability, and privacy emerge as the most prevalent concerns in responsible AI implementation in healthcare.

https://doi.org/10.1371/journal.pdig.0000810.g002

Standardized acronyms like “SHIFT” can help establish consensus on key AI challenges and protective initiatives for patients and communities. This work is crucial for educating stakeholders on AI applications in medicine. Siala and Wang highlight responsible initiatives, including linking algorithm outputs to human decision-making, implementing a centralized institutional review board, and integrating diverse patient data to enhance explainability [39].