Deep leverage points for ethical and responsible AI in long-term care

With the rapid development of AI technologies, enthusiasm for AI applications in long-term care (LTC) is evident across policy, research, and commercial sectors [1]. AI is often portrayed as a promising approach to address the growing burden of noncommunicable diseases, diverse LTC needs, and care workforce shortages [2]. Various AI applications have been used to analyze behavioral patterns, predict risks, support care planning, and personalize services [3]. Yet, AI does not simply respond to existing challenges in LTC [4]. By determining which data are collected, which risks are classified, and which outcomes are prioritized, AI may also influence how care needs are understood and addressed. Ethical risks therefore arise not only from technical performance, but also from the possibility that structural and relational problems in LTC are reframed as individual conditions that can be monitored or managed through technology [4].

These risks arise at several interconnected levels. First, at the individual and relational level, AI may affect autonomy, dignity, privacy, consent, and relationships between older adults and caregivers. AI-enabled systems may collect and process sensitive data without sufficient transparency about how the data are used, protected, and retained [5]. Excessive reliance on AI may also weaken the human connection and reciprocity that are central to care [6]. Next, at the level of social justice, biased data, opaque AI systems, and weak accountability may deepen existing inequalities, particularly for older adults living with frailty or socioeconomic disadvantage, or those with limited AI literacy [4,7]. Moreover, at the cultural and epistemic level, categories such as “dependency,” “risk,” “efficiency,” and “quality of care” are not neutral descriptors. They reflect assumptions shaped by medical, administrative, commercial, and policy institutions, and these assumptions may become embedded in datasets, classification systems, platform design, and performance indicators [8]. Decisions about what data to collect, which risks to classify, and which outcomes to optimize therefore embed particular assumptions about normal aging, desirable behavior, appropriate care, and where responsibility for care should lie. These assumptions may reinforce ageist and deficit-oriented representations while marginalizing diverse experiences of aging and social relations [7]. Furthermore, at the environmental level, scaling data-intensive AI systems also carries material costs through the use of energy, water, minerals, and data infrastructures [9].

Overall, these risks are not isolated. Organizational pressures shape what AI systems are designed to measure. Technical classifications influence how staff interpret residents’ needs. These interpretations can reshape care practices, reproduce inequalities, and increase demand for further AI-enabled interventions. In this process, older adults may be reduced to data subjects or measurable profiles, while their identities, relationships, life histories, and social belonging receive less attention [4].

Consider a hypothetical example in which an AI-powered chatbot is introduced to reduce loneliness among LTC residents. In an underfunded facility, workforce pressures may encourage staff to use the chatbot to monitor residents’ emotional states, identify signs of social isolation, and initiate conversations. The facility may then rely on the technology as a substitute for further investment in staffing, communal activities, and sustained human relationships. The system may also rely on datasets that reflect narrow assumptions about “normal” social interaction and may misinterpret culturally diverse forms of communication and social connection. Its collection and processing of personal data also create risks related to privacy and data governance, while the infrastructure required to operate such systems may carry environmental costs. Loneliness may therefore be reframed as an individual condition to be detected and technologically managed, while the organizational conditions contributing to it remain insufficiently addressed.

To date, several governments and regional institutions have established AI governance frameworks that provide important safeguards, including the EU AI Act’s risk-based approach and China’s expanding framework for AI industry and technical standardization [10,11]. Yet, compliance-oriented governance often focuses on the capabilities and risks of individual AI systems, framing responsibility mainly in terms of technical compliance rather than institutional responsibility for staffing, social infrastructure, and equitable care. It may therefore leave unexamined why particular AI systems are adopted, which problems they are expected to solve, and how organizational arrangements, power relations, and sociocultural assumptions shape their use [12].

A systems thinking perspective can complement existing regulatory approaches by shifting attention from isolated technological risks to the structures, goals, feedback processes, and underlying assumptions that continually reproduce these risks [13]. Systems thinking understands complex challenges as outcomes of dynamic relationships among technical systems, organizational arrangements, professional practices, and societal values [14]. Leverage points are places where intervention can alter how a system behaves. Changes to technical parameters, such as improving model accuracy or consent procedures, may reduce particular risks but leave the system’s underlying goals and structures intact. Deeper interventions instead seek to change how problems are defined, whose knowledge carries authority, what organizations are designed to achieve, and who has the power to redirect the system [13].

Applying a systems thinking perspective to AI-enabled LTC, we emphasize four domains in which deep leverage points may operate. These domains shape the rules, goals, knowledge hierarchies, and decision-making capacities through which AI applications are selected, designed, evaluated, sustained, modified, or discontinued: (1) societal paradigms and value systems at the level of policy and public discourse, which define what counts as “aging well,” technological progress, and quality of care; (2) epistemic authority in knowledge production and technical design, which determines whose knowledge is used to define care needs, relevant data, and desirable AI outputs; (3) systemic goals and models of AI-enabled LTC at the organizational and care practice levels, which determine what AI is introduced to achieve and whether it strengthens relational care or primarily advances efficiency, surveillance, and risk management; and (4) distributed decision-making power and user agency, which determines whether affected groups can shape, contest, modify, or discontinue AI systems.

Corresponding intervention pathways can be developed across these four domains. First, at the level of policy and public discourse, ethical and responsible AI governance requires challenging the assumption that LTC-related challenges are best addressed through greater prediction, automation, and individual risk management [4]. Quality of care may instead be defined in terms of dignity, reciprocity, inclusion, sustainability, and opportunities for meaningful social connection. Second, at the level of knowledge production and technical design, participation should extend beyond consultation [15]. Older adults, family caregivers, care workers, and community members should have a meaningful role in defining the problems to be addressed, determining which data are relevant, prioritizing outcomes, and deciding whether an AI application is needed at all [16].

Third, at the organizational level, interventions may target the incentives that shape AI adoption and use. Procurement, funding, and evaluation arrangements should reward uses of AI that strengthen relational care, support the workforce, and expand opportunities for social participation. LTC facilities should also assess whether AI applications redirect resources away from human care, intensify surveillance, or transfer institutional responsibilities to residents and caregivers. Fourth, at the level of user and collective agency, affected groups should be able to influence the conditions under which AI is used, with practical mechanisms in place to question, modify, suspend, or discontinue AI applications.

AI is transforming not only how LTC is delivered, but also how care needs, social problems, and desirable outcomes are understood. AI governance should therefore look beyond whether individual applications are safe or compliant and examine the LTC contexts in which they are introduced. This requires ongoing multidisciplinary learning about how workforce conditions, organizational priorities, social relationships, and inequalities shape AI implementation, and how AI, in turn, may reshape these conditions. Given the limited evidence on long-term effects, governance should remain precautionary and adaptive. Most importantly, AI should not reframe structural and relational problems as individual conditions to be technologically managed, particularly when doing so may reproduce underinvestment, weaken care relationships, or leave inadequate social infrastructure unaddressed.

Acknowledgments

This opinion article was developed alongside the authors’ ongoing work on AI-assisted care for older adults, including an international expert meeting they convened at the National University of Singapore on AI-powered care design and implementation for older adults with limited kin availability. The discussions at that meeting highlighted some of the ethical issues addressed here, though this article reflects only the authors’ own interpretations.