Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Assessing the usability of Accessercise to increase physical activity in adults with physical disabilities: A qualitative think-aloud study

  • James A. Haley,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom

    ⨯
  • Daniel J. A. Rhind,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Supervision, Validation, Visualization

    Affiliation School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom

    ⨯
  • David W. Maidment

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Validation, Visualization, Writing – review & editing

    D.W.Maidment@lboro.ac.uk

    Affiliation School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom

    ⨯

Abstract

Background

Mobile health (mHealth) applications (apps) offer a convenient way to increase physical activity to people with disabilities. While several mHealth apps have been developed for this population, there is limited evidence assessing app usability and how this might impact physical activity.

Objective

To investigate the usability of a novel mHealth app, Accessercise, that aims to increase physical activity in people with physical disabilities.

Methods

Twelve adults with different physical disabilities participated in a face-to-face think-aloud interview. Interviews were analysed using deductive content analysis based on the User Version of the Mobile Application Rating Scale (uMARS).

Findings

Data mapped onto 12 uMARS domains; most participants expressed positive views concerning Accessercise, namely, entertainment, customisation, tailoring to the target group, ease of use and navigation, and visual information. Some additional elements were viewed positively but required modification to improve usability, perceived credibility, and relevance, including the app’s layout, visual appeal, interactivity, and quality/quantity of information.

Conclusion

This study provides an example of how the think-aloud method can be employed to evaluate mHealth apps that aims to increase physical activity in people with physical disabilities. Therefore, developers and researchers could use this study to inform future usability evaluations in this area.

Citation: Haley JA, Rhind DJA, Maidment DW (2025) Assessing the usability of Accessercise to increase physical activity in adults with physical disabilities: A qualitative think-aloud study. PLoS ONE 20(4): e0321109. https://doi.org/10.1371/journal.pone.0321109

Editor: Festo Casmir Shayo, Muhimbili National Hospital, TANZANIA, UNITED REPUBLIC OF

Received: August 2, 2024; Accepted: February 28, 2025; Published: April 1, 2025

Copyright: © 2025 Haley et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data reported throughout this study is found within the supplementary materials.

Funding: The author(s) received no specific funding for this work.

Competing interests: No potential conflict of interest was reported by the author(s).

Introduction

Adults with physical disabilities, defined as a loss of physical function or mobility that restrict important life activities [1], experience substantial barriers to physical activity [2]. These barriers include those related to the individual (e.g., negative attitudes/beliefs, employment status) and/or environment (e.g., transportation, inaccessible facilities) [3,4]. Moreover, during the COVID-19 pandemic, national lockdown restrictions added further barriers to participating in physical activities for this population [5]. For example, it was challenging to access specific adapted equipment and it was more difficult for individuals to obtain professional guidance on how to exercise safely at home [6]. As a result, these barriers prevent adults with physical disabilities from participating in physical activity, which in turn, can increase their risk of developing non-communicable diseases, as well as increase anxiety and depression [7]. Therefore, physical activity interventions that effectively address the barriers experienced by adults with disabilities are needed as a priority [8].

One potential solution to the aforementioned barriers is health-based interventions delivered through applications (or apps) on mobile and wireless devices, such as smartphones, tablets or computers. This can be known as mobile-enabled healthcare (mHealth), defined as medical and public health support to healthcare consumers via mobile-based devices (e.g., phones, tablets) [9]. MHealth apps may present a convenient way to deliver physical activity to people with disabilities, mainly due to the sparse and geographically distant number of individuals with disabilities who experience mobility difficulties, which can make the delivery of in-person interventions difficult. MHealth apps could relieve transportation and environmental barriers to physical activity, which are significant for people with physical disabilities [4]. Additionally, use of a mHealth apps by people with disabilities can also alleviate social and health inequalities [10].

MHealth apps are cost-effective, easily accessible at any time, and allow users to personalise their experience [11]. Further, users are not required to consult a personal trainer in-person [12]. Among adults without disabilities, mHealth apps have been reported to be efficacious at increasing physical activity behaviours with small-to-moderate effects [13]. These results may indicate the potential value of using mHealth formatsto deliver physical activity interventions to individuals with physical disabilities. However, people with disabilities have not typically been a primary target for mHealth apps, with very few mHealth apps being designed specifically with (i.e., co-produced) and for people with physical disabilities [14]. This may explain, in part, why there are increasing health care service disparities between people with disabilities and the general population [15].

Some of the existing evidence-based physical activity mHealth apps for adults with physical disabilities include WHEELS [16], ParaSportAPP [17], Fisiofriend [18], SCI Step Together [19], and 9zest Parkinson’s Therapy [20]. However, most existing interventions target a specific disability (e.g., amputee, spinal cord injury). For example, WHEELS [16] is aimed at wheelchair users with SCI or lower-limb amputation, while the 9zest Parkinson’s Therapy app was created to promote physical activity in people with Parkinson’s disease. Few mHealth apps focus on multiple physical disabilities. This could restrict access to physical activity mHealth apps for people with different physical disabilities, potentially increasing sedentary behaviours and reducing physical activity participation. More recently, the Accessercise app has been developed for adults with different physical disabilities (e.g., achondroplasia, amputee). Features of the app include building your own custom workout from a library of exercises according to the user’s physical disabilities, a social hub to interact with other users, and a directory of gyms and fitness facilities based in the United Kingdom (UK) ranked by users for accessibility. A detailed description of the Accessercise app has previously been reported [21]. The focus of Accessercise is to help people with disabilities get fit, strong, and healthy, providing users with several primary functions. First, the Accessercise app features a video library tailored to participants’ needs and disabilities to help illustrate suitable exercises that are demonstrated by a role model with the same impairment. Second, the app offers monitoring function where users can log workouts, track progress, and meet or exceed their goals. Third, an explore section exists where users can search a directory of UK fitness facilities ranked for accessibility. Fourth, a social hub, where users can connect with others, share their progress with followers and groups, and be part of a diverse, supportive, and passionate community.

To ensure new mHealth apps, including, Accessercise, operate as intended, a critical first step is establishing their usability [22]. Usability relates to the formal assessment of the extent to which interaction with a product or system (i.e., mHealth app) is effective, efficient, and viewed as satisfactory by the user [23]. Usability testing aims to detect possible issues when using an app, leading to further developments to improve its functionality and increase the possibility of technology acceptance among end-users [24]. Nevertheless, poor usability is often hindered by hardware issues, such as small screens and limited input opportunities [25], which can lead to low adherence or app rejection [26]. If an end-user does stop using a mHealth app, certain health parameters are no longer controlled for and the app will no longer “facilitate the target behaviour” [27]. Therefore, undertaking usability testing is imperative in designing and (re)developing mHealth apps [28,29]. However, the usability of mHealth apps has received limited attention in the context of increasing physical activity in people with physical disabilities [30]. In the literature, only several mHealth apps for people with disabilities have been assessed for usability, including WHEELS [16], ParaSportAPP [17] and 9zest Parkinson’s Therapy [20]. Overall, these studies reported positive satisfaction with the apps’ usability, potentially leading to increased adherence, accessibility, and effectiveness in improving physical activity in these populations.

To uncover usability issues in mHealth apps, a highly effective method is to use/employ questionnaires, interviews, focus groups, think-aloud methods, and eye tracking [31]. Among these models, the think-aloud method has been widely used for identifying usability problems in digital health tools and is commonly used for evaluating health behaviour change interventions [32,33]. This method requires asking users to continuously verbalise their thoughts and feelings when performing a given task (e.g., navigating through an app) [34]. The advantages of think-aloud interviews are that participants’ thoughts and needs can be elicited during testing, and the ongoing verbalised information collected allows the researcher to better identify the source of possible problems [31]. While methods such as questionnaires and surveys are easy to use, cost-effective and can be used on a large sample, such methods cannot provide specific information on usability issues that need to be addressed and rely on user perception [35].

Unlike other usability testing methods (e.g., observations, questionnaires), think-aloud helps understand how users think about the problem independently and provides a clear picture of the reasoning process in real-time [36]. As a result, think-aloud can help discover usability issues and identify improvements, providing direct information on a user’s thinking and cognitive processes as they happen [37]. The think-aloud approach helps overcome memory/recall bias in methods like questionnaires and interviews by collecting real-time data, preventing data loss that can occur when data is collected retrospectively. [38,39]. Think-aloud interviews also reveal the mental steps users take in real-time, helping designers make informed decisions to improve usability, functionality, and user satisfaction. Additionally, the think-aloud method performed individually offers clear, authentic insights into each user’s experience, allowing them to express their thoughts without group influence or peer pressure, ensuring accurate and detailed data while reducing the risk of group bias. On this basis, the current study employed think-aloud interviews to examine the usability of the Accessercise app, specifically designed for people with different physical disabilities.

Methods

The methods are reported in accordance with the Standards for Reporting Qualitative Research (SRQR) checklist [40].

Qualitative approach and research paradigm

The current study aimed to gain a broad understanding of the usability of Accessercise. The research was grounded in the qualitative, relativist epistemology, whereby multiple realities are socially constructed, and meaning is created through the interaction between the researcher and participants [41].

Researcher characteristics and reflexivity

Qualitative data collection and analysis were undertaken by the first author (JH), who has previous research and experience working with people with disabilities (i.e., physical and sensory) in a physical activity context. Reflexivity has become an integral component of qualitative research [42], recognising that the researcher’s unconscious biases, and conscious reactions and responses can impact on the conduction and interpretation of data [43]. Therefore, reflexivity was considered throughout the research process; hand-written field notes were made by JH in a reflexive diary, which included reflexive and subjective comments after each interview. JH debated and contested the interpretations and conclusions present in the data analysis and report these to all co-authors (DJAR, DWM – who were JH’s PhD supervisors and who both had expertise in qualitative methods, including think-aloud) acting as “critical friends” to maximise reflexivity [44].

Context

One-to-one think-aloud interviews were undertaken between 1 February and 31 May 2023. Interviews were conducted at a mutually convenient time and in a private location within the School of Sport Exercise and Health Sciences at Loughborough University (LU) that was secluded, quiet, and free from distractions. All interviews were facilitated by JH to maintain consistency. The interviewer had no connection with the participants before the study, and no repeat interviews were undertaken with any participants.

Sampling strategy and participants

Between December 2022 and January 2023, twelve participants were recruited through social media platforms (e.g., Facebook, X), word of mouth, and through contacting local sports clubs. The inclusion criteria were: (i) adults aged ≥ 18 years, (ii) living with a physical disability, (iii) a resident living in the UK, and (iv) had access to a compatible internet-enabled smartphone to use the Accessercise app. Individuals were recruited using a purposive sampling strategy [45]. The demographic information of the sample is presented in Table 1. This sample size was deemed sufficient, given that a sample size of 10 to 15 participants is deemed necessary to understand usability [46].

thumbnail
Download:
Table 1. Demographic characteristics of each participant who took part in the study.

https://doi.org/10.1371/journal.pone.0321109.t001

Ethical issues pertaining to human subjects

Full ethical approval was granted by Loughborough University Ethics Review (Human Participants) Sub-Committee (Ref: 2022-8262-9840). Participants received an information sheet and provided written informed consent prior to taking part in the research. Data was stored on a password protected server with anonymity and confidentiality upheld by replacing participants’ names with numbers (e.g., participant P1, P2), as well as removing identifying data from the transcripts.

Data collection methods

Data was collected using qualitative think-aloud interviews [34] to ascertain participants’ views on the usability of the Accessercise app. Think-aloud interviews allow participants to verbalise aloud in real-time everything they think about when completing a task [47]. Therefore, participants communicate their thoughts that would have otherwise remain silent [34]. One-to-one interviews, lasting between 42 and 113 minutes, were audio recorded. The data collect were stored on LU’s password-encrypted Microsoft OneDrive system that was only accessible by the research team.

Data collection instruments and technologies

An interview guide was developed based on a discussion amongst all co-authors and a review of the literature (Supplemental Materials, S1 Table). The guide included broad, open-ended questions, allowing free expression to clarify the participants’ experiences. The interview guide was pilot tested on one participant to confirm that questions were naturally framed and to identify ambiguities or irrelevant questions. No amendments were needed, although some questions were rephrased to improve clarity. Once each interview was completed, participants were given time for open discussion. Furthermore, brief field notes were undertaken by the first author in a dedicated journal throughout each interview. The notes were later used to help contextualise the data generated during analysis. Data saturation was reached after 12 interviews, as no new themes emerged, and further data were unnecessary to answer the research question.

Data processing

Audio recordings were transcribed verbatim within 24 hours of completion of the interview by JH. To confirm immersion and familiarisation with the data set, and to verify accuracy, each transcript was checked for accuracy against the audio recording by JH and was then forwarded to the interviewees for approval, further comments, and/or corrections. The transcripts were then printed, ready for hand-written analysis using highlighters to differentiate between codes (See Supplemental Materials, S2 Table). Despite the possibility of overlooking or mis-categorising analysis by hand due to human bias, fatigue or subjective judgement [48], thematic sorting by hand was chosen because it allows researchers to engage directly with the data, ensuring that subtle or complex patterns are not overlooked [49]. Hand-analysis provides a deeper, more flexible understanding of the data, allowing the researcher full control over the identification and interpretation process [49].

Data analysis

Deductive content analysis, which involves assigning the text to one or more domains, was employed to analyse the data [50]. Content analysis, which is one of the most used techniques to analyse qualitative data [51], was deemed most suitable due to its systematic and objective approach for identifying what content and information is contained in the discourse [52,53]. The analysis was completed in several phases. The first phase began by creating a formative categorisation matrix with four pre-set categories based on components within the User Version of the Mobile Application Rating Scale (uMARS) [54] to identify patterns of meaning in the data and support researcher understanding. The categorisation matrix is a technique used to organise and categorise data into specific themes. This process involved sorting individual quotes from the dataset into specific themes, with each passage of text being coded and assigned to one or more themes based on its content. The uMARS was selected as it provides simplified readability and a reduced number of items, which better allows mHealth evaluation by end users of differing education backgrounds, ensuring a fully inclusive analysis [55]. The uMARS is a simple and validated tool that can be used by end-users to assess the quality of an mHealth app across four objective quality subscales: engagement, functionality, aesthetics, and information quality [56].

In the second phase of the analysis, the JH immersed himself in the data by reading and then re-reading all the interview transcripts to maximise familiarisation and make sense of the data. Additionally, JH read through his reflexive diary on several occasions and met three times with the co-authors to critically review and discuss any reflexive field notes. In the third step, specific relevant passages of text were highlighted and then colour coded using highlighter pens. In the fourth phase, colour-coded passages were clustered, and a descriptive code was produced. In the fifth phase, any codes were grouped and categorised based on their meanings, similarities and differences and then aligned to the prearranged categorisations within different themes and second-order themes highlighting the reappeared basic thoughts typical of the participants descriptions. Lastly, a content analysis map was constructed to share a visual presentation of themes, codes, and their relationships, involving a detailed account and description of each quotation. The presentation of the findings is based on the four pre-determined themes based on the uMARS.

Strategies to enhance trustworthiness

Trustworthiness is one of the essential components of ‘rigour’ in qualitative research [57]. Even though there is no universal definition, trustworthiness relates to the extent to which one can have confidence in the quality and appropriateness of the data and its interpretation [58]. It is the overall impression of quality associated with a research project [59]. Four criteria are widely used to appraise the trustworthiness of qualitative research: credibility, dependability, confirmability and transferability [60]. Nevertheless, this method has been critiqued in recent years, as it has been proposed to violate the philosophical underpinnings of qualitative research [44,58,61]. Researchers, including Burke [61] and Smith and McGannon [44] argue that the trustworthiness criteria are rooted in positivist paradigms, which conflict with the epistemological foundations of qualitative research that emphasises subjectivity, context, and interpretation. They contend that the criteria impose rigid standards that undermine the relational and dynamic nature of qualitative inquiry. In contrast, reflexivity is seen as a superior approach because it acknowledges the researcher’s subjectivity and the co-constructed nature of knowledge.

More recently, the phrase ‘reflexivity’ has been considered vital. Therefore, a “critical friend” technique was used to increase the rigour of the analytical process. The purpose of critical friends is ‘not to agree’ or ‘achieve consensus’ but rather to ‘encourage reflexivity’ [45]. This technique allows researchers to challenge one another’s construction of knowledge and interpretation, reducing research bias [62], which is a critical practice for ensuring qualitative goodness. All co-authors met three times to critically review and discuss the codes, sub-themes, and the hierarchy of the themes, alongside any reflexive field notes. This phase of analysis resulted in the development of several further codes and the merging of existing codes. Trustworthiness was further upheld by pilot testing the interview schedule to review its relevance and to remove ambiguity and irrelevant questions, with no amendments needed. In addition, each transcript and reflexive field notes were emailed to participants, with all participant’s being content, and no amendments were made to the final transcripts.

Findings and discussion

Four higher-order-themes (derived from the uMARS), each with several second-order-themes (generated by the research team) corresponding to the usability of the Accessercise app were reported (Table 2): (i) Engagement (e.g., entertainment, customisation), (ii) Functionality (e.g., ease of use), (iii) Aesthetics (e.g., layout), and (iv) Information (e.g., quantity and quality of information). Each higher-order theme and associated second-order themes with illustrative quotes are presented. Additional quotes are presented in the Supplemental Materials S3 Table. Significance was established through a comparative analysis with existing mHealth literature for people with disabilities. Using the uMARS scale, the study identified app features that enhanced physical activity, extending research on accessibility and user-centered design.

thumbnail
Download:
Table 2. Higher- and second-order themes from the Content Analysis of interview data.

https://doi.org/10.1371/journal.pone.0321109.t002

Engagement

Entertainment.

Entertainment was identified as a strength of Accessercise. For example, P09 (male, cauda equina syndrome) commented, “Nutrition, blogs, shops and podcasts are all amazing features on the app.” Generally, podcasts are becoming increasingly popular to gain health information [63]. Regular engagement with podcasts can lead to listeners forming a relationship with the host, leading to a level of trust [64]. As such, integrating entertainment into apps may enhance user trust, ultimately promoting sustained engagement for people with disabilities.

However, other participants shared disinterest in such entertainment features. One participant explained, “I never look really at blogs and podcasts. I’m not really into them. I’m not that interested” (P03; female, spinal cord injury). This finding could be due to people with disabilities using smartphone apps and the internet much less than the general population [65]. Such a disparity could arise from the expenses associated with smartphones, higher living costs and inherent physical difficulties, such as manual dexterity [66,67]. Therefore, people with disabilities may be reluctant to use interactive features on mHealth apps, which may result in decreased use.

Customisation.

The option to customise workouts was valued across participants, with one participant, P01 (female, spinal cord injury) stating, “I appreciate that you can set a goal and it’s not just about increasing muscle or losing weight” and “I really like that you can choose where you complete your workouts”. This finding is significant given that limited apps include customisable exercise content [68]. Customisation enables users to modify their technology, resulting in a personalised user experience [69]. The ability to exercise at home is valuable, given time and accessibility are common barriers to physical activity for people with disabilities [4]. However, users are only half as likely to engage with mHealth fitness apps with limited customisation [70]. Therefore, to enhance user engagement, app developers should provide goal customisation options, empowering users to personalise content according to their individual needs, which will likely increase user engagement.

Participants also appreciated options for adapted equipment, with one participant stating, “The ability to customise and choose adapted equipment for your workouts is an amazing idea” (P04; male, spinal cord injury). This finding is significant, considering that inadequate equipment is a prevalent obstacle to physical activity for individuals with disabilities [71]. Users exercising at home find value in the ability to choose adapted equipment, which may not be readily available in traditional gyms or may lack accommodations for specific needs [72]. As a result, adaptive equipment options may enable users to engage in environments where equipment access is limited, ultimately increasing physical activity participation.

Interactivity.

Several participants argued that Accessercise lacked any form of feedback. For instance, one participant stated, “I think Accessercise is missing logistics data. I want to review my progress” (P07; male, amputee). To overcome this, one participant stressed, “Yeah, monitoring progression through graphs and statistics would be good” (P09; male, cauda equina syndrome). Users believe feedback can encourage them to undertake more physical activity [73]. Feedback on users’ behaviour is crucial, as it allows them to align progress with their goals [74,75]. Therefore, implementing feedback on mHealth apps can foster progress towards goals, enhancing self-efficacy [76], and potentially encouraging further uptake/adherence in people with physical disabilities.

Most participants liked the option to share workouts on Accessercise. One participant mentioned, “I like the fact that you can share your goals or ideas it’s bit like you know like the Strava fitness app” (P01; female, spinal cord injury). It is essential to create social connections in mHealth apps, as this is likely to encourage participation in an activity [77,78]. This may be particularly important for individuals with disabilities, given they commonly experience loneliness, low perceived social support, and social isolation [79]. Consequently, by integrating social features including peer support and mentorship and real-time interaction (e.g., live chats/video calls) that foster connections in physical activity, mHealth apps for people with disabilities may enhance psychological wellbeing.

Notifications were appreciated by most participants, with one participant (P09; male, cauda equina syndrome) commenting, “Notifications or nudges would really help me undertake more exercise.” Despite too frequent notifications being annoying to some users and potentially discouraging them from using apps [80], push notifications have shown promise for motivating initial enrolment in physical activity interventions [81] and evoking repeated invention use [82]. Utilising push notifications enable users to easily self-monitor their behaviour to achieve their intended goals [83]. As a result, mHealth app developers could include notifications, as these may enhance uptake, use, and adherence apps, thereby facilitating increases in physical activity.

Target group.

The language used within Accessercise was discussed regularly by participants. Some of the key terms included ‘stretching’, ‘building muscle,’ and ‘increasing strength’ when selecting a workout goal. For example, one participant remarked, “The terminology they have used for each goal is good” (P10; male, achondroplasia). Consistency and clarity in terminology are crucial for an app’s success [84]. Plain language improves user experiences, making apps more functional by providing action-oriented information devoid of technical jargon [85]. Thus, when developing mHealth apps, developers should consider aligning the language in the app with everyday user language to maximise content understanding, facilitating take-up and adherence.

Selecting a disability type was highly valued, especially after one participant remarked, “I like the fact that the app is adapted for people with disabilities” (P01; female, spinal cord injury). This finding is significant given that mHealth apps rarely focus on personalisation [86]. Personalisation and tailoring may increase user engagement and adherence to mHealth apps [87]. Therefore, mHealth apps that allow personalisation may foster long-term engagement among people with disabilities as they meet end-users needs.

In addition, participants praised the map function as it allowed them to search for facilities that are accessible using their location (i.e., post/ZIP code). Users could also add accessibility rankings (from 1 [low accessibility to 5 stars [high accessibility]) for gyms and leisure centres in the UK. Specifically, one participant who noted, “I like how you can find local gyms with the explore section, that’s a really useful feature” (P03; female, spinal cord injury). This finding is helpful, given that people with physical disabilities commonly report limited knowledge of where to exercise [71], potentially restricting opportunities to engage in physical activity. On this basis, mHealth apps should consider incorporating information to enable people with disabilities to overcome knowledge and environmental barriers to physical activity. This valuable option is likely to enhance users’ knowledge, fostering positive behaviour and adherence to mHealth apps.

Functionality

Performance.

Most participants experienced technological challenges when using Accessercise due to ongoing developments. As one participant highlighted, “Obviously it’s [blogs, achievements, podcasts, online shop] still coming soon, which is a weakness of the app” (P09; male, cauda equina syndrome). Many mHealth apps are of low quality since they are designed by researchers with limited software development skills [88]. For this reason, mHealth apps can experience technical ‘bugs’, be underdeveloped, and improperly designed [89,90], with these technical challenges negatively impacting take-up and adherence [91]. Therefore, people with physical disabilities may be unlikely to adhere to mHealth apps if their development is still ongoing or poorly designed.

Additionally, one participant expressed hesitancy to purchase the premium paid-for version of the app after claiming, “I do think sixty pounds for the membership on the Accessercise app is quite expensive” (P03). This finding is not surprising given that pricing and costs associated with using apps are a noteworthy issue that impacts their adoption [92]. Therefore, mHealth apps with high subscription costs may result in low adoption rates among people with physical disabilities, especially as this population can have low incomes and are often dependent on state benefits. App developers should design flexible and affordable membership plans, including necessary fee waivers, to assist individuals facing financial obstacles [93].

Ease of use.

Participants highlighted that the user-friendliness of Accessercise played a pivotal role in its perceived ease of use. For instance, one participant stated, “It’s all clear and easy to pick this app up and use it even if you’re a beginner” (P11; male, spinal cord injury). Easy-to-use and convenient mHealth apps that align with users’ desired learning style can contribute to higher levels of user engagement [94]. Furthermore, participants valued the videos provided for each exercise, which may help people with disabilities overcome the barrier of having limited knowledge of physical activity [95]. To demonstrate, one participant remarked, “I also really like the videos because they show you how to actually do the exercise because some people don’t know how to do it properly” (P02; male, sacral agenesis). Critically, videos can aid individuals with limited education, literacy, or enthusiasm, as they involve less reading, scrolling, or swiping through written content [96].

Navigation.

The ability to navigate smoothly through the app was perceived to be helpful by most participants. For example, one participant (P07; male, amputee) highlighted, “I really like the Accessercise app. I think it will be really easy to follow when you are starting the session.” Participants also found the four features at the bottom of the home screen (i.e., social, exercises, explore, and more) particularly helpful for navigation. To illustrate, one participant (P07; male, amputee) said, “It’s quite easy to navigate mainly with the four tabs down the bottom of the screen.” To maintain user engagement, it’s important that the interface in mHealth apps is easy to navigate [97]. Poor design features, including complex navigation, are poorly accepted by users in real-world settings [98]. Apps that provide simple and logical navigation between screens can encourage continued use [99]. Therefore, for widespread use among people with physical disabilities, mHealth apps should adopt simple navigation strategies, enabling successful user engagement and promoting adherence.

Aesthetics

Layout.

Participants generally reported mixed feelings regarding the layout of Accessercise with several positives and negatives presented. Most participants appreciated the simplistic layout that Accessercise provided. For example, one participant commented, “The layout of the app is good they [Accessercise developers] have done a good job!” (P03; female, spinal cord injury). A possible explanation is due to a core group of users being involved in the app development. Apps that are user-centred in design are recommended in the development of mHealth interventions, ensuring that they meet the needs of the end user, resulting in improved usability and lower risk of failure, thus enhancing optimal uptake and adherence among end-users [100].

However, some participants reported concerns with the layout that required improvements. For instance, one participant (P12; male, spinal cord injury & amputee) said, “You want to make an app as simple as possible to use at the moment the layout is a bit dull and it involves a lot of scrolling.” A consistent layout can support users to manoeuvre around the app without getting confused [85]. To overcome this, one participant (P07; male, amputee) recommended presenting the content in alphabetical order to make it easier for users to identify content more quickly, with another (P12; male, spinal cord injury & amputee) suggesting the use of bullet points as well as numbered guides to improve layout “rather than just a block of text”. Overall, app developers should strive to implement solutions (e.g., numbered guides) that are universally designed, accessible, and usable for everyone, including people with physical disabilities [101], which may enable users to locate content quickly without confusion and difficulties.

Visual appeal.

Several participants reported positive first impressions of Accessercise, finding it visually appealing. For example, one participant said, “It comes across pleasant straight away and compared to the other fitness apps there are only four options at the bottom and it’s straight forward and clean” (P04; male, spinal cord injury). This finding is crucial as the design of apps, such as visual displays, can significantly impact users’ experience and their willingness to engage [102], as well as their trust in the content [103]. Consequently, an appealing visual design of an mHealth app will likely foster long-term user engagement.

However, some participants disliked the colour scheme, describing it as “bland” (P12; male, spinal cord injury & amputee), or “excessively monotone” (P01; female, spinal cord injury). Use of colour in an efficient manner is important to maintain user engagement, making mHealth apps more visually appealing and easier to use [104]. According to Holzinger and Errath [105], there should be a consistent colour pattern within the whole app. To address this, one participant suggested, “an area for development for the Accessercise app is having the option to change colours on the app” (P02; male, sacral agenesis).

On this basis, it is apparent that app interfaces should be customised to users’ preferences, such as high-contrast colours [106]. Colour preferences can change as people get older due to colour vision, contrast sensitivity, and visual acuity all declining with age [107]. Thus, for older individuals or those with sight impairments, colour contrasts on mHealth apps should be carefully considered to improve accessibility and acceptance. Incorporating varied colour contrasts is essential for ensuring successful engagement and reliability.

Information

Quality of information.

Participants shared appreciation for the quality of information on Accessercise. One participant expressed satisfaction regarding one section, stating, “The information about the opening hours and contact details [of local gyms/leisure facilities] is good” (P03; female, spinal cord injury). However, other participants felt the usefulness of this information was somewhat limited. For example, one participant remarked, “Yes, a phone number and opening hours are good, but it’s missing information [e.g., reviews on accessibility, cleanliness and tidiness]” (P10; male, achondroplasia). Limited information can hinder people with physical disabilities from obtaining the necessary details for engaging with and adhering to mHealth apps, which may result in low up-take rates [99].

Quantity of information.

Several participants appreciated the quantity of information provided on Accessercise, with one participant remarking, “The fact the Accessercise app has basic information and the pages are short, that’s good.” However, some users found the quantity of information excessive, a weakness needing improvement, “There is too much information per exercise with the big block of text” (P10; male, achondroplasia). To address this issue, one participant proposed recommendations to improve the layout of Accessercise after noting, “lastly on that filter section just like removing the muscle group section if it’s not like you’re doing flexibility or stretching or something like that” (P01; female, spinal cord injury).

Overall, a good quantity of information is imperative for users of mHealth apps to find reliable, accurate, and timely information with minimal effort [108]. Excess information can overwhelm users, leading to cognitive load and potential disinterest in the apps. For example, as smartphones apps contain a lot of redundant material, searching for information can often take time and effort, resulting in user tiredness due to cognitive overload [109]. Therefore, to optimise user engagement with mHealth apps, information could be presented clearly and concisely in accessible terms [110].

Visual information.

The visual information on Accessercise played a vital role in its perceived appearance; participants emphasised the significance of appealing images, “The images they have used on the app is good” (P04; male, spinal cord injury). Notably, participants shared an interest in images depicting muscles targeted during exercises and the logos used throughout Accessercise, with two participants affirming, “I like the pictures of the muscle components being worked” (P03; female, spinal cord injury), and “The little logo they have used next to each goal is creative and helpful” (P07; male, amputee). In summary, these findings underscore the significance of integrating visually appealing information on mHealth apps for individuals with physical disabilities, as it is well-received by this population. Thus, mHealth physical activity apps could persist in incorporating impactful logos and images to enhance uptake, use, and adherence among users.

Final reflections

Our study is one of the first to understand the perception of end-users with physical disabilities using think-aloud to explore the usability of a novel physical activity mHealth app, Accessercise. This study demonstrated that the Accessercise app provides a simple platform that, despite several troubleshooting/performance challenges, can be efficiently operated by various users with different physical disabilities. People with physical disabilities are expected to overcome multiple barriers (e.g., cost, time, lack of adapted equipment) to engage in regular physical activities. However, our findings indicate that Accessercise could be an effective intervention to help overcome some of these previously identified barriers. For example, the app offers high-quality exercise videos that are customised for people with varying physical disabilities using a range of easily accessible equipment (e.g., water bottle, rucksack), which may act as facilitators in the long-term to encourage regular physical activity among this population.

Therefore, these findings have helped to further our understanding of how a mHealth physical activity app can be designed and configured to meet the needs of people with physical disabilities. Such findings could help app developers reconsider the ways in which apps are currently designed based on user’s preferences, which might, in turn, increase the uptake and adherence and, potentially, result in successfully increasing their participation in physical activity.

Study limitations and future research

First, the sample population were predominantly White British, English-speaking, male, all under the age of 58 years, and had sufficient internet skills, which limits the generalisability of experiences identified to other people with physical disabilities. To ensure that Accessercise is accessible to a diverse range of users, future research that involves a more heterogenous sample is warranted. Second, participants completed a think-aloud interview under the observation of a researcher in an unfamiliar setting (i.e., a university laboratory), which is not analogous to how they would engage with the app in real-world settings (e.g., home, gym). Consequently, valuable data on the app’s performance in a more ecologically valid environment could have been overlooked. For future studies, researchers may want to supplement think-aloud interviews with extensive field testing to address such challenges.

Third, the controlled laboratory setting and audio-voice recording interviews may have influenced participants’ responses, potentially leading to overly positive feedback and reduced willingness to criticise the Accessercise app. Despite building rapport with participants, informing them they could be open and honest and that the researchers were independent from the app, such experiences may have induced or increased anxiety and impacted the participants’ ability to use the app efficiently. Future studies must consider the setting where usability tests occur more cautiously; the physical environment should be the least stressful. Fourth, some app functionalities (such as blogs, podcasts, and nutrition) were under development during the interview phase and therefore were not provided to the participants regardless of the target group’s interest, which could have affected the user’s satisfaction with the Accessercise app. Future studies should consider incorporating such functions to gather rich data on user intention and acceptance of these and whether they will support usability from its end-users.

Fifth, this study is susceptible to confirmation bias, as the researchers may have favoured data supporting pre-existing assumptions about the mHealth app’s effectiveness, while underreporting or overlooking critical usability issues. Future studies could mitigate confirmation bias by engaging a diverse group of researchers external to the primary research team when analysing the data, ensuring that both positive and negative feedback are equally weighted. Sixth, JH conducted the testing and independently analysed the data, which increased the risk of observational bias. Both roles may have influenced the interpretation of participants behaviours and feedback, introducing a subjective lens that may have impacted the objectivity and reliability of the conclusions drawn. Future studies could benefit from involving multiple researchers in both data collection and analysis to reduce potential bias and enhance objectivity.

Conclusion

This qualitative think-aloud study offers a unique contribution to the literature, as it adds to the limited number of usability studies evaluating mHealth apps in people with physical disabilities [14,30]. These studies are highly valued and needed to understand the utility of such interventions [111]. Specifically, this usability evaluation of Accessercise, using a think-aloud method, has revealed high acceptance and the potential to provide people with physical disabilities with an easy-to-use, accessible intervention to increase physical activity. Characteristics that promoted usability included ease of navigation, good aesthetics (e.g., layout, colour), high-quality information, and customisable features (e.g., selecting an impairment, tailored workout videos, and workout locations). Consequently, given limited usability studies in this area, we have shown that this method is acceptable and can be informative as we have provided an evidence-based protocol that other researchers and app developers could use to inform their usability evaluations. Future developers of physical activity interventions aimed at this population should also consider leveraging the insights acknowledged here to maximise their usability, improving user acceptance, as well as increasing health behaviour change, namely, physical activity participation. This could also have further impacts, such as reducing the risk of developing associated conditions, including diabetes, obesity and cardiovascular disease.

Supporting information

S1 Table. Interview schedule.

https://doi.org/10.1371/journal.pone.0321109.s001

(DOCX)

S2 Table. Transcripts of interviews undertaken with participants 1-12.

https://doi.org/10.1371/journal.pone.0321109.s002

(DOCX)

S3 Table. Content analysis map.

https://doi.org/10.1371/journal.pone.0321109.s003

(DOCX)

Acknowledgments

The authors would like to thank all the participants for participating in this research. The authors would also like to thank the founders of Accessercise, Sam Breary, Ali Jawad, and Yulia Kyrpa, for allowing the research team and participants free full access to the app.

References

  1. 1. Kalpakjian CZ, Kreschmer JM, Slavin MD, Kisala PA, Quint EH, Chiaravalloti ND, et al. Reproductive health in women with physical disability: a conceptual framework for the development of new patient-reported outcome measures. Journal of Women’s Health. 2020;29(11):1427–36.
  2. 2. Wilroy J, Lai B, Currie M, Young H-J, Thirumalai M, Mehta T, et al. Teleassessments for enrollment of adults with physical or mobility disability in a home-based exercise trial in response to COVID-19: usability study. JMIR Form Res. 2021;5(11):e29799. pmid:34792477
  3. 3. Williams TL, Smith B, Papathomas A. The barriers, benefits and facilitators of leisure time physical activity among people with spinal cord injury: a meta-synthesis of qualitative findings. Health Psychol Rev. 2014;8(4):404–25. pmid:25211208
  4. 4. Martin Ginis KA, Ma JK, Latimer-Cheung AE, Rimmer JH. A systematic review of review articles addressing factors related to physical activity participation among children and adults with physical disabilities. Health Psychol Rev. 2016;10(4):478–94. pmid:27265062
  5. 5. de Boer DR, Hoekstra F, Huetink KIM, Hoekstra T, Krops LA, Hettinga FJ. Physical activity, sedentary behavior and well-being of adults with physical disabilities and/or chronic diseases during the first wave of the COVID-19 pandemic: a rapid review. Int J Environ Res Public Health. 2021;18(12):6342. pmid:34208156
  6. 6. Marco-Ahulló A, Montesinos-Magraner L, González L, Morales J, Bernabéu-García J, García-Massó X. Impact of COVID-19 on the self-reported physical activity of people with complete thoracic spinal cord injury full-time manual wheelchair users. The Journal of Spinal Cord Medicine. 2022;45(5):755–9.
  7. 7. Battalio SL, Huffman SE, Jensen MP. Longitudinal associations between physical activity, anxiety, and depression in adults with long-term physical disabilities. Health Psychol. 2020;39(6):529–38. pmid:32202825
  8. 8. Ma JK, West CR, Martin Ginis KA. The effects of a patient and provider co-developed, behavioral physical activity intervention on physical activity, psychosocial predictors, and fitness in individuals with spinal cord injury: a randomized controlled trial. Sports Medicine. 2019;49(7):1117–31.
  9. 9. Khan ZF, Alotaibi SR. Applications of artificial intelligence and big data analytics in m-health: a healthcare system perspective. Journal of healthcare engineering. 2020;2020:1–5.
  10. 10. Toquero CM. Mobile healthcare technology for people with disabilities amid the COVID-19 pandemic. European Journal of Environment and Public Health. 2021;5(1)
  11. 11. LeBeau K, Huey LG, Hart M. Assessing the quality of mobile apps used by occupational therapists: evaluation using the user version of the mobile application rating scale. JMIR Mhealth Uhealth. 2019;7(5):e13019. pmid:31066712
  12. 12. Geraedts H, Zijlstra A, Bulstra SK, Stevens M, Zijlstra W. Effects of remote feedback in home-based physical activity interventions for older adults: a systematic review. Patient Educ Couns. 2013;91(1):14–24. pmid:23194823
  13. 13. Fanning J, Mullen SP, McAuley E. Increasing physical activity with mobile devices: a meta-analysis. J Med Internet Res. 2012;14(6):e161. pmid:23171838
  14. 14. Jones M, Morris J, Deruyter F. Mobile healthcare and people with disabilities: current state and future needs. Int J Environ Res Public Health. 2018;15(3):515. pmid:29538292
  15. 15. Singh K, Drouin K, Newmark L, Lee J, Faxvaag A, Rozenblum R, et al. Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Affairs. 2016;35(12):2310–8.
  16. 16. Hoevenaars D, Holla JFM, Te Loo L, Koedijker JM, Dankers S, Houdijk H, et al. Mobile app (WHEELS) to promote a healthy lifestyle in wheelchair users with spinal cord injury or lower limb amputation: usability and feasibility study. JMIR Form Res. 2021;5(8):e24909. pmid:34379056
  17. 17. Marco-Ahulló A, Montesinos-Magraner L, González L, Crespo-Rivero T, Launois-Obregón P, García-Massó X. Encouraging people with spinal cord injury to take part in physical activity in the COVID-19 epidemic through the mHealth ParaSportAPP. Healthcare. 2022;10(6):1069.
  18. 18. Bizzarini E, Chittaro L, Frezza M, Polo M, Malisan C, Menosso R, et al. A mobile app for home-based exercise in spinal cord injured persons: proposal and pilot study. Digit Health. 2022;8:20552076211070724. pmid:35140978
  19. 19. Lawrason SVC, Martin Ginis KA. Evaluating the feasibility, acceptability, and engagement of an mhealth physical activity intervention for adults with spinal cord injury who walk: a randomized controlled trial. J Sport Exerc Psychol. 2023;45(2):61–76. pmid:36870347
  20. 20. Landers MR, Ellis TD. A mobile app specifically designed to facilitate exercise in parkinson disease: single-cohort pilot study on feasibility, safety, and signal of efficacy. JMIR Mhealth Uhealth. 2020;8(10):e18985. pmid:33016887
  21. 21. Haley JA, Rhind DJA, Maidment DW. Applying the behaviour change wheel to assess the theoretical underpinning of a novel smartphone application to increase physical activity in adults with spinal cord injuries. Mhealth. 2023;9:10. pmid:37089265
  22. 22. Goldberg L, Lide B, Lowry S, Massett HA, O’Connell T, Preece J, et al. Usability and accessibility in consumer health informatics current trends and future challenges. Am J Prev Med. 2011;40(5 Suppl 2):S187-97. pmid:21521594
  23. 23. Aiyegbusi OL. Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems. Qual Life Res. 2020;29(2):325–33. pmid:31691202
  24. 24. Stonbraker S, Cho H, Hermosi G, Pichon A, Schnall R. Usability testing of a mHealth app to support self-management of HIV-associated non-AIDS related symptoms. In: Nursing Informatics 2018. 2018. pp. 106–10. IOS Press.
    • 25. Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. Journal of Interaction Science. 2013;1(1):1–6.
    • 26. Inal Y, Wake JD, Guribye F, Nordgreen T. Usability evaluations of mobile mental health technologies: systematic review. J Med Internet Res. 2020;22(1):e15337. pmid:31904579
    • 27. Fontecha J, González I, Bravo J. A usability study of a mHealth system for diabetes self-management based on framework analysis and usability problem taxonomy methods. Journal of Ambient Intelligence and Humanized Computing. 2023;14(1):5–15.
    • 28. Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst. 2015;39(2):1. pmid:25600193
    • 29. Kumar B, Mohite P. Usability of mobile learning applications: a systematic literature review. Journal of Computers in Education. 2018;5(1):1–7.
    • 30. Almeida AF, Rocha NP, Silva AG. Methodological quality of manuscripts reporting on the usability of mobile applications for pain assessment and management: a systematic review. Int J Environ Res Public Health. 2020;17(3):785. pmid:32012674
    • 31. Vanicek T, Popelka S. The think-aloud method for evaluating the usability of a regional atlas. ISPRS International Journal of Geo-Information. 2023;12(3):95.
    • 32. Yardley L, Morrison LG, Andreou P, Joseph J, Little P. Understanding reactions to an internet-delivered health-care intervention: accommodating user preferences for information provision. BMC Med Inform Decis Mak. 2010;10:52. pmid:20849599
    • 33. Broekhuis M, van Velsen L, Hermens H. Assessing usability of eHealth technology: a comparison of usability benchmarking instruments. International journal of medical informatics. 2019;128:24–31.
    • 34. Ericsson KA, Simon HA. How to study thinking in everyday life: contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity. 1998;5(3):178–86.
    • 35. Keenan H, Duke S, Wharrad H, Doody G, Patel R. Usability: an introduction to and literature review of usability testing for educational resources in radiation oncology. Technical Innovations & Patient Support in Radiation Oncology. 2022;24(1):67–72.
    • 36. Reinhart A, Evans C, Luby A, Orellana J, Meyer M, Wieczorek J, Elliott P, Burckhardt P, Nugent R. Think-aloud interviews: A tool for exploring student statistical reasoning. Journal of Statistics and Data Science Education. 2022;30(2):100–13.
    • 37. Doi T. Usability textual data analysis: a formulaic coding think-aloud protocol method for usability evaluation. Applied Sciences. 2021;11(15):7047.
    • 38. Jaspers M, Steen T, Van Den Bos C, Geenen M. The think aloud method: a guide to user interface design. International journal of medical informatics. 2004;73(11–12):781–95.
    • 39. Brusco NK, Watts JJ. Empirical evidence of recall bias for primary health care visits. BMC Health Serv Res. 2015;15:381. pmid:26373712
    • 40. O’Brien B, Harris I, Beckman T, Reed D, Cook D. Standards for reporting qualitative research: a synthesis of recommendations. Academic Medicine. 2014;89(9):1245–51.
    • 41. Kivunja C, Kuyini A. Understanding and applying research paradigms in educational contexts. International Journal of Higher Education. 2017;6(5):26–41.
    • 42. Adeagbo MJ. An ‘outsider within’: considering positionality and reflexivity in research on HIV-positive adolescent mothers in South Africa. Qualitative Research. 2021;21(2):181–94.
    • 43. Oliphant S, Bennett C. Using reflexivity journaling to lessen the emic–etic divide in a qualitative study of Ethiopian immigrant women. Qualitative Social Work. 2020;19(4):599–611.
    • 44. Smith B, McGannon K. Developing rigor in qualitative research: problems and opportunities within sport and exercise psychology. International Review of Sport and Exercise Psychology. 2018;11(1):101–21.
    • 45. Patton MQ. Qualitative research & evaluation methods. Sage; 2002.
      • 46. Schmettow M. Sample size in usability studies. Communications of the ACM. 2012;55(4):64–70.
      • 47. Davies EL. Feasibility of the prototype willingness model as the basis for school-delivered alcohol misuse prevention: a qualitative think-aloud study to explore acceptability of ‘The Alcohol Smart Quiz’ with adolescents and teachers. J Health Psychol. 2018;23(9):1196–210.
      • 48. Nowell L, Norris J, White D, Moules N. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods. 2017;16(1):1609406917733847.
      • 49. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res psychol. 2006;3(2):77–101.
      • 50. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15. pmid:18352969
      • 51. White M, Marsh E. Content analysis: A flexible methodology. Library trends. 2006;55(1):22–45.
      • 52. Shuyler KS, Knight KM. What are patients seeking when they turn to the Internet? Qualitative content analysis of questions asked by visitors to an orthopaedics Web site. Journal of medical Internet research. 2003;5(4):e898.
      • 53. McKay FH, Cheng C, Wright A, Shill J, Stephens H, Uccellini M. Evaluating mobile phone applications for health behaviour change: A systematic review. J Telemed Telecare. 2018;24(1):22–30. pmid:27760883
      • 54. Stoyanov S, Hides L, Kavanagh D, Wilson H. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR mHealth and uHealth. 2016;4(2):e5849.
      • 55. Shinohara Y, Yamamoto K, Ito M, Sakata M, Koizumi S, Hashisako M, et al. Development and validation of the Japanese version of the uMARS (user version of the mobile app rating system). International Journal of Medical Informatics. 2022;165:104809.
      • 56. Chasiotis G, Stoyanov SR, Karatzas A, Gravas S. Greek validation of the user version of the Mobile Application Rating Scale (uMARS). J Int Med Res. 2023;51(3):3000605231161213. pmid:36924222
      • 57. Eryılmaz Ö. Are dissertations trustworthy enough? The case of Turkish ph. d. dissertations on social studies education. Participatory Educational Research. 2022;9(3):344–61.
      • 58. Stahl N, King J. Expanding approaches for research: understanding and using trustworthiness in qualitative research. Journal of Developmental Education. 2020;44(1):26–8.
      • 59. Rose J, Johnson C. Contextualizing reliability and validity in qualitative research: toward more rigorous and trustworthy qualitative social science in leisure research. Journal of Leisure Research. 2020;51(4):432–51.
      • 60. Lincoln YS, Guba EG. Naturalistic inquiry. Sage; 1985.
        • 61. Burke S. Rethinking ‘validity’and ‘trustworthiness’ in qualitative inquiry: How might we judge the quality of qualitative research in sport and exercise sciences? In Routledge handbook of qualitative research in sport and exercise. 2016. p. 352-362. Routledge.
          • 62. Sparkes A, Smith B. Qualitative research methods in sport, exercise and health: From process to product. Routledge; 2013
            • 63. Dingfelder S. Giving psychology away, one podcast at a time. PsycEXTRA Dataset. 2010.
            • 64. Ragusea A. Three ways podcasts and radio actually aren’t quite the same. The Current. 2015.
            • 65. Duplaga M. Digital divide among people with disabilities: Analysis of data from a nationwide study for determinants of Internet use and activities performed online. PLoS One. 2017;12(6):e0179825. pmid:28662125
            • 66. Cho J, Lee H. Contextualization of motivations determining the continuance intention to use smart devices among people with physical disabilities. Telematics and Informatics. 2017;34(1):338–50.
            • 67. John E, Thomas G, Touchet A, Morciano M. Disability price tag. Policy report. 2019.
              • 68. Lai B, Young H-J, Bickel CS, Motl RW, Rimmer JH. Current Trends in Exercise Intervention Research, Technology, and Behavioral Change Strategies for People With Disabilities: A Scoping Review. Am J Phys Med Rehabil. 2017;96(10):748–61. pmid:28398967
              • 69. Bol N, Høie NM, Nguyen MH, Smit ES. Customization in mobile health apps: explaining effects on physical activity intentions by the need for autonomy. Digit Health. 2019;5:2055207619888074. pmid:31807312
              • 70. Razaghizad A, McKee T, Malhamé I, Friedrich M, Giannetti N, Coristine A, et al. Mobile health fitness interventions: impact of features on routine use and data sharing acceptability. JACC: Advances. 2023;2(8):100613.
              • 71. Rimmer JH, Riley B, Wang E, Rauworth A, Jurkowski J. Physical activity participation among persons with disabilities: barriers and facilitators. Am J Prev Med. 2004;26(5):419–25. pmid:15165658
              • 72. Diaz R, Miller EK, Kraus E, Fredericson M. Impact of Adaptive Sports Participation on Quality of Life. Sports Med Arthrosc Rev. 2019;27(2):73–82. pmid:31046012
              • 73. Tong HL, Coiera E, Laranjo L. Using a Mobile Social Networking App to Promote Physical Activity: A Qualitative Study of Users’ Perspectives. J Med Internet Res. 2018;20(12):e11439. pmid:30578201
              • 74. Fjeldsoe BS, Marshall AL, Miller YD. Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med. 2009;36(2):165–73. pmid:19135907
              • 75. Voth E, Oelke N, Jung M. A theory-based exercise app to enhance exercise adherence: a pilot study. JMIR mHealth and uHealth. 2016;4(2):e4997.
              • 76. Sullivan A, Lachman M. Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Frontiers in Public Health. 2017;4:222909.
              • 77. Steinhubl SR, Muse ED, Topol EJ. The emerging field of mobile health. Sci Transl Med. 2015;7(283):283rv3. pmid:25877894
              • 78. Peever N, Vella K, Johnson D, Ploderer B, Klarkowski M, Mitchell J. Understanding initial experiences with Mindmax, an mHealth app that draws on shared interests in sports and video games. Proceedings of the 29th Australian conference on computer-human interaction. 2017:438–42.
              • 79. Emerson E, Fortune N, Llewellyn G, Stancliffe R. Loneliness, social support, social isolation and wellbeing among working age adults with and without disability: Cross-sectional study. Disabil Health J. 2021;14(1):100965. pmid:32843311
              • 80. Heffernan K, Chang S, Maclean S, Callegari E, Garland S, Reavley N, et al. The potential of eHealth apps to support targeted complex health messages. J Gen Practice. n.d.;182(182):80.
              • 81. Bardus M, Blake H, Lloyd S, Suggs L. Reasons for participating and not participating in a e-health workplace physical activity intervention: A qualitative analysis. International Journal of Workplace Health Management. 2014;7(4):229–46.
              • 82. De Leon E, Fuentes L, Cohen J. Characterizing periodic messaging interventions across health behaviors and media: systematic review. Journal of Medical Internet Research. 2014;16(3):e2837.
              • 83. Woodward J, Chen Y, Jurczyk K, Ross K, Anthony L, Ruiz J. A survey of notification designs in commercial mHealth apps. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021:1–7.
              • 84. Couture B, Lilley E, Chang F, DeBord Smith A, Cleveland J, Ergai A, et al. Applying User-Centered Design Methods to the Development of an mHealth Application for Use in the Hospital Setting by Patients and Care Partners. Appl Clin Inform. 2018;9(2):302–12. pmid:29742756
              • 85. Broderick J, Devine T, Langhans E, Lemerise A, Lier S, Harris L. Designing health literate mobile apps. NAM Perspectives. 2014.
              • 86. Cavallo FR, Toumazou C. Personalised lifestyle recommendations for type 2 diabetes: Design and simulation of a recommender system on UK Biobank Data. PLOS Digit Health. 2023;2(8):e0000333. pmid:37647301
              • 87. Jakob R, Harperink S, Rudolf AM, Fleisch E, Haug S, Mair JL, et al. Factors Influencing Adherence to mHealth Apps for Prevention or Management of Noncommunicable Diseases: Systematic Review. J Med Internet Res. 2022;24(5):e35371. pmid:35612886
              • 88. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. Experiences and Results of Applying Tools for Assessing the Quality of a mHealth App Named Heartkeeper. J Med Syst. 2015;39(11):142. pmid:26345452
              • 89. Paschou M, Sakkopoulos E, Tsakalidis A. easyHealthApps: e-Health Apps dynamic generation for smartphones & tablets. J Med Syst. 2013;37(3):9951. pmid:23666429
              • 90. de la Torre-Díez I, Martínez-Pérez B, López-Coronado M, Díaz JR, López MM. Decision support systems and applications in ophthalmology: literature and commercial review focused on mobile apps. J Med Syst. 2015;39(1):174. pmid:25472731
              • 91. Bentley C, Powell L, Potter S, Parker J, Mountain G, Bartlett Y, et al. The use of a smartphone app and an activity tracker to promote physical activity in the management of chronic obstructive pulmonary disease: randomized controlled feasibility study. JMIR mHealth and uHealth. 2020;8(6):e16203.
              • 92. Aydin G, Silahtaroglu G. Insights into mobile health application market via a content analysis of marketplace data with machine learning. PLoS One. 2021;16(1):e0244302. pmid:33406100
              • 93. Oyebode O, Alqahtani F, Orji R. Using machine learning and thematic analysis methods to evaluate mental health apps based on user reviews. IEEE Access. 2020;8:111141–58.
              • 94. World Health Organization. Patient engagement. 2016:1–50.
              • 95. Martin Ginis KA, van der Ploeg HP, Foster C, Lai B, McBride CB, Ng K, et al. Participation of people living with disabilities in physical activity: a global perspective. Lancet. 2021;398(10298):443–55. pmid:34302764
              • 96. Ben-Zeev D, Brian RM, Aschbrenner KA, Jonathan G, Steingard S. Video-based mobile health interventions for people with schizophrenia: bringing the “pocket therapist” to life. Psychiatr Rehabil J. 2018;41(1):39–45. pmid:27295133
              • 97. Al-Abbadey M, Fong MM-W, Wilde LJ, Ingham R, Ghio D. Mobile health apps: an exploration of user-generated reviews in Google Play Store on a physical activity application. Digit Health. 2021;7:20552076211014988. pmid:34017609
              • 98. Ting Y. The pitfalls of mobile devices in learning: a different view and implications for pedagogical design. Journal of Educational Computing Research. 2012;46(2):119–34.
              • 99. Morey SA, Barg-Walkow LH, Rogers WA. Managing heart failure on the Go: usability issues with mHealth apps for older adults. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2017 Sep (Vol. 61, No. 1). Los Angeles, CA: SAGE Publications; 2017. p. 1–5.
                • 100. Humayoun SR, Dubinsky Y, Catarci T. A three-fold integration framework to incorporate user–centered design into agile software development. In Human Centered Design: Second International Conference, HCD 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9-14, 2011. Proceedings 2 2011 (pp. 55–64). Springer, Berlin Heidelberg; 2011.
                  • 101. Henni SH, Maurud S, Fuglerud KS, Moen A. The experiences, needs and barriers of people with impairments related to usability and accessibility of digital health solutions, levels of involvement in the design process and strategies for participatory and universal design: a scoping review. BMC Public Health. 2022;22(1):35. pmid:34991540
                  • 102. Lazard AJ, Babwah Brennen JS, Belina SP. App Designs and Interactive Features to Increase mHealth Adoption: User Expectation Survey and Experiment. JMIR Mhealth Uhealth. 2021;9(11):e29815. pmid:34734829
                  • 103. Lowry P, Wilson D, Haig W. A picture is worth a thousand words: Source credibility theory applied to logo and website design for heightened credibility and consumer trust. Int J Hum-Comput Interact. 2014;30(1):63–93.
                  • 104. Kaur E, Haghighi PD. A context-aware usability model for mobile health applications. In Proceedings of the 14th International Conference on Advances in Mobile Computing and Multi Media 2016 Nov 28 (pp. 181–9).
                    • 105. Holzinger A, Errath M. Mobile computer Web-application design in medicine: some research based guidelines. Universal Access in the Information Society. 2007;6:31–41.
                    • 106. Ramdowar H, Khedo KK, Chooramun N. A comprehensive review of mobile user interfaces in mHealth applications for elderly and the related ageing barriers. Universal Access in the Information Society. 2023;1(1):1–7.
                    • 107. Isaković M, Sedlar U, Volk M, Bešter J. Usability pitfalls of diabetes mHealth apps for the elderly. J Diabetes Res. 2016;2016:1604609. pmid:27034957
                    • 108. Roy S, Poonia RC, Nayak SR, Kumar R, Alzahrani KJ, Alnfiai MM, et al. Evaluating the usability of mhealth applications on type 2 diabetes mellitus using various mcdm models. Healthcare. 2022;10(1):4.
                    • 109. Pang H, Ruan Y. Can information and communication overload influence smartphone app users’ social network exhaustion, privacy invasion and discontinuance intention? A cognition-affect-conation approach. J Retail Consum Serv. 2023;73:103378.
                    • 110. Riis A, Hjelmager DM, Vinther LD, Rathleff MS, Hartvigsen J, Jensen MB. Preferences for web-based information material for low back pain: qualitative interview study on people consulting a general practitioner. JMIR Rehabilitation and Assistive Technologies. 2018;5(1):e8841.
                    • 111. Hanghøj S, Boisen KA, Hjerming M, Elsbernd A, Pappot H. Usability of a mobile phone app aimed at adolescents and young adults during and after cancer treatment: qualitative study. JMIR Cancer. 2020;6(1):e15008. pmid:31895046