Eligibility criteria.

Individuals eligible to participate in the study met the following criteria: (i) assigned male at birth; (ii) aged 18 to 50 years (i.e., age group at highest risk for testicular diseases); (iii) residing in the Republic of Ireland; and (iv) players and/or coaches in Gaelic Athletic Association (GAA) clubs. GAA is Ireland’s largest international amateur sporting and cultural organization, with over 500,000 members in 2,200 clubs nationally [34]. GAA is focused primarily on promoting indigenous Gaelic games and pastimes, which include the traditional Irish sports of hurling, camogie, Gaelic football, Gaelic handball, and rounders [34]. Of note, given that the focus of our study was on process and feasibility rather than efficacy, individuals with a history of testicular disease were eligible for inclusion.

Individuals who were younger than 18 years or older than 50 years, not assigned male at birth, residing outside the Republic of Ireland, and not involved in GAA as either players or coaches were excluded. VR rarely causes motion sickness and the risk for seizures is negligible [35]. However, for safety reasons, individuals with a history of seizures and/or motion sickness were excluded. The four researchers who delivered E-MAT_VR and E-MAT_E and collected data were also eligible to participate in the process evaluation to explore their experiences conducting the study.

Setting.

GAA clubs are increasingly adopting “The Healthy Club” philosophy to improve the health of their members [32]. Therefore, participation was primarily sought from geographically dispersed Healthy GAA clubs in the south of the Republic of Ireland.

An invitation letter was e-mailed on two occasions to the executive committee of each club. Clubs that responded to the e-mail and expressed their interest in participating were asked to advertise the study on their Facebook^TM and X^TM (formerly Twitter^TM) pages. Social media posts explained what the study involved and comprised a link that club members accessed to provide their contact details and register their interest in participating. Ten posters with a quick response (QR) code with the same link were displayed in each participating club. Each recruitment strategy (i.e., Facebook^TM, X^TM, and poster with QR code) was implemented over two weeks per club at random, for a total of six weeks [36].

Club members who registered their interest in participating were contacted by the researchers to assess their eligibility. Potential participants who were eligible were provided with an information leaflet explaining the study process and the risks and benefits from their participation and were given an opportunity to speak to a member of the research team if they had any questions or concerns. They were then asked to provide written informed consent in the presence of research personnel, and those who did were enrolled into the study. Participants were required to sign a second informed consent form if they wished to participate in qualitative interviews as part of the process evaluation. Researchers were also asked to provide written informed consent if they wished to be interviewed.

Procedures.

Data were collected at baseline (T0), immediately post-test (T1), and three months post-test (T2) using the Castor EDC software, a secure clinical data management platform that enables researchers to easily capture and integrate data [37]. Participants completed a baseline questionnaire (T0). They were then individually randomised to either trial arms. Immediately after exposure to E-MAT_VR or E-MAT_E, they completed a second questionnaire (T1). Four research personnel collected T0 and T1 data in-person in participants’ respective GAA clubs, usually before a training session.

Three months post-test (T2), participants received a link by e-mail to complete the third and final questionnaire. Two subsequent reminders were sent, and non-respondents were considered lost to follow-up. Participants who completed T0, T1, and T2 questionnaires received a gift voucher as a token of appreciation.

Process evaluation outcomes.

Participants who consented were interviewed individually either in-person immediately after T1 or later via telephone. The research team participated in a virtual focus group to glean their perspectives of delivering the intervention/control. A semi-structured interview guide was used for participants and researchers (S1 Table in S3 File). Interview guides were developed by four senior academics and researchers who have experience in qualitative research (MMS, MMC, MPD, JH). The interviews and focus group explored how E-MAT_VR and E-MAT_E were perceived by participants and researchers, if/why this varied, and how these perceptions affected intervention/control receptivity. The interviews and focus group also explored the mechanisms through which the intervention/control brought about changes in testicular awareness, which is crucial to understanding how the effects of the intervention/control occurred and how these effects might be replicated in the future [30].

Context which includes anything peripheral to the intervention/control that may influence its implementation, reach, or effects was measured using a survey. The interview and focus group topic guide explored in what context the intervention/control was effective or ineffective, and whether they might be transferrable to other contexts. The usability and satisfaction survey contained a checklist of items with three of those items examining the impact of context on the implementation of the intervention/control. This included determining whether the intervention/control was applicable to real life, men across a range of ages (18 to 50 years), and men from different ethnic backgrounds.

Fidelity checks were conducted to check if E-MAT_VR and E-MAT_E were delivered as intended. This involved having a researcher observe participants during testing and recording (tick box) pre-specified items (e.g., participants’ behaviour, interest, and engagement) [43]. The quality of delivery was also assessed during the fidelity checks. These checklists were purposefully developed for use in the current study. Checklist items were mapped onto the processes and contents of the E-MAT_VR intervention and E-MAT_E control. These checklists were pilot tested with four participants. Data from these participants were not included in the final analysis. No changes were made to these checklists following pilot testing. Data were collected on dose exposure, dose completeness, and dose satisfaction. The time taken to deliver E-MAT_VR and E-MAT_E and the units of intervention/control completed (dose completeness) were captured in minutes. Data were recorded automatically within and extracted from the VR headset for E-MAT_VR participants. The time it took participants to complete E-MAT_E in minutes was recorded manually by the observing researcher. Participants’ satisfaction with the intervention/control, interactions with research staff (dose satisfaction), and extent to which participants were receptive to the intervention/control (dose exposure) were collected in the usability and satisfaction survey.

Reach refers to the proportion of the intended population that participated in the study measured by the number of participants within each arm [29]. In our study, reach was determined by comparing the number of participants who consented to participate relevant to the number of GAA players and coaches who registered their interest in participating but did not.

Feasibility and satisfaction were measured using instruments which assessed usefulness, ease of use, and level of satisfaction with use of both E-MAT_VR and E-MAT_E [23, 42].

Finally, four open-ended questions explored elements of the study that worked and did not work for participants, and any possible changes they would like to make [23, 42]. Process evaluation data including timing, difficulties experienced during E-MAT_VR/E-MAT_E, frequency of adverse events, and implementation problems were also collected by researchers during the study. Implementation problems identified during the trial were categorised as they arose, including the mechanisms by which the issues were addressed.

Pilot efficacy outcomes.

The primary efficacy outcomes were testicular knowledge and testicular self-examination behaviours. The testicular knowledge questionnaire measured participants’ knowledge of the anatomy of the testes, common testicular symptoms and diseases, and testicular self-examination using 12 multiple choice questions [24].

At T0, three items explored whether participants performed testicular self-examination within the past year and month [24]. They were also asked if a healthcare professional ever examined their testes. At T2, one item explored whether participants performed testicular self-examination over the past three months (i.e., since exposure to the intervention/control). All item responses were dichotomous (Yes/No).

The secondary efficacy outcomes included testicular awareness, perceived risk of testicular diseases, help-seeking intentions, implementation intentions, and recommendation of testicular self-examination to others. The testicular awareness scale is a 5-point Likert scale that assessed participants’ testicular awareness, that is their familiarity with their own testes, knowledge of what is normal and what is not normal, and ability to differentiate between what is normal and what is not normal [24, 39].

Participants’ perceived risk of testicular diseases was assessed using one item. The level of agreement for this item was measured on a 5-point Likert scale [24].

Participants’ intentions to seek help for testicular symptoms were assessed using the General Help Seeking Questionnaire [24, 40]. Here, three items assessed participants’ intentions to seek help for three different symptoms (i.e., testicular swelling, lumpiness, and pain) from various sources of help.

Implementation intentions “specify the when, where, and how of responses leading to goal attainment” (p.121) [41]. The Implementation Intentions Scale comprised three items assessing participants’ intentions to feel their own testes in the shower/bath at least once over the coming month and their intentions to advise at least one man to do the same [24]. The level of agreement for each item was measured on a 5-point Likert scale.

One item assessed men’s behaviours in relation to advising at least one man to feel their testes at T2. Item responses were dichotomous (Yes/No) [24].

A sociodemographic questionnaire was administered at T0. It comprised 11 items as follows: age; gender; nationality; marital status; name of GAA club; role in GAA club; highest level of education; current occupation; personal history of testicular disease(s); prior experience with VR; and where participants heard about the feasibility trial. Of note, the names of participating GAA clubs are not disclosed in this paper for confidentiality purposes.

Quantitative data analysis.

Prior to analysis, data underwent extensive quality checking. Given the feasibility aims of the trial, analyses of participant outcomes were largely descriptive. However, we did estimate between-arm contrasts for the knowledge scores and the testicular self-examination behaviours on an intent-to-treat basis. For the former, we estimated the between-arm difference in mean scores using linear regression with adjustment for baseline scores; for the latter we estimated odds ratios using logistic regression. Resulting effect estimates were reported with 95% confidence interval (CI) and p-values. Missing data were addressed using complete case analysis, following from the small number of missing values. Because the pilot feasibility trial was not designed to return a high-powered test of the efficacy of the intervention, we report these results on a purely exploratory basis, and for the purposes of informing the design of a subsequent RCT. Simple between-arm tests for ordinal responses were conducted using Kruskal-Wallis rank sum tests. For multi-item scales, we evaluated the internal consistency of items using Cronbach’s alpha [46], or the equivalent Kuder-Richardson 20 (KR-20) metric [47] when the items were binary. All reported p-values are based on two-sided tests with statistical significance defined as p<0.05.

Analyses were conducted using R (v 4.2.0) and RStudio (2023.06.0 Build 421). All analysis steps, including cleaning or modifications to study database, were fully scripted and replicable. Upon completion, the study database was prepared according to the findability, accessibility, interoperability, and reusability (FAIR) data principles and made available along-side the analysis scripts via the Open Science Foundation (https://osf.io/m5wb7/).

Qualitative data analysis.

Qualitative data from the interviews with participants and focus group with researchers were analysed using qualitative content analysis with pre-defined main categories reflective of the key constructs within the logic model, with an openness to emergent themes [48, 49]. All qualitative data were coded and analysed in NVivo. Similar codes were grouped to form sub-categories which were then collated under main categories. Qualitative data analysis was conducted by one author (MMC) and checked for accuracy by two authors (MPD, JH) to minimise errors and improve data credibility and confirmability [50].

Data from the four open-ended questions were analysed using quantitative content analysis which involved systematically categorising and recording textual material so that they can be analysed [51].

Context.

All participants in E-MAT_VR and 97.3% (n = 36) of those in E-MAT_E either agreed or strongly agreed that the intervention was applicable to men aged between 18 and 50 years. There were similar findings regarding applicability to those from different ethnic and cultural backgrounds; the majority of E-MAT_VR (n = 33, 89.2%) and E-MAT_E (n = 36, 97.3%) participants either agreed or strongly agreed. Thirty-six (97.3%) participants in both E-MAT_VR and E-MAT_E either agreed or strongly agreed that the intervention was close and applicable to real life (S2 Table in S3 File).

Fidelity.

High quality of delivery was associated with both E-MAT_VR and E-MAT_E (fidelity score ≥80%). All participants in E-MAT_VR received a safety briefing and an explanation on the use of the VR headset and controllers. All participants used the VR equipment and began the VR experience as instructed. Two participants in E-MAT_VR (5.4%) ended the VR experience early. One due to nausea and one due to a technical problem in the VR headset. All E-MAT_E participants could use the tablet (n = 37, 100%) and almost all E-MAT_E participants appeared to read everything on the tablet (n = 35, 94.6%).

Dose and reach.

Data on dose exposure were not available for seven participants in E-MAT_VR and one participant in E-MAT_E. The mean completion time was 9.45 minutes (range 7.15 to 12.31 minutes) for E-MAT_VR and 4.07 minutes (range 1.50 to 7 minutes) for E-MAT_E. As for reach, the target sample size of 74 participants was reached.

Usability and satisfaction.

Most participants (67/74, 90.5%) either agreed or strongly agreed that the device was comfortable to use in both E-MAT_VR (n = 34, 91.9%) and E-MAT_E (n = 33, 89.2%). All participants in E-MAT_VR agreed or strongly agreed that the device was easy to use and that they were engaged to learn by the device. While 33/37 E-MAT_E participants (89.2%) agreed or strongly agreed with these statements. Similarly, 34/37 E-MAT_VR participants (91.9%) either agreed or strongly agreed that the time it took to complete was reasonable and all E-MAT_E participants agreed with this statement. All 74 participants either agreed or strongly agreed that the information provided by the device was clear (S2 Table in S3 File).

All 74 participants were either extremely satisfied or somewhat satisfied with their experience of using the PDF reader on the tablet (E-MAT_E) or VR system (E-MAT_VR) to learn about testicular diseases. Similarly, all E-MAT_VR participants and 36/37 E-MAT_E participants (97.3%) were either extremely satisfied or somewhat satisfied with their overall experience of participating in the study (S3 Table in S3 File).

Open-ended questions and qualitative interviews with participants.

All 74 participants answered the four open-ended questions at T1. The full answers are presented in S4 Table in S3 File. Participants in E-MAT_VR (n = 37) described the VR game as “interactive” (n = 16, 43.2%), “easy” (n = 13, 35.1%), “fun” (n = 7, 18.9%) and close to “real life” (n = 4, 10.8%). Elements of the VR game that participants did not favour were the risk of “nausea” and “sickness” (n = 6, 16.2%), as well as “dizziness” (n = 5, 13.5%) and difficulties with “navigation” (n = 4, 10.8%). Most E-MAT_VR participants (n = 30, 81.1%) did not recommend making any changes to the VR game. Recommended changes included having “more interaction” (n = 2, 5.4%) and “more content” (n = 1, 2.7%).

Participants in E-MAT_E (n = 37) mainly liked the tablet because it was “easy” (n = 22, 59.5%), “quick” (n = 6, 16.2%), contained “pictures” (n = 4, 10.8%), and “informative” (n = 3, 8.1%). The main elements of the tablet not favoured related to the “touchscreen function” (n = 7, 18.9%). Others reported “repetition” (n = 1, 2.7%) and “lack of interaction” (n = 1, 2.7%) as drawbacks of E-MAT_E. Most participants in E-MAT_E (n = 24, 64.9%) responded with “none/N/A” when asked about elements of the tablet that they recommend changing.

Responses were explored in greater depth during qualitative interviews with nine participants. As a result, seven main categories and 22 sub-categories were identified. The seven main categories were: Positive aspects of the study; positive aspects of E-MAT_VR; positive aspects of E-MAT_E; challenging aspects of E-MAT_VR; challenging aspects of E-MAT_E; motivation for participating; and recommendations for future research. The main categories, sub-categories, and codes are presented in Table 3. The letter “P” is used to designate participants hereafter.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Findings from qualitative interviews with participants on their views of E-MAT.

https://doi.org/10.1371/journal.pone.0307426.t003

Positive aspects of the study. Participants in both, E-MAT_VR and E-MAT_E described their experience of participating in the study as “enjoyable” (P2, P3, P9) “easy” (P2, P3, P4) and “informative” (P3). One participant seemed motivated to perform testicular self-examination following his participation in the study: “It brings it more to the front of your mind to do checks on yourself” (P8).

Positive aspects of E-MAT_VR. VR technology was perceived as “engaging” (P4, P6), “interactive” (P3, P9), “fun” (P3, P4), and “quick to use” (P4). Although E-MAT_E was described as “more efficient” (P9), E-MAT_VR was perceived as “more interactive” (P9). As a result, participants felt that E-MAT_VR was “much better than writing on a piece of paper” (P5). E-MAT_VR was also perceived as a “better way of learning’” (P5) and a “different type of learning” (P4), encouraging self-directed learning: “you were in control of your learning” (P3).

Positive aspects of E-MAT_E. Many participants described E-MAT_E as “easy to use” (P1, P6, P7, P8, P9), “handy” (P1, P8) and “straight forward” (P8). E-MAT_E was also perceived as “informative’ (P6), “educational” (P8), written in “simple language” (P2) and presented “in a simple format” (P8) making it “simple for a layman to understand” (P8). In addition, the use of the tablet during E-MAT_E enabled participants to “go through the slides at your own speed, your own pace…” (P6).

Challenging aspects of E-MAT_VR. For some participants, navigating the VR environment proved challenging initially as the “controllers were a bit confusing at first” (P4). One participant reported “feeling slightly dizzy at the start” (P4). Fear of the unknown was also an initial concern voiced by one participant: “I felt a tiny bit anxious at the very beginning…because you don’t know” (P5). A technical issue was reported by another participant: “It [E-MAT_VR] froze at the end” (P3).

Challenging aspects of E-MAT_E. Participants felt that E-MAT_E “wasn’t very interactive” (P2) or interactive only “to a certain degree” (P7). Some participants reported that E-MAT_E had “too much text” (P8) and a “good bit of information” (P7). As a result, one participant reported information overload: “Hard to take it all in” (P7).

Motivation for participating. Some participants were motivated to participate in the study to “find out more about men’s health” (P9), particularly in relation to testicular diseases, as described by one participant: “I didn’t really have too much information on the disease, only a small bit there in secondary school” (P7). Others volunteered to participate because the study was “easy” (P6), “quick” (P8), “simple” (P9), and “wasn’t time consuming” (P9). For many, the “50-euro voucher” (P6, P7, P8, P9) given to participants following completion of the third and final questionnaire at T2 incentivised them to participate.

Recommendations for future research. Participants made several recommendations to improve E-MAT_VR and E-MAT_E. Both, E-MAT_VR and E-MAT_E participants recommended making the VR game and tablet “more interactive” (P5, P8). Several E-MAT_E participants advised including “videos” (P6, P7, P8) and “visual interaction” (P7). One participant recommended incorporating VR into secondary school and university curricula: “Maybe it’s something that should be introduced in secondary schools, colleges, as part of a curriculum…to show it to older classrooms, older age groups in secondary schools” (P6). Others recommended using the tablet because it is “straightforward” (P8), “easy to follow” (P6), and “people are well used to using the device” (P7).

Focus group with researchers.

Five main categories and 21 sub-categories were identified from the focus group with the four researchers who collected data. The five main categories were: Positive aspects of E-MAT_VR; challenges to engaging with E-MAT_VR; resources incurred; surveys at T0 and T1; and recommendations for future research. The main categories, sub-categories, and codes are presented in Table 4. The letter “R” is used to designate researchers hereafter.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Findings from the focus group with researchers.

https://doi.org/10.1371/journal.pone.0307426.t004

Positive aspects of E-MAT_VR. Three research personnel discussed the benefits of using VR in various contexts and settings (R1, R3, R4). One researcher discussed the benefits of using VR in education: “So yes across education, it could be provided in any context really” (R1), while another spoke about the value of using VR in other health contexts: “So I know as well you have a student working a breast cancer one…you can do anything with it technically” (R4). In addition, researchers discussed the “enjoyment” (R3) element of VR which made it a “fun way for them [participants] to learn” (R1). One researcher felt E-MAT_VR was valuable for educating students prior to going on clinical placements: “…they [participants] get that sense of what it might be like before they go on placement…that experience and that practice beforehand–I think it could be really helpful” (R1). Another researcher felt that E-MAT_VR served as a “humorous way to give the information” (R4) which might help “keep people interested” (R4) and educated about testicular diseases. In contrast to E-MAT_E which was described by one researcher as “more boring” (R1), E-MAT_VR was described as “more interesting and interactive” (R2).

Challenges to engaging with E-MAT_VR. Information delivery proved challenging, as described by one researcher: “They [participants] are really just being told to point at something” (R4). Technical issues were also reported. These related to VR headsets: “Making sure the batteries didn’t die on the VR because they were used back-to-back” (R1), poor graphics due to ill-fitting headset: “Even people with glasses. I think we had a situation where you know the screen went blurry for them” (R1), and researchers having to use their personal Facebook^TM accounts to sign in: “The Oculus^TM [VR headset brand] portal…that is kind of a bit of a tricky situation…when you can’t make dummy Facebook^TM accounts for it, you have to use personal Facebook^TM accounts” (R2). Another challenge relating to E-MAT_VR was the risk of nausea: “I think, maybe one or two people on the day…maybe one guy in particular got very nauseated from it” (R3).

Resources incurred. Researchers discussed the obstacles they encountered while conducting data collection, specifically in relation to lack of time and personnel available: “Trying to do fidelity checks at the same time while they were doing the interventions was actually a little bit tricky” (R1). Therefore, “allocating the time” (R1) for participants, ensuring there was “enough research members there” (R1), and having “more manpower…somebody doing the checks specifically” (R2) were highlighted as key during data collection. Another challenge related to recruiting participants, particularly among those who initially registered their interest in participating but did not respond to the invite to participate: “When they [participants] are confronted with the sort of the email to say ‘oh thank you for signing can we organise something’, that initial interest maybe drops off” (R2).

Surveys at T0 and T1. Researchers felt that some survey questions at T0 and T1 were “similar” (R3, R4) and “overlapping” (R3). This led to confusion among some participants: “A few people did get a bit confused with the T1 questions…they thought they were answering the same thing again” (R4). Therefore, explaining briefly to participants that the questions are similar in the surveys was an important lesson learned: “Just mention that like these are going to be similar questions again you already answered…it takes a second to say it to them” (R4). In addition, the surveys did not capture “technical feedback” (R4) or information about the “usability of the software” (R4). However, one researcher noted that “putting extra [questions] would have been impossible” (R4) due to the length of the surveys at T0 and T1.

Recommendations for future research. Researchers recommended incorporating VR into education and learning by “bringing them [VR headsets] around to secondary schools…GAA clubs” (R4), and having a designated location “where people can learn in their own time” (R4) such as “[University] library…so if someone wants to spend time they can book that out and they have the headsets” (R4). Making VR accessible was also perceived as important, with one researcher discussing the need to “bring them [VR headsets] to the people…people aren’t very likely to download them” (R4) so “it [E-MAT_VR] has to exist somewhere where people can go to it and…freely” (R4). Ensuring VR is accessible to everyone and not just GAA players was also discussed: “I would go for sports players in general…not just limited to GAA” (R4). Accommodating different levels of VR experiences among participants was also advised by researchers, particularly for “those who struggled with the VR” (R1) and needed “a longer tutorial” (R1) or an “extra bit of guidance of at the start” (R2). Going forward, one researcher recommended having “a process set in stone…to say that this is how we do it…and this is how we deal with people who aren’t so used to VR” (R4). Another key recommendation by researchers was to have “more training” (R1, R4) for the use of Castor EDC software for data collection to help make the process “a little more…streamlined” (R2). Finally, one researcher suggested more “user testing” (R4) to help “find bugs” (R4) in E-MAT_VR. Another researcher suggested “tweaking” (R3) the survey questions to get more “concise” (R3) and “insightful” (R3) answers.

Lesson 1: Using virtual reality to promote the health of young men.

The number of VR users is growing. The global VR market size is projected to increase from <12 billion U.S. dollars in 2022 to >22 billion U.S. dollars by 2025 [65]. Advancements in VR have led to the development of user-friendly, lightweight, affordable, accessible, and wireless VR technology [42]. Most VR users (94%) are aged 18 to 54 years, with the majority (60%) being men [66]. Therefore, VR can serve as a tool to engage hard to reach populations (i.e., younger men) in health promotion initiatives. This could explain the high satisfaction and the high retention rate (89.2%) among our participants. Having said that, while testicular knowledge and testicular self-examination behaviour improved significantly among all participants, these outcomes did not differ significantly by trial arm. However, this was a feasibility trial and a full RCT, with a sample size powered to detect the efficacy of the intervention would be needed to establish efficacy.

Lesson 2: Estimating resources incurred by the interventions.

Interactive digital interventions tend to outperform non-interactive ones [67]. Such interventions, however, are expensive. Indeed, resources incurred in our study were identified by researchers as a potential challenge for a future RCT. Therefore, investigating the cost effectiveness of E-MAT_VR versus E-MAT_E is important to inform policy, resource allocation decisions, and the design and conduct of a future RCT. This would involve, for example, estimating intervention delivery costs inclusive of personnel and material, investigating healthcare utilisation regarding testicular disease at the conclusion of the RCT, and estimating participants’ willingness to pay for E-MAT_VR and E-MAT_E. It is notable that VR as a medium is becoming much more mainstream with the target population more likely to own VR devices in the future which will help with costs [42, 65, 66].

Of note, a feasibility economic evaluation was conducted as part of our study to explore the practicality of conducting a full-scale economic evaluation in a future RCT. Results from the feasibility economic evaluation have been analysed but not published yet. Initial results, however, suggest that we proceed with an economic evaluation for a full RCT.

Lesson 3: Revisiting and refining the intervention.

Some minor technical issues, with a very small number of participants, may have impacted their experience of the intervention. Therefore, there is value from revisiting E-MAT_VR to help identify technical issues/glitches. There is also value from incorporating more visual content, interactive elements, and gamification features to maximise engagement and potentially improve the effectiveness of E-MAT_VR.

Lesson 4: Accounting for confounders.

Given that the focus of our study was on process and feasibility rather than efficacy, we decided from the outset not to exclude participants who had a personal or family history of testicular disease. We do acknowledge, however, that such individuals may be more familiar with signs, symptoms, treatment, burden, and the overall experience of a patient with testicular disease. Furthermore, participants were GAA players and coaches who are more susceptible to testicular injuries and consequent disease. These individuals could also be more aware of testicular diseases. However, one could argue that given their susceptibility to the disease, the target population is appropriate. Nevertheless, these factors could have affected participants’ knowledge at baseline. Notwithstanding this potential confounder, only 2 of the 74 participants reported a personal history of testicular disease. The inclusion of participants who have a personal and/or family history of testicular disease ought to be considered in the design of a future RCT.

Lesson 5: Ensuring diversity within the sample.

The limited ethnic diversity within the sample may have limited our ability to assess the potential problems with the experience of the research process or the intervention in a more diverse sample. This arises from the relatively ethnically homogenous population in Ireland, though it is changing, and is particularly ethnically homogenous for indigenous sports like the GAA. Therefore, there is value from expanding the study population beyond the GAA, to include a more diverse sample of men as well as targeting more heterogenous male-dominated environments.

Lesson 6: Encouraging patient and public involvement.

The importance of patient and public involvement in health research is widely recognised and advocated [68]. The contribution of our public and patient representatives (AO’C, MO’R) was instrumental in ensuring that study documents were clear, and the study was relevant to younger men. In addition, both public and patient representatives played a key role in recruiting several participants from their GAA clubs. This strength ought to be leveraged while planning recruitment in a future RCT.

Lesson 7: Incentivising participants.

As identified by participants and researchers, the provision of a conditional incentive at the conclusion of the study was one of the key contributors to the high retention rate. This warrants consideration in a future RCT. Of note, the model of payment that we used is an appreciation model offered to participants at the end of the study, therefore avoiding undue inducement [69].

Lesson 8: Examining survey length and content.

Lengthy and repetitive surveys tend to lead to missing data, participant disengagement, and drop-out [70]. Indeed, information overload and repetition within the surveys were identified as problematic by some of our participants and researchers, despite having a panel of patient and public representatives review the instruments a priori. To this end, there is value from carefully examining the survey length and content. This would help ensure optimal participant engagement and data collection in a future RCT.

Lesson 9: Using qualitative research.

Qualitative research alongside RCTs provides added value [71, 72]. In our study, qualitative research, particularly in the process evaluation, helped provide an in-depth account of men’s experiences of participating as well as researchers’ experiences of delivering the interventions. Qualitative research also informed the lessons learnt which will be considered in the design and conduct of a future RCT.