Introduction

Phishing is the spoofing of Internet websites or emails aimed at tricking users into entering sensitive information, such as usernames and passwords [1]. It is one of the most severe and challenging threats to Internet security. As Internet accessibility continues to increase, and as the sophistication of phishing attacks continues to improve [2], individuals of all ages and backgrounds are increasingly susceptible to this threat. Falling victim to a phishing attempt may lead to identity theft and loss of financial resources [3]. In older adults, these outcomes may have important consequences for living independently and for maintaining social, emotional, and medical health.

An abundance of research in the computer sciences and human factors fields has addressed phishing susceptibility and its association with demographic variables. For instance, Metzger et al. [4] reported that students were more likely to trust all information channels and less likely to verify online information compared to nonstudents. Sheng et al. [5] found that women and people between 18 and 25 years old were less suspicious of phishing than people of other ages. The effect of age on phishing susceptibility was largely explained by prior experience with phishing, experience using the Internet, perception of financial risk, and years of education. The average age of their sample, recruited through Amazon’s Mechanical Turk, was 30; but because no other information about the participants’ age was reported, the composition of older adults in that study is unclear [5]. Using a role-play paradigm similar to Sheng and colleagues [5], Welk et al. [6] found that “personality characteristics that support reserved behavior, low impulsivity, and distrust decreased phishing susceptibility within an email-based decision making task” (p. 13). The increased impulsivity that is often characteristic of younger adults has been proposed as another mechanism governing the increased phishing susceptibility reported in this cohort. However, it should be noted that some studies have failed to demonstrate increased susceptibility in the 18–25 age group compared to older groups; for example, Zielinska et al. [7] found no difference between young people (age 18–25) and other people (age ≥ 26) in phishing susceptibility. Nevertheless, no studies of phishing susceptibility to date have focused specifically on older adults (i.e., age 50+).

Similarly, few studies have focused on examining the value of cognitive abilities in predicting phishing susceptibility. Vishwanath and colleagues have reported a series of studies showing that the degree to which participants pay attention to and recall the details of a phishing email are related to whether or not victimization occurs [8–10]. Hong and colleagues [11] measured working memory, crystallized intelligence, spatial skills, and sustained attention in their sample of younger adults, but did not report data related to the association between these cognitive abilities and performance on a phishing detection task. Finally, Purkait et al. [12] found positive associations between phishing detection and experimental measures of attentional vigilance and short-term memory. There are no other studies known to have evaluated the predictive validity of cognitive measures for phishing outcomes.

In the field of clinical neuropsychology, there is a large corpus of research that has focused on the use of cognitive tests to predict real-world functional outcomes, including instrumental activities of daily living (IADLs) such as financial management, driving, and medication adherence [13]. These ecological validity studies are commonly pursued in an older adult population due to the increased risk of aging-related cognitive disorders and their associated functional disabilities [14]. Given the now prevalent use of the Internet and smartphone applications for the organization and management of daily tasks such as shopping, banking, and social networking, predictions about real-world functional outcomes ought to include measures of technology use and its associated risks. The use of technology such as online shopping and banking may represent an enhanced activity of daily living (EADL; [15]) that serves to improve quality of life, whereas one’s ability to resist phishing and other scams may be best conceptualized as an IADL. Despite the threats posed by phishing, one’s ability to recognize and avoid phishing attacks has not been a common focus of research from the perspective of clinical neuropsychology and ecological validity. The current study was designed to bridge this gap in the literature, with the goals of adding a clinical and ecological perspective to the existing security-focused literature on phishing as well as taking the first steps toward promoting the assessment and prediction of phishing susceptibility across age groups.

Current older adult cohorts may be particularly vulnerable to phishing attempts. In fact, older adults appear to be disproportionately targeted for many types of fraud, including financial exploitation [16] as a result of their greater potential for wealth; decreased familiarity, comfort, and self-efficacy using computers; less experience with web browsing [17–20]; and increased risk for age-related cognitive decline [21]. Even in the absence of mild cognitive impairment (MCI) or dementia, older adults may exhibit impairments in decision-making [22]. For example, age-related decline in executive functioning is associated with a decline in functional capacities, such as financial management [23–25]. Research has consistently demonstrated that tests of executive functioning can be useful for the prediction of IADLs, especially in older adults with cognitive impairment due to neurodegenerative diseases [26–29]. For example, the Iowa Gambling Task (IGT) measures decision-making in ambiguous situations, which is believed to be important for performing IADLs [30]. Other research has demonstrated associations between IADL functioning and scores on tests of judgment and decision-making [31,32], maze navigation [33], abstract reasoning and concept formation [34], and verbal fluency [35].

Avoiding online fraud is becoming an important component of one’s capacity to function independently. This is especially true for emerging cohorts of older adults who rely heavily on electronic storage of personal information, as through online banking applications, electronic medical records, and cloud-based storage systems. Clinicians working with older adults with known or suspected cognitive impairment may wish to assess this capacity to predict susceptibility and intervene when appropriate [36]. Recent advances in the conceptualization and assessment of capacity have cautioned against treating this construct as a single entity; instead, it is recommended that capacity be conceptualized as applying to a particular function (e.g., capacity to make financial decisions) [37]. Under this approach, the prediction of phishing susceptibility represents a specific type of capacity that may wish to be addressed to protect vulnerable individuals from financial abuse and identity theft. As current cohorts of digital natives age into older adulthood, assessing the capacity to detect phishing attempts will undoubtedly become an essential focus of clinical assessment.

The current study sought to identify and compare the relationship between cognitive functioning—in particular, executive functioning ability—and phishing susceptibility in younger and older adult samples. Because older adults are generally less familiar with computers [18–20] and are more likely to experience declines in abilities such as judgment and decision-making [21], we hypothesized that the older adult cohort would be more vulnerable to phishing attempts than the younger adult cohort. In addition, we hypothesized that commonly used tests of executive functioning, especially those measuring judgment and decision-making, would be predictive of phishing susceptibility.

Materials and methods

Procedure

The study received human subjects approval from the University of Colorado Colorado Springs Institutional Review Board. A trained graduate or undergraduate research assistant conducted all data collection in Psychology lab space on campus. At the outset of the study, participants provided written informed consent, answered a demographics questionnaire, and were administered a battery of executive functioning tests (described below). Participants were then asked to perform a series of web-browsing activities, including evaluation of websites’ login notifications, and to then rate their stylistic preferences by site. Participants were not aware that these activities would involve phishing attempts to obtain their login credentials.

Immediately before the web-browsing tasks, the examiner emphasized the voluntary nature of the study to participants and provided reassurance that the tasks could be discontinued at any time without penalty. After the examiner left the room, participants visited four different websites, two of which were legitimate sites and two of which were phishing sites that we created. Of the two total phishing sites, one login utilized traditional methods (i.e., entering one’s username and password for that site) and one utilized single sign-on (SSO) methods (i.e., logging into a website such as Sears.com using an identity provider account, such as Facebook or Google). The order of the legitimate and phishing websites, and the traditional vs. SSO websites, was counterbalanced. Participants’ interactions with these sites were monitored remotely to determine whether and where login attempts were made. No sensitive login information was recorded, transmitted, or stored when participants interacted with the phishing websites.

Phishing websites were hosted on university servers and designed to accurately mimic the target websites, so that activities such as clicking on links, searching, and logging in would behave identically to target websites. URLs of the phishing websites were created with the explicit purpose of alerting participants to the forgery site (e.g., http://www.amazon.jigdee.com instead of http://www.amazon.com). Figs 1 and 2 present examples of phishing websites used in this study. Fig 1 shows a screenshot of the phishing testbed’s replica of the Amazon.com homepage and Fig 2 shows a dynamically generated deep-level replica of the Amazon.com website. A more complete description of the phishing testbed has been published previously [1]. The phishing testbed can also be accessed through our project website, hosted at http://inside.mines.edu/~chuanyue/NSF/NSF1624149.html.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Example of a Phishing Home Page Encountered by Study Participants.

A dynamically generated homepage on the Amazon phishing website (it presents the same content that is actually displayed on the legitimate Amazon website).

https://doi.org/10.1371/journal.pone.0171620.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Example of a Deep-Level Phishing Web Page Encountered by Study Participants.

A dynamically generated deep-level webpage on the Amazon phishing website (any level of webpages can be generated and presented to users in real-time).

https://doi.org/10.1371/journal.pone.0171620.g002

After participants completed the web-browsing task, the examiner returned to the room and administered several questionnaires about the web-browsing activity without disclosing the phishing attempts. The questionnaires asked whether the participants noticed anything suspicious about the websites, whether their approach to the web-browsing task would have differed if they were at home rather than in a lab, whether they had prior knowledge of phishing, and whether they had previously been victimized by phishing. A copy of the questionnaire is included as S1 Appendix. After the completion of these questionnaires, participants were debriefed about the true purpose of the study and compensated for their time.

Results

Participant demographic data, cognitive test results, and answers to phishing questionnaires are presented in Table 1. On the IGT, three participants admitted and two demonstrated (through perfect or nearly perfect scores) prior knowledge of the task; therefore, we omitted IGT scores for these five participants. As described above, only 12 were not vulnerable to the experimental phishing attacks. All 12 participants who chose not to enter their login information to the phishing sites were older adults. Of these 12 older adult participants, 10 reported prior knowledge of phishing and 4 reported being victims of phishing in the past; their mean age was 69.17 (SD = 9.4) and their mean years of education was 16.5 (SD = 2.11). Compared to the number of participants who were victimized by the experimental phishing attacks in the current study, there was more variability about reported suspiciousness, with 68 participants reporting something suspicious about one or more of the websites.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Participant Demographics, Cognitive Test Scores, and Questionnaire Responses.

https://doi.org/10.1371/journal.pone.0171620.t001

Overall, the younger adult group contained a greater proportion of individuals of non-Caucasian race, χ² (df = 1) = 11.89, p = 0.001, and Hispanic/Latino/Spanish ethnicity, χ² (df = 1) = 17.53, p < 0.001. Older adults had more years of education, t (df = 168.32) = -9.10, p < 0.001. The younger adult group performed better on NAB Mazes, t (df = 183.67) = 7.35, p < 0.001, and NAB Categories, t (df = 190.82) = 2.58, p = 0.011, while the older adult group demonstrated better overall executive functioning on the demographically-corrected NAB Executive Functions Index score, t (df = 179.58) = -6.73, p < 0.001, and produced higher scores on the NAB Word Generation test, t (df = 188.19) = -2.99, p = 0.003. More older adults than younger adults reported knowledge of phishing, χ² (df = 1) = 7.56, p = 0.006. They were also more likely to have been victimized by phishing in the past, χ² (df = 1) = 13.46, p < 0.001, and more likely to report that their web-browsing behaviors were different in the laboratory setting versus at home, χ² (df = 1) = 16.33, p < 0.001. No other group differences were observed on any of the variables reported in Table 1.

The results of the logistic regression demonstrated that the predictor variables contributed to a 22.7% advantage for the hypothesized model over an intercept-only model (Nagelkerke R²), χ² (df = 25) = 34.85, p = 0.092. Detailed results of the logistic regression model are presented in Table 2. As can be seen from Table 2, there were two significant main effects (years of education and NAB Mazes raw score); in addition, prior knowledge of phishing and scores on NAB Mazes were significantly moderated by Age Group. These effects are depicted graphically in Fig 3. The model correctly classified participants’ phishing suspiciousness with 73.1% (95% CI [66.0, 80.3]) accuracy.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Predictors of Phishing Susceptibility Derived from Logistic Regression.

Main effect of years of education on reported phishing suspiciousness (A). Age group interaction effects with prior phishing knowledge (B) and NAB Mazes score (C). Error bars and shaded bands represent 95% confidence intervals.

https://doi.org/10.1371/journal.pone.0171620.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results of the Logistic Regression Analysis Predicting Phishing Suspiciousness.

https://doi.org/10.1371/journal.pone.0171620.t002

Discussion

This study sought to determine whether older adults are more susceptible to computer-based phishing attacks and whether cognitive test scores can predict phishing susceptibility. Because very few participants declined to log in to the experimental phishing websites, we used reported suspiciousness as our dependent variable. Overall, the logistic regression model was able to achieve a classification accuracy of 73.1% (95% CI [66.0, 80.3]) in predicting whether or not participants would be suspicious of the experimental phishing attacks. The results presented here failed to support our hypothesis that older adults would be more susceptible to phishing than younger adults. Although the results indicate that being an older adult was associated with 3.69 times greater odds of being suspicious relative to younger adults, this parameter was estimated too imprecisely to interpret with confidence (Table 2). There was partial support for our hypothesis that executive functioning tests would be predictive of phishing susceptibility. One test, NAB Mazes, was predictive of phishing susceptibility. Other executive functioning test scores were not important predictors of phishing susceptibility. More specific findings are discussed below.

Compared to younger adults, older adults were more likely to have knowledge of phishing and more likely to have been victimized by phishing in the past. In addition, older adults appeared to have been less trusting of the research setting, as they were more likely to report using different browsing behaviors in the lab versus at home, and all 12 participants who resisted the phishing attacks were older adults. Older adults’ suspiciousness varied markedly depending on whether or not they possessed prior knowledge of phishing. When broken down by prior knowledge (Fig 3B), the most suspicious of the subgroups was the older adults with prior knowledge of phishing, whereas the least suspicious subgroup was the older adults without prior knowledge of phishing. In contrast, younger adults without prior knowledge of phishing reported more suspiciousness than those with prior knowledge. Because all of the younger adults—whether suspicious or not—entered their login information into phishing sites, this suggests that younger adults may have been too trusting of the laboratory environment. Consistent with the findings of Metzger and colleagues [4], our younger adult participants, all of whom were students at the university where this study was performed, trusted the experimental setting and the website content. For older adults, personal experience (having been victimized by phishing) and associated prior knowledge may have offered useful protection against fraud. For younger adults, experience and information may have had the opposite effect by creating implicit trust in the research setting, which likely includes trust about IT security. Also as expected, people with more years of education tended to be more suspicious of the phishing attempts (Table 2), which is consistent with previous research suggesting that years of education moderates the influence of age group on susceptibility to phishing [5]. It is likely that educational background is related to factors associated with learning and cognition, but it may also serve as a proxy for experience and familiarity with computers, as has been previously reported [17]. Unlike other studies (e.g., [5,11,45]), we found no differences in phishing suspiciousness between men and women.

Only one cognitive test, NAB Mazes, was strongly associated with phishing suspiciousness. The interaction between this variable and age group produced an unexpected pattern, however. In older adults, better test scores were associated with more suspiciousness, as expected. But in younger adults, better test scores were associated with less suspiciousness (Fig 3). The best explanation for this pattern may be that the abilities measured by NAB Mazes (processing speed, planning) serve different roles for younger and older adults: for younger adults, these executive functions contribute to efficient, automatic processing, especially in situations that are perceived as trustworthy (e.g., using campus computers); for older adults, these executive functions may contribute to more deliberative, self-protecting behaviors, especially when they are knowledgeable about risks (e.g., phishing and web browsing) [46]. Similarly, it may be the case that the NAB Mazes test measures slightly different abilities in older versus younger adults. For example, in older adults, the NAB Mazes test may draw more heavily on skills such as visual scanning and spatial navigation; older adults who are better at these skills may be more adept at noticing webpage anomalies that increase suspiciousness. In younger adults, in contrast, the NAB Mazes test may draw more heavily on information processing speed and mental efficiency; younger adults who are better at these skills may have used cognitive shortcuts, such as the familiarity heuristic, which could have suppressed detection of webpage anomalies and reduced suspiciousness [47]. The possibility that the NAB Mazes test draws on different cognitive abilities in younger and older adults suggests measurement invariance, which should be a focus of future research on this test and the rest of the NAB Executive Functions module.

In contrast to the effects discussed above, statistical significance was not reached for several predictor variables, including past phishing susceptibility, lab versus home browsing approach, sex, race, ethnicity, three tests from the NAB Executive Functions module, and the IGT. These results suggest that most demographic variables, including age cohort, are insufficient predictors of phishing victimization. Although race and ethnicity were associated with some of the larger effect sizes (Table 2), these parameters were estimated extremely imprecisely due to the limited representation of ethnic and racial minorities in the current sample, especially among older adults (Table 1). Examining phishing susceptibility differences across ethnic and racial groups is important, as evidence suggests that these groups report different Internet usage patterns [17]. These results also suggest that some tests of executive functioning are limited in their utility to identify potential victims of phishing and possibly other forms of exploitation. It should be pointed out, however, that both the main and interaction effects of the IGT were very close to reaching statistical significance (Table 2). Nevertheless, these parameters were estimated with good precision, suggesting that the ability of IGT scores to predict phishing suspiciousness is of limited clinical significance regardless of statistical significance. Taken together, the predictor variables used here were able to predict—with an accuracy of 73.1% (95% CI [66.0, 80.3])—whether or not an individual participant would report suspiciousness. However, because the logistic regression model only produced a 22.7% reduction in error variance compared to the null model, it is likely that there are additional unmeasured variables that could greatly contribute to the prediction of phishing suspiciousness and susceptibility. The current results are best interpreted as the first step in identifying ecologically valid neuropsychological approaches for identifying phishing susceptibility and are not suited for classification purposes at the individual level. Because the current results have not been cross-validated or validated in out-of-set data, additional studies are needed before using neuropsychological tests to classify individuals based on predicted phishing susceptibility.

Neuropsychological assessment instruments have been described as having limited ecological validity [48,49]. The results of the current study also suggest considerable room for improvement in the use of neuropsychological tests for the prediction of real-world functional outcomes. Neuropsychologists are often asked to evaluate individuals to render judgments about functional capacities such as financial management and ability to avoid exploitation [50]. Because phishing represents an emerging threat to individuals lacking in those capacities, it is necessary for neuropsychologists to have access to clinical tools that are valid for making such judgments. These results highlight the importance of continued development of assessment techniques that can jointly measure cognitive and functional abilities and that are predictive of real-world outcomes. Cognitive abilities have been shown to play an important role in older adults’ ability to learn and adapt to advances in computing technology [17], which suggests that the development of novel neuropsychological tests that can better predict computer-related EADLs and IADLs is of considerable importance.

The current study is limited by the salient influence of the laboratory environment. Out of the 29 individuals who reported prior knowledge of phishing, prior victimization by phishing, and suspiciousness of the websites visited, 26 (89.7%) entered their login information into the spoofed web pages. We attempted to minimize the influence of the laboratory setting by emphasizing that participants could discontinue any task without penalty, by having examiners leave the room while participants browsed the Internet, and by asking participants to enter their private login data into real (or seemingly real) websites, some of which may have been used to store sensitive data (e.g., credit cards). This aspect of the study represents a strength relative to other research, which often asks participants to indicate how they might respond to a role-play scenario (e.g., [5,45]) or to simply identify whether a website is legitimate (e.g., [11]). However, it appears as though the powerful influences of trust, obedience, and authoritarianism affected our participants [51]. Past research has implicated trust as an important factor influencing phishing susceptibility [52], and current methods used by websites to demonstrate their own trustworthiness are of marginal effectiveness [53]. However, because so few participants were able to avoid becoming susceptible to phishing, our resulting shift in focus from susceptibility to suspiciousness may be capturing other non-specific personality traits, such as skepticism, paranoia, or anxiety, that were not directly related to the phishing attempts [17,54]. Further, our study may have been improved through the inclusion of additional cognitive predictors of phishing suspiciousness, such as more basic aspects of attention, which have been proposed as a mechanism for phishing susceptibility in the past (e.g., [9]). However, given the fundamental role of attention skills in completing more complex tasks of executive functioning, it is likely that individual variability in attention skills was captured indirectly through the battery of executive functioning tests used here [55].

This study is also limited by the fact that our older adult sample was self-selected to participate in this study and is therefore unlikely to have good external validity for the entire population of Internet-using older adults, or older adults who use the internet less frequently than the minimum of once per month needed to meet our study’s inclusion criteria. Our older adult sample was highly educated, in possession of higher than average executive functioning abilities, and lacking in racial and ethnic diversity. Approximately 88% of the older adult sample reported knowledge of phishing (even more than the 71% reported in younger adults), which may be higher than in the general population of Internet-using older adults. Because almost 53% of the older adults in our sample reported past victimization by phishing attacks, it is possible that this high rate of phishing knowledge in older adults is caused by high rates of past victimization. It should also be mentioned that our study’s operationalization of younger and older age cohorts may affect the results. For example, some studies have used age 65 as the cutoff for defining "older" adults. Similarly, two individuals older than 40 were included in our younger adult group. Additional limitations include the fact that some parameters were not estimated with desirable precision (Table 2), and the fact that we did not ask participants about their familiarity with the target websites. Although we ensured that every phishing webpage was based on a website at which participants owned a real user account, participants may have differed in the frequency with which they used these sites. Finally, we excluded participants with cognitive impairment, so our results cannot speak to the risk of phishing susceptibility in older adults with MCI or dementia. Future research should evaluate susceptibility to phishing in individuals with MCI, as this group will continue to grow in size with the emerging baby-boomer cohort, many of whom are avid or frequent technology users.

Many phishing attempts are predicated upon trust and not necessarily technological sophistication. As such, the current results suggest that both older and younger adults can be protected from phishing attempts through methods that provoke discomfort and suspiciousness at the expense of automatic processing. Educational initiatives that increase phishing awareness while also directly linking phishing threats to concrete negative consequences may provide the necessary skills to avoid being phished. Past research supports the use of education as a strategy for intervention and means for decreasing vulnerability; similar initiatives have shown promise in increasing knowledge and suspiciousness about phishing in both older and younger adults. A training intervention employed by Sheng and colleagues showed that phishing education could help reduce users’ tendency to become victims [5]. Carpenter et al. reported that warnings are effective at reducing online personal information disclosure [56]. Kumaraguru et al. developed an email-based anti-phishing education system, PhishGuru, and an online game, Anti-Phishing Phil, that uses learning science instructional principles to teach users how to use cues in URLs to avoid falling into phishing attacks. Their results suggested that an education approach facilitated the identification of phishing emails and websites [57]. In addition to the above, future research should attempt to investigate phishing susceptibility outside of a laboratory environment where the presence of implicit trust, especially on the part of students, may be influencing behaviors and decreasing the external validity of the results.

In conclusion, the results of this study reveal that older and younger adults both become suspicious about phishing at approximately equal rates, a previously understudied effect within older adults, but that the factors driving phishing susceptibility have similarities and differences between these two age cohorts. For both groups, more years of education appears to be protective against phishing susceptibility. In older adults, lack of knowledge about phishing and lower scores on NAB Mazes contributed to reduced suspiciousness and, presumably, greater susceptibility to phishing. These results also suggest an increased need for development of ecologically valid assessment instruments to identify individuals who are most vulnerable to phishing and similar exploits [49,50]. As access to technology continues to increase worldwide, the potential for victimization by this technology will also increase. Neuropsychologists and other clinicians who wish to identify individuals lacking the capacity to avoid this victimization are in need of more ecologically valid tools than those currently available.

Supporting information

Acknowledgments

The authors would like to thank Caleigh Findley and Josh Beaston for assistance with data collection and entry.