Personality influences decision making and ethical considerations. Its influence on the occurrence of research misbehavior has never been studied. This study aims to determine the association between personality traits and self-reported questionable research practices and research misconduct. We hypothesized that narcissistic, Machiavellianistic and psychopathic traits as well as self-esteem are associated with research misbehavior.
Included in this cross-sectional study design were 535 Dutch biomedical scientists (response rate 65%) from all hierarchical layers of 4 university medical centers in the Netherlands. We used validated personality questionnaires such as the Dark Triad (narcissism, psychopathy, and Machiavellianism), Rosenberg's Self-Esteem Scale, the Publication Pressure Questionnaire (PPQ), and also demographic and job-specific characteristics to investigate the association of personality traits with a composite research misbehavior severity score.
Machiavellianism was positively associated (beta 1.28, CI 1.06–1.53) with self-reported research misbehavior, while narcissism, psychopathy and self-esteem were not. Exploratory analysis revealed that narcissism and research misconduct were more severe among persons in higher academic ranks (i.e., professors) (p<0.01 and p<0.001, respectively), and self-esteem scores and publication pressure were lower (p<0.001 and p<0.01, respectively) as compared to postgraduate PhD fellows.
Machiavellianism may be a risk factor for research misbehaviour. Narcissism and research misbehaviour were more prevalent among biomedical scientists in higher academic positions. These results suggest that personality has an impact on research behavior and should be taken into account in fostering responsible conduct of research.
Citation: Tijdink JK, Bouter LM, Veldkamp CLS, van de Ven PM, Wicherts JM, Smulders YM (2016) Personality Traits Are Associated with Research Misbehavior in Dutch Scientists: A Cross-Sectional Study. PLoS ONE 11(9): e0163251. https://doi.org/10.1371/journal.pone.0163251
Editor: Pablo Dorta-González, Universidad de las Palmas de Gran Canaria, SPAIN
Received: February 11, 2016; Accepted: September 5, 2016; Published: September 29, 2016
Copyright: © 2016 Tijdink et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are available within the paper and its Supporting Information files.
Funding: This work was supported by the ZonMw, the Netherlands Organisation for Health Research and Development. ZonMw had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Little is known about the psychology and personality of biomedical scientists. We like to think that scientists are open, eager to collaborate, self-confident, curious and creative . However, there’s anecdotal evidence that this is not universally so . Success in science requires publishing in high Impact Factor journals and acquiring research grants, all in a hypercompetitive climate. This may tempt scientists to rush into print, cut corners, exaggerate findings and overstate the importance of their research . In addition, the so-called Dark Triad of personality, referring to Machiavellianism, psychopathy, and narcissism has been found to predict behaviors like abusive supervision and employee theft . Self-esteem appears to be negatively associated with cheating, at least among students [5,6].
The influence of personality traits on scientific practice is understudied. However, a negative relationship between narcissism and cynicism on the one hand, and aspects of ethical decision-making in future scientists on the other has been demonstrated consistently.  As these traits have also been found to predict cheating in scholastic, financial, and work settings, it is conceivable that they are associated with an increased likelihood of engaging in research misbehavior (fraud and Questionable Research Practices or QRPs). Research misbehavior has received substantial attention in the last decade . There is increasing evidence that research misbehaviors, specifically QRPs, are relatively common, seriously impact the scientific process, and compromises the validity of scientific results .
Preliminary data indeed suggest that specific personality characteristics are indeed associated with scientific misbehavior. Psychiatrist Kornfeld distinguished different categories of fraudulent scientists, suggesting that certain personality profiles are more common among fraudsters, and found a relationship between publication pressure and research misbehavior. This qualitative evidence was supported by quantitative results .
In this study we aimed to provide more insight into the psychology of research misbehavior. We hypothesized that some scientists are more susceptible to committing research misbehavior than others. Specifically, we postulated that high self-esteem, Machiavellianism (a person’s tendency to be unemotional, detached from conventional morality and hence inclined to deceive and manipulate others, to focus on unmitigated achievement, and to give high priority to their own performance), narcissistic traits (a person’s tendency to pursue gratification from vanity or egotistic admiration, and to obtain recognition of their own attributes), and psychopathic traits (enduring antisocial behavior, diminished empathy and remorse, and disinhibited or bold behavior), are associated with research misbehavior. The Dark Triad questionnaire with the 3 specific character traits (narcissism, Machiavellianism and psychopathy) was chosen to analyze the influence of these specific traits on research behavior. We hypothesized that narcissism, psychopathy and Machiavellianism could all play a role in an individual’s susceptibility to commit research misbehavior. If you are ‘bad’ in terms of dark personality traits, you are more likely to engage in bad behavior such as research misbehavior.
We hypothesized that publication pressure and academic position were mediating and possibly moderate/modify the effect of personality traits on research misbehavior; the higher the scores on the Dark Triad, the higher the scores on the research misbehavior sum score. More profound personality traits may, however, lead to more publication pressure, which in turn may result in more misbehavior (mediation). More profound personality traits may also make it more likely that someone occupies a higher academic rank, which may be related to more misbehavior (mediation).
Alternatively, more pronounced personality traits may be more strongly associated with research misbehavior in case of high academic position (moderation) due to researchers in top positions being more focused towards success in research. More pronounced personality traits may also be more strongly associated with research misbehavior in case of high publication pressure (moderation).
Conversely, we hypothesized that self-esteem could be beneficial for responsible conduct of research; if you feel confident in your job you have fewer reasons to misbehave. We therefore included self-esteem as a character trait in our predefined analysis model. (Fig 1)
In this study we furthermore wanted to determine whether publication pressure and academic position influence the relation between personality traits and research misbehavior.
The medical Ethics Review Committee of VU University reviewed the study protocol and decided that the Medical Research Involving Human Subjects Act (WMO) does not apply to our study and that an official approval of our study by this committee is therefore not required.
Participant selection and procedure
All 1833 biomedical scientists from 4 university medical centers in the Netherlands were invited by email to participate in the survey. Scientists were eligible if they were sufficiently proficient in English (as the questionnaires were in English), were scientifically active (studying research misbehaviors is relevant only for active scientists), and provided consent by following the link to the online questionnaire.
The research councils (the departments in charge of the policy of research-related matters within an institution) were essential in providing the e-mail addresses of potential participants in the 4 participating medical centers and inviting the participants. Two preclinical, three clinical (internal medicine, surgery, and psychiatry) and two supportive departments (i.e. methodology, statistics) were selected. To create heterogeneity among the participants we also included the most and least publishing departments per fte. The invitation e-mail (See Text A in S1 File) explained the objective of the study, using neutral terms such as achievement, motivation, personality and scientific success, and provided a link to an anonymous online questionnaire (Text B in S1 File) on a protected website. The e-mail also included the name and a link to the e-mail address of the lead investigator (JT) to opt out of participation.
Scientists who did not respond within 2 weeks were sent two reminders. After the second reminder we asked invited participants who still did not respond to fill out a 15 second ultra-brief questionnaire to determine their reason for declining participation.
Psychometric information on questionnaires
The survey consisted of seven validated questionnaires (see Text B of S1 File) and a set of demographic questions. We used the Dark Triad  to measure Machiavellianism, narcissism and psychopathy. The Dark Triad consists of 27 items (9 items per subscale) on which participants indicate agreement on a five-point Likert scale, ranging from ‘strongly disagree’ to ‘strongly agree’. This scale is reliable (Cronbach’s α = 0.77–0.80). We used the Rosenberg Self-Esteem scale  to measure self-esteem. It consists of 10 items for which agreement is indicated on a four-point Likert scale, with answer options ranging from ‘strongly agree’ to ‘strongly disagree’. The Rosenberg Self-Esteem scale is also reliable (α = 0.77–0.88). To measure publication pressure we used the validated 14-item publication pressure questionnaire (PPQ) . The biomedical scientists indicated to what extent they agree with each of the 14 statements, answer options ranging from ‘strongly disagree’ to ‘strongly agree’. Reliability of this scale is moderate (Lambda-2 reliability = 0.69).
For the primary outcome we constructed a research misbehavior (fraud and QRPs) severity questionnaire, which yielded a composite research misbehavior severity score. This questionnaire was based on questionnaires used by other research misbehavior investigators (see following cited references), with additional items gathered from different landmark publications on research misbehavior [12,14,21]. Based on these previous reports, we selected items deemed most suitable for biomedical scientists. The questionnaire consisted of 22 different types of research misbehaviors. Respondents were asked to report to what extent they had committed specified types of research misbehaviors the past three years. Answers were given on a 5-point scale (never, once, occasionally, frequently, often). To construct a Research Misbehavior Severity Score (RMSS), the scores of the items were dichotomously translated (behavior yes/no) and items were assigned different weights. The most severe type of misbehavior (based on the definition of fraud, i.e. fabrication, falsification and plagiarism) was 3 points. The other item scores were based on consensus in the research group and were assigned 1 (for moderate) and 2 (for severe) misbehavior. Positive answers (committing the behavior at least ‘once’ in the past 3 years) to the most severe misbehavior questions (items 1, 2, 8, 9, 12, 15 and 19) were assigned three points, positive answers of the severe research misbehavior questions were assigned two points (items 4, 7, 10, 14, 16, 18 and 20) and positive answers to the moderate research misbehavior questions were assigned one point (items 3, 5, 6, 11, 13, 17, 21 and 22). Scores were added up to calculate the composite research misbehavior severity score (RMSS) (maximum range: 0–43).
Survey characteristics and outcomes
In Fig 1 we present the predefined analysis model. We postulated that 4 traits (narcissism, Machiavellianism, psychopathy, and self-esteem) are related to the outcome variable (RMSS). We hypothesized that this relation could be modified/moderated or, alternatively, mediated by publication pressure or academic position. To measure publication pressure we used the validated publication pressure questionnaire (PPQ) . Academic position was operationalized by self-reported rank (postgraduate PhD-fellows, postdoctoral researchers, assistant professors, associate professors, and full professors).
Respondents provided demographic information on gender, age, academic position, type of biomedical specialty, number of years working as a scientist, percentage of time spent on research and (if known) Hirsch-index.
Associations between personality traits and the outcome measure (RMSS) were first tested by means of linear regression analyses using a separate regression model for each personality trait. In order to satisfy the normality assumption for the residuals the outcome variable (RMSS) was transformed and log(1+RMSS) was used as the dependent variable. Personality traits were standardized by subtracting their sample mean and dividing by the standard deviation. The exponentiated regression coefficients refer to the relative increase of geometric means of RMSS+1 associated with an increase of 1 standard deviation on the personality trait scale. Effect modification/moderation was assessed by including the candidate modifier (publication pressure or academic position) in the regression model together with its interaction with the personality trait at issue. A variable was considered to be an effect modifier/moderator if it showed a significant interaction (p<0.05) with the personality trait. To assess whether publication pressure and academic position mediated the relationship between personality traits and scientific misbehavior we divided the total effect into a direct effect of the personality trait and an indirect effect via the candidate mediator. A variable was considered a mediator if both the total effect and the indirect effect were significant. In the mediation analyses for academic position we used a probit-link to relate academic position to the personality trait. To assess mediation we used a path model which regressed the mediator on the personality trait and then regressed the (transformed) RMSS score on the mediator. The coefficients of these two regression equations quantified the indirect effect from personality to RMSS score via the mediator. In addition, the path models directly regressed the RMSS score on the personality trait. The total effect was estimated in a model where RMSS score was regressed on the personality trait.
To assure validity of the analyses residuals from the models were checked for normality by means of visual inspection of histograms and QQ-plots.
In a secondary analysis, ANOVA tests were used to compare mean scores for personality traits, self-reported research misbehavior and publication pressure between respondents with different academic positions. In case of a significant overall difference, pairs of groups were compared in post hoc analyses using a Bonferroni correction.
To analyze the correlates of ethical behavior we performed a secondary post hoc analysis where we used logistic regression to test for associations of personality traits, PPQ and academic position with an RMSS of 0 (vs. RMSS > 0).
Regression analyses and ANOVA tests were performed in SPSS version 22. Analyses of mediating variables were performed in M-Plus version 7 using the product of coefficients methods.
In total, we used 1833 email addresses. Of these, 182 bounced because the email addresses no longer existed or were inactive. Of the remaining 1651, 1098 invitees opened the email. Among them, 715 started the survey (response rate 65%) and 537 completed the survey (completion rate 49%). We excluded 2 participants who declared they were not scientifically active. The demographic data of the complete responders are summarized in Table 1.
Research Misbehavior Severity Score (RMSS)
The items of the research misbehavior questionnaire are tabulated in Table 2. The results of item 11 serve as an example of which items are included in the severity score: Almost 60% of the participants admitted having added authors who had made no significant contribution to the author list at least once. Fabrication and falsification were less common, although almost 5% admitted they had selectively deleted data to confirm a hypothesis. The median RMSS score in the sample was 3 (range 0–39).
Machiavellianism predicts research misbehavior
Tables 3 and 4 present relations of personality traits with the research misbehavior severity score. Higher scores on Machiavellianism were significantly associated with higher research misbehavior severity scores. There was a trend (0.05≤p≤0.10) for narcissism and psychopathy having a similar association with research misbehavior. In multivariate analysis that included all three personality traits, only Machiavellianism was significantly associated with the RMSS (data not shown).
Exponentiated regression coefficients, 95% confidence intervals and p-values for total, direct and indirect effects associated with 1 standard deviation increase in the personality trait.
Table 4 shows the mean values of the Dark Triad: (Machiavellianism, narcissism, and psychopathy), self-esteem and PPQ sum score and comparison of means between groups using an ANOVA. Kruskal-Wallis test used to compare RMSS sum scores between groups.
Publication pressure as measured with the PPQ and academic position failed to significantly modify/moderate the relationship between the personality traits and research misbehavior severity: neither showed significant interaction with any of the personality traits (see S1 Table).
Variance in misbehavior explained by personality traits was only small, with 1.6% of variance explained by Machiavellianism (R2 = 0.016). Models for moderation/effect modification by academic position had R2 values ranging between 0.042 and 0.053. Models for moderation/effect modification by publication pressure had R2 values ranging between 0.031 and 0.038. Similar R2 values were found for models assessing mediation (see S3 Table).
Role of publication pressure and academic position
We considered whether publication pressure or academic position mediated the relation between the Dark Triad and RMSS scores. The results of the mediation analyses are displayed in Table 3. In all mediation analyses PPQ and higher academic position were both found to be positively associated with RMSS scores (all p <0.001). R2 values for mediation models obtained from Mplus are included in S3 Table.
Personality traits and demographic factors
Table 4 provides the means of the measured personality traits, PPQ and primary outcome (RMSS) stratified for academic position. Mean scores on narcissism and self-esteem were different between academic positions. Post-hoc tests revealed that postdocs, assistant and associate professors, and professors all scored higher on narcissism (Bonferroni corrected p < 0.05 and p <0.01 respectively) and lower on self-esteem than postgraduate PhD-fellows (Bonferroni corrected p < 0.05 and p < 0.01 respectively). No significant differences in personality trait scores were found between postdocs and professors. Furthermore, the RMSS scores differed between the academic position groups and post-hoc tests found RMSS scores to be higher for professors and postdocs as compared to postgraduate PhD-fellows (Bonferroni corrected p < 0.01 and p<0.001 respectively). No significant difference in RMSS was found between postdocs and professors. Finally, the publication pressure questionnaire score was found to differ between respondents with different academic positions. Post-hoc tests revealed a significant difference in PPQ scores between postgraduate PhD-fellows and full professors (Bonferroni corrected p < 0.05), with professors reporting lower publication pressure.
In the secondary analyses of the correlates of ethical behavior we looked at factors associated with an RMSS of 0 and we found that RMSS = 0 is more likely when PPQ is lower (OR 0.94 95% CI 0.91–0.97). There was a trend for academic position with higher academic position associated with RMSS > 0. (overall p = 0.053; postdoc compared with PhD-student OR 0.68 95% CI 0.43–1.09 and professor compared to PhD OR 0.39 95% CI 0.16–0.96). See S2 Table.
To our knowledge, this is the first study to investigate personality traits in relation to research misbehavior among biomedical scientists. Our results suggest that Machiavellianism is associated with self-reported research misbehavior. Secondary analyses reveal that narcissismand research misbehavior are positively associated with higher academic rank. Furthermore, unlike lower academic positions, higher academic positions are associated with both lower publication pressure and lower self-esteem.
Although evidence from earlier research is lacking, our results seem largely in agreement with the qualitative narrative analysis of Kornfeld , who gathered case histories of 146 individuals found guilty of research misbehavior and categorized them based on different psychological traits. According to Kornfeld, scientific fraud is the product of a combination of individual personality traits and an intense fear of failure, or the lure of academic and/or financial rewards . Furthermore, most subjects indicated an intense pressure to publish as the main reason for their behavior, reasoning that publications boost their career potential and financial rewards. Our study is larger and non-selective and addresses a pre-defined hypothesis with quantitative data. In addition, Kornfeld’s study had no comparison group of non-fraudulent scientists. Also, personality traits in fraudsters who were caught may differ from those who are not (yet) caught.
In our analysis, Machiavellianism was associated with self-reported research misbehavior. Machiavellianism is best described as ‘a person’s tendency to be unemotional, detached from conventional morality and hence to deceive and manipulate others, to focus on unmitigated achievement and give high priority to own performances’. This description intuitively explains that Machiavellianistic scientists more easily engage in research misbehavior. Moreover, the intellectual legacy of Niccolo Machiavelli confirms our findings as well. This is best illustrated by some of his quotes: ‘Whosoever desires constant success must change his conduct with the times’ and ‘One who deceives will always find those who allow themselves to be deceived’. Comparison with existing literature on the general population revealed that the levels of the subscales of the Dark Triad (including Machiavellianism) in our study are comparable with the most recent literature, suggesting that Dark Triad traits are not stronger in biomedical scientists than in the general population.
Our secondary analyses suggest that narcissistic and psychopathic traits are more common in higher academic ranks and that scientists in higher academic ranks have less self-esteem. This could imply that the personality traits narcissism and psychopathy offer some kind of ‘survival benefit’ in academia. Whether these traits in higher academic ranks are desirable and ultimately will ultimately lead to the most ethical research behavior remain to be seen. Publication pressure was also lower in higher academic ranks, which is in line with earlier results.
To our knowledge, this is the first study to investigate personality traits and their relation with major and minor research misbehaviors among biomedical scientists. The response rate was high compared to other online surveys. We chose anonymous questionnaires because of the potential relucatance of respondents to answer the questions on research misbehavior truthfully.[25,26]
The results of our study should, however, be interpreted cautiously. First, internet-based questionnaires can be influenced by response bias, e.g. by attracting participants who are not engaged in research misbehavior and reticence on the part of participants who have. To minimize this risk, we did not convey the purpose of this study in our invitational email and formulated the study goal in neutral terms (i.e. ‘We invite you to participate in this brief questionnaire that addresses personal characteristics of biomedical scientists in relation to science practice’, see Text A in S1 File). Furthermore, we asked participants who did not wish to participate to fill in an ultra-brief non-response survey. Of the 41 biomedical scientists who completed this survey, 56% cited lack of time, 22% cited that they were not involved in research and 10% cited the length of the questionnaire as the main reason for non-response. This together with the high response rate suggests that the findings may be generalizable to the total population of biomedical scientists in the Netherlands (and possibly the rest of the industrialized world).
The research misbehavior severity score should also be interpreted with caution. The 22 items are all self-reported, are prone to different interpretations and were measured at one time point only. The composite score should also be cautiously interpreted as we composed the score based on a self-designed (and thus arbitrary) one-dimensional score sheet and the input of earlier studies.
Taking these limitations into account, one should be hesitant in translating the results directly to the practice of, for example, hiring scientific personnel. However, they might have some implications for those who hire biomedical scientists in academia. One should be conscious about the possible influence of character traits on research behavior. Global assessment of the personality of applicants can certainly have some value in determining the right candidate for the job, but it should never play a central role in appointment procedures. It also can help increase awareness of certain personality traits in researchers and research groups. This awareness can help scientists gain more insight into and control over their behavior when they are in the middle of the scientific process. This insight might help individuals to become more aware of their misbehaviors and can ultimately reduce the risk and incidence of research misbehavior.
Taken together the results suggest that, although narcissism and psychopathy may be associated with research misconduct at first sight Machiavellianism is the personality trait that is most strongly associated with research misbehavior. These results may inform those involved in the recruitment of scientific personnel, as well as people involved in scientific quality and integrity monitoring and those responsible for institutional research integrity policy and for responsible conduct of research training.
Invitational Email (Text A). Questionnaires included in the electronic survey (Text B).
S1 Table. Exponentiated regression coefficients Exp(beta) for linear regression of RMSS on personality traits and results of tests effect modification by PPQ and academic position (to clarify the beta scores: an increase of 1 standard deviation in Machiavellianism is associated with an increase of 12% in the geometric mean of RMSS+1).
S2 Table. Table with subanalysis of RMSS = 0.
- Conceptualization: JT LB YS CV PvdV JW.
- Data curation: JT PvdV.
- Formal analysis: JT YS PvdV.
- Funding acquisition: JT YS.
- Investigation: JT CV.
- Methodology: JT YS LB PvdV.
- Project administration: JT.
- Resources: JT YS LB.
- Software: JT.
- Supervision: YS LB JW.
- Validation: JT CV.
- Visualization: JT PvdV.
- Writing – original draft: JT.
- Writing – review & editing: JT LB CV PvdV JW YS.
- 1. Isohanni M, Kotiranta H (2006) What is a top scientist like? Nord J Psychiatry 60: 189. P1M977865261383M [pii]; pmid:16720508
- 2. Fetzer J (2003) Wolfish behavior…the successful scientist as leader of a pack or a lone wolf. Anal Bioanal Chem 377: 785–786. pmid:12955283
- 3. Alberts B, Kirschner MW, Tilghman S, Varmus H (2014) Rescuing US biomedical research from its systemic flaws. Proc Natl Acad Sci U S A 111: 5773–5777. [pii]; pmid:24733905
- 4. O'Boyle EH, Forsyth DR, Banks GC, McDaniel MA (2012) A meta-analysis of the Dark Triad and work behavior: a social exchange perspective. J Appl Psychol 97: 557–579. 2011-24470-001 [pii]; pmid:22023075
- 5. Aronson E, Mettee DR (1968) Dishonest behavior as a function of differential levels of induced self-esteem. J Pers Soc Psychol 9: 121–127. pmid:5662732
- 6. Ward D.A. (1986) Self-Esteem and Dishonest Behavior Revisited. The Journal of Social Psychology 126: 709–713.
- 7. Antes A.L., Brown R.P., Murphy S.T., Waples E.P., Mumford M.D., Connelly S., et al. (2007) Personality and Ethical Decision-Making in Research: The Role of Perceptions of Self and others. Journal of Empirical Research on Human Research Ethics 2: 15–34. pmid:19385805
- 8. Nathanson C., Paulhus D.L., Williams K.M. (2006) Predictors of a behavioral measure of scholastic cheating: Personality and competence but not demographics. Contemporary Educational Psychology 31: 97–122.
- 9. Jones D.N. (2014) Risk in the face of retribution: Psychopathic individuals persist in financial misbehavior among the Dark Triad. Personality and individual Differences 67: 109–113. http://dx.doi.org/10.1016/j.paid.2014.01.030.
- 10. Cohen TR, Panter AT, Turan N, Morse L, Kim Y (2014) Moral character in the workplace. J Pers Soc Psychol 107: 943–963. 2014-34008-001 [pii]; pmid:25133716
- 11. Wicherts JM (2011) Psychology must learn a lesson from fraud case. Nature 480: 7. pmid:22129686
- 12. Fanelli D (2009) How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One 4: e5738. pmid:19478950
- 13. Kornfeld DS (2012) Perspective: research misconduct: the search for a remedy. Acad Med 87: 877–882. pmid:22622208
- 14. Tijdink JK, Verbeke R, Smulders YM (2014) Publication pressure and scientific misconduct in medical scientists. Journal of Empirical Research and Human Research Ethics 9: 64–71. pmid:25747691
- 15. Christie R. and Geis F. (1970) Studies in Machiavellianism. NY: Academic Press.
- 16. Clarkin J.F., Fonagy P., Gabbard G.O. (2010) Narcissistic personality disorder. In: Psychodynamic Psychotherapy for Personality Disorders: A clinical Handbook. Arlington: American Psychiatric Publishing. pp. 263.
- 17. Skeem JL, Polaschek DL, Patrick CJ, Lilienfeld SO (2011) Psychopathic Personality: Bridging the Gap Between Scientific Evidence and Public Policy. Psychological Science in the Public Interest 12: 95–162. pmid:26167886
- 18. Jones DN, Paulhus DL (2014) Introducing the short Dark Triad (SD3): a brief measure of dark personality traits. Assessment 21: 28–41. [pii]; pmid:24322012
- 19. Rosenberg M (1965) Society and the adolescent self-image. Princeton, NJ: Princeton Press.
- 20. Tijdink JK, Smulders YM, Vergouwen AC, de Vet HC, Knol DL (2014) The assessment of publication pressure in medical science; validity and reliability of a Publication Pressure Questionnaire (PPQ). Qual Life Res 23: 2055–2062. pmid:24522963
- 21. Martinson BC, Anderson MS, DeVries R. (2005) Scientists behaving badly. Nature 435: 737–738. pmid:15944677
- 22. Detmold C.E. (1882) The Historial, political, and diplomatic writings of Niccolo Machiavelli. Boston: J.R. Osgood and company.
- 23. Niccolo Machiavelli (1515) The Prince. In. pp. 85.
- 24. Cook C, Heath F, Thompson RL (2000) A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement 60: 821–836.
- 25. Joinson A (1999) Social desirability, anonymity, and Internet-based questionnaires. Behavior Research Methods, Instruments, & Computers 31: 433–438. pmid:10502866
- 26. Tourangeau R, Smith T.W. (1996) Asking sensitive questions the impact of data collection mode, question format, and question context. Public opinion quarterly 60: 275–304.