Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Rasch Analysis of the Premature Ejaculation Diagnostic Tool (PEDT) and the International Index of Erectile Function (IIEF) in an Iranian Sample of Prostate Cancer Patients

  • Chung-Ying Lin,

    Affiliation Department of Rehabilitation Sciences, Faculty of Health and Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong

  • Amir H. Pakpour ,

    pakpour_amir@yahoo.com; apakpour@qums.ac.ir

    Affiliation Social Determinants of Health Research Center, Qazvin University of Medical Sciences, Qazvin, Iran

  • Andrea Burri,

    Affiliations Health and Rehabilitation Research Institute, Auckland University of Technology, Auckland, New Zealand, Waitemata Pain Service, Department of Anaesthesia and Perioperative Medicine, North Shore Hospital, Auckland, New Zealand

  • Ali Montazeri

    Affiliation Mental Health Research Group, Health Metrics Research Centre, Iranian Institute for Health Sciences Research, ACECR, Tehran, Iran

Rasch Analysis of the Premature Ejaculation Diagnostic Tool (PEDT) and the International Index of Erectile Function (IIEF) in an Iranian Sample of Prostate Cancer Patients

  • Chung-Ying Lin, 
  • Amir H. Pakpour, 
  • Andrea Burri, 
  • Ali Montazeri
PLOS
x

Abstract

Background

Male sexual dysfunction is an increasing problem across a variety of general and clinical populations, such as cancer populations; especially among prostate cancer patients who tend to receive treatments that often result in erectile dysfunction (ED) and/or premature ejaculation (PE). Therefore, in order to diagnose ED and PE in these populations, adequate and efficient instruments such as the International Index of Erectile Function 5-item version (IIEF-5) and the Premature Ejaculation Diagnostic Tool (PEDT) are needed. However, since this is an important topic additional evidence of psychometric properties of the IIEF-5 and the PEDT in such samples are required. Thus the aim of the present study was to use Rasch models to investigate the construct validity, local dependency, score order, and differential item functioning (DIF) of both questionnaires in a sample of prostate cancer patients.

Methods

Prostate cancer patients (n = 1058, mean±SD age = 64.07±6.84 years) who visited urology clinics were invited to fill out the IIEF-5 and the PEDT. Construct validity was examined using infit and outfit mean square (MnSq) and local dependency using correlations between each two residual Rasch scores. Score order was investigated using step and average measures of difficulty and DIF using DIF contrast.

Results

All IIEF-5 and PEDT items had acceptable infit and outfit MnSq. Step measures revealed that all but two items had disordered categories in terms of scores 1 to 3. Only one local dependency was found, and no items displayed DIF across age, educational level, and help seeking.

Conclusions

The results showed that both the IIEF-5 and the PEDT had sound psychometric properties in the Rasch analyses, although some score disordering could be detected in both instruments. The results of no DIF items in both instruments suggest using them to compare ED and PE across age and educational level is adequate.

Introduction

One in five men suffers from some sort of sexual problem, with prevalence rates showing a steady increase [1,2]. Erectile dysfunction (ED), together with premature ejaculation (PE), is the most common male sexual disorder. The largest follow-up study published on the prevalence of ED found that only 39.31% of men reported not suffering from ED, whereas 25.14% had mild ED (that is, experienced ED sometimes), 18.79% moderate ED (that is, usually experienced) and 16.77% complete ED [3]. Reported prevalences for PE range from 10% to 40% [46]. It is estimated that worldwide up to 322 million men will suffer from erectile dysfunction (ED) in 2025 [7]. Although ED and PE have shown to heavily impact on the quality of life of men [8], very often sufferers do not seek help from urologist or other specialists because of feelings of shame and embarrassment. In addition, many affected may not have the insight as to how serious the ED and/or PE problem is, and may resist seeking help until the sexual problem becomes extremely severe.

While prevalence rates of ED and PE are already high in general population, the estimates tend to be even higher in clinical samples, such as prostate cancer patients [9,10]. Prostate cancer and its treatments (e.g. surgery, radiation, chemotherapy) can have negative impacts on a patient’s sex life and functioning. Cancer can directly impact on sexual organs, as it is the case with prostate cancer. It also can affect body image and psycho-emotional health. In addition, side effects of cancer treatments such as fatigue, pain or anxiety can severely impact on libido and consequently affect erectile function and sexual satisfaction [9,10]. Many cancer patients may feel uncomfortable discussing the issue with healthcare professionals. In other words, they may feel and notice that their sexual function has decreased but they do not know whether this decrease should be taken serious and or may feel embarrassed to talk about the problem and actively seek for help. Therefore, there is an urgent need for validated self-report instruments that allows the patients to efficiently and privately assess their sexual function, and to decide whether seeking help is necessary. Two commonly used self-reports instruments—the International Index of Erectile Function (IIEF) for the assessment of ED [8,11] and the Premature Ejaculation Diagnostic Tool (PEDT) for the assessment of PD [12] have been designed for the above mentioned purpose but validation in prostate cancer patient population is urgently needed.

While widely applied, the evidence of the psychometric properties of both the IIEF-5 and PEDT seems insufficient because as to the best of our knowledge, the existing validation studies [8,1214] used primarily classical test theory (CTT). CTT has the major drawback that it treats the scoring methods inappropriately (e.g., means and standard deviations), and does not differentiate the estimated parameters between items (item difficulty) and respondents (person ability) [15]. Other statistical methods for the assessment of psychometric properties such as the Rasch models are able to resolve the CTT drawbacks as they separately assesses person’s ability and item difficulty [16,17], and convert both item difficulty and person ability into a ratio scale using the identical unit called logit.

In addition, Rasch models investigate the issues related to the score orders, local dependency, and differential item functioning (DIF), while these issues seem to have never been investigated in both the IIEF-5 and PEDT. The score orders indicate whether the rated score reflects the respondents’ condition or not [16]. For example, a respondent who scores 1 (very low) on the IIEF-5 item should have less severe ED than does a respondent who scores 2 (low). The local dependency tests whether the IIEF-5 and PEDT items contain latent traits other than ED and PE, respectively [18]. The DIF shows that whether respondents with different characteristics (e.g., different educational level) interpret IIEF-5 and PEDT item differently [19,20].

The aim of the present study therefore was to add evidence on the psychometric properties evidence (including construct validity, local dependency, score order, and DIF) of the IIEF-5 and the PEDT in a sample of prostate cancer patients using Rasch models.

Material and Methods

Study population

Between March 2014 and August 2015 a sample of 1202 men who had a diagnosis of prostate cancer were invited to participate. The sample was recruited from urology clinics affiliated to medical universities in Tehran, Qazvin, Ahvaz, Guilan, and Tabriz, in Iran. Inclusion criteria were (1) aged ≥ 18 years, (2) being in a stable sexual relationship with a female partner for at least 6 months, (3) good cognitive function, and (4) voluntary participation. Sixty-three patients declined to participate, and 81 patients were not eligible because of their impaired cognitive function according to the Mini-Mental State Examination (MMSE; score ≤ 23) [21], resulting in a final sample of 1058 patients. After having their eligibility ascertained by the visiting urologist, participants were asked to complete a set of study questionnaires (see below) in a private clinic room. All patients provided written informed consent and the study protocol was approved by the Ethics Committee of Qazvin University of Medical Sciences.

Main outcome measures

Five-item version of the International Index of Erectile Function (IIEF-5).

The IIEF-5 represents a short version of the 15-item version of International Index of Erectile Function used to measure erectile function [8]. The IIEF has received extensive psychometric and cross-cultural validation and translation [11,22,23], the IIEF-5 seems to be feasible because it contains only five items and has strong evidence on psychometric properties. Recently, an Iranian version of the IIEF-5 has been developed, showing good psychometric properties [14]. Each item of the IIEF-5 is rated from 1 (very low; almost never or never; extremely difficult) to 5 (very high; almost always or always; not difficult), with a lower score indicating more erectile difficulties.

In addition, the sensitivity and specificity of the IIEF-5 have been tested with satisfactory values in area under receiver operating characteristic curve (AUC). The AUC was 0.97 on a sample recruited from both the US and the UK [8]. In other words, IIEF-5 has a probability of 97% to accurately identify a man with ED or without ED.

Premature Ejaculation Diagnostic Tool (PEDT).

The PEDT has been designed based on the diagnostic principles of the DSM-IV-TR for PE [24]. Previous validation studies have shown satisfactory feasibility, reliability and validity of the PEDT [12]. Similarly, the recently translated Iranian version of the PEDT also showed good psychometric properties [13]. Originally, each PEDT item is rated from 0 (not difficult at all; almost never or never; not at all) to 4 (extremely difficult; almost always or always, extremely), with a higher score indicating more difficulties with premature ejaculation. In the present study, however, the PEDT scores were recoded from 1 (extremely difficult; almost always or always, extremely) to 5 (not difficult at all; almost never or never; not at all) in order to correspond to the direction of the IIEF-5 score. As a result, the higher PEDT scores in our current study indicate less ejaculation problems.

In addition, the sensitivity and specificity of the PEDT have been tested with satisfactory values in AUC. The AUC was 0.89 for PEDT on an Iranian sample [13]. In other words, PEDT has a chance of 89%.

Statistical analysis

Descriptive analyses were conducted using SPSS 17.0 (SPSS Inc., Chicago, IL, USA). For Rasch rating scale models WINSTEPS was used [25]. Because ED and PE were considered as two separate types of sexual problems, two Rasch models were performed separately—one for IIEF-5 and another for PEDT. In addition to estimating the difficulty of each IIEF-5 item and PEDT item, we used information-weighted fit statistic (infit) mean square (MnSq) and outlier-sensitive fit statistic (outfit) MnSq to determine any redundant (infit or outfit MnSq < 0.5) or out-of-concept (infit or outfit MnSq > 1.5) item [26]. In other words, if the item fit well in its belonging construct (ED or PE), both infit and outfit MnSq should be between 0.5 and 1.5. Rasch models provide separation reliability and separation index (both included person and item separation), and a value >0.7 suggests good reliability; > 2 suggests good index [16].

The ordering of the response scores was examined using average difficulty of each response score (i.e., average measure) and step difficulty of each threshold or boundary between every two nearby scores (i.e., step measure). Satisfactory ordering of the response scores should monotonically increase average and step difficulties [27]. We also used Rasch models to test the local dependency. For that, the correlations (r) of the Rasch residuals between every two items were computed. That is, we examined whether some items are still correlated after the same underlying concept has been taken into account, and an r ≤ 0.4 is acceptable [28]. Finally, differential functioning item (DIF) for both the IIEF-5 and PEDT were tested across different age groups (Group 1: <65 years vs. Group 2: ≥65 years), different education level (Group 1: <6 educational years vs. Group 2: ≥6 years), and across help seeking (Group 1: no vs. Group 2: yes). According to other studies [26,29]a DIF was considered substantial when the DIF showed an absolute contrast (the difficulty for Group 1 minus the difficulty for Group 2) >0.5, meaning that the same item was interpreted in different ways by the two groups.

Results

The mean±SD age (n = 1058), and mean diagnosis duration were 64.07±6.84, and 6.14±3.47 years, respectively. Nearly half of the participants were at stage 2 (n = 462, 43.7%) of prostate cancer and nearly half were at a medium grade at time of participation (n = 438, 41.4%) according to the Gleason grade (Table 1). The mean (SD) scores were between 2.36 (1.70) and 3.10 (1.55) in the IIEF-5; between 1.97 (1.33) and 2.66 (1.45) in the PEDT (Table 2).

thumbnail
Table 2. Observed mean score, item difficulty and fit statistics for International Index of Erectile Function-5 (IIEF-5) and Premature Ejaculation Diagnostic Tool (PEDT) (N = 1054).

https://doi.org/10.1371/journal.pone.0157460.t002

Four participants did not respond to all the IIEF-5 and PEDT items and were therefore not included in the analyses, resulting in a sample of N = 1054 patients used for the Rasch analyses. The mean score of each item ranged from 2.36 to 3.10 for the IIEF-5 and from 1.97 to 2.66 for the PEDT (Table 2). The difficulty was −0.66 to 0.33 for the IIEF-5 and −0.45 to 0.54 for the PEDT. Infit (0.68 to 1.42 for the IIEF-5; 0.77 to 1.18 for the PEDT) and outfit MnSq (0.57 to 1.43 for the IIEF-5; 0.70 to 1.17 for the PEDT) were acceptable for all individual items. In addition to the slightly low value for person separation reliability (0.66 for the IIEF-5 and 0.68 for the PEDT), item separation reliability (0.99), person separation index (≥1.40), and item separation index (≥9.63) were all satisfactory (Table 2).

Although the average measures were monotonically increased by the categories for all IIEF-5 and PEDT items, step measures revealed that all but two items (P1, P2) had disordered categories in terms of scores 1 to 3 (Table 3). The disordered pattern showed that participants intended not to select score 2 (Fig 1a), and the probabilities of rating on scores 2 to 4 were low even in the ordering items (Fig 1b).

thumbnail
Fig 1. Disordering graph.

(A) An example of disordering graph for IIEF-5 (Item 1); the rating scale is a 5-point-Likert scale with 1 represents the worst and 5 the best conditions. (B) An example of ordering graph for PEDT (Item 1); the rating scale is a 5-point-Likert scale with 1 represents the worst and 5 the best conditions.

https://doi.org/10.1371/journal.pone.0157460.g001

thumbnail
Table 3. Threshold disordering tests for International Index of Erectile Function-5 (IIEF-5) and Premature Ejaculation Diagnostic Tool (PEDT).

https://doi.org/10.1371/journal.pone.0157460.t003

Only one local dependency was found for the IIEF-5, and none for the PEDT (Table 4). In addition, no substantial DIF was found in both the IIEF-5 and PEDT across age (<65 years vs. ≥65 years; DIF contrast = −0.18 to 0.11), educational level (<6 educational years vs. ≥6 educational years; DIF contrast = −0.13 to 0.13), and seeking help (no vs. yes; DIF contrast = −0.23 to 0.34) (Table 5).

thumbnail
Table 4. Tests of local dependency for International Index of Erectile Function-5 (IIEF-5) and Premature Ejaculation Diagnostic Tool (PEDT).

https://doi.org/10.1371/journal.pone.0157460.t004

thumbnail
Table 5. Tests of differential item functioning (DIF) for International Index of Erectile Function-5 (IIEF-5) and Premature Ejaculation Diagnostic Tool (PEDT).

https://doi.org/10.1371/journal.pone.0157460.t005

Discussion

The construct validity of the IIEF-5 and PEDT has been previously confirmed by means of factor analysis using CTT methods [13,14,30], and our present results using Rasch models are in line with these results also showing satisfactory construct validity for the IIEF-5 and the PEDT, indicating the usefulness of the two instruments. However, our results additionally reveal other issues related to the questionnaires’ psychometric properties in terms of their score ordering, local dependency, and DIF items.

Except for the score ordering, all psychometric tests performed in this study suggested that both IIEF-5 and PEDT are good instruments to assess erectile and ejaculatory problems in men suffering from prostate cancer. All items fit well in their embedded ED or PE construct without noise from other unknown concepts as evidenced by our low local dependency analysis. Moreover, no items displaying DIF indicated the appropriate use of combining and comparing respondents with different demographics [31]. However, because we only tested DIF across age, education and help seeking behavior, Clinicians should use both questionnaires with caution since DIF may be different across other demographic groups. Future studies may want to further probe into this issue.

The results of the score ordering indicate that most participants preferred not choosing score 2 when filling out both the IIEF-5 and PEDT. There are several explanations to this finding. First, the patients may not have sufficient cognition to understand the descriptors of score 2. However, because we only recruited patients who had a MMSE score > 23, therefore indicating unimpaired cognition, this explanation may not be supported. Second, only few of our participants showed impaired sexual function thus, this may be reflected in the few score 2 responses. Unfortunately, we were unable to obtain the patients’ records with the clinical diagnosis to confirm this hypothesis. Third, sexual dysfunction patients may classify the problem into four levels or less. In other words, when they have a little problem on sexual dysfunction, they may consider it as no problem. Also, our data cannot answer the third hypothesis; future studies with sufficient information and solid design are warranted to examine our second and third hypotheses.

There are some limitations in the present study. First, with the available data we were unable to explore how many categories should be used in the response scores. Although our results indicated one disordering score, we are unable to make any assumption or confirm on whether a 4-point Likert scale would fit better and would show no disordering. Future studies should further investigate this issue by administering questionnaires using different response scales (e.g., 3-point vs. 4-point Likert scales). Second, our participants were all diagnosed with a prostate cancer, and our results may not be generalized to general population or other clinical samples. Third, we did not use any gold standard measures, including objective measures of ED [32] and PE [33], to test the validity of the IIEF-5 and PEDT. However, this was not considered a serious limitation since previous studies reported high correlations between both instruments and the Urologists’ diagnoses [5,12].

Conclusions

Overall, the IIEF-5 and PEDT are two feasible and useful instruments for self-assessment of ED and PE. All the items fit well in their embedded construct without serious local ependency. In addition, no items displayed substantial DIF, suggesting that both instruments can be used in respondents from various demographic backgrounds. However, a certain degree of score disordering could be detected in both instruments, and future studies will need to further examine whether using a 4-point Likert scale could perform better than the current 5-point Likert scale.

Author Contributions

Conceived and designed the experiments: AHP C-YL. Performed the experiments: AHP. Analyzed the data: C-YL. Contributed reagents/materials/analysis tools: AHP CY-L. Wrote the paper: CY-L AM AB.

References

  1. 1. Derogatis LR, Burnett AL. The epidemiology of sexual dysfunctions. J Sexual Med. 2008; 5: 289–300.
  2. 2. Saigal CS, Wessells H, Pace J, Schonlau M, Wilt TJ Predictors and prevalence of erectile dysfunction in a racially diverse population. Arch Intern Med. 2006; 166: 207–212. pmid:16432090
  3. 3. Weber MF, Smith DP, O’Connell DL, Patel MI, de Souza PL, Sitas F, et al. Risk factors for erectile dysfunction in a cohort of 108 477 Australian men. Med J Aust. 2013; 199: 107–111. pmid:23879509
  4. 4. Lee SW, Lee JH, Sung HH, Park HJ, Park JK, Choi SK, et al. The prevalence of premature ejaculation and its clinical characteristics in Korean men according to different definitions. Int J Impot Res. 2013; 25: 7–12.
  5. 5. Tang WS, Khoo EM. Prevalence and correlates of premature ejaculation in a primary care setting: a preliminary crosssectional study. J Sex Med. 2011; 8: 2071–2078. pmid:21492404
  6. 6. Brock GB, Bénard F, Casey R, Elliott SL, Gajewski JB, Lee JC. Canadian Male Sexual Health Council survey to assess prevalence and treatment of premature ejaculation in Canada. J Sex Med. 2009; 6: 2115–2123. pmid:19572961
  7. 7. Ayta IA, McKinlay JB, Krane RJ. The likely worldwide increase in erectile dysfunction between 1995 and 2025 and some possible policy consequences. BJU International. 1999; 84: 50–56. pmid:10444124
  8. 8. Rosen RC, Cappelleri JC, Smith MD, Lipsky J, Peña . Development and evaluation of an abridged, 5-item version of the International Index of Erectile Function (IIEF-5) as a diagnostic tool for erectile dysfunction. Int J Impot Res. 1999; 11: 319–326. pmid:10637462
  9. 9. Andersen BL. Sexual functioning morbidity among cancer survivors: current status and future research directions. Cancer. 1985; 55: 1835–1842. pmid:3978569
  10. 10. Siegel T, Moul JW, Spevak M, Alvord WG, Costabile RA. The development of erectile dysfunction in men treated for prostate cancer. J Urol. 2001; 165: 430–435. pmid:11176390
  11. 11. Rosen RC, Riley A, Wagner G, Osterloh IH, Kirkpatrick J, Mishra A. The international index of erectile function (IIEF): a multidimensional scale for assessment of erectile dysfunction. Urology. 1997; 49: 822–830. pmid:9187685
  12. 12. Symonds T, Perelman MA, Althof S, Giuliano F, Martin M, May K, et al. Development and validation of a premature ejaculation diagnostic tool. Eur Urol. 2007; 52: 565–573. pmid:17275165
  13. 13. Pakpour AH, Yekaninejad MS, Nikoobakht MR, Burri A, Fridlund B. Psychometric properties of the Iranian version of the Premature Ejaculation Diagnostic Tool. Sex Med. 2014; 2: 31–40. pmid:25356299
  14. 14. Pakpour AM, Zeidi IM, Yekaninejad MS, Burri A. Validation of a translated and culturally adapted Iranian version of the International Index of Erectile Function. J Sex Marital Ther. 2014; 40: 541–551. pmid:24308814
  15. 15. Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technol Assess. 2009; 13: 1–177.
  16. 16. Chang K-C, Wang J-D, Tang H-P, Chang C-M, Lin C-&. Psychometric evaluation using Rasch analysis of the WHOQOL-BREF in heroin-dependent people undergoing methadone maintenance treatment: further item validation. Health Qual Life Outcomes. 2014; 12: 148. pmid:25277717
  17. 17. DeRoos YS, Allen-Meares P. Rasch Analysis. J Soc Serv Res. 1993; 16: 1–17.
  18. 18. Wang W-C, Wilson M. Exploring local item dependence using a random-effects facet model. Appl Psychol Meas. 2005; 4: 296–318.
  19. 19. Amin L, Rosenbaum P, Barr R, Sung L, Klaassen RJ, Dix DB, et al. Rasch analysis of the PedsQL: an increased understanding of the properties of a rating scale. J Clin Epidemiol. 2012; 65: 1117–1123. pmid:22910540
  20. 20. Khan A, Chien CW, Brauer SG. Rasch-based scoring offered more precision in differentiating patient groups in measuring upper limb function. J Clin Epidemiol. 2013; 66: 681–7. pmid:23523550
  21. 21. Su C-T, Ng H-S, Yang A-L, Lin C-Y. Psychometric evaluation of the Short Form 36 Health Survey (SF-36) and the World Health Organization Quality of Life Scale Brief Version (WHOQOL-BREF) for patients with schizophrenia. Psychol Assess. 2014; 26: 980–9. pmid:24796341
  22. 22. Cappelleri JC, Rosen RC, Smith MD, Mishra A, Osterloh IH. Diagnostic evaluation of the erectile function domain of the International Index of Erectile Function. Urology. 1999; 54: 346–351. pmid:10443736
  23. 23. Karakiewicz P, Shariat SF, Naderi A, Kadmon D, Slawin KM. Reliability of remembered International Index of Erectile Function domain scores in men with localized prostate cancer. Urology. 2005; 65: 131–135. pmid:15667878
  24. 24. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, fourth edition, text revision: DSM-IV-TR. Washington DC: American Psychiatric Association; 2000.
  25. 25. Linacre JM, Wright BD. A User’s Guide to WINSTEPS. Chicago: MESA Press; 2009.
  26. 26. Lin C-Y, Yang S-C, Lai W-W, Su W-C, Wang J-D. Rasch models suggested the satisfactory psychometric properties of the WHOQOL-BREF among lung cancer patients. J Health Psychol. Epub 2015 Sept 8.
  27. 27. Jafari P, Bagheri Z, Safe M. Item and response-category functioning of the Persian version of the KIDSCREEN-27: Rasch partial credit model. Health Qual Life Outcomes. 2012; 10: 127. pmid:23078650
  28. 28. Chang C-C, Su J-A, Tsai C-S, Yen C-F, Liu J-H, Lin C-Y. Rasch analysis suggested three unidimensional domains for Affiliate Stigma Scale: additional psychometric evaluation. J Clin Epidemiol. 2015; 68: 674–683. pmid:25748074
  29. 29. Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, et al. A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. J Clin Epidemiol. 2009; 62: 288–295. pmid:18774693
  30. 30. Kriston L, Günzler C, Harms A, Berner M. Confirmatory factor analysis of the German version of the international index of erectile function (IIEF): a comparison of four models. J Sex Med. 2008; 5: 92–99. pmid:17466059
  31. 31. Strobl C, Kopf J, Zeileis A. Rasch trees: A new method for detecting differential item functioning in the Rasch model. Psychometrika. 2015; 80: 289–316. pmid:24352514
  32. 32. Bodie J, Lewis J, Schow D, Monga M. Laboratory evaluations of erectile dysfunction: an evidence based approach. J Urol. 2003; 169: 2262–2264. pmid:12771765
  33. 33. Waldinger MD, Zwinderman AH, Schweitzer DH, Olivier B. Relevance of methodological design for the interpretation of efficacy of drug treatment of premature ejaculation: a systematic review and meta-analysis. Int J Impot Res. 2004; 16: 369–381. pmid:14961051