Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Determinants of Cervical Cancer Screening Accuracy for Visual Inspection with Acetic Acid (VIA) and Lugol’s Iodine (VILI) Performed by Nurse and Physician

Abstract

Background

Visual inspection with acetic acid (VIA) and Lugol’s iodine (VILI) are used to screen women for cervical cancer in low-resource settings. Little is known about correlates of their diagnostic accuracy by healthcare provider. We examined determinants of VIA and VILI screening accuracy by examiner in a cross-sectional screening study of 1528 women aged 30 years or older in a suburb of Kinshasa, Democratic Republic of Congo.

Methods

We used a logistic regression model for sensitivity and specificity to estimate the diagnostic accuracy of VIA and VILI, independently performed by nurse and physician, as a function of sociodemographic and reproductive health characteristics.

Results

Nurses rated tests as positive more often than physicians (36.3% vs 30.2% for VIA, 26.2% vs 25.2% for VILI). Women’s age was the most important determinant of performance. It was inversely associated with sensitivity (nurse’s VIA: p<0.001, nurse’s VILI: p = 0.018, physician’s VIA: p = 0.005, physician’s VILI: p = 0.006) but positively associated with specificity (all four combinations: p<0.001). Increasing parity adversely affected sensitivity and specificity, but the effects on sensitivity were significant for nurses only. The screening performance of physician’s assessment was significantly better than the nurse’s (difference in sensitivity: VIA = 13%, VILI = 16%; difference in specificity: VIA = 6%, VILI = 1%).

Conclusions

Age and parity influence the performance of visual tests for cervical cancer screening. Proper training of local healthcare providers in the conduct of these tests should take into account these factors for improved performance of VIA and VILI in detecting cervical precancerous lesions among women in limited-resource settings.

Citation: Raifu AO, El-Zein M, Sangwa-Lugoma G, Ramanakumar A, Walter SD, Franco EL, et al. (2017) Determinants of Cervical Cancer Screening Accuracy for Visual Inspection with Acetic Acid (VIA) and Lugol’s Iodine (VILI) Performed by Nurse and Physician. PLoS ONE 12(1): e0170631. https://doi.org/10.1371/journal.pone.0170631

Editor: Marcia Edilaine Lopes Consolaro, Universidade Estadual de Maringa, BRAZIL

Received: October 18, 2016; Accepted: January 7, 2017; Published: January 20, 2017

Copyright: © 2017 Raifu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. A fully anonymized data set has been made available as a Supporting Information file.

Funding: Financial support for the Congo Cervical Screening Study was provided by the Program for Appropriate Technology in Health (PATH) (Grant to Ghislain Sangwa-Lugoma), an International Agency for Research on Cancer Postdoctoral Fellowship (to Ghislain Sangwa-Lugoma), and by the Cancer Research Society Division of Epidemiology.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: ASC-US, Atypical squamous cells of undetermined significance; CI, Confidence interval; CIN, Cervical intraepithelial neoplasia; HSIL, High-grade squamous intraepithelial lesion; LSIL, Low-grade squamous intraepithelial lesion; NILM, Negative for intraepithelial lesions or malignancy; VIA, Visual inspection with acetic acid; VILI, Visual inspection with Lugol’s iodine

Introduction

Cervical cancer is the third most common cancer diagnosed among women worldwide and ranks seventh among all causes of death, with an estimated 529,000 new cases and 275,000 deaths in 2008 [1,2]. Nearly 85% of cases occur in developing countries, mostly in Eastern Africa, Melanesia, Southern Africa, and Middle Africa [1]. Cervical cancer is a preventable disease if precancerous lesions are detected early through effective screening programs, but establishment and successful implementation of the latter is challenging in low-income countries. For cervical cancer screening to be successful in resource-limited settings, the screening test, diagnosis, and treatment must either be provided on-site or in clinics accessible to the majority of women at risk. In search of simple, cost-effective screening methods for cervical cancer prevention in low-resource settings, visual inspection with acetic acid (VIA) and visual inspection with Lugol’s iodine (VILI) have been considered as alternative, less technically complex tests to conventional cytology. However, their diagnostic accuracy in detecting high-grade precursor lesions and invasive cervical cancer varies across studies. VIA sensitivity and specificity ranged from 55% to 96% and from 49% to 98%, respectively, whereas VILI sensitivity and specificity ranged from 44% to 98% and from 75% to 91%, respectively [3–7].

The diagnostic performance of VIA and VILI for cervical cancer screening, and determinants of their positivity have been extensively evaluated [4,6–12]. However, no study has investigated how patient characteristics might influence their diagnostic accuracy. The objective of the present study is to examine demographic, sexual behavior, and clinical determinants of the sensitivity and specificity of VIA and VILI tests, independently performed by nurse and physician, in a community-based cross-sectional cervical cancer screening study conducted in a primary health care setting in the suburbs of Kinshasa, the Democratic Republic of Congo.

Materials and Methods

Study design and population

A detailed description of the design, study population, eligibility criteria, data collection methods, and study outcomes of the Congo Community-Based Screening Study is provided elsewhere [7,13,14]. Fig 1 presents an overview of the study population and procedures. Briefly, 1699 women aged 30 years and older were invited to attend an educational session on cervical cancer prevention provided by a trained nurse, between 2003 and 2004 at the Mbuku Healthcare Center, Kinsenso commune. Of these, 1571 women accepted, and 1528 eligible women were interviewed by a trained nurse using a structured, standardized questionnaire to collect information on sociodemographic, reproductive, clinical, lifestyle, and sexual behavior characteristics.

thumbnail
Download:
Fig 1. Overview of the Congo Community-Based Screening Study population and procedures.

Eligible women were screened by conventional cytology, HPV DNA testing, VIA, VILI, and colposcopy.

https://doi.org/10.1371/journal.pone.0170631.g001

Women were screened by conventional cytology (smears read offsite in Lyon, France), HPV DNA testing (Hybrid Capture 2), VIA, VILI, and colposcopy. Cytological diagnoses were based on the Bethesda system [15], i.e., negative for intraepithelial lesions or malignancy (NILM), atypical squamous cells of undetermined significance (ASC-US), low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL), and cancer (no other findings were recorded in this study). Cytology smears were read by a cytotechnologist and reviewed by a gynecologic-cytopathologist without knowledge of disease status or results of the other screening tests. VIA and VILI were independently performed by one of six attending nurses and by an experienced gynecologist (GSL) who trained the nurses. Colposcopy-directed biopsies were obtained from women with cervical abnormalities. Cervical biopsies were also performed in a random sample of women (~20%) who had normal colposcopic findings. The histopathological slides were shipped to Montreal and read at the pathology lab of the Jewish General Hospital, a teaching institution affiliated with McGill University. Biopsy slides were read in research capacity by an experienced gynecologic pathologist who was blinded to the screening test results and colposcopic impressions. Disease was defined as cervical intraepithelial neoplasia (CIN) of grade 2 or worse (CIN2+). All women provided verbal or written informed consent, depending on their level of literacy. A local community health representative independent from the study ascertained that all instances of verbal consent were given after subjects listened attentively to the full text of the consent form read by the study staff. The PATH Human Subjects Protection Committee and the Institutional Review Board of the Kinshasa School of Public Health approved the study, the informed consent, and the above approach for obtaining it from subjects.

Statistical analysis

We used a method originally derived by Cornfield (1962) to estimate a subgroup risk of coronary heart disease [16], and later applied by Coughlin et al. (1992) to estimate the sensitivity and specificity of a screening test [17]. Four multivariate logistic regression models were used to estimate the diagnostic accuracy indices and corresponding 95% confidence intervals (CI) for VIA and VILI tests performed by nurses and clinician. Compared to the crude approach of calculating sensitivity and specificity of screening tests, this model-based approach allows for evaluating and adjusting for predictor variables, which was integral to our analysis. In addition, it makes use of all data on women with and without cervical lesions through inclusion of the gold standard variable in the model (in this case: histology supplemented by colposcopy).

We chose predictors based on a backward stepwise selection procedure, using an arbitrary p-value of 0.15 and the likelihood ratio test, to accommodate important covariates. We also used judgment based on biological plausibility for variables that could potentially influence the performance of VIA and VILI, e.g., menopausal status. The final models included age (30–39, 40–49, 50–59, ≥ 60 years), age at first sexual intercourse (< 18 years, ≥ 18 years), number of lifetime sexual partners (0–1, 2–3, ≥ 4), history of sexually transmitted diseases (no, yes), parity (0–1, 2–5, 6–9, ≥ 10), menopausal status (pre vs. post), healthcare provider (medical doctor, other), history of Pap test (never, ever), HPV DNA test results (negative, positive), and cytology test results (NILM, ASC-US, LSIL, HSIL, and cancer). The disease status variable was based on the results from the colposcopy supplemented with histological verification if a biopsy had been taken (Fig 1). Eight variables had missing values; number of lifetime sexual partners (0.3%), age at first sexual intercourse (0.4%), parity (0.5%), history of Pap smear (0.6%), healthcare provider (2.8%), menopausal status (3.2%), cytology (9.3%), and HPV DNA (11.5%). Our preliminary logistic regression analyses showed that 379 observations (25% of the data) would be omitted due to case-wise deletion. To avoid loss of information, all statistical analyses were based on a derivative dataset using multiple imputation to replace missing data by expected values.

A non-parametric test developed by Cuzick (1985) [18] was used to assess trends across categorically ordered predictors. For binary predictors, a 2-sample z-test for proportions was performed to compare the screening accuracy of VIA and VILI between the two categories. We also calculated, across predictors, the differences in sensitivity and specificity of VIA and VILI between nurse and physician along with corresponding 95%CIs. All statistical analyses and multiple imputation were carried out using the Stata statistical software package (StataCorp, Release 12.1, College Station TX) with two-sided p-values.

Results and Discussion

The mean age of participants was 42.7 years (range: 30–85 years). Among the 1528 women enrolled, nurses reported a positive VIA for 555 (36.3%) and a positive VILI for 401 (26.2%), whereas VIA and VILI positivity according to the physician’s assessment were recorded for 462 (30.2%) and 385 (25.2%) women, respectively. Table 1 shows the distribution of VIA and VILI positivity, independently performed by nurse and physician, according to selected sociodemographic, sexual behavior, and clinical variables. The proportions of VIA and VILI positivity were lower for physician-assessed compared to nurse-administered tests across most non-clinical characteristics.

thumbnail
Download:
Table 1. Characteristics of participants with positive test results from visual inspection methods performed by nurse and physician, the Congo Community-Based Screening Study.

https://doi.org/10.1371/journal.pone.0170631.t001

Tables 2 and 3 show the diagnostic accuracy of VIA and VILI, respectively, for nurse and physician, stratified according to different characteristics and mutually adjusted for each other. Irrespective of test and examiner, there was a trend for sensitivity to decrease with the age of the woman, an observation that was mirrored, expectedly, on menopausal status. The trends were opposite for specificity, and were statistically significant. Increasing parity adversely affected sensitivity and specificity, but the effects on sensitivity were significant for nurses only and the effects on specificity were significant for physician-performed tests only. Although the examiners did not know the HPV test results, HPV positivity seemed to improve the examiner’s propensity to detect disease, but not significantly. On the other hand, irrespective of test and examiner, specificity degraded significantly when women were HPV positive. With respect to cytology, except for VIA done by a nurse, there were similar trends of increased sensitivity and decreased specificity as grade of lesion increased.

thumbnail
Download:
Table 2. Determinants of VIA sensitivity and specificity performed by nurse and physician: the Congo Community-Based Screening Study.

https://doi.org/10.1371/journal.pone.0170631.t002

thumbnail
Download:
Table 3. Determinants of VILI sensitivity and specificity performed by nurse and physician: the Congo Community-Based Screening Study.

https://doi.org/10.1371/journal.pone.0170631.t003

In general, there was a significant difference in performance between the physician and the nurse. For VIA, the sensitivity of the physician’s assessment was 13% higher in absolute terms than the nurse’s (95% CI: 0.09–0.16). The physician scoring was also more specific than the nurse’s (difference = 6%; 95% CI: 0.06–0.07). The same was observed for VILI: difference in sensitivity: 16% (95%CI: 0.12, 0.20); difference in specificity: 1% (95% CI: 0.01, 0.02), favoring the physician assessment. The differences observed across most characteristics show that the physician tended to be more accurate than nurses, with few exceptions.

We used the Coughlin’s model approach [17] to estimate sensitivity and specificity of VIA and VILI cervical cancer screening tests as a function of potential influences in performance and to identify potential characteristics that would lead to inadequate screening performance in field work conditions. We found that younger age, higher parity, pre-menopausal status, HPV positivity, and Pap cytology test results can be important influences on the performance of VIA and VILI tests in this study population. We found that a physician’s examination affords higher sensitivity and specificity for these visual screening tests than that performed by nurses, across most predictors.

Although previous studies have investigated predictors of VIA positivity [9–11], they focused on estimating odds ratios, an approach that is better suited for evaluation of effectiveness rather than the diagnostic accuracy of screening tests [19]. Hence, it may be difficult to compare our findings with previous reports, especially for VILI which has been less extensively studied. Nonetheless, an effect of age on VIA positivity rate was repeatedly observed [5,9,10,20]. We found that sensitivity of VIA and VILI decreased while specificity increased as age increased. Women with higher parity were reported to have an increased risk of a positive VIA in some studies [10,21], but not all [8]. VIA positivity was not found to be associated with cytology results [11] or with HPV positivity [10]. Conversely, women who tested positive for HPV were significantly more likely to have a positive VIA (15%, 95% CI: 12.9–17.2) than HPV negative women (6.3%, 95% CI: 5.7–6.9) [9]. Moreover, a CIN2+ diagnosis and HPV positivity with a higher viral load were found to be determinants of VIA positivity [9].

Our results show variation in nurse- and physician-based VIA and VILI positivity rates, that were also higher, especially for VIA, than previously reported figures [9–11]. Variability in VIA positivity across studies could be attributed to several factors, such as the skill and experience of the health care provider performing the test, variation in the light source, procedure for preparation of 4–5% acetic acid solution and its storage, underlying prevalence of sexually transmitted diseases or cervical disease, and the choice of gold standard or non-uniformity of the gold standard for disease definition [21,22].

Studies of inter-rater agreement on the performance of visual inspection techniques for precancerous lesions of cervical cancer are very limited, but good concordance is generally reported. Moderate to substantial agreement for VIA performance was found between test providers [6,23,24], with no adjustment for patient characteristics. A VIA by nurse compared to that performed by physician had higher sensitivity (100% vs. 87.5%) and lower specificity (53% vs. 63%), but no statistically significant differences in the diagnostic accuracy of VIA were found between nurse and physician [6]. Another study found higher sensitivity (88.9% vs. 80.0%) and specificity (69.8% vs. 54.9%) for VIA by physician compared to VIA by nurse, but the differences were not statistically tested [23]. An overall agreement of 66.7% [Kappa value = 0.57 (95% CI: 0.48, 0.66)] was reported for three independent raters assessing cervical photographs after acetic acid wash, but sensitivity and specificity were not estimated [24]. In the current study, the overall sensitivity (VIA: 90.3% vs. 77.4%; VILI: 90.3% vs. 74.2%) and specificity (VIA: 71.0% vs. 64.5%; VILI: 76.2% vs. 74.7%) were higher for the physician compared with nurses as examiners. The significant differences in the sensitivity and specificity observed between nurse and physician could be due to adjustment for other risk factors in the logistic regression models.

Subjectivity in the interpretation of visual test results by different test providers and high false positive rates are acknowledged limitations of visual inspection screening tests [6,12,25]. In our study, the false positive rates of VIA and VILI differed significantly between nurse and physician (VIA: 35.4% vs. 29%; VILI: 25.3% vs. 23.8%). Other limitations that need to be underscored include the cross-sectional nature of the study, and our inability to explore inter-nurse variability as a function of field experience with VIA and VILI techniques, as this information was not available.

The main strength of our study is that it is the first report to use a statistical approach (the Coughlin model) to calculate sensitivity and specificity of VIA and VILI tests while accounting for potential patient characteristics that can influence test accuracy. Moreover, ours is the first to investigate whether the accuracy of VIA and VILI tests depends on the test provider, while simultaneously controlling for other patient characteristics. Other strengths of this study include its population-based design, high response rate, the fact that cytopathologists and pathologists were blinded to the results of the screening tests, and that all women underwent colposcopy, thus minimizing the likelihood of verification bias which arises when the only individuals to undergo the reference standard test for verification of disease are those who have screened positive [26,27].

Conclusions

The diagnostic accuracy of VIA and VILI tests performed by nurse and physician were significantly related to being younger, having high parity, pre-menopausal status, HPV positivity, and cytology with an ASC-US cut-off status. Our results suggest that improved training of healthcare providers (prior to their being deployed as screeners) to take these determinants into account will enhance the performance of VIA and VILI in detecting cervical precancerous lesions among women in low-resource settings. To improve their diagnostic accuracy, more training of the nurse on case definition and interpretation of definite aceto-white cervical epithelium of VIA test is also warranted.

Supporting Information

S1 Table. An anonymized version of the non-imputed dataset available in an excel format.

https://doi.org/10.1371/journal.pone.0170631.s001

(XLSX)

Acknowledgments

Investigators in the Congo Screening Study Group include: Canadian Research Staff: Mariam El-Zein, Alex Ferenczy, Eduardo L. Franco (Principal Investigator), Salaheddin Mahmud, Samih Nasr, Amidu O. Raifu, Agnihotram Ramanakumar, Stephen D. Walter. Congo Research Staff: Patrick Kayembe, Ghislain Sangwa-Lugoma, Rahma. International Research Staff: Jean Liaras (Lyon, France), Attila Lorincz (London, UK).

Author Contributions

  1. Conceptualization: AOR GSL AR SDW ELF.
  2. Data curation: MEZ AR.
  3. Formal analysis: AOR AR SDW ELF.
  4. Funding acquisition: GSL ELF.
  5. Investigation: GSL ELF.
  6. Methodology: AOR GSL ELF.
  7. Project administration: ELF.
  8. Resources: GSL ELF.
  9. Software: AOR AR.
  10. Supervision: SDW ELF.
  11. Validation: AOR MZ.
  12. Writing – original draft: AOR MEZ ELF.
  13. Writing – review & editing: AOR MEZ GSL AR SDW ELF.

References

  1. 1. Ferlay J, Shin H-R, Bray F, Forman D, Mathers C, Parkin DM. Estimates of worldwide burden of cancer in 2008: GLOBOCAN 2008. Int J Cancer [Internet]. 2010 Dec 15 [cited 2014 May 23];127(12):2893–917. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21351269 pmid:21351269
  2. 2. Arbyn M, Castellsagué X, de Sanjosé S, Bruni L, Saraiya M, Bray F, et al. Worldwide burden of cervical cancer in 2008. Ann Oncol [Internet]. 2011 Dec [cited 2014 May 26];22(12):2675–86. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21471563 pmid:21471563
  3. 3. Denny L, Quinn M, Sankaranarayanan R. Chapter 8: Screening for cervical cancer in developing countries. Vaccine [Internet]. 2006 Aug 31 [cited 2014 Oct 8];24 Suppl 3:S3/71–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16950020
  4. 4. Sankaranarayanan R, Basu P, Wesley RS, Mahe C, Keita N, Mbalawa CCG, et al. Accuracy of visual screening for cervical neoplasia: Results from an IARC multicentre study in India and Africa. Int J Cancer [Internet]. 2004 Jul 20 [cited 2014 Jul 7];110(6):907–13. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15170675 pmid:15170675
  5. 5. Arbyn M, Raifu a O, Autier P, Ferlay J. Burden of cervical cancer in Europe: estimates for 2004. Ann Oncol [Internet]. 2007 Oct [cited 2014 May 27];18(10):1708–15. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17369600 pmid:17369600
  6. 6. Bhatla N, Mukhopadhyay A, Joshi S, Kumar A, Kriplani A, Rm P, et al. Visual Inspection for Cervical Cancer Screening ; Evaluation by Doctor versus Paramedical Worker. Indian J Cancer. 2004;41(1):32–6. pmid:15105577
  7. 7. Sangwa-Lugoma G, Mahmud S, Nasr SH, Liaras J, Kayembe PK, Tozin RR, et al. Visual inspection as a cervical cancer screening method in a primary health care setting in Africa. Int J Cancer [Internet]. 2006 Sep 15 [cited 2014 Jul 4];119(6):1389–95. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16619217 pmid:16619217
  8. 8. Ogunbowale T, Lawoyin TO. Cervical cancer risk factors and predictors of cervical dysplasia among women in south-west Nigeria. Aust J Rural Health [Internet]. 2008 Dec [cited 2014 Oct 23];16(6):338–42. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19032205 pmid:19032205
  9. 9. Castle PE, Qiao Y-L, Zhao F-H, Chen W, Valdez M, Zhang X, et al. Clinical determinants of a positive visual inspection after treatment with acetic acid for cervical cancer screening. BJOG [Internet]. 2014 May [cited 2014 Oct 23];121(6):739–46. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24575872 pmid:24575872
  10. 10. Kahesa C, Kjaer SK, Ngoma T, Mwaiselage J, Dartell M, Iftner T, et al. Risk factors for VIA positivity and determinants of screening attendances in Dar es Salaam, Tanzania. BMC Public Health [Internet]. BMC Public Health; 2012 Jan [cited 2014 Oct 23];12(1):1055. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3552680&tool=pmcentrez&rendertype=abstract
  11. 11. Vedantham H, Silver MI, Kalpana B, Rekha C, Karuna BP, Vidyadhari K, et al. Determinants of VIA (Visual Inspection of the Cervix After Acetic Acid Application) positivity in cervical cancer screening of women in a peri-urban area in Andhra Pradesh, India. Cancer Epidemiol Biomarkers Prev [Internet]. 2010 May [cited 2014 Jun 12];19(5):1373–80. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2913449&tool=pmcentrez&rendertype=abstract pmid:20447927
  12. 12. Arbyn M, Sankaranarayanan R, Muwonge R, Keita N, Dolo A, Mbalawa CG, et al. Pooled analysis of the accuracy of five cervical cancer screening tests assessed in eleven studies in Africa and India. Int J Cancer [Internet]. 2008 Jul 1 [cited 2014 Jun 8];123(1):153–60. Available from: http://www.ncbi.nlm.nih.gov/pubmed/18404671 pmid:18404671
  13. 13. Sangwa-Lugoma G, Ramanakumar A V, Mahmud S, Liaras J, Kayembe PK, Tozin RR, et al. Prevalence and determinants of high-risk human papillomavirus infection in women from a sub-Saharan African community. Sex Transm Dis [Internet]. 2011 Apr [cited 2014 Oct 22];38(4):308–15. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21150817 pmid:21150817
  14. 14. Mahmud SM, Sangwa-Lugoma G, Nasr SH, Kayembe PK, Tozin RR, Drouin P, et al. Comparison of human papillomavirus testing and cytology for cervical cancer screening in a primary health care setting in the Democratic Republic of the Congo. Gynecol Oncol [Internet]. Elsevier Inc.; 2012;124(2):286–91. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22062546 pmid:22062546
  15. 15. Solomon D, Davey D, Kurman R, Moriarty A, O’Connor D, Prey M, et al. The 2001 Bethesda System: terminology for reporting results of cervical cytology. JAMA [Internet]. 2002 Apr 24;287(16):2114–9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/11966386
  16. 16. Cornfield. Joint dependence of risk of coronary heart disease on serum cholesterol and systolic blood pressure: a discriminant function analysis JEROME CORNFIELD. Fed Proc. 1962;21(11):58–61.
  17. 17. Coughlin SS, Trock B, Criqui MH, Pickle LW, Browner D, Tefft MC. The logistic modeling of sensitivity, specificity, and predictive value of a diagnostic test. J Clin Epidemiol. 1992;45(1):1–7. pmid:1738006
  18. 18. Cuzick J. A Wilcoxon-type test for trend. Stat Med [Internet]. 1985;4(1):87–90. Available from: http://www.ncbi.nlm.nih.gov/pubmed/3992076 pmid:3992076
  19. 19. Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004;159(9):882–90. pmid:15105181
  20. 20. Cremer M, Conlisk E, Maza M, Bullard K, Peralta E, Siedhoff M, et al. Adequacy of visual inspection with acetic acid in women of advancing age. Int J Gynaecol Obstet [Internet]. International Federation of Gynecology and Obstetrics; 2011 Apr [cited 2014 Oct 23];113(1):68–71. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21272884 pmid:21272884
  21. 21. Mahé C, Gaffikin L. Screening test accuracy studies: How valid are our conclusions? Application to visual inspection methods for cervical screening. Cancer Causes Control. 2005;16(6):657–66. pmid:16049804
  22. 22. Parashari A, Singh V. Reasons for variation in sensitivity and specificity of visual inspection with acetic acid (VIA) for the detection of pre- cancer and cancer lesions of uterine cervix. Asian Pac J Cancer Prev [Internet]. 2013;14(12):7761–2. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24460365 pmid:24460365
  23. 23. Sherigar B, Dalal A, Durdi G, Pujar Y, Dhumale H. Cervical Cancer Screening by Visual Inspection with Acetic Acid—Interobserver Variability between Nurse and Physician. Asian Pac J Cancer Prev. 2010;11:323–6. pmid:20843109
  24. 24. Sellors JW, Jeronimo J, Sankaranarayanan R, Wright TC, Howard M, Blumenthal PD. Assessment of the cervix after acetic acid wash: Inter-rater agreement using photographs. Obstet Gynecol. 2002;99(4):635–40. pmid:12039126
  25. 25. Sankaranarayanan R, Nessa A, Esmy PO, Dangou J-M. Visual inspection methods for cervical cancer prevention. Best Pract Res Clin Obstet Gynaecol [Internet]. Elsevier Ltd; 2012;26(2):221–32. pmid:22075441
  26. 26. Zhou X. Comparing accuracies of two screening tests in a two phase study for dementia. Appl Stat. 1998;47:135–47.
  27. 27. Franco EL. Statistical issues in human papillomavirus testing and screening. Clin Lab Med. 2000;20(2):345–67. pmid:10863644