Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Historical Analysis of the Brazilian Cervical Cancer Screening Program from 2006 to 2013: A Time for Reflection

  • Ricardo Filipe Alves Costa ,

    ricardofacosta@gmail.com

    Affiliations Graduate Program of Oncology, Barretos Cancer Hospital, Barretos, São Paulo, Brazil, Barretos School of Health Sciences Dr. Paulo Prata – FACISB, Barretos, São Paulo, Brazil

  • Adhemar Longatto-Filho,

    Affiliations Molecular Oncology Research Center, Barretos Cancer Hospital, Barretos, São Paulo, Brazil, Laboratory of Medical Investigation (LIM 14), Faculty of Medicine, São Paulo University, FMUSP, São Paulo, Brazil, Life and Health Sciences Research Institute, ICVS, School of Health Sciences, Minho University, Braga, Portugal, ICVS/3B’s - PT Government Associate Laboratory, Braga/Guimarães, Portugal

  • Céline Pinheiro,

    Affiliations Barretos School of Health Sciences Dr. Paulo Prata – FACISB, Barretos, São Paulo, Brazil, Molecular Oncology Research Center, Barretos Cancer Hospital, Barretos, São Paulo, Brazil

  • Luiz Carlos Zeferino,

    Affiliation School of Medical Sciences, Women’s Hospital CAISM, Unicamp, Campinas, São Paulo, Brazil

  • José Humberto Fregnani

    Affiliation Research and Teaching Institute, Barretos Cancer Hospital, Barretos, São Paulo, Brazil

Historical Analysis of the Brazilian Cervical Cancer Screening Program from 2006 to 2013: A Time for Reflection

  • Ricardo Filipe Alves Costa, 
  • Adhemar Longatto-Filho, 
  • Céline Pinheiro, 
  • Luiz Carlos Zeferino, 
  • José Humberto Fregnani
PLOS
x

Abstract

Background

The Cervical Cancer Database of the Brazilian National Health Service (SISCOLO) contains information regarding all cervical cytological tests and, if properly explored, can be used as a tool for monitoring and managing the cervical cancer screening program. The aim of this study was to perform a historical analysis of the cervical cancer screening program in Brazil from 2006 to 2013.

Material and Methods

The data necessary to calculate quality indicators were obtained from the SISCOLO, a Brazilian health system tool. Joinpoint analysis was used to calculate the annual percentage change.

Results

We observed important trends showing decreased rates of low-grade squamous intraepithelial lesions (LSIL) and high-grade squamous intraepithelial lesions (HSIL) and an increased rate of rejected exams from 2009 to 2013. The index of positivity was maintained at levels below those indicated by international standards; very low frequencies of unsatisfactory cases were observed over the study period, which partially contradicts the low rate of positive cases. The number of positive cytological diagnoses was below that expected, considering that developed countries with low frequencies of cervical cancer detect more lesions annually.

Conclusions

The evolution of indicators from 2006 to 2013 suggests that actions must be taken to improve the effectiveness of cervical cancer control in Brazil.

Introduction

Cervical cancer is the fourth most commonly diagnosed cancer in women worldwide, with an estimated 528,000 new cases annually, and it is the fourth most common cause of cancer deaths in women, with 266,000 estimated deaths annually. More than 85% of new cervical cancer cases and more than 87% of the deaths from cervical cancer occur in developing countries [1].

In Brazil, cervical cancer, excluding non-melanomas, is the third most common cancer in women, with an estimated incidence of 15.33 cases per 100,000 women. Cervical cancer is also the third most common cause of cancer deaths among women in Brazil [2].

The lower incidence and mortality of cervical cancer in developed countries can be explained by well-structured cytological screening programs and better medical infrastructure, which are not frequently found in low- and middle-income countries [3].

The Brazilian Ministry of Health recognized the need for a national program for cervical cancer control, and its coordination is the responsibility of the Brazilian National Cancer Institute (INCA) [4]. Brazilian health authorities recommend cytology-based screening at three-year intervals for women between 25 and 64 years of age who have initiated sexual activity. In recent years, multiple approaches to develop an effective cervical screening program have been implemented in Brazil. In 2005, the Action Plan for the Control of Cervical and Breast Cancer proposed six strategic guidelines: increased coverage of the target population, laboratory quality assurance, strengthening of the information system, professional training development, social mobilization strategies and research development [4]. In 2012, to improve the quality and reliability of cytopathological exams, a Quality Management Manual for Cytopathology Laboratory was published by the Brazilian National Institute of Cancer and the Ministry of Health. This manual presents some important indicators for the monitoring of laboratory results, which assess overall and individual performance [5]. The Department of Informatics of the Public Health System (DATASUS) created the Information System of Cervical Cancer Screening (SISCOLO), which contains information regarding all Papanicolaou (Pap) tests collected in the public health system. The SISCOLO was implemented for the management and monitoring of the cervical cancer screening program [4] and is publicly available at http://www2.datasus.gov.br/DATASUS/index.php.

Although the SISCOLO was implemented for monitoring and assisting in the management of the above-mentioned program, this data resource has been poorly exploited and deserves more attention. As a result, the present work is a historical analysis of the cervical cancer screening program quality indicators in Brazil from 2006 to 2013 based on data collected from the SISCOLO.

Materials and Methods

This is a time series study of the cervical cancer screening program quality indicators in Brazil. Data regarding cytopathological exams (n = 81,322,700) were collected from the SISCOLO according to the location of collection (Brazilian state) and age of the women who voluntarily participated in the opportunistic Governmental Brazilian program of cervical cancer prevention from 2006 to 2013. Data regarding the female population was obtained from the DATASUS from 2006 to 2012. All data collected from public access databases (SISCOLO and DATASUS) are anonymous. This study was approved by the Ethics Committee of the Barretos Cancer Hospital (identification: CAAE 26354114.0.0000.5437).

The following quality indicators were determined for women of 25 to 64 years of age: (1) productivity rate; (2) percentage of exams performed during the target age (25–64 years); (3) positivity index (PI); (4) percentage of unsatisfactory exams; (5) percentage of rejected exams; (6) ASC-US (atypical squamous cells of undetermined significance) percentage; (7) ASC-H (atypical squamous cells—high grade) percentage; (8) LSIL (low-grade squamous intraepithelial lesion) percentage; (9) HSIL (high-grade squamous intraepithelial lesion) percentage; (10) ASC (atypical squamous cell) percentage; (11) ASC/abnormal exam rate; and (12) ASC/SIL ratio.

The transformation zone (TZ) percentage, i.e., the percentage of scrubs with representation of the transformation zone, was calculated for all women. This indicator was calculated for women aged < 50 years and women aged ≥ 50 years.

The formulas used to calculate these indicators are presented in Table 1.

Processing of data and statistical analysis

All data were stored in.csv format, which allowed for management of the data using R software (The R Foundations for Statistical Computing) and Microsoft Excel 2010 (Microsoft Corporation 2010). R software and Microsoft excel were used to organize the large volume of data from the DATASUS and allowed for the creation of new spreadsheets.

The annual percentage change (APC) for each indicator was calculated using the Joinpoint Regression Program Version 4.1.1 (August 2014; Statistical Methodology and Applications Branch, Surveillance Research Program, National Cancer Institute). Joinpoint analysis allows for interpretation of changes over time to determine whether these changes are statistically significant. Joinpoint software allows for adjustment of data series using a minimum number of inflection points (zero, in which case the trend is represented by a single line segment) and tests whether the inclusion of more inflection points (joinpoints) in the model has statistical significance. To test for significance, the software uses the Monte Carlo permutations method, and to determine the APC, the software uses the natural logarithm of the rates calculated using the following formulas: y = mx+b, y = ln(rate) and x = calendar year, and APC = 100x(em-1). The APC and estimates of the trend are used, with calendar year as the regression variable. Each significant point indicates an increase or decrease in the rate [6]. To describe the linear trend for each period, the APC values with 95% confidence intervals (95% CIs) are calculated for each trend.

Results

From 2006 to 2013, 81,322,750 cytopathological exams were performed in Brazil, with 62,397,698 (76.7%) women within the screening target age range from 25–64 years.

Table 2 shows the distribution of exams according to Pap test results and age. Table 3 shows the values for each indicator, and Table 4 shows the corresponding APC values. Figs 1 and 2 show the main results.

thumbnail
Table 2. Number of exams according to Pap Smear results and age for Brazil 2006–2013.

http://dx.doi.org/10.1371/journal.pone.0138945.t002

thumbnail
Table 3. Values of the cervical cancer screening program quality indicators in Brazil 2006–2013.

http://dx.doi.org/10.1371/journal.pone.0138945.t003

thumbnail
Fig 1. Historical description of the following indicators.

(A) % exams performed (2006–2013: APC = -0.7 CI 95%: -2.1; 1.2). (B) % positivity index (2006–2013: APC = 0.56 CI 95%: -0.5; 1.7). (C) % unsatisfactory (2006–2013: APC = -2.1 CI 95%: -4.3; 0.1). (D) % rejected (2006–2010: APC = 4.5 CI 95%: -5.1; 15.0) (2010–2013: APC = 35.4 CI 95%: 16.3; 56.6).

http://dx.doi.org/10.1371/journal.pone.0138945.g001

thumbnail
Fig 2. Historical description of the following indicators.

(A) % ASC-US (2006–2013: APC = 2.2 CI 95%: 0.9; 3.5). (B) % ASC-H (2006–2009: APC = 2.1 CI 95%: -2.0; 6.4) (2009–2013: APC = 6.8 CI95%: 4.0; 9.6). (C) % LSIL (2006–2013: APC = -4.6 CI 95%: -6.6; -2.5). (D) % HSIL (2006–2013: APC = -1.8 CI 95%: -3.2; -0.6). (E) % ASC (2006–2013: APC = 3.0 CI 95%: 2.1; 3.8). (F) ASC/SIL (2006–2013: APC = 7.0 CI 95%: 5.2; 8.9).

http://dx.doi.org/10.1371/journal.pone.0138945.g002

The productivity rate, positivity index and percentage of unsatisfactory exams remained relatively stable over the entire study period. A yearly significant increase was observed in the percentage of exams performed on women in the target age range (APC = 0.8). Additionally, yearly significant increases in ASC-US percentage (APC = 2.2), ASC percentage (APC = 3.0) and ASC/SIL ratio (APC = 7.0) and a yearly significant decrease in the percentage of LSIL (APC = -4.6), percentage of HSIL (APC = -1.8) and percentage of TZ were observed in women aged <50 years (APC = -0.7).

Finally, four indicators of quality showed one inflection point, indicating a change in the trend; these indicators included the percentage of rejected exams, ASC-H percentage, rate of ASC in abnormal exams and percentage of TZ in woman aged ≥ 50 years. The percentage of rejected exams significantly increased (APC = 34.4) from 2010 to 2013; prior to 2010, this rate was relatively stable. The ASC-H percentage remained stable until 2009 and then increased significantly (APC = 6.8) until 2013. The ASC/abnormal exam rate increased significantly (APC = 3.8) from 2006 to 2010, after which time it remained stable, and the percentage of TZ in women aged ≥50 years significantly decreased from 2006 to 2010 and then remained stable.

Discussion

The SISCOLO is an important tool for improving the Brazilian opportunistic cervical cancer program, as it contains a significant amount of data regarding Pap smear tests that can be used to calculate quality indicators. Through these data, it is possible to identify fragilities and strengths and to evaluate indicators to adjust the course of action. However, the SISCOLO information has not yet been used at its maximum potential because it does not allow for the identification of women or the calculation of the actual number of women who effectively underwent Pap screening. In fact, the SISCOLO only provides the overall number of tests that were performed. In addition, the SISCOLO data are only for women under the National Health System (SUS) and do not cover women who use supplementary health services [7]. Indeed, when collecting information from the SISCOLO, we noticed that some data were incomplete (e.g., 2013 data from Amapá state), and this may have been the result of the lack of a well-established flow of information between institutions. In order to overcome some of the above-mentioned limitations, the Brazilian Ministry of Health is implementing the Cancer Information System (SISCAN) web platform, which integrates the information systems of cervical (SISCOLO) and breast (SISMAMA) cancer screening programs. SISCAN will be associated with the National Health Registry and will comprise a module to identify and convene women register in the SUS, to carry out screening tests according to the periodicity and recommended age [8].

Focusing on the general results, we observed that the productivity rate, which is the ratio of the number of Pap tests and the number of women in the target age range (25–64 years), remained stable over the study period. If we consider the three-year interval of screening based on the recommendation of the Brazilian guidelines, we observed a cumulative productivity rate of approximately 45%. However, this number cannot be used as the real coverage rate, as a significant number of women with a normal Pap test underwent screening more than once in a three-year period [9, 10], possibly due to the overuse of Pap smears by physicians and a lack of women’s knowledge of Pap test periodicity [11, 12]. In addition, approximately 25% of the population had private health insurance [13], and the percentage of women who underwent a Pap test at private health insurance laboratories is not included in the SISCOLO system. This simple estimation showed that the Pap test coverage rate in Brazil is below 70%. The results presented herein do not support those found in the Brazilian National Household Sample Survey (PNAD) performed in 2008, which reported a coverage rate higher than 80% over a three-year period. PNAD is a research system that uses nationwide household data to produce basic information to study the socioeconomic development of the country [14]. However, if we consider that the productivity rate is an overestimation of the coverage rate and that we are counting the number of exams rather than the number of women who underwent a Pap test, the coverage rate appears to be lower than 70%.

Based on the percentage of exams performed on women in the target age range, the goal of the cancer screening program was achieved, as there was a significant increase in this value over the study period. However, approximately 25% of tests are performed on women outside the target age range, which implies the existence of unnecessary financial resource consumption. This high frequency of unnecessary exams outside the target population is frequently observed in opportunistic programs [1517], as in Brazil.

The analysis of the pre-analytical Pap test results showed that the percentage of unsatisfactory exams did not vary over the study period, and importantly, this indicator follows the World Health Organization (WHO) recommendation for unsatisfactory exams (<5%). However, the percentage of rejected tests significantly increased in recent years (2010–2013). This result can be explained by the lack of care in handling, transportation and identification of samples, which underscores the importance of focusing not only on sample collection but also on all of the steps between sample collection and sample analysis. Another important pre-analytical indicator is the presence of epithelial cells from the transformation zone (TZ), as the presence of these cells increases the likelihood of lesion identification, and the majority of lesions begin to develop in this region [1820]. Although the results showed that the percentage of TZ cells was close to the Brazilian National Cancer Institute reference value, it significantly decreased over the years, which may be a consequence of problems related to sample collection.

Regarding cytological alterations, the positivity indexes, which indicate the prevalence of cell alteration and characterize the sensitivity of the screening process for detecting lesions, were below those recommended by the Brazilian guidelines (3–10%) [5]. It is concerning that these indexes are significantly below the indexes observed in developed countries, which have already achieved control of cervical cancer incidence, such as Norway (4.9%) [21], the USA (6.8%) [22] and Great Britain (6.5%) [23]. Interestingly, we observed a reduction in LSIL and HSIL detection. The observed HSIL value was approximately 0.3% per year, which is below the expected range (0.5–1.0%) [5] and below the values reported in developed countries, such as the USA and Canada (0.5% and 0.6%, respectively) [24]. In addition, intraepithelial lesion (LSIL and HSIL) detection rates significantly decreased over the study period, which may be explained as a result of a complex network of events that could have limited the quality of cytology-based screening. For example, prior to analyzing the results of Pap exams, events such as inappropriate sample collection and slide preparation are likely to occur. Additionally, difficulty in the detection of intraepithelial lesions by cytotechnologists and/or cytopathologists can occur as a consequence of suboptimal slide preparation, high workload routine, limitations related to the continued education of the professionals or even inadequate training of the professionals. Conversely, this decrease may also be explained by a real reduction in the prevalence of intraepithelial lesions in the population; however, the high ASC-US and ASC-H percentages, which have been found to be increasing, support the first hypothesis of problems in the collection phase and/or the interpretation of the samples. This result highlights the need for better training of the professionals who collect and analyze these samples.

In general, the pre-analytical and analytical phase indicators suggest a suboptimal Pap test performance, which could be attributed to the low professional efficiency in the collection, preparation and reading of the samples. To maintain an adequate level of competence, it is important that the laboratory processes a minimum number of samples per year. In this context, the Pan American Health Organization (PAHO) considers it essential that a laboratory process at least 15,000 exams annually [25]. However, according to a study conducted in Brazil in 2002 on 739 participating laboratories, only 18.9% of the laboratories had performed at least 15,000 exams/year [26], suggesting that the majority of laboratories may be working with examiners who have not adequately developed their skills.

Even laboratories that perform a large amount of exams can show poor indicators due to excessive overload. As elegantly reported by Renshaw and colleagues, the reading of more than 50 slides per day (6 hours of work) initiates an ascendant curve of progressive errors related to fatigue and loss of attention [27]. This variable is far from being controlled in many Brazilian laboratories. In fact, the low price of Pap tests paid by the Government (circa US $2.0) pushes cytotechnicians to examine more than the recommended 50 slides per day. This fact generates a pernicious circle resulting in cytotechnicians getting paid less to analyze more slides than recommended.

Despite the above-mentioned limitations of the current screening program, it is important to note that the increase in access to Pap tests has been associated with a decrease in cervical cancer-related mortality, which has been observed in developed regions in Brazil (South, Southeast and Midwest). However, this success is only partial, as the underdeveloped regions (North and Northeast) have experienced an increase in cervical cancer-related mortality [28].

Based on the findings of the present study and according to the optimal quality control measures, the government should invest more funds in internal quality control during both the pre-analytical phase, via review of the adequacy of collection, fixation and transportation of samples to the laboratory, and the analytical phase, via the training of cytotechnologists to improve their professional skills. Additionally, as some studies show the importance of external quality control of laboratories in the improvement of professionals’ skills in detecting cervical cancer precursor lesions, a systematic improved external control should be implemented at all cytopathology laboratories that provide services to the public health system. The current external system of quality control implemented by the Brazilian Government is limited to a small number of samples in selected laboratories examined periodically [29, 30], which may result in an unrealistic panorama of the laboratories’ performances.

Importantly, one should also keep in mind the recent breakthrough in cervical cancer prevention due to the introduction of human papillomavirus (HPV) vaccines. This event promises to modify the burden of cervical cancer incidence and mortality. However, as the vaccines only protect against oncogenic HPVs, which are responsible for approximately 70% of all cervical cancers, it is important to increase the sensitivity and reproducibility of the screening program. In this regard, HPV DNA testing combined or followed by Pap smear test in HPV-positive cases is as an attractive approach for cervical cancer screening in the present scenario [31].

Author Contributions

Conceived and designed the experiments: RC ALF CP LCZ JHF. Performed the experiments: RC JHF. Analyzed the data: RC JHF. Contributed reagents/materials/analysis tools: RC JHF. Wrote the paper: RC ALF CP LCZ JHF.

References

  1. 1. Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, et al. Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer. 2015;136(5):E359–86. doi: 10.1002/ijc.29210 pmid:25220842.
  2. 2. INCA. Estimativa 2014: Incidência de Câncer no Brasil Rio de Janeiro 2013. Available: http://www.inca.gov.br/estimativa/2014/.
  3. 3. Mathew A, George PS. Trends in incidence and mortality rates of squamous cell carcinoma and adenocarcinoma of cervix—worldwide. Asian Pac J Cancer Prev. 2009;10(4):645–50. pmid:19827887.
  4. 4. INCA. Programa Nacional de Controle do Câncer do Colo do Útero. Rio de Janeiro: INCA; 2011.
  5. 5. INCA. Manual de Gestão da Qualidade para Laboratórios de Citopatologia. Rio de Janeiro: INCA; 2012.
  6. 6. Kim HJ, Fay MP, Feuer EJ, Midthune DN. Permutation tests for joinpoint regression with applications to cancer rates. Stat Med. 2000;19(3):335–51. Epub 2000/01/29. pmid:10649300.
  7. 7. INCA. Diretrizes Brasileiras para o Rastreamento do Câncer do Colo do Útero. Rio de Janeiro: INCA; 2011.
  8. 8. Silva DSMd, Silva AMN, Brito LMO, Gomes SRL, Nascimento MdDSB, Chein MBdC. Rastreamento do câncer do colo do útero no Estado do Maranhão, Brasil. Ciênc saúde coletiva [Internet]. 2014; 19:[1163–70 pp.]. Available: doi: 10.1590/1413-81232014194.00372013.
  9. 9. Vale DB, Morais SS, Pimenta AL, Zeferino LC. [Assessment of the cervical cancer screening in the Family Health Strategy in Amparo, Sao Paulo State, Brazil]. Cad Saude Publica. 2010;26(2):383–90. pmid:20396853.
  10. 10. Freitas RA, Carvasan GA, Morais SS, Zeferino LC. Excessive Pap smears due to opportunistic cervical cancer screening. Eur J Gynaecol Oncol. 2008;29(5):479–82. pmid:19051817.
  11. 11. Correa Mda S, Silveira DS, Siqueira FV, Facchini LA, Piccini RX, Thume E, et al. [Pap test coverage and adequacy in the South and Northeast of Brazil]. Cad Saude Publica. 2012;28(12):2257–66. pmid:23288059.
  12. 12. Pinho AdA, França I Junior, Schraiber LB, D'Oliveira AFPL. Cobertura e motivos para a realização ou não do teste de Papanicolaou no Município de São Paulo. Cad Saude Publica [Internet]. 2003; 19 (Supl.2):[S303–S13 pp.]. Available: doi: 10.1590/S0102-311X2003000800012.
  13. 13. IBGE. Pesquisa Nacional de Saúde 2013. Rio de Janeiro: IBGE; 2014.
  14. 14. IBGE. Pesquisa Nacional por Amostra de Domicilio. Um panorama da saúde no Brasil: acesso e utilização de serviços, condições de saúde e fatores de risco e proteção à saúde 2008. Rio de Janeiro: IBGE; 2010.
  15. 15. Madlensky L, Goel V, Polzer J, Ashbury FD. Assessing the evidence for organised cancer screening programmes. Eur J Cancer. 2003;39(12):1648–53. pmid:12888358.
  16. 16. Zeferino LC, Derchain SF. Cervical cancer in the developing world. Best Pract Res Clin Obstet Gynaecol. 2006;20(3):339–54. doi: 10.1016/j.bpobgyn.2006.01.018 pmid:16563869.
  17. 17. Adab P, McGhee SM, Yanova J, Wong CM, Hedley AJ. Effectiveness and efficiency of opportunistic cervical cancer screening: comparison with organized screening. Med Care. 2004;42(6):600–9. pmid:15167328.
  18. 18. Manrique EJC, Tavares SBdN, Souza NLA, Albuquerque ZBP, Zeferino LC, Amaral RGU. A revisão rápida de 100% é eficiente na detecção de resultados falsos-negativos dos exames citopatológicos cervicais e varia com a adequabilidade da amostra: uma experiência no Brasil. Rev Bras Ginecol Obstet [Internet]. 2007; 29:[402–7 pp.]. Available: doi: 10.1590/S0100-72032007000800004.
  19. 19. Franco R, Amaral RG, Montemor EBL, Montis DM, Morais SS, Zeferino LC. Fatores associados a resultados falso-negativos de exames citopatológicos do colo uterino. Rev Bras Ginecol Obstet [Internet]. 2006; 28:[479–85 pp.]. Available: doi: 10.1590/S0100-72032006000800007.
  20. 20. Dufloth RM, Messias-Silva SM, Andrade LA, di Loreto C, Munhoz DM, Zeferino LC. Nuclear alterations of cells and atypical metaplastic cells in cervical smears are predictive criteria of high-grade cervical intraepithelial neoplasia. Eur J Gynaecol Oncol. 2005;26(2):186–90. pmid:15858905.
  21. 21. Nygard JF, Skare GB, Thoresen SO. The cervical cancer screening programme in Norway, 1992–2000: changes in Pap smear coverage and incidence of cervical cancer. J Med Screen. 2002;9(2):86–91. pmid:12133929.
  22. 22. Davey DD, Neal MH, Wilbur DC, Colgan TJ, Styer PE, Mody DR. Bethesda 2001 implementation and reporting rates: 2003 practices of participants in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology. Arch Pathol Lab Med. 2004;128(11):1224–9. pmid:15504056.
  23. 23. HSCIC. Cervical Screening Programme, England 2012–13 2013. Available: http://www.hscic.gov.uk/searchcatalogue?productid=12601
  24. 24. Eversole GM, Moriarty AT, Schwartz MR, Clayton AC, Souers R, Fatheree LA, et al. Practices of participants in the college of american pathologists interlaboratory comparison program in cervicovaginal cytology, 2006. Arch Pathol Lab Med. 2010;134(3):331–5. doi: 10.1043/1543-2165-134.3.331 pmid:20196659.
  25. 25. OPS. Módulo de citologia: procedimientos. Washinton D.C.: OPS; 2000.
  26. 26. Thuler LSC, Zardo LM, Zeferino LC. Perfil dos laboratórios citopatológicos do Sistema Único de Saúde. J Bras Patol Med Lab [Internet]. 2007; 43(2):[12 p.]. Available: doi: 10.1590/S1676-24442007000200006.
  27. 27. Renshaw AA, Elsheikh TM. Sensitivity and workload for manual and automated gynecologic screening: best current estimates. Diagn Cytopathol. 2011;39(9):647–50. doi: 10.1002/dc.21439 pmid:20945459.
  28. 28. Girianelli VR, Gamarra CJ, Azevedo e Silva G. Disparities in cervical and breast cancer mortality in Brazil. Rev Saude Publica. 2014;48(3):459–67. pmid:25119941; PubMed Central PMCID: PMC4203079.
  29. 29. Freitas HGd, Thuler LCS. Monitoramento externo da qualidade dos exames citopatológicos cervicais realizados pelo Sistema Único de Saúde (SUS) no Estado de Mato Grosso do Sul. Rev Bras Ginecol Obstet [Internet]. 2012; 34:[351–6 pp.]. Available: doi: 10.1590/S0100-72032012000800002. pmid:23080277
  30. 30. Azara CZ, Manrique EJ, Tavares SB, de Souza NL, Amaral RG. Internal quality control indicators of cervical cytopathology exams performed in laboratories monitored by the External Quality Control Laboratory. Rev Bras Ginecol Obstet. 2014;36(9):398–403. pmid:25272360.
  31. 31. Franco EL, Mahmud SM, Tota J, Ferenczy A, Coutlee F. The expected impact of HPV vaccination on the accuracy of cervical cancer screening: the need for a paradigm change. Arch Med Res. 2009;40(6):478–85. doi: 10.1016/j.arcmed.2009.06.003 pmid:19853188.