Screening for Chronic Obstructive Pulmonary Disease: Validity and Reliability of a Portable Device in Non-Specialized Healthcare Settings

Introduction and Objectives The underdiagnosis of chronic obstructive pulmonary disease (COPD) could be improved through screening using portable devices simpler than conventional spirometers in specific healthcare settings to reach a higher percentage of the at-risk population. This study was designed to assess the validity and reliability of the COPD-6 portable device to screen for COPD in non-specialized healthcare settings. Methods Prospective cohort study to validate a diagnostic test. Three cohorts were recruited: primary care (PC), emergency services (ES) and community pharmacies (CPh). Study population: individuals with risk factors for COPD (>40 years, smoking >10 pack-years, with respiratory symptoms). The values measured using the COPD-6 were FEV1, FEV6 and the FEV1/FEV6 ratio. Subsequently, participants underwent conventional spirometry at hospital, using a post-bronchodilator FEV1/FVC value <0.7 as the gold standard criterion for the COPD diagnosis. Results 437 participants were included, 362 were valid for the analysis. COPD was diagnosed in 114 patients (31.5%). The area under the ROC curve for the COPD-6 for COPD screening was 0.8.The best cut-off point for the FEV1/FEV6 ratio was 0.8 (sensitivity, 92.1%) using spirometry with the bronchodilator test as the gold standard. There were practically no differences in the COPD-6 performancein the different settings and also regarding age, gender and smoking status. Conclusions The COPD-6 device is a valid tool for COPD screening in non-specialized healthcare settings. In this context, the best cut-off point for the FEV1/FEV6 ratio is 0.8.


Introduction
Chronic obstructive pulmonary disease (COPD) is a very common disease, with a prevalence of 10.2% in Spain, higher in men than in women, and its frequency increases with age. However, underdiagnosis can be as high as 73% [1,2]. The ability to perform spirometry-essential test for COPD diagnosis-in primary care (PC) centres would improve this situation, and such a recommendation has been made by the Spanish Society of Pneumology and Thoracic Surgery (SEPAR) [1] and by the National COPD Strategy of the Spanish National Health System [2]. However, in PC we find not only a limited availability of spirometry but also that the test is inadequately performed in a large number of centres [3][4][5], though this varies considerably between the different Spanish regions [6,7].
One of the causes of an insufficient quality of spirometry is difficulty in obtaining the forced vital capacity (FVC) correctly. An alternative parameter, the forced expiratory volume in six seconds (FEV6), has therefore been accepted as a substitute [8,9]. This simplifies spirometry technique and thus improves accuracy in the diagnosis of airway obstruction in non-specialist settings [6]. In addition, if our aim is to improve out-of-hospital screening for COPD to reduce underdiagnosis, a key element would be to employ portable devices that are easier to operate than conventional spirometers, such as the Vitalograph COPD-6 device [10].
In view of the recognised benefits of the early diagnosis of COPD, both the Spanish COPD Strategy [2] and GesEPOC (the Spanish COPD Guidelines) [1] recommend proactive screening in individuals at-risk. PC would appear to be ideal for the implementation of these programs [1,2,11], but testing could also be extended to those settings in which there are health professionals with basic training in performing respiratory function tests, and easily accessible to the general public, such as community pharmacies (CPh), where many users report respiratory symptoms (without having consulted their doctor), and emergency services (ES), where patients may attend for an exacerbation. Spirometry with a bronchodilator test, the gold standard, should then be used to confirm a suspected diagnosis.
The COPD-6 device measures FEV1 and FEV6 and calculates its ratio. Our group has already validated this device for the detection of obstructive respiratory disease in a hospital setting [12]. The next step would be to demonstrate whether the COPD-6 device could also be useful as a screening tool for COPD in the out-of-hospital setting, administered by staff who does not have extensive experience in performing spirometries.
The objective of this study was to establish the validity and reliability of the COPD-6 device to screen for COPD among individuals at high COPD risk in non-specialized healthcare settings (PC, ES and CPh). Conventional spirometry with a bronchodilator test was used as the gold standard for comparison. This objective falls within the scope of COPD research proposed in the official declaration of the American Thoracic Society (ATS) and the European Respiratory Society (ERS) [13].

Material and Methods
Prospective, multi-cohort study to validate a diagnostic test in various settings. The study was carried out between 2011 and 2014 in the Vigo University Hospital Complex (CHUVI) in Galicia, Spain, using consecutive cohorts: 1) a cohort drawn from PC patients (PC cohort), with the participation of 8 Health Centres in the Vigo Health District in Galicia, Spain; 2) a cohort recruited from users of Community Pharmacies(CPh cohort), with the participation of 15 pharmacies in the Vigo area, with the collaboration of the Pontevedra Official College of Pharmacists; and 3) a cohort recruited from patients attending the emergency services (ES cohort), undertaken in of the CHUVI (Xeral Hospital and Meixoeiro Hospital) and in the Redondela and Porriño emergency services for PC.
Ten PC doctors, 13 hospital and out-of-hospital ES doctors and 15 pharmacists participated in the study. All were previously trained in the use of the COPD-6 device in a 2-hour theoretical and practical session given by chest physicians and nurses who were participating in the study.Training timerequired to perform conventional spirometries adequately is longer.

Study population
Individuals who attended a PC consultation (for a respiratory or other problem), who used a participating CPh or who presented to the hospital or out-of-hospital ES with symptoms of a respiratory exacerbation, were asked to participate in the study if they satisfied the inclusion criteria.
Inclusion criteria. The same inclusion criteria were used for all 3 cohorts: patients aged over 40 years, smokers or ex-smokers of more than 10 pack-years, and symptoms suggestive of COPD (dyspnea, cough, sputum production). These criteria were consistent with the recommendations of the Spanish COPD Strategy [2] and GesEPOC [1], though it should be recognised that the GesEPOC recommendations were published after initiation of the first phase of our study and lowered the age criterion for the high-risk group to 35 years.
Exclusion criteria. The following exclusion criteria were applied: the individual declined to participate in the study, was unable to attend the hospital to perform the test for diagnostic confirmation, or presented any of the absolute contraindications for performing spirometry [14]. Individuals who had already been diagnosed with a respiratory disease were also excluded.
Study participants were recruited consecutively in a non-systematic manner after the participating doctor or pharmacist (in PC, ES or CPh) had checked that they satisfied the inclusion criteria and had obtained written informed consent after explaining the nature of the study.

Procedures
Measurement of the functional parameters, FEV1, FEV6 and FEV1/FEV6 ratio, were performed using the portable COPD-6 device (model 4000, Vitalograph Ltd., Ennis, Co. Clare, Ireland). The patient was required to perform manoeuvres similar to those used in forced spirometry with the difference that, in this case, it was only necessary to maintain exhalation for 6 seconds. Measurements were repeated until at least 2 reliable measurements were achieved, and the best values obtained for each patient were recorded. The participating PC physicians and pharmacists (for the PC and CPh cohorts) then contacted the chest physicians of the research team by mobile phone to schedule the second phase of the study, which was carried out within the following 24 hours. In the case of the ES cohort, the participating doctors in the ES remitted the patients' data to the research team by internal post; as the patients in this cohort had attended because of an exacerbation, they were scheduled for the hospital phase of the study one month after the emergency consultation, during a clinically stable phase.
In the hospital, blinded to the results of the tests performed with the COPD-6 device, the research staff performed conventional spirometry using a Datospir 120 spirometer (Sibelmed, Barcelona, Spain), recording the FEV1, FVC and FEV1/FVC ratio. The test was performed in accordance with SEPAR guidelines [14]. When an FEV1/FVC ratio less than 0.7 was detected, the study was completed with a bronchodilator test, with measurement of the same parameters after the administration of 400 μg of salbutamol via a spacer device.

Statistical analysis
A descriptive analysis of the variables was performed for each cohort and for the overall sample. Sensitivity (S), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV) and the positive and negative likelihood ratios (PLR/NLR) were calculated for the different cut-off points of the FEV1/FEV6 ratio.
The area under the receiver operating characteristic (ROC) curve was determined for the FEV1/FEV6 ratio calculated using the COPD-6 device employed to screen for COPD. An FEV1/FVC ratio less than 0.7 obtained on spirometry after the bronchodilator test was used as the gold standard criterion for the diagnosis of COPD. In addition, the areas under the ROC curves adjusted for a series of variables were compared using the Chi-squared test for homogeneity of areas, taking the median value as the cut-off point for age and for pack-years of the cumulative smoking history.
The analysis was performed using the Statistical Package for Social Sciences version 15.0 (SPSS, Chicago, Illinois, USA).
Presentation of the study was based on STARD methodology for reporting studies of diagnostic accuracy.

Results
A total of 437 individuals were included in the three cohorts; 362 subjects were valid for analysis (75 were withdrawn because they presented exclusion criteria or did not attend the hospital visit).
The characteristics of the participants in each of the cohorts and in the overall sample are shown in Table 1. The mean age of the participants in the overall study population was 55 years, 61.9% men, and 72.4% active smokers.
COPD was diagnosed in 114 patients, a prevalence of 31.5%; the majority (96 patients, 84.2%) were in Global Initiative for Chronic Obstructive Lung Disease (GOLD) stage I or II.
The ROC curves of the FEV1/FEV6 ratio calculated by the COPD-6 are shown in Fig 1 for each of the cohorts and for the overall patient sample, in which the area under the curve (AUC) was 0.8 (95% confidence interval, 0.752-0.847).
A comparison was also performed of the area under the COPD-6 ROC curves as a function of sex, age, smoker status and cumulative smoking history; no significant differences were found between these groups. These results are shown in Table 2. We also observed that COPD 6 is valid for screening even in more severe categories of COPD. With the cutoff point of 0.8, no patient with severe COPD was false negative in the screening. Tables 3-6 show the sensitivity (S), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV),positive likelihood ratio (PLR) and negative likelihood ratio (NLR)for the different cut-off points of the FEV1/FEV6 ratio in the PC, CPh and ES cohorts and in the overall study population. Based on the overall population, a cut-off point of 0.8 for the FEV1/ FEV6 ratio had a S of 92.1%, Sp of 52.8%, NPV of 93.6% and PPV of 47.3% for the detection of COPD.

Discussion
This study is the first to show that the COPD-6 is a valid and reliable tool for the detection of COPD in different non-specialized healthcare settings. The validity indexes obtained are very good and show no significant differences according to sex, smoking history or age. Furthermore, the results hardly differ when the COPD-6 is applied in different settings and for different conditions such as gender, age group or smoking status. These results reinforce the validity and reliability of the COPD-6 test. COPD screening is important because the early detection of COPD and intervention for smoking cessation can delay lung function decline, reduce the burden of COPD symptoms, reduce healthcare costs, and improve patient´s quality of life [2]. In a previous study in the hospital setting, our group has validated the COPD-6 for the detection of obstructive pulmonary disease [12]. The results of the present study are highly relevant because they provide evidence of the reliability of simple spirometric manoeuvres with an easy-to-use device to COPD screening, performed by staff who have not been trained in pulmonary function testing and who have only received a brief theoretical-practical course on the use of the device. The use of the FEV1/FEV6 ratio as a substitute for the FEV1/FVC ratio has already been accepted [8,9].According to the PLATINO longitudinal study data, FEV1/FEV6 is a better indicator of airflow obstruction, likely due to comparing volumes at fixed times of the expiratory manoeuvre and avoiding inconsistencies due to changes in the quality of the spirometries, especially in forced expiratory time across different technicians, centres, or along time [15]. Postbronchodilator FEV6 and FEV1/FEV6 as percentage of predicted are also independent prognostic factors in stable outpatients with COPD [16]. In any case,there is no consensus on the FEV1/FEV6 cut-off point considered to define obstruction [8,9,12]. The election of the cut- Table 4. Community pharmacies cohort. Values for sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio for different cut-off points for the FEV1/FEV6 ratio determined during screening for COPD using the COPD-6 device in CPh (gold standard criterion for comparison: post-bronchodilator spirometry FEV1/FVC <0.7). off point for the FEV1/FEV6 ratio must take into account that this test is being used for screening, and a high sensitivity is thus desirable. We therefore propose raising the cut-off point to 0.8 to achieve this high sensitivity and a high negative predictive value, even with a lower specificity. Although there were small differences in the 3 healthcare settings in which the screening studies were performed, it would appear that cut-off points close to 0.8 for the FEV1/FEV6 ratio are those with greatest validity for screening. On this basis, individuals at risk of COPD who have an FEV1/FEV6 ratio greater than or equal to 0.8, as determined by the COPD-6, would have a much lower probability of being diagnosed with COPD or, if this were not detected, they would have mild disease. In these cases, spirometry could theoretically be deferred, and follow-up scheduled. When the result is below 0.8, spirometry should be performed with a bronchodilator test to confirm or exclude the diagnosis of COPD. An AUC value of 0.8 seems also acceptable for screening tests performed in ambulatory settings or by staff not having extensive training in lung function tests.
In the present study we diagnosed COPD in a significant number of at-risk patients: 31.5% in the overall group, and 40% in the PC cohort. The majority of these patients presented a mild or moderate degree of obstruction, though more than 16% of patients had severe obstruction. This would support a recommendation for proactive screening of susceptible individuals in order to initiate appropriate therapeutic strategies at the earliest opportunity [1].
A curious finding in our results was the difference between the percentage of male patients in the CPh cohort (40%) compared with the other 2 cohorts (70.6% and 71.6%). We believe that this simply reflects that women are more likely than men to attend community pharmacies.Several reports have been published on the use of portable devices to screen for COPD in non-specialized settings. Miravitlles et al [17] analysed the diagnostic yield of the FEV1/FEV6 ratio measured using the COPD-6 device to screen for chronic airways obstruction, concluding that it was useful for this purpose and the best cut-off point was 0.75. But the authors did not specify who performed the COPD-6 measurements, a factor that could affect the results. In our study, the measurements in the PC cohort were made by the participating doctors who recruited the patients. This design would provide results that were a more reliable reflection of daily practice, in which all doctors are required to be able to use portable devices as screening Table 6. Overall sample. Values for sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio for the different cut-off points for the FEV-1/FEV6 ratio determined during screening for COPD using the COPD-6 device in non-specialist healthcare settings (gold standard criterion for comparison: post-bronchodilator spirometry FEV1/FVC <0.7). tools, even if they have not received extensive specific training on pulmonary function. Frith et al [18] validated another portable device, the PiKo-6, using a spirometry-measured postbronchodilator FEV1/FVC ratio <0.7 as the gold standard diagnostic criterion. They demonstrated the usefulness of the FEV1/FEV6 ratio for COPD screening in PC (area under the curve 0.85, similar to our results), and proposed a cut-off point of 0.75 for the FEV1/FEV6 ratio (S 81%; Sp71%). In our opinion, if we wish to use these devices for screening, we must look for a high sensitivity, even with a lower specificity. Our cut-off point of 0.8 has also been proposed by other authors [9,19].There are other studies that have observed different AUCs such as the ones performed by Van den Bemt et al [20] or Thorn et al [21]. The reason for the observed differences might be due that they studied populations with different characteristics or to the fact that microspirometry was performed by staff with higher experience in lung function assessment compared with our staff, who received a 2-hour session training. With regard to other healthcare settings, few studies have been published. In the FarmaEPOC study [22], patients at risk of COPD (selected by a clinical questionnaire) were screened in the pharmacy by conventional spirometry performed by staff trained on a 4-day course. An expert in spirometry telematically reviewed the quality of the studies. Only those patients with an FEV1/ FVC <0.7 (without bronchodilatation) were referred to PC to complete the study. The results in PC were only reported to the pharmacies in 15% of cases, which is a significant limitation. Furthermore, it was not explained how the diagnosis of COPD was reached in PC. This screening strategy would require pharmacies to be equipped with spirometers, lengthier training for the staff performing the studies, and time dedication by an expert healthcare professional to review the quality, and it would therefore be much more costly and difficult to apply in the majority of CPh. For that reason, in our CPh cohort, we opted to use a portable device, the COPD-6, much simpler and cheaper than conventional spirometry, an essential aspect if we wish to extend its use widely to CPh. Solidoro et al [23] in Italian pharmacies also used a portable device, the PiKo-6, to detect obstructive and restrictive lung disease, not specifically COPD; furthermore, in contrast to our study, those authors did not take into account whether patients had a previous diagnosis nor did they define the standard comparator test used to confirm the diagnosis of COPD.
There are very few data on the results of COPD screening programmes in ES. Recently, a French group [24] published a study in which they performed COPD screening in an ES, using the Nèo6 device. Spirometry was performed in those who had a questionnaire suggestive of COPD and an FEV1/FEV6 ratio <0.8, the same as the cut-off point that we propose. In the group of patients studied by spirometry, only 8,91% presented COPD, a surprisingly low prevalence of COPD compared with the results from our ES cohort (31.4%).
One of the strengths of our study is that we took the measurements using a baseline COPD-6 test, without bronchodilator, which is how the study would be performed in daily practice in any of the non-specialized settings, and we compared this with spirometry after bronchodilatation, which is the gold standard test for the diagnosis of COPD. This could be one of the reasons for the difference in results compared with our study performed in the hospital setting [12]. A further strength is the sample size, which was over 350 individuals from different settings; this increases the validity of the results obtained. Furthermore, the study was performed on an unselected population that we believe reflects reasonably and accurately the characteristics of the general population. Finally, the fact that the spirometry was performed blinded to the results of the COPD-6 eliminates observer bias and increases the validity of our results. Moreover, our research falls within the scope of COPD research proposed in the official declaration of the American Thoracic Society (ATS) and the European Respiratory Society (ERS) [13], recommending studies that evaluate case-finding strategies using mini-spirometers.
However, there are certain limitations to our study, as the use of a post-bronchodilator FEV1/FVC ratio <0.7 as the gold standard, criticised and questioned for possible inaccuracy [25], but it continues to figure in current guidelines [1,2]. A further limitation could be that the participating healthcare professionals in these studies were highly motivated. We do not know whether our results might be extrapolated to all these settings, although it is likely that with a brief training course on the use of the COPD-6 device, any health professional could use it in daily practice and achieve similar results.
In conclusion, the COPD-6 device is a valid tool for screening for COPD in non-specialized healthcare settings, and can be used by staff who has received a brief training session. For COPD screening, the best cut-off point for the FEV1/FEV6 ratio is 0.8. Obviously, the COPD-6 device will never replace spirometry, but it enables those individuals with COPD high risk to be selected when they visit their PC physician, their local pharmacy or attend an ES for an exacerbation of respiratory symptoms. The proposed cut-off point is highly reliable for excluding those individuals who do not require spirometry due to the very low likelihood that they have the disease. These results are relevant, as they will enable us progressively to reduce COP-Dunderdiagnosis in a relatively simple and cost-effective manner.