Ethics Review

This study was approved by the National Healthcare Group ethics review board. Written informed consent was obtained from all study participants. All study healthcare workers received Singapore Ministry of Health accredited training on point-of-care HIV rapid testing.

Setting and Design

From December 2008 to August 2010, a cross-sectional study was conducted at 4 HIV test sites in Singapore: the Communicable Disease Centre (CDC) outpatient clinic (Singapore’s reference HIV treatment centre), the Department of Sexually Transmitted Infections (STI) Control (DSC) clinic (Singapore’s reference STI treatment centre) and two private general practice clinics. These represent a diverse spectrum of potential HIV self-test end-users in Singapore.

Study Population

The DSC, which provides point-of-care rapid testing to STI patients, and the two general practice clinics, which were government-approved anonymous point-of-care HIV rapid test sites, were included to recruit “at-risk” participants who did not yet know their HIV status. As new HIV diagnoses were expected to be rare among at-risk participants, we also recruited known HIV-positive participants from the CDC, the national HIV treatment centre, to facilitate evaluation of self-testing among confirmed HIV-seropositive individuals. Additionally, any effect of sociodemographic differences between known HIV-seropositive individuals and at-risk individuals on study endpoints would be detected.

As there were two trained healthcare workers recruiting from 4 study sites, convenience sampling was conducted at the sites when the healthcare workers were available. There was no predefined order for the 2 healthcare workers to rotate to the various sites. Additionally, at CDC and DSC, sampling of one participant each from blocks of 2 was performed, with pre-determined random selection of the order to be selected in each pair. Due to smaller numbers of at-risk attendees, all at-risk persons seeking rapid HIV tests at the private clinics whenever the healthcare workers were available were approached to participate. Only consenting individuals aged 21 years and older who had never undergone HIV rapid testing were recruited.

Study Procedures

Pre-test counseling, as per Singapore Ministry of Health training on point-of-care HIV rapid testing, was provided immediately following study consent. Following pre-test counseling, participants were provided a 31-question pre-test survey on demographics and knowledge and attitudes toward HIV self-testing (Appendix S1). For English-illiterate participants (defined as self-reporting unable to read or write English), pre- and post-test surveys were administered verbally by the trained healthcare worker, using either Mandarin or Malay, to the participant (the other two main languages in Singapore). English-illiterate participants were included to determine if English literacy would be a factor in test accuracy or interpretation. To determine sensitivity and specificity comparing self-testing by untrained persons to healthcare worker testing (reference standard), participants performed self-testing guided solely by an 11-step pictorial guide, followed by a repeat test by a trained healthcare worker.

The OraQuick ADVANCE Rapid HIV 1/2 Antibody Test (OraSure Technologies, Bethlehem, Pennsylvania, USA), an approved point-of-care rapid test in Singapore, was used. Our study kit included all commercial kit testing equipment and an 11-step pictorial instruction sheet with English instructions designed by the study team based on test kit instructions, which replaced the product insert (Appendix S2). These 11 steps included test kit preparation (Steps #1–6), collection of oral fluids (#7–8), insertion of the specimen into the reagent vial (#9, 10), and interpretation of results (#11). Test results were read based on the presence or absence and position of 2 reddish-purple lines in the result window. An instruction sheet with 7 pictures representing the range of results was provided for test result interpretation. The pre- and post-test questionnaires and information sheets were modeled on those used in a prior study [14].

To reduce bias from verbal instructions by the healthcare worker, no verbal instructions were given in the course of self-testing. Both test results (the first by self-testing, and the second by healthcare worker testing) were interpreted by the healthcare worker. To determine the ability of untrained persons to accurately interpret test results, participants were then provided 3 model test results (positive, negative and an invalid result) to interpret. Interpretation of model test results were based on the reference pictorial guide provided. Finally, a 14-question post-test survey was administered (Appendix S1). Results of self-test and healthcare worker conducted test, both read by the healthcare worker, were revealed only after the participant had interpreted the model tests and completed the post-test survey. All at-risk participants received pre and post-test counseling and standard-of-care confirmatory HIV blood test, according the MOH guidelines [16]. Post-test counseling, as per Singapore Ministry of Health training on point-of-care HIV rapid testing, was conducted following completion of all study procedures. CDC participants did not undergo additional HIV blood tests and were counseled that the self-testing results would not alter their HIV status.

Statistical Analysis

Key demographic variables were compared between at-risk individuals and known HIV-positive patients, using the chi-square test to compare proportions for categorical variables, and the Mann-Whitney test to compare medians for continuous variables. Inter-rater agreement, measured by the κ value, was estimated in two analyses – one including invalid results, and one excluding invalid results. For the analysis of self-testing sensitivity and specificity, pairs with invalid test results were excluded. To assess the variability in accuracy estimates due to invalids, in a second analysis, the 3 invalid results were then assumed to be discordant pairs. This was as sensitivity and specificity estimates could only be calculated on dichotomous outcomes for both the test under evaluation (self-test) and the reference test (healthcare worker test). In the analysis of interpretation results, sensitivity analysis was performed, by eliminating incorrect results that were reported as invalid. This was to estimate the proportion of misinterpretations which resulted in a positive or negative result.

Due to presence of zero cells, Fisher’s exact test was used to explore demographic and behavioral factors associated with false negative self-testing. Univariate and multivariate logistic regression analysis was used to explore demographic and behavioural associations with wrongly interpreting a positive or invalid sample result as negative. Fisher’s exact test was used to compare rates of incorrect model result interpretation between known HIV-seropositive individuals and at-risk individuals. Covariates analyzed were gender, age, marital status, ethnicity, highest education attained, monthly income, English literacy and sexual orientation.

Complete case analysis was used for missing data. Analyses were performed on Stata Release 11.1 (Stata Corporation, College Station, Texas, USA).

HIV Self-test Sensitivity and Specificity

For analysis of sensitivity and specificity, the eight result pairs with missing data and 3 invalid self-tests were excluded, leaving 983 result pairs for analysis. Self-testing had a sensitivity of 97.4% (95% CI: 95.1% to 99.7%) and specificity of 99.9% (95% CI: 99.6% to 100%) compared to healthcare worker testing (Table 2). The most common pattern of discordant results was a false-negative self-test paired with a positive healthcare worker test in 5 known HIV-positive participants. Based on healthcare worker assessment of the 5 participants with false-negative results, 4 (80%) had incorrect collection of oral fluid and 2 (40%) had incorrect test kit preparation. In qualitative discussions with study healthcare workers, the commonest error for oral fluid collection was using the collection pad to swab the external lips, instead of the gingival crevice. For incorrect test kit preparation, the commonest error was touching the collection pad during removal from packaging, or spilling the test solutions. There were no independent demographic or behavioral predictors of a false-negative self-test. Due to small numbers, multivariate analysis of predictors of discordant results was not performed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Pattern of self-test results, by healthcare worker testing results.^a

https://doi.org/10.1371/journal.pone.0045168.t002

Sensitivity and specificity were re-estimated with the 3 test-pairs with invalid results assumed to be discordant. The estimates were minimally affected. Self-testing had a sensitivity of 96.9% (95% CI: 94.4% to 99.4%) and specificity of 99.6% (95% CI: 99.1% to 100%).

Model Test Result Interpretation

Positive, negative and invalid sample tests were interpreted correctly by 949 (96.0%), 925 (93.1%) and 939 (95.2%) participants respectively (Table 3). The commonest pattern of misinterpretation was interpreting a negative sample result as invalid, a finding in 61 (6.1%) of participants. Five (0.5%) of participants misinterpreted a positive sample result as negative and 29 (2.9%) misinterpreted an invalid result as negative. Thirty-two participants (3.2%) wrongly interpreted a positive or invalid sample result as negative. Compared to at-risk participants, a higher proportion of known-HIV positive participants misinterpreted model positive (91.0% vs. 97.2%, P<0.001), negative (85.5% vs. 95.0%, P<0.001), and invalid results (91.9% vs. 96.1%, P = 0.023). There were no independent demographic or behavioral predictors of wrongly interpreting a positive or invalid sample result as negative.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Participant interpretation of sample test results, by site.^a

https://doi.org/10.1371/journal.pone.0045168.t003

In the analysis excluding tests reported as invalid, the rates of correct interpretation improved, with 99.5% of participants and 99.1% of participants correctly identifying the model positive and negative sample respectively.

Survey Results on User Acceptability

The questionnaire revealed that 56.4% of participants had heard of HIV rapid tests, with the highest awareness among private clinic at-risk participants (83.2%) (Table 4). More than 90% of all participants preferred to receive their test results within one hour. Overall, 87.4% of participants would purchase an over-the-counter rapid test kit, with the highest proportion being among private clinic at-risk participants (92.4%). 89% wanted to conduct HIV testing in private and only 36.4% would agree to have their names recorded when undergoing a HIV test. 72.5% felt that pre-test counseling was necessary.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Selected participants’ opinions regarding self-testing before and after self-testing, by site.^a

https://doi.org/10.1371/journal.pone.0045168.t004

Post self-testing, approximately 95% felt that test kit instructions were easy to understand, the kit was convenient to use, and that they would recommend it to others. Accordingly, almost 90% felt that the kits should be available over-the-counter. 74.9% felt that post-test counseling was necessary. Only 28% would pay at least USD15 for the kit with the highest proportion being among private clinic at-risk participants (42.5%).

Sensitivity and Specificity of HIV Self-testing

Our study suggests that a HIV oral-fluid self test has markedly better accuracy compared to a blood-based self-test in Singapore. In a previous study by our group, blood-based self-testing had a κ of 0.277. Out of 350 self-tests conducted, 197 (56.3%) were invalid in the prior study. In our current study, the κ value for inter-rater agreement comparing self-testing to healthcare worker testing was 0.97 (95% CI: 0.95–0.99, P<0.001). This suggests that, in Singapore, participants are much more able to perform oral fluid sample collection compared to finger-prick based sample collection.

Our sensitivity and specificity results, and number of invalid tests, are similar to previous studies involving observed self-testing using oral-fluid HIV tests. In an observed use regulatory study submitted to the US FDA involving 531 known HIV positives and 500 persons of unknown status, HIV oral-fluid self-test sensitivity was 97.9% (95% CI:95.0% to 99.4%), and specificity 99.79% (95% CI: 98.1% to 100.0%) [17]. A new metric, the Test System Failure Rate (TFR), measured the rate of self-tests with neither a positive or negative result (thereby including invalids). The overall TFR in the above observed study was 3.3%, compared to 0.3% in our study. In a study involving 478 emergency department attendees who conducted self-testing using oral fluid or blood-based tests, Gaydos et al. in Baltimore determined a HIV self-testing accuracy of 99.6% [18]. Ninety-one percent of self-tests were conducted using an oral-fluid kit. In a study of 260 participants, who received a brief demonstrating of oral-fluid testing and then conducted supervised oral HIV self-testing in Blantyre, Malawi, Choko et al. determined a HIV self-test accuracy of 99.2% with two false negative tests [19].

Incorrect oral fluid collection is a potential factor in false negative results. Four of the 5 participants in our study with false-negative results were observed to incorrectly perform oral fluid collection. For accurate results, it is important that oral mucosal transudate be collected from the crevicular space between the gums and teeth as this fluid is rich in immunoglobulin G (IgG), the main class of antibodies detected for a positive HIV test [20]. Sample collection should avoid the cheeks, floor and roof of the mouth, as saliva has low IgG levels and contains bacterial and salivary proteases which may degrade IgG [21]. Another reason for false-negative results could be decreased anti-gp41 antibody titres among known HIV positive participants on treatment. This could affect test results as the oral fluid-based rapid test used in this study detects anti-gp41 antibodies [22]. The current product insert categorically warns known HIV positive patients not to use the test [17].

Accuracy of Model Result Interpretation

Accuracy of model result interpretation in our study was very similar to a device interpretation study reported to the FDA for self oral-fluid HIV test approval. In the regulatory study, the rates of correctly identifying model results were: high positive model tests 95.00%, negative results 93.80% and invalid results 92.10% [17]. In analysis excluding results interpreted as invalid in the regulatory study, 96.74% and 95.57% of participants accurately identified high positive and negative results respectively. This suggests that emphasizing seeking re-testing at an anonymous HIV test site should a user read a result as invalid would significantly help mitigate errors in interpretation in the course of actual use of a home self-test.

When interpreting test results, false positive interpretation could result in unnecessary anxiety, while false negative interpretation could result in an erroneous sense of security and continued high-risk behavior [7], [23]. The majority of interpretation errors were positive and negative results read as invalid. The implication of this error could be limited by including in test instructions that patients who test positive or invalid should present to HIV test sites for confirmation. The effects of a false negative interpretation are harder to mitigate. A potential solution to reduce misinterpretation while maintaining test confidentiality could be to use existing mobile communication technology to capture and transmit a digital image of the self-test result for digital or visual verification by a computer or trained person respectively. The verified result could be conveyed to the self-tester via text message or phone-call [24].

Survey Results

In agreement with most published studies, questionnaire responses revealed a high level of user support for oral fluid-based HIV self-testing. The majority of respondents (>85%) would purchase over-the-counter test kits which provide results within an hour, liked anonymous testing in private, and would recommend the kit to others. In surveys from California and Seattle representing diverse populations, up to a third of respondents were supportive of self-testing [25]–[27]. In a previous self-test study in Singapore involving a blood-based kit, 89% of at-risk persons preferred testing in private and 97% would recommend the kit to others [14].

However, the current study supports the concern that poorer at-risk populations may not access HIV self-testing [28]. Private clinic participants, the wealthiest group among the test sites, had the highest level of positive responses in support of self-testing, and were more likely to agree to pay at least USD15 for the test kit (the approximate laboratory testing costs currently). If HIV self-testing is to be employed, measures to decrease the financial burden of self-testing warrant consideration.

Although 89% of participants favored testing in private, about three quarters wanted pre- and post-test counseling. One possible measure to provide counseling and linkage to care while maintaining confidentiality would be the use of phone-based counseling [29]. The results from a study of the first 175,000 users of anonymous home collection HIV tests are reassuring – of 400 newly diagnosed HIV clients, telephone counseling and release of results was associated with 74% accepting referral for care, and the rest refusing because they had an existing healthcare provider [9].

Implications for Possible HIV Self-testing in Singapore

In Singapore, HIV self-testing can be operationalized as an extension of the currently available MOH-approved anonymous HIV test sites. Anonymous test sites are an exception to the legal requirement to report all HIV positive tests to a name-based MOH registry. Unlike provider-based anonymous testing, alternative forms of pre and post-test counseling and linkage to care would obviously be necessary for self-testing. As discussed above, phone based counseling is a potential option. In the United States, the commercial company marketing the FDA approved HIV self-test has committed to maintaining a 24-hour toll-free number providing counseling, advice on test conduct, and referral services for care. This strategy could be used in Singapore.

Limitations

A major limitation of our study was that test assessment and interpretation was conducted in an observed setting. This could introduce the potential for overestimation of accuracy as participants could have additional information otherwise not available in private. In the United States, the follow on study of unobserved self-testing submitted to the FDA involving 5558 participants reported a lower sensitivity of 93.0% (95% CI: 86.6% to 96.9%) compared to the observed testing study. A similar study in the Singapore context would further strengthen estimates of accuracy in actual practice. A related limitation was known HIV-positive participants may be less invested in self-test results, and bias sensitivity estimates lower. However, the above unobserved study in patients on unknown status did not further improve sensitivities appreciably. It is likely that HIV-positive participants were less invested in accurately interpreting model test results, as misinterpretation was significantly more prevalent among HIV-positive participants compared with the at-risk population. Selection bias could have occurred as convenience sampling was employed. The questionnaire responses of known HIV-positive participants may be influenced by healthcare provision and education but this was unlikely to affect the ability to perform and interpret tests as all participants had no prior experience with HIV rapid tests. Issues relevant to self-testing not addressed by the current study include the psychological impact of lack of face-to-face counseling, the effect on surveillance data collection, and measures to prevent abuse of over-the-counter availability of self-tests (for example, the use of such tests to coerce individuals to undergo HIV testing).

Self-testing was associated with high specificity, and a small but significant number of false negatives. Incorrectly identifying model results as invalid was a major reason for incorrect result interpretation. Survey responses demonstrated strong support for making self-testing available. In Singapore, HIV self-testing could be operationalized as an extension of the currently available anonymous HIV test sites. Further studies in unobserved use could aid in the discussion on making self-testing available in Singapore.