Introduction

The potential impact of childhood cancer treatment on male fertility is a significant issue for both families at the time of diagnosis, and the young adult survivor [1, 2]. Treatment at any age, with chemotherapy agents, particularly high doses of alkylating agents, and pelvic radiotherapy, may damage the testes resulting in impaired sperm production[2–7]. While semen analysis remains the gold standard, a serum biomarker of sufficient accuracy, for example Follicle Stimulating Hormone (FSH) would provide a useful indirect assessment of fertility.

The feedback relationship between the seminiferous tubule and the hypothalamus/pituitary underpins the putative value of FSH and inhibin B in the quantitative assessment of spermatogenesis [8]. FSH concentrations are negatively related to sperm concentration in both normal men and in those with testicular dysfunction, whereas serum inhibin B is positively related [9–11]. Both can be used to aid discrimination of obstructive vs non-obstructive azoospermia in infertile men [12] without clear benefit of one over the other, likely reflecting their interdependence and relationship to maturational stages of spermatogenesis [13].

The ready availability and acceptability of serum FSH analysis compared to semen analysis makes it of potential value as a predictor of azoospermia in childhood cancer survivors (CCS), but the literature contains conflicting reports of the sensitivity and specificity of plasma concentrations of FSH in this context. Green et al. [14] found that FSH was unsuitable as predictor of azoospermia in CCS whilst Romerius et al. [15] concluded that FSH was an excellent predictor. It is possible that sources of heterogeneity such as diagnosis, treatment regimens or pubertal status may account for this difference. It is also possible that there is little or no inherent heterogeneity, in which case data can be combined from multiple studies in order to provide a dataset suitable for improved assessment of the true level of diagnostic strength.

In this study we identified studies that have reported FSH and sperm concentrations in CCS, and used them to (a) test the data for homogeneity and (b) to assess the value of FSH as a diagnostic predictor of azoospermia in CCS.

Materials and methods

Using an established methodology [16–18], a scoping search was carried out using relevant MeSH headings which generated 680 results on PubMed and 973 on Scopus. The Medline search strategy used was 1. 'Follicle Stimulating Hormone/b 2. FSH.ti,ab. 3. Inhibin/bl 4. Inhibin В/Ы 5. Follicle stimulating hormone.ti,ab. 6. exp Sperm Count/ 7. spermato$.ti,ab. 8. semen/су 9. (male adj3 fertil$).ti,ab. 10. azoospermia.ti,ab. 11. semen analysis.ti,ab. 12. sperm concentration.ti, ab. 13. oligospermia.ti,ab. 14. semen.ti,ab. 15. 1 or 2 or 3 or 4 or 5 16. 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 orm14 17. 15 and 16 18. (HUMANS not ANIMALS).sh. 19. 17 and 18. Only publications in written in English were screened.

The abstracts of all studies identified were screened, and any studies in cancer survivors that had data on semen analysis and FSH levels were read in full. Studies were selected if they met the following criteria: (i) they contained both serum FSH concentration and semen concentration (either as explicit values or reported in a scatterplot), (ii) World Health Organisation (WHO) certified methods [19] were used in the semen analysis; (ii) the study participants were all childhood cancer survivors, or data was clearly demarcated between childhood cancer survivors and normal controls, in which case only cancer survivor data was extracted; (iii) all study designs were included except case reports. Searches were performed by TWK, LM and WHBW using the PRISMA guidelines for reporting in systematic reviews and meta-analyses [20] between June 2015 and March 2017 (S1 PRISMA checklist). Data extraction was performed by TWK, LM and AIU using version 3.02 of WebPlotDigitizer (http://arohatgi.info/WebPlotDigitizer)); intra- and inter-observer errors were less than 1% for all values, and the extracted datasets closely match the originals in terms of descriptive statistics. Analysis of study quality was performed by LM, RTM and WHBW using the Scottish Intercollegiate Guidelines Network (SIGN) methodology (http://www.sign.ac.uk/assets/sign104_ev_levels.pdf)); all included studies were assessed as 2+ or higher.

In addition to data identified from a systematic search of the literature, we included our own data (S1 Table) used (but not explicitly reported or given as a scatterplot) in a CCS semen quality study [21]. This study involved 33 male survivors of childhood cancer recruited from the oncology database at the Royal Hospital for Sick Children, Edinburgh, from whom FSH levels were obtained in addition to semen concentrations determined according to WHO protocols. For each study participant, we recruited two age-matched controls (n = 66). The volunteers were recruited by means of advertisement in local media and through hospital out-patient clinics, and selected on the basis of the absence of any clinical evidence, on history or physical examination,of reproductive health problems. The Lothian Paediatric and Reproductive Medicine research ethics subcommittee approved the study, and all patients provided written informed consent.

While recognising that different FSH assays were used in the studies included, a detailed comparison has shown ‘fair to strong consistency’ between the relevant assays [22] with most of the variability at the lower end of the normal range thus we have used extracted data without further conversion.

Approval was not required from an ethics committee or institutional review board since our research was limited to use of previously collected, non-identifiable data that has been published in peer reviewed journals which is specifically excluded from Research Ethics Committee review by the National Research Ethics Service guidelines of the UK Health Research Agency [23].

The risk of publication bias was visually assessed by constructing funnels plots, in which calculated diagnostic accuracy is set against statistical precision [24]. In addition, we performed a linear regression of log diagnostic ratios on the inverse root of effective sample sizes as a test for funnel plot asymmetry, where a non-zero slope coefficient is suggestive of significant asymmetry and small study bias [25].

Initial analysis considered the heterogeneity or otherwise of the included studies. This was tested using four distinct techniques: visually by forest plots [26], numerically by calculating the slope of the affine regression equation linking the study diagnostic odds ratios (DOR) to the study thresholds [27, 28] (where a slope close to zero shows homogeneity of the studies), and statistically by (i) calculating the p-value for the chi-squared test of the hypothesis that the studies are heterogeneous (a high p-value suggests homogeneity) and (ii) calculating Higgins I² statistic for measuring inconsistency in meta-analyses [29] (a small value suggests homogeneity). Two statistical tests were used as the interpretation of I² can be misleading, since the importance of inconsistency depends on several factors and the magnitude and direction of effects could lead to a small I² despite a large chi-squared p-value [30].

After combining the data into a single set of (FSH, azoospermic or not azoospermic) pairs, a ROC curve was constructed. 95% confidence intervals for the AUC were calculated using 200 bootstraps of the data set, as were the optimal threshold (i.e. the level of FSH that maximizes the probability of a randomly-selected (azoospermic, not azoospermic) pair from the CCS population being correctly diagnosed) and the 95% confidence intervals for the specificity and sensitivity at each threshold value. All analyses were performed using the mada and pROC packages for the R statistical language [31].

Results

The application of inclusion and exclusion criteria to the studies found in the literature yielded four sources of FSH and semen concentration in CCS (Table 1, S2 Table, Fig 1) [5, 14, 32, 33]. Studies identified for full-text analysis, but excluded are listed in S1 References together with reasons for exclusion. The Chi-squared statistical test for funnel plot asymmetry (Fig 2) did not reach statistical significance (p = 0.32 for sensitivity; p = 0.17 for specificity), suggesting that neither studies with small sample size nor studies with results lacking statistical significance are missing from the literature. As all the included studies used WHO protocols, we conclude that they are at low risk of bias and have low concern about applicability, as specified by the QUODAS-2 and STARD frameworks for reporting diagnostic accuracy [34, 35].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Flow-chart of systematic search methodology.

n = number of studies; N = number of childhood cancer survivors fulfilling criteria.

https://doi.org/10.1371/journal.pone.0181377.g001

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Funnel plots for specificity (upper panel) and sensitivity (lower panel) relating study size to reported diagnostic accuracy for the five studies listed in Table 1.

The Chi-squared statistical test for funnel plot asymmetry did not reach statistical significance (p = 0.32 for sensitivity; p = 0.17 for specificity), suggesting a lack of publication bias.

https://doi.org/10.1371/journal.pone.0181377.g002

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the included studies.

https://doi.org/10.1371/journal.pone.0181377.t001

The confidence intervals for the log-adjusted DOR for each study have similar ranges, suggesting a lack of significant study heterogeneity (Fig 3). Visual inspection shows that each study is statistically significant in its own right, that the intervals overlap to a great extent, and that therefore the studies are unlikely to be heterogeneous. The slope of the regression equation linking the study log DOR to the study FSH thresholds was close to zero (slope = -0.01), providing numerical evidence for study homogeneity. The chi-squared p-values were 0.32 for study sensitivity and 0.17 for study specificity, supplying no statistically significant evidence for the hypothesis that the studies are heterogeneous. The Higgin’s I² statistic was 0%, the lowest possible indication of study heterogeneity. Taken together, and in conjunction with the lack of publication bias, we conclude that the studies are homogeneous in terms of dependency on FSH thresholds to determine diagnostic accuracy, and hence that combining the study data into a single set results in a representative sample of the CCS population in terms of FSH levels and sperm concentrations.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Forest plot of 95% confidence limits for the log-adjusted diagnostic odds ratio for the five studies listed in Table 1.

The vertical dashed line denotes the line of no effect. Visual inspection shows that each study is statistically significant in its own right, that the intervals overlap to a great extent, and that therefore the studies are unlikely to be heterogeneous.

https://doi.org/10.1371/journal.pone.0181377.g003

For the combined data (n = 367, SI 1, SI 2) the optimal FSH threshold was 10.4 IU/L with specificity 81% (95% CI 76%–86%) and sensitivity 82% (95% CI 76%–88%). The AUC was 0.89 (95%CI 0.85–0.92), demonstrating that FSH is a strong predictor of azoospermia for CCS (Fig 4).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Receiver-operator characteristic (ROC) curve analysis of FSH as predictor of azoospermia (combined cohort: n = 367).

Area under the curve: 0·89 (95% CI 0·85–0·92. The optimal diagnostic threshold is 10.4 mIU/mL, with sensitivity 0.814 and specificity 0.823.

https://doi.org/10.1371/journal.pone.0181377.g004

The optimal threshold maximizes the chance of a correct classification for an arbitrary survivor of childhood cancer. In order to quantify FSH levels that minimize misdiagnosis of azoospermia, a range of threshold FSH values for the diagnosis of azoospermia were calculated, together with the median and 95% confidence intervals for their associated sensitivities and specificities (Table 2). A diagnostic threshold of 17 IU/L for FSH gives 94% probability of avoiding misdiagnosis of azoospermia, with 95% confidence interval 90–97% (Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Sensitivity and specificity of FSH-based azoospermia diagnosis for a range of threshold values.

Median and 95% CI are calculated from 2,000 stratified bootstrap replicates of the combined data (n = 367).

https://doi.org/10.1371/journal.pone.0181377.t002

Discussion

We have shown that FSH has strong diagnostic power, with 89% probability that FSH levels will correctly classify as azoospermic, not azoospermic a randomly chosen survivor of childhood cancer (i.e. positive predictive value) [36], with 95% confidence that this probability is within 85% and 92% (Fig 4). For the combined data (n = 367) the optimal Follicle Stimulating Hormone threshold was 10.4 IU/L with specificity 81% (95% CI 76%–86%) and sensitivity 83% (95% CI 76%–89%). The AUC was 0.89 (95%CI 0.86–0.93). This study provides strong supporting evidence for the use of serum Follicle Stimulating Hormone as a surrogate biomarker for azoospermia in adult males who have been treated for childhood cancer. We have also calculated clinically-useful diagnostic levels for a range of FSH thresholds (Table 2).

We have assessed heterogeneity of existing studies using visual, numeric modelling and two distinct statistical tests; none of these suggested any important level of heterogeneity (Figs 2 and 3). This result is of clinical and biomedical interest in its own right, but also allows us safely to combine the data into a single set, which has greater power for statistical analysis than any single study reported to date. While different FSH assays were used in the studies included in this analysis, there is good concordance between them [22].

FSH, inhibin B, and more recently anti-Mullerian hormone have been previously investigated as biomarkers of seminiferous tubule function, often to attempt to predict the surgical recovery of sperm in azoospermic men [12]. The latter two are products of the Sertoli cell, with potentially an additional contribution to serum inhibin B from germ cells [37]. In the post-chemotherapy testis, the key pathology determining azoospermia or not is the presence or absence of spermatogonial stem cells at the end of treatment. This differs therefore from the situation in the more general male infertility population, where disorders of spermatogenic maturation are relatively common, with likely impact on the germ cell-Sertoli cell interaction and production of inhibin B, and feedback regulation of FSH. It is thus possible that serum biomarkers of spermatogenesis may be more accurate in post-chemotherapy assessment than with the wide range of pathologies seen in the general infertile population. We considered the potential value of including these biomarkers in a joint model, but there were insufficient data to do this at present.

Our calculated optimal FSH threshold for classifying a CCS as azoospermic is 10.4 UI/L, where optimal means providing the best tradeoff between sensitivity (i.e. minimized prediction of non-zero sperm concentration for CCS who are in reality azoospermic) and specificity (i.e. minimized prediction of azoospermia for CCS who in reality have non-zero sperm concentration). In clinical practice of long-term follow up of CCS, however, we suggest that a more conservative threshold is more appropriate, since a wrong diagnosis of azoospermia is worse than a false negative. It should also be emphasized that in some azoospermic CCS it is possible to obtain sperm by micro-TESE [38]. The bootstrap sampling used to provide the optimal threshold (necessarily) allows calculation of confidence intervals for sensitivities and specificities for all potential thresholds, and from these we observe that a diagnostic threshold of 17 IU/L for FSH has a 94% probability of avoiding this misdiagnosis, with 95% confidence interval of 90%–97% (Table 2). Our specificity results are in quantitative agreement with a study that reported mean FSH of 22 IU/L in 21 azoospermic CSS compared to 9 IU/L in 10 controls with 81% specificity at a 10 IU/L cutoff [39] compared to our value of 78%. However our calculated sensitivity at this cutoff is higher: 56% [39] compared to 83%.

Serum assessment of FSH is therefore a useful test before the patient is ready to submit a semen sample for analysis, and the present analysis indicates high predictive accuracy. Attempts to survey CCS with universal semen analysis have demonstrated the reluctance of these patients to submit semen samples. In contrast, a blood test is less intrusive and more acceptable to these young CCS [32]. The use of hormone measurement in dried blood spots sent by post has recently been evaluated in the analysis of reproductive function in female cancer survivors [40], and this technique has clear potential to be useful in the male case.

This study provides strong supporting evidence for the use of serum FSH as a useful surrogate biomarker for spermatogenesis in adult males who have been treated for childhood cancer, however semen analysis should always be encouraged and remains the gold standard test of spermatogenesis.

Supporting information