Introduction

The National Institutes of Health is the largest source of funding for medical research in the world. As such, it is a major source of hope for individuals afflicted with specific diseases and their associated advocacy groups. In 1998, the Institute of Medicine (IOM) issued a report entitled “Scientific Opportunities and Public Needs” noting that there were problems regarding the public’s perception of how research funds were allocated at NIH [1]. Specifically, it said that some believed “that NIH cares more about curiosity than cure, more about fundamental science than clinical application”. The IOM report highlighted the need for public input into NIH using “a formal mechanism through which the public can inform the priority setting”. One of the recommendations of the report was that “NIH should improve the quality and analysis of its data on funding by disease”. Similar discussions subsequently took place among members of Congress, and in 2006, as part of the NIH Reauthorization Act, Congress mandated that “the Director of NIH shall establish an electronic system to uniformly code research grants and activities…The electronic system shall be searchable by a variety of codes, such as the type of research grant, the research entity managing the grant, and the public health area of interest” [2].

The result of this congressional mandate was the NIH Research, Condition, and Disease Categorization (RCDC) system, a publicly available online computerized database, updated annually, that reports the amount being spent by each NIH institute for each disease category. The RCDC includes extramural grants and contracts as well as intramural research projects for all 27 NIH institutes and centers. The database includes the name and institution of the principal investigator; the project’s funding history; an abstract summarizing the research; and a statement regarding the project`s public health relevance. The explicit intent of Congress was to make such information available to the public. As stated on the RCDC website: “The American people want to know how the NIH spends their tax dollars. The RCDC process categorizes the NIH research projects funded with those tax dollars” [3].

The RCDC database is thus a potentially rich research resource that could be useful for disease advocacy groups. Surprisingly, the database appears to be little known or used. A Medline search of articles related to the RCDC identified only two articles, on statistics [4] and disease burden [5]. Another article used the RCDC database without naming it to ascertain NIH research expenditures for cystic fibrosis and sickle cell disease [6]. The present report illustrates how the RCDC can be used by advocates to evaluate NIH research for particular diseases. We used schizophrenia research funded by the NIMH to illustrate how this information can inform the public about funding priorities.

Schizophrenia is one of the nation’s most important diseases. Its one-year prevalence in the United States, as reported by the National Institute of Mental Health (NIMH) to Congress in 1993, is 1.5% among adults 18 and over and 1.2% among children 9 to 17 [7]. These numbers were based on the Epidemiologic Catchment Area study, the last in depth prevalence study carried out in the U.S. Based upon the estimated 2018 population of 327.2 million, this translates into 4.25 million Americans affected by this disease. Schizophrenia is also a major contributor to the problems of homelessness and the overcrowding of jails and prisons. The most recent estimate of the annual cost of schizophrenia in the United States is $155.7 billion [8].

In recent years, concerns have been raised regarding the relevance of schizophrenia research carried out by NIMH. An editorial published in the British Journal of Psychiatry in 2016, authored by 20 current or former members of the NIMH National Advisory Mental Health Council, including one of the present authors (Dr. Rush), noted that NIMH research had become increasingly focused on basic research, especially genetics and neural circuits, instead of more clinical research that might help people currently afflicted. They therefore called “for an increase in public discussion of how to appropriate funding resources across mental health research domains” [9].

Others have expressed similar concerns. Dr. Steven Hyman, a former NIMH Director (1996–2001), recently claimed that “no new drug targets or therapeutic mechanisms of real significance have been developed for more than four decades” [10]. Dr. Thomas Insel, another former NIMH Director (2002 to 2015), observed that despite spending $20 billion “on the neuroscience and genetics of mental disorders… I don’t think we moved the needle in reducing suicide, reducing hospitalization, [or] improving recovery for the tens of millions of people who have mental illness” [11].

Two advocacy groups in the United States focus specifically on serious mental illness, especially schizophrenia: the Treatment Advocacy Center in Arlington, Virginia (www.treatmentadvocacycenter.org) and Mental Illness Policy Org in New York City (www.mentalillnesspolicy.org). Other advocacy groups such as The National Alliance on Mental Illness (NAMI) and Mental Health America focus on mental disorders more broadly but do not specifically target schizophrenia. Based on the reports of NIMH’s apparent failure to carry out sufficient clinical research (9–11), the authors undertook discussions with the two schizophrenia-focused advocacy groups regarding how to assess the NIMH research portfolio.

Methods

The following research was not based on a formal protocol but rather evolved from the discussions cited above. One of the authors (reviewer one) became aware of the RCDC database and proceeded to do a rating of the projects to determine the feasibility of such research. Reviewer one then recruited reviewers two and three to independently do a similar rating, masked to the results of the other two assessors. All three reviewers had an association with one or both of the advocacy groups and shared a belief that NIMH should support a balanced portfolio between basic and clinical research, acknowledging that both were necessary. Each of the reviewers had had extensive clinical experience with patients with schizophrenia as well as extensive research experience, including having been funded by NIMH for research on schizophrenia and/or bipolar disorder. Dr. Rush served as both a member or Chair of three different research study sections at NIMH, as a member of the NIMH National Advisory Mental Health Council and as coauthor of the 2016 consensus statement referenced above [9].

Each reviewer was asked to estimate the relative likelihood (likely, possible, very unlikely) that the completion of the research project would improve the symptoms and/or quality of life for persons with schizophrenia during the next twenty years. The terms “likely”, “possible”, and “very unlikely” were not formally defined. Instead the reviewers relied on general medical knowledge regarding how long it has taken basic brain research in such areas as genetics and neural circuits to be translated into clinically useful knowledge. The twenty-year period was chosen based on the fact that the median age of all Americans is 38 and individuals with schizophrenia have a 15–25 year shortened life expectancy [12]. It was assumed that individuals affected with schizophrenia should have a reasonable chance of profiting from ongoing NIMH research during their remaining lifetime.

The public RCDC database was accessed for 2018, the most recent year for which final data was available at the time of the study. See accompanying box for details on accessing this system. In the RCDC system, disease relevance is ascertained by a computerized analysis of all funded proposals.

How to access the NIH schizophrenia–related research grants:

• Go to: https://report.nih.gov/categorical_spending.aspx.
• Scroll down the table until you find schizophrenia.
• Go to the 2018 column which has $248 million;
• Click on the $248 million. This will give you a listing of all 542 NIH schizophrenia–related grants for 2018.
• For details of any given grant click on the grant number. This will provide you with the project information, including an abstract, a statement on public health relevance, and the grant’s funding history.

For 2018 the RCDC database listed 542 new and ongoing NIH-funded projects as being schizophrenia-related, which included 428 funded by NIMH and 114 funded by other NIH institutes. Of the latter, the largest number (n = 26) were basic brain studies funded by the National Institute of Neurological Disorders and Stroke (NINDS). Other examples of such grants and contracts included studies of smoking cessation in schizophrenia funded by the National Institute on Drug Abuse and the development of new Positron Emission Tomography (PET) ligands funded by the National Institute of Biomedical Imaging and Bioengineering.

The results of the independent ratings were summarized qualitatively and were also tested for inter-rater reliability using SAS software, version 9.4, to calculate the intraclass correlation coefficient for Fliess kappa. In reporting these results the Standard Reporting Qualitative Research (SRQR) guidelines were also used [13].

Results

The NIMH schizophrenia-related research portfolio for 2018 consisted of 428 projects costing a total of $201 million, which represented 11.5% of NIMH’s total budget of $1.755 billion for 2018. It included 418 extramural grants and contracts costing $176 million or an average of $421,000 per project per year. It also included $25 million for schizophrenia-related intramural projects which included eight separate laboratory projects, $3.9 million for the Office of the Intramural Scientific Director and $10.2 million for “NIMH space activation, maintenance and improvement”. The information available in the abstracts on the RCDC database was sufficient for assessing most research projects but less so for the intramural projects or for research centers when multiple projects were summarized in a single abstract.

The average duration of funding for the 428 NIMH research projects in 2018 was four years (range 1–31 years). Thirty-eight extramural projects had been funded for 10 years or more and 11 of these for 20 years or more. The longest funded projects were two training grants funded respectively for 31 and 28 years along with a genetics study and a study of sensory processing, each funded for 27 years. Among the intramural projects a study of neuroimaging had been funded for 26 years.

Geographically, the 418 extramural awards went to 107 institutions. There was a significant clustering of awards with seven institutions receiving a total of 120 awards. The University of Pittsburgh received the most awards with 30, followed by Johns Hopkins University with 20. Columbia University, the University of Maryland, Icahn School of Medicine at Mount Sinai, and UCLA each received 15 or more. Only five awards were made to non-American principal investigators—four Canadians and one Swedish investigator—although a few others had foreign components administered by an American principal investigator.

Table 1 and Fig 1 summarize the assessments of the three individual reviewers regarding the likelihood of the research projects improving the symptoms and/or quality of life for persons with schizophrenia during the next 20 years. At least one reviewer rated 90% (386/428) of the research projects as being very unlikely to improve the symptoms and/or quality of life for persons with schizophrenia during the next 20 years, and all three reviewers agreed on this 71% (302/428) of the time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Venn diagrams to indicate relationship amongst reviews’ ratings of the research projects.

https://doi.org/10.1371/journal.pone.0241062.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Likelihood of research projects (n = 428) improving the symptoms and/or quality of life for persons with schizophrenia within 20 years.

https://doi.org/10.1371/journal.pone.0241062.t001

Similarly, at least one reviewer rated 29% (126/428) of the research projects as possibly able to improve the symptoms and/or quality of life for persons with schizophrenia during the next 20 years. When the categories of “possible” and “likely” are combined, all three reviewers unanimously agreed that only ten percent (43/428) of the research projects held such promise. Overall, the intraclass correlation coefficient revealed a moderate level of agreement among the three raters (Fliess kappa = 0.473). Agreement was highest for those rated “very unlikely” (0.570); intermediate for those rated as “possible” (0.427); and poor for those rated as “likely” (0.165). This is also reflected in Fig 1.

Taken together, the individual reviewers selected a total of 30 projects (30/428) or 7% as being likely to improve the symptoms and/or quality of life for persons with schizophrenia within the next twenty years. Seven of these thirty were selected by two of the reviewers but none was selected by all three. For twenty-one of the thirty, all reviewers agreed that the project was either possible or likely. Examples of projects selected as likely included attempts to improve social function, medication adherence, symptoms such as auditory hallucinations, and clozapine usage; Table 2 lists all thirty research projects. The total cost of the thirty projects selected by one or more reviewer as likely to improve the symptoms and/or quality of life for persons with schizophrenia was $11,120,544; this was 5.5% of the schizophrenia-related research portfolio or 0.6%of the total NIMH budget for 2018. Note also that the average cost of a research project rated as likely was $371,000; this was significantly less than the $421,000 average cost for all extramural schizophrenia-related projects.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The 30 research projects selected by at least one of the reviewers as likely to improve the symptoms and/or quality of life for persons with schizophrenia within 20 years.

https://doi.org/10.1371/journal.pone.0241062.t002

Discussion

In keeping with the intent of Congress, the RCDC public database was used to assess the schizophrenia-related research projects funded by NIMH in 2018. Three experienced reviewers independently but unanimously agreed that 71% of the research projects were very unlikely to improve the symptoms and/or quality of life for individuals with schizophrenia over the next 20 years. The reviewers also unanimously agreed that only 10% of the research projects held any possibility for such improvement. One or more reviewers selected 30 out of the 428 total research projects as actually likely to lead to such improvement but no project was so rated by all three reviewers. The results of the study would appear to confirm the opinion of 20 current or former members of the National Advisory Mental Health Council that the NIMH research portfolio is disproportionately focused on basic research instead of clinical research that might help people currently affected [9].

The information generated by this study will be useful to the schizophrenia-related advocacy groups that co-authored this study as well as to other advocacy groups and individuals with an interest in this research. The advocacy groups will encourage their followers to contact NIMH and their representatives in Congress. From past experience we have found that advocates for individuals with schizophrenia and other serious mental illnesses feel very strongly about the paucity of research on these diseases. When provided with specific information, they will often contact their congressional representatives as well as writing letters to newspapers. A follow up study using the RCDC database looking at NIMH’s funding over time is already underway. The present study also demonstrates the feasibility of using the RCDC database to generate information of value to other disease advocacy groups. The authors plan to send the current study to the major disease advocacy groups in the United States to make them aware of this resource and its potential for influencing future research funding decisions by NIH. This is consistent with the intent of Congress in creating this database.

In assessing the NIMH schizophrenia-associated research portfolio, the reviewers were also impressed by the research areas being ignored. Foremost among these were attempts to find better treatments for individuals with schizophrenia. Among the 428 research projects there was just one treatment trial using a pharmacological agent, the anticonvulsant levetiracetamin, to treat individuals with schizophrenia. Another trial used amphetamine to enhance the effects of cognitive therapy and a third trial used a gluten-free diet. Two additional treatment trials used forms of transcranial stimulation in attempts to improve the symptoms of schizophrenia. These five trials together, representing 1.2% of the 428 research projects, were the only such trials.

We contend that NIMH should invest much greater resources to find better treatments for schizophrenia, including testing off label and potentially repurposed medications and other compounds for which there is no intellectual property protection, including prebiotics and probiotics. A major program should also be undertaken to find medications effective against schizophrenia’s negative symptoms. Cost-benefit studies should be undertaken for clozapine, the most effective anti-psychotic but markedly underutilized in the U.S. Head to head efficacy studies for various long-acting injectable antipsychotic would also be very useful.

Associated with the paucity of treatment trials for schizophrenia in the NIMH research portfolio was the absence of clinically useful studies on the optimal use of antipsychotic drugs. Twenty years ago, NIMH funded the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE), which provided clinicians with useful information regarding the selection of antipsychotic medication. It was arguably the most important schizophrenia research funded by NIMH in the last two decades. Nothing similar has been subsequently funded by NIMH despite many important and outstanding questions regarding which antipsychotic or combination is likely to be effective for specific subgroups of patients. Pharmaceutical companies are reluctant to fund head-to-head antipsychotic studies so unless NIMH does so such studies will not be done. This situation is similar to the need for head-to-head drug trials for such disorders as hypertension and rheumatoid arthritis.

NIMH should also invest resources in improving the lives of individuals with schizophrenia who are homeless or incarcerated. For example, it should study the efficacy of assisted outpatient treatment (AOT) in reducing homelessness as suggested in preliminary studies. For individuals with schizophrenia who are incarcerated for major crimes, NIMH should study the relative effectiveness of conditional release, Forensic Assertive Community Treatment (FACT) teams, and Psychiatric Security Review Boards in reducing recidivism.

Another research area conspicuously absent from the 428 NIMH research studies was the epidemiology of schizophrenia, its treatment or treatment outcomes; there was not a single epidemiological study. This stands in marked contrast to Europe where epidemiological studies are generating useful hypotheses regarding schizophrenia`s etiology, including marked differences in the incidence of schizophrenia in different parts of Europe [14] and in the incidence of schizophrenia among various immigrant groups [15,16]. The last major epidemiological study of schizophrenia funded by NIMH was the 1980s Epidemiologic Catchment Area (ECA) study; one finding from this study was a very low lifetime prevalence of schizophrenia among Mexican Americans in Los Angeles, a finding that was never followed up [17].

In summary, the main strengths of this project are the use of a novel NIH database and its demonstration as a disease advocacy tool; the use of experienced psychiatric reviewers; and the collaboration between the reviewers and the advocacy groups. The study’s main limitation is the subjective nature of the review criteria.

Recommendations

The 1998 Institute of Medicine report on NIH research suggested the need for “a formal mechanism through which the public can inform the priority setting” [1]. The RCDC database provides this mechanism. Accordingly, the authors suggest the following.

1. Publicize the availability and encourage the use of the RCDC database for research advocacy groups for all medical and psychiatric disorders.
2. At least 50% of NIMH schizophrenia-related projects should be classifiable by advocacy groups as possible or likely to improve the symptoms and/or quality of life for individuals with this disease in less than 20 years.
3. Put a much greater funding priority on developing better neurobiological or psychological treatments for schizophrenia and/or enhance their delivery in clinical care settings. Include especially the many off-label medications and plant-derived compounds that may be useful in treating schizophrenia and for which there is insufficient intellectual property protection for these compounds to be tested by industry.
4. Request that the National Academy of Medicine convene a review group to examine the NIMH intramural research program and assess both its mission and its cost effectiveness. In 2018, according to the RCDC database, it included eight schizophrenia-related research projects at a cost of $25 million.
5. Incentivize public and private community healthcare systems to conduct research to address the clinical needs of patients and providers in order to bring basic/translational research results into practice more rapidly.

In conclusion, the NIMH schizophrenia research portfolio is disproportionately weighted toward basic research that promises better treatments in the distant future. One is reminded of the advice given in similar circumstances in 1971 by a leading cancer researcher: “We cannot afford to sit and wait for the promise of tomorrow so long as stepwise progress can be made with the tools at hand today” [18].

Supporting information

Acknowledgments

We thank Dr. Robert H. Yolken for his help in planning this project and Corrie Brown for her help with the Venn Diagrams. We also thank Dr. Thomas Carmody for his assistance with the statistics.