Search strategy and dataset

In May 2024, a bibliographic search was performed in OpenAlex with the equation: “cocaine OR cannabis OR heroin” in the Fulltext field and filtered in the Type field by dataset. A total of 2782 records were retrieved. The metadata of the records were downloaded as comma-separated values (csv) and tabulated in an.xlsx file.

The study sample was selected based on the expertise of the authors of the present work, who are part of a research group on addictive disorders and have been responsible for the Documentation Center on Drug Dependence and other Addictive Disorders website (cendocbogani.org) for more than 20 years.

Characterization of the datasets

A frequency analysis was performed on the qualitative variables, publication_date (1963–2024) and primary_location_source_display_name (SN). These analyses are shown in Fig 1 and Tables 1 and 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Diachronic occurrence of dataset collection.

https://doi.org/10.1371/journal.pone.0339653.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Occurrence of records by type of repository.

https://doi.org/10.1371/journal.pone.0339653.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Chronological evolution of dissemination of datasets by five-year period (1974-2023).

https://doi.org/10.1371/journal.pone.0339653.g001

Similarly, the sample was described through the frequency of subject classifications, using the variables called concepts_display_name and topics_display_name (this analysis can be found in S1 and S2 Tables).

To determine the completeness of authorship and institution, a dichotomous analysis (yes/no) was performed on the occurrence of content in the author_names (AN) and author_institution_names (IN) fields (this analysis is shown in Table 3). Table 4 shows this association for each SN.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Author-institution contingency.

https://doi.org/10.1371/journal.pone.0339653.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Author-institution contingency by repository.

https://doi.org/10.1371/journal.pone.0339653.t004

Documentary typology of the files included in the registers

The evaluation of the documentary typology of the files included in the retrieved records was carried out by means of a stratified random analysis of datasets (n = 771; 27.7% of the records), weighted by the records of each repository (n = 41). Selection was based on a 95% confidence level (z = 1.96) and a sampling error of ±3%. Access to the dataset from OpenAlex and to the file hosted in the repository was then manually examined by all authors. In this way, the original sources were accessed and through manual verification the documentary typologies were classified. This analysis is presented in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Typologies of documents retrieved as “datasets” and occurrence of repositories.

https://doi.org/10.1371/journal.pone.0339653.t005

Similarities in the records

To identify possible similarities between records, the DOI and display_name metadata were examined. Thus, (a) records with the same DOI were identified using the logical function IF and; (b) records with identical titles or with small differences in characters (upper/lower case, punctuation) were manually reviewed and the analysis was performed by three professionals with expertise in health sciences and scientific documentation (CR, JCVZ, RLD) to determine why identical titles occurred. This analysis is presented in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Title similarities.

https://doi.org/10.1371/journal.pone.0339653.t006

Characterization of the datasets

A total of 2782 records classified as datasets in 41 different repositories were obtained, belonging mainly to the subject classifications (named concepts_display_name) of Psychology, Medicine and Psychiatry and (named topics_display_name) “Endocannabinoid System and Its Effects on Health”, “Neurobiological Mechanisms of Drug Addiction and Depression” and “Epidemiology and Interventions for Substance Use” (S1 and S2 Tables). The diachronic evolution of the sample is distributed between 1963 (2 records) and 2024 (15 records). There is an upward trend per quinquennium in the number of records classified as datasets. Only two quinquennia have fewer records than the previous one (1979–1983 and 2014–2018). The changes between the quinquennia average a difference of 99.22 records (SD 159.25). Percentage-wise, three periods can be distinguished, (a) the first two quinquennia accumulate a percentage of less than 1% of the records, (b) the period of quinquennia between 1984 and 2003 presents percentages of records between 3% and 8%, and (c) the last four quinquennia show percentages of 12% and 15% in the first three quinquennia and an increase to 33.07% (1:3) in the last quinquennium (Fig 1).

The results show that there is an increase in the deposit of datasets over the years, which has a point of ascent from the five-year period 1999–2003, coinciding with the 3B of Open Access [39,40], and another more notorious peak from 2014–2018, coinciding with the implementation of Open Access in relation to the funding and regulation of open data, both in Europe with the 2020 horizon and other international policies [13].

Regarding the source where the bibliographic reference of the dataset is indexed and through which it has been included in OpenAlex, it is observed that Crossref provides 58.84% of the datasets (n = 1637), mainly in the last five quinquennia (n = 1409) (Table 1). Second, the largest indexing of bibliographic references to datasets comes from DataCite, which provides 25% of the total datasets for the period 2019–2023 (n = 681). Both indexes contribute 89.79% of the datasets (n = 2498) (Table 1).

In the context of research information management infrastructure, Crossref [41], launched in 2000, complements the coverage of other widely used commercial sources such as WoS and Scopus, while PubMed dominates among open databases in the health sciences [42].

One of the advantages of Crossref, OpenCitations, DataCite, OpenAIRE and OpenAlex is that they offer a wide range of metadata for any purpose free of charge, without limiting the maximum amount of metadata that can be retrieved [43].

Furthermore, OpenAlex, is popular because it has an even greater documentary reach than the main databases that are its main sources (Crossref and Microsoft Academic Graph) [42]. The results obtained in this study, which place CrossRef as the main repository for addictive substances datasets, are consistent with the fact that CrossRef has established itself as one of the preferred open metadata repositories [16].

In second place is DataCite, which contains records from almost 3,000 data repositories [44,45]. However, the temporal coverage of DataCite, which started in 2009, is inferior to that of Crossref, which, unlike the former, retrieves data back to its creation date (2000).

Analysis of the variable primary_location_source_display_name, which refers to the repositories (n = 41) where the dataset is hosted, shows that the PsycEXTRA dataset is the repository with the highest number of datasets (n = 997; 35.84%), followed by Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature (n = 582; 20.92%), GNIF Global Biodiversity Information Facility (GNIF) (n = 434; 15.6%) and Figshare (n = 373; 13.40%). These four repositories provide 85.77% of the records (n = 2386) (Table 2).

As expected, among the diversity of repositories found, the PsycEXTRA Dataset stands out, which is a specific repository on psychology and behavioural sciences created by the American Psychological Association [46], closely related to our field of study, drugs. In this sense, 6 of the top 10 repositories are thematic, highlighting PsycExtra and PsycTESTS Dataset, Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature, GNIF Global Biodiversity Information Facility, ISRCTN Registry of clinical trials, or ICPSR Data Holdings, while the rest are general repositories with multidisciplinary themes such as Figshare, Zenodo, Authorea, DATA.GOV. Overall, 65.85% (n = 27) are specialised repositories and 34.15% (n = 14) are generalist repositories. Specialised repositories account for 80.3% of the datasets while generalist repositories account for 19.7% of the datasets. These results are in line with previous studies [45] confirming that the authors of the datasets, may have a preference for certain specialized repositories over more generalist ones, following the recommendations of some publishers who encourage the deposit of data in discipline-specific repositories [47]. With respect to generalist repositories, there is a dispersal of datasets across the wide variety of non-specialist repositories, which contributes to the difficulty of locating and reusing datasets.

Analysis of the AN variable shows that of the 2782 records retrieved on the topic of addictive substances, 839 (30.2%) did not provide authorship information (Table 3). Of these, 434 were from the GNIF Global Biodiversity Information Facility repository, 309 were from the PsycExtra repository, and 105 were distributed among 21 repositories (Table 4).

Regarding the variable author_institution_names, 1919 (69%) had an empty field (Table 3). Of these, the largest distributions are in the PsycEXTRA dataset (n = 844), GNIF Global Biodiversity Information Facility (n = 434), and Figshare (n = 318). The remaining empty fields (n = 327) in the IN metadata are distributed across 38 repositories (Table 4).

This problem with authorship and institutional affiliations has also been described by Krause & Mongeon [32]. In the present study, we have also observed how the contingency of the values of both variables (yes/no), the co-occurrence probabilities are in the range of 8.6 (between 30.2% and 38.8%). On the one hand, the probability of finding neither authors nor institutional affiliation is 30.2%, the probability of finding authors with affiliation is 31%, and the probability of finding author information without institutional affiliation is 38.8% (Table 3).

It has been reported that OpenAlex indexes more than 200 million authors through their ORCID or OpenAlex ID (OAID), and more than 100,000 institutions through their Research Organisation Register (ROR) or OAID [42]. However, in our study, despite the registration of an extensive network of affiliations, it has been observed that there are some gaps (omissions) in certain fields, which, firstly, makes it difficult to properly cite the document and, secondly, affects the citations estimation of this document and evaluation of its authors and institutions. This creates an even bigger problem, which extends to the systems for obtaining indicators fed by OpenAlex, and also affects the evaluation system of researchers who deposit their datasets to promote Open Science.

This can lead to inaccurate citation counts, which are important to suggest in a “bug report or improvement suggestion” to improve OpenAlex. Therefore, given the inaccuracy of citations that can result from these issues, it is advisable to use different bibliographic databases to verify the citation counts received for each dataset, to provide a more accurate picture of the research impact of datasets [42].

The results of the present study on the lack of identification of institutions in the datasets retrieved through OpenAlex are in line with those of Zhang et al. [30], who found that about 60% of the scientific journal articles in OpenAlex in the social sciences and humanities did not contain information on the institution [30]. In this respect, Ortega and Delgado-Quirós [16] postulate that the cause of the omissions may be due to the third-party sources from which they are taken, in this case OpenAlex, such as PubMed, DataCite or Crossref. Data on the author and/or institution may not be included in the metadata of the repository in which they are hosted or are difficult to access, as is the case with GNIF Global Biodiversity Information Facility (100% absence of author and institution labels). Moreover, the way in which each database and repository labels the Author, Institution and/or Cites fields can make proper retrieval difficult [16].

In contrast, Velez-Estevez et al. [43] indicate that OpenAlex, Dimensions and Scopus Affiliations are the most complete databases for identifying the institutions of publications, and ORCID, Scopus Authors and Publons (now integrated with Web of Science Research ID) for author-based analyses. In addition, one of the features that sets OpenAlex apart is that it allows author and affiliation information to be retrieved via the Research Organisation Register (ROR) ID by implementing this process [43].

However, as the RORs and the affiliationIdentifier, affiliationIdentifierScheme and schemeURI have only been integrated into Datacite in version 4.3 [48], and in Crossref until 2021 [49], it is possible that problems are currently creeping in with the identification (tagging) of author affiliation information of a dataset hosted in this collector for a period prior to this version.

At the same time, recent scholarly literature confirms the incompleteness of some information fields in the records hosted in the repositories, which is consistent with our results regarding the lack of institutions in the retrieved datasets, e.g., 85.25%, 17% and 68.4% in Figshare, Zenodo and ICPSR respectively (Table 4). Furthermore, in the case of ICPSR, 31.6% of the records were missing author and institution [45].

Document typology of files identified as ‘Dataset’ resource type

The analysis of the random sample stratified by repository shows that 30% of the records retrieved are datasets, according to definition of dataset [9,10], deposited in 10 different repositories, representing 24% of the total number of repositories providing datasets (120 in GNIF, 72 in Figshare and 19 in Zenodo). Among the other most common document types, 21.4% were comments on scientific research (n = 165) in the Faculty Opinions repository, 15% were monographs and 8% were reports, both in the PsycExtra repository (Table 5).

Almost a quarter of the files analysed are ‘comments on research’, which is consistent with research by Johnston et al. [45], who in their study found records which, despite being described as ‘datasets’ in the metadata, are not. This is the case for the repository Faculty Opinions Ltd (now H1 Connect) (https://connect.h1.co/about), which does not contain any identifiable datasets per se, although it uses the term ‘dataset’ as a resource type for indexing in Crossref, as there is no other term in the resource type field that fits this type of document [45].

These results show that, although the sharing of data in the field of addiction has been encouraged by the creation of various institutional initiatives (U.S. NIH, NIDA), firstly, it becomes evident that there is a lack of knowledge of the concept of a dataset among the researchers who are responsible for depositing and appropriately tagging the metadata of the documents registered in the repositories. Secondly, the ambiguity between the records recovered under this document typology differs from other types of documents whose classification does not give rise to confusion (clinical trials, original articles). In turn, Scholarly Knowledge Graphs (SKG) systems would support and ameliorate publication workflows improving and guaranteeing metadata quality [50]. Furthermore, it would be highly recommended that all academic repositories implement the roadmap developed by the Repositories Expert Group in support of researchers as part of the Data Citation Implementation Pilot (DCIP) project, an initiative of FORCE11.org and the NIH-funded BioCADDIE project (https://biocaddie.org).

In parallel, data repositories would facilitate by one hand, the location, re-use and citation of data sets through an appropriate process of metadata creation with semantic enrichment [51], and on the other hand, by providing a metadata field clearly dedicated to associated publications [52]. The ultimate goal, is to ensure a correct repository of datasets with all required tags properly completed [53]. To date, 8.9% of the 41 repositories in this study have followed and committed to this roadmap [54].

Finally, one suggestion for OpenAlex to improve the collection of datasets would be to check the repositories and the information they collect through Crossref and Datacite. In the present study, 997 records retrieved from Crossref under the repository name PsycEXTRA dataset do not match the original name (APA PsycEXTRA).

On the other hand, it is important to remember that, as Velez-Estevez et al. [43] point out, open databases may contain more errors or less accuracy in some of their data, due to the cataloguing or indexing process, since the curation and pre-processing of a large amount of scientific data requires a great deal of effort and economic resources that non-profit organizations sometimes cannot afford.

Similarities in the records

Of the 2782 records retrieved, a total of 48 records (1.73%) are duplicates, i.e., they appear twice with the same DOI and the same OpenAlex record id. Therefore, 24 OpenAlex records are duplicated in the search results. Furthermore, 165 title similarities were found in the datasets, of which 132 similarities occur in two records and 33 similarities occur in more than two records (Table 6).

The above results indicate an error in the collection of records after the bibliographic search, i.e., there are no duplicate records, but the 24 records are duplicated in the results grid provided by OpenAlex, as well as in the download of the data. On the other hand, a co-occurrence of title similarities is observed. This aspect may be related to the concentration of interest in some data, whether they are versions of the same data or a coincidence of thematic data, giving us an idea of the data with the greatest flow or interest. At this point it is relevant to consider good practices in the title given to the datasets, as it is the main description of the data and when searching for datasets a massive repetition of titles can be perceived as duplicity, so it would be highly recommended to provide more precise titles that disambiguate the datasets, adding unambiguity to each dataset.

The inaccuracy found is in line with previous studies. In the case of Gerasimov et al. [55], in a study focused on the citation of datasets by scientific papers, it can be observed how Crossref omits the DOI of access to the dataset, even in publications that reference datasets; most of these documents belonging to publications edited by Elsevier. On the other hand, Johnston et al. [45], attribute its origin to the fact that some repositories issue DOIs for each file, while others assign DOIs at the level of the study, e.g., Zenodo assigns a different DOI to each deposited dataset and also to each of its versions, then OpenAlex identifies each of these DOIs as an independent dataset record. The resulting dispersion leads to a significant lack of precision.