Definitions of unstructured data and digital unstructured data enrichment

We developed the following working definition for digital unstructured data in accordance with the literature: unstructured data are raw data that fulfill at least one of the following conditions: a) are not in a pre-defined structure (e.g., tables), b) and/or data that are not ready-to-use and require substantial pre-processing or feature extraction efforts to extract the desired information for analyses (e.g., free text data stored in table) [3,4,12,18]. Substantial pre-processing and feature extraction efforts refers to the process of transforming raw unstructured data into data that are ready-to-use, credible and meaningful for a research question and research project [23]. For example, unstructured sensor data might require signal processing techniques such as noise filtering, data interpolation or outlier removal for meaningful information extraction. We added the second condition to emphasize that digital unstructured data can exist either in its original unstructured format or within a structured format. For both cases, the data need significant pre-processing before they can be used, which adds complexity to the process of data enrichment. This is also reflected in the literature where sensor data or biosignal data, although available in a structured format, are considered to be unstructured data because of their associated high pre-processing efforts [1,24,25]. High pre-processing burden associated with our working definition of digital unstructured data has connections to the concept of big data. Big data is commonly defined by the 4V attributes (volume, velocity, variety, and veracity) [4,26,27], and applies to structured as well as unstructured big data. We consider unstructured big data to represent a subset of digital unstructured data.

We further define digital unstructured data enrichment as the use of digital unstructured data in combination with other data sources to augment the available evidence base and contribute to relevant domain knowledge in health research and clinical practice. We envision a situation where digital unstructured data are already available (e.g., clinician notes or collected wearable data), and researchers wish to utilize these digital unstructured data in combination with structured survey data from the same patients. Within this definition we also consider the complexity and challenges associated with digital unstructured data itself, which can create difficulties when linking it with other data sources.

Included types of digital unstructured data

In this review, we consider text data, including unstructured data from electronic health records (EHR), unstructured big data, and sensor data from wearables and other devices, including electroencephalogram (EEG) as common sources of digital unstructured data. Despite their widespread use in health research, we did not consider imaging and video data in this review, as these data are often bound with additional technical challenges in the enrichment process that may not generalize to other unstructured data types [28,29].

Search strategy

We conducted a systematic narrative review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement [30] (S1 PRISMA Checklist). Fig 1. PRISMA flowchart provides the flowchart for the screening and study selection process. Our study selection was guided by the inclusion and exclusion criteria displayed in Textbox 1 and Textbox 2, respectively. We performed our search on PubMed and PsycInfo for 1) general overview articles, 2) primary research articles, and 3) articles describing databases, all including relevant information on digital unstructured data enrichment. The complete search syntax including all keywords can be found in S1 Text.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flowchart.

https://doi.org/10.1371/journal.pdig.0000347.g001

Screening was conducted in two phases. In the first step, we screened the titles and abstracts from the studies based on the inclusion criteria (Textbox 1). In the second step, we performed a full-text screening of the articles selected in the first step and excluded articles that matched the criteria outlined in Textbox 2. In both steps, one investigator (JS) assessed all articles and a second investigator (PD) performed checks on a randomly selected sample of articles for each screening phase. Any disagreements were discussed and, if required, a decision was achieved through the principal investigator (VvW).

Data extraction and synthesis

Data extraction was systematic yet developed iteratively. The initial data extraction included study characteristics and attributes relevant to our research question. During the full-text screening, seven overarching topics related to digital unstructured data enrichment were identified and used for data extraction. The topics were the following: 1) medical field and subfield of the study, 2) main motivation for digital unstructured data enrichment, 3) data enrichment scope (e.g., gathering accurate information about disease severity), 4) type(s) of unstructured data, 5) limitations of unstructured data (e.g., quality/completeness), 6) challenges of digital unstructured data enrichment, and 7) proposed or discussed approaches for overcoming the mentioned challenges. The extracted data are to be found in S1 Data.

A narrative synthesis of the results was conducted to provide an overview of the challenges and proposed solutions related to digital unstructured data enrichment. This choice was also motivated by the heterogeneity of included studies that ranged from overview papers to original research studies. To address study aims 1 and 2 (i.e., description of common challenges and their possible solutions), the extracted study data on the topics 5 and 6 (i.e., limitations and challenges associated with enabling digital unstructured data enrichment) were grouped into challenge areas. The challenge areas include not only topics directly connected with data enrichment, but also related to the unstructured data use itself, as this is an essential requirement to enable digital unstructured data enrichment. For each challenge area, relevant possible solutions to tackle the challenges were summarized. For study aim 3 (i.e., providing guidance), we developed a preliminary checklist based on findings from our literature review to guide early study planning and feasibility assessment steps for studies that aim to include unstructured data in their methodology. To this end, the identified challenge areas from study aim 1 were reformulated into checklist questions and ordered according to the common study planning stages in health research [31]. Finally, the checklist was complemented and refined based on domain-specific expertise represented by the interdisciplinary team.

General description of included studies

A description of the 30 included articles [3–5,8–13,16,18,32–50] is presented in S1 Table. The most frequently discussed types of unstructured data sources in the selected articles were electronic health records (n = 13) and sensor data (n = 7). The most commonly cited motivations for digital unstructured data enrichment were to include more objective measures in their research, for example, to improve understanding of disease mechanisms and disease prediction, and to strengthen the existing evidence base in precision medicine, real-time monitoring, and real-world data collection.

The most prevalent challenge areas in enabling digital unstructured data enrichment were: 1) the lack of meta-information for unstructured data (n = 6), 2) standardization issues (n = 21) 3) data quality and bias in data (n = 13), 4) infrastructure and human resources (n = 12), 5) finding suitable analysis tools, methods and techniques (n = 15), 6) alignment of unstructured data with a research question and design (n = 11), as well as 7) legal and ethical issues (n = 11). These challenges span across all study stages involving data in a health research study: from data collection to data interpretation. Definitions of the main challenge areas and a brief explanation of their relevance for health research are given in S2 Table.

Challenge areas

In the next sections, we summarize the seven identified challenge areas associated with enabling digital unstructured data enrichment in the field of cardiology, neurology and mental health and the proposed possible solutions to address them.

1. 1. Lack of meta-information for unstructured data

CHALLENGES: Lack of meta-information (e.g., describing data structure and properties or sample population) has been acknowledged as an obstacle for unstructured data findability, integration, interchangeability, and interpretation [3,5,8,18]. Insufficient meta-information might limit the translation of a study’s findings into clinical practice [3,8] as important contextual information, such as information on the time in which the data were collected might be missing to assess the usability and correct interpretation of data and, consequently, their combination with other data sources [3].

POSSIBLE SOLUTIONS: Proposed possible solutions included the standardization of meta-information (e.g., through a standardized format for meta-information through open science standards), which may also resolve issues of data interpretation and their alignment with research questions and designs [3–5,18,43]. Specifically, one suggestion was to provide information for four important aspects in each study: subjects included in the data, context of collection, observations, and time of data collection [3]. Moreover, a greater availability of standardized meta-information was suggested, as this would facilitate the determination of suitable specific unstructured data for a given research question and their linkage with other data sources [3,4].

1. 2. Standardization issues

CHALLENGES: The most frequently discussed challenge (n = 21) was the lack of a standardized framework for the description of disease phenotypes (e.g., symptoms, clinical presentation), as well as a lack of commonly defined terminologies, ontologies, and data labels [3–5,8–10,13,16,18,32–34,36,37,43,45,48,50]. For example, different terms may be used for a seizure with alteration of consciousness by different physicians [32] or for the administration of a specific dose of a given drug [33]. These issues are particularly prevalent in EHRs or clinical annotations [18] where, for example, terminologies and phenotyping may differ across healthcare settings or change over time [4,32,38]. This poses a notable challenge for data linkage because relevant features might vary in the different data sets and prevent their proper linkage. There is also an observed lack of standardized data management methods [8,45] and regulatory standards to guide and assess the use of novel technology and their associated unstructured data in clinical applications such as clinical trials [3,12].

POSSIBLE SOLUTIONS: In most articles, harmonization of data formats, data models, terminologies, ontologies, and analytical tools, as well as working practices were proposed as possible solutions to standardization issues [3–5,9,11,33,34,39]. A consensus of standards across the entire data flow [3–5,9,33,36], the effective use of datasets [5], data optimization [34], data consistency [38,49] and replicability of the studies [4,11] were also suggested as a means to foster data sharing. The adoption of unified data standards was considered to be important in both academic and industry settings [9,36].

To improve standardization efforts and data sharing, the systematic adoption of FAIR (Findable, Accessible, Interoperable, Reusable) Guiding Principles for scientific data management [51] was proposed [9]. Other authors mentioned the need for specialized organizations to promote harmonization of terminologies in health research [4,9]. An example is the consortium behind the Fast Health Interoperability Resources (FIHR) standard to enable “interoperable communication and information sharing between various healthcare systems” [9].

1. 3. Data quality and biases in data

CHALLENGES: Data quality of unstructured data was frequently cited as an important challenge for evidence creation [4,5,8,11,18,34,37,38,40,50]. Unstructured data are often collected for purposes other than research and may lack systematic collection methods and scientific rigor, thus often leading to missing data [4,5,8,11,18,34,38,47]. In medical records, for example, missing data can occur because health care professionals may omit some information or because of patients’ refusal to share data [8]. The challenge of data quality is reinforced by data inconsistencies and inaccuracies [4,8,11,34,38,40]. Other recurrent challenges stem from biases in data collection—mainly in the form of selection and information bias [5,8,18,34,37–39]—and confounding [4,39]. Selection bias was mentioned, for example, in the context of studies where the sample comprised only of individuals who have the digital literacy skills or interest to share unstructured data from social media or wearable sensors [5,12]. Information bias, such as observer bias, was often mentioned in the context of making errors with data in EHRs use and big data analytics [8,37–39]. Further biases may establish themselves in analyses if processing algorithms were trained on biased data [18,38,39]. Finally, the quality and continuity of data might be negatively impacted by technical issues that can arise, for example, by software updates of wearable sensors [5].

POSSIBLE SOLUTIONS: Several strategies were proposed for assessing and ensuring data quality [3–5,8,18,38]. For studies that use digital health technologies, one study cited a recommendation from the European Medicines Agency (EMA) urging researchers to define “small, well-defined, meaningful measures followed by a data-driven development path” [5,52]. Furthermore, possible data quality issues should be considered for all study phases, including preprocessing, feature extraction or analysis [18]. First, preprocessing should yield only verified and valid dataset that properly combines the unstructured data with other data sources, for example by ensuring that study samples are representative of the populations that are being studied. Second, following feature extraction, data should be critically assessed for their validity and meaning. Finally, analytical methods should be aligned with the research goals of description, prediction, or prescription of the study in such a way that bias is reduced.

Other studies highlighted the need for a data quality standard checklist such as Data Access Quality and Curation for Observational Research Design (DAQCORD) [18,53]. This checklist should provide a priori guidance for planning of large-scale study data collection and pre-processing [54], thereby countering the pervasive practice of post-hoc methods for data cleaning [18]. Other proposed possible solutions included the use of meta-information to increase data quality, to detect potential biases in the data [3], to enable cross-referencing of multiple data sources involving the same individuals, as well as to encourage the comparison of results [4]. Imputation procedures for addressing missing data, as well as algorithms for checking data quality were also recommended [18]. Furthermore, the inclusion of study participant feedback can inform data collection and processing and improve the relevance of study findings for the intended target population [5].

1. 4. Infrastructure and Human Resources

CHALLENGES: Several studies pointed out challenges related to infrastructure availability, including databases, or open-source platforms [3–5,8–10,18,33]. Infrastructure challenges can be particularly problematic when healthcare data are spread across multiple medical systems that lack connection or interoperability, thus creating isolated data clusters [9,10,33,36]. Difficulties in data linkage can also emerge when information system architectures cannot accommodate data standardization and other linkage processing tools [18]. Furthermore, the lack of skills and formal training opportunities for infrastructure utilization or inadequate knowledge of novel statistical tools and methods for combining unstructured data with other data sources can inhibit their use in health research [4,5,8].

POSSIBLE SOLUTIONS: Improvements such as searchable catalogues, databases and the availability of open platforms can mitigate infrastructure-related challenges [4,5,9,11,16,18,32,39]. Similarly, the availability of infrastructure for the storage and combination of unstructured data with other data sources can enable collaborative efforts, facilitate standardization, and foster the alignment of unstructured data with good research question development and research design [4]. Furthermore, the availability of secure collaborative platforms and repositories for data sharing through open science can enable independent knowledge gain and foster new research studies [32,34]. Meta-databases or catalogues that facilitate the discovery of open data and linking data across public repositories can also facilitate digital unstructured data enrichment [4,9]. Several studies further suggested that platforms for combining different datasets from various sources should have a modular, flexible, and scalable structure [9,18,32] and recommended to define the purpose and goals of such platforms during their development [11,39]. Open data and open software repositories also provide more opportunities for external validation of novel algorithms or (electronic) clinical outcome measures [16]. Finally, awareness about novel digital unstructured data enrichment methods, their methodological requirements, and the need for specialized training opportunities should be increased [4,11].

1. 5. Finding suitable analysis tools, methods, and techniques

CHALLENGES: The complexity of analyses and appropriate methodological choices associated with the unstructured data enrichment in health research are challenges that were addressed in multiple studies [4,8,9,18,32,38,39]. Typical features of unstructured data such as high volume or complexity may be overwhelming for researchers due to a lack of methodological knowledge and might discourage researchers to use them in combination with other data sources for enrichment purposes [8]. Furthermore, the validity of results may be decreased by algorithms that are either not trained properly or may need recurrent fine-tuning to ensure that they create a model representative of its intended purpose and without biases [8,16,18]. Working with unstructured data requires specific expertise, typically from data scientists. However, the lack of supply of data scientists or the failure to build effective collaborations with external experts were also cited as impediments to managing the complexity of unstructured data [4,9,39]. Furthermore, there is a lack of guidelines and standards to guide decisions on which tools, methods, and analytical approaches to use when using unstructured data in health research [4,8].

We further observed a discrepancy in approaches to reduce the complexity of unstructured data (e.g., using feature extraction) in our studies. While some authors argued that complexity reduction is a feasible and appropriate method to enhance the combination of unstructured data with other data sources, others voiced concerns that complexity reduction can also reduce richness of unstructured data—particularly in the context of EHRs [13,32].

POSSIBLE SOLUTIONS: The complexity of unstructured data calls for increased collaboration among different experts. The increasing need for interdisciplinary efforts among health researchers, data scientists, biostatisticians, and health-care professionals was highlighted by most sources [3–5,11,16,32,34,37]. Some authors emphasized the need for a novel profession that combines expertise in health research and informatics [4,34]. Many also called for greater attention to trainings of health researchers regarding novel methods for using and combining unstructured data with other data sources [4,11,16]. The need for specific sets of skills, resources, and guidelines for the successful implementation of big data tools into clinical workflows was further mentioned as a requirement to manage unstructured data complexity [38]. Furthermore, some authors called for more efforts to develop and establish validated algorithms to process and integrate data [8]. One suggestion was to “provide AI with more ‘functional’ information, such as domain-specific medical reasoning processes and policies based on heuristic-driven search methods derived from human diagnostician methods” [9]. It was also suggested that the use of multiple data sources can improve the performance of ML models, such as through combination of theory-driven and data-driven approaches [50]. In the field of mental health, it was advised to complement data-driven research with qualitative research to strengthen the relevance and meaning of results [39].

1. 6. Alignment with a research design and/or research question

CHALLENGES: The difficulty of finding suitable datasets and their subsequent, critical evaluation for clinical relevance was discussed from several perspectives [4,8,16,34,37–39]. One study strongly warned against adjusting the research agenda to the data that are available [39]. Furthermore, the fact that unstructured data or technologies generating these data were not designed for scientific purposes [11,16,37,39] might lead to misinterpretation of the data [39]. The lack of contextual (meta) information, for example about the data generation process, and observational nature of many sources of unstructured data may limit the value of the data for their use in robust, replicable confirmatory analyses (e.g., regarding disease etiology or intervention) [37,39]. The need for further and robust validation of results or outcomes from unstructured data analyses was a further topic of concern [11,38,39,50]. For example, predictive models need further validation before being integrated into clinical settings [38] and informing clinical decision-making [8]. Similarly, while linked EHRs are suitable for generating research questions, unstructured data should not be used for influencing clinical practice without prior validation [4].

POSSIBLE SOLUTIONS: It should be ensured that unstructured data used in combination with other data sets are relevant for a research question and desired therapeutic effect [16]. When working with data from digital health technologies, the EMA recommendation framework—that was developed with the collaboration with industry representatives with the aim to provide insights and guidance on validation and qualification processes of digital technologies [54]—could be consulted for guidance with research question design [5].

Another recommendation was to align large-scale research projects using unstructured data with clinical priorities and outcome-focused research [11]. Similarly, the choice of analytical tools depends on the goals of health research: description, prediction, or prescription [18]. Thus, setting clear research goals might help with the choice of appropriate analytical tools and methods. Finally, unstructured data should be used rather with complementary and enrichment purposes than as a replacement of other traditional methods or datasets [12,16,38,39].

1. 7. Ethics & Legal Issues

CHALLENGES: The most frequently mentioned ethical challenges concerned privacy protection, informed consent and preservation of individual agency over data use [4,5,8–11,18,38,40]. Further challenges connected with digital unstructured data enrichment include inappropriate patient profiling [38] and decreased participants diversity due to low digital literacy skills reducing some participants’ contributions to certain types of unstructured data (e.g., from social media use) [5]. Furthermore, current deidentification and anonymization practices may still allow patient-linkage when combining different data sets. This is, for example, enabled when a combination of data on unusual physical conditions of a patient from a local hospital or a combination of gender, age and admission date might be unique enough to identify a subject and connect it with consumer-level data [13,18].

POSSIBLE SOLUTIONS: Strategies for preserving data privacy and security were discussed in multiple studies [4,5,8,10,11,16,18,43]. Some authors proposed to develop a new social contract and a broad consent model to balance the benefits of data usage and privacy concerns [4,8,11]. Unified rules for data governance across fields and sectors might contribute to systematic privacy protection and confidentiality [18], such as through unified procedures for data anonymization. Additionally, the importance of engagement with regulatory agencies in early stages of research was emphasized to ensure alignment of unstructured data processing with best practices [5]. Finally, independent agencies or governing bodies were proposed to oversee and ensure safe data sharing, preservation of intellectual property and valid applications [16,18].

Collaborations with all stakeholders

Several sources stressed the importance of stakeholder collaboration in health research when combining different data sources for knowledge enrichment [5,9,11]. The inclusion of with public and patient advocacy groups and other relevant stakeholders was highly recommended [11] to ensure wide public acceptance and patient trust [4,43]. Broad stakeholder involvement was also seen as crucial to increase data sharing and to minimize wasted efforts from research study duplication [5]. Collaborative efforts among academic and commercial organizations (e.g., digital device or sensor manufacturers) can facilitate large-scale data integration and create synergies [9,11]. Stakeholder and patient engagement during in the digital unstructured data enrichment, analysis, and interpretation provides relevant context and feedback on the meaningfulness of results [5,11].

Documentation and transparency

Proper documentation and transparency during the entire data flow were repeatedly mentioned as essential steps to achieve reliability, replicability, reproducibility and validity of studies, as well as facilitating the standardization efforts to ultimately enable unstructured data enrichment [5,9,37,39]. The EMA framework emphasizes documentation as an important means to achieve reliability, repeatability, accuracy, clinical validity, generalizability, and clinical applicability of the novel methodologies [5]. In the context of digital health technologies, United States Food and Drug Administration (FDA) recommendations suggest documenting the device and algorithm input and output, and to provide plans for data loss minimization, missing data handling, or patient inclusion for results. Furthermore, the FDA recommendations call for transparency of all processing steps from raw data to algorithm and at all data workflow stages [5]. Transparency regarding the analysis process can also assist with the assessment of whether study findings were clinically significant [39]. Specifically, studies relying on large databases will produce many statistically significant, but clinically meaningless results. This “overpowering” of statistical tests by large sample sizes should be made transparent through reporting of effect size determinants and complementation by clinical interpretation [37].

Proposal for a feasibility and planning checklist for unstructured data enrichment

Many studies highlighted the need for further research and guideline development on best practices to use and integrate unstructured data in health research [4,9,11,16,33,34,38]. In Table 1, we provide a set of guiding questions to inform early study planning and the assessment of the feasibility of studies. These questions are based on the described challenge areas, which have been expanded to align with the breadth of proposed solutions from our review.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The checklist for early study planning and the assessment of the feasibility of studies using digital unstructured data.

https://doi.org/10.1371/journal.pdig.0000347.t001

Summary of findings

Our systematic narrative review provides an overview of challenges and best practices associated with the combination of unstructured data with other data sources in the fields of cardiology, neurology and mental health, which we refer to as digital unstructured data enrichment. In our review, we identified seven prevalent challenge areas in enabling digital unstructured data enrichment: 1) the lack of meta-information for unstructured data, 2) standardization issues, 3) data quality and bias in unstructured data, 4) infrastructure and human resources, 5) finding suitable analysis tools, methods and techniques, 6) alignment of unstructured data with a research question and design, as well as 7) legal and ethical issues. For each challenge area, we summarized proposed possible solutions. Additionally, we derived two additional recommendations that span across all challenge areas. We also compiled literature and experience-based checklist questions to inform initial study planning about the feasibility of research studies aiming to complement existing health data with digital unstructured data.

Description of main requirements and solutions to enable digital unstructured data enrichment

All our studies revealed challenges of unstructured data use in health research, many of which might endanger scientific rigor and quality of health studies that may inhibit digital unstructured data enrichment. For example, the frequently unclear suitability of digital unstructured data to address concrete research questions or allow for proper research study design [4,8,16] may lead to possible biases, threatening the external and internal validity of studies. The validity of studies might also be endangered by applying unsuitable analytical tools and methods. Furthermore, the findings of the study may lack generalizability limiting its use to specific research tasks and questions (e.g., hypothesis-generation) [4,8,16,37]. The lack of meta-information might hinder a proper interpretation of the data and consequently limit their use for enrichment purposes. Further problems are that the data can be placed so centrally that any bias will be strongly reflected in the results. The most discussed challenge of standardization issues might hinder replicability and generalizability of research studies. Finally, ethical and legal issues, such as the risk of patient re-identification when disparate data sources are combined, pose additional challenges to digital unstructured data enrichment.

While many of the challenges to enable digital unstructured data enrichment are not specific to the use of unstructured data and are well known (e.g., data quality or standardization issues), other challenges, such as difficulties to align data with research questions or challenges pertaining to special skills or infrastructure needs, may be aggravated with the use of unstructured data for enrichment purposes due to their complexity. One of the key challenges might be the lack of open and collaborative platforms that can foster not only joint standardization but also validation efforts [3,4,9]. Oftentimes, the attractive characteristics of unstructured data that might add value to research are the ones that pose the most challenges. The data granularity and large, often international, population-based samples can enhance disease understanding or monitoring but also lead to methodological challenges, for example, regarding validity and choice of tools for analyses [8,39].

The review revealed that the possible solutions to enable digital unstructured data enrichment are less frequently and systematically discussed than the challenges. In particular, several sources discussed challenges without referring to the existing solutions or offering proposals for possible solutions. The difficulties with enabling digital unstructured data enrichment are also reflected in the fact that possible solutions may need to cover multiple challenge areas. Interdisciplinary collaboration, open science and transparency were the most mentioned possible solutions. Overall, the reported possible solutions and additional recommendations are important to sustain interchangeability, validity, reliability, generalizability, and reproducibility of studies.

Requirement for guidance on and reporting of digital unstructured data enrichment

Our review also revealed that, despite the widespread use of unstructured data in health research, there is a lack of a systematic approach and guidelines for researchers to address challenges specific to digital unstructured data enrichment. Several of the selected articles acknowledged the need for more guidance [4,9,11,16,33,34,39], oversight or monitoring from agencies [5,13,16] and interdisciplinary teamwork and exchange to establish methodological approaches in the context of utilizing unstructured data in combination with other data sources in health research [4,8]. Only a few studies directly mentioned existing frameworks and standards such as EMA recommendations [5], FAIR principles [9], openHR [4] or DAQORD framework [18] in the context of unstructured data use. Recent efforts to provide guidelines are mainly focused either on a specific type of unstructured data or on specific challenges, for example, guidelines and standards for the use of social media data [61,66], guidelines regarding the use of EHRs [67,68], checklists and frameworks for evaluating the measurements made by digital technologies [69] or algorithms used for data analysis [70,71]. However, it is up for discussion whether these specific frameworks and guidelines are suitable to provide general guidance on challenges connected with digital unstructured data enrichment.

Our findings also reveal an underreporting of information relevant to digital unstructured data enrichment in health research. Even though digital unstructured data are used in combination with other data sources, the specific challenges connected with the linkage of these data sources are only sparsely reported. This is reflected in this review, where the reported challenges could be applicable to other types of data, such as big data, and to challenges of unstructured data without data enrichment. This rather general presentation of results points to the insufficient methodological guidance as well as reporting of challenges specific to digital unstructured data enrichment in currently available research. The lack of reporting of specific challenges and barriers of digital unstructured data enrichment could be addressed by providing researchers with reporting guidelines that include unstructured data-enriched methods. For example, current reporting guidelines such as STROBE [72] do not cover unstructured data-enriched analyses. Furthermore, the assessed studies rarely reported challenges relevant to combining unstructured data with other health data sources. Rather, the studies usually provided a description of the data collection and preprocessing steps, such as issues of noisy and missing data. However, they often lacked a description of data limitations or strategies to ensure high data quality for the analyses. In light of the growing volume of unstructured data in health research, experience sharing should be increasingly encouraged—either in published literature (e.g., also in appendices) or in other outlets. The lack of reporting and unavailability of guidelines not only hampers study reproducibility but also presents missed opportunities for learning and capacity building. Similarly to the guidance and research on the linkage of structured datasets,¹¹ there is a need to provide more guidance and research on specific challenges connected with the linkage of digital unstructured data with other data sources and to assess their quality.

All this points to a growing need to define systematic ways of how to approach digital unstructured data enrichment in health research. This need is enhanced considering the interdisciplinary nature of studies working with unstructured data enrichment. The numerous challenges linked with unstructured data use or digital unstructured data enrichment should be reflected in systemic guidance on how to properly combine digital unstructured data with other data sources in health research. This can also facilitate interdisciplinary collaborations that are essential for digital unstructured data enrichment. Our review identified a special need for guidance to establish common standards to enable digital unstructured data enrichment to help researchers in the first stages of study planning and to assess the feasibility of studies combining unstructured data with other data sources [5,16,32]. The checklist derived from our review provides a first, pragmatic step towards classifying challenges and developing methodologies in health research involving digital unstructured data enrichment. In next steps, we hope to encourage specific research fields to dive deeper into our proposed checklist and adapt it to terminologies and issues that might be of a greater relevance in their respective research fields.

Limitations

First, our definition of “digital unstructured data” may not be universal and definite. Further, although based on a systematic search and extraction process, we restricted our search to a few specific research fields due to the prevalent and growing use of unstructured data in these fields. We did not include books and book chapters and included only articles that mentioned the digital unstructured data enrichment already in the abstract. This might have led to the exclusion of articles that discussed the limitations of digital unstructured data enrichment in the discussion part. Therefore, our overview is likely not comprehensive. Furthermore, we a priori excluded imaging data and bioinformatics data from our literature search, which are an important source of unstructured data, but are often analyzed with highly specialized tools. For example, only considering various image acquisition methods and different techniques to handle noise in the image data would already add complexity that cannot be generalized to other unstructured data types [28,29]. In the systematic narrative review, we did not specifically discuss challenges and obstacles that are linked with learning algorithms used for unstructured data linkage or data analysis or interpretation. However, there is also precaution and guidance needed for choices about learning algorithms. Machine learning and deep learning algorithms are not immune to errors, biases and other limitations that can negatively impact validity, objectivity, and reproducibility of studies. Finally, the included studies predominantly reported challenges of unstructured data use in health research in the enrichment context. This observed underreporting limits our review results from describing specific challenges and solutions directly associated with digital unstructured data enrichment. However, we find that this is an important finding and calls for more efforts to report challenges and methods of digital unstructured data enrichment in health research.