Introduction

The use of digital platforms for delivery of instruction and information at school and at home is now requisite for students at all levels, from elementary school through higher education. The increased use of digital materials alongside paper-based materials in learning environments has motivated research into the efficacy of reading and learning in one format versus the other (e.g., [1–4]), and although there is an overall finding for a paper-based advantage, the outcomes have been characterized by small effect sizes and variability associated with participant, stimulus, and task factors (e.g., [5, 6]). Some reports have indicated no differences between print and digital media with respect to story understanding or comprehension measures [7–10], or test scores [11]. An eye-tracking and EEG investigation [12] revealed no differences across media in fixation times or theta oscillatory power (a correlate of memory encoding) for younger adults, although older adults showed shorter mean fixations and lower EEG theta power when reading from tablet computers compared to e-readers or paper.

As noted by Clinton [1], reports of reading times in different mediums have been inconsistent, with some authors reporting longer reading times for paper than digital environments (e.g., [13–16]), others reporting the reverse [17, 18], and yet others finding no difference between media [7, 19]. Effects of reading time on comprehension accuracy are also mixed. Some of those reporting shorter reading times for computer-based reading also reported a decrease in reading comprehension accuracy in this medium [14, 15]; although Chen and Cantrambone [13] reported shorter reading times for digital texts but no difference in comprehension between mediums; while Singer Trakhman et al. [16] found differences between distinct reading profiles with respect to both speed and comprehension, with overall scores reflecting longer processing times and lower accuracy in digital mediums. However, Kim and Kim [20] found that teenagers read faster in the paper-based condition compared to a digital format with a scrolling feature, and also that they scored significantly higher on exams when they studied via paper-based texts.

Others [1, 19] reported no difference in reading times between the two media but observed higher comprehension scores in the paper-based condition, suggesting a metacognitive moderating factor. This proposition is supported by results showing that outcomes are poorer on computer-based exams when time is constrained, in contrast to self-paced exams, perhaps because students find it more difficult to self-regulate, monitor task progress, and manage goals and time in digital space [21–23].

The effects of medium on metacognitive regulation of learning have been further investigated in a series of studies [22, 24, 25]. Lauterman and Ackerman [25] evaluated differences between mediums under distinct learning constraints (time pressure, interrupted study, and free regulation). Their sample showed no effect of medium when participants were permitted to freely regulate their learning activities, but there was a clear screen inferiority effect under time pressure. These findings interacted with participants’ stated preferences for digital vs. paper-based learning, such that those who preferred digital formats were able to achieve similar results when learning via screens and via paper. However, paper learning still conferred an advantage; efficiency of learning dropped over time for participants working from screens, whereas participants learning from paper maintained learning efficiency throughout the study process. Metacognitive effects of reading medium in adolescent readers were investigated by Ronconi et al. [26], who pointed out the importance of this age group for reading research. The pervasive use of screens and digital media by adolescent students, in and outside of school, has been exacerbated by the pandemic-related shift to long-term distance learning. In addition, motivation for reading has been observed to decrease in this age group as they transition from elementary to middle and high school (described by Chall & Jacob [27] as “the fourth grade slump”; see also [28]). In their study of adolescent reading on tablets vs. printed page, Ronconi et al. [26] measured reading time, reading comprehension (at three levels: main idea, key points, and related information), and calibration bias–that is, the difference between participants’ judgments of their own comprehension performance, and their actual performance. Although reading speed was not affected by medium overall, findings were mediated by sex of the participants, with boys reading faster from screens than paper and also showing greater calibration bias than girls during screen reading. Calibration bias was found to mediate effects of reading medium on text comprehension, at the levels of main idea and key points.

In addition to metacognitive factors, reading comprehension seems also to be moderated by the level of analysis required of participants; for example, recall of overall gist or main idea versus specific details from a written passage. Comprehension scores measured by understanding of gist have been repeatedly shown not to differ between narrative and expository texts, regardless of the medium of text presentation [1, 2, 6, 15], although Ronconi et al. observed a print advantage for comprehension at the level of main idea. The effects of text genre (expository or narrative) are also mixed; for instance, Mangen et al. [29] observed a comprehension advantage for print over digital media for both narrative and expository texts, whereas others have shown that reading narrative texts yields similar comprehension outcomes regardless of the medium [1, 2, 30]. However, in a recent meta-analysis, Salmerón et al. [4] showed that level of comprehension (textual and inferential) was not moderated by text genre (narrative or expository), whether the texts were read on a tablet/computer or paper.

These varied outcomes may be attributed to a number of factors, such as differences in age and grade-level of study participants, their learning goals, and learned strategies. For elementary students, medium of presentation has been shown to have little influence on comprehension of simple texts [7, 8]. Lenhard et al. [14] found that elementary and middle school children were faster at completing a reading comprehension assessment on computer compared to paper under time constraints, but at the expense of accuracy. Støle et al. [31] evaluated the effects of medium (computer vs. print) with 10-year-olds, and found that participants’ reading comprehension scores were higher on paper-based than computer-based tests; this held true across low, medium and high comprehension skills. Critical reading skills of high school and college students were compared by Eshet-Alkalai and Geri [32], who found that younger students performed better when reading news in digital formats compared to paper, while college students performed better on the same task when reading in paper formats.

Against this lack of clarity in the behavioral findings, there has been little brain imaging work to further elucidate the mechanisms that underpin reading in print versus digital formats. Two studies used functional near-infrared spectroscopy (fNIRS) to investigate memory encoding and retrieval processes supporting learning [33, 34]. Lee et al. [33] examined differences in neuronal efficiency during encoding and retrieval of information read from paper versus a digital tablet. Participants read and memorized a mini-novel (the information encoding phase) and then completed multiple choice questions (the memory retrieval phase). Concentrations of oxygenated hemoglobin (HbO) were measured in areas of the prefrontal cortex (PFC) associated with working memory. Performance on the multiple-choice task did not differ between mediums, but lower HbO concentrations were observed for the paper-based group during encoding, suggesting greater encoding efficiency in this medium. There were no differences in HbO during the memory retrieval phase. Anuardi et al. [34] conducted a reading span task on paper and tablets, and participants were asked to memorize an underlined word in each presented sentence. Scores on the reading span task for the two mediums were equivalent, but again prefrontal cortex activity was lower during task performance in the paper-based condition than the tablet condition, suggesting greater resource efficiency when learning from paper.

Umejima et al. [35] also investigated encoding and retrieval from reading in different media, using functional resonance imaging (fMRI). For the encoding phase, participants read appointment information in three conditions: paper notebooks, digital tablets, and mobile phones. They then entered the information on a calendar displayed in each medium. For the retrieval phase participants responded to a series of multiple-choice questions about the scheduled appointments, while fMRI data were recorded. During encoding, participants were much quicker when making calendar entries in notebook than digital formats. During retrieval, neither overall accuracy nor response times for the memory retrieval test questions differed significantly among the three groups, though accuracy for simple recall questions was significantly higher for the paper notebook group compared to the others. Brain activations during retrieval were significantly higher in areas of the pre-frontal cortex for the notebook compared to tablet and phone conditions. Umejima et al. suggest that together the outcomes indicate a possible advantage for information encoding when working in the paper notebook format.

Relevant to the current study, a few studies have applied electroencephalography (EEG) methods to explore the neurocognitive processes underlying reading in different mediums. Kretzschmar et al. [12] recorded electroencephalography (EEG) during their eye-tracking paradigm, that was designed to evaluate whether stated preferences for the printed medium (versus one of two digital devices) correlated with indices of text engagement in young and older adults. Comprehension accuracy did not differ with text presentation medium for either group, but the older adults showed shorter mean fixation durations and lower EEG theta band voltage density when reading from a tablet computer in comparison to an e-reader or a printed page. Younger adults did not show any such differences, and Kretzschmar et al. interpreted the observed differences as relating to limitations on memory encoding and retrieval for the older adults, affected by reduced contrast sensitivity, that could be somewhat ameliorated by the backlit display of the tablet computer. In another EEG study comparing brain responses to reading in different mediums, Zivan et al. [36] examined brain activation differences in EEG power bands for 6-8-year-olds reading expository texts from screens versus printed paper. They observed a higher theta-beta ratio for screen reading, an index of greater difficulty in attentional allocation to a given task. This built on earlier observations by Zivan et al. [37], who had observed increased theta-beta ratios in preschoolers who were exposed to story-telling via screens during a 6-week period compared to those who were exposed to the same stories read live. To the best of our knowledge, however, there exist currently no reports of event-related potential (ERP) measures applied to the question of reading in different media, specifically print vs. digital text processing in children.

For our investigation, we drew upon depth of processing theory, first posited by Craik and Lockhart [38]. The premise of this theoretical framework is that shallow information processing yields less durable episodic memory traces, while deeper processing results in more durable traces. The central claim is that the more deeply information is processed, the more durable the associated memory traces. Kintsch [39–41] has described text comprehension as a dynamic process of constructing meaning from semantic relations among words in the text and stored knowledge about subject matter. According to seminal work by Craik and Tulving [42], processing of verbal text information requires the use of semantic processes (protocols concerning the ways in which words work together to create meaning); hence, text processing strategies for reading may involve drawing on contextual, semantic, grammatical, and phonemic knowledge in systematic ways to work out what information is conveyed by a text. Such strategies would allow an encoded unit to be integrated with knowledge of the world or “semantic memory” (e.g., [43]). At retrieval, informational cues would then tap into this semantic memory structure to reconstruct an initial encoding [42]. Likewise, Kintsch [39] proposed that lexical-semantic information is integrated into comprehension via activation in a network of related concepts. As semantic activation spreads within such networks, it can facilitate word recognition as well as supporting connections between inferentially-related or overlapping propositions.

There exists a large amount of evidence suggesting that text processing is typically shallow, much of it based on psycholinguistic experiments demonstrating the difficulty of identifying local and global anomalies in texts (e.g., [44–47]). However, different factors may influence the relative depth or shallowness of lexical-semantic processing of words. Using text-change tasks wherein words in a passage are changed on a second reading and participants are asked to identify the altered word, Sturt et al. [48] demonstrated that depth of processing is affected by interactions between linguistic focus and semantic distance, while Sanford et al. [49] showed that syntactic complexity is also a factor affecting depth of lexical-semantic processing with failure to recognize the change indicative of shallow lexical-semantic processing for that lexical item. Based on this framework, we propose that the medium whereby readers engage with text/reading material could also be a crucial determinant of differences in depth of lexical-semantic processing, and consequently the durability of the semantic memory structure that is established in response to a written text. Congruous encoding between a semantic structure already established by a reader and a semantic structure associated with a newly encoded unit, as conceptualized in models like that developed by Kintsch [39], should facilitate efficient comprehension of a text, first because a meaning-referenced elaborated trace network is formed, and second because robust congruent semantic encoding also entails alignment with the structure, rules, and organization of semantic memory [42, 50].

Consistent with this view of semantic structure and encoding processes, we hypothesized that depth of lexical-semantic processing is key for reading comprehension and for congruency between existing semantic structures and the semantic structures encoded by probe words. Based on previous research, semantic encoding of text presented on paper is deeper than that of text presented digitally [1, 29]. Therefore, our experimental approach to measuring reading comprehension in the brain made use of a signature of electrophysiological activation associated with semantics in language processing: the N400 event-related potential (e.g., [51, 52]).

The N400 event-related potential (ERP) indexes brain response differences between expected and unexpected stimuli. Since we hypothesized that the encoding of word meaning during the reading experience is critical for comprehension, then we should be able to index shallow vs. deep information processing of text delivered in print or digitally by observing differences in N400 responses to probe words that were selected to be related, moderately related, or unrelated in meaning to written passages. Based on this hypothesis and given that both the culturally prevailing view and data meta-analytic studies [1, 2, 21, 53, 54] suggest that reading digitally presented text promotes shallower engagement than print, our predictions for the electrophysiological index were as follows: 1) In the digital reading condition, the N400 amplitude response to related word probes was predicted to be attenuated compared to moderately related and unrelated word probes, with amplitude differences between moderately related and unrelated word conditions expected to be equivalent; and 2) In the print reading condition, the N400 amplitude response to the three conditions is predicted to be graduated. Specifically, the amplitude measures were predicted to increase in their negativity such that the response to the related words would be most attenuated, followed by the moderately related words, with words that are unrelated to the text passage eliciting the greatest negativity. Differences in the N400 ERP response between the two mediums for the moderately related word conditions may offer essential insights about the neurocognitive processing underlying reading comprehension, and whether readers in some situations process text somewhat more shallowly under conditions of digital text presentation than when processing text via print presentation.

Materials and methods

Data collection

Data were collected in three phases: Phase 1 for the online administration of psychometric assessments; Phase 2 for the EEG recordings and the immediate passage recall comprehension measure; and Phase 3 for the online administration of the passage retention comprehension measure. All informed consent and experimental procedures were carried out with approval of the Teachers College, Columbia University Institutional Review Board (Protocol # 22–173). Study recruitment began in February 2022 and continued until March 2023. Written informed consent for participation in the study was obtained from the parent or guardian of each minor participant, and both verbal and written assent were obtained from all individual participants directly. There were separate approved informed consent and assent forms for each of the three study phases.

Phase 1: Psychometric assessments.

In Phase 1, responses to behavioral assessments were collected by two trained assessment administrators online during video conference sessions. The parent/guardian received a study overview, consent, and assent forms in advance of the scheduled appointment. They were asked to select a quiet setting with the home with minimal distractions where the participant could complete the assessments. Online, the assessment administrator reviewed the materials and responded to questions before the parent/guardian and their child completed the consent and assent forms via a Qualtrics survey. The sessions were approximately 25 to 30 minutes in length and audio recordings were stored for the purpose of second-scoring of measures.

Phase 2: EEG recording.

In Phase 2, participants and their accompanying parent/guardian attended the Neurocognition of Language Lab at Teachers College, Columbia University. Informed consent and assent procedures were conducted, this time via face-to-face conversations (for the verbal assent of minors) and paper-and-pen administration (for written consent from parents/guardians and written assent from minors). High-density EEG data were continuously recorded in NetStation 4.3.1, using a 128-channel HydroCel Geodesic Sensor Net (MagStim Electrical Geodesics, Inc.). Signals were amplified using a NetAmps 200 series amplifier. Samples were collected at a rate of 500 Hz; an online low-pass filter of 200 Hz and high-pass filter of .1 Hz were applied. Impedances were kept below 40 kilo-ohms and were re-checked between blocks. Participants completed sessions in an electrically shielded and sound-attenuated room, seated 65 cm from a computer monitor with a brightness of 75 cd/m².

Each participant was first exposed to texts that were presented via either a paper booklet (print) or a laptop screen (digital). For the digital reading condition, visual readability variables (contrast, brightness, text size) between the laptop screen and the stimulus presentation screen were held equivalent. The order of medium and passage presentation was balanced between participants. We counterbalanced the order of medium (digital-first vs. print-first), and the sequence of presentation for the themed pairs of passages. With this approach, we were able to systematically vary the order of passage presentation across participants, ensuring that each individual passage appeared an equal number of times in each medium, and that each themed pair appeared equally often as the first, second, or third set of texts read by a participant. For each passage, reading time was recorded, and then participants completed two tasks, presented using E-Prime 3.0 (Psychology Software Tools, LLC).

First, participants read one text passage in their assigned first medium. Then they completed the semantic relatedness judgment task in response to single probe words presented on a computer screen. They responded to each word by pressing one button to indicate that a word was related to the passage, and another if they thought the word was unrelated (see Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Timeline for example trials from the semantic relatedness task.

https://doi.org/10.1371/journal.pone.0290807.g001

After the semantic relatedness judgment task, the SVT recall comprehension test items were displayed, and participants were prompted to respond. This procedure was repeated for two additional passages in the selected medium (either print or digital). Then, the medium of presentation was switched, and the process was repeated for another three passages.

Phase 3: Passage retention measure.

Following the EEG recording session, participants were emailed a link to the follow-up Qualtrics retention comprehension survey. As in Phase 1, consent and assent were indicated via Qualtrics survey questions prior to participants completing the comprehension items online. This assessment consisted of the same SVT recall comprehension test items that they had completed during the EEG portion of the study, and was included to provide an indication of information retention. Participants were asked to complete the measure within 24 hours of their visit to the lab, but survey responses were accepted up to seven days after their lab visit.

EEG data analysis

Pre-Processing.

EEG data were pre-processed using the Harvard Automated Processing Pipeline for Electroencephalography (HAPPE; [75]), specifically the event-related extension (HAPPE+ER; [76]). The sensitivity of the HAPPE procedures allows for more trials to be kept and averaged when dealing with high-variance data such as those associated with children. Globally bad channels were detected and removed from the remainder of the pipeline. Across all participants, an average of 93.6% (SD: 4.4%) of channels were good, with a range of 61.2% to 99.2%. A hard wavelet threshold was applied to remove artifacts from the continuous EEG data, a technique that improves upon previous methods of detecting artifacts to retain more of the EEG signal instead of rejecting segments at this stage [76]. A pre-established bandpass filter from 0.1–40 Hz was utilized, and data were segmented from 100 milliseconds (ms) before stimulus presentation to 750 ms post-presentation.

Segmented data were subjected to baseline correction, whereby the average of the EEG recorded during the baseline period for each epoch was subtracted from the post-stimulus period. Bad data within each segment were interpolated and segments were rejected based on a joint probability criterion as well as amplitude cutoffs of -150 and 150 microvolts. Globally bad channels were replaced based on spherical spline interpolation of data from surrounding electrodes, and data were re-referenced offline to the average of the left and right mastoid channels (electrodes 57 and 100).

Participants were excluded from further analysis if more than 40% of trials for any passage were rejected. Of 65 participants, one was excluded due to low numbers of trials in the final analysis and others due to inability to use behavioral data; analyses were therefore based on data from 59 participants. For all retained participants, at least 50% of trials were deemed usable; on average, 66.5% of trials were usable (SD: 5.8%; range: 53.2% to 82.3%). The numbers of trials per participant did not vary significantly across medium or passage. For the related and unrelated conditions, error trials (trials in which a participant had misidentified a related word as unrelated, or vice versa) were also excluded from further analysis. All trials were kept for the chimera condition, as their intermediate level of relatedness makes them hard to accurately categorize in a binary fashion. In the print medium, 910 related trials, 2,024 chimera trials, and 1,961 unrelated trials were used in subsequent analyses; in the digital medium, trial numbers came to 909 related trials, 2,016 chimera trials, and 2,028 unrelated trials.

Baseline-corrected epochs for each word condition were then averaged together for each individual participant, providing individual averages per medium and condition. Individual event-related potentials were interrogated for mean amplitude of the target component within an a priori-established time window, 300–500 milliseconds post-stimulus. Individual averages per condition were then grand averaged to generate group ERP waveforms.

Montaging.

N400 montages vary across studies (e.g., [77]). The electrode montage for investigation of the N400 component was selected based in part on the N400 context and discourse literature [78–84]. Fig 2 below indicates the montage of interest; all plots of the derived event-related potentials relate to this montage.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Montage for N400 analysis.

Electrodes included in the analysis montage are indicated in green: electrode numbers 54, 55, 61, 62, 67, 71, 72, 76, 77, 78, 79.

https://doi.org/10.1371/journal.pone.0290807.g002

Results

Phase 1: Psychometric assessments

All participants completed a set of standardized assessments, plus an assessment of auditory working memory. Table 1 below provides mean scores and standard deviations for each assessment for all included participants (n = 59). We applied a criterion to include only those participants whose scores on all assessments were within 3 standard deviations of the sample mean for each assessment. All participants met this criterion. While we did not have any outliers that needed to be removed from the data analysis, a range of abilities was represented within this sample of middle school students.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Mean scores and standard deviations for psychometric reading assessments.

https://doi.org/10.1371/journal.pone.0290807.t001

We evaluated correlations between these measures to determine which assessments of working memory (digit span and LSST) were correlated with the measures of language skill from the WRMT-III. A review of the relationships between different measures revealed no significant correlations between the Listening Sentence Span Task (LSST) and traditional working memory measures (forward and backward digit span: r = .088, p = .507 and r = .155, p = .241, respectively). However, there was a significant correlation between the digit span scores (r = .559, p < .001). The LSST measure was found to be positively correlated with both word attack and passage comprehension scores (see Table 2, below).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Correlations between scores on working memory and language assessments.

https://doi.org/10.1371/journal.pone.0290807.t002

These findings indicate that (a) working memory, as measured by both the Listening Sentence Span Task (LSST) and the digit span Forward and Backward subtests, was within a typical range across the group of participants; (b) that working memory is important to control in experimental approaches to reading comprehension; and (c) that working memory is not likely to be a factor influencing neurophysiological response differences between passages in this experiment.

Phase 2: Event-Related potentials

We examined the grand-averaged N400 responses to all probe word conditions (i.e., related, chimera, unrelated) within each medium (i.e., following texts presented in digital vs. print media). Plots displaying the grand-averaged waveforms of participant responses for each probe word condition within each presentation medium condition are shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Grand-averaged waveforms in response to the semantic relatedness task following digital text presentation.

Includes all retained participants, correct response trials only for related and unrelated word conditions, and all responses to chimera words (no error criterion for this condition). Variance around the mean waveforms is shown as shadow. Green: Related condition; blue: Chimera condition; red: unrelated condition.

https://doi.org/10.1371/journal.pone.0290807.g003

A mixed model two-way ANOVA was conducted to compare the factors of text medium and probe word category. No significant main effect of medium was detected (F (1, 58) = .158, p = .692, η² = .003), but the main effect of word category was significant (F (2, 116) = 13.533, Greenhouse-Geisser corrected p < .001, η² = .189). Across both mediums, N400 amplitudes in response to related words were significantly more positive than responses to either chimera words (mean difference = .974 μV, t (117) = 2.96, p = .011, d = .273) or unrelated words (mean difference = 1.624 μV, t (117) = 4.40, p < .001, d = .405). Amplitude values in response to chimera words were also significantly more positive than to unrelated words (mean difference = .650 μV, t (117) = 2.84, p = .016, d = .262). A significant interaction between medium of presentation and word category was also found (F (2, 116) = 4.017, Greenhouse-Geisser corrected p = .025, η² = .065). Further examination of simple main effects revealed no significant differences between responses to each probe word category following presentation of either print or digital texts (p > .10 in all conditions).

Significant differences were observed when comparing amplitude values between word probe categories within each of the two mediums. Following text presented in the digital medium, the N400 response to related words was significantly different from the response to both chimera words (mean difference = 1.644 μV, t (58) = 3.562, p = .002, d = .463) and unrelated words (mean difference = 2.204 μV, t (58) = 4.055, p < .001, d = .527). The response to chimera words was not significantly different than the response to unrelated words in the digital text condition (mean difference = .560 μV, t (58) = 1.718, p = .273, d = .223). Following texts presented in the print medium, the response difference between related and unrelated words was significant (mean difference = 1.043 μV, t (58) = 2.755, p = .023, d = .358). The difference between related and chimera words was not significant (mean difference = .304 μV, t (58) = .798, p = 1.00, d = .103), but a significant difference between chimera words and unrelated words was observed (mean difference = .739 μV, t (58) = 2.546, p = .040, d = .332). All t-tests were controlled for multiple comparisons via Bonferroni correction within medium.

Experimental results: Behavioral findings

Following each passage, participants were asked to decide whether each word shown on screen was related or unrelated to the passage they had just read. Related words were both scored as “correctly identified” if the participants indicated they were related to the passage; unrelated words were similarly coded if they were marked as unrelated. Responses are summarized in Table 3 below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Percentages of words in each probe category identified as related to the preceding text.

https://doi.org/10.1371/journal.pone.0290807.t003

A two-way repeated measures ANOVA revealed no interaction between medium and category (F (1, 58) = .179, p = .674) or main effect of medium (F (1, 58) = .505, p = .480); however, there was a main effect of condition (F (1, 58) = 277.261, p < .001). Planned comparisons (t-tests) confirmed significant differences in accuracy between conditions, with unrelated words being identified significantly more accurately than related words (following text reading in the digital medium: t (58) = -16.314, p < .001; print medium: t (58) = -15.382, p < .001).

Reaction times for each word were also recorded for each participant, and are summarized by probe word category in Table 4 below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Mean reaction times for each probe word category.

https://doi.org/10.1371/journal.pone.0290807.t004

A two-way repeated measures ANOVA revealed a significant interaction between medium and category (F (2, 116) = 4.278, p = .016). There was no significant effect of medium, confirming that reaction times to individual words following reading in print or on a screen did not differ. A significant simple main effect was found for word category (F (2, 116) = 23.334, p < .001), and planned comparisons (paired-samples t-tests) confirmed that, in the digital medium, reaction times to the related words were significantly longer than to either the chimera (t (58) = 3.352, p < .001) or the unrelated words (t (58) = 4.922, p < .001); however, reaction times did not differ significantly between chimera and unrelated words (t = 2.684, p = .005). In the print medium, reaction times to the related and chimera words were both significantly longer than to the unrelated words (related vs. unrelated: t (58) = 4.389, p < .001; chimera vs. unrelated: t (58) = 5.216, p < .001), but the reaction times were not different between related and chimera words (t (58) = -0.790, p = .433). To verify these findings, we conducted additional generalized linear mixed model (GLMM) analyses; the additional analyses confirmed that participant and item variance did not reduce the effects of any fixed factors. The GLMM is further detailed in Supplementary Materials (S1 File).

Comprehension accuracy

Immediate recall comprehension task.

The reading of each passage was followed by a set of eight sentence verification items to evaluate participants’ comprehension of the preceding passage. The eight items were of four different types, as described above: explicit, paraphrase, meaning change, and unrelated. These four types of questions were designed to probe different aspects of understanding of the text and different levels of difficulty with respect to recall as well as recognition of ideas and concepts from the texts.

Responses to the sentence verification items were not recorded for 9 of the 59 participants due to software malfunction during data collection. Thus, the results below include data for 50 participants. Accuracy for these items is presented below in Table 5, separated by medium.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Mean percent correct responses for each sentence verification item type, immediate presentation.

https://doi.org/10.1371/journal.pone.0290807.t005

A two-way repeated measures ANOVA was run to determine the statistical significance of the interaction between medium of presentation and accuracy across question types. No significant interaction was found (medium x item type: F (3, 147) = 0.89, p = .449), and the main effect of medium was also non-significant (F (1, 49) = 0.561, p = .457). However, the main effect of question type was significant (F (3, 147) = 85.105, p < .001), and planned comparisons (t-tests) revealed that accuracy for each of the question types was significantly different, in the following order from most to least accurate: Unrelated > Explicit (t (198) = 5.528, p < .001); > Paraphrase (t (192.19) = 3.586, p < .001); > Meaning Change (t (173.14) = 8.107, p < .001).

Delayed (retention) comprehension task.

In addition to collecting responses to the sentence verification items about each passage immediately following presentation, we asked participants to answer the same questions again within 24 hours after completing the lab session. However, the survey responses were accepted up to 168 hours (seven days) following the lab session. The goal was to gauge retention of the information presented in the passages, and to compare retention between media. Mean accuracy for each item type is presented below in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Mean percent correct responses for each sentence verification item type, delayed presentation.

https://doi.org/10.1371/journal.pone.0290807.t006

The pattern of responses to the delayed sentence verification task is similar to that of the immediate recall comprehension evaluation: meaning change items were responded to with the lowest accuracy, followed by paraphrase items. In this case, the accuracy for explicit and unrelated items appears equivalent, while overall accuracy is slightly lower for delayed vs. immediate evaluation. These results were confirmed with statistical analysis. A two-way repeated measures ANOVA was conducted, and no significant interaction between medium and item type was found (F (3, 147) = .195, p = .90). The main effect of medium was also non-significant (p = .75), but the main effect of item type was found to be significant (F (2.37, 115.91) = 41.240, p < .001). Planned comparisons (t-tests) showed that accuracy for unrelated and explicit items was not significantly different, but both were responded to significantly more accurately than paraphrase and meaning change items. Accuracy of responses to the paraphrase question type was greater than to the meaning change question type.

Additionally, we sought to identify significant differences between immediate recall comprehension (SVT items presented during the experiment run) and later retention accuracy (SVT items completed via online survey after at least 24 hours elapsed). Two separate two-way repeated measures ANOVAs were run, to observe the effects of time (immediate vs. delayed) and item type separately across the two mediums. For the digital passages, a significant interaction between time and item type was found (F (3, 132) = 3.204, p = .030), and the main effects of time (F (1, 44) = 13.02, p < .001) and item type (F (3, 132) = 49.697, p < .001) were also significant. Similarly, for the print passages, there was a significant interaction between time and item type (F (3, 132) = 5.448, p = .001) as well as significant main effects (time: F (1, 44) = 13.020, p < .001; item type: F (2.26, 99.64) = 60.197, p < .001). The effects of time reflected that total accuracy was significantly higher in the immediate responses to comprehension items than for the delayed responses (t (397.73) = 2.187, p = .03), while the interaction was driven by a difference in accuracy rates for the unrelated SVT items: on average 16.67% higher when responded to immediately after the passage reading task, compared to delayed responses (t (180.88) = 4.698, p < .001). The significant main effects of item type reflected that accuracy rates continued to follow the general pattern previously observed (Unrelated > Explicit: t (379) = 3.669, p < .001; > Paraphrase: t (392.61) = 5.814, p < .001; > Meaning Change: t (365.09) = 11.66, p < .001). With respect to delayed responses, the meaning change item type again yielded significantly fewer accurate responses than all other question types (Unrelated: t (165.42) = 11.699, p < .001; Explicit: t (192.3) = 13.85, p < .001; Paraphrase: t (189.09) = 8.433, p < .001). Paraphrase item types yielded significantly fewer accurate responses than the explicit and unrelated items (t (197.57) = -4.671, p < .001; t (186.27) = -4.273, p < .001, respectively). However, responses to the explicit and unrelated items did not differ with respect to accuracy (t (182.24) = 0.327, p = .744).

To determine whether performance on the SVT was affected by working memory, we calculated correlations between performance on the each of the working memory assessments (Digit Span Forward, Digit Span Backward, and the LSST) and accuracy on the SVT items, both immediate and delayed. Spearman correlations (Bonferroni-corrected for multiple comparisons) were used due to the ordinal nature of the LSST and possible non-normal distributions of the variables. All correlations can be viewed in the Supplementary Materials to this paper (S2 File). The only correlations that remained significant when applying the Bonferroni correction were between LSST scores and immediate SVT scores, both total (r = .459, p = .000801, corrected p = .036) and responses to the paraphrase items (r = .467, p = .0006287, corrected p = .028). No other correlations between comprehension measures and working memory measures were significant. Such a result could indicate that an increased ability to recall sentences may be helpful in accurately assessing the paraphrase items, but it does not suggest that working memory was a significant mediator for the different neurophysiological responses to digital and print items in this experiment.

Discussion

As alluded to above, this study took place against a complex background of research and environmental factors that contribute to the importance of the findings. The COVID pandemic was a time of unprecedented disruption to our educational systems, with as-yet little understood consequences for students. Amid pre-existing doubts about the impact of digital media on the development of reading and related skills, children were abruptly forced into online instruction and even more of their engagement with text, at all levels, now happens through various digital devices. These disruptions highlighted a challenge already being faced by educators: to understand how reading comprehension and learning are changing in the age of digital information. This investigation of the neural correlates of depth of lexical-semantic processing following text reading across mediums in middle-school students is the first to apply event-related methodologies to this question, and is novel in its use of the N400 as an index. We drew upon the depth of processing theory introduced by Craik and Lockhart [38] to provide a theoretical framework for the investigation, alongside Kintsch’s [39–41] view that text comprehension is a dynamic process of constructing meaning from semantic relations among words in the text and one’s stored knowledge about subject matter. We proposed that how readers engage with text/reading material may be a crucial determinant of differences in depth of processing for lexical-semantic information contained in a text, consequently affecting the robustness of semantic memory structures that are established in support of reading comprehension. We extended the standard applications of the N400 to provide an index of processing depth associated with two mediums of text presentation: digital (via a laptop screen) and print (via a printed page).

We predicted that N400 responses to single words following texts presented in digital and print formats would differ. These predictions were largely supported by the data presented above. The waveforms indicate distinct brain responses to the semantic probe words that were presented following text reading across the two mediums. Consistent with our predictions, when passages were read on a laptop (digital), responses to subsequently presented words in the chimera (moderately related/moderately unrelated) category evoked activations similar to those associated with words that were unrelated to the text. This finding can be observed in the waveforms (Fig 3), and is supported by the lack of statistical significance in amplitude differences between chimera and unrelated word responses in the digital condition. The N400 waveforms in these two conditions can be observed to differ significantly from the response to related words.

In the print medium, we predicted that the N400 responses for the three conditions would be graduated, with unrelated words producing the greatest negativity, the response to related words being the most attenuated, and responses to chimera words falling between. However, the N400 waveform patterned differently than expected (Fig 4). Mean amplitude values within the N400 time window were significantly different between related and unrelated words, and between chimera and unrelated words–consistent with our predictions. However, contrary to prediction, the amplitude differences in response to related and chimera stimuli were not significant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Grand-averaged waveforms in response to the semantic relatedness task following digital text presentation.

Includes all retained participants, correct response trials only for related and unrelated word conditions, and all responses to chimera words (no error criterion for this condition). Variance around the mean waveforms is shown as shadow. Green: Related condition; blue: Chimera condition; red: unrelated condition.

https://doi.org/10.1371/journal.pone.0290807.g004

Within the context of the depth of processing theory [38–41], the primary experimental manipulation in this study related to the chimera word stimuli. Prior behavioral studies have suggested that reading on a digital device promotes shallow reading. When classifying the chimera words, we stated that these stimuli could be perceived as related or unrelated given the center clustering of their word relatedness ratings. Whether chimeras are perceived as related or unrelated to the text may depend on the strength of the encoded memory traces, and/or the lexical-semantic activations, that are established during text discourse processing. Therefore, perception of chimera words as unrelated words would be consistent with shallow lexical-semantic processing as hypothesized in digital text reading, whereas chimera words perceived as related would be consistent with deeper lexical-semantic processing as observed in print reading.

Behaviorally, there was no distinction between classification of the chimera words by study participants following the digital or print presentations of texts; in both conditions, chimera words were most frequently identified as being “unrelated” to the text. The longer reaction times to related words than words in other probe conditions likely reflect response competition (e.g., [85]), and the patterning of reaction times between chimera and related words in the print condition, and between chimera and unrelated words in the digital condition, is an expected finding given the study prediction that more robust semantic networks were expected to develop following exposure to print vs. digital texts. However, observations of the ERP responses to chimera words provide a deeper insight.

The semantic judgement task prompted participants to decide whether presented probe words were related or unrelated, potentially shaping brain responses specific to the task at hand. Therefore, how deeply a participant read the text would likely contribute to whether they perceived the chimera word probes as either related or unrelated. Keeping in mind the general expectation of shallowness in text processing [44–47], we were looking for relative differences in depth of lexical-semantic processing related to the medium of preceding text presentation. These seem to be borne out in the waveforms and statistics: responses to the chimera words track with responses to the unrelated words in the digital condition, and with responses to the related words in the print condition. The observed responses to the chimera word condition may index the robustness of context models for the text: if robust models are created, chimeras can be situated within the model affording greater processing efficiency, whereas when such words are situated within a less robust contextual model, as would be generated under shallower reading, the opposite would be expected. Under this interpretation, these ERP responses align with the study hypothesis and may indicate that a more “robust” semantic network was derived in response to texts presented in the print medium. Hence, we propose that the N400 brain responses observed are consistent with a finding of deeper text processing in print compared to digital media.

The increased use of digital materials alongside paper-based materials in learning environments has motivated many studies on the efficacy of reading and learning in one format versus the other (e.g., [1, 2, 6]). Investigations of reading comprehension and learning measured in terms of reading ability, reading rate, eye movement, and factual recall, have found no differences in student performance between working in the two mediums (e.g., [9, 12, 86, 87]. The present study is the first to evaluate depth of processing for print and digital informational texts in middle-school children using a brain measure (N400 ERP). Our findings contribute to this landscape by providing insights about the neurocognitive processing underlying reading comprehension. The study outcomes reveal differences in how the brain processes expository text when presented in digital and print mediums, with the former suggesting more shallow engagement and the latter conferring deeper engagement. This effect could indicate a “print advantage” with respect to depth of processing, in support of previous behavioral research [2].

Conclusions

As we have described here, in this study we systematically applied neurophysiological methods to understand the implications and neural underpinnings of reading in print vs. digital media, at a crucial stage in literacy acquisition. An important question raised by these findings concerns the implications for classroom instruction of reading and learning via paper-based texts compared to texts delivered on digital platforms. The question is particularly relevant given the near ubiquitous use of digital platforms for delivery of instruction and information at school and at home.

For reasons related to study delimitations and limitations we think it too early to generate a set of recommendations for adaptation in the classroom. However, we do think that these study outcomes warrant adding our voices to those of Delgado et al. [2] in suggesting that we should not yet throw away printed books, since we were able to observe in our participant sample an advantage for depth of lexical-semantic processing when reading from print. Applications for digital reading should not be dismissed, either: the observation of a potential print advantage does not negate the value of rapid access to information that could be supported by digital reading. It may be that classroom practices should strategically match reading strategies and mediums to task, such that printed media are employed when deeper processing is required while digital access to text is utilized for other needs.

Another reason not to dismiss digital reading platforms is their potential to benefit children with reading disabilities. Research in this area suggests that digital reading strategies may be utilized in support of reading proficiency [99] and comprehension [100] in this population. However, reading disabilities are vastly heterogeneous, and there are concomitant difficulties with identification (e.g., [101]), alongside a corresponding array of interacting causal mechanisms that need to be described at multiple levels—at least, behaviorally, neurophysiologically, and genetically (e.g., [102]). Hence, further investigations of the effectiveness of digital and print text presentations for dyslexia and other reading disabilities will be needed.

Finally, in this study, we predicated a relationship between the activation of semantic networks related to text propositions, and a subsequent classification of those relationships instantiated in single word presentations. We observed differences in brain responses indexing the relatedness of single words to previously-presented texts in print and digital mediums. These data were suggestive of lexical-semantic information being processed in the context of a previously-established network of related concepts, and may indicate that the extent to which such processing is facilitated depends, among other variables, on the medium of text presentation. However, in general, much more work is needed to elucidate the relationships between concepts, words, and the comprehension of discourse or texts.

Supporting information

Acknowledgments

A special thanks to the incredible research team from the Neurocognition of Language Lab at Teachers College, including Sarah Bennett, M.S., Girija Chatufale, M.S., Kenny Fabara, M.S., Kai Gilchrist, M.S., Zeina Haidar, M.S., Luiza Lodder, M.S., Emily Mantaro, M.S., Michela Thomsen, M.S., and Huiyu (Bonnie) Yang, M.S. We also thank Dakota Egglefield, Ph.D., for her contributions to every aspect of our behavioral data collection; and Alexa Monachino for support during data analysis applications. We want to thank Dr. Kara Federmeier for her invaluable suggestions on earlier versions of this article, and Dr. Trey Avery for his continued support with the technology and the data. We thank Prof. Bryan Keller at Teachers College for guidance on Gaussian mixture modeling. The detailed comments provided by two anonymous reviewers of an earlier draft were invaluable. This research would not be possible without the wonderful families and participants who let us borrow their magnificent brains–our deep appreciation for their interest, support, and time.