Background

9^th March 2020 represents an unforgettable date in recent Italian history. On that date, the former Italian Prime Minister declared the start of the national lockdown [1]. The pandemic has dramatically changed people’s life worldwide, impacting all sectors, with repercussions also for the school sector. Indeed, different countries closed educational institutions to prevent the spread of COVID-19 [2], causing an interruption of regular learning. The same has been done in Italy, where schools remained closed for almost 40 weeks [3, 4]. After some days/weeks of total school interruption, most countries, including Italy, started to adopt distance learning [5, 6]. However, distance learning is a complex process that needs a careful design to lead to effective learning [7]. Due to the unpreparedness that led to the implementation of the distance learning, several studies revealed some achievement gap as an effect of school closure on the students’ learning quality at different educational levels [8, 9].

König & Frey [10] conducted a meta-analysis including 18 studies from different countries on primary and secondary school students, showing that school learning tended to be negatively affected by school closures, with a particular impact on primary school students. These results can also be found in Betthäuser et al. [11] and Di Pietro [12], comprising respectively 42 and 39 studies from different geographical areas on primary and secondary education students–even tertiary in Di Pietro [12]. Both meta-analyses agreed that students were unable to recover from the learning loss caused by school closures due to COVID-19. Several studies agreed that school closures mainly affected the most marginalized students, both scholastically [13, 14] and emotionally [15, 16]. These students are identified as vulnerable due to disadvantaged educational, environmental, and social factors, including socio-economic status (SES), cultural differences, learning disabilities and special education needs (SEN). Also, the students included in these categories are accompanied by additional scholastic support compared to their classmates, which is challenging to dispense online, and may thus represent an additional source of difficulties in times of pandemic. This shows how the learning environment is essential to the student’s well-being [17–19]. As a consequence of the online transition, students experienced a period of forced isolation at home, away from other places of interest and their peers, with negative psychological consequences [20]. Several studies highlighted a fluctuation in the emotional behavior of students during the pandemic, with a significant increase in anxiety, depression, sadness [21, 22], suicide attempts [23], and behavioral problems such as restlessness and inattention [24].

The debate on the quality of distance learning was also accentuated by the presence of problems related to teaching and assessment. Among the difficulties related to teaching, there are technological complications, such as the presence of adequate support tools, computers, good quality broadband. Another challenge is creating an ideal study environment [25–27]. Furthermore, ensuring the proficiency adequate of both teachers and students, along with their parents, in the use of e-learning instruments and practices is essential [28, 29]. Engzell et al. [30] reported that the Netherlands–whose schools were closed for eight weeks–did not experience such technological problems. However, the study refers to a country with an excellent school performance according to the Organization for Economic Cooperation and Development (OECD) average, high academic autonomy, and the greatest presence of broadband among European countries. Despite the lack of such technological problems, learning losses were also found in Dutch students, highlighting that low socio-economic background and parental education/support negatively influenced students’ learning [31].

Based on the information previously mentioned, it appears that the closure of schools due to COVID-19 and the subsequent distance learning affected students across educational levels worldwide, with implications extending beyond academic performance. It is plausible that similar circumstances have also transpired in Italy. Indeed, according to the Italian school calendar [32], Italian students lost over a year of regular learning during the pandemic. The national institute for the evaluation of the education and training system (INVALSI) annually tests the learning outcomes in language, mathematics, and English of Italian 2^nd and 5^thgraders. The 2022 INVALSI data, compared with 2019 and 2021 cohorts, revealed a containment of the pandemic effects on learning of Italian language and a general slight worsening in mathematics [33]. Moreover, these data show that students with a linguistic disadvantage achieved lower results compared to an average student of the same age, highlighting the post-pandemic difficulties of students with low SES and a migrant background. However, although these data also consider socio-economic, cultural, and migratory aspects, they do not consider further aspects that may have influenced students’ learning outcomes during the pandemic. Several international studies highlighted how the transition to distance learning led to serious emotional consequences and behavioral changes on students [15, 16, 34, 35]. Assessment of distance learning is still a widely debated topic, as digital skills among teachers and parents are heterogeneous–both in terms of the ability to use and of the availability of adequate tools [36, 37]. There is also international agreement that distance learning has further affected students already on the margins of school community, such as SEN students or those with cultural and economic difficulties, increasing social inequalities [21, 38, 39]. For all these reasons, the Italian situation may represent a good case to be studied to understand the effects of long-time school closures. We decided to focus our investigation on primary school level, as it is a crucial period for pupils’ learning outcomes.

Aim of the present study

The current investigation intends to provide an overview of the effects of the school closure due to the COVID-19 pandemic on the learning outcomes of Italian primary school students (target population) according to their opinion, and the opinions of their teachers and/or parents. As shown in the Introduction, the school closure affected not only the academic achievement of students, but also several dimensions. Hence, our work intends to identify all dimensions related to the scholastic well-being invalidated from the school closure due to COVID-19. Moreover, we wanted to investigate if there was agreement or not between the opinion of the students themselves, and their parents and teachers’ opinion, as well as set off differences or similarities with international literature. Therefore, our work intends to answer three specific Research Questions:

• Research Questions #1: Beyond academic performance, are there other aspects linked to scholastic well-being of primary school students that were affected by school closures?
• Research Questions #2: Is there agreement between the opinions of parents, teachers, and students regarding the different effects of school closures on Italian primary school students?
• Research Questions #3: Are there any differences between the effects of school closures on Italian primary school students and the students in other countries as appears in the international literature on the topic?

With these aims in mind, we intend to follow a rigorous search strategy to identify all studies focused on the effects of school closures due to COVID-19 on Italian primary school students, based on the point of view of their parents, teachers, and the students themselves. We are aware that there are already numerous reviews on the effects of the pandemic on education. However, we believe that our research has many points of originality, and it has the potential to add new and important information to the literature in the field. First, our systematic review is the only review addressing the pandemic’s effect on education on an Italian primary students population. This represents an important adding point to the literature not only because of the difference in nationality per se, but also because the investigated phenomenon impacted Italy, with respect to all the other European countries, in a peculiar and severe manner that made Italy a unique source of investigation of the impact of this phenomenon on education, regardless of the nationality. Thus, we strongly believe that the Italian point of view in this field is not just one other point of view together with the one of other countries, but it has a dramatic privilege to be the point of view of the European country in which children experienced the longest school closures and one of the greatest COVID-19 impact in terms of deaths and infections in the first pandemic months. Telling about the impact of COVID-19 pandemic on the Italian students’ population means telling the international literature about the European country in which this tragic event has potentially had the greatest effect.

Literature search and search strategy

The articles were searched according to the PRISMA guidelines [40]. The study was preregistered on the Open Science Framework (https://doi.org/10.17605/OSF.IO/P4FGM –March 17^th, 2023). On February 2^nd, 2023, the first and second authors searched on 4 different databases: Web of Science; Pubmed; EBSCOHost (including PsycArticles, PsycInfo, Psychology and Behavioral Sciences Collection); Scopus. The same researchers examined the preprint servers PsyArXiv, Open Science Framework, and PROSPERO, searching papers, preprints, or preregistered works on the same topics. Limiting the identification to the topic section (title, abstract, keywords), they used the following search string: (COVID OR CORONA OR “SARS-COV-2”) AND (ITAL*) AND (SCHOOL OR “PRIMARY SCHOOL” OR “ELEMENTARY SCHOOL” OR SCHOLASTIC) AND (LEARN*). Next, we used both the backward (the works cited in the selected articles) and forward (the studies that cited the considered articles) methods to identify any further studies.

Inclusion and exclusion criteria

We retained any article published between March 9^th, 2020 (first COVID-19 lockdown in Italy) and February 2^nd, 2023 (article search date); written in English or Italian. Study participants had to be Italian primary school students and/or their teachers/parents, but always with a specific focus on Italian primary school students’ leaning during COVID-19 lockdown. We excluded studies that did not satisfy those criteria. Furthermore, we excluded non-journal papers, editorials, dissertations, letters to authors, comments on published articles, and grey literature in general.

Screening process

Eligible studies were screened using Rayyan [41]. After removing duplicates, the first two authors assessed the titles and abstracts of all identified articles compared to the predefined criteria, discussing any disagreements later with the last two authors. The same researchers reviewed the text of the selected publications and performed reliability using the Cohen’s Kappa (κ) to measure agreement between two reviewers. Cohen’s Kappa was interpreted using Landis & Koch [42] convention as Poor (< 0.00), Slight (0.00–0.20), Fair (0.21–0.40), Moderate (0.41–0.60), Substantial (0.61–0.80), and Almost Perfect (0.81–1.00) Agreement.

Quality assessment

Given the heterogeneity of the studies included, we used different tools to assess the quality of the evidence and the risk of bias. We used the Risk of Bias in Non-randomized Studies–of Interventions tool (ROBINS-I) [43] for the assessment of the studies that used an experimental methodology [44]. All the other studies used an observational methodology, therefore we used the Newcastle-Ottawa Scale (NOS) [45] in three different versions, depending on the type of study: cohort studies (NOS–C); case-control studies (NOS–CC); cross-sectional studies (NOS–CS). Our systematic review includes: 3 cohort studies [46–48]; 1 case-control study [49]. The remaining 29 are cross-sectional studies. The first two authors conducted an independent assessment of study quality, calculating percent agreement and Cohen’s K [50]. Any concerns were discussed subsequently with the other authors.

Data extraction

380 papers potentially relevant were found: 176 papers from Scopus, 116 from Web of Science, 61 from PubMed, and 27 from EBSCOhost. See Fig 1 for a PRISMA flowchart of the literature search process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flowchart of the literature search and screening process.

https://doi.org/10.1371/journal.pone.0303991.g001

First, we detected and removed 181 duplicates. We screened the titles and abstracts of the remaining 199 articles, removing 146 papers: 82 because of the wrong outcome (no reference to student learning); 63 for the wrong population examined (pre-school, middle-school, high-school, university, or others); 1 for the paper’s language (different to English or Italian). Four records were excluded as the full text was not available. The first and second author independently reviewed the remaining 49 eligible studies, according to the criteria defined in advance. This process led to the exclusion of 17 papers: 13 for a different outcome (not focused on Italian primary school students’ learning); 3 for the population (different from an Italian sample); 1 for the type of publication (dissertation). Then, we included 2 articles consulting the papers’ references via backward and forward search. We computed a Cohen’s Kappa (κ) of 0.80 –a substantial agreement level [42]. We discussed emerging discrepancies with the last two authors. We retained 34 articles suitable for inclusion.

Risk-of-bias assessment

The first two authors independently assessed the methodological quality of the studies with ROBINS-I [43], and NOS [45] in three different versions (cohort, case-control, and cross-sectional studies). When the independent evaluation was finished, the two authors obtained a percentage of agreement of P(a) = 95% and a Cohen’s K of 0.64 –a substantial agreement level [50]. All differences were discussed subsequently. Regarding the single study with a non-randomized experimental methodology [44], no concerns and/or high risk of bias were highlighted, and the study was judged to be at low risk of bias for all domains (see S1 Fig). The same was true for the three studies evaluated through the NOS-C [46–48], that obtained a score of 7 (good score level–see S1 Table). All these studies presented an unsatisfactory value only in the Outcome section, due to the lack of information about the method of evaluating the results (e.g., independent blind assessment). Only one study [49] was evaluated through the NOS-CC, and obtained a total score of 5 (satisfactory–see S2 Table). The unsatisfactory scores are mainly due to the lack of information about the rate of non-responders and the self-report data collection method (survey). 29 studies were evaluated through the NOS-CS (S3 Table). Only one study achieved a good level of evaluation [51] and only 2 a satisfactory criterion [52, 53]. There is still much debate within the scientific community on the inclusion of studies highly vulnerable to bias within systematic reviews and meta-analyses [54, 55]. We decided to maintain studies with high risk of bias for different reasons. First, assessing the risk of bias does not necessarily imply the possibility that a study is biased, but only that it is susceptible to bias [56]. Second, we decided to preserve our research by Tipe II error, thus the possibility to exclude informative papers because of a too stringent methodology impossible to adopt in consideration of the pandemic period and the covered topic. Third, it should be noted that these high risks of bias levels are mainly due to the choice of sampling methods (snowball) and the evaluation methods used (surveys or structured interviews). This choice is to be considered binding as the pandemic period in Italy consisted in both the closure of research institutions and the impossibility of researchers to conduct experimental studies involving schools.

An overview of the effects of the school closure due to the COVID-19 pandemic

In addition to the scholastic achievement, several studies included in our systematic review found that the school closure due to Covid-19 affected Italian primary school students on an emotional level (see, for example, Ref. [57]), particularly for those who were already on the margins of the educational system prior to the COVID-19 pandemic, due to socio-cultural, socio-economic difficulties, and/or difficulties due to one or more diagnostic conditions (e.g., Ref. [53]). According to the studies included in our work, some students may have encountered difficulties during distance learning, from a technical-practical point of view, such as: difficulties in following the online lesson, technical problems or competence of teachers/students with the new learning methodology (for example, Ref. [58]). Hence, we categorized the articles according to the specific dimensions investigated, namely: emotional aspects of the learning context; marginality-inequality effects; didactic problems; learning quality. See Table 1 for a synthesis of the studies included and the investigated dimensions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Synthesis of studies included.

https://doi.org/10.1371/journal.pone.0303991.t001

The section called “Emotional aspects of the learning context” includes studies investigating emotional aspects experienced by students in distance learning during the pandemic period. By the general term “emotional” we mean studies that considered both short-term (e.g., fear, joy, sadness, anger) and long-term (e.g., anxiety, frustration, serenity) emotional states. We also include cognitive processes linked to emotional responses, such as attention and motivation to learn [83, 84]. We divided the studies according to the involved sample’s opinion: parents; teachers; students. The study by Crescenza et al. [65] was the only one that investigated these aspects from the perspectives of both teachers and parents. Therefore, we included the study in both sub-categories. In summary, 2 studies collected information from a sample of teachers, 8 from parents, and 1 from students. We repeated the same distinction also in the other sections.

Within the section “Marginality-inequality effects”, we included the papers that have at-risk students within their sample: students with specific learning disorders (SLDs), neurodevelopmental disorders (e.g., autism spectrum disorder, ASD; attention deficit and hyperactivity disorder, ADHD), or neurodiverse students in general. We also included students with SEN, linguistic-cultural difficulties, and participants with low SES. Of the 20 papers included, 7 collected information from a sample of parents, 6 from teachers, and 7 from students. However, Baschenis et al. [44] and Inguscio et al. [72] collected data through questionnaires to parents and tested children through psychometric tests; therefore, we placed them in both subsections. Consequently, the section includes 7 papers that collected information from a sample of parents, 6 from teachers, and 9 from students.

The new teaching approach adopted during distance learning has brought difficulties in evaluating student performance, as well as didactic problems encountered by parents or teachers themselves. The category called “Didactic problems” includes studies that considered problems encountered during distance learning, such as the provision of the teaching itself, technical problems, or competence of the teachers/pupils in the use of the new learning tools, issues with the online assessment of students’ achievements. This section includes 11 studies that dealt with these issues in the pandemic period: 2 according to the parents’ opinion; 9 to the teachers’ opinion. No articles evaluated this topic according to students’ opinion.

The category called “Learning quality” includes studies that compared academic performance during the pandemic with pre-pandemic performances. Of the 9 papers included, 3 collected information from a sample of parents, 3 from teachers, and 3 from students.

Therefore, this work includes 10 articles dealing with emotional aspects of the learning context, 20 related to marginality-inequality effects, 11 on didactic problems, 9 on learning quality. Some articles investigated two or more areas of learning; therefore, they were included in each dimension investigated. See Table 2 for sample characteristics and demographic information about the studies included.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Descriptive synthesis of the included studies.

https://doi.org/10.1371/journal.pone.0303991.t002

Parents’ opinion.

Fourteen studies have delved into the effects of distance learning during the COVID-19 pandemic from the perspective of parents [44, 53, 57, 63, 65, 68, 69, 72, 74, 76, 78, 80–82]. Online questionnaires were the main method of data collection (see Table 3 for a description of the studies included in this section). Consensus among these studies is that distance learning has posed significant challenges for children. Concerns about the quality of learning during distance education were widespread among parents, with many expressing dissatisfactions with the effectiveness of online instruction [57, 69, 76]. Parents have reported difficulties in supporting their children academically [65], leading to disruptions in children’s emotional well-being, including increased aggression and restlessness [63, 76]. There has also been a noticeable decrease in attention, concentration, and engagement during home study sessions [80, 82]. Additionally, concerns have been raised about increased screen time and decreased participation in school and extracurricular activities [57, 78, 82]. Despite these challenges, one study found a positive effect on children’s sleep and family relationships due to parents’ remote work [74]. Furthermore, several parents reported amplified difficulties faced by marginalized and neurodiverse students during distance learning. Regarding to the marginality-inequality condition, parents of students with special educational needs (SEN) or neurodevelopmental disorders have expressed dissatisfaction with the support received and highlighted the negative impact on their children’s academic performance and psychological well-being [53, 80, 81], along with higher levels of fidgetiness [65]. Similarly, parents of dyslexic children have reported significant difficulties in online learning compared to parents of neurotypical children [44]. Inguscio et al. [72] investigated the perception of online learning by students (and their parents) with or without hearing loss (HL). The presence or absence of HL did not influence the parents’ estimation of the difficulties with the distance learning. Over 50% of the parental population defined the distance learning as quite useful, although with lower scores than the direct relationship (in class) between child-teacher and schoolmates. Furthermore, instructional problems have been noted, with some children unable to participate in distance learning due to technological constraints or perceived inefficacy of remote learning methods [68, 76]. The investigation conducted by Gaggi et al. [68] found that 4.5% of the children in their sample (22 out of 490) never participated in distance learning during the pandemic, due to family technological difficulties or school impreparation, or even because some teachers considered it a suboptimal method for teaching young students. More than 70% of the parental sample of Scarpellini et al. [76] reported that they were not in favor of the learning methods imposed by the distance learning, and many of them (80%) found themselves in great difficulty in giving technological help to children.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Parents’ opinion.

https://doi.org/10.1371/journal.pone.0303991.t003

Teachers’ opinion.

Fourteen studies have investigated teachers’ opinions on emotional aspects, marginality-inequality effects, instructional issues, and the quality of their students’ learning [51, 52, 58–62, 64–67, 70, 71, 75]. Only two studies investigated emotional status of students according to their teachers [51, 65]. The teachers’ sample by Doz et al. [51] observed a decline in students’ attention and motivation, particularly among those lacking parental support. Over 60% of the surveyed teachers by Crescenza et al. [65] acknowledged the importance of prioritizing the emotional well-being of students and families. However, they also expressed concerns about their perceived lack of proficiency in fulfilling this role. It is indisputable that the pandemic has had disruptive effects on student populations who already presented a difficult clinical-family picture; within the school context there was inevitably a loss or a slowdown in learning [58, 59]. Teachers reported that their students with special educational needs (SEN) have faced significant challenges, including changes in relationships, decreased autonomy, and passive participation [62, 66], or even no online lesson activities for these students, given the difficulties in delivering their personalized teaching plans online [64, 71]. While some teachers recognized the benefits of distance learning, such as innovative teaching methods, many encountered difficulties in building relationships with students and adapting to technological limitations [52, 70]. As regards didactic problems, digital competence has emerged as a critical issue, with widespread resistance to digital methods among older teachers [51, 52, 60, 66]. This resistance has negatively influenced student performance, underscoring the importance of ongoing training and support for teachers [70]. Additionally, assessing students’ academic performance in a remote environment has posed significant challenges, necessitating adaptation of assessment methods. Capperucci et al. [61] reported that the most common methods in primary school, during the pandemic period, were homework (78.7%), tests (63.3%), oral questions (62.0%), unspecified exercises (59.0%). Over 60% of teachers who responded to the survey by Ferretti et al. [67] did not believe assessment was feasible in distance learning environments. See Table 4 for a description of the studies on teachers’ opinion.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Teachers’ opinion.

https://doi.org/10.1371/journal.pone.0303991.t004

Students’ opinion.

As highlighted in the preceding sections, students have faced various emotional challenges and learning difficulties due to schools closure during the pandemic period. This section provides a summary of the results of 10 studies that investigated students’ own opinions [44, 46–49, 72, 73, 75, 77, 79]. See Table 5 for a description of every study included in this section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Students’ opinion.

https://doi.org/10.1371/journal.pone.0303991.t005

Segre et al. [77] conducted interviews revealing increased levels of anxiety (over 75%) and concerns about family members contracting COVID-19 among almost 80% of students in their sample. Dyslexic students faced heightened difficulties with online lessons and homework compared to their peers without specific learning disorders [44], although such difficulties in following online lessons were not highlighted by the sample of SLDs students investigated in Marchese et al. [73]. However, their sample (21 students from a single primary school, in southern Italy) cannot be considered representative. Inguscio et al. [72] did not find a significant impact of hearing impairment on academic well-being, while Contini et al. and Ranieri et al. [48, 75] highlighted exacerbated educational disparities, particularly among socioeconomically disadvantaged students. The pandemic has worsened existing disparities among students, especially those from lower socioeconomic backgrounds, as Borgonovi & Ferrara [47] demonstrated a non-linear relationship between the pandemic’s effects on learning and Economic, Social, and Cultural Status (ESCS) [85], with medium-low SES students experiencing the greatest setbacks. Termine et al. [49, 79] also noted significant challenges for students with neurodevelopmental disorders (NDDs), attributing the difficulties to the inadequacy of distance learning for their individualized educational needs. Termine et al. [49] collected information on a sample of 49 patients between 6 and 18 years old affected by tic disorders (common hyperkinetic movement disorder in childhood). More than 50% of their sample reported an increase in the severity of tics during the pandemic, with an increase in restlessness in 30% of the participants and in perceived pain in 21% of them. According to the regression analyses carried out by Termine et al. [49], the persistence and increase of tics was directly associated with a learning difficulty in the distance learning modality. In 2021, Termine et al. [79] conducted a study also in a hospital setting, but with a larger sample size, including patients with tic disorders (n = 46) and not only: 781 with SLD, 125 with ADHD, 42 with ASD. The analyses carried out by Termine et al. [79] reported a strongly negative effects of distance learning condition in students with Neurodevelopmental Disorders (NDD), adapting less than the control group at this new condition for several reason: distance learning difficulties were more pronounced for children with NDD; students with NDD felt more often reprimanded by their parents during the COVID-19 lockdown; they tended to feel less the absence of their peers, indicating a negative impact on the socialization.

It is interesting to note that no studies have been found that highlight whether Italian primary school students experienced difficulties in using the new distance learning methodology, or any technical didactic problems (e.g.: lack of adequate devices, quality of connection Internet). Regarding the quality of learning, three studies used INVALSI data, revealing adverse effects on academic performance, particularly in mathematics, among primary school students during the pandemic. Borgonovi & Ferrara and Bazoli et al. [46, 47] observed a decline in mathematical skills, while reading scores varied across regions. In fact, the results by Borgonovi & Ferrara [47] were in line with those by Bazoli et al. [48], except for the distribution of learning loss on the Italian territory. While Bazoli et al. [46] divided the area of residence into regions, identifying a territorial homogeneity with respect to learning loss, Borgonovi & Ferrara [47] divided the area into Italian provinces (thus a smaller division compared to Bazoli et al.), identifying a somewhat heterogeneous condition. Regarding mathematics, there was more negative results in the provinces further north-west, and in the regions further south (Puglia and Basilicata, and a part of Sicily) compared to the entire national territory. For reading, the areas most affected by learning loss were the north and central north of Italy and the islands. Contini et al. [48] also noted greater losses for students with pre-existing academic difficulties, low socioeconomic status, and female students, especially those with highly educated parents.

Research Questions #1: Beyond academic performance, are there other aspects linked to scholastic well-being of primary school students that were affected by school closures?

Our systematic review found that, in addition to the scholastic achievement, the school closure due to Covid-19 affected Italian primary school students on an emotional level (see, for example, Ref [57]), particularly for those who were already on the margins of the educational system prior to the COVID-19 pandemic, due to socio-cultural, socio-economic difficulties, and/or difficulties due to one or more diagnostic conditions (e.g., Ref [53]). According to the studies included in our work, some students may have encountered difficulties during distance learning, from a technical-practical point of view, such as: difficulties in following the online lesson, technical problems or competence of teachers/students with the new learning methodology (for example, Ref [58]). In order to achieve an overview of the effects of school closures due to COVID-19 on Italian primary school students, we focused on an analysis of the affected dimensions, according to the studies identified and included in our systematic review. Hence, we subdivided the articles according to the specific dimensions investigated, namely: emotional aspects of the learning context; marginality-inequality effects; didactic problems; learning quality.

Research Questions #2: Is there agreement between the opinions of parents, teachers, and students regarding the different effects of school closures on Italian primary school students?

Our results highlight several similarities and differences among the perspectives of parents, teachers, and students regarding the effects of COVID-19 school closures on students’ learning and well-being. A key similarity concerns the negative impact on learning and emotional well-being during the transition to remote learning. All three groups acknowledged a decline in students’ attention, motivation, and engagement during online classes, as well as an increase in anxiety levels and concerns. Additionally, both parents and teachers highlighted the significant challenge in providing necessary support to students during remote learning, especially for students with special educational needs or neurodevelopmental disorders. This suggests an agreement among the perceptions of different stakeholders regarding the difficulties encountered by students during the pandemic. Such disadvantageous condition is not identified in students with hearing loss [72], neither according to the opinion of the students themselves nor their parents.

However, there are also differences in perspectives related to the main problems due to COVID-19. For example, parents expressed concerns about excessive screen time and decreased participation in extracurricular activities, while teachers mainly focused on the need to improve digital competence and the challenge of adapting assessment methods. Moreover, teachers acknowledged some advantages of remote learning, such as innovation in teaching methods, which parents seem not have directly noticed.

As previously mentioned, no study investigated potential difficulties among students in approaching the new instructional methodology. One possible explanation could be attributed to their adaptability to technology. It is plausible that these students, who are typically exposed to various digital devices and platforms from an early age, may have a propensity to navigate and engage with online learning tools. The early exposure to technology may have facilitated a smoother transition to distance learning, potentially reducing the perceived challenges or obstacles encountered during the adaptation process [86]. However, further research is needed about this point.

Students also highlighted specific difficulties related to their learning disorder and emphasized the negative impact on learning quality, thus supporting the conclusions of the other two groups (parents and teachers). It is interesting to note how both parents and teachers considered support for students crucial in this new mode of learning, although each stakeholders pointed to the other: parents were resentful of the teachers’ lack of attention towards their pupils, especially those with previous difficulties; teachers were dissatisfied with the lack of attention and support from parents. Concerning the lack of support from some teachers, it is conceivable as expressed by some studies considered [52, 66]: teachers with more years of professional experience, accustomed to certain learning methodologies, encountered greater difficulties in reinventing themselves, especially towards technological novelty. The reported lack of parental support by teachers could be related with the level of education highlighted by Contini et al. [48], which suggest that students who have been most affected by COVID-19 school closures are those with highly educated parents. It is possible to postulate retrospectively that, the higher the level of education attained, the greater the professional responsibilities achieved by parents, and therefore the greater the workload, reducing the daily available time and hindering careful instructional support for their children. However, it is important to note that the relationship between parents’ level of education and their involvement in their children’s learning is complex and can vary depending on a range of factors, including cultural, socioeconomic, and individual context [30, 87].

Research Questions #3: Are there any differences between the effects of school closures on Italian primary school students and students in other countries as appears in the international literature on the topic?

Finally, we examined eventual differences between the effects of school closures on Italian primary school students and the international literature on the topic. This inquiry was thoroughly conducted through the analysis of the opinions of different populations (parents, teachers, students), while maintaining a focus on students’ learning and well-being. This comparison could provide valuable insights into the specificities of the Italian experience and its relevance in the global context, thus contributing to delineating targeted strategies and interventions to address the effects of the pandemic on students’ education.

Concerning the emotional aspects, our findings are in line with research conducted overseas. In their international systematic review, Viner et al. [24] analyzed the association between school closure and the well-being of K19 students. Compared to primary school students, school closure and the obligation to stay at home drastically reduced healthy extracurricular activities, such as sports, increasing sedentary habits (more time spent in front of the screen). Furthermore, a relationship between school closure and an increase in emotional and behavioral difficulties is once again confirmed. Our results are also in line with a recent systematic review of Italian K12 students during school closure [88]. Their results confirmed an increase in emotional dysfunction demonstrated by an increase in anxiety, anger, difficulty concentrating, and even sleep disturbances. However, it is interesting that Picca et al. [74] was the only study that found a positive effect of the pandemic period, precisely on children’s sleep. Further research on this matter is necessary. Still, Panchal et al. [89] carried out an intercontinental systematic review on the mental health of children and adolescents (≤ 19 years) exposed to the pandemic lockdown. Like them, our national data indicate that lack of adequate extracurricular activities and increased time spent in front of electronic tools negatively impacted children’s emotional health and psychological well-being, increasing perceptions of negative emotions associated with school closures for COVID-19 (feeling of loneliness, anger, irritability, fear, and stress). Furthermore, the authors also identified a higher risk of mental health problems and increased stress for students with special educational needs, although this also depended on space availability (e.g., having a garden or adequate space for studying at home) and family socio economic status. Further international research highlighted how students already on the margins of school community suffered scholastically [13, 14] and emotionally because of the school closure more than their peers [15, 16]. Even in this area, our results do not differ from international literature, although we highlighted that the condition differed based on the difficulty experienced by the students (e.g., students with special education needs vs. low socio-economic status vs. linguistic-cultural differences).

We did not find international reviews that specifically addressed technical-practical educational issues, such as the knowledge and competence of teachers in adapting to new teaching methodologies or the availability and quality of technological devices in the context of distance learning. However, a study conducted by Engzell et al. [30] sheds some light on this matter. This study reported that Dutch schools were closed for “only” eight weeks during a period of pandemic-related disruptions, without experiencing complications in the continuation of education. One of the factors that contributed to the maintenance of an adequate education was the excellent quality of broadband in the country. As regards digital preparation of teachers in this new teaching method, our results are aligned with a survey conducted in Germany by König et al. [90], about the importance of teachers possessing digital competence in achieving meaningful educational objectives. In summary, while technical-practical issues were not extensively explored in secondary studies, the primary evidence, and data available do underscore the significance of factors such as broadband quality and teachers’ digital skills in addressing the challenges of distance learning effectively.

As regards academic achievement, our review showed a learning loss in mathematical skills, in particular for females and students with learning difficulties, and a slight improvement in reading skills [8, 10–12]. A comparison with international review is needed due to the several differences and similarities between the Italian context and other countries. Contrary to Italian context, the international study by Betthäuser et al. [11] did not find an effects across age (no difference between primary and older students in learning loss). Similarly to our results, they found a greater learning loss in math than reading, although we highlighted a slight improvement in reading skills of Italian primary students. Our data are in line with the Danish study by Birkelund & Karlson [91], that found a leaning gain in reading performance of pupils in Grades 2 and 4, and a leaning loss in the same topic between older students (grade 8). It is important to note they lived a different time range of school closure: only eight weeks for Danish primary school students, and 22 weeks for Danish Grade 8 students. Hence, it is conceivable that prolonged school closures may have a detrimental effect on children’s learning (see for example Ref. [92]), raising long-term concerns for countries that experienced prolonged school closures, such as Italy. Our data are also in line with the US study by Kuhfeld, Lewis & Peltier [93], where they highlighted a larger decline for primary school students compared to secondary school students, but not in the same school subjects as the ones that were analyzed in the current review: while Italian studies highlighted greater learning loss in mathematics and a slight improvement in reading, they found similar pre- and post-pandemic results for mathematics, and a greater loss in reading among younger students. Furthermore, our study highlighted a worsening in existing disparities among students, especially those with learning disabilities or special education needs (SEN), and from lower socioeconomic backgrounds. These results are aligned with several international studies. As regards students with previous scholastic difficulties, two US studies [94, 95] highlighted that learning loss in reading and mathematics was substantially greater for students with prior difficulties than for the general population. As previously reported, it is conceivable that the structured education provided by the schools that these students require could not be adequately delivered in the new learning modality. This once again underlines the need to identify new, more inclusive learning methodologies. Compared to students with different family backgrounds, heterogeneity is highlighted at an international level. While the study by Kuhfeld, Lewis & Peltier [93], in line with our findings, highlighted considerable differences between ethnic groups, with lower performance among students with high socioeconomic poverty, not all studies found an association between learning loss and socioeconomic status [91]. We might assume that this difference is due to the concept of “socio-economic status” varies considerably from country to country (in this case: USA vs. Denmark). This may mean that measures used to assess the socioeconomic level of study participants may not be perfectly comparable across different nations or regions. Furthermore, country-specific cultural, political, and economic factors could significantly influence how socioeconomic level affects student learning losses. Furthermore, it should be remembered that the sample of the Danish primary school of Birkelund & Karlson [91] experienced a school closure of only 8 weeks, while the study of Kuhfeld, Lewis & Peltier [93] referred to US students, where the school closure lasted several months. This would further confirm the hypothesis that the rapid return to school reduced the risk of learning losses for students with previous difficulties, also in relation to the possible lack of adequate devices to follow distance learning (e.g., electronic devices; good quality or total absence of Wi-fi) within the family unit. The comparison between Italian and international studies highlighted an extremely heterogeneous picture, with numerous differences and similarities that showed a more or less harmful effect of school closures due to COVID-19 based on different factors: territory; duration of school closure; pre-pandemic disadvantaged conditions. Further research on the topic is needed to highlight long-term post-pandemic effects on the quality of learning of primary school students, not only Italians.

Limitations

The authors are aware of the limitations of this systematic review. First, although a good number of studies have been identified, few of them have achieved a satisfactory level of risk of bias. This may be attributable to the fact that, due to the pandemic condition, most of the studies took place through the completion of online questionnaires and snowball sampling, and in some cases validated measures were not used to evaluate the pandemic effects of interest to them. Also, except for Baschenis et al. [44], all studies were observational, and most were cross-sectional. A further limitation is, paradoxically, the number of studies. Taking into consideration all the included studies, an adequate number of 34 papers is reached. However, having examined 4 distinct and fundamental categories of learning during the pandemic period and as many as 3 different samples (parents, teachers, pupils), the number of studies (divided according to section and reference sample) has never exceeded the number of 9 papers. Hence, further research is needed regarding the perception of distance learning and its consequences on learning by all stakeholders. Moreover, the substantial differences in the analysis of the results (sometimes even absent) of the studies identified did not allow the development of a meta-analytic study. Consequently, the qualitative nature of the studies included in this systematic review, did not permit the evaluation of a publication bias, aimed exclusively at evaluating a bias within quantitative studies (therefore, meta-analyses). However, an effort was done so that to avoid publication bias, following a rigorous research methodology, including PRISMA flowchart and checklist, a Risk-of-Bias Quality Assessment, and a search strategy. Another important limitation of our study is that it was not always possible to quantify the differences between the reference sample and the pre-pandemic period. Most of the studies, in fact, are based on the opinion of students, parents and/or teachers about the differences between the two time periods, collecting these values through surveys and reporting values in percentages. However, it has not always been possible to draw a comparison between the two periods. With respect to academic performance, the studies in this regard mainly focused on INVALSI data, relating to reading, mathematics, and English, although no identified study reported information on the latter school subject. Consequently, it was not possible to identify a change–positive and/or negative–with respect to all primary school subjects.