Evaluation of the Virtual Learning Environment by School Children and Their Parents in Saudi Arabia during the COVID-19 Pandemic After School Closure

Background: No previous studies have involved school children or their parents in the evaluation of virtual learning environment (VLE). Thus, this survey was performed to develop a novel questionnaire which was then used to evaluate the satisfaction of school children and their parents with the VLE in the Kingdom of Saudi Arabia during the COVID-19 pandemic. Methods: A cross-sectional survey utilizing a validated and reliable questionnaire was distributed online for VLE evaluation. A median value >3 indicated participant satisfaction in each of the 5 domains of the questionnaire as well as overall VLE satisfaction. Results: Six hundred and ninety three participants including 571 Saudi citizens and 122 non-Saudi residents participated in this survey. The number of school children who agreed or strongly agreed were signi�cantly lower than the number of children who disagreed or strongly disagreed with preferring the VLE over traditional education (p<0.001). The participants evaluated the VLE experience as unsatisfactory with a median value ≤ 3 for 4 out of 5 questionnaire domains with an overall satisfaction value of 2.8. Among the 117 participants who gave further written opinions/comments, 42(35.9%) participants supported the VLE as an alternative to traditional classrooms, if equipment and internet are made available and for the safety of their children. Conclusions: This is the �rst powerful population study, both globally and nationally, to utilize a novel validated and reliable questionnaire for exploring VLE satisfaction. This study showed the participants’ unsatisfactory VLE experience. The VLE is accepted as an alternative to traditional classrooms to keep up with learning and to maintain the safety of children and it can be a supplementary learning method but many measures are still needed to develop the VLE.


Background
Much has been reported about the impact of the ongoing COVID-19 pandemic on the health and lives of people throughout the world. As almost every aspect of life was affected, the educational process at every level has also been in uenced [1]. Quick modi cation of the method of delivering education and of the assessment process was needed. The answer was a complete shift to distant e-learning constituting a virtual learning environment (VLE).
In VLE, remote learners receive their learning materials electronically via the internet [2]. Although the VLE/e-learning has already experienced high growth/development in the past few years to as an alternative to traditional classrooms, the dependence on and the use of VLE have increased signi cantly since the start of the COVID-19 pandemic [3]. Compared to the physical classroom environment, the VLE employs technology to conduct courses and lessons, offer quizzes, and provide various assessment tools [4].
However, the VLE has many disadvantages such as the lack of prompt live interaction and feedback in asynchronous e-learning, the excess time that instructors need for preparation, and potentially more frustration especially in the absence of high speed internet or the required technical experience or support [2,4]. Additionally, school children and their caregivers may not have the required skills and experience to adapt and use VLE technology incurring the risk of academic failure of students who are unable to complete their homework and assignments [5]. However, the COVID-19 pandemic left us with no choice but to adopt and implement the VLE with all its advantages and disadvantages as billions of school children around the world had to undergo a sudden unplanned shift from traditional classrooms to home education through the VLE with due support from many concerned organizations [3,6].
School closure during the COVID-19 pandemic deprives children from essential learning and opportunities for adequate growth and development. Home con nement will have deleterious effects in terms of longterm socioemotional imbalance [7]. The cessation of traditional education is expected to disintegrate the sense of normality that used to be provided by schools and have serious effects on the physical and mental health of children including childhood obesity, reduced cardio-respiratory tness, unfavorable diet plans, sedentary lifestyles, irregular sleep patterns and increased television/smart-phone screen time during the home lockdown [8]. Moreover, with school closure, parents are more responsible for facilitating the home learning of their children and they usually struggle to perform this task. This is particularly evident for parents with limited education and resources [9]. Thus, parents, psychologists, governments, non-governmental organizations and in particular pediatricians are very concerned about alleviating the harmful effects of the COVID-19 pandemic on the health/development of children and adolescents, who are the most vulnerable to many of the devastating effects of COVID-19.
The majority of higher education institutions in Western world were partially utilizing the VLE even before the pandemic. They were equipped with the required skills/experience to embrace this sudden change. This is not the case for e-learning at the school level as many students, caregivers, teachers and institutions are de cient either in skill or equipment or both [10]. In the past, the use of VLE in the Kingdom of Saudi Arabia (KSA), was mainly oriented towards university level education whereas its use at the school level remained limited [11][12][13].
To the best of our knowledge, no previous published studies, either globally or in the KSA have involved school children and their parents in the evaluation of the VLE, and no prior published questionnaires have been designed speci cally for this purpose. Therefore, this study was performed to develop a novel validated reliable questionnaire which was then used as a screening tool to evaluate the satisfaction of school children and their parents with the main aspects of the VLE in the KSA during the COVID-19 pandemic after school closure.

Study design and participant selection
A cross-sectional questionnaire-based survey was designed to collect data on the main aspects of VLE.
This survey was conducted in the KSA from March-May, 2020 during the COVID-19 pandemic. Participating school children and their parents were randomly selected and completed an electronic online questionnaire. Parents were invited to answer this questionnaire if they were Saudi citizens or residents and had school children aged 6-18 years who had gone through the VLE experience. Participants living outside the KSA or having children outside the speci ed age range of 6-18 years as well as incomplete questionnaires were excluded from this study.

Measuring Tool (questionnaire Development)
A questionnaire based on the main components and basic requirements of e-learning was developed by the researchers as an exploratory screening tool to evaluate the overall experience/satisfaction of school children/learners and their parents with the VLE experience and to identify di culties encountered while using the VLE.
The questionnaire included 3 sections. The rst section included a full explanation of the concept, objectives, and bene ts of this survey. The second section obtained the socio-demographic data of participating family (Saudi citizen or resident, residence area, number of children, age, gender, and school grade of the eldest child; and age, education level and occupation of both mother and father). The third section had 24 questions to evaluate the learner and parental perspective on experience with the VLE. At the end of the questionnaire, the participants were invited to add any further opinions or comments.
The content of the questionnaire was validated by 5 faculty members from outside the research team and any amendments were made accordingly. The questionnaire was translated and back-translated (English-Arabic) and checked by two bilingual experts.

Statistical Validation/reliability Analysis Of The Questionnaire
Exploratory factor analysis was conducted for 24 variables (questions) using parallel analysis to determine the number of factors to retain with direct oblimin rotation.
According to exploratory factor analysis, the whole questionnaire consisted of 5 main domains: The answers to the questions were rated on a 5-point Likert scale from 1-5 (strongly disagree = 1, disagree = 2, true sometimes = 3, agree = 4 and strongly agree = 5). The answers to the questions related to the traditional classroom and facilities/support domains were reverse scored because their answers were in the opposite direction to the answers for all other questions as they expressed the participants' dissatisfaction with some negative aspects/drawbacks of the VLE,. The reliability of all domains as well as the individual domains of the questionnaire was analysed by Cronbach's alpha test and overall reliability of the questionnaire was excellent (Cronbach alpha coe cient = 0.92).
The questionnaire used and technical details of statistical validation and reliability analysis of the questionnaire are given as supplementary material in the additional le 1.

Questionnaire Implementation And Distribution
Both English and Arabic questionnaires were converted into Google Forms, so that participants cold choose to complete the questionnaire that was more convenient for them. Then, the links for both questionnaires were sent via social media such as WhatsApp's, Twitter, and Facebook to the participants.
The questionnaire was directed to the parents of school children who could complete the whole questionnaire, including responses to questions directed to their children after rst taking opinions and answers from their eldest children. Furthermore, the parents could use the questionnaire link again to ll out another questionnaire(s) for other children between 6-18 years of age. The student/learner was also allowed to directly answer learner-related questions if desired, and if he/she understood the questions with or without help from his/her parents.
In this study, learner and parental VLE satisfaction was the main dependent or outcome variable. A score with a median value above 3 indicated overall satisfaction with the VLE experience as well as satisfaction with each individual domain, while a median value below 3 indicated dissatisfaction.

Sample Size And Study Power
The collected cross-sectional sample size of 693 was selected to achieve 91% power to detect a difference of 0.3 between the null hypothesis mean satisfaction of 3.0 and the alternative hypothesis mean of 3.3 with a signi cance level (alpha) of 0.05 using a two-sided test carried out with PASS software (provided in the additional le 2).

Statistical analysis
The data were analysed by the Statistical Package for the Social Sciences (SPSS) version 25.0 (IBM corporation, Armonk, NY) after checking for completeness and inconsistencies. The data were scrutinized and double-checked before and after entry into the SPSS program. Frequencies and percentages were used to present categorical variables while the mean, standard deviation (SD) and range were used for quantitative variables. The Kolmogorov Smirnov test was used to test data normality. The Kruskal Wallis test was used to test the differences between the median satisfaction values of the studied variables. P values < 0.05 were considered signi cant.

Results
Six hundred and ninety-three participants, including 571 Saudi citizens and 122 non-Saudi residents from 5 main regions of the KSA completed the questionnaire. The participants were parents of 343 (49.5%) males and 350 (50.5%) females, with a mean and SD of 12.6 ± 3.8 years of age for all included school children. The participating parents included 167 (24.1%) fathers with a mean age and SD of 44.5 ± 8.3 years and an age range from 23.0-78.0 years; and 526 (75.9%) mothers with a mean age and SD of 39.0 ± 7.1 years and an age range from 20.0-60.0 years. Four hundred and ten participants (59.2%) had 3 or more children. A high proportion of fathers (73.8%) and mothers (77.5%) had university and postgraduate higher education. Regarding job status, 624 (90%) of fathers and 350 (50.5%) of mothers were employed. The sociodemographic data of the participants are presented in Table 1. In one direct question of this questionnaire to express preference for the online VLE over traditional education, the number of school children who agreed or strongly agreed (n = 158, 22.8%) was signi cantly lower than the number of children who disagreed or strongly disagreed (535, 62.8%) on preferring the VLE over traditional education (p < 0.001).
According to the de nition of satisfaction, participants evaluated the VLE experience as unsatisfactory with a median value of 3 or less for all domains except the parental role domain (median value of 3.3). Additionally, the overall evaluation revealed dissatisfaction with a median score of 2.8 ( Table 2). Parental perspective on facilities/support 2.8 1 5 Parental role/perspective on the VLE 3.3 1 5 Overall satisfaction with the VLE experience 2.8 1 5 Except for the facilities/support domain, Saudi citizens had signi cantly more satisfaction with all domains and more overall satisfaction than non-Saudi residents (P values < 0.05, Table3, Fig. 1). A comparison of overall VLE satisfaction between Saudi citizens and non-Saudi residents by child gender, study grade, parental and maternal education is shown in Fig. 1. with a university level and above demonstrated signi cantly higher satisfaction with the facilities/support domain than fathers with less than a university level of education (P = 0.0001). Mothers with a university level and above demonstrated signi cantly higher satisfaction with the skills/interaction and facilities/support domains than mothers with less than a university level of education (P = 0.02, and P = 0.0001, respectively, Table 3).
Regarding further opinions/comments at the end of the questionnaire, 117 (16.9%) participants provided their comments. From these comments, the participants could be clearly identi ed as opponents or supporters of the VLE. There were signi cantly more opponents of the VLE than supporters:

Discussion
The global COVID-19 pandemic and its resultant prolonged lockdown measures have seriously impacted all aspects of people's lives. While the immediate concern during this pandemic focused on avoiding or treating COVID-19 to maintain people's health and lives, it is increasingly recognized that children, the world's future, will be the most evidently severely impacted victims of this pandemic crisis with unprecedented risks to the rights, safety and development of all children.
The long-term serious impacts on children worldwide are likely to be destructive, despite the less severe symptoms and lower mortality rates in children who are infected with COVID-19 compared to other age groups [14]. Children of all ages and in all countries especially low-income countries are at extremely high risk of signi cant psychological and socioeconomic impacts, learning problems and potential long-term persistent effects of the delayed implementation of sustainable development goals, particularly if the battle to contain SARS-CoV-2 virus is prolonged [15].
To mitigate the spread of SARS-CoV-2 among children, considering that their safety is a priority, the governments of 188 countries introduced social distancing and lockdown measures with unprecedented worldwide closure of face-to-face child services such as schools interrupting education for > 90% of the world's students or 1.5 billion children and youth establishing a learning crisis [14,16,17] While more than two-thirds of countries have introduced a national distance learning platform, only 30% of low-income countries have done so [18,19]. The COVID-19 pandemic implies limited or no education for many children. Not only did school closure in many countries with planned extended lockdowns interrupted teaching but many exams have also been postponed, rescheduled or cancelled [1,16]. The longer schools remain closed with its dramatic major consequences on children, the less likely children will be able to catch up with learning and essential life skills that support a healthy transition to adulthood.
The VLE has been utilized mainly for university students and in the professional e-training of postgraduate students [20] but no previous published studies, either globally or in the KSA, have ever involved school children in the evaluation of their experience with the VLE. Therefore, it was initially recognized by the research team that it is essential to develop a novel validated and reliable questionnaire and to apply this questionnaire in the evaluation of the satisfaction of school children/learners and their parents with the VLE experience in the KSA after school closure.
In this online survey, 693 participants provided a robust study with a power of 91% that evaluated the VLE satisfaction/experience of the targeted population of primary, intermediate and secondary school children and their parents in 5 main regions of the KSA. The higher participation from the western and central regions of the kingdom can mainly be attributed to the presence of the highest population densities in these 2 regions as well as the uneven distribution of online questionnaires which depended on social media and internet resources/accessibility. However, this online questionnaire was the only way to reach participants in view of the inability to directly approach participants in different regions due to lockdown measures.
In this survey, the participants evaluated the VLE experience as unsatisfactory with a median value of ≤ 3 for most of the studied domains and with an overall satisfaction value of 2.8. These results can clearly demonstrate that the participants were not very satis ed with the VLE experience and 62.8% of school children disagreed or strongly disagreed that they preferred the VLE over traditional classrooms. Such a preference for traditional education could also be recognized from the analysis of the opinions/comments of the participants at the end of the questionnaire as opponents of the VLE were signi cantly more prevalent than the supporters (p < 0.0001). The most important reasons given by VLE opponents were missing social skills essential for normal child development, problems in the availability of equipment/internet, working parents who cannot supervise/help their young primary school children, the requirement of more time and effort, and more convenience of the VLE for older secondary school children. Parents were only satis ed with their role in supporting the VLE (median = 3.3, Table 2) of their children because it seems that they were convinced and felt that they did their best to facilitate the VLE of their children even though they most likely faced many challenges out of their control and beyond their abilities.
A high proportion of fathers (73.7%) and mothers (77.5%) participating in this survey were highly educated (university level and above) with 90% of the fathers and 50.5% of the mothers being employed. Therefore, they could have more resources for equipment and internet accessibility and a greater ability to respond and participate in this survey. Thus, they were more able to support the VLE of their children as evidenced by signi cantly more maternal and paternal satisfaction with the facilities/support domain compared to parents with lower education levels (p = 0.0001, Table 3). On the other hand, a high proportion of families (59.2%) had 3 or more children and 47.3% of children were in primary schools requiring more supervision/help than older students. Therefore, these families experienced a di cult challenge and tough complex situation to allocate time and effort to care and support the VLE of 3 or more children and to provide them with the necessary equipment/internet resources at the same time.
The signi cantly higher satisfaction values of secondary school children in the skill/interaction domain compared to primary and intermediate school children (Table 3) was expected because secondary school children are mature enough to be responsible and control their learning and because they have possibly had more training and experience in the use of e-learning technology and the internet. Furthermore, parents expressed that the VLE is more convenient for secondary school children than for primary school children. Therefore, it seems logical to recommend the VLE mainly for intermediate and secondary school children, and school reopening can be resumed initially for primary school children under complete preventive measures given the consistent universal agreement that COVID-19 is less severe in young children than in adults particularly in young children more than 2 years and less than 15 years of age [21][22][23][24].
In this survey, Saudi children and their parents had signi cantly more overall satisfaction with the VLE than non-Saudi residents (Table 3) even though asynchronous e-learning was more commonly applied in Saudi governmental schools than in international schools which mainly serve non-Saudi students. This satisfaction regardless of the somewhat possible lower degree of VLE quality may be related to the greater preference of Saudi children for e-learning because they may be more skillful, more trained and more adapted to using such technology as the majority of the Saudi population is actively using the internet and social media [25,26].

Strengths And Limitations
This study has important strengths because it is the rst powerful study, both globally and in the KSA, to utilize a novel questionnaire as a screening tool to evaluate the VLE satisfaction of school children and their parents. All necessary steps and statistical procedures to guarantee the validity and reliability of the questionnaire to obtain robust/sound results and solid conclusions were undertaken.
The limitations of this study included the lower participation from eastern, northern and southern regions of the kingdom due to the unavoidable previously explained reasons and inability of the questionnaire to reveal speci c details of the defects/problems in some aspects of the VLE including the role of the instructor and assessment evaluation. However, this questionnaire was designed as a screening tool to discover overall satisfaction and the main disadvantages/problems that emerged during the VLE experience. If a more comprehensive questionnaire was used, lower responses or more incomplete responses might be expected because the inclusion of more questions can result in a tedious questionnaire requiring more time and effort from participants.

Conclusions
The participants in this survey were not very satis ed with the Saudi VLE experience and 62.8% of school children (62.8%) still preferred traditional education for many given reasons. However, fortunately, the Saudi VLE experience is better than no education or the lack of facilities for any VLE for populations who are digitally-excluded without internet access in resource-poor countries. The VLE was accepted as an alternative to traditional classrooms in the current COVID-19 pandemic to keep up with learning and to maintain the safety of children and it can be just a supplementary learning method. VLE technology can play a major role in helping school children learn and develop new skills because future education seems to be more digitally-dependent. It is recommended to support e-learning advancement in the future with successful VLE implementation at the school level whether during normal or emergency situations. Many measures are still needed to develop and improve the VLE, such as managing technical problems especially the free supply of equipment/internet, including their cost in school fees, training stakeholders, developing more innovative and interesting teaching tools, and implementing a more synchronized VLE.