http://orcid.org/0000-0002-7092-4538
Figures
Abstract
Our work applies Universal Design criteria for producing and using Mathematics videos for primary education students, at a time when many countries are shifting towards inclusive education policies. We have focused on how the accessibility criteria used for students with visual impairments might affect non-disabled students. For this, we reviewed applicable Universal Design principles as well as best practices in multimedia learning. We took into account the roles, procedures, tools and standards involved in the multimedia lifecycle. We then undertook an experiment consisting of producing two videos about prime numbers with the same pedagogical contents; one video was accessible for students with visual impairments and the other one was not accessible to them. We conducted a trial in real world school settings with 228 non-disabled children, who were randomly assigned a version, either accessible or not accessible, and were then asked to take a test to measure objective aspects of their learning concerning retention and transfer as well as several subjective aspects, including the attractiveness of the videos. Results indicate that there were no significant differences in the scores obtained by students using either video, although the group who watched the accessible video obtained higher score medians in the retention questions. Moreover, students found the accessible video significantly more attractive (p = 0.042). Our study provides recommendations for different stakeholders and stages within the process of producing multimedia mathematics materials that are accessible to primary students with visual impairments, as well as evidence demonstrating that everybody can benefit from the recommendations for developing good quality, accessible multimedia material.
Citation: Rodriguez-Ascaso A, Letón E, Muñoz-Carenas J, Finat C (2018) Accessible mathematics videos for non-disabled students in primary education. PLoS ONE 13(11): e0208117. https://doi.org/10.1371/journal.pone.0208117
Editor: Filomena Papa, Fondazione Ugo Bordoni, ITALY
Received: May 10, 2018; Accepted: November 12, 2018; Published: November 28, 2018
Copyright: © 2018 Rodriguez-Ascaso et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Multimedia material is being used more and more as a learning resource [1–4]. It is known that this kind of material, if designed effectively, can enhance learning ([5,6]; [7][8]). Furthermore, the use of teacher-created videos may increase significantly in the near future. Relevant educational paradigms include “the flipped classroom” [9] where “with teacher-created videos and interactive lessons, instruction that used to occur in class is now accessed at home, in advance of class, that becomes the place to work through problems, advance concepts, and engage in collaborative learning”, and MOOCs (massive open online courses) [10], which are “based on multimedia collections”. More specifically, the use of multimedia for teaching mathematics in primary education has been previously investigated in [11] and in [12].
The use of multimedia material should also benefit students with disabilities. According to the Salamanca Statement of UNESCO, “those with special educational needs must have access to regular schools which should accommodate them within a child-centred pedagogy capable of meeting these needs” [13]. In agreement with this statement, the education policies of many countries are shifting towards inclusive education [14]; [15]). As an example of the impact this trend is causing in the accessibility of multimedia materials, by early 2015 advocates for the deaf filed federal lawsuits against Harvard and M.I.T., citing violations of the Americans with Disabilities Act (ADA) by failing to provide closed captioning for the multimedia material in their MOOCs. This led to a settlement of the Justice Department with edX Inc. to comply with ADA and WCAG 2.0 [16] within 18 months [17]. The current situation is that edX courses have captions, the latest version of the edX multimedia player includes the option to enable/disable captions [18], and edX authors are provided with guidelines on how to create accessible multimedia [19].
The theory of Universal Design for Learning (UDL) [20] and its subsequent guidelines (UDL Guidelines 2.0 [21]) provide guidance to identify and remove barriers for all students from teaching methods and curriculum material. UDL applies in the learning context the principles of Universal Design (UD) [22], i.e., equitable use, flexibility in use, simple and intuitive to use, perceptible information, tolerance for error, low physical effort, and size and space for approach and use. In particular, adopting a User Centered Design (UCD) [23] approach for the production of all learning materials, whether they are electronic or tangible, and providing adaptations for learning materials where necessary, constitute key strategies to make elearning accessible to all, and involve students, teachers and most of the stakeholders within the elearning arena [20,24–26][27] [28] [29]. However, “simply encouraging, or even mandating, the implementation of inclusive education practices does not guarantee improved outcomes”, while training and supporting school managers and teachers adequately constitute key levers for making real inclusion to happen [30].
Furthermore, previous studies indicate that inclusive education shows neutral to positive effects on students both with and without disabilities, e.g. generic studies such as [31,32], as well as studies on learning through multimedia, such as [33].
Within the described scope, the general objective of our work is to assess the effect that accessible multimedia materials on Mathematics have on the learning process of non-disabled students. For doing so, we have carried out an experience consisting of: a) producing two multimedia objects with exactly the same pedagogic content, one of them produced without accessibility in mind (video B), while the other one is accessible for students with visual impairments (video A); b) assessing the quality of the learning achieved with each of the two videos A and B through an anonymous evaluation conducted in real settings with 228 non-disabled students in three primary schools.
To achieve the objective mentioned above, the present paper has been structured as follows: we first review applicable Universal Design principles as well as best practices in multimedia learning, both from the student and from the professional viewpoints, and define the purpose of the study (section 2). Next, we describe the multimedia materials and the tools used in our experience (section 3). Then, we describe the methodology of our experience (section 4). Next, we discuss the analysis and results of the experience and provide recommendations for different stakeholders (section 5). Finally, we come to the conclusions derived and future work (section 6).
Accessible multimedia materials
Related work
In this section we review the corresponding literature, techniques and standards applying to each of the principles of Universal Design that are applicable to our study, namely: perceptible information, equitable use and low physical effort.
The principle of perceptible information establishes that “the design communicates necessary information effectively to the user, regardless of ambient conditions or the user's sensory abilities”, see [22]. There exist international de facto standards on the accessibility of electronic content, such as WCAG 2.0 [16]. The standard includes the following applicable guidelines:
- Use of Color: Guideline 1.4.1 reads “Color is not used as the only visual means of conveying information, indicating an action, prompting a response, or distinguishing a visual element”.
- Contrast: Guideline 1.4.3 reads “The visual presentation of text and images of text has a contrast ratio of at least 4.5:1”. Guideline 1.4.6 is identical to Guideline 1.4.3, except for that the required contrast is higher (7:1).
- Visual presentation. Guideline 1.4.8, as for displaying multimedia, establishes that “for the visual presentation of blocks of text, a mechanism is available to achieve the following: a) Width is no more than 80 characters or glyphs; b) Text is not justified (aligned to both the left and the right margins); c) Line spacing (leading) is at least a space-and-a-half within paragraphs, and paragraph spacing is at least 1.5 times larger than the line spacing”.
- Perception of audio. Guidelines 1.4.2 (audio control) and 1.4.7 (low or no background audio). In a context like ours where the sound coming out from the loudspeakers of the classroom is adjusted by the teacher and not by the end-users (students), these guidelines recommend “to ensure that the background sounds are at least 20 decibels lower than the foreground speech content”.
- Audio description or media alternative. Guideline 1.2.3 establishes that “an alternative for time-based media or audio description of the prerecorded video content is provided for synchronized media, except when the media is a media alternative for text and is clearly labeled as such”. Audio description (AD) is the most widespread practice to describe videos orally, and can be defined as the “verbal depiction of key visual elements in media and live productions, where the description of media involves the interspersion of these depictions with the multimedia’s original audio” [34]. The benefits of AD for people with visual impairments have been reported elsewhere in the literature [35]; [36], as well as its benefits for their learning activities [37,38].
The principle of equitable use establishes that [22] the design should be “useful and marketable to people with diverse abilities”. Our study focuses on assessing the effect that a particular design has on non-disabled students. In the case of AD, its beneficial effect on people without visual impairments has been addressed elsewhere [39]; [33]). Furthermore, the Equitable use principle also states “regardless of ability, all users should find the design appealing to use” [22]. In line with this, principle III of UDL Guidelines 2.0 includes the following generic statement: “affect represents a crucial element to learning, and that learners differ markedly in the ways in which they can be engaged or motivated to learn”. Like in [40] we believe that the visual aspect of the design has to do with this part of the principle. In this respect, [41] established that the visual design of multimedia material has an affective function, in addition to the cognitive aspects addressed by Mayer and colleagues in the Cognitive Theory of Multimedia Learning (CTML). Furthermore, [42] showed that the visual design of multimedia learning environments ¨was able to induce positive emotions that in turn facilitated comprehension and improved performance”. In line with this, the children's attitude to the use of technology in education has been studied in the literature [43].
The principle of low physical effort establishes that “the design can be used efficiently and comfortably and with a minimum of fatigue” ([22]). Similarly to the approach taken by [40], in our multimedia context we are considering “the cognitive effort required to use the design efficiently, comfortably and with a minimum of fatigue”, in line with applicable aspects of the (CTML) proposed by Mayer and colleagues, and more specifically with the temporal-contiguity effect described in [44] and referenced in the multimedia elearning classification produced in [45]. This effect refers to multimedia learning enhancement when visual and spoken materials are temporally synchronized, that is, presented simultaneously rather than successively. Authors of [44] measure two different learning indicators: retention, which is the ability to remember material at some later time in much the same way it was presented during instruction; and transfer, which is the ability to use what was learned to solve new problems, answer new questions or facilitate learning new subject matter [46].
Also in relation with the principle of low physical effort, videos with aural description of the visual information have a longer duration than those without it, and this could have an effect on learning, as educational experts agree that video is best shown in short segments so as to maximize concentration [47].
Purpose of the study
The objective of our work is to assess the effect that accessible multimedia materials on Mathematics have on the learning process of non-disabled students in primary education. In agreement with the related work described above, both objective (retention and transfer) and subjective indicators have been measured.
Materials
This section presents the pedagogic and accessibility aspects of the multimedia materials that have been used in our experience. To carry out the research, two versions of a video (A, accessible, B, not accessible) about prime and composite (non-prime) numbers were developed:
- Video B: http://youtu.be/52O1T7EFVBc
- Video A: http://youtu.be/lCsWVByY7Gk
The process to develop the videos was as follows: First, a teacher with a long experience in the creation of learning multimedia materials, but with no specific experience in accessibility, authored a Mathematics video addressing the concept of prime numbers with the support of the staff from the media production unit (video B, not accessible), see Fig 1.
More specifically, the teacher designs the material, writes the script, and produces a set of slides with the static visual information. Then, the teacher records the video with the support from the multimedia production unit, who provide the required technology and assist the teacher during the recording. The recorded multimedia material consists of:
- Video information: the slides (static information) plus the images/text that the teacher manually draws/writes on them and that are recorded by the system (dynamic information). The face of the teacher is also recorded.
- Audio information: the teacher’s voice is recorded as the narration of the video.
Then, in order to produce the accessible version of the video (video A), a group of experts in teaching mathematics to visually impaired and blind students identified the accessibility issues of video B, and provided a set of recommendations to make a new video with the same pedagogic objectives, and at the same time accessible for that target group. Then, the teacher designed and recorded a new video in which those recommendations were followed, again with the support of the media production unit. The accessibility specialists assessed the accessibility of video A, and considered that it is accessible for visually impaired and blind students. The overall process is illustrated in Fig 2.
Pedagogical features.
Both videos A and B have the same learning content and objectives. Through them, the students are taught about the three concepts defined in the 6th grade Mathematics syllabus for the “prime numbers” topic, namely:
- The definition of prime numbers and composite numbers,
- The geometrical interpretation of prime and composite numbers,
- The use of the Eratosthenes sieve to identify the prime and composite numbers.
The video material is to be used with students who have just been through the concept of divisors.
Regarding the Design Principles [45] the production of both videos has been guided mostly by the principles of the CTML [6]. Additionally, we used other features proposed by [8], i.e. synchronizing mathematical handwritten text with teacher’s oral narration, and [12], i.e., “the content was segmented into clear steps; the video contents were connected to previous mathematical knowledge; clear visuals were used when necessary to illustrate key aspects of problems; important elements in problems were highlighted in order to focus student attention; a conversational, relaxed voice was used to engage users; and the length of each clip was kept to a minimum to address issues of limited attention span”. Also, the face of the teacher is displayed on the video, in agreement with [48].
Accessibility features.
Video B was produced first. Then, its appropriateness to use with students with visual impairments (either low vision or blindness) was assessed by a team of accessibility experts. The reviewers -two math teachers, a computer teacher and a teacher from a non-scientific area- from the Spanish National Organisation for the Blind (Organización Nacional de Ciegos Españoles, ONCE) are experts with experience of at least 25 years in teaching Science and Mathematics to students with visual impairments. Within the review, that included tests with students with different visual conditions, the experts followed the ONCE guidelines for ensuring that digital learning materials are accessible for students with visual impairments [49].
After the review of Video B, the experts proposed the following recommendations in order to improve its accessibility:
- Improve the contrast by changing the color of the pen used by the teacher to write on the screen.
- Use thicker lines for both characters and drawings.
- Increase the size of the pictures in the drawings.
- Verbally explain the visual contents in the video, such as the Eratosthenes sieve and the list of prime numbers.
With the support of the media production unit, the experts’ recommendations were adopted, and a new video called Video A was developed.
First, with the aim of procuring an aural description of the visual information, the teacher wrote a new script for Video A. The only difference with the script of Video B was that the new script included the aural description of visual information that is relevant from the pedagogical viewpoint (see Table 1). As a consequence of the oral description of visual information in video A, its duration is 10 minutes 15 seconds, while Video B’s duration is 8 minutes 43 seconds. During the recording of both videos, the teacher used a tele-prompter to adhere to the script.
Second, in order to optimize the perception of visual information, video A was produced with improved contrast and size of its visual elements (drawings, text and teacher’s image). For doing so, the media production unit modified the visual settings of the recording tool accordingly. Furthermore, both videos (A and B) comply with WCAG guidelines 1.4.1 (Use of Color) and 1.4.8 (Visual Presentation). See Fig 3 and Table 2 for details and samples of the differences between videos A and B.
Screenshots of video A (left) and B (right). The person in the image is Emilio Letón, one of the work co-authors, who has given written informed consent for its publication.
At the end of the process, video A was reviewed by the team of accessibility experts, including tests with blind students and students with low vision, who considered that video A is accessible for students with visual impairments.
Methodology
Participants
The participants in this trial were 6th grade students (12-year-ods) attending three schools that belong to the same school system and follow the same curriculum. All participants were non-disabled.
The trial was carried out with the permission of Fuhem (http://www.fuhem.es/), the institution to which the three participating schools belong. The methods and materials were previously agreed on with the teachers, and the director of studies authorized the experience to be included in their program of study. Furthermore, the subject of the exploratory lesson, “prime numbers”, and the time when the research experience took place were agreed on with the teachers of the three schools, according to their program of study for mathematics.
Because participants were minors, the participant consent was handled as follows: each school sent a letter to the students’ parents or guardians, where they were informed about the purpose of the experience, its characteristics, the date, and the researchers’ contact information. Furthermore, the letter informed them that their participation was voluntary, that only anonymous information was to be gathered, and that they had the right to refuse their children’s participation.
Immediately after the experience was carried out, each teacher resumed her/his program of study as usual, including the “prime numbers” lesson.
Design
The trial was designed to assess the quality of the learning achieved with two types of videos: A and B. Both videos have the same pedagogic content, one of them produced without accessibility in mind (video B), while the other one is accessible for students with visual impairments (video A).
In order to avoid bias, groups were formed from random samples at each school, and were stratified in order to ensure a homogeneous distribution by classroom and by gender in each group. For that purpose, a pack of playing cards that had been previously prepared according to the number of female and male students in each classroom was dealt separately to girls and boys. For girls, we prepared the same number of black suit cards as red suit ones. We did the same for boys. Thus, students were initially divided into two randomly assigned groups, those who got a red suit were to sit in a classroom where Video A would be shown and those with a black suit would sit in a classroom showing Video B.
Both videos A and B were projected simultaneously in the different classrooms. This was the first time the students watched the videos. The visual conditions of the projection [51] were similar in all classrooms: they were made through smart boards (all of them were of the same model), the blinds were closed and the lights were on. Regarding the auditory conditions of the projection (WCAG 2.0 guidelines 1.4.2 and 1.4.7), in each classroom the teacher used the regular settings.
A brief introduction to the experiment was made by each class teacher and one member of the research team. Once the video finished, students were asked to take two tests (objective and subjective) in a maximum of 25 minutes. Both tests are described further in the next section. Finally, the completed questionnaires were collected, and students were thanked for their participation in the experiment.
Assessment tools
According to [43], asking for the minimum of information required for the purpose is a general principle for designing surveys that is specifically important when participants are children. This, together with the need of running the whole experience within the 45 minutes’ duration of a regular class, were the reasons to keep the questionnaire as short as possible.
As for the objective indicators, we have followed [6,44] that consider retention and transfer aspects of the learning process. The objective test included 10 questions (Q1-Q10), which were agreed on with the teachers. Each question was assigned a maximum grade of 1 point. The aim of this test was to measure the retention and transfer aspects of learning, taking prior knowledge into account:
- Prior knowledge: we are not aiming to assess either prior knowledge in 6th grade Mathematics or in the “prime number” topic, but the students’ knowledge about the atomic concept of “divisor”. It was agreed that this concept would be previously taught by the teacher in a regular class, as it necessarily preceded the concept of “prime number”. It is important to note that both videos include statements that apply the concept of divisor in the very beginning (e.g. “The divisors of 6 are: 1, 2, 3, and 6”, 00 m 45 s). Hence, the purpose of Q1 was to measure prior knowledge about the “divisor” concept.
- Retention: Q2-Q7 questions address the pedagogical contents that are explicitly mentioned in the video, and that are distributed homogeneously throughout the video timeline. Regarding their content validity, these questions cover the three concepts comprised within the “prime numbers” topic, described in the Materials section (pedagogical features), namely: 1) definition of prime and composite numbers, 2) geometrical interpretation of the prime numbers, and 3) the procedure to distinguish prime from composite numbers. These concepts cover the whole “prime numbers” chapter of the 6th grade Mathematics syllabus, and were agreed on with the teachers.
- Transfer: each of the transfer questions (Q8-Q10) covers one different basic concept of the syllabus, hence we assume their content validity.
The subjective test included questions on the video’s image and sound quality, comprehensibility, duration, and helpfulness for learning, as well as overall satisfaction with the video. Regarding its content validity, the questions address each of the principles of Universal Design that are applicable to our research, namely:
- Perceptible information: Question 1 (“Clearly seen”) addresses the perception of visual information, and question 2 (“Clearly heard”) deals with the perception of auditory information.
- Low physical effort: Question 3 (“Easy to understand”) focuses on the cognitive effort, and question 4 (“Short”) focuses on using “the design efficiently, comfortably and with a minimum of fatigue”.
- Equitable use: Question 5 (“Helpful”) addresses the usefulness aspect, while question 6 (“I liked it”) addresses the appealing aspect of the principle.
Its design and piloting followed the recommendations in [43] for measuring children’s opinions of technology. In line with this work, a Visual Analogue Scale based around a 1–5 Likert scale was used. Children were asked to indicate the extent to which they agreed with the corresponding statements by circling one of 5 “smileys” ranging from strongly agree (5), agree (4), normal (3), disagree (2), strongly disagree (1). [43] considers that this method is easy to complete, quick to complete, requires limited reading ability, and requires no writing. In order to reduce the satisficing effect, the test has been designed and piloted so that the questions are especially easy to understand, and the answers are easy to complete.
Furthermore, in agreement with [43], the objective and subjective questions were piloted with 20 students, in the same conditions of the large-scale trial. The aim of this pilot experiment was to identify if there were test questions that were not properly understood or had inappropriate language for the age of the students. As a result of this pilot experiment, abstract questions were replaced by specific questions (for example, "Given any number, how many divisors does it have at least?" was replaced by “What are the divisors of the number 17?”) and negative questions were replaced by affirmative questions (for example, "can you give an example of an odd number that is not prime?" for "can you give an example of an odd number that is composite?). Also the term “product” was replaced by “multiplication”. Another objective of the pilot experiment was to check that the time needed to complete the survey was appropriate.
The objective and the subjective tests are included in S1 Student Test.
Results
A total of 228 participants (112 males and 116 females) took part in the research experience. 117 participants watched video A (accessible) and 111 participants watched video B. The data have been analyzed with the R software.
The data measured is presented using the mean and the median as centralized statistics, and the standard deviation (SD) and the interquartile range (IR) as dispersion statistics. To compare the differences in quantitative variables between the groups, we established to use a parametric test (Student’s t-test) if the assumptions of normality and homoscedasticity were met, and a nonparametric test (Mann-Whitney test) if any of these assumptions were not met. To assess the normality we used the Shapiro-Wilk statistic, and for the homoscedasticity we used the Levene test. To compare differences in qualitative variables between the groups we used the Chi-square test. In all the tests the significance level of 0.05 was taken. The reliability of the assessment tools, in terms of internal consistency of their questions, was measured through Cronbach’s alpha.
The time it took participants to take the tests as described in Table 3 confirms that the time allowed within the experiment was appropriate.
Results of the objective test
Results for retention and transfer.
Regarding the reliability, Cronbach’s alpha is 0.761 for the quantitative questions (Q2-Q10), 0.747 for the retention questions (Q2-Q6), and 0.322 for the transfer questions (Q7-Q10). The scores obtained by participants in the retention questions are presented in Table 4. There is no statistical significance in the scores of the retention questions with respect to the two groups (Mann-Whitney, p = 0.875), although the median in group A (2.30) is higher than the median in group B (1.70), see Fig 4.
Scores obtained in the retention (left) and transfer questions (right) by group.
The scores obtained by participants in the transfer questions are presented in Table 5. There is no statistical significance in the scores of the transfer questions with respect to the two groups (Mann-Whitney, p = 0.339), being the median in group A (0.200) equal to the median in group B, see Fig 4.
Results for retention and transfer, when prior knowledge is considered.
Given that question Q1 measures the prior knowledge about the “divisor” concept, which is key for understanding both videos, we have analyzed the scores by group (A and B), and by the results obtained in Q1 (see Table 6). We only considered two groups according to the score obtained in the previous knowledge question: those students who scored 0, and those with a score > 0.
As expected, students who had a better previous knowledge scored higher in the retention test. Among them, the median in group A (2.80) was higher than the median in group B (2.50), see Fig 5, although these results were not statistically significant (Mann-Whitney, p = 0.953). In the case of individuals without previous knowledge the results were again not significant (Mann-Whitney p = 0.412) with a median in group A of 0.95 and 1.05 for group B, see Fig 5.
Scores obtained in the retention (left) and transfer questions (right), by group and by score obtained in the previous knowledge question (PKQ): PKQ = 0 (PKQ0), PKQ>0 (PKQ1).
We analyzed the scores obtained by participants in the transfer questions, both by group (A and B) and by the score obtained in the previous knowledge question (see Table 7). In both cases (PKQ = 0 and PKQ>0) the results were not statistically significant (Mann-Whitney, p = 0.570 and p = 0.403, respectively).
Results of the subjective test.
Regarding the reliability of the subjective questions, their Cronbach’s alpha is 0.807. Tables 8 and 9 present the accumulated percentages, and the mean ratings to the statements for students who chose either “strongly agree”, “agree” or “normal”, respectively. In the case of how participants rated the videos against the statement “I liked the video” we found statistical difference between the groups A and B (Chi-squared, p = 0.042). For that question, the accumulated percentages for students who chose either “strongly agree”, “agree” or “normal” was 87.9 (group A) and 78.9 (group B), see Table 8. Furthermore, in the case of the statements “The video could be seen clearly”, “The video could be clearly heard”, “The video was easy to understand”, “The video has been short”, and “The video has helped me learn” there was no statistical difference between groups A and B (Chi-squared, with p = 0.116, p = 0.461, p = 0.255, p = 0.214, p = 0.710, respectively).
Discussion
To assess the learning experience with each of these two multimedia materials, an anonymous evaluation was conducted through two sets of questions, aiming at assessing the quantitative and qualitative indicators, correspondingly. The internal consistency of the questions has been measured through Cronbach’s alpha, with acceptable values (>0.7) except for the transfer questions (0.322). With respect to the content validity, the quantitative questions cover all the content in the “prime numbers” chapter of the 6th grade Mathematics syllabus, and were agreed on with the teachers. The qualitative questions address each concept within the Universal Design principles that are applicable to our study.
The learning experience was carried out in real world settings, i.e., real classrooms where smart boards were used to provide image and sound to the students, following [4] we used a random assignment to the groups in order to fill the gap of previous studies noticed in [12].
Regarding the “Perceptibility of information”, the accessibility experts consider that only video A is fully perceptible for people with visual impairments. No differences in the material perception by the students (i.e., subjective test questions “The video could be seen clearly”, “The video could be clearly heard”) came out during the experience between videos A and B, which is not surprising as all who took part in the experience were non-disabled.
As for the “Equality of use”, there are no statistical differences either in the retention or in the transfer questions of the two groups of students. However, the score median in the retention questions is higher in the case of the group using the adapted video (2.30 vs. 1.70, see Table 4). In the case of students with better previous knowledge (i.e., knowledge about the “divisor” concept), they obtained better results both in the retention and in the transfer questions. Furthermore, within the group of students with better previous knowledge, those who watched the adapted video scored better both in retention (median of 2.80 vs. median of 2.50, see Table 6) and in transfer (median of 0.40 vs. median of 0.20, see Table 7), although these results are not statistically significant. In line with our results, [33] describe an eye-tracking study whose results confirm that audio description focuses the attention of non-disabled children, especially when new concepts are introduced, just as in our case.
Furthermore, the “Equality of use” also implies that design should be appealing to be used for all. In this respect, the analysis of the students’ ratings of the statement “I liked the video” in the subjective test indicates that the adapted video was found significantly more attractive than the original counterpart (p = 0.042), see Table 8. Our results are in line with the conclusions of the literature review in [42] indicating that higher levels of saturation and lightness of the color create a positive attitude toward the material, and that the color red should be avoided, see Fig 3 and Table 2. Our results also confirm those by [39], who concluded that audio description did not detract from the enjoyment of a TV program for elderly users with normal vision.
Regarding the “Low physical effort” principle, there are two effects derived from implementing the aural description of visual information in the adapted video (A), which was required by the accessibility experts to ensure the material’s perceptibility. On the one hand, the adapted video is necessarily longer (10 minutes 15 seconds vs. 8 minutes 43 seconds), while experts’ opinion says that shorter videos maximize learners’ concentration [47]. On the other hand, when producing the aural description we have adopted the strategy of writing a new script and recording a new video, rather than including the audio-description in the pauses of the dialogue, see Table 1. By doing so, we ensured that the description was concurrent with the visual information and therefore minimizing the cognitive load, accordingly with [44] and with [40]. In any case, and as stated above, using the adapted video did not hinder the learning process in our experience.
Limitations of the study
We acknowledge that the lack of internal consistency of our transfer questions limits the validity of the results obtained for the variable “transfer”. Hence, these questions should be redesigned in order to improve Cronbach’s alpha. Furthermore, considering that this was the very first time that the participants faced the concept of prime numbers, we could expect that most of the learning gained would be in the retention side, and that it was too early to measure the learning transfer. An alternative approach for measuring differences in learning transfer may require the re-design of the learning experience, with additional learning activities are included after the students use the video, and before they take the transfer quiz [48].
Other limitations derived from the scope of our study have to do with the type of multimedia content we have considered, its application in different curricula, methods of inclusion, levels of students’ achievement, etc.
Conclusions and future work
In this paper we apply and discuss a subset of the Universal Design criteria (specifically those addressing the needs of students with visual impairments) in order to produce and use Mathematics multimedia for primary education students. Also, this paper has analyzed how applying accessibility criteria for people with visual disabilities when creating Mathematics multimedia material, can affect the amount and depth of learning for non-disabled students, as well as the organizational implications of adopting Universal Design. To our knowledge it has never been studied before.
Our study is in agreement with others that indicate that inclusive education shows neutral to positive effects on students both with and without disabilities, e.g. generic studies such as [31,32], as well as studies on learning through multimedia, such as [33]. In particular we have found statistical significance in the preference of accessible multimedia. So, we can say that our study provides scientific evidence towards demonstrating that everybody can benefit from the recommendations for developing good quality, accessible multimedia material. Such evidence is one of the critical components of a successful collaboration between teachers and specialist educators (accessibility experts in our case) [29].
Furthermore, this should encourage the increase of awareness about accessibility as well as the creation and adoption of guidelines that suit the emerging ways to produce multimedia in primary education (e.g., [9]). Having such guidelines seems to be especially necessary in this context, as most primary education teachers who produce their own videos can’t rely on accessibility experts. Adopting those guidelines from the very beginning of the authoring process complies with applicable best practices in multimedia learning [44] and universal design of multimedia learning materials [40]. These measures need the support from policy-makers since, in agreement with [28], “policy can go far in articulating an organizational commitment on accessibility, and building an ‘accessibility-first’ mindset across the organization”. Authors of [30] state that teachers should have the skills necessary to use effective practices (“those validated through rigorous research”) and school managers should have tools to support both teachers and students. As a prerequisite for the latter, college and university preparation programs that train teaching professionals should be improved [52].
Regarding the measurement of the learning transfer, the set of questions should be modified in order to increase its internal consistency. We believe that the number of questions should be increased, and that more specific questions should replace the current ones, which might result a bit abstract for primary education students. Furthermore, the use of additional learning activities could contribute to a better training for the students before they face the transfer quiz, and to measure potential differences among the two groups. Additionally, we need to note that the videos used in our study do not include either complex visual information or complex mathematical expressions. Accommodating any of these contents to visually impaired students might require other techniques that are not considered in our work. Hence, further research is needed for understanding the implications of Universal Design in the case of videos with those types of contents. Still, more research is needed to answer that question bearing in mind different curricula, methods of inclusion, levels of students’ achievement, etc. As an improvement to our research experience, we are planning to use affective computing to complement the information provided by subjective tests, as well as to explore the use of interactive learning objects [53], including interaction with annotated videos, see [54] and [55].
Supporting information
S1 Student test. Appendix.
This file contains the test used during the learning experience.
https://doi.org/10.1371/journal.pone.0208117.s001
(DOCX)
S1 Data. Data file.
This file contains the data collected during the learning experience.
https://doi.org/10.1371/journal.pone.0208117.s002
(XLS)
Acknowledgments
The authors would like to thank the Fuhem (http://www.fuhem.es/ Madrid, Spain), a not-for-profit institution, and its three schools where the study took place.
References
- 1. Griffin DK, Mitchell D, Thompson SJ. Podcasting by synchronising PowerPoint and voice: What are the pedagogical benefits? Comput Educ. septiembre de 2009;53(2):532–9.
- 2. He Y, Swenson S, Lents N. Online Video Tutorials Increase Learning of Difficult Concepts in an Undergraduate Analytical Chemistry Course. J Chem Educ. Agosto de 2012;89(9):1128–32.
- 3. Kay RH. Exploring the use of video podcasts in education: A comprehensive review of the literature. Comput Hum Behav. mayo de 2012;28(3):820–31.
- 4. Yang F-Y, Chang C-Y, Chien W-R, Chien Y-T, Tseng Y-H. Tracking learners’ visual attention during a multimedia presentation in a real classroom. Comput Educ. marzo de 2013;62:208–20.
- 5.
Sweller J. Instructional Design in Technical Areas. Australian Education Review, No. 43. [Internet]. PCS Data Processing, Inc., 360 W. 31st, New York, NY 10001; Tel: 212-564-3730; Fax: 212-967-0928.; 1999 [citado 29 de julio de 2015]. Disponible en: http://eric.ed.gov/?id=ED431763
- 6.
Mayer RE. Multimedia learning. Cambridge university press; 2009.
- 7.
van Merriënboer JJG. Training complex cognitive skills: a four-component instructional design model for technical training. Englewood Cliffs, NJ: Educational Technology; 1997.
- 8. Luzón JM, Letón E. Use of animated text to improve the learning of basic mathematics. Comput Educ. octubre de 2015;88:119–28.
- 9. Tucker B. The flipped classroom. Educ Next. 2012;12(1):82–83.
- 10.
Balaouras P, Gatzonis M, Tsimpanis K, Apostolidis H, Tsiatsos Τ. Towards an online video platform for MOOCs. Multimed Tools Appl [Internet]. 12 de julio de 2017 [citado 20 de julio de 2017]; Disponible en: http://link.springer.com/10.1007/s11042-017-4913-5
- 11. Loomes M, Shafarenko A, Loomes M. Teaching mathematical explanation through audiographic technology. Comput Educ. Enero de 2002;38(1–3):137–49.
- 12. Kay R, Kletskin I. Evaluating the use of problem-based video podcasts to teach mathematics in higher education. Comput Educ. septiembre de 2012;59(2):619–27.
- 13.
UNESCO. The Salamanca Statement Framework for Action [Internet]. UNESCO; 1994. Disponible en: http://www.unesco.org/education/pdf/SALAMA_E.PDF
- 14.
Mittler P. Working towards inclusive education: Social contexts. David Fulton; 2000.
- 15. Freire S, César M. Evolution of the Portuguese special education system. A deaf child’s life in a regular school: Is it possible to have hope? Educ Child Psychol. 2002;19(2):76–96.
- 16.
World Wide Web Consortium. Web Contents Accessibility Guidelines (WCAG) 2.0. W3C; 2008.
- 17.
Lewin T. Harvard and M.I.T. Are Sued Over Lack of Closed Captions. The New York Times. 12 de febrero de 2015;
- 18.
edX. Do edX courses have transcripts or captions? [Internet]. 2016. Disponible en: https://support.edx.org/hc/en-us/articles/207205917-Do-edX-courses-have-transcripts-or-captions-
- 19.
edX. Accessibility Best Practices for Developing Course Content—Building and Running an edX Course documentation [Internet]. [citado 8 de junio de 2017]. Disponible en: http://edx.readthedocs.io/projects/edx-partner-course-staff/en/latest/accessibility/best_practices_course_content_dev.html#creating-accessible-media
- 20.
Rose DH, Meyer A. Teaching every student in the digital age: Universal design for learning. ERIC; 2002.
- 21.
CAST. Universal Design for Learning Guidelines version 2.0. Wakefield, MA: Author; 2011.
- 22.
Connell BR, Jones M, Mace R, Mueller J, Mullick A, Ostroff E, et al. The principles of universal design. En: The universal design file. New York, NY, USA: State University Press; 1997. p. 34–5.
- 23.
ISO. ISO 9241–210:2010 Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems. ISO; 2010.
- 24. Burgstahler S. Distance Learning: Universal Design, Universal Access. AACE J. 2002;10(1):32–61.
- 25. Burgstahler S, Corrigan B, McCarter J. Steps toward making distance learning accessible to students and instructors with disabilities. Inf Technol Disabil. 2005;11(1).
- 26. Marsico MD, Kimani S, Mirabella V, Norman KL, Catarci T. A proposal toward the development of accessible e-learning content by human involvement. Univers Access Inf Soc. 21 de junio de 2006;5(2):150–69.
- 27.
Kinash Shelley, Crichton Susan. Supporting the disabled student. En: Moore Michael G., editor. Handbook of Distance Education. 2nd ed. Routledge; 2007. p. 193–204.
- 28.
Horton S, Sloan D. Accessibility in Practice: A Process-Driven Approach to Accessibility. En: Langdon MP, Lazar J, Heylighen A, Dong H, editores. Inclusive Designing: Joining Usability, Accessibility, and Inclusion [Internet]. Cham: Springer International Publishing; 2014. p. 105–15. Disponible en: http://dx.doi.org/10.1007/978-3-319-05095-9_10
- 29. Garderen D van, Scheuermann A, Jackson C, Hampton D. Supporting the collaboration of special educators and general educators to teach students who struggle with mathematics: An overview of the research. Psychol Sch. Enero de 2009;46(1):56–78.
- 30. Smith DD, Tyler NC. Effective inclusive education: Equipping education professionals with necessary skills and knowledge. PROSPECTS. 30 de agosto de 2011;41(3):323.
- 31. Cole CM, Waldron N, Majd M. Academic progress of students across inclusive and traditional settings. Ment Retard. abril de 2004;42(2):136–44.
- 32. Kalambouka A, Farrell P, Dyson A, Kaplan I. The impact of placing pupils with special educational needs in mainstream schools on the achievement of their peers. Educ Res. Diciembre de 2007;49(4):365–82.
- 33.
Krejtz I, Szarkowska A, Krejtz K, Walczak A, Duchowski A. Audio Description As an Aural Guide of Children’s Visual Attention: Evidence from an Eye-tracking Study. En: Proceedings of the Symposium on Eye Tracking Research and Applications [Internet]. New York, NY, USA: ACM; 2012 [citado 29 de julio de 2015]. p. 99–106. (ETRA ‘12). Disponible en: http://doi.acm.org/10.1145/2168556.2168572
- 34.
Described and Captioned Media Program. Description Key for Educational Media [Internet]. Described and Captioned Media Program; 2008. Disponible en: https://www.dcmp.org/descriptionkey/
- 35. Pettitt B, Sharpe K, Cooper S. AUDETEL: Enhancing television for visually impaired people. Br J Vis Impair. 1 de mayo de 1996;14(2):48–52.
- 36.
Greening J, Rolph D. Accessibility: raising awareness of audio description in the UK. Rodopi, Amsterdam, Holland; 2007.
- 37. Peli E, Fine EM, Labianca AT. Evaluating visual information provided by audio description. J Vis Impair Blind. 1996;90:378–385.
- 38. Kirchner C, Schmeidler E. Adding Audio Description: Does it make a difference? J Vis Impair Blind JVIB. 2001;95(04).
- 39.
Carmichael A, Rabbitt PMA. Audio description of television for the visually disabled elderly: Assessment of the effect of audio description of elderly people’s comprehension and memory for a television program (AUDETEL Project Deliverable 15, Part 2 of Final Report). Brussels: Commission of Europeans Communities Directorate General XIII; 1993.
- 40. Udo JP, Fels DI. The rogue poster-children of universal design: closed captioning and audio description. J Eng Des. 1 de junio de 2010;21(2–3):207–21.
- 41. Moreno R. Optimising learning from animations by minimising cognitive load: Cognitive and affective consequences of signalling and segmentation methods. Appl Cogn Psychol. 2007;21(6):765–781.
- 42. Um E, Plass JL, Hayward EO, Homer BD, others. Emotional design in multimedia learning. J Educ Psychol. 2012;104(2):485–98.
- 43. Read JC. Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology. Cogn Technol Work. 22 de mayo de 2007;10(2):119–28.
- 44. Mayer RE, Anderson RB. The instructive animation: Helping students build connections between words and pictures in multimedia learning. J Educ Psychol. 1992;84(4):444–52.
- 45. Stanisavljevic Z, Nikolic B, Tartalja I, Milutinovic V. A classification of eLearning tools based on the applied multimedia. Multimed Tools Appl. 2015;74(11):3843–80.
- 46. Mayer RE, Wittrock MC. Problem-solving transfer. Handb Educ Psychol. 1996;47–62.
- 47. Questioning Shephard K., promoting and evaluating the use of streaming video to support student learning. Br J Educ Technol. junio de 2003;34(3):295–308.
- 48. van Gog T, Verveer I, Verveer L. Learning from video modeling examples: Effects of seeing the human model’s face. Comput Educ. marzo de 2014;72:323–7.
- 49.
ONCE. Dirección de Educación. Grupo de Accesibilidad Plataformas Educativas. Pautas para el diseño de entornos educativos accesibles para personas con discapacidad visual [Internet]. ONCE; 2005. Disponible en: http://educacion.once.es/documentacion-1/accesibilidad/Pautas_Ent_Educ_2005_cast.pdf
- 50.
Webaim. PowerPoint Accessibility [Internet]. Webaim; 2014. Disponible en: http://webaim.org/techniques/powerpoint/
- 51.
Willings C. Material adaptations [Internet]. Teaching Students With Visual Impairments; 2013. Disponible en: http://www.teachingvisuallyimpaired.com/material-adaptations.html
- 52.
Holdheide LR, Reschly DJ. Teacher Preparation to Deliver Inclusive Services to Students with Disabilities: TQ Connection Issue Paper [Internet]. National Comprehensive Center for Teacher Quality; 2008 [citado 14 de septiembre de 2018]. Disponible en: https://eric.ed.gov/?id=ED543818
- 53. Arroyo I, Woolf BP, Burelson W, Muldner K, Rai D, Tai M. A Multimedia Adaptive Tutoring System for Mathematics that Addresses Cognition, Metacognition and Affect. Int J Artif Intell Educ. 30 de septiembre de 2014;24(4):387–426.
- 54. Yu HQ, Pedrinaci C, Dietze S, Domingue J. Using linked data to annotate and search educational video resources for supporting distance learning. Learn Technol IEEE Trans On. 2012;5(2):130–142.
- 55.
Encelle B, Ollagnier-Beldame M, Pouchot S, Prié Y. Annotation-based video enrichment for blind people: a pilot study on the use of earcons and speech synthesis. En: The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility. Dundee, Scotland, UK: ACM; 2011. p. 123–30.