Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Optimizing learning outcomes in physical education: A comprehensive systematic review of hybrid pedagogical models integrated with the Sport Education Model

Abstract

Background

There is a notable gap in systematic reviews concerning hybrid pedagogical models (PMs) integrated with the Sport Education Model (SEM) and their impact on students’ outcomes.

Purpose

Which hybrid PMs incorporating SEM are currently the mainstream choices in research, and what are the main factors supporting their integration? How does SEM function as a foundational model in these hybrid teaching approaches? What learning outcomes are optimized through the hybrid models that combine SEM with other PMs?

Methods

A systematic search was conducted in major databases in December 2023 following PRISMA guidelines. Out of the identified 1342 studies, 30 met the eligibility criteria, all of which were deemed to be of high quality.

Results

Seven hybrid types were identified, primarily composed of two PMs, among which the blend of SEM and Teaching Games for Understanding (TGfU) emerges as the mainstream in current research. SEM, serving as the foundational structure, provides a stable framework for the hybrid, termed the "SEM + 1 model," yielding positive effects on enhancing students’ learning outcomes.

Conclusions

Pedagogical models align with PMs’ motivational aspects, thus enhancing learning outcomes. However, evidence for partial hybrids is lacking. Future research should explore diverse interventions, addressing coherence and teacher competence, while maintaining fidelity.

1. Introduction

The Pedagogical Model (PM) is a blueprint that "describes the specific procedures for organizing content, task structures, and the sequence of learning activities" [1]. Each model has specific design specifications and is considered a framework [2], allowing teachers to choose the most effective methods for delivering the model in different local contexts based on their perceived relevant teaching skills [3]. Furthermore, Pedagogical Models undergo a process of generation, testing, refinement, and further testing in various situations, reflecting their continuous modification and development [3]. Consequently, physical education (PE) has been immersed in a process of continually updating the teaching methods and PMs used in the classroom [4].

Research on teaching methods in PE encompasses various PMs that allow students to acquire breadth and depth of PE knowledge in diverse contexts [3]. As an alternative to direct instruction, a range of pedagogical models (PMs) have been introduced, including second-generation models such as Teaching Games for Understanding (TGfU) by Bunker and Thorpe [5], the Sport Education Model (SEM) by Siedentop [6], Cooperative Learning, and the Teaching Personal and Social Responsibility (TPSR) model by Hellison [7]. These PMs drive the shift from teacher-centered to student-centered instructional approaches, featuring key design characteristics that enhance not only student learning but also motivation [8]. For instance, the Sport Education Model (SEM) aims to provide students with authentic and educationally meaningful movement experiences within the school sports context, with six key structural characteristics: seasons, affiliation, formal competition, culminating activities, record-keeping, and festivals [9]. Cooperative Learning (CL) emphasizes learning together, from each other, and for each other, aiming to foster five essential elements: interpersonal skills, processing, positive interdependence, interaction promotion, and individual responsibility [10]. The core idea behind Teaching Games for Understanding (TGfU) is to shift the focus from technical aspects of gameplay to situations (tactical considerations) through modification, representation, and exaggeration [11]. It centers on placing learners in game situations where tactics, decision-making, and problem-solving are non-negotiable features, while skill practice is used to correct habits or reinforce skills. TGfU is based on six structural steps: games, game appreciation, tactical awareness, making appropriate decisions, skill execution, and performance [12]. The original TGfU model has evolved into several frameworks worldwide, such as the Step-Game Approach (SGA), Invasion Games Competence Model (IGCM), Tactical Game Approach (TGA) [13], Game Sense (GS) [14], Play Practice (PP) [15], Tactical Decision Learning Model (T-DLM) [16], and Developmental Game Stages Model (DGSM) [17], categorizing them under the umbrella term "game-centered approaches" (GCA). Health-Based Physical Education (HBPE) encourages healthy lifestyles by integrating health education into physical education programs, aiming to enhance students’ physical, mental, and social well-being [18]. Social and Emotional Learning (SEL) focuses on developing students’ social and emotional skills, such as self-awareness, self-management, social awareness, relationship skills, and responsible decision-making, which are crucial for success in school and life [19]. Self-Regulated Physical Education Model (SPRM) promotes students’ ability to manage their own learning processes, including goal setting, self-monitoring, and self-reflection, to become autonomous and motivated learners [20]. Finally, Teaching Personal and Social Responsibility (TPSR) is a model that focuses on fostering individual and social responsibility through five developmental goals: (1) Respect and self-control; (2) Participation and effort; (3) Self-direction; (4) Leadership and caring; and (5) Transfer. By addressing these goals, TPSR enhances students’ basic psychological needs, thereby increasing their motivation and engagement in physical activities beyond PE classes [21].

Currently, the aforementioned PMs have the potential to be beneficial for students’ development. However, it is widely acknowledged that no single model can be universally applicable to all PE environments [2, 22, 23]. Each model has its own strengths and limitations, and their effectiveness can vary depending on the context in which they are implemented. This is because each model is developed for specific curriculum objectives, and thus, each model has its limitations when implemented in isolation [17]. For example, while SEM aims to cultivate competent, literate, and enthusiastic individuals, it might not simultaneously foster social skills improved through Cooperative Learning (CL) group tasks or a sense of responsibility in students enhanced through Teaching Personal and Social Responsibility (TPSR) instruction. Moreover, SEM can sometimes be limited by its focus on sports and competition, which may not appeal to all students and could potentially marginalize those less interested in competitive sports [1]. Another limitation is that SEM requires significant time and resources to implement effectively, which may not always be feasible in all school contexts [2]. Additionally, SEM’s structured seasons and formal competitions may not leave enough room for spontaneous or student-led activities, potentially reducing opportunities for creativity and flexibility in learning [22]. These limitations provide compelling reasons for hybridizing SEM with other PMs. Hybrid models can address the shortcomings of individual PMs by combining their strengths. For instance, integrating SEM with CL can enhance social interaction and teamwork, while blending SEM with TPSR can promote personal and social responsibility alongside sports competence. Evaluating the effects of hybrid PMs, which include SEM, is crucial to understanding their potential benefits in diverse educational settings and to developing more comprehensive and effective physical education programs [2325].

In pursuit of maximizing benefits for students in teaching or achieving specific teaching goals, some researchers have attempted to introduce how to integrate two PMs to create a new hybrid PM [2629]. Hybridizing PMs involves extracting and combining key features from multiple models or using one model as a base and incorporating additional essential elements from another model. For instance, in SE-TGfU, the SEM provides key organizational features (seasons, persisting teams, formal competition, record keeping, festivity, and a culminating event), while the Teaching Games for Understanding (TGfU) contributes the primary instructional approach (teacher-mediated formats (e.g., active teaching and teaching through questioning) and student-mediated formats (e.g., peer tutoring, small-group/cooperative learning approaches) to assist educators in identifying key tactical issues within each team, such as (a) tactical or strategic awareness (What do you do to keep the ball away from other players?), (b) skill execution (How do you keep the ball away from other players?), (c) time (When is the best time to pass?), (d) space (Where should you move when your teammate is trying to restart play?), (e) risk (What options do you have if your defender is near?), and (f) rationale (Why should you move after passing?) [30]. In addition, Casey and Dyson combined the emphasis on encouraging adaptation to individual differences, fair teaching, and personal and social development of CL with the TGfU, which prioritizes the development of correct decision-making and tactical awareness over skilled performance. They found that hybridizing these two PMs is an extremely complex and challenging task, with potential benefits for both students and teachers if the models have similar characteristics and goals [31]. Likewise, recent research has demonstrated the feasibility of hybridizing TPSR and SEM by observing that they share similar learning theories and complementary objectives. For example, both models emphasize student-centered learning and personal development [32]. Combining TPSR’s focus on personal and social responsibility with SEM’s structure and organization can enhance student engagement and learning outcomes [32]. Moreover, hybrid models can address the limitations of individual PMs by leveraging their strengths, leading to more comprehensive educational experiences [25].

Previous studies also indicated that hybridization can enhance game performance and motor skills [26, 29] and yield positive psychosocial outcomes, such as enjoyment, willingness for physical activity, and responsibility [33, 34]. Hybrid PMs serve as effective resources for developing flexible physical education programs within multimodal projects. Teachers can choose and blend appropriate PMs based on students’ situations to meet the diverse needs of different teaching objectives. Therefore, this is considered an innovative trend and a necessary means for achieving higher-quality learning outcomes [22, 35].

Despite an increasing number of studies on hybrid PMs in recent years, systematic reviews regarding the hybridization of PMs remain scarce. Indeed, only two systematic reviews have comprehensively examined various hybridizations between 2019 and 2022 [3, 35]. The first review in this field by González-Víllora et al. primarily provides fundamental information on PM hybridization in PE, covering aspects such as Hybridization, Research Focus, Participants and Context, Sport/Content, Length of Implementation, Data Sources and Analysis, and Outcomes [35]. It acknowledges that hybridization benefits students in terms of game-related skills and psychosocial variables. To extend the work of González-Villora et al., Shen & Shao further investigate the impact and mechanisms of hybrid models on students’ learning outcomes (i.e., motor, cognitive, emotional, and social) and provide a comprehensive review of empirical studies on various PM hybridizations. The results suggest that the duration of implementation and teacher familiarity are the primary limiting factors for the implementation of hybrid PM teaching [3]. Notably, from the two systematic reviews, it is evident that SEM is the most frequently appearing model in hybrid PM teaching, constituting 90% and 88% in the first and second reviews, respectively. This raises the questions: which hybrid pedagogical models incorporating SEM are currently the mainstream choices in research, and what are the main factors supporting their integration? How does SEM function as a foundational model in these hybrid teaching approaches? What learning outcomes are optimized through the hybrid models that combine SEM with other pedagogical models? However, these aspects have not been extensively explored or explained in previous research. Therefore, building on prior work, this study aims to conduct a comprehensive systematic review of the role of hybrid PMs, combined with the SEM, in optimizing learning outcomes in physical education. Specifically, this study will investigate the impact of these hybrid models, integrated with SEM, on students’ learning outcomes. Additionally, the study seeks to explore the advantages of SEM as a foundational model in hybrid PMs.

2. Methods

2.1 Protocol and registration

The PRISMA statement was followed in reporting this systematic review and meta-analysis [36], and the review protocol has been registered on Inplasy.com: [INPLASY202410027].

2.2 Data sources and search strategy

On December 3, 2024, a search of four electronic databases, namely Web of Science, EBSCO host, PubMed, and SCOPUS, was conducted to identify relevant articles on the topic. Previous reviews [3, 35] were consulted to guide the formulation of the search strategy, with keywords and Boolean operators considered both individually and in combination during the search across the four databases (see S1 Table). The search employed terms and operators such as "Sport Education," "pedagogical model," "curriculum model," "instructional models," "physical education," "hybrid*," "Integrate*," and "combine*." Additionally, a manual search of Google Scholar was performed to retrieve missing studies, including article citations and free-text searches. Moreover, all identified articles underwent reference list screening to identify any publications not initially found during the database searches (See S1 Checklist for details).

To ensure the robustness of the data collection process, experienced librarian Miss Chen assisted. To mitigate selection bias, two knowledgeable authors (J.Z. and M.A.), both familiar with SEM, independently screened and selected studies. In cases of disagreement, a third reviewer (K.G.S) was involved to reach a consensus. Following the review of titles and abstracts, publications not meeting the inclusion criteria were excluded. Full-text examination was conducted based on exclusion criteria, and the final set of 30 articles was included for systematic review and analysis (Fig 1 and S1 File).

2.3 Eligibility criteria

A PICOS framework [18] was used to rate studies for eligibility. The criteria include the following:

  1. Inclusion Criteria: (1) Published in peer-reviewed international journals; (2) Implementation of a hybrid PM incorporating the SEM in physical education (PE) environment; (3) Primary findings report at least one aspect of the impact of the hybrid PMs on student learning outcomes; (4) Articles published and written in English; (5) Empirical studies utilizing quantitative, qualitative, or mixed research methods.
  2. Exclusion Criteria: (1) Books, book chapters, conference proceedings, master’s theses, and doctoral dissertations not subjected to independent peer review are excluded; (2) Studies published in non-peer-reviewed journals and/or not indexed in the Journal Citation Reports (JCR) or Science Journal Rankings (SJR) are excluded; (3) To align with the research objectives, studies that do not design hybrid PMs with SEM and those not specifically measuring any aspect of learning outcomes are excluded.

2.4 Data extraction

Based on the pertinent reviews in the realm of the physical education pedagogical model [3, 35], we have succinctly outlined the distinctive features of each retained study: Sources, Purpose, Sample Characteristics, Hybrid, Length of Unit/Content, Data Collection, Study Design & Analysis, Learning Outcomes (Table 1). This process involved two reviewers (J.Z. and M.A.) obtaining information on each study through Microsoft Excel spreadsheets (Microsoft Corporation, Redmond, WA, United States), and a third reviewer (K.G.S) subsequently verified its accuracy.

Previous studies have indicated that learning outcomes encompass motor learning, cognitive learning, affective learning, and social learning [3, 9]. Motor learning is defined as body growth-related physical characteristics and technical skills [3]; cognitive learning primarily explains learning about strategies and decision-making abilities [63]; affective learning typically includes psychological factors such as confidence, self-esteem, motivation, and self-worth [33]; social learning comprises (a) interpersonal communication skills; (b) the ability to build relationships and listen to team members’ opinions; (c) the sharing of beliefs, perspectives, and the collaborative construction of new understandings [10]. This serves as the reference basis for data extraction in this section.

2.5 Quality assessment and level of evidence

The updated PRISMA framework [36] was initially employed to assess the quality of this systematic review. Subsequently, the quality of published manuscripts was evaluated using a checklist adapted from the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement [64]. Referring to the 9 assessment criteria chosen by Shen and Shao for the structure of typical publications in this research domain: (1) Description of PM Hybridizations, (2) Characteristics of the Participants, (3) Reasonable Design of the Study, (4) Detailed Data Collection, (5) Detailed Data Analysis, (6) Validity and Reliability, (7) Inclusion of Models’ Fidelity, (8) Report of Learning Outcomes, and (9) Discussion of Results. Each item was scored as 1 (yes) or 0 (no). The total quality score for each included study was determined by summing individual scores. Studies scoring 7 or above were categorized as "high quality," those scoring between 4 and 6 were classified as "moderate quality," and those scoring below 4 were classified as "low quality" [3]. Manuscripts had to score at least 4 points for inclusion. Two independent researchers (J.Z. and M.A.) assessed the selected studies. The final scores were reviewed and discussed by a research team composed of other co-authors, with any discrepancies negotiated with a third researcher (K.G.S) until a consensus was reached.

3. Results

3.1 Study background

The majority of the 30 studies were conducted in Western countries, with the highest number in Spain (13), followed by Portugal (10), China (3), the United States (2), Turkey (1), and Australia (1). The earliest article in this field was published in 2000 [52], with 20 articles published in the last five years, indicating that research on the hybridization of PM with SEM in physical education has become a recent focal point.

3.2 Participants

The majority of studies examined the effects of hybrid PMs in physical education classes for primary and secondary school students, with a combined sample size of 2,399 participants. Specifically, 70% of the studies focused on middle school students, totaling 1,238 participants, while seven studies targeted elementary school students, with 733 participants. Interestingly, no studies addressed university students. Additionally, two studies reported participant numbers but did not specify the ages of the students. The hybrid PMs curriculum primarily centered around sports such as soccer, basketball, volleyball, handball, taekwondo, frisbee, and various game-based activities like invasion games and batting/fielding games.

3.3 Hybrid curriculum implementation

Seven hybrid types were identified across all literature. One study employed a combination of three PMs: SEM-CL-HBPE [29], while the rest involved combinations of two PMs: SEM-TGFU (14), SEM-SGA (6), SEM-TPSR (4), SEM-IGCM (3), SEM-SEL (1), SEM-SPRM (1). Most studies emphasized the organization structure based on SEM seasons, incorporating elements of other PMs, forming a "SEM + 1" model. The learning tasks and teaching content of SEM seasons mainly originated from game-centered models (TGFU, IGCM, and SGA) and TPSR.

3.4 Study design and data collection

Different research designs were employed for various instructional objectives. Qualitative studies primarily collected data through interviews and observations, while quantitative studies mainly utilized scales and questionnaires. There was a total of 20 quantitative studies, 7 qualitative studies, and 3 mixed-methods studies.

3.5 Quality assessment and level of evidence

The results of the quality assessment are presented in Table 2, indicating that all 30 studies are of high quality. The primary factors influencing article quality are validity and reliability, and the inclusion of models’ fidelity. Among these, 12 studies did not conduct validity and reliability testing, and 22 studies lacked models’ fidelity procedures.

thumbnail
Table 2. Study quality checklist with quality scores assigned.

https://doi.org/10.1371/journal.pone.0311957.t002

3.6 Optimization of learning outcomes by hybrid pedagogical model

This section aims to demonstrate the impact of hybrid PMs included in the literature on learning outcomes.

3.6.1 The impact of hybrid SEM-TGFU on students’ learning outcomes.

Existing research indicates that hybrid SEM-TGFU benefits students in terms of motor learning, cognitive learning, and affective learning outcomes.

Motor Learning: Stran et al. found that SEM-TGFU increased student engagement in the classroom [61]. Guijarro & MacPhail concluded that the use of hybrid SEM-TGFU units surpassed SEM in decision-making, support, overall game performance, and game participation, leading to improved game performance and engagement [51]. Jia found a significant impact of SEM-TGFU on students’ motor skills [54]. Notably, Oliveros & Fernandez-Rio’s results showed that female students scored significantly lower than males in high-intensity physical activity (MVPA) during hybrid SEM-TGFU, emphasizing that the hybrid PM itself may not assist girls in achieving MVPA scores similar to boys. Additionally, the study found that MVPA for lower-grade students was significantly higher than for higher-grade students [57]. Gouveia et al. concluded that SEM-TGFU significantly reduced classroom sedentary time [50]. López-Lemus et al. [55] demonstrated that the implementation of hybrid models SEM/TGFU could enhance students’ game involvement and performance, as well as increase enjoyment, perceived competence, and intention to be physically active, in both boys and girls.

Cognitive Learning: Hastie & Curtner-Smith found that students could understand, appreciate, and execute some basic hitting, throwing, and outfield throwing tactics and strategies, as well as grasp the overall principles, rules, and structures of hitting/batting/fielding games, realizing their importance [53]. Jia found a significant impact of SEM-TGFU on students’ cognitive performance [54].

Affective Learning: Gil-Arias et al. reported that students showed significant improvements in autonomy, competence, and enjoyment [46]. Gil-Arias et al. further found a significant increase in students’ input, praise for autonomous behavior, perceived competence, and enjoyment [47]. Two additional articles from their team, published in the same year, added comparisons between students of different genders, highlighting a larger effect size in girls, emphasizing the importance of TGFU/SEM units in improving student motivation outcomes, especially for girls [51, 55]. Additionally, García-González et al. studied the impact of hybrid SEM-TGFU on students at different Relative Autonomy Index (RAI) levels, emphasizing a larger effect size in "medium" or "low" level students, although SEM-TGFU units in volleyball teaching were beneficial to students at all three levels [4]. The results of Buendía et al.’s study showed a significant difference in improving student enjoyment with SEM-TGFU [42]. Pan et al. found that SEM-TGFU had positive learning effects on students’ motivation, enjoyment of physical activity, sense of responsibility, and game performance, surpassing the effects of the TGFU model [58].

3.6.2 The impact of hybrid SEM-SGA on students’ learning outcomes.

Existing research indicates that hybrid SEM-TGFU benefits students in terms of motor learning, cognitive learning, affective learning, and social learning outcomes.

Motor Learning: Araújo et al. conducted three assessments of game performance using hybrid SEM-SGA for students of different genders and skill levels. The results showed improvement in all indicators of game performance, including game performance, game involvement, decision making, adjustment, skill efficiency, and skill efficacy, for both boys and girls from pre-test to post-test. Additionally, students with lower skill levels gained greater benefits, and there were no significant changes in all indicators between the post-test and retention test [37]. This reinforces the idea that the implementation of this hybrid approach in the future should be adjusted based on different skill levels in terms of content and learning tasks. Araújo et al. applied hybrid SEM-SGA to study the long-term development of students’ volleyball play performance over three years. The results of the survey showed an improvement in the play performance levels of all 18 students, and the study also concluded that the implementation over three seasons would gradually eliminate the gap in students’ skill levels [39]. It implies that the improvement effect of students with low skill levels is higher than that of students with high skill levels. Farias et al. investigated students’ game-play development using hybrid SEM-SGA, revealing improvements in most game variables (such as "serving," "receiving," and "setting") and the "game performance index" [45].

Cognitive Learning: Araújo et al. studied students’ pedagogical content knowledge of coaches using hybrid SEM-SGA, demonstrating the effectiveness of SEM-SGA in enhancing students’ coaches’ pedagogical content knowledge [38]. Araújo et al. proposed that students’ tactical understanding during hybrid SEM-SGA instruction significantly improved [40]. Farias et al. investigated the specialized content knowledge (SCK) of pre-service teachers using hybrid SEM-SGA, revealing a significant improvement in pre-service teachers’ SCK over time [45].

Affective Learning and Social Learning: Existing research suggests that hybrid SEM-SGA benefits students in terms of cognitive learning, affective learning, and social learning outcomes. Silva et al. conducted a unique survey analysis of pre-service teachers using hybrid SEM-SGA, employing inductive and deductive theme analysis. The results showed that the use of this hybrid approach helped teachers act as facilitators of learning, increased students’ levels of responsibility for their learning experiences, promoted students’ autonomy and sense of active control over classroom activities, facilitated the development of student’s abilities and volleyball fundamental knowledge, and increased their interest and engagement in physical education [60].

3.6.3 The impact of hybrid SEM-TPSR on students’ learning outcomes.

Existing research indicates that hybrid SEM-TPSR benefits students in terms of motor learning and social learning outcomes. Hastie & Buchanan concluded that hybrid SEM-TPSR allows achievement in a powerful triad of goals-motor skill competence, social responsibility, and personal empowerment [52]. Menendez-Santurio & Fernandez-Rio conducted hybrid SEM-TPSR teaching targeting students with disabilities and found that it helped them and their peers establish connections both inside and outside the classroom [34]. This result seems to broaden the range of beneficiaries. At the same time, it enriches the future research direction. The results of Fernandez-Rio & Menendez-Santurio’s study indicated that SEM-TPSR significantly enhances students’ responsibility and exercise self-efficacy [32].

3.6.4 The impact of hybrid SEM-IGCM on students’ learning outcomes.

Existing research indicates that hybrid SEM-IGCM benefits students in terms of motor learning, cognitive learning, and social learning outcomes.

Motor Learning and Social Learning: Mesquita et al. provided students with opportunities to improve skill execution and tactical decision-making through a hybrid SEM-IGCM soccer unit. Furthermore, inclusive participation had a strong impact on learning for girls and students with lower skill levels [56]. Farias et al. concluded that hybrid SEM-IGCM promoted improvements in students’ soccer game performance and understanding, increasing the correlation between the two concepts [43]. Farias et al. examined game performance in three consecutive basketball, handball, and soccer units based on the hybrid SEM-IGCM model. The results showed significant improvements in game performance for handball and soccer units, but not for basketball. This means that hybrid PMs may have different effects on different sports. However, when breaking down game performance into tactical structure indicators, improvements were observed in all seasons. The correlation between tactical structure indicators and game performance increased over time. The predictive model of game performance improvement demonstrated correlations with expanded team contextual features, coordinated interpersonal dynamics in the game, the nature of peer teaching mediation, and the game format [44].

3.6.5 The impact of hybrid SEM-SEL, SEM-SPRM, and SEM-CL-HBPE on students’ learning outcomes.

Due to the limited number of included studies for hybrid SEM-SEL, SEM-SPRM, and SEM-CL-HBPE (each with only one study), they will be collectively described here. Evangelio et al.’s study is the only one among the included literature that hybridized all three models. The results indicate that the current hybrid SEM-CL-HBPE can contribute to shaping habitual, motivated, critical, and informed advocates [29].

Using a 16-week SEM-SEL volleyball program based on the SEM, Arikan (2020) compiled a SEL. The results showed that the SEM-SEL volleyball program is effective in enhancing students’ emotional intelligence levels, advocating for further exploration of the compatibility of SEM and SEL hybridization.

Quiñonero-Martínez et al.’s results revealed different outcomes in motor learning. Traditional methodology, including standing long jump and speed-agility scores, showed significant improvement. Conversely, in SEM-SPRM, neither of the tests demonstrated significant improvement [59].

4. Discussion

4.1 Hybrid curriculum implementation

Our review of the literature reveals a lack of detailed descriptions regarding fixed or optimal methods for implementing the "SEM + 1" model. This may be due to the inherent flexibility of the "SEM + 1" approach, which allows teaching strategies to be adjusted based on specific instructional goals and contextual factors, effectively leveraging the strengths of both models [8]. Some studies have provided intervention programs that emphasize combining the characteristics and principles of the two models, offering guidance to both future researchers and frontline teachers [48, 58]. However, certain challenges remain in implementing these hybrid models. First, these intervention programs are often developed in specific educational contexts, which may lead to issues with adaptability when applied in different settings. Second, while some studies offer guidance, they may not fully address the practical difficulties teachers face and the professional support required for effective implementation, potentially impacting the model’s effectiveness. Additionally, there is often a gap between the theoretical design of these hybrid models and their practical execution in real-world classrooms. Teachers may struggle with balancing the dual demands of integrating two pedagogical models while maintaining coherence in their teaching, which can hinder the overall implementation process. Future efforts should focus on providing clearer implementation guidelines and professional development opportunities to better support educators in adopting these hybrid approaches.

4.2 The impact of hybrid SEM-TGFU on students’ learning outcomes

The popularity of SEM-TGfU as a hybrid pedagogical model can be attributed to three key factors: the theoretical foundation for constructing these PMs [32, 63, 64], the characteristics of the models themselves [9], and the motivational aspects they bring to physical education [65].

Firstly, while constructivist learning theory and game theory serve as the theoretical underpinnings for both SEM and TGfU, it is important to highlight that these theories support the effectiveness of the hybrid approach. Constructivism provides the rationale for creating active, student-centered learning environments, which is the cornerstone of SEM and TGfU. This theoretical foundation justifies why combining these models can enhance students’ learning experiences by promoting autonomy and engagement [32, 63, 64].

Secondly, the distinct characteristics of SEM and TGfU complement each other, making their integration beneficial for students. SEM’s emphasis on structured seasons, affiliation, formal competition, culminating events, record-keeping, and festivity helps students experience authentic sporting events through various roles [9]. However, SEM may not sufficiently address tactical understanding and skill development [65]. TGfU, on the other hand, fills this gap by focusing on game-based learning, tactical awareness, and skill execution through a structured approach [66]. By integrating TGfU’s instructional steps into SEM’s framework, students gain a more comprehensive understanding of sports, which includes both the organizational structure and tactical execution [46].

Lastly, the motivational aspects of the hybrid SEM-TGfU model further enhance student engagement. Both models encourage cooperative learning, which fosters a positive team environment and supports improved decision-making, game performance, and overall involvement [67]. This approach also reduces sedentary behavior, increases physical activity, and positively influences students’ motivation, enjoyment, and sense of responsibility [45, 68].

4.3 The impact of hybrid SEM-SGA on students’ learning outcomes

In addition to research on hybrid SEM-TGFU, there is a relatively significant amount of research on SEM-SGA. SAG emphasizes providing students with an appropriate framework for the development of tactical and technical skills in net sports to ensure the success of the game [37, 56].

Combining SEM with SGA retains SEM’s core elements such as stable teams, formal competition, and role-playing, while integrating SGA’s learning tasks and sports skills taught during the season. This combination enables students to enhance the acquisition of motor skills and performance in the game [37, 39, 45]. The primary purpose of this hybrid is also to address SEM’s lack of specific teaching strategies to cultivate students’ tactical competence [37].

4.4 The impact of hybrid SEM-TPSR on students’ learning outcomes

The integration of TPSR with SEM leverages SEM’s six characteristics (seasons, affiliation, formal competition, culminating event, record keeping, and festivity) to provide a structured and authentic learning environment. SEM-TPSR encourages cooperation, self-esteem development, and sustained social interactions [34]. This hybrid model supports students’ autonomy and responsibility, fostering attention to their rights, feelings, and the needs of others [32, 34].

In investigating how SEM functions as a foundational model, it becomes clear that the organizational strengths of SEM complement TPSR’s focus on personal and social responsibility. This synergy addresses key factors supporting their integration, such as shared theoretical foundations and mutual reinforcement of learning outcomes. Consequently, SEM-TPSR optimizes learning outcomes by promoting interpersonal skills, social responsibility, and overall student motivation [10, 33, 69].

4.5 The impact of hybrid SEM-IGCM on students’ learning outcomes

SEM-IGCM is also a prevalent choice in hybrid teaching due to its alignment with constructivist principles and the complementary strengths of both models. The Invasion Game Competence Model (IGCM), like the Step-Game Approach (SGA) and Teaching Personal and Social Responsibility (TPSR), originates from Teaching Games for Understanding (TGfU) and is rooted in constructivist learning theory and game theory. IGCM focuses on game-centered approaches, emphasizing structured planning, the development of tactical skills, decision-making, and skill execution in invasion sports. It provides a comprehensive framework for assessing competence in invasion games, considering overall game performance and participation.

When combined with the Sport Education Model (SEM), IGCM benefits from SEM’s structured framework, including stable teams, formal competitions, and role-playing, while IGCM enhances the learning plans and sports skills taught during the season. This hybrid approach allows students to improve skill acquisition and game performance [39, 43, 56].

4.6 The impact of hybrid SEM-SEL, SEM-SPRM, and SEM-CL-HBPE on students’ learning outcomes

The hybrid models of SEM-SEL, SEM-SPRM, and SEM-CL-HBPE facilitate the development of students’ sociological and affective learning abilities [29, 40]. The emphasis on cooperative learning and student engagement in these models likely contributes to their positive impact. SEM’s stable teams create a consistent platform for fostering social skills through collaborative learning [70], while HBPE promotes healthy lifestyles, motivating team members to prioritize physical well-being [71]. This group teaching approach enhances students’ ability to work within teams, improving social relationships. However, further research is needed due to the limited evidence in the literature.

Additionally, in SEM-SPRM, performance tests for standing long jump and speed and agility did not show significant improvement, whereas the traditional teaching group demonstrated significant enhancement. This suggests a need for further clarification on the compatibility of SEM and SEL hybrids [3].

4.7 Implementation limitations and future applications

First of all, since the premise of hybrid PM is that teachers have a comprehensive understanding of the theories, methods, and procedures of each mode and have teaching experience in implementing them, this creates major obstacles and challenges for pre-service teachers [34, 48, 61]. At the same time, teachers need to accurately select suitable hybrid pedagogical models (PMs) based on teaching needs. The implementation of hybrid PMs often centers around the six key features of the Sport Education Model (SEM). Teachers must repeatedly emphasize basic team agreements, individual roles, and how to conduct independent team activities. This focus can sometimes reduce the time available for the instructional content of the other teaching model, which may impact the overall progress of the class [53]. Therefore, in the future application of hybrid PMs, it is important to strengthen the early training of PE teachers on the use of hybridization. This training should include a focus on maintaining model fidelity, ensuring that teachers can accurately and consistently implement the hybrid models as intended. Without this fidelity, the benefits of hybrid PMs may not be fully realized, and the variability in implementation could lead to inconsistent outcomes. PE teachers should accurately understand student needs, consider the most suitable hybrid PM, and master the two PMs in hybridization. If implementation challenges arise or teaching progress is hindered, selecting one model may be advisable. Moreover, proficient teachers should adjust teaching content and tasks according to students’ skill levels to optimize learning outcomes.

5. Conclusion

This review examines hybrid pedagogical models (PMs) integrated with SEM to optimize students’ learning outcomes. It identifies seven hybrid PMs, primarily composed of two models, with the blend of SEM and Teaching Games for Understanding (TGfU) emerging as the mainstream in current research. SEM serves as the foundational structure, creating a stable framework for these hybrids, referred to as "SEM + 1 models." These hybrids have shown positive effects on enhancing students’ learning outcomes across various domains. While the motivational aspects of these models contribute to improved learning outcomes, evidence for some hybrid models remains limited. Future research should focus on exploring diverse interventions and addressing issues related to teaching coherence and teacher competence. Ensuring model fidelity and expanding empirical support are essential for the effective development and implementation of these hybrid models.

References

  1. 1. Hastie P A, Casey A. Fidelity in models-based practice research in sport pedagogy: A guide for future in-vestigations. Journal of Teaching in Physical Education. 2014; 33(3):422–31.
  2. 2. Kirk D. Educational value and models-based practice in physical education. Educational Philosophy and Theory. 2013; 45(9):973–86.
  3. 3. Shen Y, Shao W. Influence of hybrid pedagogical models on learning outcomes in physical education: A systematic literature review. International Journal of Environmental Research and Public Health. 2022; 19(15):9673. pmid:35955027
  4. 4. García-González L, Abós Á, Diloy-Peña S, Gil-Arias A, Sevil-Serrano J. Can a hybrid sport education/te-aching games for understanding volleyball unit be more effective in less motivated students? An examin-ation into a set of motivation-related variables. Sustainability. 2020; 12(15):6170.
  5. 5. Bunker D, Thorpe R. A model for the teaching of games in secondary schools. Bulletin of Physical Educ-ation. 1982; 18(1):5–8.
  6. 6. Siedentop D. Sport education: Quality PE through positive sport experiences. Champaign: Human Kineti-cs; 1994.
  7. 7. Hellison D. Teaching physical and social responsibility through physical activity just what is the system[J]. Physical Education and Sport Pedagogy, 2015, 83: 1–16.
  8. 8. López-Lemus I, Del Villar F, Rodríguez-Gutiérrez A, González-Silva J, Moreno A. Could the hybridizati-on of the SE/TGfU pedagogical models be an alternative for learning sports and promoting health? Sch-ool context study. Children. 2023; 10(5):877. pmid:37238425
  9. 9. Bessa C, Hastie P, Ramos A, Mesquita I. What actually differs between traditional teaching and sport ed-ucation in students’ learning outcomes? A critical systematic review. Journal of Sports Science and Med-icine. 2021; 20(1):110–25. pmid:33707994
  10. 10. Casey A, Goodyear V A. Can cooperative learning achieve the four learning outcomes of physical educ-ation? A review of literature. Quest. 2015; 67(1):56–72.
  11. 11. Barba-Martín R A, Bores-García D, Hortigüela-Alcalá D, González-Calvo G. The application of the tea-ching games for understanding in physical education: Systematic review of the last six years. Internatio-nal Journal of Environmental Research and Public Health. 2020; 17(9):3330. pmid:32403272
  12. 12. Morales-Belando M T, Kirk D, Arias-Estero J L. A systematic review of Teaching Games for Understa-nding intervention studies from a practice-referenced perspective. Research Quarterly for Exercise and Sport. 2022; 93(4):670–81. pmid:34705604
  13. 13. Gouveia É R, Gouveia B R, Marques A, Kliegel M, Rodrigues A J, Prudente J, et al. The effectiveness of a tactical games approach in the teaching of invasion games. Journal of Physical Education and Sport.2019; 19(3):962–70.
  14. 14. Jarrett K, Harvey S. Similar, but not the same: Comparing the game-based approaches of Teaching Gam-es for Understanding (TGfU) and Game Sense. Ejournal de la Recherche sur l’intervention en Éducation Physique et Sportive. 2016; (39):24–43.
  15. 15. Zhan P, Ward P, Li W, Sutherland S, Goodway J. Effects of play practice on teaching table tennis skills. Journal of Teaching in Physical Education. 2012; 31(1):71–85.
  16. 16. Gréhaigne J F, Wallian N, Godbout P. Tactical-decision learning model and students’ practices. Physical Education and Sport Pedagogy. 2006; 11(3):297–315.
  17. 17. Rivera-Pérez S, Fernandez-Rio J, Gallego D I. Effects of an 8-week cooperative learning intervention on physical education students’ task and self-approach goals, and emotional intelligence. International Jour-nal of Environmental Research and Public Health. 2021; 18(1):61. pmid:33374777
  18. 18. McKenzie J E, Brennan S E, Ryan R E, Thomson H J, Johnston R V, Thomas J. Defining the criteria for including studies and how they will be grouped for the synthesis. In: Higgins J P T, Thomas J, Chandler J, Cumpston M, Li T, Page M J, et al., editors. Cochrane Handbook for Systematic Reviews of Intervent-ions. Wiley; 2019. p. 33–65.
  19. 19. Durlak J A, Weissberg R P, Dymnicki A B, Taylor R D, Schellinger K B. The impact of enhancing stud-ents’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Dev-elopment. 2011; 82(1):405–32. pmid:21291449
  20. 20. Zimmerman B J. Becoming a self-regulated learner: An overview. Theory into Practice. 2002; 41(2):64–70.
  21. 21. Pienimäki I. Scoping review: Teaching personal and social responsibility through physical activity (TPS-R-MODEL). 2019. https://jyx.jyu.fi/handle/123456789/64838
  22. 22. Casey A. Models-based practice: Great white hope or white elephant? Physical Education and Sport Pe-dagogy. 2014; 19(1):18–34.
  23. 23. Metzler M W. Instructional models for physical education. 3rd ed. Routledge; 2011.
  24. 24. MacPhail A, Kirk D, Griffin L L. Throwing and catching as relational skills in game play: Situated learn-ing in a modified game unit. Journal of Teaching in Physical Education. 2008; 27(2):100–15.
  25. 25. Casey A, MacPhail A. Adopting a models-based approach to teaching physical education. Physical Edu-cation and Sport Pedagogy. 2018; 23(3):294–310.
  26. 26. Costa L. C. A. D., Nascimento J. V. D., & Vieira L. F. Teaching invasive team sports in the school environment: From theory to practice from the perspective of a hybrid model. Journal of Physical Education, 2016; 27, e2709.
  27. 27. Threatt A S, Simonton K L, Layne T E. Motivating students through content and curriculum models: Bu-ilding a Sport Education Tactical Games Model (SETGM) unit on Spikeball. Journal of Physical Educa-ti-on, Recreation & Dance. 2022; 93(4):31–8.
  28. 28. Alexander K, Penney D. Teaching under the influence: Feeding Games for Understanding into the Sport Education development-refinement cycle. Physical Education and Sport Pedagogy. 2005; 10(3):287–301.
  29. 29. Evangelio C, Fernández-Rio J, Peiró-Velert C, González-Víllora S. Sport education, cooperative learnin-g, and health-based physical education: Another step in pedagogical models’ hybridization. Journal of P-hysical Education, Recreation & Dance. 2021; 92(6):24–32.
  30. 30. Griffin J I, Butler L L, Sheppard M A. Athlete-centred coaching: Extending the possibilities of a holistic and process-oriented model to athlete development. Routledge; 2018.
  31. 31. Casey A, Dyson B. The implementation of models-based practice in physical education through action research. European Physical Education Review. 2010; 15(2):175–99.
  32. 32. Fernandez-Rio J, Menendez-Santurio J I. Teachers and students’ perceptions of a hybrid sport education and teaching for personal and social responsibility learning unit. Journal of Teaching in Physical Educat-ion. 2017; 36(2):185–96.
  33. 33. Dyson B, Griffin L L, Hastie P. Sport Education, Tactical Games, and Cooperative Learning: Theoretica-l and Pedagogical Considerations. Quest. 2004; 56(2):226–40.
  34. 34. Menendez J I, Fernandez-Rio J. Hybridising Sport Education and Teaching for Personal and Social Res-ponsibility to include students with disabilities. European Journal of Special Needs Education. 2017; 32(4):508–24.
  35. 35. González-Víllora S, Evangelio C, Sierra-Díaz J, Fernández-Río J. Hybridizing pedagogical models: A s-ystematic review. European Physical Education Review. 2019; 25(4):1056–74.
  36. 36. Page M J, McKenzie J E, Bossuyt P M, Boutron I, Hoffmann T C, Mulrow C D, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 st-atement. Journal of Clinical Epidemiology. 2021; 134:103–12. pmid:33577987
  37. 37. Araújo R, Mesquita I, Hastie P, Pereira C. Students’ game performance improvements during a hybrid sport education—step-game-approach volleyball unit. European Physical Education Review. 2016; 22(2):185–200.
  38. 38. Araújo R M F, Hastie P A, Pereira C H de A B, Mesquita I M R. The evolution of student-coach’s peda-gogical content knowledge in a combined use of sport education and the step-game-approach model. Ph-ysical Education and Sport Pedagogy. 2017; 22(5):518–35.
  39. 39. Araújo R, Hastie P, Lohse K R, Bessa C, Mesquita I. The long-term development of volleyball game pl-ay performance using Sport Education and the Step-Game-Approach model. European Physical Education Review. 2019; 25(2):311–26.
  40. 40. Araújo R, Delgado M, Azevedo E, Mesquita I. Students’ tactical understanding during a hybrid sport ed-ucation/step-game approach model volleyball teaching unit. Movimento. 2020; 26:1–16.
  41. 41. Arikan N. Effect of Sport Education Model-Based Social-Emotional Learning Program on Emotional In-telligence. International Education Studies. 2020; 13(4):41.
  42. 42. Gómez Buendía Á, Sánchez-Alcaraz Martínez B J, Cifo Izquierdo M I, Gómez Mármol A. Effects of a hybrid teaching model (SEM + TGfU) and the model of personal and social responsibility on sportsmanship and enjoyment in 4° Secondary and 1° Baccalaureate students (Efectos de un modelo híbrido de enseña-nza (MED + TGfU) y el modelo de respon). Retos. 2021; 43:550–59.
  43. 43. Farias C F, Mesquita I R, Hastie P A. Game Performance and Understanding Within a Hybrid Sport Ed-ucation Season. Journal of Teaching in Physical Education. 2015; 34(3):363–83.
  44. 44. Farias C, Mesquita I, Hastie P A. Student game-play performance in invasion games following three co-nsecutive hybrid Sport Education seasons. European Physical Education Review. 2019; 25(3):691–712.
  45. 45. Farias C, Teixeira J, Ribeiro E, Mesquita I. Effects of a two-stage physical education teacher education programme on preservice teachers’ specialised content knowledge and students’ game-play in a student-centred Sport Education-Step Game approach. European Physical Education Review. 2022; 28(4):816–34.
  46. 46. Gil-Arias A, Harvey S, Cárceles A, Práxedes A, Del Villar F. Impact of a hybrid TGfU-Sport Education unit on student motivation in physical education. PLoS One. 2017; 12(6):1–17. pmid:28658267
  47. 47. Gil-Arias A, Claver F, Práxedes A, Del Villar F, Harvey S. Autonomy support, motivational climate, en-joyment and perceived competence in physical education: Impact of a hybrid teaching games for understa-nding/sport education unit. European Physical Education Review. 2020; 26(1):36–53.
  48. 48. Gil-Arias A, Diloy-Peña S, Sevil-Serrano J, García-González L, Abós Á. A hybrid TGfU/SE volleyball teaching unit for enhancing motivation in physical education: A mixed-method approach. International Journal of Environmental Research and Public Health. 2020; 18(1):1–20. pmid:33375232
  49. 49. Gil-Arias A, Harvey S, García-Herreros F, González-Víllora S, Práxedes A, Moreno A. Effect of a hybr-id teaching games for understanding/sport education unit on elementary students’ self-determined motivation in physical education. European Physical Education Review. 2021; 27(2):366–83.
  50. 50. Gouveia É R, Lizandra J, Martinho D V, França C, Ihle A, Sarmento H, et al. The Impact of Different Pedagogical Models on Moderate-to-Vigorous Physical Activity in Physical Education Classes. Children. 2022; 9(12):1790. pmid:36553234
  51. 51. Guijarro E., MacPhail A., Arias-Palencia N.M., & González-Víllora S. (2021). Exploring Game Perf-ormance and Game Involvement: Effects of a Sport Education Season and a Combined Sport Education—Teaching Games for Understanding Unit. Journal of Teaching in Physical Education, 1–14.
  52. 52. Hastie, P.A., & Buchanan, A.M. (2000). Teaching Responsibility Through Sport Education: Prospects of a Coalition.
  53. 53. Hastie P.A., & Curtner-Smith M.D. (2006). Influence of a Hybrid Sport Education—Teaching Games for Understanding Unit on One Teacher and His Students. Physical Education and Sport Pedagogy, 11(1), 1–27.
  54. 54. Jia Z.-R.R. (2021). Effects of Teaching Games for Understanding Integrated Sport Education Model on College Students’ Football Cognitive Performance and Motor Skills. Revista de Cercetare și Intervenție Socială, 72, 274–287.
  55. 55. López-Lemus I., Del Villar F., Rodríguez-Gutiérrez A., González-Silva J., & Moreno A. (2023). Cou-ld the Hybridization of the SE/TGfU Pedagogical Models Be an Alternative for Learning Sports and P-romoting Health? School Context Study. Children, 10(877). pmid:37238425
  56. 56. Mesquita I., Farias C., & Hastie P. (2012). The Impact of a Hybrid Sport Education—Invasion Games Competence Model Soccer Unit on Students’ Decision Making, Skill Execution and Overall Game Perform-ance. European Physical Education Review, 18(2), 205–219.
  57. 57. Oliveros M., & Fernandez-Rio J. (2022). Pedagogical Models: Can They Make a Difference to Girls’ In-Class Physical Activity? Health Education Journal, 81(8), 913–925.
  58. 58. Pan Y.-H., Huang C.-H., & Hsu W.-T. (2023). A Comparison of the Learning Effects Between TGfU-SE and TGfU on Learning Motivation, Sport Enjoyment, Responsibility, and Game Performance in Ph-ysical Education. Frontiers in Psychology, 14. pmid:37519367
  59. 59. Quiñonero-Martínez A.L., Cifo-Izquierdo M.I., Sánchez-Alcaraz Martínez B.J., & Gómez-Mármol A. (2023). Effect of the Hybridization of Social and Personal Responsibility Model and Sport Education Model on Physical Fitness Status and Physical Activity Practice. Frontiers in Psychology, 14, 1–12. pmid:37915524
  60. 60. Da Silva R.M.R.R., Farias C.F.G.G., Ramos A.G.A.A., & Mesquita I.M.R.R. (2022). A Novice Teac-her as Facilitator of Learning During a Hybrid Sport Education/Step-Game Approach Volleyball Seaso-n. Journal of Sports Science and Medicine, 21, 153–163. pmid:35719230
  61. 61. Stran M., Sinelnikov O., & Woodruff E. (2012). Pre-Service Teachers’ Experiences Implementing a Hybrid Curriculum: Sport Education and Teaching Games for Understanding. European Physical Educ-at-ion Review, 18(3), 287–308.
  62. 62. Wei C., Su R., & Hsu M. (2020). Effects of TPSR Integrated Sport Education Model on Football Less-on Students’ Responsibility and Exercise Self-Efficacy. Revista de Cercetare și Intervenție Socială, 71, 126–136.
  63. 63. Luoma J., & Martela F. (2021). A Dual-Processing View of Three Cognitive Strategies in Strategic De-cision Making: Intuition, Analytic Reasoning, and Reframing. Long Range Planning, 54(102065).
  64. 64. Von Elm E., Altman D.G., Egger M., Pocock S.J., Gøtzsche P.C., & Vandenbroucke J.P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. Bulletin of the World Health Organization, 85(11), 867–872. pmid:18038077
  65. 65. Guijarro E., MacPhail A., González-Víllora S., & Arias-Palencia N.M. (2021). Relationship Between Personal and Social Responsibility and the Roles Undertaken in Sport Education. Journal of Teaching in Physical Education, 40(1), 76–85.
  66. 66. Pozo P., Grao-Cruces A., & Pérez-Ordás R. (2018). Teaching Personal and Social Responsibility Mod-el-Based Programmes in Physical Education. European Physical Education Review, 24(1), 56–75.
  67. 67. Farias C., Segovia Y., Valério C., & Mesquita I. (2022). Does Sport Education Promote Equitable Ga-me-Play Participation? Effects of Learning Context and Students’ Sex and Skill-Level. European Phys-ical Education Review, 28(1), 20–39.
  68. 68. Hastie P.A., De Ojeda D.M., & Calderón A. (2011). A Review of Research on Sport Education: 2004 to the Present. Physical Education and Sport Pedagogy, 16(2), 103–132.
  69. 69. Goodyear V.A., Casey A., & Kirk D. (2014). Tweet Me, Message Me, Like Me: Using Social Media to Facilitate Pedagogical Change Within an Emerging Community of Practice. Sport, Education and Society, 19(7), 927–943.
  70. 70. Dyson B., Shen Y., Xiong W., & Dang L. (2022). How Cooperative Learning is Conceptualized and I-mplemented in Chinese Physical Education: A Systematic Review of Literature. ECNU Review of Education, 5(2), 185–206.
  71. 71. Haerens L., Cardon G., De Bourdeaudhuij I., & Kirk D. (2011). Toward the Development of a Pedag-ogical Model for Health-Based Physical Education. Quest, 63(3), 321–338.