Using a Delphi Technique to Seek Consensus Regarding Definitions, Descriptions and Classification of Terms Related to Implicit and Explicit Forms of Motor Learning

Background Motor learning is central to domains such as sports and rehabilitation; however, often terminologies are insufficiently uniform to allow effective sharing of experience or translation of knowledge. A study using a Delphi technique was conducted to ascertain level of agreement between experts from different motor learning domains (i.e., therapists, coaches, researchers) with respect to definitions and descriptions of a fundamental conceptual distinction within motor learning, namely implicit and explicit motor learning. Methods A Delphi technique was embedded in multiple rounds of a survey designed to collect and aggregate informed opinions of 49 international respondents with expertise related to motor learning. The survey was administered via an online survey program and accompanied by feedback after each round. Consensus was considered to be reached if ≥70% of the experts agreed on a topic. Results Consensus was reached with respect to definitions of implicit and explicit motor learning, and seven common primary intervention strategies were identified in the context of implicit and explicit motor learning. Consensus was not reached with respect to whether the strategies promote implicit or explicit forms of learning. Discussion The definitions and descriptions agreed upon may aid translation and transfer of knowledge between domains in the field of motor learning. Empirical and clinical research is required to confirm the accuracy of the definitions and to explore the feasibility of the strategies that were identified in research, everyday practice and education.


Background
Motor learning is a central issue in sports, but has recently received increased attention in the context of rehabilitation [1][2][3][4][5][6]. A departure point for explanatory models of motor learning, in a variety of healthy and rehabilitation target groups, is the broad distinction between conscious and non-conscious attributes of the motor learning process. The distinction is often delimited by an implicit-explicit conceptualisation first made popular in cognitive psychology [7], which proposes that implicit motor learning targets more non-conscious attributes of the motor learning process, whereas explicit motor learning targets more conscious attributes of the motor learning process [8,9].
Although investigation of implicit and explicit forms of motor learning has become common-place in recent years, particularly in the sport-related motor literature [9,10], for the most part, terminology has been insufficiently uniform. This has raised a barrier that hampers exchange of knowledge within and between fundamental domains of research and practical target groups.
For example, in describing or defining implicit and explicit (motor) learning, independent research groups have focused on the type of knowledge accrued during the learning process, e.g. [8], the amount of attention or awareness needed to learn, e.g. [11][12][13], or the way the results of the learning process are measured, e.g. [11,14]. The terms implicit and explicit are sometimes also used to refer to the underlying memory systems that are involved, e.g. [15], as synonyms associated with declarative and procedural knowledge, e.g. [12] or to describe the actual application of learning in practice, e.g. [16]. However, there is some overlap in underlying conceptualisations of implicit and explicit motor learning. For example, in many definitions and descriptions explicit motor learning is associated with conscious cognitive processes, whereas implicit learning is associated with nonconscious cognitive processes, e.g. [8,17]. The lack of agreement sometimes results in different, or even conflicting, applications of implicit and explicit learning in study paradigms, clinical practice and education.
Intervention strategies, such as discovery learning, analogy learning and errorless learning, have been used to shape the motor learning process in clinical or non-clinical studies within different target groups [18][19][20]. In general, intervention strategies that lead to high conscious awareness of how the motor behavior is accomplished are applied to promote explicit motor learning, whereas intervention strategies resulting in low conscious awareness of how the motor behavior is accomplished are applied to promote implicit motor learning [8,9].The theoretical underpinning for this implicit/explicit distinction proposes that motor learning is a process in which solutions to the motor problem are discovered either explicitly through a process of hypothesis testing made possible by the human ability to temporarily manipulate and store information consciously in working memory, or implicitly through a process of discovery that does not rely on conscious manipulation and storage of information by working memory [8,9,21]. For example, reducing the amount of errors made during the motor learning process (errorless or error-reduced learning) is thought to moderate the need for hypothesis testing about possible motor solutions, which minimizes working memory involvement in movement and promotes implicit motor learning [19,22]. Further, it has been argued that learning a motor task while performing a concurrent cognitive task (dual task learning) prevents working memory from temporarily storing conscious information related to motor solutions because working memory must engage in completing the cognitive task. Thus, the motor behaviour is learned more implicitly than if a cognitive secondary task was not performed concurrently [8]. However, not all intervention strategies are used unambiguously with respect to the implicit/explicit distinction. For many motor learning strategies, it seems unclear whether they promote implicit or explicit motor learning or whether their ability to promote  either form of learning is a function of the target population or the specific learning context in which they are applied. For example, discovery learning is regarded by some researchers as likely to result in predominantly implicit learning outcomes, whereas, other researchers argue that predominantly explicit outcomes result [23]. Yet, trial and error learning, which in practice seem little different from discovery learning, has been described to promote explicit motor learning [19,24]. For therapists, coaches, researchers and teachers, uniform terminology is particularly important. Effective transfer of research results to clinical practice and education is promoted by clear terminology, and allows therapists and coaches to speak a common language among themselves (e.g., to set up treatment plans) or when instructing students [25]. The aim of this study was therefore to seek consensus regarding the definitions, descriptions and classification of terms related to the general distinction between implicit and explicit forms of motor learning.

Method
A survey consisting of a series of sequential rounds interspersed by controlled feedback [26] was performed to collect and aggregate informed judgments about motor learning from a group of experts. The survey consisted of three rounds, which were designed and distributed using an online survey programme (SurveyMonkey Inc, SurveyMonkey.com, California, USA). More detailed information about the method and rationale for the entire survey is presented elsewhere [27].
A Delphi technique was embedded into the first two rounds of the survey to seek consensus regarding definitions, descriptions and classifications related to the explicit/implicit distinction in motor learning. Although there is minimal scientific evidence available to inform decisions about the number of survey rounds appropriate for a Delphi technique, two or three rounds have typically been employed [28]. Information regarding the content and the results of the third round is not presented here as this round was not used as part of the Delphi technique.
The Central Ethics Committee Atrium-Orbis-Zuyd (Institutional Review Board) was contacted and formal written permission to perform the study described in the protocol [26] was obtained (13-N-144). The study was excluded from IRB review, because under the law, Medically Scientific Research with people (WMO), it does not submit people to actions or impose specific behaviors on them. Definition provided in second round Explicit motor learning can be defined as learning which generates verbal knowledge of movement performance (e.g., facts and rules), involves cognitive stages within the learning process and is dependent on working memory involvement % agreement 95.5%

Comments after second
NAspect of an 'internal focus' should be involved (n = 0/1) Procedure A referee group consisting of seven researchers with backgrounds in epidemiology, physiotherapy, occupational therapy, movement sciences and psychology supervised and monitored the process. The group conducted the literature search, identified experts to be approached to complete the surveys, and prepared the questions for each survey round. Between each survey round, the group performed a preliminary analysis of data blinded to the identity of the experts. In addition, two members of the referee group (MK, SB) were responsible for distributing and monitoring the survey (e.g., sending reminders and feedback reports).
A panel of international experts was invited to contribute to the study. Panel members were initially selected on the basis of literature search or the networks of the referee group. Criteria for selection of an expert were based on either scientific publication(s) in the field of motor learning (researcher) or at least three years of working experience applying motor learning in practice plus involvement in education or research (therapist, coach, lecturer). Definition provided in second round Implicit motor learning can be defined as learning which progresses with no or minimal increase in verbal knowledge of movement performance (e.g., facts and rules) and without awareness. Implicitly learned skills are (unconsciously) retrieved from implicit memory.  In a preliminary recruitment round, eligible experts were invited by mail to participate in the study. They were given a comprehensive introduction to the aims and content of the survey rounds, informed of the expected amount of time necessary to complete each survey and provided with a personal link to the online survey program. After informed consent was obtained from the experts, they were asked to provide personal information (e.g., background, years of experience, special interests). Those experts who were invited to participate were asked to recommend other

71.4%
The learner must (be able) to detect the error (n = 6); Learning is an iterative process (n = 1); Correction of errors should not be emphasised (n = 1) Learning by repeatedly attempting to perform a task during which the learner detects errors and corrects them.

Observational learning
Learning by observing a movement. The observer determines the key spatial and/or temporal features of the task through observation, thereby creating a cognitive representation of the action pattern.

69.4%
Unsure about/delete ''cognitive representation'' (n = 7); The demonstrator/ therapist can also direct the learner to the key features (n = 1) Learning by observing a movement. The observer determines the key spatial and/or temporal features of the task through observation, and/or is directed to these features by the demonstrator/ therapist.

Errorless learning
Learning facilitated by constraining the learning environment so that very few errors occur.

67.3%
Learning environment and the instructions and skill difficulty can be constrained as part of the learning environment (n = 3); Should be applied particularly in early phase of learning (n = 1); Replace ''very few errors'' with ''no errors'' (n = 1) Learning facilitated by constraining the learning environment (e.g., instructions, skill difficulty) so that very few errors occur.

Movement Imagery
Learning by imagining oneself undertaking the skilled movement without actually doing the movement.

71.4%
Imagery should be from the first person perspective (n = 2); Exchange ''undertaking'' with ''performing'' (n = 1); Suggestions for terming the strategy (mental rehearsal, motor imagery) (n = 2) Learning by imagining oneself performing the skilled movement (in the first or third person perspective) without actually physically performing the movement.

Discovery learning
Learning without guidance, instructions or feedback from another person.

57.1%
Without information from other sources (book, website) (n = 2); It is necessary to give instructions or feedback (n = 2); Learning is facilitated by (constrained) context (n = 3); Use (pure) discovery learning as a synonym for Trial and error (use this description) (n = 1) Learning without guidance or feedback from another person or information source.

Dual-task learning
Learning of a skill during simultaneous performance of another skill. The secondary task can be a motor or cognitive task.

61.2%
The (second) task must be of equal importance/difficulty and attention demanding (n = 5); Doubts about whether dual task is a form of learning (n = 3) + Learning of a skill while simultaneously performing another task. The second task can be a motor or cognitive task but must be attention demanding 81.8% agreed; 9.1% preferred description from round one

Analogy learning
Learning facilitated by metaphors. The complex structure of the to-be-learned skill is integrated in a simple biomechanical metaphor that the learner is provided with 51.0% Did not agree with term 'biomechanical' (n = 6) Learning facilitated by metaphors. The complex structure of the to-be-learned skill is integrated into a simple metaphor that the learner is provided with. experts in the field (so-called snowball sampling method), who were subsequently contacted in the same manner. No clear guidelines regarding the optimal panel size for a Delphi study exist [29]. Consistent with another study using a Delphi technique [30], a minimum panel size of 30 experts was targeted, comprising approximately ten motor learning researchers in rehabilitation, ten in healthy individuals and sports and ten with experience applying motor learning in daily practice.
Content of the survey rounds and analysis. Within the survey rounds, we distinguished between definitions and descriptions. The term ''definition'' was used when referring to forms of learning (e.g., implicit, explicit), whereas the term ''description'' was used when referring to motor learning strategies (e.g., errorless learning, trial and error learning). We made this distinction because the term 'definition' implies theoretical attributes/features of learning, while the term 'description' implies elements of how a strategy is applied.
Additionally, implicit and explicit motor learning have been described as representing a dichotomy in learning and also as representing tail ends of a learning continuum, so for the categorisation of intervention strategies we used answering categories that left room for both perspectives. Table 1 presents the content of the survey rounds. Each survey round was divided into two parts. The first part of Round 1 focussed on creating a basic definition of implicit and explicit motor learning. The second part focussed on identifying, describing and classifying learning strategies. Questions in part one of Round 2 and 3 were used to verify responses in the first round and second round respectively and to elaborate issues identified by the expert panel. Questions in part two of Round 2 and 3 addressed other predefined topics (results are not presented in this article).
In preparation of the Delphi, the referee group performed a literature search in different fields of motor learning (sports, rehabilitation, fundamental research) using both scientific research articles and grey literature. The group identified several search terms, implicit, explicit, motor learning, skill acquisition, and used MeSH terms when possible. From these resources, e.g. [1,8,11,17,[31][32][33][34][35], statements which were related to implicit and explicit motor learning were extracted and compared. The referee group tried to improve readability of the statements by using comparable formulation.
Analysis of the data was conducted blind to the names and characteristics of the expert respondents. Open comments and additions made by the experts were clustered in themes and carefully considered by the referee group. Consistent with other studies, consensus was considered to have been reached when $ 70% of the experts agreed on a certain topic [30,36,37]. Differences in values and beliefs within the different professions represented by the experts might have influenced the results, so in cases where $70% of the experts agreed, the referee group checked for a profession-based imbalance in the responses, which was not the case.
If consensus was achieved, final definitions and descriptions were formulated. If no consensus was achieved, the topic and answers were presented to the expert panel in the following round.
After each round, panel members received a feedback report that summarised the response percentages for each question, as well as responses to open questions and additional comments. In these feedback reports, the results were clustered but not analysed or interpreted.

Results
The recruitment process is shown in Figure 1. In total, 155 experts were invited to participate. Thirty-nine experts agreed to participate initially and recommended a further 49 experts. Fortynine experts completed the Round 1 survey and 44 completed the Round 2 survey. Characteristics of these experts are shown in Table 2. Experts were heterogeneous with regard to age, background and current working situation. Although the expert panel was internationally diverse, most were based in Europe. Of the 11 experts who did not respond to invitations or reminders, only two reported lack of time as the reason for non-response.

Definitions of implicit and explicit motor learning
The results with regard to the definitions of explicit and implicit motor learning are shown in Tables 3 and 4. Of the experts, 95.5% agreed in general with the following definition of explicit motor learning: learning which generates verbal knowledge of movement performance (e.g., facts and rules), involves cognitive stages within the learning process and is dependent on working memory involvement.
Six experts (12.2%) proposed additional attributes of the definition of explicit learning that were not provided initially (category ''other''). The additions included practical attributes (e.g., ''using an internal focus of attention'', ''using feedback'') and attributes related to the result of explicit learning (e.g., ''responsible for rapid early improvement''), but they were not incorporated into the definition as each was only suggested by one or two experts.
Of the experts, 88.6% agreed in general with the following definition of implicit motor learning: learning which progresses with no or minimal increase in verbal knowledge of movement performance (e.g., facts and rules) and without awareness. Implicitly learned skills are (unconsciously) retrieved from implicit memory.
Eight experts (16.3%) proposed additional attributes of the definition of implicit motor learning, which were eventually not incorporated into the definition, despite their importance for discussion (e.g., an ''external focus of attention'' is used; the learning is ''goal orientated''; relies on ''functional practice in a meaningful environment''). One expert pointed out that implicit learning is ''not non-cognitive'' and ''not unconscious'' but rather ''non-verbal''. This expert further pointed out that implicit learning ''often does involve awareness of trying to accomplish something''.
Thirteen existing definitions or extractions from literature were preferred or deemed to be as good as the new definitions, but none were mentioned more than once. Table 5 provides an overview of 10 motor learning intervention strategies that were identified from the literature and gives the percentage of experts who indicated that they were aware of the strategies and/or used the strategies. Seven strategies were known by more than 70% of the experts and were therefore included in Round 2. After modification of the strategy descriptions, based on the comments in Round 1, consensus was achieved for six of the seven descriptions (see Table 6). Only for observational learning did percentage agreement decrease slightly after the description was reformulated (from 69.4% to 68.2%).

Classification of strategies
Responses regarding classification of whether the strategies are likely to result in (more) implicit or (more) explicit forms of motor learning were diverse (see Table 7). Table 8. Examples of the best known strategies provided by the experts.

Strategy Two random selected examples provided by experts
Trial and error learning ''Structure the learning environment so that errors will be made, but a positive outcome is achievable. Inform the learner that following the practice session they will be asked to describe the different techniques they Round 1 suggested that none of the strategies can be categorized as promoting just one form of motor learning. For the errorless, dual-task and analogy learning strategies, there was however a slight trend for the experts to consider these strategies as likely to result in a more implicit form of learning. Depending on the strategy, between 1 and 5 experts did not classify the separate strategies into one of the provided categories, but chose the option 'other' (between 4.5-16.7% of the sample).
A common argument in the open comment box was that the strategy could promote both implicit and explicit motor learning. Factors, such as, instructions, constraints in the environment, type of task/skill and the abilities of the learner were all deemed to have an influence on the outcomes of the learning strategy. According to the experts, manipulation of these factors has a profound influence over the degree to which a strategy results in implicit or explicit motor learning.

Additional strategies
Twenty-two alternative motor learning strategies were suggested, which were not included in the initial list presented to the experts (e.g., win shift lose stay, verbal overshadowing, blocked practice, applied behaviour analysis). None of these strategies were mentioned by more than one expert and were therefore not incorporated in the following surveys rounds. Other suggestions were related more generally to the focus of attention during learning, the provision of feedback, the repetition and variability of practice and manual facilitation.
As a result of the diversity in answers and additional statements made by the experts, the referee group decided not to strive to seek consensus with regard to the classification but rather to explore the reasons for diversity. This was done in the third survey round and resulted in an overview of practical experiences, opinions and verifications of statements (results not presented).

Examples of the application of the strategies in clinical practice
The number of examples of the application of different strategies in clinical practice ranged from 30 (discovery learning) to 41 (errorless learning). For each strategy, the referee group chose two examples to present in this article (Table 8).

Discussion
The aim of this study was to seek consensus on the definitions and descriptions of terms related to the conceptual distinction between implicit and explicit motor learning. Within a heterogeneous international group of experts, consensus regarding definitions of implicit and explicit motor learning, and descriptions of the best known strategies used in the context of implicit and explicit motor learning, was reached. Both definitions incorporate central aspects of motor learning (e.g., form of memory and type of knowledge). Incorporation of more than one central aspect of motor learning is preferable to outlining a single aspect, as sometimes occurs in definitions, and suggests that there was at least some degree of consensus by the experts.
Consensus suggests that experts from the different fields represented within the study may think about and describe motor learning and the underlying processes in a comparable way, at least at a more theoretical level.
According to the responses of the experts in this study analogy learning, errorless learning and dual task learning seem to promote more implicit learning in general. However, no consensus was reached within the expert panel on the classification of motor learning strategies for promoting a (more) implicit or (more) explicit form of motor learning. Based on the results of this study, it seems that most intervention strategies do not naturally promote implicit or explicit motor learning. They can promote either form of learning depending on their use in a specific learning situation and/or target population. It is probably impossible, and perhaps not even desirable, to achieve consensus. This result might be a consequence of the complexity of applying motor learning strategies in everyday practice. For example, athletes or patients usually need a tailored approach and the application of learning strategies is determined by multiple factors. Consequently, for research and education it is even more important that the application of an intervention strategy in a specific context is always described in detail.

Critical reflection on the study and the study results
To our knowledge, this is the first study that uses a Delphi technique in the field of motor learning. The results generated and summarized within the study are based on knowledge, opinions and practical experiences of an international expert panel. Consequently, the results should be interpreted tentatively; an expert group's opinion rather than empirical evidence.
Importantly, although consensus was obtained regarding the definitions, this does not mean that all of the experts, or indeed the authors (referee group), agree with the final definitions and descriptions. For instance, the final definition of implicit motor learning suggests that learning progresses ''without awareness'', but there are clearly occasions (e.g., sport, rehabilitation) when a person has intention to learn and is aware of learning, especially when outcome feedback is readily available [21,38]. It also seems unlikely that learning ever progresses with 'no' increase in verbal knowledge.
Although the Delphi technique is a well-accepted method for investigating opinions, there is currently no agreement on the meaning of consensus [28]. In our study, consensus was regarded as agreement within a selected group of leading experts on a certain topic, based on a criterion of 70% agreement or greater. However, lack of consensus (i.e., less than 70% agreement) does not directly imply that a statement was invalid, but may suggest that more plausible possibilities exist or that no alternatives exist yet. Numerous comments and statements were made by individual experts in response to the open questions. Although all were distributed in the feedback report, most were not carried back into the survey. As the aim of the study was to achieve consensus (a quantitative approach), we unfortunately were not able to take all single statements into account (a more qualitative approach). Consequently, comments/statements which other experts may have agreed upon might have been overlooked.
The quality of the findings from a Delphi study is strongly related to the heterogeneity and representativeness of the expert panel. Although the response rate in the current study was low, the experts who participated can be described as heterogeneous with regard to their backgrounds, special interest and working experience. Further, the different practical areas of motor learning are represented by the expert panel. We tried to overcome selection bias within the sample by using snow-ball sampling; nevertheless, some selection bias may have occurred, as most of the experts who participated were based in Europe. This might be explained by the fact that six of the seven referee group members, whose networks were used to identify experts, were also based in Europe. We do not know to what extent the origin and background of the experts influenced the results, so we acknowledge that cultural values may account for some of our findings (especially, lack of consensus).
Despite the limitations already discussed, we believe that it is important to use uniform terminology when describing the content of motor learning studies and practical sessions, and within education. The study is a first important step towards helping therapists, researchers and other professionals to communicate about motor learning in general and to distinguish fundamentally between implicit and explicit motor learning more specifically. The added value of the study is that the definitions and descriptions that emerged are based on the opinions of an expert panel from different fields of motor learning, which might help to promote a common language across different fields.
Future applied research is needed to confirm the findings. Underlying neurophysiological and behavioural aspects of the definitions should be investigated by fundamental research. Clinical research investigating clearly defined and described techniques is needed to investigate whether the definitions of implicit and explicit motor learning, as well as the descriptions of the strategies, are feasible and applicable within clinical practice.