Development and validation of interprofessional learning assessment tool for health professionals in continuing professional development (CPD)

Introduction Interprofessional learning (IPL) is increasingly recognized as a promising strategy in addressing the burgeoning challenges in healthcare. Its assessment remains to be perplexing and requires accurate measurements. Thus, this study intended to develop a valid and reliable reflective tool in assessing IPL as a Continuing Professional Development (CPD) outcome. Methods A one-group post-posttest pre-experimental design with tool development was employed to establish the validity and reliability of the “Inventory of Reflective Vignette–Interprofessional Learning” (IRV-IPL). This tool was developed from an extensive literature review and designed with three segments to assess interprofessional competencies before, after, and what if scenarios using vignettes. After it was validated by education experts (n = 5) and written consent forms were signed by the participants, the IRV-IPL was pilot tested among healthcare professionals (n = 10) for analysis and improvement. During the actual implementation, it was administered to healthcare professionals (n = 45) who participated in a university-provided CPD event. Collected data underwent validity and reliability testing. Results IRV-IPL generated excellent internal consistency (α = 0.98), and across all segments of collaboration (α = 0.96), coordination (α = 0.96), cooperation (α = 0.96), communication (α = 0.97), and commendation (α = 0.98). Items exhibited significantly positive large correlations (r > 0.35, p < 0.05) in all segments showing beneficial measures for postdictive validity in recalling prior interprofessional competencies, and predictive validity in estimating interprofessional learning as an outcome of CPD and alternative interventions. Conclusion This study provided a piece of groundwork evidence on the use of IRV-IPL as a reflective assessment tool for interprofessional learning in CPD contexts. Further studies are essential to explore the educational utility of IRV framework in crafting relevant assessments and to establish construct validity of IRV-IPL using exploratory and confirmatory factor analyses.


Introduction
Interprofessional learning (IPL) is increasingly recognized as a promising strategy in addressing the burgeoning challenges in healthcare.Its assessment remains to be perplexing and requires accurate measurements.Thus, this study intended to develop a valid and reliable reflective tool in assessing IPL as a Continuing Professional Development (CPD) outcome.

Methods
A one-group post-posttest pre-experimental design with tool development was employed to establish the validity and reliability of the "Inventory of Reflective Vignette-Interprofessional Learning" (IRV-IPL).This tool was developed from an extensive literature review and designed with three segments to assess interprofessional competencies before, after, and what if scenarios using vignettes.After it was validated by education experts (n = 5) and written consent forms were signed by the participants, the IRV-IPL was pilot tested among healthcare professionals (n = 10) for analysis and improvement.During the actual implementation, it was administered to healthcare professionals (n = 45) who participated in a university-provided CPD event.Collected data underwent validity and reliability testing.
As the interplay of collaboration, interprofessional competencies and learning become more valued, assessment in these areas are increasingly important [30].Numerous studies display a variety of interprofessional assessments using different approaches and measures [33,34] For instance, Morison & Stewart [33] developed specific performance and program assessments.In both tools, the behavioral indicators include professional knowledge, performance, communication, and teamwork.Meanwhile, the readiness for interprofessional learning scale (RIPLS) consists of teamwork, identity, and roles constructs [35,36].Curran & colleagues [37], for their part, developed an interprofessional collaborator assessment rubric (ICAR) with criteria on communication, collaboration, roles and responsibilities, collaborative client-centered approach, team functioning, and conflict resolution.Recently, Hayward and others [38] have revised this rubric, yet they still have the same categories.Similarly, the assessment of interprofessional team collaboration (AITCS) comprised of partnership, cooperation, and coordination subscales [39].In assessing global curricular outcomes, Arif et al. [40] devised a survey tool with patient care and professionalism domains.For its part, the Agency for Health Research and Quality [41] released the team strategies and tools to enhance performance and patient safety (TeamSTEPPS), and developed tools for assessing teamwork and communication such as the Teamwork Perceptions Questionnaire (T-TPQ), Teamwork Attitudes Questionnaire (T-TAQ), Team Assessment Questionnaire (TAQ) and Team Performance Observation Tools (TPOT).These instruments have focused on team structure, leadership, situation monitoring, mutual support and communication domains [3,42,43].Aside from T-TAQ, Brock et al. [3] utilized their AMUSE model in assessing interprofessional competence in terms of attitude, motivation, utility, and self-efficacy.Likewise, other interprofessional assessment combinations [44,45] may include attitudes toward health care teams scale (ATHCTS) with care quality and physician centrality subscales [46], interprofessional education perception scale (IEPS) with competency, cooperation needs, and actual cooperation subscales [47], and interprofessional collaboration (IPC) scale [48] with communication, accommodation and isolation factors.Whereas, other scholars developed their assessments for their own programs in general [2,[49][50][51][52][53][54][55][56].
Interprofessional learning is seen beneficial in promoting quality, sustainable and safe healthcare practice [44].Parsimoniously, many existing interprofessional assessments are mainly focused on measuring attitude and perceptions [57] that makes an assessment of interprofessional learning challenging [5,58].Currently, there are limited attempts to reliably measure the impact of interprofessional learning constructs in healthcare education [59], and the demand for an assessment tool is becoming increasingly important [17,60].Consequently, there is a lack of a reflective tool to measure the indirect effects of interprofessional interactions in many CPD, which in this paper refers to any lifelong learning activities among health professionals.Thus, this study aimed to fill this current gap through the development and psychometric assessment of a reflective tool that can be used for CPD involving IPL.

Research design
A one-group posttest only (also one-shot case study or one-shot experiment) with tool development design suits the existing contextual circumstances of the study.In this pre-experimental subdesign, the effect on the outcome is measured after a single group of participants has received a predetermined intervention [61,62].This study specifically adapted the Ruzafa-Martinez et al. [63] tool development procedures to ensure the integrity of the collected data, which comprises of five (5) steps, namely: (a) content design mediated by review methods (e.g.literature review and existing tool synthesis), (b) content validation by field experts, (c) pilot testing with health professionals, (d) preliminary analysis for initial reliability and validity, and (e) actual implementation with final validation and analysis.
Content design.The content design of the Inventory of Reflective Vignette (IRV) as a framework for tool development considers the strategy of MacDonald, Stodel, Thompson, & Casimiro [64] and Bottenberg et al. [44] in embedding the pretest items into the posttest survey to minimize possible bias.MacDonald & colleagues [65] referred to this as the "post-posttest design," which enables improvement in the tool sensitivity as well as reflection on prior and current conditions.To achieve these goals, the tool incorporates the use of research vignette.At large, a vignette question presents a carefully designed situation to a respondent [62].Several studies [66,67] have combined vignette and questionnaire in measuring the variables and validating research instrument.In this case, the vignette on conventional CPD delivery (i.e.didactics or lectures) creates an opportunity to surface the construed judgments of the participants in connection with their previous (or vicarious) experiences.
The assessment of Interprofessional Learning (IRV-IPL) was constructed using the designed IRV framework.This innovative instrument is divided into two columns, one for the assessment items and another for the rating responses.A 6-point Likert-type scale (i.e., 1 = Emerging; 2 = Developing; 3 = Minimal; 4 = Proficient; 5 = Advanced; 6 = Excellent) is devised to allow deeper reflection, yet eliminate a neutral value for clearer measurement.
Despite the advantage of having a midpoint as a respite for sensitive topics, this study involves mature health professionals, who are presumably more thoughtful and critical.The latter column is further divided into three segments for (a) before CPD event, (b) after CPD participation, and (c) a vignette (i.e. if participated in a traditional lecture format).Additionally, the instruments include a section for the respondent information.
Content validation.During the content validation, the study identified five IPL constructs (Fig 1) using synthesized evidences based on relevant literature review.There are five essential constructs for interprofessional learning in spite of the program purpose, namely: (a) collaboration, (b) coordination, (c) cooperation, (d) communication and (e) commendation.To begin with, collaboration centers on purposeful creation of a certain outcome.This coincides on the view that collaboration focuses on working relationships with others [37] in achieving a common goal [68].Secondly, coordination seeks to inform other units in ensuring harmony leading towards a single direction.This explicitly emphasizes awareness of the action, but not so much on the results.As for cooperation, it highlights making contributions in a team [39].Although this allows sharing thoughts and working together, it also fosters divergent thinking.Now, communication respectfully expresses information with others for understanding.This may include verbal and nonverbal strategies, as well as transmission and acquisition activities [33,37,48].Lastly, commendation is conceptually described as the appreciation of others' competencies, accomplishments, performances, professions, roles, and identities.These attributes may offer responsive if not pervasive (i.e.direct effect on health outcomes) measures of interprofessional learning.The constructs of the IRV-IPL tool were reviewed independently by experts (n = 5) for relevance, applicability, and improvement.
Pilot testing and preliminary analysis.The designed tool was pilot tested on October 2017 in a university offering a CPD program for health professionals (n = 10).The institution was accredited by the National Professional Regulation Commission in delivering CPD programs.

Data collection on the actual implementation
During the actual implementation on November 2017, the IRV-IPL was administered to health professionals (n = 45), who completed the university-provided CPD programs with constructivist teaching methods, which allow participants to learn interactively and to work primarily in groups.CPD units were given to the participants after successful completion.The participants in the pilot and actual testing were both registered and practicing physicians, nurses, and allied professionals like pharmacists, technicians, and therapists.

Data analysis
The collected data was processed using IBM SPSS Statistics version 22 to estimate its internal reliability using Cronbach's alpha and validity coefficient using item-total correlation.

Ethical approval
Our Lady of Fatima University Institutional Ethics Review Committee-Level 2 Accredited Research Ethics Committee of the Philippine Health Research Ethics Board approved this study.All participants (pilot and actual) signed a written informed consent after careful orientation on the study objectives and plan.

Results
The study analyzed the utilization of the IRV-IPL in assessing interprofessional learning among health professionals during the pilot (n = 10) and actual (n = 45) implementations.
The pilot study also tested IRV-IPL for statistical validity (Table 2) by correlating each item with the total score (i.e. the sum of all segments).Given that the tool underwent content and face validation, it can serve as a valid measure of interprofessional learning.The validity coefficients showed the inconsistency of estimates resulting in items with insignificant correlations (p > 0.05) with exception to the construct of commendation.Based on the initial validation and focus group discussion, most items were reviewed and reworded to improve further its measures.

Actual implementation: Descriptive test responses to IRV-IPL
As shown (Table 3), the highest mean responses of the participants on the IPL items are consistently observed after the CPD program.All the ratings fall within the range of 4.5-5.49or Proficient.
Ratings on the instrument "if lecture" was used as an intervention are consistently higher compared to the ratings before the CPD program.Most of the items in "if lecture was used" fall within the range of 3.5-4.49or Advanced.Two items namely (1) Explain the roles/tasks of each team member and (2) Negotiate tasks/responsibilities with other participants which fall within the rage of 2.5-3.49 of Minimal.Evaluations before CPD fall within two ranges 2.5-3.49(Minimal) and 3.5-4.49(Advanced).Ratings on the Coordination dimension fall within the range of 2.5-3.49Minimal while the ratings on Communication dimension fall within 3.5-4.49or Advanced.Majority of the items under Commendation fall within 3.5-4.49or Advanced.The only item "Give constructive feedbacks to other participants" fall within 2.5-3.49or Minimal.Majority of the items under Collaboration fall within 2.5-3.49or Minimal.Only item "Use team an approach to achieve the goals/outcomes" fall within 3.5-4.49or Advanced.

Actual implementation: Reliability and validity
The designed tool demonstrated excellent internal consistency (α = 0.98) across all segments (Table 4).As compared to the pilot test, there was also notable increased reliability estimates among the identified constructs, namely: collaboration (α = 0.96), coordination (α = 0.96), cooperation (α = 0.96), communication (α = 0.97), and commendation (α = 0.98).This suggests that IRV-IPL was able to measure consistently the identified constructs, which in turn could provide a reliable measure of IPL.Further analysis (Table 5) shows significant positive limited relationship (r < 0.70, p < 0.05) across all IPL constructs when (a) before and after, and (b) before and if were correlated.Moreover, this is specifically present when correlating after and if segments in the construct of coordination.This perhaps may imply the distinction of each segment.It means that IRV-IPL can also provide reliable and reflective assessments of (a) baseline interprofessional competencies, (b) IPL as outcomes of an intervention, and (c) comparative IPL measure for alternative situations.
The actual validity coefficients (Table 6) of the corresponding items for all constructs exhibited significantly positive large correlations (r > 0.35, p < 0.05) in all segments except for one

IRV-IPL as a reliable assessment tool
The IRV-IPL exhibited good psychometric properties as evidenced by a remarkably high internal consistency coefficient score.This tool possesses inherent characteristics of a reliable instrument-suitability of length, validity of content, practicality of administration, inclusivity of user, usability of tool, and clarity of structure.
The IRV-IPL's accuracy to measure interprofessional learning is beneficial in generating consistently reproducible results under related conditions and subjects [69,70] on several items under the areas of collaboration, coordination, cooperation, communication, and commendation.Measurement of these constructs is significant for it measures collaboration attribute as a central concept of interprofessional learning.For instance, a healthcare professional with the inability to work together with other members of the healthcare team may translate to poor team learning outcomes and eventually to poor healthcare delivery in practice [71,72].In evidence, most of the tools developed to assess interprofessional learning focuses on the quality of interpersonal interactions and behavior within teams [59,73].As an outcome of positive interactions, healthcare workers who are members of interprofessional groups are expected to function cohesively [74] under shared leadership, decision-making and co-shared accountability [75].

IRV-IPL as a valid assessment tool
IRV-IPL is a valid measurement of interprofessional learning through its constructs that were carefully crafted using global evidences found in previous studies and enhanced through effective negotiations between tool designers and users.It is composed of harmonized relevant key elements of interprofessional learning broken down into core values (i.e.Collaboration, Coordination, Cooperation, Communication, and Commendation).The tool clearly distinguishes one construct to another through concrete definitions.This is substantial since interprofessional learning involves a variety of attributes of interprofessionalism [76] beyond generic and profession-specific competencies [77].Previous studies (e.g.[78]) underscored the need to identify individual features which is essential in identifying areas of strength and weakness in learning.
The validity of the IRV-IPL as a tool is useful to educators who intend to come up with an assessment backed by evidence and theoretical supports [79] to interpret interprofessional learning regardless of the type of healthcare provider test takers [80,81].By and large, learning measurement under interprofessional scope is incommensurable even under same learning outcomes for different purposes.Interprofessional learning as a form of educational exercise was acknowledged to occur in divergent environments due to differing professional standards, social structures, individual responses, learner behaviors, and learning resources [82].Education planners may benefit from valid assessment tools as sources of evidence-based data for the curriculum and instructional development and improvement from profession-specific to interprofession-sensitive learning.Interestingly, IRV-IPL focuses on positive affinity toward IPL in contrast with other tools that include both exemplar and non-exemplar performances.This may lead to an acceptable conclusion based on sound reasoning, evidence, and justification, and further enhance tool sensitivity [83].

IRV-IPL as a reflective assessment tool
IRV-IPL is a tool that promotes reflection through critical introspection.It involves the use of an odd-response scale that removes neutrality, promotes thoughtfulness, compels looking back, and minimizes midpoint bias.Following a post-posttest design that integrates pretest and posttest in a single administration preventing tool desensitization, it can be administered to economically obtain viable comparisons between self-assessments in interprofessional learning and alternative environments.
When juxtaposed to previous tools, an apparent highlight of the IRV-IPL is the advantage of evaluating educational impact through reflective assessments and parallel evaluation of current and previous understandings through the innovative use of anchoring vignettes [84,85].This approach fosters deeper contemplation of past experiences in foreseeing possible outcomes given a different context.In this study, anchoring vignettes pertain to the hypothetical case [86] of the respondents in a lecture scenario which can be rooted from their previous experiences.Vignettes that underscore "historical moments" [87] are proven effective in comparing insights in various life courses occurring in different levels [88] and are positively correlated with actual outcomes [89].

Conclusion
The study offered a valid and reliable assessment tool for interprofessional learning (IPL) utilizing the Inventory of Reflective Vignette (IRV) as a framework for tool development.Based on the study findings, IRV-IPL can measure IPL consistently due to its excellent internal reliability and capture variances among segments using post-posttest strategy.It also demonstrated face and content validity supported by significant beneficial item-total correlations.Although the study was limited by the number and demographics of samples, it was able to provide empirical evidences to substantiate IRV-IPL as a reflective tool.Health professions leaders and educators are encouraged to use this tool in assessing IPL so as to identify appropriate policies, strategies, and interventions.Further studies are needed to: (a) explore the educational utility of IRV framework in designing relevant assessments, (b) investigate demographic influence on IRV-IPL by finding its determinants and correlates, (c) strengthen statistical evidences for IRV-IPL's criterion validity by identifying appropriate criteria, and (d) establish construct validity of IRV-IPL using exploratory and confirmatory factor analyses.It is hoped that the IRV-IPL can provide a better IPL estimate to promote interprofessionalism in achieving better healthcare outcomes through collaboration.

Table 5 . Relation matrix of IPL constructs as to segments.
Such a remarkable result can be attributed to the effective strategy used in improving tool items during pilot testing.This clearly indicates the established validity of IRV-IPL as an assessment tool.In addition, the instrument was able to measure interprofessional learning at different segments.It also shows that IRV-IPL demonstrated beneficial measures for postdictive validity in recalling prior interprofessional competencies, and predictive validity in estimating IPL as an outcome of CPD and alternative interventions.