Assessing university guidance and tutoring in higher education: Validating a questionnaire on Ecuadorian students

This study was intended to explore and confirm the factorial structure and to analyze the psychometric properties of an instrument for university guidance and tutoring, apply it, and detect differences between sociodemographic variables. A total of 1,048 students from five universities in the province of Manabi (Ecuador) participated. The study was divided into two phases with differentiated samples. An exploratory phase, made up of 200 subjects (19.1%), and another confirmatory phase, made up of 848 (80.9%), where the questionnaire was also applied. The results supported the three-factor structure of the instrument called “Questionnaire for the Assessment of Guidance and Tutoring in Higher Education” (Q-AGT), with of a total of 21 items. The indices of goodness of fit, reliability and internal consistency of the model were considered satisfactory. The application of the questionnaire did not show statistically significant differences in the assessment of university guidance and tutoring between men and women, with a high value given by both sexes to the importance of tutoring, the demands and the competences of the teaching staff in the university. The differences were mainly found between universities and branches of knowledge. Among the main conclusions, what stands out is the achievement of a valid and reliable instrument to measure the development of guidance and tutoring in Latin American universities. This contributes to the assessment of university guidance and tutoring as a strategy for the integral development of the student- personally, academically and professionally- and as a possible protective factor against academic dropout.


Introduction
Guidance and tutoring are especially important in university institutions, as it is a strategy that allows individual counseling and guidance to students about their university studies [1][2][3]. This, in addition to preventing academic failure and student dropout, also benefits the institution itself by reducing dropout rates and increasing the graduation rates established in the plans and programs [4,5]. Tutoring is related to higher performance and persistence [6][7][8]. For this reason, many universities consider the development of guidance and counseling strategies for students to be among the most important aims of their training programs, taking other national and international universities as a reference [9][10][11].
In recent years, various European researchers have taken up the analysis of the quality of guidance and tutoring in universities [12,13], which has led to an increase in studies focused on the design and validation of reliable instruments of measurement [14][15][16][17]. However, although in Latin American countries there is concern with improving the quality of academic tutoring or counseling at the university [18][19][20][21][22], there is a paucity of measurement instruments allowing scientific knowledge in this field to be obtained [1,3,19,22]. For this reason, it is essential to study the assessment of university tutoring in the countries of South America more deeply, and this involves the creation of reliable measurement instruments. Obtaining reliable instruments for university guidance and tutoring in this context would allow scientific evidence of their quality to be obtained, which would facilitate the extraction of strong and weak points from the programs proposed by the different universities. The improvement of the guidance and tutoring system should be seen as a prevention strategy for academic dropout, an essential matter for any university institution to consider.

University guidance and tutoring: The case of Ecuador
The Higher Education system in Ecuador is committed to measures that improve the educational quality of universities and guarantee access to all citizens. The introduction of new structures and services aimed at promoting the comprehensive development of students and extending their inclusion at all educational levels, has led to an organizational management model in agreement with the current demands of society in this country [23][24][25]. The Organic Law of Higher Education [26] establishes guidance as a student right and a teaching role for the faculty. This law creates an administrative unit or department of Student Welfare, among whose functions is the vocational and professional orientation of students. Its mission is to promote the comprehensive development of all students, attending to their interests, needs and personal aspirations, all through principles of prevention, continuity and quality of care and services implemented for the purpose.
Similarly, Art.45 the Organic Law of Intercultural Education [27], on the role of the Deputy Director or Vice Rector, determines that among the functions of this office is to implement pedagogical support and academic tutorials for students according to their needs. Furthermore, the Regulation of Career Path and Ranking of Lecturers and Researchers in the Higher Education System [28] endorses the existence of teacher-tutor guides to accompany students in their academic activities. The purpose is to provide personalized support for the effective use of the opportunities and services that the university offers students, attending to their personal needs, which increases the likelihood of remaining in the institution [22,29].
However, although education law and the aforementioned regulations have favored an increase in the quality of the Universities of Ecuador, according to the technical report issued by the National Council for the Evaluation and Accreditation of University [30], there are important limitations in a number of areas, among which are the low rate of graduates and the lack of teacher training. Based on the results of this report, the universities began a continuous process of assessment and supervision supported by the Council for the Evaluation, Accreditation and Quality Assurance for Higher Education, to guarantee educational quality and improve the services and infrastructures created for this purpose. In 2013, this council issued the first assessment report [31,32] and the results obtained in relation to the criterion of academic efficiency, in which the effective strategies for admission, retention and accompaniment in the educational process of students [32], were that most of the universities in category A and B in Ecuador have a fairly high level.
However, there are still very important limitations [30], related to the insufficient professional training of teachers and the small number of them who have postgraduate studies. Research studies in Latin America [19,24,[33][34][35] show that one of the reasons for this situation is that the professional work of the university lecturer is mostly made up of teaching. That is, "teaching classes" is established as a priority activity, and didactic and pedagogical training, and their professional advancement, are of secondary importance. According to Moscoso and Hernández [35], in the university institutions of Ecuador, less importance is given to such training, compared to disciplinary training.
Institutions are currently taking steps to improve the results reported on the low rate of graduate students [30]. Tutoring here becomes importance in Ibero-American universities, since it is considered essential for providing personalized attention to the student, guiding them in their personal, academic and professional development, and as a prevention strategy against academic dropout [20]. However, to be effective, tutoring must be linked to the teaching-learning process [11,22] acting as an anchoring link between the teaching function and the overall learning of students. To respond to these needs [36], the role of the teaching staff is defined as that of tutor and advisor, who assumes responsibilities beyond teaching practice.

Scales for university guidance and tutoring
The scientific literature includes many studies on the design and validation of scales, addressing different aspects and dimensions of university orientation and tutoring [16,[37][38][39]. Most of these studies assess the practice of university tutoring from the perspective of those involved-students and/or teaching staff- [24,36,40], by carrying out an extensive review of the management structures and systems used by university institutions for their promotion or improvement.
Some studies focusing on the design and validation of scales consider different dimensions of tutoring in the university setting. Despite the efforts to provide reliable measurement scales for university guidance and tutoring, there are many discrepancies to be considered when addressing the concept [16,17,36,41].
Pérez-Cusó et al. [36] describe and check two subscales for measuring satisfaction with university tutoring: 1) satisfaction with the tutor; and 2) satisfaction with the organization and content. These same authors [1] subsequently set out a scale to assess the needs of university guidance and tutoring, comprising five factors: a) adaptation to the university context; b) identity; c) integration and interpersonal development; d) teaching-learning process; and d) professional development. Other authors who contributed a five-factor scale were López-Castro and Pantoja-Vallejo [17], using the following dimensions: 1) functions and tasks of the tutorial activity; 2) satisfaction with the tutorial action in the center; 3) difficulties of tutorial practice; 4) use of technologies in the development of tutorial activities; and 5) knowledge and satisfaction with the work of the counselor.
León-Carrascosa and Fernández-Díaz [16], on the other hand, identify four factorial dimensions for guidance and tutoring in universities: 1) functions of the tutor with the students; 2) functions of the tutor with the families; 3) tutorials; and 4) assessment of tutorials. Another validation study looking at guidance and tutorial practice in universities [13], also opted for a four-factor structure: 1) academic orientation; 2) personal orientation; 3) career guidance; and 4) guidance and ICT.
Among the studies that describe assessment of tutoring with a three-factor model, the study carried out by Delgado-García et al. [15] stands out, proposing the following areas for consideration: 1) functions of the tutor (e.g.: reporting on institutional academic issues); 2) profile of the tutor (e.g.: affectivity, empathy, attitude. . .); and 3) student needs (e.g.: transition to university).

The current study
The literature gives great importance to university guidance and tutoring as a strategy in institutions for the development of students, not only academically, but also personally and professionally, as well as offering protection against academic dropout. This leads to the need to evaluate whether this process is carried out properly, with a n increasing amount of research aimed at producing valid and reliable instruments [15,16,36,41,42]. However, the discrepancies found between studies in relation to their factorial dimensions, and the scarce literature on this type of research in Latin America, require more studies of this kind. For all these reasons, a study to validate an instrument on university guidance and tutoring in Ecuador is here proposed, with the following objectives: 1. To explore and confirm the factorial structure of an instrument on university guidance and tutoring.
2. To analyze the psychometric properties of the instrument.
3. To apply the instrument to identify differences between sexes, universities and university courses.
Regarding the objectives, the following hypotheses were addressed: 1. The instrument will possess a multidimensional factorial structure.
2. The instrument will have optimal psychometric properties and positive correlation between dimensions.
3. The student body will show similar scores regardless of gender, universities and university degrees.

Instruments
An ad hoc questionnaire was developed, called "Questionnaire for the Assessment of Guidance and Tutoring in Higher Education" (Q-AGT), made up of a set of questions related to sociodemographic data (e.g.: sex, age, degree, level, university), and a set of 21 items related to university tutoring. Each of the items was assessed by the students using a Likert-type scale with five response options that ranged from 0 ("completely disagree") to 4 ("completely agree"). The items related to the instrument were grouped into 3 subcomponents of tutoring: a) 5 items on the importance of university tutoring; b) 9 items on the demands of university tutoring; and c) 7 competencies of the teaching staff for university tutoring.

Procedure and data analysis
Data collection proceeded by contacting each of the university institutions, to inform them and to obtain approval for participation in the research. Once approval was obtained, the questionnaires were completed, which took approximately 30 minutes. The participants, all of legal age, provided written informed consent, were informed about the voluntary nature of the questionnaire, respect for anonymity and the confidentiality of the data provided. All doubts were addressed and data collection proceeded without incident. The process was conducted in accordance with the ethical principles of the Declaration of Helsinki and was approved by the Bioethics Committee of the Technical University of Manabi.
The procedure developed to perform the study involves two different phases. The first phase was exploratory in nature, and examined the factorial structure of the instrument on university tutoring. Specifically, the Factor 10.9.02 program [43] was used to carry out an Exploratory Factor Analysis (EFA) to determine the number of underlying factors. Since the data were ordinal, it was decided to generate a polychoric correlation matrix [44] with the ULS (Unweighted least squares) extraction method. Following the recommendations of various authors [45,46], Promin oblique rotation was used, as it is adapted to the exploratory nature of the study. For the item-factor saturations, values greater than 0.40 were used as criteria [46,47].
In the second, confirmatory, phase of the study, the factorial structure of the instrument examined in the preliminary phase was validated, its psychometric properties analyzed and it was then applied. This was done via a Confirmatory Factor Analysis (CFA) using the EQS 6.2 program. In line with the previous phase, on detecting absence of normality, the robust estimation method was used [48]. The fit of the model was interpreted using the Satorra-Bentler chi squared method (x 2 S-B) and x 2 S-B/df, considering �3 to be optimal, and �5 acceptable. Other indices not affected by the sample size were also considered, such as the NNFI; NFI; CFI; IFI, taking values �0.95 as criteria for assuming a good fit [49]. Finally, values of RMSEA between 0.05 and 0.08 were also considered to indicate good model fit [50].
Likewise, in this second phase, descriptive and comparative analyzes were carried out with the SPSS 20 program. First, Student's t-test was used to determine the existence of differences between the sex of the participants and the factors found in the scale related to college tutoring. The ANOVA test was also used to verify the existence of differences between the students of the different universities and of the different university degrees with regard to their assessment of university tutoring. To determine among which groups there were differences to be found, Tukey-b or Games-Howell statistics were used, depending on the Levene test. Finally, the effect size (Cohen's d) was evaluated in the t test, considering scores < 0.5 as small, between 0.5 and 0.8 as moderate, and > 0.8 as large [51]. The confidence level was 95% (p <0.05) and 99% (p <0.01).

Exploratory factor analysis of Q-AGT
In the results obtained from the EFA, the Kaiser-Meyer-Olkin test (KMO = 0.78) and the Bartlett's Sphericity Test (Bartlett's Test = 2131.8; p <0.01) indicated that the sample was suitable for carrying out the CFA. The indices on simplicity −index S− and simplicity of loading −index LS− [52,53] indicated simplicity of factors and that the items were exclusively related to one factor. Table 1 shows the descriptions of each item, as well as the factorial weights and the factor to which each item belongs. The three factors found were called: a) F1 or Importance (IMP); b) F2 or Demand (DEM); and c) F3 or Competence (COM).

Confirmatory factor analysis of Q-AGT
In the second phase, the robust maximum likelihood estimation method was used, due to the lack of normality of the data (Mardia coefficient = 318.9760).
The various goodness of fit indices of the model were considered optimal: x2S-B (186) = 624.3760; p = 0.00 [50]. Those that evaluate the relative fit of the model and are not affected by the sample size, also returned high values, indicating a good fit to the model: NFI = 0.982; NNFI = 0.986; CFI = 0.987; IFI = 0.987; RMSEA = 0.053.
Polychoric correlations between items of the scale were positive (Table 2), ranging from 0.328 y 0.766.
The indices of reliability and internal consistency obtained are considered satisfactory for the instrument in general (α = 0.930) and for each of its factors: a) F1 or Importance (IMP), α = 0.853; b) F2 or Demand (DEM), α = 0.895; and c) F3 or Competence (COM), α = 0.894. Table 3 shows the participants' sociodemographic data in relation to the 21 items that make up the Q-AGT.

PLOS ONE
Validation a questionnaire for university guidance and tutoring As seen in Table 4, the results did not show statistically significant differences in the assessment of university tutoring between men and women. Both gave a high valuation of the importance of tutoring in the universities of Ecuador, the demands that students make on it and the skills of the teaching staff in undertaking it, related to guidance and counseling.
Regarding the different participating universities, statistically significant differences were detected between universities in the three factors: importance, demand and teaching skills. Table 5 shows the results obtained from the post hoc test, where it is mainly the students of the Taking into account the university degrees of the participants, the students in the branch of sciences expressed a greater appreciation of the importance of university tutoring (see Table 6). However, no differences were detected between students of the Science and Social Sciences degrees in the factors related to the demands of tutorials and the skill at university tutoring of the teaching staff.

Discussion
The first two objectives of this study were to explore and confirm the factorial dimensions of an instrument that we call "Questionnaire for the Assessment of Guidance and Tutoring in Higher Education" (Q-AGT), based on the review of the scientific literature on scales that evaluate university guidance and tutoring [13,24,36,40]. In agreement with other studies, an analysis of the psychometric properties of the instrument was carried out to evaluate the state of tutoring in Latin American universities [16,37,38], in order to respond to the lack of research into this matter in Ecuador.
In accordance with previous work [16,17,36,41], the multidimensional nature of the assessment of university tutoring is confirmed. Specifically, and according to other research [15], the instrument was defined by three factors. Factor 1 -Importance-addresses the importance attributed by students to information, organization of the center and the curriculum; adaptation and integration with the university; help with academic, personal and professional development. The second factor -Demand-, assesses the levels expressed by students about the need for information; monitoring academic, personal and professional guidance; problem solving and difficulties; decision making; help in level transitions; and attention to diversity. Finally, the third factor -Competence-addresses the teaching competence that, from the student perspective, is necessary for the proper practice of university guidance and tutoring, such as competence in university tutoring and organization; competence in social insertion;   knowledge and skill in the techniques of tutoring; inter-and intrapersonal competence; knowhow and acceptance of criticism.
The three-factor model proposed for Q-AGT has confirmed the construct validity, showing an appropriate metric quality and a satisfactory fit of the model, and giving a scale with high reliability indices. In this way, a scale is obtained that is useful for generalizing results on guidance and tutoring in Latin American university students. This contributes to the assessment of various questions that have appeared as deficient, or little studied, in previous research, such as: a) attending and understanding the academic, personal and professional needs of the students through individualized support [14,22,29]; b) possible low skill in tutoring and university guidance in the teaching staff of Latin American universities [30]; and c) contribute to evaluating a criterion of academic effectiveness related to admission, retention and support in the educational process of students [32]. In this sense, Q-AGT would be considered of great importance for initial courses, as an early detection mechanism for the prevention of academic dropout.
We agree with Ponce Ceballos, Aceves Villanueva, and Boroel Cervantes [54], who point out the importance of continuing to explore the evaluation of guidance and tutoring for the improvement of the student support processes and the training of teachers to undertake this activity in the university environment. This is related to superior performance and also greater persistence [8].
With regard to the application of Q-AGT, in the first factor -Importance-, the students attribute great value to university tutoring as an environment in which to promote personal development, especially autonomy, self-esteem and identity. In line with previous studies [1,14,16,55], the importance of guidance and tutoring in the university context for personal, academic and professional development is underlined. These three types of development, which must be addressed by responding to the needs of students throughout their time at the university, support the idea of the progress of university tutoring towards a comprehensive model in which the personal dimension becomes especially important [15].
Looking at the second factor obtained -Demand-, the students express the need for guidance to help with job placement, as well as support and academic monitoring by the teaching staff of their university studies. These results are consistent with those obtained in other studies carried out in Europe and Latin America [9,54], where students say that they feel a greater need for backup and support, specifically in important areas related to their future skills and employability −e.g. analysis of strengths and weaknesses, strategies for coping with stress and adversity, etc.−. This type of skill is in high demand from employers, as shown by the recent studies of González [56] and Bartual and Turmo [57].
Finally, with regard to the third factor −Teaching competence− the scale groups together the variables related to the competence of university teaching staff for tutorial work, such as intra-and interpersonal, technical, academic skills, and skills to help with job placement. Similarly, Pérez-Serrano et al. [58], highlight the importance of orientation and "Face to face" advice for university tutorials. Other authors, such as Pantoja-Vallejo et al. [13], in their study to validate a scale for measuring and assessing the guidance and tutorial practice, concluded that to the skills required for the tutorial work of the teaching staff should be added a crosscurricular competence, which comes from the domain of ICT.
Applying the scale has also made it possible to extract relevant, interesting information from the study. It has not been shown that women and men [20,24] disagree on the assessment of university tutoring. Both sexes value highly the importance of tutoring in the universities of Ecuador, the demands that are met through it, and the competence of the teaching staff to carry it out.
The discrepancies between different universities suggest the strengths of university guidance and tutoring programs from the student's point of view. Although all the universities expressed a good, high valuation in this regard. Those students who are studying science [59,60] value the importance of college tutoring more highly.

Strengths, limitations, and future lines of research
One of the main limitations of this study is that it would have been desirable to expand the sample with subjects from other Latin American universities. In addition to university students, the opinion of other agents involved in university guidance and tutoring, such as teachers, would also be of interest.
Although the methodology, a cross-sectional approach using a survey, is a method commonly used in the study of the subject, it prevents the identification of causal relationships with other constructs. Also, the absence of studies carried out in Ecuador means that comparisons cannot be made that might contribute to deepening the scientific knowledge related to university tutoring and guidance.
Despite everything mentioned above, work is being done to overcome these limitations. First, data is currently being collected from a larger sample and the intention is to extend the assessment of counseling and mentoring to other Latin American countries. It is likewise intended to expand the group, in future studies, to university faculty, and to include the record of other variables related to the psychological well-being of students and teachers.

Conclusions
The main conclusion of this study is that a valid and reliable instrument has been obtained to measure the performance of guidance and tutoring in Ecuadorian universities, which is theoretically grounded and operationally defined based on three factors: a) Importance; b) Demand; and c) Competence. The results of the Exploratory Factor Analysis show that the factors obtained and their loads correspond to the prior theoretical approaches.
The students have given a high assessment of the importance and demand of tutoring in the Ecuadorian university context, and of the teaching competence of the staff to meet the demands that satisfy thereby, all regardless of gender.
In relation to the participating universities, the student body of the Technical University of Manabi is the one that gives less importance to tutoring and to the demands generated by the student body. In contrast, the students of the Agricultural Polytechnic School of Manabi attach great importance to tutoring and believe that their teachers have good skills in tutoring. It is true, however, that despite the discrepancies, all universities scored highly in the assessment of tutoring and university guidance.
Considering the difference between the university degree courses in Ecuador, the results show that the students of degrees in Sciences are those who most value university tutoring.
Finally, it gives great importance to university orientation and tutoring as a strategy for the holistic development of the student -personal, academic and professional-and as a preventive against academic dropout. This study assesses whether this process is carried out properly, in line with other recent research aimed at the development of valid and reliable instruments [15,16,41]. Likewise, it is expected that this study will contribute to shed light on the discrepancy found between studies in terms of the factorial dimensions of tutoring, and provide knowledge in light of the very limited literature in Latin America, and specifically in Ecuador.