Introduction

Several lines of evidence indicate that medical schools have been failing to adequately nurture empathy and the ethical dimension in their graduates [1–5]. In 1994 Feudtner et al [6] coined the term “ethical erosion” to denote a process that affects many medical students during their training and which is characterized by the abandonment of certain moral values, changes in attitude and in ethical identity. In alignment with that perception, Leape and colleagues [3,7] made a strong argument that the healthcare field is paved with a diffuse culture of disrespect which plays a central role in the genesis and maintenance of medical errors, itself a major source of avoidable deaths, incapacity and wasted resources [8].

Most experts identify the phenomena of declining empathy and ethical erosion during medical education as a consequence of the hidden curricula of healthcare organizations [2,9,10]. The hidden curriculum has been defined as the learning derived from the cultural and organizational environment of institutions [11,12]. It consists of unwritten, taken-for-granted rules and customs that provide lessons about what is important, valued and acceptable or not in the relationships with patients and colleagues on a daily basis. Negative examples that medical students and residents observe during their training represent the dimension of the hidden curriculum that plays the most important role in the phenomena of declining empathy and ethical erosion [2,6,9,11,13–16].

It has been proposed that to change that picture for the better, medical schools should adopt evaluation strategies that foster a cultural environment of respectful and trustworthy relationships among professionals, students and patients [7,12,17]. The rational behind this proposal is that evaluation strategies are an important driver of change in institutions as administrators and staff may value them just as students consider carefully what is expected of them to succeed in their finals [12,18–20]. Despite the apparent logic of those arguments there is a significant gap in the knowledge base concerning if and how medical schools have been using evaluation strategies with those aims. As a matter of fact, we were not able to identify any report of any survey of medical schools examining such strategies. Hence, we conducted the present survey to examine the instruments used by Brazilian medical schools to assess clerkship rotations from the perspective of students, with a main focus on items associated with the ethical domain. Our main hypothesis was that in most cases the available instruments would not address possible experiences of unethical situations and behaviors during those rotations.

Materials and methods

We conducted a national survey of instruments used by Brazilian medical schools to evaluate clerkship rotations from the perspective of medical students. Data collection took place between August 2014 and September 2015. Every medical school that had at least one group of students graduated by the first semester of 2014 was eligible to participate in the survey. In May 2014 there were 176 medical schools in Brazil that fulfilled that criterion.

We calculated a sample size of 121 schools to attain a margin of error of 5% and a confidence interval of 95% considering a response distribution of 50% regarding the presence of instruments assessing the ethical dimension during clerkship rotations [21]. The selection of medical schools to participate in this study proceeded as follows. We extracted a list of all medial schools in Brazil from the website of the Brazilian Ministry of Education and organized those schools in alphabetical order in a numbered electronic spreadsheet. We then used a computer-based random number generator to produce a sequence of numbers corresponding to the schools to be invited to participate in the study.

We used multiple strategies to invite medical schools to participate in the study, as follows:

• We asked the Brazilian Medical Education Association to send an electronic invitation for its member schools.
• We made phone calls and sent invitation e-mails to the administration offices of each medical school.
• We asked for the assistance of members of the Foundation for the Advancement of International Medical Education and Research in Brazil through their e-mail server.
• We also contacted representatives of medical schools in person during a regional meeting of the Brazilian Medical Education Association.

Once we managed to establish communication with the administration office of a medical school, we sent them e-mail containing the consent form to be signed by a representative of that medical school, and a questionnaire to be answered by a key informant appointed by the administration office of each medical school.

We performed at least 5 communication attempts in different days and hours before classifying a school as non-respondent in the study. Once we achieved communication by phone or e-mail with the administration office of a medical school, we waited about one week to contact it again asking for an update concerning its participation in the study. If no feedback was provided, we made further weekly contact attempts with phone calls and/or e-mails for at least 5 weeks before we discarded a medical school from the study.

The questionnaire asked for information concerning the duration of clerkship in each institution, where the rotations took place (e.g., academic hospital belonging to the medical school, other hospitals not belonging to the medical school, public primary care clinics, etc.), and whether the school used any standardized instrument to evaluate the clerkship rotations according to the point-of-view of the medical students. If the answer to that last item was positive, we asked whether the students’ evaluations according to that instrument were anonymous and requested a copy of the assessment instrument.

We also obtained some descriptive data regarding each of the participating medical schools from the electronic registry of the Brazilian Ministry of Education. Those data included the location of the medical school (city and state), the number of years of functioning, the number of positions offered each year for new medical students, and the administrative nature of the institution (public or private).

Two independent researchers using a standardized electronic sheet extracted information concerning the assessment instruments that medical schools used to evaluate the clerkship rotations according to the perspective of students. A third researcher arbitrated any disagreements regarding data extraction.

We classified each item of the assessment instruments according to the dimension that it evaluated into one of 4 categories: a) organizational/administrative, b) technical, c) ethical, and d) nonspecific. We defined the organizational/administrative dimension as that associated with elements such as general orderliness of the rotation, neatness, punctuality of staff and activities. The technical dimension was outlined as involving aspects such as specific knowledge of diseases and the performance of diagnostic or therapeutic procedures. The ethical dimension was conceptualized as associated with moral and behavioral aspects of interpersonal relationships such as trust, confidentiality, respect, beneficence, justice, non-maleficence and autonomy. We used the nonspecific category for quite general questions that could be associated with more than one of the three previous dimensions (e.g., “How would you rate this rotation in general?”).

We also searched each instrument for items concerning the following issues:

1. Was there at least one question within the instrument that was directly related to the ethical dimension of the experiences and/or the learning that occurred during the rotation?
2. Was there at least one question concerning the performance of the preceptors of that rotation related to the ethical dimension?
3. Was there at least one question regarding the performance of multidisciplinary healthcare teams related to the ethical dimension?
4. Was there at least one question within the instrument assessing ethical implications of the physical environment where the clerkship rotations took place? For instance, if the physical environment allowed patients confidentiality and privacy when discussing sensitive issues or exposing their bodies.
5. Was there at least one question related to the occurrence of positive or negative experiences during the rotation?
6. Was there at least one question within the instrument asking students specifically about having witnessed or experienced mistreatment in the relationships involving preceptors, the multidisciplinary healthcare team, patients and/or other students?

If any of the answers to items 2, 3 or 6 above was positive, we further examined if those questions were more general or involved specific members of the team (e.g., physicians and nurses), patients and/or students.

For the purposes of data collection we defined preceptors as any healthcare professional responsible for the supervision of students during the rotations. We also defined “mistreatment” as any behavior associated with disrespect, humiliation, threats or prejudice of any kind.

We used simple measures of frequency data with 95% confidence intervals [22]. We compared means using Student’s t test and proportions with the Chi-square statistic. We also performed an exploratory multivariable logistic regression of the relationship between the presence of at least one item within the instruments concerning specifically some aspect of the ethical dimension and the following variables: years of functioning, number of vacancies for admission of new students per year and the administrative nature of each medical school (public or private).

Missing data were reported as such and were not included in the relative frequencies reported in our results.

We used the R software 3.3.2 (Vienna, 2016) [23] for all statistical analyses. We adopted 0.05 as the value of α for statistical significance.

The local Ethics Review Board of Botucatu Medical School approved the study protocol and a representative from each participating medical school provided written informed consent. We followed the STROBE guidelines [24] for presentation of reports of cross-sectional studies and a completed checklist is available as a supplementary file.

Results

We provide a summary of the process of selection and invitation of medical schools to participate in the survey in Fig 1. Forty-nine (40.5%) out of 121 schools accepted our invitation to participate in the study. Nonparticipating institutions had lower mean time of functioning when compared with participating institutions (Mean Difference (MD): 35.8 years, 95%CI: 25.5–46.1, P<0.001). There were no statistically significant differences when comparing nonparticipating with participating schools concerning the mean number of vacancies for new first-year students (MD: 15.0, 95% CI: -3.0 to 33.0, P = 0.10) and the proportion of public to private schools (0.71 vs. 0.96, respectively, P = 0.54).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flow diagram of the inclusion of medical schools in the study.

https://doi.org/10.1371/journal.pone.0217717.g001

Of the 49 participating institutions 25 were private and 24 were public schools. The median time of functioning for all schools was 44.2 years (interquartile range (IQR): 6.7 to 51.1). For 46 schools the clerkship period lasted for 2 years, for 2 schools it lasted for 1.5 year and for 1 school it lasted for 2.5 years. All but 2 schools had at least one clerkship rotation in institutions that did not belong directly to the medical school or to its university. Only 16 schools had a university hospital of their own and 39 schools used other hospitals outside of its direct administration as a training field for at least one rotation. Forty-five schools used public primary care clinics as a training field for their students.

Twenty-six (53.1%, 95%CI: 39.4–66.3%) schools reported using at least one institutional instrument to evaluate the quality of clerkship rotations according to the perspective of students. Twenty-four (92.3%, 95%CI: 75.9–97.9%) of those 26 schools stated that assessment instruments allowed student anonymity. However, we could confirm that the assessment instruments were clearly anonymous (e.g., there was no field for student identification) for just 12 (54.5%, 95%CI: 34.7–73.1%) of the 22 schools that had sent us copies of those documents. For 4 (18.2%, 95%CI: 7.3–38.5%) other medical schools there was some written information within their instruments stating that student identification was optional.

Table 1 describes the composition of the assessment instruments according to the 4 dimensions we adopted for our analyses. Table 2 shows examples of questions regarding those 4 dimensions within the instruments analyzed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Mean numbers of items by assessment instrument and overall number of items among the 29 assessment instruments analysed according to the dimension addressed by each item.

https://doi.org/10.1371/journal.pone.0217717.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Examples of items addressing ethical, technical, organizational/administrative and nonspecific dimensions within the instruments analysed.

https://doi.org/10.1371/journal.pone.0217717.t002

Only 13 (26.5%, 95%CI: 16.2–40.3%) of 49 schools had an instrument with at least one item addressing issues related to the ethical dimension. The most common items associated with that dimension involved the relationship between preceptors and students (13 schools, 26.5%, 95%CI: 16.2–40.3%), preceptors and patients/families (7 schools, 14.3%, 95%CI: 7.1%-26.7%), preceptors and the multidisciplinary healthcare team (3 schools). The instruments of only 2 (4.1%, 95%CI: 1.1–13.7%) schools included 1 question regarding specifically the experience of negative ethical events during a rotation and just 1 (2.0%, 95%CI: 0.4–10.7%) school asked for positive ethical experiences. Likewise, merely 1 medical school specifically asked if students had witnessed any situation of patient mistreatment during a rotation, and no similar questions were made concerning mistreatment against the students themselves or towards members of the healthcare team. Additionally, no school (0%, 95%CI: 0–7.3%) asked students about aspects of the physical environment associated with the ethical dimension, such as if it allowed for privacy.

The exploratory logistic regression did not disclose any significant relationship between the presence of at least one item within the questionnaires concerning specifically some aspect of the ethical dimension and the following variables: years of functioning (OR: 1.00, 95%CI: 0.98–1.02), number of vacancies for admission of new students per year (OR: 1.00, 95%CI: 0.99–1.02) and the administrative nature of each medical school being public instead of private (OR: 0.81, 95%CI: 0.19–3.49).

Discussion

To our knowledge this is the first national survey of medical schools in any country focusing specifically on the institutional instruments used to assess the quality of clerkship rotations according to the perspective of medical students. The first relevant finding of our research was that almost half of the participating medical schools did not have any institutional instrument designed to assess the quality of the clerkship rotations according to the perspective of students. That finding is important because it discloses a major missed opportunity to unveil aspects of institutions and of their functioning that could benefit from quality improvement initiatives concerning both medical education and the care of patients. It is especially relevant in the present context where most schools have several rotations taking place outside their institutions. It also represents a wasted chance to communicate students and staff about the most important values and quality models that schools want to nurture within their domains in a more concrete manner than through institutional discourses.

Another important result lies in the absence of several ethics-related items within the instruments that we analyzed. For instance, the finding that only 13 of 29 instruments had at least one question encompassing the ethical dimension suggests that there is often a divide between the usual emphasis that medical schools give to that dimension in their official discourse and the actual importance it receives within the hidden curricula of institutions. That interpretation is consistent with the results from Stern and colleagues [25] who showed significant differences between the “recommended curriculum” and the one actually taught in inpatient general medical wards. Our results allow us to hypothesize that the dearth of assessment instruments emphasizing the ethical behaviors recommended within formal curricula might be a contributing factor for the phenomenon they described.

There is a plethora of evidence demonstrating that anti-ethical behaviors such as harassment and other forms of mistreatment are common in healthcare institutions around the world [9,15,16,26–29]. Despite that fact, only 2 instruments asked students directly about negative ethical experiences during the rotations and only one school asked if students had witnessed any occurrence of disrespect towards patients. No school asked students whether they had either experienced or witnessed mistreatment toward themselves, other students or staff members. When schools fail to ask students specifically about those issues within their institutional assessment framework, they are turning a blind eye to those unfortunate situations, which certainly facilitates their perpetuation.

A possible explanation for the absence of ethics-related questions is that many leaders and administrators of medical institutions were not trained about how to confront such problems and end up choosing not to face those issues [30].

Our findings are in marked contrast with the American Academy of Medical Colleges Graduation Questionnaire, which has an entire section devoted to “Behaviors experienced during medical school” where students are asked straightforwardly for mistreatment experiences ranging from public humiliation to sexual harassment [31,32].

It was reassuring to notice that most (24 out of 26) medical schools understood their assessment instruments allowed for student anonymity. This is important because mechanisms that allow for anonymity may decrease fear of retaliation, which is a significant barrier for the reporting of mistreatment and disruptive behaviors by faculty and staff [33–36]. However, we were not able to analyze how effectively anonymity was ensured by internal procedures within each institution, especially in the 10 cases of medical schools where there was a field for student identification within the assessment instruments.

The absence of any clear relationship between the presence of at least one item addressing the ethical dimension within the instruments that we evaluated and the number of years of functioning, the number of vacancies for admission of new students per year and the administrative nature of each medical school was not surprising. It is probably a consequence not only of the relatively limited number of participating schools but more likely of the fact that the characteristics of the assessment instruments from individual institutions depend on much more complex variables than we were able to measure.

Although our results are restricted to Brazilian medical schools, reports from other areas of the world have also identified several problems concerning numerous aspects of medical education, ranging from the use of outdated sources of information and lack of appropriate supervision of medical students to fraud [37–41]. Furthermore, the rapid growth in the numbers of new medical schools in the world has raised concerns that many of them might have started functioning on inappropriate grounds [42]. Hence, we hypothesize that the dearth of assessment instruments and of ethics-related items within instruments that we found in this study may also be common in other countries.

This study has several potential limitations. The participation ratio of medical schools was much below the intended number and due to logistical reasons and limitations regarding funding we were not able to expand data collection to schools outside our initial randomization list. Therefore, our results may not be generalizable to the remaining medical schools in the country. However, it is probable that nonparticipating schools were more likely not to have any institutional instrument to assess clerkship rotations, since that situation could have contributed to their nonparticipation and because nonparticipating schools had significantly less years of functioning than participating schools. Hence, our results might represent a conservative and overoptimistic picture. We adopted a very broad conceptualization of the ethical dimension during the analysis of the assessment instruments. For instance, if instruments had any question about the quality of the interpersonal relationships between preceptors and students, they were classified as having at least one item addressing the ethical dimension. In addition, we were not able to assess how the ethical dimension is assessed in other courses of participating medical schools or the existence of other initiatives that were designed to allow the reporting of harassment, mistreatment and other types of unethical behavior (e.g. ethics hotlines). Finally, we were not able to examine how medical schools deal with the results from the assessments they received from students concerning clerkship rotations or other strategies in place to promote an ethical cultural learning environment.

In summary, our results show there are several missed opportunities in the way medical schools assess the quality of clerkship rotations regarding ethical domains. Closing the gap between usual institutional discourses regarding ethics and how those dimensions are assessed within clerkship rotations might represent an important step towards the improvement of medical education and healthcare systems.

Supporting information

Acknowledgments

The authors wish to express their gratitude to Vanessa dos Santos Silva for her helpful comments regarding this work; to Jacqueline Caramori and Joelcio Abbade, for their support in contacting medical schools.