Methodological issues

Despite the similarities between Italy and Brazil, there is a marked difference between the two sociocultural contexts. However, some methodological issues should be properly addressed to avoid some sources of significant bias in the statistical tools to be used. The sample design was an important step, as nationwide samples best represent the complexity of the several sociocultural influences. As we were targeting the main religious group in both countries (Catholics), no oversampling was necessary. Non-Catholic Christians would be part of the sample, but no specific conclusions could be drawn considering the different religious affiliations within the group.

Brazil is a country where religious practices are diverse and encompass several different belief systems and traditions, which reflect elements of European (Catholic and Protestant), African, Oriental, and indigenous religions, among others. This diversity has its foundation in the colonization process of European immigrants from 1500 onwards, African descendants, and local indigenous tribes [46].

The group of non-Catholic Christians is very heterogeneous, and our database aggregated several different affiliations in the two countries: Orthodox, Lutheran, Presbyterian, Baptist, Adventist, Jehovah’s Witnesses, Assembly of God, Christian Congregation of Brazil, International of God’s Grace, World of God’s Power, and the Universal Church of the Kingdom of God. Therefore, we consider that the comparison between Catholics in the two countries was more suitable for this study, as it is a more homogeneous group. The term “evangelical” represents the universe of non-Catholic Christians considered by Brazilian society as “Brazilian Protestantism”, which is constituted by historical Protestants, Pentecostals, and Neo-Pentecostals and led by Pentecostals, with a massive dissemination by Neo-Pentecostals. Brazil is considered the country with the largest number of Roman Catholics in the world. However, this hegemony has been decreasing in recent decades, which reveals a changing trend in the religious composition of the population [47]. Data from the population census in Brazil point to the decline of the Roman Catholic religion and a continuous rise of protestant groups in the last decades [48].

The age level was an essential component for sampling, as elderly people tend to be more conservative than young students. Taking advantage of an international project focusing on youngsters who were beginning high school level in both countries, we prepared separate sets of questions about evolution so that we had the very same questions answered by students of the same age in the same time frame, during the first months of the year 2014. Targeting students at the same age level in two different countries allows for closer comparisons than multi-age-wide groups. In the next section, a more detailed description of the sampling procedures in both countries is given.

There is an important distinction that we must keep in mind when comparing the concepts of belief and acceptance. Differences between them have long been discussed in the philosophical literature and the specific field of evolution education [25,27,43,49]. The overall picture takes belief as a subjective disposition to consider p. simply as a premise under the influence of the wide cultural context. A well-known distinction is recognized between belief and conjecture since the latter is not committed to sensible things and applies even to the world of shadows. In this context, it is important to separate the notions of belief of classical philosophers, such as Plato and Aristotle, from the revision carried out by Thomas Aquinas, who considers belief in terms of "thinking with assent" related to faith [49]. In our theoretical framework, belief refers to the sensible world only, with the reality of things that exist. However, one can take p. as a belief, but it does not in itself imply an admission of the objective validity of the notion under consideration. Meanwhile, to accept p. implies not only recognizing the existence of some objective reality but also a further step in terms of judgment, admitting p. as valid under some specific circumstances; in short: “acceptance implies a commitment to a policy of premising that p.” [50].

The instrument used in the recent work by Kuschmierz et al. [19], ATEVO, relies on eight items with statements about evolution, and students show their degree of agreement with each of them [51]. In the German context, the acceptance of evolution is defined as a “positive attitude” (“positive Einstellung”) towards evolution. In this regard, “attitude” (“Einstellung”) is defined as the level of association between a term or fact and its subjective evaluation, in terms of strength (“Starke”) and/or ability to bind with other terms or ideas (“Valenz”). On the opposite side, a “negative attitude” (“negative Einstellung”) is seen as “rejection” (“Ablehnung”). However, the result of ATEVO test may be the result of very different processes. On the one hand, “it remains unknown whether a person answers negatively for most items about evolution because he/she has a ‘bad feeling’” (“schlechtes Gefühl”) about evolution, in the sense of a preexisting emotional refusal to think on the subject. On the other hand, the rejection may have been the result of a long reflection leading to a rational conclusion that evolution is not plausible [51, p. 12].

We adopted a clear definition of evolution acceptance despite the complexity of the discussion on the subject in different languages, taking acceptance as the expression of explicit recognition of the objective validity of known scientific statements about evolution under absolute anonymity. This definition considers two steps. The first is associated with scientific statements about evolution, which must be clear and well known, avoiding issues under discussion, for instance, about the origin of life. Students must show not only a positive attitude towards evolution but also express clearly that a statement based on biological evolution is considered a valid premise to construct a judgement about the real world.

The second step refers to objective conditions in which a person may admit his/her positive judgment about a certain scientific statement. One may know a proposition but to refuse to show public recognition of it, or, on the contrary, manifest an opinion different from his/her deep feeling due to the suspicion that it may reach a wider audience. The perception of secrecy in everyday life is actually a major focus of concern worldwide. Strong evidence published in the literature shows that anonymity reduces social desirability distortion and increases self-disclosure [52,53].

Therefore, anonymous participation must be guaranteed to every person who is responding to questions about religion-related subjects, such as biological evolution, especially when surveys are carried out within conservative social contexts. Under secrecy, a person may accept a proposition as a valid premise for argumentation, making inferences, deliberations, etc. as a mental, individual, and innermost act. However, this may not happen if there is a perception of the possibility of third parties identifying the individual opinion. This means, for instance, that phone calls or traceable connections using electronic devices such as smartphones could be perceived by respondents as possible ways of infringing the inviolability of the anonymous character of the individual manifestation.

In our case, the students answered questions in their school environment and were told their answers would be under absolute anonymity. Previous analyses of this data bank showed surprisingly high mean scores of evolution acceptance within fundamentalist religious groups in Italy and Brazil [54]. This was also the case in a previous survey using a different methodology, carried out in 2010–11, where roughly half of Evangelical high school students in Brazil declared under absolute anonymity that their religion was not opposed to evolution [55].

In line with the tantalizing results of the study by Barnes et al. [6] already mentioned, our research team proposed a revision to some methods adopted in the previous survey. As mentioned before, we were carrying out data collection as part of an international project focusing on students’ interests and attitudes on science and technology, which had started in 2007 [56]. The main research instrument was based on Likert scale items but allowed a separated section, which we added in both countries. However, we considered a different approach for these evolution items, as we were going to present students with factual statements that were regarded as right or wrong by scientists.

Items following the Likert attitude data consist of favorable or unfavorable statements about an entity, admitting a spectrum of responses ranging from strong positive to strong negative answers, with a numerical scale from 1 to 5. The resulting data can be analyzed using several models that underlie latent variables. However, attitudes as well as response styles can affect the result. It is well known that numerical comparisons rely on the degree of precision of measures, for instance, to estimate the mean height of a group of people. The significance of the different statistical analyses are dependent on the precision of the instrument used in data collection. Likert scaling is used in instruments measuring attitudes, beliefs, and opinions about statements, allowing for an expression of moderation of opinion from agreement to disagreement. The key is to find ways of calibrating how strongly or mildly a statement should be worded. These scales are therefore suitable for issues related to open constructs and less suitable for scientific claims based on presumed factuality. An item likely to produce extreme responses, either full agreement or strong disagreement, would do a poor job of discriminating across the full spectrum of respondents [57]. Therefore, Likert scales may not be a good choice with statements expressing well-known scientific facts. In addition, in the case of theories of evolution, factuality is supported by the school context in which the questionnaires were filled out. The battery of items on evolution was proposed with dichotomous questions, given that each of them can have minimal variability.

When evolution is considered, opinions can be expressed in terms of favorable and unfavorable subjective statements, such as “There is a significant body of data which supports evolutionary theory” [33], “There is strong, reliable evidence to support the theory of evolution” [24], or “I think there is reliable evidence to support the theory that describes how humans were derived from ancestral primates” [33]. In these three examples, latent variables can be inferred in terms of personal attitudes, which can range from weak to strong, and grading would help to correctly appraise the level of agreement. However, well-known factual scientific statements cannot be graded as weak or strong scientific facts. For instance “The age of the Earth is at least 4 billion years” [33] or “I think that humans and apes share an ancient ancestor” [15], should be recognized as sound scientific statements (or not). We have here a black/white choice, and respondents would be able to express their judgment about these scientific statements as true or false (or declining to express their opinion). When a well-known scientific statement is marked as false, this means an explicit refusal to admit it as a valid premise to analyze a certain situation.

One could argue that the level of admission or rejection can also be variable, but we cannot overlook the fact that respondents are aware they are invited to make a judgment different from simply recognizing the existence of an idea. In other words, respondents know what is expected from them. The scientific statement about the age of the planet is widely known, both from the scientific and the fundamentalist religious points of view. When ticking “false”, the respondents are confirming their rejection of a statement known to be accepted by a wide scientific community. A weak disagreement in a Likert scale should not be taken as equivalent as rejection, as one could say that there is some disagreement regarding the presented cipher, as a professional paleontologist probably would argue about the precise age of our planet. This disagreement cannot be taken as similar to the one from a Young Earth creationist, but in a Likert scale, both would have similar scores (1 and 2 out of 5 points in the Likert scale).

Thus, in such cases, weak and strong disagreement may represent completely different attitudes, but with similar numerical values, undermining the measure’s precision. The option to change the instrument’s true/false options could enhance measurability, a critical aspect of survey sampling [58]. Thus, contrary to some of our previous pieces of research, we did not adopt Likert scale items in the questionnaire used in this study, which is presented below.

Survey instrument

We sought to measure evolution acceptance based on five clear statements related to the evolutionary theory: Earth age, the fossil record, common ancestry, and the origin of human beings. We included an item about understanding, related to deep time, as it has been considered a relevant barrier to evolution acceptance [59], while statements about human evolution are highly controversial even amongst biology teachers [60]. As mentioned before, the section of evolution provided scientific statements as a separated set of items as part of a longer questionnaire within the Relevance of Science Education (ROSE) project [see 56]. One seminar was held in Venice (Italy) and another in Brasília (Brazil) in the year 2012, bringing together researchers from both countries for the construction of the research instrument. Collaborative arrangements were defined for conducting joint data collection using the same instrument, whose final wording was defined after fieldwork and validation with students from two high schools in the following year and applied in early 2014 in both countries [54].

Eight items related to evolution were offered to the students under a command line asking to simply tick each of the following statements as true or false, and a gentle refusal to answer the question (“I wouldn’t know how to put it”). The Italian expression (“non saprei dire”) was considered similar to the Portuguese translation adopted in Brazil (“não saberia dizer”). The statements are presented below, according to their identification in Section G of the questionnaire, which was then called the “SAPIENS Barometer” of the second round of the ROSE project in Italy and Brazil. The items read as follows:

• G75—The formation of our planet occurred some 4.5 billion years ago;
• G76—Fossils are evidence of living beings that lived in the past;
• G77—Present-day species of animals and plants originated from other species of the past;
• G78—Evolution occurs in both plants and animals;
• G79—Humans are descended from other primate species;
• G80—The human species has inhabited planet Earth in the last 100,000 years;
• G81—Different organisms may have a common ancestor;
• G83—The first humans were prey to carnivorous dinosaurs.

As item G82 was inadvertently absent in the Italian questionnaire, it was not considered here for comparisons between Brazilians and Italians.

Sample design and research ethics

Sample design is basically composed of two aspects: a selection process, defining the rules and operations leading to the definition of clear targets in a wide population, and the estimation process, leading to sample estimates of population values [58]. These two aspects were standardized in both countries since the two research teams were taking part in the same international project above mentioned [56]. This international project targeted students who were beginning high school level for several reasons; one of them is related to the fact that at this age level, moral development is already completed in most cultures, predicting long-term behavior [61]. The study was conducted using stratified sampling both in Italy and Brazil.

The Brazilian sample was designed as a sub-sample of the 2009 OECD-PISA sample and considered characteristics such as the offering of high school levels, the administrative organization of the school (public or private), the location (rural or urban, including all capitals and cities in the interior of each state), and the Human Development Index (HDI) of the state. There were three explicit stratification variables (state, grade status, and certainty), with a total of 82 explicit strata, and three implicit stratification variables (origin of funding, urban/rural, and HDI level) [62]. The PISA sample comprised 976 schools from the 27 units of the Brazilian Federation, from which 120 schools were randomly drawn, with implicit stratification to allow all federation units to be part of the subsample. This total included 20 schools for replacement.

Parcels with paper questionnaires and instructions were sent by ordinary mail to 100 schools located in 87 different municipalities all around the country after phone/email contact with the schools’ principals [63]. Differences from the PISA procedures were explained. They were to select one classroom from the first year of high school (“primeiro ano do Ensino Médio”) by a random drawing. A reasonable number of schools (n = 78) returned the paper questionnaires after a few weeks, with short reports describing how the process was carried out and a signed letter (see below). After reading the report, the questionnaires were scanned for machine-reading, leading to a digital database with answers for 96 items and personal and family details, such as age, gender, religious membership, and socioeconomic indicators. The number of students reached 2,404 from 72 municipalities of 26 federative units of the country (see Appendix 1 in Oliveira [63, p. 288–9]).

The sampling was carried out by the EDEVO-Darwin Research Nucleus in the first half of 2014, following all the applicable ethical procedures. The contact with school principals included an explanation of how the school was selected, differences from OECD/PISA technical procedures, the voluntary nature of participation, students’ right to withdraw at any time with no penalties, and the maintenance of the confidentiality of school participation and guarantee of student anonymity. School principals demonstrated their accordance with the ethical procedures, sending back the parcel with the questionnaires to the postal address of the research nucleus by ordinary mail, together with a signed letter of informed consent (see Appendix A in Pinafo [64, p. 437–442]). The expected risks and benefits were explained, and mutual advantages were effectively achieved [65]. The database built by the EDEVO-Darwin Research Nucleus was already anonymized, and, in addition, the schools were recoded to prevent identification. Very similar procedures were adopted by Observa, the institution that constructed the Italian database in the same time-frame (see below).

The Italian sample was designed following the stratification model, with random drawings from the universe of secondary schools (n = 2,862), with two explicit stratification variables (geographical area, Grade 2 middle school status), with a total of 18 explicit strata, and two implicit stratification variables (school type, town type). The official list included high schools and enrolled students in Grade 2 of middle school (“secondo anno della Scuola Media di Secondo Grado”), in the year 2013/2014 [66], which corresponds to the same age level of the Brazilian sample.

The sampling in Italy was proportional to the number of students in the grade that year in each one of the geographical areas in which the country is divided, as follows: North-West (25%), North-East (18%), Center (18%), South (27%), and Islands (12%). A random selection of 100 schools followed the above-mentioned proportions. Instead of having the urban/rural school-type stratification, standard sampling procedures adopted in similar surveys in Italy recommend dividing the sample into two parts, drawing half of the schools from the group of regional major towns (“comuni capoluogo”), and the other half from all other municipalities (“territorio provinciale”) [67]. The questionnaires were sent to 103 schools after 14 replacements were randomly selected, having received materials in return from 99 schools from 88 municipalities from all geographical areas of the country, comprising 3503 students. Differently from what happened in Brazil, the procedures for database entry from the paper questionnaires were done manually, with no machine-reading.

In Italy, the questionnaires were delivered to two classrooms for each selected school. The teachers explained to the students the objectives of the ROSE survey in each classroom by specifying how to fill it out and ensuring that the questionnaire was completely anonymous, along with all the other conditions in compliance with Italian legislation at the time (2014) [68]. Subsequently, after a brief introduction to the questionnaire, the students filled it out in paper or in the school computer lab, under teacher supervision, assuring they were working individually. Regulations for research on education in Italy are similar to the Brazilian ones and were followed, with no identification of the students or schools in the public databank, which was merged with the Brazilian one.

The aggregated databank has been used for independent analyses, generating doctoral theses at different institutions and in several publications [46,63,64,69]. Using an anonymous database with public access, the authors did not submit the statistical analysis protocol to an Institutional Research Review Board in advance.

Data collection

The survey took place between March and May 2014 in both countries. Paper-and-pencil questionnaires were used in both countries, as well as electronic forms in some schools in Italy, all under absolute anonymity. The instrument consisted of a section regarding personal data (gender, age, living place), some socioeconomic questions, 73 Likert scale items, and the true/false section on evolution (G Section), which is the focus of the present study (see Annex A in Oliveira [63]).

The students had roughly the same age level in both countries, being around 15 years old. Valid cases comprised only students from 14 to 16 years old. There was a slight difference in the gender proportion in the two countries, as girls slightly outnumbered boys in Brazil (55%) and Italy (52%). The declaration of religion is a sensitive issue, and each country had a different presentation of the question. In Brazil, there was a direct question about religious affiliation (yes/no) and 18 options for answers (Catholic/Orthodox/Evangelical denominations (10 options), Jewish/ Buddhist/African denominations (2 options), Kardecist, and an “other” option). Some 12% of Brazilian students declared not to follow any religion. A high proportion of Brazilian religious students declared to be Catholic (56%); the largest non-Catholic Christian group was the Pentecostal Christians (21%), followed by Mission Evangelicals (10%), which were composed mainly of Baptists (6%) and Seventh-Day Adventists (2%). The number of Orthodox, Lutheran, and Anglican Christians was low, less than 1%. Other religions amounted to some 13% [63].

In the Italian context, this section asked a general question about religion or transcendent philosophy. Two-thirds of the students declared to follow the Catholic religion (67%), and 22% declared not to follow any religion or transcendent philosophy. Of the respondents, 3% were non-Catholic Christians, and roughly the same were “other religions” (3.5%) [48]. Therefore, the total number of Catholics in our sample from the two countries was very high, reaching over 3,000 valid cases. The number of Brazilian non-Catholic Christians was reasonable, reaching 549 valid cases. However, they composed a very conservative group compared to, for instance, the group in Britain who responded to a survey about evolution [24], where the proportion of Anglicans and Lutherans was far higher than Pentecostal Christians and Independent Evangelical Christians, the two main groups in Brazil.

Statistical and numerical analyses

All analyses were performed using the statistical software IBM SPSS Statistics 26. We used Chi-square statistics to determine if there were statistically significant differences between the answers from the two countries, different religious affiliations in the same country, and between Catholics of the two countries. Column proportion tests were also performed, testing whether, for each row, the proportion of respondents in one column was significantly different from the proportion in the other column. For each item, a global significance level of 5% was adopted. As we were working with multiple comparisons, the significance level for each individual test was adjusted according to the Bonferroni correction method [70] to maintain a global significance of 5%. Additional tables with general results can be found in the Supporting Information (S1 and S2 Tables). As the results show very significant differences, we tackled the problem of association among a set of variables using a multivariate method, multiple correspondence analysis (MCA) [71]. Another set of tables was provided, excluding the neutral option, and allowing for an exploratory graph (biplot) to be drawn with the true/false answers from seven items with significant values (G75, G76, G77, G79, G80, G81, and G83) from the samples of the two countries. The reasons to exclude G78, taking its results as outliers, are presented in the Supporting Information (S1 File). For this analysis, the normalization method followed the main variable, using default procedures of IBM SPSS Statistics v.26. Each item was recorded according to the individuals’ nationality, religion, and option (true/false), generating eight labels:

• 1 = BC + (Brazilian Catholic who answered “True”);
• 2 = BC—(Brazilian Catholic who answered “False”);
• 3 = BNC + (Brazilian non-Catholic Christian who answered “True”);
• 4 = BNC—(Brazilian Christian non-Catholic who answered “False”);
• 5 = IC + (Italian Catholic who answered “True”);
• 6 = IC- (Italian Catholic who answered “False”);
• 7 = INC + (non-Catholic Christian Italian who answered “True”);
• 8 = INC- (Italian Christian non-Catholic who answered “False”).

This led to a generalization showing four clear groups of students separated by both religion and nationality. The total number of students in the two countries who declared their religion was 3,944, but 63 had missing values in all seven items and were not considered; therefore, a total of 3,881 cases were processed. S3 Table in the Supporting Information shows the case-processing summary, S4 Table presents the numbers of every religious group in the two countries, and S5 Table shows the degree of association of each item with the two dimensions of the analysis. Supporting Information (S2 File) brings a detailed study of the MCA analysis (S6.1 to S6.10 Table in S2 File), leading to the conclusion that there is no evidence that the MCA results have any significant bias when comparing the responses of valid active cases with those with missing values. In the same file, a link provides access to the databank and the SPSS syntaxes.

We then estimated the differences between the Christian denominations (Catholics and non-Catholics) within the same country (Brazil), such that BCat—BNCat = ΔBChr. In addition, we estimated differences within the same religion (Catholics) between countries (Brazil and Italy), such that ICat—BCat = ΔCat. Thereupon, we created an intercultural index (IntcI) to compare these differences, such that (1)

If the intercultural index was higher than 1 (IntcI>1), this meant that the Brazilian and Italian Catholics had more differences amongst themselves than among Brazilian Catholics and Brazilian non-Catholics Christians. This indicated that sociocultural factors played a more important role in the acceptance of evolution than religion itself. If the index was lower than 1 (IntcI<1) this indicated the opposite, that religion played a major role.