History and overview of conceptual change

While the investigation of conceptual change is not new to the field, there is not yet a unified framework for approaching the study of conceptual change. Many early misconception studies were even devoid of unified theory (for review see [14; 16]). Presently, there are two broad theoretical approaches on knowledge structure that are applied to this area of study: (1) knowledge-as-theory and (2) knowledge-as-pieces [16]. The knowledge-as-theory perspectives are widely influenced by Piaget’s assimilation and accommodation concepts. One of the most widely noted models of conceptual change within the knowledge-as theory-perspective is the Conceptual Change Model (CCM) proposed by Posner and colleagues (1982). According to this model, in order for conceptual change to occur, the following must occur: (1) the learner becomes dissatisfied with their prior knowledge, (2) the new concept is intelligible and plausible to the learner, and (3) the new concept must be fruitful for explanation [13]. Much of the prior research is based on the idea that learners hold an incorrect theory that must be corrected in light of new information (e.g. [1,5,17]). In contrast to the knowledge-as-theory approach is the knowledge-in-pieces approach, in which learners’ understanding is based on a collection of multiple, semi-independent elements of knowledge (e.g. [16,18,19]). According to the knowledge-in-pieces perspective, learners may hold intuitive ideas that do not need to be rejected, but rather can be incorporated into a stronger conceptual factor that is reinforced through learning [18,19].

The knowledge-as-theory and knowledge-in-pieces perspectives both offer insights into the process of conceptual change. However, it is not our intent to put these approaches to the test in the present study. Rather, our goal is to provide an empirical investigation of two factors that have been implicated in conceptual change. The first factor is text structure, particularly the question of whether expository or refutation texts are better for outdating misconceptions. The second factor is the learner’s epistemic beliefs (e.g., beliefs about knowledge, and how changeable it is). These factors have been explored in previous research [20,21,22], but as we discuss below, some questions remain about the impact that each factor has on conceptual change.

Before moving on to a discussion of text structure and epistemic beliefs, we first provide our working definition of conceptual change. For our purposes, conceptual change refers to whether a learner, who previously held a misconception, will successfully acquire new knowledge that refutes the misconception after reading text in support of new knowledge. This view of conceptual change can be operationalized as the difference in performance between a pre- and post-test in the relevant knowledge domain.

Designing optimal text

Successful text comprehension involves constructing a mental representation of the text and continuously updating it as the text unfolds. Mental representations of the text are augmented by prior knowledge, resulting in a model that blends the learner’s existing knowledge with input from the current text [9,10]. One approach to facilitating conceptual change in science is to ensure the text highlights the new concept in a way that the learner recognizes that the misconception is inadequate for explaining scientific phenomena, and that the new ideas are useful for explaining phenomena [13]. It has been argued under the Knowledge Revision Components (KReC) Framework that a key step to encouraging conceptual change (also referred to as knowledge revision in this literature) is making the learner explicitly aware there is a conflict between previously acquired knowledge and new information [23]. The awareness component is critical because the old and new ideas must be co-activated to induce cognitive conflict. Learners must determine what happens in the presence of conflict. In successful cases of conceptual change, learners will integrate the new information into their knowledge structure, thus decreasing activation of the misconception [23].

There evidence suggesting the quality and quantity of explanation matters in order for conceptual change to occur. Initial investigation of the text factors that promote knowledge was in the domain of narrative text comprehension (e.g. [24,25,26,27]). In the inconsistency paradigm, a character is introduced as having a certain behavior (e.g. “Mary, a health nut, has been a strict vegetarian for ten years.”) ([27], p. 1209). At the end of the narrative, however, a critical sentence reveals the character engaging in a behavior that is inconsistent with the behavior described in the beginning of the text (e.g. “Mary ordered a cheeseburger and fries.”) ([27], p. 1209). In between the introduction and the last sentence were the key manipulations of different explanations of the character’s behavior that would possibly make that critical sentence less disruptive to the reader. Findings suggest that causal explanations of character’s behavior lead to less disruption in comprehension compared to qualified explanations, as indexed by lower reading times in the causal explanation condition [25]. The quantity of explanation also makes a difference in how much conceptual change the reader exhibits. Explanations that provide an explanation for why a behavior changed (e.g. “She wasn’t getting enough vitamins because of her diet so her doctor said she had to start eating meat.”) ([25], p. 865) showed less disruption of an old characteristic compared to an explanation that simply outdates the old behavior (e.g. “Recently Mary had stopped worrying about eating nutritious foods and began to eat more meat and fast food.”) ([27], p. 1209). Thus, causal explanations could help learners undergo more conceptual change than qualified explanations because there is more information substantiating why the new idea holds more support than the old one.

There are many different styles of text, but the main ones of interest in the conceptual change literature are refutation and expository text. Expository text is the text style that is typically encountered in textbooks: it disseminates factual knowledge. Refutation text structure is similar to expository text with one notable difference: it explicitly highlights a misconception. Consider an example in which the targeted misconception is “meteors that land on Earth are hot.” Both refutation and expository texts explain that the outer molten layer of a meteor burns off while entering Earth’s atmosphere, and the meteor does not have time to heat up before it lands. The key difference between these texts will be in how this explanation is prefaced. Refutation texts may start with a preface such as, “Kate warned everyone not to touch the meteor because it would be hot and they could get burned. However, Jerry said that they should not worry because it actually should not be hot.” ([28], p. 396). This preface explicitly states that meteors are hot when they land on Earth is a misconception. Expository text, on the other hand, does not explicitly point out the misconception. For example, the preface may be something like this: “Kate was excited and curious because she had never seen a meteor on the ground before. Jerry said he could look up more about meteors in the astrophysics book that he had.” ([28], p. 396).

A number of studies have found that refutation text is better at inducing conceptual change than expository text among school-aged and college students (see [29] for review, [30]). However, a smaller number of studies find the two styles do not differ in inducing conceptual change (e.g. [31,32]). Divergence in these findings can be attributed to differences in structuring the two text styles and different methodologies, such as reading times (e.g. [28,33]), protocol analysis or think alouds (e.g. [34,35,36]), and eye tracking (e.g. [32,37]). Despite the conflicting evidence, the prevailing view in the field remains that refutation texts are better for conceptual change than expository text for students from elementary school through university (e.g. [19,38]). The main argument is when refutation text has its optimal characteristics of including an explicit statement that a misconception is false, an explicit statement that the new idea is true, and there is sufficient supporting evidence favoring the new idea (e.g. a substantive causal explanation), a reader will exhibit more conceptual change than when reading an expository text (e.g. [23,28]). While refutation text could help facilitate conceptual change, additional reader characteristics, such as epistemic beliefs could illuminate key factors that are important for conceptual change. We review this topic in the next section.

Role of epistemic beliefs

Even with the best-designed text structure, conceptual change may also depend on the learner’s ability to engage in conceptual change. Epistemic beliefs–individual beliefs about the nature of knowledge and the process of knowing—are one of the more interesting characteristics (e.g. [21,39,40,41]). There are many different ways to approach and classify epistemic beliefs. For instance, there are the terms “naïve—sophisticated” (e.g. [42]) and “less advanced—more advanced” (e.g. [32]). Naïve and sophisticated refer to the extent to which a learner believes knowledge is stable versus subject to change. For example, a learner with more sophisticated epistemic beliefs often perceive knowledge as tentative and subject to change whereas a learner with naïve epistemic beliefs tend to perceive knowledge as stable and unchanging [42].

It has been noted that students with more naïve epistemic beliefs have faced more challenges to learning than students with complex epistemic beliefs (see [22] for review). Previous studies have noted that students with more advanced epistemic beliefs exhibited more conceptual change at the immediate posttest compared to students who had less advanced epistemic beliefs, which is consistent with other findings on this relation [20,43,44,45]. An investigation of the interaction of text with epistemic beliefs showed that among students reading refutation text, those with more advanced epistemic beliefs demonstrated more conceptual change than students with less advanced epistemic beliefs [20]. This interaction, however, was not observed with expository text: there were no differences in conceptual change between students with more advanced epistemic beliefs compared to those with less advanced epistemic beliefs. These patterns persisted into the delayed posttest, even though the scores were overall lower than the immediate posttest [20].

Despite this common finding in the literature on conceptual change and epistemic beliefs, it is important to note that it may not always be the case that having more sophisticated epistemic beliefs will lead to an advantage in learning, or even conceptual change, compared to having naïve epistemic beliefs. The view of naïve—sophisticated, or certain—uncertain, epistemic beliefs has been argued to be insufficient as the context for learning itself could modify when it would be more appropriate to have naïve (or certain) or sophisticated (or tentative) epistemic beliefs. Elby and Hammer (2001) point out that it is not correct to broadly apply the principle knowledge is tentative to every concept, as the context for learning may call for an approach that knowledge is certain. For example, when elementary school students learn the Earth is flat, it is neither correct nor productive for them to approach knowledge as tentative since the Earth being round is a verifiable fact. Thus, it is more productive for students to accept that, in this instance, knowledge is certain and learn the fact the Earth is round. For theories of dinosaur extinction, on the other hand, it would be more productive for students to take a tentative approach to knowledge as that information is not definitive and subject to revision, especially when new scientific evidence can present evidence that conflicts with previous knowledge in the field [42].

Some research suggests that aligning text to learner’s epistemic beliefs may be more beneficial to conceptual change than just having sophisticated epistemic beliefs. Franco and colleagues (2012) found that learners with a rational approach to epistemic beliefs (e.g. belief that knowledge is derived with reason and logic) exhibited more conceptual change when given a text with a rational approach compared to a metaphorical approach (e.g. belief that knowledge is derived via intuition). Similarly, learners with a metaphorical approach to epistemic beliefs exhibited more conceptual change when reading text with a metaphorical approach compared to a rational approach [21]. People’s current topic domain knowledge may also play a role in their epistemology. For instance, someone with little knowledge about physics in general may perceive the field as precise due to the methods used to measure physical quantities. On the other hand, someone with extensive knowledge about research in physics may perceive the field as imprecise because various methods are still in development [46]. Furthermore, there are also ontological differences in epistemology depending on the discipline examined. For example, knowledge in math and science is often considered to be more certain than knowledge in psychology or history because of the different ways with which knowledge is dealt across disciplines [46]. Thus, optimal learning in different disciplines may require less advanced epistemic beliefs compared to others. For example, a learner understanding mathematical processes would benefit more from less advanced epistemic beliefs because it would be more productive to learn the process rather than be open to alternatives that could lead to inaccurate answers. On the other hand for history, new archaeological and anthropological evidence may present new information about a historical time period that conflicts with previous knowledge about that era; therefore, it would be more fruitful for the learner to understand that knowledge in history is uncertain and be more adaptable to learning the new information.

While there are different approaches to defining epistemic beliefs, in the present work, we will refer to epistemic beliefs on the spectrum of less advanced to more advanced (see [20] for similar approach to classifying epistemic beliefs). As epistemic beliefs were measured using the Connotative Aspects of Epistemic Beliefs questionnaire [47], we were tapping into two different aspects of epistemic beliefs: texture and variability. Texture refers to the extent to which learners believe knowledge to be unstructured and vague. Variability refers to the extent to which students believe knowledge to be flexible and dynamic. Following the guidelines from previous studies (e.g. [20,47]), in this study, more advance epistemic beliefs will refer to higher scores in texture while less advance epistemic beliefs will refer to higher scores in variability. Our understanding of the relationship between individual characteristics of the learner and the nature of the learning materials (such as the text that is being read) is still emerging and requires further investigation [20,21,22,43,44,45]. Thus, while epistemic beliefs appear to play a role in conceptual change, its role is still not well understood. Furthermore, it is important to understand the extent to which having more or less sophisticated/advanced epistemic beliefs helps or hinders the learner from undergoing conceptual change in different disciplines.

Present study

The existing literature points to the importance of text type and epistemic beliefs in promoting conceptual change. However, there is conflicting evidence as to whether or not refutation text is better at inducing conceptual change in readers compared to expository text. Much of this evidence comes from studies with between subjects designs (e.g. [20,37,40,48]), which limits insight as to how individual readers may approach the different styles of text, and subsequently, how much they learn from them. This study aims to replicate previous studies investigating how text style and epistemic beliefs influence conceptual change (e.g. [20]), and also extend those findings by conducting a within subjects investigation to determine if an individual reader exhibits different amounts of conceptual change after reading a refutation text versus an expository text.

The primary aim of this study is to investigate whether and how the interaction of text style (refutation versus expository) and epistemic beliefs (ranging from less advanced to more advanced, per the guidelines suggested in Mason and Gava, 2007 [20] for describing the continuum of epistemic beliefs) impacts conceptual change in different subject areas. If text style and epistemic beliefs play a role in conceptual change, then it is expected that refutation text will induce more conceptual change than expository text. Additionally, learners who score higher in texture and variability will exhibit more conceptual change than learners who score lower in texture and variability.

Preregistration disclosure

The study was preregistered on Open Science Framework (OSF): osf.io/msq7w. The methods and model described below are in accordance with the preregistration. Any divergence or elaboration from the preregistration on OSF is noted. The data from this study is available on the OSF website.

Participants

The Florida State University Office of the Vice President for Research Human Subjects Committee approved this study on 6/8/2017 (Approval number: 2017.21233) and renewed it on 3/21/2018 (Approval number: 2018.23500). Consent was obtained through a written form requiring the participant's signature before starting the study. One hundred and seventy-two college-aged participants were recruited from Florida State University’s Department of Psychology and were compensated with course credit for participation (M = 19.28 years, male = 37). 157 participants completed both sessions and were included in the final sample. This sample size was generated following the sequential sampling guidelines described in Frick (1998), which allow researchers to be flexible and efficient in their sample sizes while also controlling for Type I errors [49]. We proposed an initial target sample size of 100 college-aged adults. Once this sample sized was reached, we did preliminary data analyses to determine whether the critical effects (involving text type and epistemic beliefs) had p-values greater than .35 or less than .01 (e.g., Frick’s stopping rule). As the effects did not meet this criterion, we continued data collection toward our pre-registered maximum sample size of 150. We collected data from more than 150 participants in an effort to ensure that our final sample had more than 150 participants once participant attrition was taken into account.

Text

Following a similar procedure outlined by Lassonde and colleagues [50], refutation and expository texts were written for each topic (see S1 File for example). Five disciplines were included (physics, astronomy, genetics, economics, and geography) and each discipline had two topics that were tested (see Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. List of disciplines and topics assessed in the study.

https://doi.org/10.1371/journal.pone.0220766.t001

All texts contained the following sections: introduction, premise, explanation, and conclusion. The key difference between the refutation and expository versions of the texts for each topic was the premise section. The premise section for the refutation text explicitly stated the misconception and noted the misconception is incorrect (see example below).

Many people think red stars are hotter than blue stars because red is often associated with fire and other notably hot surfaces or objects. Astronomers, however, have found this idea to be inaccurate. Quantitative measures of starlight color and stellar temperature reveal that blue stars are actually the hottest stars while red stars are the coolest stars.

The premise section for the expository text presented further information about the topic, but does not explicitly state the misconception (see example below).

Whether or not blue stars or red stars are the hottest is not that clear to the casual stargazer. Close investigation of the visible light spectrum of stars, however, does reveal vast temperature differences. Astronomers have devised measures for quantifying the colors of light the stars give off and then using those colors to determine stellar temperatures.

We carefully controlled the length of each text and its subsections. Across all texts, there was a range of 346–349 words. The premise section for each version of each text was within one word of each other. The introduction, explanation, and conclusion sections were exactly the same length for each text version. We also ran the Flesch Reading Ease and Flesch-Kincaid Grade Level tests to obtain information about the reading level of each text (see Table 2 for results). Higher Flesch Reading Ease scores indicate the text is easier to read [51], while higher Flesch-Kincaid Grade scores indicate the text is appropriate for the given numerical grade level in the U.S. [52]. It is generally recommended that most standard text in the U.S. should score between 60–70. Overall, the text ranged from 10^th grade to first year of college reading level and the texts were generally more difficult read.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results from the Flesch Reading Ease and Flesch-Kincaid Grade Level tests.

https://doi.org/10.1371/journal.pone.0220766.t002

Experiment measures

Procedure

This study was a two-part, within-subjects design. Participants were informed in advance they would be participating in a two-session study with each session taking place approximately one week apart, lasting about one hour each. Time lag between sessions ranged from 5–9 days. During the first session, participants completed the CAEB scales, completed a pretest on one of the topics from each of the five disciplines, read text concerning the topics, and completed an immediate posttest. All tasks were completed on a Qualtrics survey that was saved on lab computers and links were not externally shared or available to anyone aside from the principal investigator. To avoid carryover effects of text style, participants read either (1) expository or (2) refutation text during the first session.

The second session was completed in a separate room in the lab, and participants read text on different topics from the same five disciplines, but in style they did not read during the first session. The second session had a similar structure with participants completing a pretest on different topics from the first session, reading text, and completing an immediate posttest on a Qualtrics survey. Participants would complete a pretest, text reading, and posttest per discipline, one at a time. It is important to note in this within-subjects design, participants read text in a style that was different from style they read in the first session (e.g. if they read expository previously in session 1, they would read refutation text in session 2). The order of topics tested was randomized across participants and order of text style was counterbalanced across participants.

Data coding

We coded our responses as correct (1) or incorrect (0). While we declared in our preregistration guidelines that missing items would be scored as incorrect, further examination of the data suggested that missing responses were largely a result of participants not finishing the task rather than skipping items themselves. In light of that finding, we report the models with missing responses removed. Items with missing responses were removed from analysis (100 items; 4.73% data loss).

The dependent measure for the study was response accuracy, which was dichotomously coded as either correct or incorrect at the item-level (1 = correct, 0 = incorrect). Each question included in this study was a multiple-choice question pulled from preexisting inventories, with the exception of geography which was developed by the primary author (see Experiment Materials, Pretests and Posttests for further information). Each came with an answer key (or one was developed by the primary author for the geography items). The answer keys noted which answers correct for each question. If the selected answer was not the correct one, it was coded as incorrect.

The following independent variables were also included in analysis: text type, test time, and epistemic beliefs. Text type was contrast coded for expository (-1) and refutation (1) text. Test time was also contrast coded for pretest (-1) and posttest (1). The two-factors for epistemic beliefs, texture and variability, were included as continuous variables. Both texture (α = 0.76) and variability (α = 0.38) were centered around their means prior to analysis.

Data analysis

We ran descriptive statistics and reliabilities for the pretest and posttest for each topic areas as well as for texture and variability. Results can be seen in Table 2. The main statistical analysis comprised a mixed models logistic regression predicting the log odds of correctly answering a test question. Conceptual change would be demonstrated by a change in the log odds of answering a question correctly between the pre-test and post-test. The model included the following predictors (and interactions between these predictors): text type (refutation vs. expository), time (pretest vs. posttest), and texture and variability (continuous predictors). We expected that participants would generate more correct answers on the post-test (i.e., show more conceptual change) when they read refutation texts and score higher in texture and lower in variability than when they read expository text and scored lower in texture and higher in variability. The model also included participants and items as crossed random factors. The preregistration specified that we would run models with the full complement of random slopes and remove random slopes if necessary, to get the model to converge. Because we did not specify a method for removing random slopes, we decided use the model-fitting approach to compute a model with the best fitting random slopes.