Development and validation of an instrument to measure student knowledge gains for chemical and physical change for grades 6–8

Abstract

Teachers tend to instruct topically, which requires the student to use multiple and interconnected concepts to succeed in each instructional unit. Therefore, it is beneficial to combine research on related concepts to form topic driven instruments to better assist teachers in assessing and instructing students. Chemical and physical change as a topic is often students' first introduction to chemical reactions; however, understanding the distinction is contingent upon a host of advanced ideas. As such, chemical and physical change is best evaluated with a strong emphasis on particle properties and behavior. To date, there was no instrument specifically designed for grades 6–8 (ages 11–14) that focused on the topic of chemical and physical change including its underlying conceptions such as the particulate nature of matter. By making use of well-designed and tested items from previously published research and item repositories, the chemical and physical change assessment (CPCA) was designed to translate findings from research to a practical and classroom-ready assessment. The CPCA went through two versions to result in a 19-item assessment that has generated valid and reliable data to date. Revisions to the assessment were driven by psychometric properties of the items as well as student validation interviews.

---

Background

Chemistry is generally thought of as a course for either secondary or college level students. While it is true that there is little explicit chemistry instruction in grades 6–8, the framework supporting the academic content standards in the U.S. provide a gateway to chemistry concepts. The conceptual framework for the new K-12 science education standards in the U.S. contains Matter and Its Interactions as the first core idea in the physical sciences. The expectation for students by the end of grade 8 (ages 13–14) is to know that substances are composed of atoms, the difference between gases, liquids, and solids at the submicroscopic level, and understand that chemical processes result in a change of atomic connections producing new substances (National Research Council, 2011). These expectations require core content knowledge about the particulate nature of matter (PNM), which is fundamental to the study of chemistry.

The instructional topic of chemical and physical change is an area where Matter and Its Interactions conceptual framework is highlighted. The U.S. focuses on the nature of chemical and physical change in grades 6–8, ages 11–14 (McREL, 2012). The national curriculum from the United Kingdom places the content of physical and chemical change to be administered by the end of Key Stage 2, ages 7–11 (Department for Education, 2011). Finland begins to address the properties of substances in grades 5–6, ages 11–13 (Finnish National Board of Education, 2004). The key to successfully understanding chemical change (and later chemical reactions) is rooted in the concept of the particulate nature of matter (Hesse and Anderson, 1992; Ahtee and Varjola, 1998; Johnson, 2000). For this reason, chemical and physical change is best evaluated with a strong emphasis on particle properties and behavior. This can encourage a deeper conceptual understanding and discourage memorizing classic examples of chemical and physical change such as burning paper and tearing paper, respectively.

While there are methods to teach without the notion of the particle concept, strong understanding of matter and change is only possible with the application of particulate models (Barke et al., 2009). Evaluating students' understanding of chemical and physical change emphasizing particle behavior is not a simple task, as misconceptions, lack of understanding, and student application about the fundamental ideas of the particulate nature of matter are widespread (Novick and Nussbaum, 1978, 1981; Osborne and Cosgrove, 1983; Gabel et al., 1987; De Vos and Verdonk, 1987; Griffiths and Preston, 1992; Abraham et al., 1994; Garnett et al., 1995; Harrison and Treagust, 2000; Eilam, 2004; Barke et al., 2009). In Andersson’s (1990) literature review of pupils' conceptions of matter and its transformations he highlighted the largest difficulty with correctly understanding chemical and physical changes is students' view of matter as continuous and static. Johnson (2000) discovered a strong tendency of children relying on the history and origin of a substance in order to describe products from chemical changes in terms of a mixture of reactants rather than a new substance.

Even with the knowledge of atoms and molecules, grade 8 students have difficulty applying these concepts consistently (Nakhleh et al., 2005). This theme perpetuates into high school where students fail to use atoms and molecules in explanations of chemical changes and rely on analogies rather than chemical principals. Consequently, students struggle to distinguish between chemical and physical changes as well as understanding the role of gaseous substances in reactions (Hesse and Anderson, 1992). The gas phase in particular has spurred confusion among students (Stavy, 1988; Johnson, 1998a, 1998b; Nussbaum, 2009). Moreover, familiar everyday occurrences such as the three phases of water come with their share of misconceptions (Osborne and Cosgrove, 1983), making gas properties and behavior prominent in the treatment of chemical and physical changes.

Due to the high level of difficulty with understanding the PNM, it may be suggested that the idea of particles is only appropriate to teach to older students. However, it is just not the abstract nature of PNM that causes difficulty, but also students' scientifically inaccurate ideas about matter and epistemological understanding of scientific models (Wiser and Smith, 2008). Thus, the true challenge comes with determining “when is learning about atoms part of the problem, and when is it part of the solution?” (Wiser and Smith, 2008, p. 205). Wiser and Smith advocate that by having students learn about atoms and molecules early (around ages 10–13), it will prove vital in connecting grounded macroscopic understandings about matter while providing the framework for future development of scientific ideas (2008). In support, it has been shown that students are capable of understanding abstract phenomena such as the PNM as early as grade 2 (ages 7–8) when supplied with high quality science instruction (Novak and Musonda, 1991). Therefore, while addressing chemical and physical change in grades 6–8 is challenging, the reality is that this topic remains imbedded in the curriculum for this age group. Introduction to chemical and physical change at this level can be framed as a critical transition between the ideas of matter accumulated from life experiences' and ideas presented in formal classroom instruction that challenge such naïve ideas. The value of the topic of chemical and physical change not only lies in the opportunity to formally introduce students to chemistry, but also to assess their prior knowledge of the structure and properties of matter. This assessment task requires a tool that can measure all of the prerequisite ideas to understanding chemical and physical change.

Learning progressions (LPs) are a useful guide for examining the development of student ideas in order of increasing complexity. They can have an influence on guiding instruction as well as assessment practices and tools. To aide in using LPs, one method is to fashion a construct map, which provides a framework for assessment design and has the benefit of yielding an assessment that can measure progress at designated levels (Wilson, 2009). A construct map is less complex than a LP, and is based on a well-researched ordering of different levels of performance relating to one component.

Study purpose

Constructivism pinpoints how learners do not simply echo what they hear, but rather construct meaning based on their understanding (von Glasersfield, 1984). This learning theory emphasizes how each individual has a unique interpretation of events. For this reason, the most critical aspect to learning is what the learner already understands, because it is within this framework that new information is processed (Ausubel, 1968). Unfortunately, the prior knowledge that a learner possesses is not always correct. Research in science education is rich with investigations of students' ideas, most focused on a specific science concept (Duit, 2009). However, teachers tend to instruct topically, which requires the student to use multiple and interconnected concepts to succeed in each instructional unit. For example mastering the topic of chemical and physical change requires understanding underlying concepts related to the particulate nature of matter, characteristics of matter, and characteristics of atoms/molecules (Fig. 1). Therefore, it is beneficial to combine research on related concepts to form topic driven instruments to better assist teachers in assessing and instructing students.

Fig. 1 Topic of chemical and physical change with underlying concepts.

To date, there was no instrument specifically designed for grades 6–8 focused on the topic of chemical and physical change including its underlying conceptions. Previous research has predominately focused on qualitative data or instruments designed with a more narrow focus. By making use of well-designed and tested items from previously published research and item repositories, the chemical and physical change assessment (CPCA) was designed to translate findings from research to a practical and classroom-ready assessment. Due to the popularity of classroom examples with water and the numerous published misconceptions concerning the gas phase, a special emphasis on the gas phase of water was added in the construction of the CPCA.

The goal of this study was to develop a paper and pencil instrument to evaluate concept learning of chemical and physical change for students in grades 6 (ages 11–12) to directly address the topic of chemical and physical change in the curriculum as well as contribute to a larger study with a grade 6 target population. To accomplish this, the following research questions were addressed: (1) What are the psychometric properties of the instrument in the 6th grade population? (2) How can revisions guided by student interview data improve the psychometric properties of the instrument? (3) How valid and reliable are the data generated from the instrument? Because the main criterion of student item validity was in language usage and reading comprehension, this assessment could extend to grade 7 and 8 (ages 12–14), but would require further evaluation.

Methods

Setting and sample

This study was conducted at a middle school located in the midwest United States. The school's student population is 4% non-white and 23% economically disadvantaged (Great Schools, 2008). The testing population was limited to 6th grade students (ages 11–12). Since the participants were minors, consent for collecting both quantitative and qualitative data was obtained from the parent/guardian.

Two teachers' classes were used in the study. For the pilot in spring 2011, 67 students completed the α-version of the CPCA. From this group, 14 students were interviewed to further inform revisions on the CPCA. The study was conducted in the following academic year in fall 2011 with the revised β-version of the CPCA. The same teachers were approached generating a total of 145 students who completed the CPCA β-version.

Students took the CPCA during normal class time and were allowed unlimited time to complete the assessment. The 25-item α-version took approximately 20 min to complete, whereas the 20-item β-version took approximately 16 min to complete.

Instrument development (α-version)

Several resources contain individual items that either address the topic of chemical and physical change and/or highlight its subordinate concepts. These items can be found in published science education research articles and item repositories. Similar to a construct map, Smith et al. organize the LP of matter and atomic-molecular theory around three key questions with answers at differing levels of complexity by grade level (2006). Due to the overlap between the chemical and physical change curriculum discussed earlier and the ideas in Smith's LP, it was useful to use the 6–8 level in the LP as a basis for CPCA item selection. Refer to supplementary materials for an overview of LP for matter and atomic-molecular theory from Smith et al. mapped onto items (including distractors) of the CPCA (2006).† An internet search was conducted and 24 items were gathered to form the CPCA α-version (AAAS; Ardac and Akaygun, 2004; BouJaoude, 1991; MOSART; Osborne and Cosgrove, 1983; Yezierski and Birk, 2006). The items were chosen based on their intended grade level, relation to chemical and physical change, and breadth of question types. CPCA α-version consisted of 5 true and false, 19 multiple choice, and 1 free response item for a total of 25 questions. The free response item was not drawn from a published resource, but instead was well-connected to classroom curriculum material. While previously developed items provided a starting point, the individual items were originally developed with an age range of kindergarten through college, depending on the respective resource. Therefore, the CPCA α-version was evaluated for reliability and validity with the targeted population. The collected data informed minor revisions resulting in a 20-item CPCA β-version composed of 19 multiple choice and 1 free response item.

Instrument development (β-version)

Quantitative and qualitative data informed revisions to the CPCA α-version. The quantitative data consisted of outputs based on classical test theory: item difficulty and discrimination, and measure of internal consistency. The qualitative data consisted of student interviews which took place within eleven days of assessment administration. The semi-structured interviews lasted for approximately 15 min and were think out-loud style. Each interviewee was asked to describe his/her thought process for an item as well as interpret all answer choices focusing on language and comprehension for at least 10 items from the CPCA α-version. The items for each interview were chosen to provide each participant with variety of items initially scored as correct and incorrect while ensuring that each item on the assessment received multiple interpretations by different students. In addition, administration observations were taken into account during the revision process.

Results and discussion

Quantitative

Items on the CPCA where either marked as correct or incorrect with no partial credit awarded. A rubric was used to score the free response item and yielded 78% agreement between two raters. Teacher equivalence was determined for both the pilot and study groups using two one-tailed t-tests with an interval of (1, −1) (Lewis and Lewis, 2005). The two teachers' students were equivalent only for the research group, justifying grouping of students for analysis for only the 20-item CPCA β-version. However, since changes between groups were not a focus or concern for the development of the instrument, all students were combined in analyzing the α-version as well.

The CPCA α-version with a maximum possible score of 25 was administered to 67 students. The distribution of scores was non-normal with a mean of 14 ± 4 and a range of 7–25. The assessment had a Cronbach's alpha of 0.736 indicating acceptable internal consistency (Pallant, 2007). A plot of item discrimination versus item difficulty was generated for the CPCA α-version (Fig. 2). The ideal instrument contains items that both discriminate well (above 0.3) and have a difficulty between 0.2 and 0.8. Three true and false items (α-items 1, 2, and 5) failed to discriminate well, with two items (α-items 1 and 5) resulting in more than 80% of the sample having the correct response. Furthermore, a Spearman's Rho correlation showed that four of the five true and false items were significantly correlated with other items on the CPCA. These results as well as administrative trouble with students skipping the true and false section or asking for clarification as to why the quiz started with number 6, resulted in the true and false section to be removed. Perhaps the true and false section's format looked too much like a paragraph of directions that was systematically skipped over by the participants, generating the confusion. After removal of the true and false section (items 1–5) the Cronbach's alpha for the CPCA α-version remained acceptable at 0.723 and the total scores still had a non-normal distribution with a shift of the mean 11 ± 4, now with a possible maximum score of 20.

Fig. 2 Items with difficulty below 0.2 are said to be hard, while above 0.8 are said to be easy. Items that discriminate above 0.3 are said to discriminate well (Popham, 2005).

On the other extreme, α-item 9 (difficulty (P) 0.15, discrimination (D) 0.111) showed poor discrimination as well as proving to be a very difficult item with only 15% of the sample answering the item correctly. This was not surprising to the authors, as the item addresses a very prevalent misconception about the composition of bubbles in boiling water. Therefore, no change was made to item 9 based on the quantitative results. The other two multiple-choice items that initially raised concern were α-item 22 (P 0.49, D 0.278) and α-item 16 (P 0.85, D 0.222). Item 22 relied on students' interpretation of visual diagrams, promoting the need for qualitative interviews to diagnose item performance. Item 16 fell out of the ideal range due to its easy nature resulting in both low and high student performers answering the item correctly. Since the item deals with the relative spacing of particles in the three phases of matter, an important aspect of understanding PNM, the item was retained for the β-version of the CPCA.

Qualitative

The student interviews guided minor revisions to 5 items' stems, 13 answer choices, and a handful of formatting changes (Table 1). The nature of the changes ranged from simple clarifications to reorganizing item stems to reduce misinterpretations. For example, consider one student's ideas about α-version item number 19 about molecular motion in a cup of liquid water (Table 1). The student eventually decides on answer choice A as the best possible option for the question highlighting a misconception that molecules in the solid state are motionless. The student understands that molecules in the liquid state are in motion, but was too caught up in the language of the item to notice answer choice D as the correct answer. This was likely fundamentally a result of her beliefs about solid state, however, the language and answer choices were clarified so that the stem of the question would not appear suggestive. Only one item required major revisions due to misinterpretation by the students (α-item 21). The item addresses the process of burning, a popular example of chemical change. In an effort to retain this concept on the CPCA, the expressed student opinions from the interviews were used to rewrite the item into β-item 16. Due to the major changes, β-item 16 was reevaluated by additional student interviews after the administration of the CPCA β-version.

Table 1 CPCA α-version's items with representative sample of supporting data for revised β-version

α Items		Supporting data	β Items^a
a All changes are italicised and correct choices are bolded.
True False Section 1. All substances are composed of atoms. T 2. An atom of copper is bendable. F 3. During phase changes matter can disappear. F 4. Evaporation of perfume is a chemical change. F 5. Burning of wood is a chemical reaction. T		Administrative trouble with students unfamiliar with how to mark answer and/or skipping section entirely. 4 of the 5 items correlated with other items on the CPCA. Three items have poor discrimination. Omission of true and false section does not affect total score distribution or internal consistency of the CPCA.	Section Omitted
6. Which is an example for a physical change?  (a) Baking a cake  (b) Spoiling of milk  (c) Formation of clouds by evaporation  (d) Digestion of food in the stomach		“D… There is acid in your stomach so it breaks down the food, so it can fit through your intestines, so that’s a chemical reaction.” [… read the question again] “Oh, physical”	1. Which is an example for a physical change?  (a) Baking a cake  (b) Spoiling of milk  (c) Formation of clouds by evaporation  (d) Digestion of food in the stomach
7. Which is an example of a chemical change?  (a) Breaking bread into pieces  (b) Frying an egg  (c) Bending a metal wire  (d) Melting of candle wax		“I said melting of candle wax. Because when you burn when you light a candle the wax melts and then it burns then it kind of turns into smoke, I’m not sure exactly what it turns into, I think it is just carbon dioxide”	2. Which is an example of a chemical change?  (a) Breaking bread into pieces  (b) Frying an egg  (c) Bending a metal wire  (d) Melting a green crayon
8. Trumpets are made of brass. A clean brass rod is made mostly of two metals, copper and zinc. If you cut the rod in half and looked at the newly cut end, it will look like:		“It said you would cut the trumpet, but I didn’t know where you would cut it. On the end, or… I don’t play instruments so I don’t know what the rod is” ----- “Brass on the outside, it’d be that you’d see brass on the outside, but there’d be a different thing on the inside. So, it’d be like copper on the inside or something. That’s basically what A means”	3. A clean brass rod is made mostly of two metals, copper and zinc. If you cut the rod in half what would the newly cut end look like?  (a)Identical to the brass outside  (b) Bits of copper and bits of zinc  (c) Copper  (d) Zinc  (e) None of the above
(a) The brass outside  (b) Bits of copper and bits of zinc  (c) Copper  (d) Zinc  (e) None of the above
9. A pot of water is placed on a hot stove. Small bubbles begin to appear at the bottom of the pot. The bubbles rise to the surface of the water and seem to pop or disappear. What are the bubbles made of?  (a) Heat  (b) Air  (c) Gaseous oxygen and hydrogen  (d) Gaseous water  (e) None of the above		“And then gaseous oxygen and hydrogen, it’s pretty much gaseous water, it’s both of them. So if there are two answers that are exactly the same thing, it can’t be either, because then there would be two answers” [so when you see answer choice C, are you seeing oxygen and hydrogen connected OR are you seeing oxygen and hydrogen as separated] “I’m seeing them together, like water”	4. A pot of water is placed on a hot stove. Small bubbles begin to appear at the bottom of the pot. The bubbles rise to the surface of the water and seem to pop or disappear. What are the bubbles made of?  (a) Heat  (b) Air  (c) Oxygen gas and hydrogen gas  (d)Water vapor  (e) None of the above
13. You spill a little water on a tile floor but don’t have time to wipe it up. A few hours later, most of the water is gone. What happened to the water?  (a) The water molecules were destroyed  (b) The water molecules got smaller and now take up less space  (c) The water molecules became a gas and are now part of the air  (d) The water molecules broke down into hydrogen and oxygen atoms, which are now in the air		“…. I would be between C and D. I would say that it is not C because I don’t think it becomes part of the air. But in D it says in the air. And I think it is more in the air than part of the air.”	8. You spill a little water on a tile floor but don’t have time to wipe it up. A few hours later, most of the water is gone. What happened to the water?  (a) The water molecules were destroyed  (b) The water molecules got smaller and now take up less space  (c) The water molecules became a gas and are now in the air  (d) The water molecules broke down into hydrogen and oxygen atoms, which are now in the air
15. Which of the following will make water molecules larger?		Grammar Edit	10. Which of the following will make a water molecule larger?
(a) Freezing  (b) Melting  (c) Evaporation  (d) Condensation  (e) None of the above			(a) Freezing  (b) Melting  (c) Evaporation  (d) Condensation  (e) None of the above
17. Which statement is correct for combustion (burning) reactions?  (a) Oxygen (O2) is needed for combustion to take place  (b) All substances can burn  (c) There will be no new substance formed from burning  (d) Carbon dioxide (CO2) is needed for combustion to take place		“Now, I it combustion, is a word don’t really use that much. So, burning, burning rather does help to figure out what it is” Overall students were highly attracted to oxygen being contained in the right answer. Therefore, it was buried more amongst the other distractors.	12. Which statement is correctfor combustion (burning) reactions?  (a) All substances can burn  (b) Oxygen (O2) is needed for combustion to take place  (c) There will be no new substance formed from burning  (d) Carbon dioxide (CO2) is needed for combustion to take place
19. In a cup of liquid water, when would the water molecules stop moving?  (a) The molecules would stop moving if the liquid water in the cup became a solid  (b) The molecules would stop moving if the liquid water in the cup became a gas  (c) The molecules would stop moving if the liquid water in the cup became still  (d) The molecules would not stop moving in the cup of liquid water		“Well, if they’d do stop, well the molecules would not stop in a cup of liquid water, that’s just kind of restating the question mostly, because no they wouldn’t stop moving in liquid water, but it asks when they would stop moving, so that’s like, not even close to correct …D is the only one that could catch people if they didn’t read it… Because it says they would not stop moving in a cup of liquid water. It is the right answer! Just not that for that question. It’s true, but that’s not what the question is asking. So if you didn’t read that, that would be an answer. But that’s just something someone could skip over cause they weren’t reading.” Student chose A as her answer	14. Suppose you have a cup of liquid water, which is TRUE?  (a) The molecules would stop moving if the liquid water in the cup became a solid  (b) The molecules would stop moving if the liquid water in the cup became a gas  (c) The molecules would stop moving if the cup became still  (d)The molecules would not stop moving in the cup
21. When wood burns ashes are formed. What is true about the ashes?  (a) Ashes formed weigh the same as the wood  (b) Ashes absorbs oxygen and weigh more than the wood  (c) Ashes formed weigh less than the wood  (d) Ashes have no weight		“Think it’s C because, because when the wood, it’s like, the ashes, um, like there’s smaller, there’s air in the wood, so some of that goes out when it burns” ----- “It would be kind of like, like getting littler, then it’s like after 10 min or something, it would just be sitting it there with little pieces of dust and stuff… it would weigh less”	16. When wood burns ashes are formed. The remaining ashes weigh less than the starting wood. There is a loss of weight because:  (a) Energy was released by the process of burning, causing the remaining ashes to weigh less  (b) Some of the carbon from the wood reacts with oxygen gas and is released as carbon dioxide gas into the atmosphere  (c) Air pockets in the wood made the wood heavy and after burning, air was released  (d) Ashes are formed that are thinner and smaller than the wood
22. Four experiments were conducted. Which experiment shows a physical change?		Okay. Are these two solutions in B are they the same type or are they different types of solutions? “Um, I think they are the same type of solution cause there is no, real change” ----- Do you see colorless solution A in answer choice A and colorless solution A in answer choice B, being the same or different? “Same” What about, the B’s? “Oh.. uh,” Same or different? “Different. They have to be different”	17. Four experiments were conducted. Which experiment shows a physical change?
23. If you were to hammer some gold into a thin sheet, the atoms:  (a) Would each flatten out  (b) Weigh less  (c) Are pushed closer together  (d) Are unchanged		“They’re like pushing more atoms from the gold into the…wait, is that a sheet of paper?”	18. If you were to hammer some gold into a thin gold metal sheet, the atoms:  (a) Would each flatten out  (b) Weigh less  (c) Are pushed closer together  (d) Are unchanged
24. When liquid water is changed to a gas, it is changed to:  (a) Hydrogen and oxygen  (b) Hydrogen only  (c) Gaseous water  (d) Air, hydrogen, and oxygen  (e) Oxygen only		“And it’d stay [hydrogen and oxygen atoms with water] that but I don’t if that would be confusing to some people because it makes it sound like the “to” it’d be something else with it” ----- “Gaseous water would be hydrogen and oxygen, but hydrogen and oxygen is like breaking it down more explaining it I guess you could say…, A would be explaining it better as to what is in the water… They [hydrogen and oxygen atoms] all would be connected still”	19. When liquid water is changed into a gas, it is changed into:  (a) Hydrogen gas and oxygen gas  (b) Hydrogen gas only  (c) Gaseous water  (d) Air, hydrogen gas, and oxygen gas  (e) Oxygen gas only

---

CPCA β-version. The CPCA β-version with a maximum possible score of 20 was administered to 145 students. The distribution of scores was normal as evaluated by a Kolmogrov-Smirnov distribution test (p = 0.225) with a mean of 9 ± 3 and a range of 2–15. The assessment had a Cronbach's alpha of 0.597. It should be noted that the α-version was piloted post instruction while the β-version was administered pre instruction due to its use as a pre-assessment for an intervention for another study. The lack of instruction no doubt contributed to the lower total score performance on the CPCA β-version versus the α-version. A plot of item discrimination versus item difficulty was generated for the CPCA β-version (Fig. 3).

Fig. 3 Items with difficulty below 0.2 are said to be hard, while above 0.8 are said to be easy. Items that discriminate above 0.3 are said to discriminate well (Popham, 2005).

Two items identified from the plot are β-items: 4 (P 0.35, D −0.051), and 17 (P 0.36, D 0.205). β-Item 4 is the same as α-item 9, and the misconception of the nature of boiling water bubbles was expected by the researchers even though the negative discrimination was a surprise showing that more low achieving students answered the item correctly than high achieving students. This could be in part due to a high frequency of guessing for the item. β-Item 17 (α-Item 22) once again fell in the same region with poor discrimination, but ideal difficulty. However, due to the previously conducted validation interviews which showed proper interpretation of the item, there was no concern for item performance. Two items were both hard items and poor discriminators, β-items: 19 (P 0.05. D 0.025) and 16 (P 0.17, D 0.077). β-Item 19 was expected to be low performing due to the nature of the question requiring understanding of the phase change of water at the particulate level. However, item 16 posed concern with only 17% of the student sample answering the item correctly, as this item received major revisions after the α-version administration. The item addresses the topic of burning wood, a very common chemical change example, and asks why the remaining ashes weigh less than the starting wood (Table 1). After administration of the CPCA β-version, interviews were conducted with 7 students following the same format as earlier. The interview transcripts demonstrated that there are numerous concepts students must have in order to correctly answer the item:

1. Gases have mass

2. Burning wood releases gases (specifically need to know carbon dioxide gas)

3. Wood contains carbon

4. Chemical potential energy in the wood is converted to thermal energy

5. Air is not synonymous with carbon dioxide

6. The size and mass of the ashes does not account for the loss of weight

Students who did not possess strong conceptual knowledge in all areas were unable to get the item correct or misinterpreted the distractor choices. A solution may have been to supply multiple items on combustion; however, if several items were connected to a specific type of chemical change this would inhibit the student score on the CPCA representing breadth of understanding. Due to the complexity of the item and poor psychometric properties (discrimination and difficulty), this item was deleted from the analysis. Consequently, the deletion of β-item 16 renumbered the CPCA forming the final 19-item γ-version, available in the supplementary materials (Table 2).† The Cronbach's alpha of CPCA γ-version is 0.612. This increase in internal consistency is additional evidence for removing the item about burning wood. Content validity of the items arises from the item sources. Table 3 shows the resources corresponding to each item on the 19-item γ-version of the CPCA.

Table 2 Alteration of item numbers throughout refinement of the CPCA

α-Version item #	β-Version item #	γ-Version item #
1–5 True/False	Omitted	Omitted
6	1	1
7	2	2
8	3	3
9	4	4
10	5	5
11	6	6
12	7	7
13	8	8
14	9	9
15	10	10
16	11	11
17	12	12
18	13	13
19	14	14
20	15	15
21	16	Omitted
22	17	16
23	18	17
24	19	18
25 Free Response	20 Free Response	19 Free Response

---

Table 3 Corresponding resource with item number from 19-item CPCA γ-version

Resource	Item(s)^a
a Free response item #19 is a general topic not drawn from a resource. b Items 4 and 18 were developed based on Osborne and Cosgrove (1983).
AAAS	6, 7, 8, 11, 14 and 15
Ardac and Akaygun (2004)	1, 2, 5, 12, 13 and 16
MOSART	3, 9 and 17
Yezierski and Birk (2006)	4^b, 10 and 18^b

---

Conclusions

The psychometric properties of the instrument in the 6th grade population were determined for the α-version of the CPCA. The results along with student interviews guided improvements to produce the β-version of the CPCA. Further validation resulted in the final CPCA γ-version. Reliability of the instrument was measured throughout the revision process resulting in a final internal consistency of 0.612 as measured by Cronbach's alpha.

Limitations

A limitation of the study is that it was only conducted at one school. Schools with similar demographic make-ups may result in similar testing results. The testing of the instrument in multiple schools and with a higher number of students would have strengthened its development. However, even with a small sample size, instrument quality is demonstrated by good psychometric properties. Another limitation relates to CPCA administration timing. Results pre- and post-instruction expectedly varied; however, the diversity in administration times possibly captured a greater range of conceptions during the CPCA development making it more widely adaptable.

Implications for teaching and future research

The development of the CPCA aimed to provide an assessment to measure the successful incorporation of particulate behavior in the processes of chemical and physical change. Located in the supplemental material is a copy of the CPCA γ-version as well as an accompanying teacher guide with suggestions for successful classroom implementation. The CPCA can be used to diagnose students' preconceptions before instruction and then again as a postassessment to determine if instruction was effective at improving students' ideas. Furthermore, use of the CPCA as just a postassessment can be valuable in supplying well-designed and validated items for grades 6–8. The CPCA, like other rigorously tested concept inventories, is a crucial source of accurate information about students' understandings that teachers can use to effectively plan future instruction that addresses areas requiring remediation or reinforcement. Currently, the CPCA has already been applied to help researchers evaluate an intervention on the topic of chemical and physical change with a 6th grade population (Christian and Yezierski, 2012).

Acknowledgements

The authors thank Cindy Kettlewell, Tracy Vu, and all of the students who participated in the study. Also, the authors wish to acknowledge the reviewers for their insightful comments that enhanced the manuscript.

References

1. Abraham M. R., Williamson V. M. and Westbrook, S. L., (1994), A cross-age study of the understanding of five chemistry concepts. J. Res. Sci. Teach., 31, 147–165.
2. Ahtee M. and Varjola I., (1998), Students' understanding of chemical reaction, Int. J. Sci. Educ., 20, 305–316.
3. American Association for the Advance of Science (AAAS). Science Assessment. http://assessment.aaas.org.
4. Andersson B., (1990), Pupils' conceptions of matter and its transformations (age 12–16). Stu. Sci. Educ., 18, 53–85.
5. Ardac D. and Akaygun S., (2004), Effectiveness of multimedia-based instruction that emphasizes molecular representations on students' understanding of chemical change. J. Res. Sci. Teach., 41, 317–337.
6. Ausubel D. P., (1968), Educational psychology: A cognitive view. New York: Holt, Rinehart and Winston.
7. Barke H., Hazari A., Yitbarek S., (2009), Misconceptions in Chemistry: Addressing Perceptions in Chemical Education. Berlin Heidelberg: Springer-Verlag.
8. BouJaoude S. B., (1991), A study of the nature of students' understanding about the concept of burning. J. Res. Sci. Teach., 28, 689–704.
9. Christian B. and Yezierski E., (2012), Learning in science museums: Measuring student knowledge gains from a chemistry museum exhibit in a classroom environment. Oral presentation at the 243rd American Chemical Society National Meeting, San Diego, CA.
10. Department for Education, (2011), Science: Sc3 Materials and their properties. Retrieved January 11, 2012, from: http://www.education.gov.uk/schools/teachingandlearning/curriculum/primary/b00199179/science/ks2/sc3.
11. De Vos W. and Verdonk A. H., (1987), A new road to reactions: Part 4. The substance and its molecules. J. Chem. Educ., 64, 692–694.
12. Duit R., (2009), Bibliography – Students' and Teachers' Conceptions and Science Education (STCSE). Retrieved from http://www.ipn.uni-kiel.de/aktuell/stcse/stcse.html.
13. Eilam B., (2004), Drops of water and of soap solution: Students' constraining mental models of the nature of matter. J. Res. Sci. Teach., 41, 970–993.
14. Finnish National Board of Education, (2004), National Core Curriculum for Basic Education. Part III: Chapters 7.4–7.9. Retrieved February 5, 2012, from: http://www.oph.fi/download/47672_core_curricula_basic_education_3.pdf.
15. Gabel D. L., Samuel K. V. and Hunn D., (1987), Understanding the particulate nature of matter. J. Chem. Educ., 64, 695–697.
16. Garnett P. J., Garnett P. J. and Hackling M.W., (1995), Students' alternative conceptions in chemistry: A review of research and implications for teaching and learning. Stu. Sci. Educ., 25, 69–95.
17. Great Schools, (2008), Data on Interested School. http://www.greatschools.org. (last accessed January 2012).
18. Griffiths A. K. and Preston K. R., (1992), Grade-12 students' misconceptions relating to fundamental characteristics of atoms and molecules. J. Res. Sci. Teach., 29, 611–628.
19. Harrison A. G. and Treagust D. F., (2000), Learning about atoms, molecules, and chemical bonds: A case study of multiple-model use in grade 11 chemistry, Sci. Educ., 84, 352–381.
20. Hesse J. J. and Anderson C. W., (1992), Students' conceptions of chemical change. J. Res. Sci. Teach., 29, 277–299.
21. Johnson P. M., (1998a), Children's understanding of changes of state involving the gas state, Part 1: Boiling water and the particle theory. Int. J. Sci. Educ., 20, 567–583.
22. Johnson P. M., (1998b), Children's understanding of state involving the gas state, Part 2: Evaporation and condensation below boiling point. Int. J. Sci. Educ., 20, 695–709.
23. Johnson P., (2000), Children’s understanding of substances, Part 1: Recognizing chemical change. Int. J. Sci. Educ., 22, 719–737.
24. Lewis S. E. and Lewis J. E., (2005), The same or not the same: Equivalence as an issue in educational research. J. Chem. Educ., 82, 1408–1412.
25. McREL, (2012), Benchmark Standards Database, Standard 8. Retrieved January 11, 2012, from: http://www.mcrel.org/compendium/standardDetails.asp?subjectID=2&standardID=8.
26. MOSART: Misconceptions-Oriented Standards-Based Assessment Resources for Teachers, President and Fellows of Harvard College. http://www.cfa.harvard.edu/smgphp/mosart/index.html (last accessed March 2011).
27. Nakhleh M. B., Samarapungavan A. and Saglam Y., (2005), Middle school students' beliefs about matter. J. Res. Sci. Teach., 42, 581–612.
28. National Research Council, (2011), A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Committee on a Conceptual Framework for New K-12 Science Education Standards. Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
29. Novak J. D. and Musonda D., (1991), A twelve-year longitudinal study of science concept learning. Am. Educ. Res. J., 28, 117–153.
30. Novick S. and Nussbaum J., (1978), Junior high school pupils understanding of the particulate nature of matter: An interview study. Sci. Educ., 62, 273–281.
31. Novick S. and Nussbaum J., (1981), Pupils' understanding of the particulate nature of matter: A cross-age study. Sci. Educ., 65, 187–196.
32. Nussbaum J., (2009), The Particulate Nature of Matter in the Gaseous Phase. In R. Driver, E. Guesne, and A. Tiberghien (Eds)., Children's Ideas in Science (124–144). New York, NY: Open University Press.
33. Osborne R. J. and Cosgrove M. M., (1983), Children's conceptions of the changes of state of water. J. Res. Sci. Teach., 20, 825–838.
34. Pallant J., (2007), SPSS Survival Manual (3rd ed.). New York, NY: Two Penn Plaza.
35. Popham W. J., (2005), Classroom Assessment: What teachers need to know (4th ed.). Boston: Pearson.
36. Smith C. L., Wiser M., Anderson C. W. and Krajcik J., (2006), Implications of research on children's learning for standards and assessment: A proposed learning progression for matter and the atomic molecular theory. Measurement: Interdisciplinary Research and Perspectives, 14, 1–98.
37. Stavy R., (1988), Children's conception of gas, Int. J. Sci. Educ., 10, 553–560.
38. von Glasersfield E., (1984), An Introduction to Radical Constructivism. In P. Watzlawick (Ed.), The Invented Reality: How Do We Know What We Believe We Know? (17–40). New York: Norton.
39. Wilson M., (2009), Measuring progressions: assessment structures underlying a learning progression. J. Res. Sci. Teach., 46, 716–730.
40. Wiser M. and Smith C. L., (2008), Chapter 8: Learning and Teaching about Matter in Grades K-8: When Should the Atomic-Molecular Theory be Introduced? In S. Vosniadou (Ed.), International Handbook of Research on Conceptual Change (205–239). New York: Routledge.
41. Yezierski E. J. and Birk J. P., (2006), Misconceptions about the particulate nature of matter: Using animations to close the gender gap. J. Chem. Educ., 83, 954–960.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: 10.1039/c2rp20041d

---