Messages about valued knowledge products and processes embedded within a suite of transformed high school chemistry curricular materials
Adam G. L.
Schafer
,
Thomas M.
Kuborn
,
Cara E.
Schwarz
,
Megan Y.
Deshaye
and
Ryan L.
Stowe
*
Department of Chemistry, University of Wisconsin–Madison, 1101 University Ave., Madison, WI 53706, USA. E-mail: rstowe@chem.wisc.edu; Tel: +1-608-890-2568
First published on 5th September 2022
Abstract
The way high school chemistry curricula are structured has the potential to convey consequential messages about knowledge and knowing to students and teachers. If a curriculum is built around practicing skills and recalling facts to reach “correct” answers, it is unlikely class activities will be seen (by students or the teacher) as opportunities to figure out causes for phenomena. Our team of teachers and researchers is working to understand how enactment of transformed curricular materials can support high school chemistry students in making sense of perplexing, relatable phenomena. Given this goal, we were surprised to see that co-developers who enacted our materials overwhelmingly emphasized the importance of acquiring true facts/skills when writing weekly reflections. Recognition that teachers’ expressed aims did not align with our stated goal of “supporting molecular-level sensemaking” led us to examine whether the tacit epistemological commitments reflected by our materials were, in fact, consistent with a course focused on figuring out phenomena. We described several aspects of each lesson in our two-semester curriculum including: the role of phenomena in lesson activities, the extent to which lessons were 3-dimensional, the role of student ideas in class dialogue, and who established coherence between lessons. Triangulation of these lesson features enabled us to infer messages about valued knowledge products and processes materials had the potential to send. We observed that our materials commonly encouraged students to mimic the structure of science practices for the purpose of being evaluated by the teacher. That is, students were asked to “go through the motions” of explaining, modeling etc. but had little agency regarding the sorts of models and explanations they found productive in their class community. This study serves to illustrate the importance of surfacing the tacit epistemological commitments that guide curriculum development. Additionally, it extends existing scholarship on epistemological messaging by considering curricular materials as a potentially consequential sources of messages.
Introduction
The Framework for K-12 Science Education, and the Next Generation Science Standards (NGSS) derived from it, articulate ways in which learners could purposefully integrate science knowledge (disciplinary core ideas) and science activities (science & engineering practices) to understand aspects of their experience (National Research Council, 2012; NGSS Lead States, 2013). However, the journey from defined performance expectations to enactment of learning ecosystems capable of supporting meaningful engagement in these performances is long and arduous. Status-quo curricula, assessments, and instructional strategies which foreground recall of knowledge and performance of procedures fall far short of what the NGSS envision (Schwarz et al., 2017; Lowell et al., 2020; Manz et al., 2020). Accordingly, many groups of scholars and classroom teachers have created curricula, professional learning opportunities, and assessments that aim to embody a focus on knowing and doing science (e.g., Anderson et al., 2018; Edelson et al., 2021). Our team of researchers and teachers is one such group.Drawing from scholarship examining college chemistry learning environments, we used the curriculum Chemistry, Life, the Universe, and Everything (or CLUE) as a template for what an NGSS-aligned high school course could look like (Cooper and Klymkowsky, 2013). CLUE was a promising model because, when appropriately prompted, CLUE-enrolled college students are better able to construct written explanations for phenomena such as dissolution of an ionic solid, acid–base reactions, and differences in boiling point than similar cohorts of students enrolled in other, more traditional, courses (Cooper et al., 2016; Kararo et al., 2019; Ralph et al., 2022). Additionally, CLUE learning objectives clustered well under the NGSS performance expectations related to high school chemistry (Stowe et al., 2019a). Our team of teachers and researchers worked to remove bits of CLUE (e.g., assessments, readings, slides) that were beyond the scope of NGSS expectations and add materials where CLUE did not fully address desired performances. A pilot of this revised curriculum (creatively termed High School CLUE or HS-CLUE) demonstrated that HS-CLUE enrolled students (like their undergraduate peers) appeared better positioned to correctly explain a difference in boiling point between two substances than students enrolled in differently structured high school chemistry learning environments (Stowe et al., 2019b).
Development, analysis, and refinement of HS-CLUE focused largely on getting students to produce knowledge products (like explanations, models, and arguments) that mimicked those a practicing chemist might create. Implicit in this focus was the view that, because professional chemists “do X and think like Y … learners of science should also do X and think like Y” (Russ, 2014). That is, the views on knowledge and knowing that informed course design align with what Russ calls the epistemology of science. Here epistemology refers to one's beliefs on what it means to know and learn science (Hofer and Pintrich, 1997). Importantly, one's epistemologies are not necessarily explicit or conscious (Berland et al., 2016). Indeed, we were not conscious of the epistemological commitments guiding creation of HS-CLUE. Russ notes that there are a variety of reasons the epistemology of science model is appealing: the way scientists do things is a product of iterative refinement and enables one to understand the world using a range of powerful models, professional science has norms we can recognize, and we can make claims about authenticity by comparing what students do to what scientists do (Chinn and Malhotra, 2002; Russ, 2014).
Russ and others also note that an epistemology of science model assumes meaningful distinctions between scientists and learners of science (Hofer and Pintrich, 1997; Hammer and Elby, 2002; Russ, 2014). This parallels expert-novice models of learning and positions experts as “scientists” and novices as individuals who learn about what the scientists do/did (Chase and Simon, 1973; Chi et al., 1981). This has the effect of minimizing the productive ways of thinking learners have cultivated from reasoning about how the world works. Additionally, the reason scientists act in particular ways is absent from the epistemology of science model (Russ, 2014). Scientists do not construct particular knowledge products with the end-goal of pleasing a teacher. Instead, explanations, arguments, models and the like are part of an ensemble of activities directed at figuring out how aspects of the world work or designing solutions to problems (Jiménez-Aleixandre et al., 2000; Berland and Hammer, 2012b; Schwarz et al., 2017). That is, scientists engage in activities that are productive for advancing knowledge construction aims they have (aims such as plausible mechanisms for perplexing phenomena). We agree with Russ that learning science ought not be materially different from doing science. The actions of learners should, for and from the perspective of those learners, be productive for making sense of phenomena. This stance assumes a model in which learners adopt epistemologies for science (Fig. 1; Russ, 2014).
 | ||
| Fig. 1 Representations depicting the tacit model of epistemology of science (left) and epistemology for science (right). Note that, in the model of epistemology for science, what scientists and learners do is mediated by actions productive for sensemaking. Adapted from Russ 2014 with permission from Wiley. | ||
During initial development of HS-CLUE, we saw no discontinuity between our tacit commitment to epistemologies of science and our stated goal of supporting molecular-level sensemaking. By sensemaking, we are referring to the construct defined by Odden and Russ as, “a dynamic process of building or revising an explanation in order to ‘figure something out’ – to ascertain the mechanism underlying a phenomenon in order to resolve a gap or inconsistency in one's understanding” (Odden and Russ, 2019a, pp. 191–192). Here, we examine how our tacit commitment to epistemologies of science was reflected by the structure and emphasis of the materials we built. We then argue the importance of curriculum-embedded messages about knowledge and knowing in affecting teachers’ assessment and instructional decisions.
The following research question guided this work:
What messages about valued knowledge products and processes were embedded in transformed curricular materials for high school chemistry?
Why we should care about epistemology
There is a large amount of evidence, across many contexts and age ranges, that learners’ in-the-moment views on knowledge and knowing (i.e., epistemologies) affect what they learn and how they learn it (Rosenberg et al., 2006; Russ et al., 2008; Ke and Schwarz, 2020). Redish describes epistemology as a control structure which bounds the ideas and connections that seem reasonable in a given context (Redish, 2004). If, as Lising, Elby, and Redish observe, students view science class as a place where only formal knowledge is “allowed”, they may not draw on potentially productive ideas from their everyday life (Lising and Elby, 2005). Relatedly, students’ epistemologies affect how and why they engage in given learning activities. Berland and Hammer observed that differing understandings of the aims of class knowledge-construction work resulted in qualitatively distinct manners of student engagement. When students aimed to persuade their peers that they had the “best” evidence-based argument, they were engaged with competing arguments voiced by their peers and directing the course of class conversation. By contrast, when the goal of an interaction was to get “credit” for voicing an idea, students tended to center their attention on being noticed by the teacher, who in-turn directed the flow of the conversation (Berland and Hammer, 2012b).Epistemologies also have implications related to the transfer of learning. Engle observed that students who framed their learning as “part of an ongoing context in which [they] are integral participants” were able to apply what they learned in a different context (Engle, 2006, p. 490). Hammer and colleagues propose a mechanism by which epistemologies may underpin scenarios such as what Engle observed (Hammer et al., 2005). They argue that “transfer” occurs when ones’ perspective on knowledge and knowing in one context is seen (implicitly or explicitly) as relevant in a different context. From this perspective, cultivating epistemologies useful for post-school life in formal environments becomes a worthwhile goal.
Epistemological resources
The central importance of epistemologies to learning has led to many efforts to model them. Some have proposed that individuals hold relatively stable perspectives on knowledge and learning that advance in predictable developmental stages (King and Kitchener, 1994; Kuhn 1999). Others model epistemologies as “theory-like” systems individuals are conscious of and intentionally apply to their lives (Hofer and Pintrich, 1997). Neither “epistemologies as developmental traits” nor “epistemologies as conscious theories” models can account for the fragmented, variable, and context-dependent nature of epistemological ideas that has been documented in K-16 learning environments (e.g., Rosenberg et al., 2006; Scherr and Hammer, 2009; Irving et al., 2013; Shar et al., 2020). Students have been shown to transition between epistemologies at the timescale of minutes, and these transitions are associated with a range of contextual cues (e.g., from peers, from the teacher; Rosenberg et al., 2006).To be consistent with the ample evidence discussed previously, we adopt a model in which one's epistemology is made of many fine-grained “pieces” that can be activated (or not) depending on context (Minstrell, 1992; DiSessa, 1993; Sherin, 2006). These pieces (or “resources”) are “smaller and more general than theories or traits (Hammer and Elby, 2002, p. 176). Thus, one does not have a “pre-compiled” (Hammer et al., 2005, p. 95) view on knowledge and knowing but rather many small epistemological ideas that are compiled in real-time.
Epistemological messages
Assuming a resources model, one might reasonably wonder what leads someone to compile their epistemological understandings in a particular way in a given moment. To help us answer this question, we leverage work by Russ that argues for the importance of tacit messages about knowledge in shaping students’ in-the-moment epistemologies (Russ, 2018). For example, a teacher’s encouragement to “start with what you know” signals that it is useful to call on prior knowledge when engaged in class activities (Rosenberg et al., 2006). In contrast, responding to a student answer with “close, but not quite” sends the message that class is about producing right answers. Russ claimed that epistemological messages of this sort are “ever present in student-teacher interactions” (Russ, 2018, p. 98) and may be either domain-general (i.e., applicable in domains from chemistry to civics) or domain-specific (e.g., applicable to knowledge construction in science class). The message that you should “start with what you know” is a domain-general message related to the source of useful knowledge in that moment. A message that students should “remember to explain how and why!” as they construct models for a phenomenon relates to a science knowledge product (i.e., this is a domain-specific message, Ke and Schwarz, 2019). We agree with Russ that both domain-general and domain-specific epistemological messages matter in science class. After all, students are unlikely to see value in supporting causal mechanisms with evidence (domain-specific) unless “figuring out why things happen” is a thing one does in class (domain-general). It should be noted that we do not expect students’ experiences with and understandings of epistemological messages sent during class to be stable one-to-one maps of these messages. It is likely emerging epistemological understandings of a course will involve complex, dynamic negotiations between students’ experiences in similar courses, messages from the class, understandings of course content, etc.Curriculum-embedded epistemological messages
Existing scholarship on epistemological messaging has focused almost exclusively on messages embedded in student–teacher verbal interactions (Rosenberg et al., 2006; Russ, 2018; Ke and Schwarz, 2019). This work has persuasively argued for the potential impact of instruction-embedded messages on how students engage in knowledge construction and with what knowledge they do so (Rosenberg et al., 2006; Russ et al., 2009). However, we contend that messages about knowledge and knowing are also embedded in other aspects of a learning environment. Here, our focus is primarily on those messages that might be embedded in student- and teacher-facing curricular materials (e.g., textbooks, lecture slides, worksheets, exams).To help illustrate how curricular materials contain embedded messages about knowledge and knowing, consider the three-day lesson plan shown in Table 1.
| Monday | Tuesday | Wednesday |
|---|---|---|
| Overview | Overview | Overview |
| Teach structure of chemical equations | Practice balancing chemical equations | Quiz on balancing chemical equations |
| Learning goals | Learning goals | Learning goals |
| (1) Students can identify products vs. reactants | (1) Students can balance chemical equations | (1) Students can identify products vs. reactants |
| (2) Students can balance chemical equations | (2) Students can balance chemical equations | |
| (3) Students can state the meaning of a coefficient | (3) Students can state the meaning of a coefficient | |
| Materials needed | Materials needed | Materials needed |
| PowerPoint slides for Unit 4 | Whiteboards and markers, Practice sheet 3 (Unit 4) | Quiz 3 (Unit 4) |
| Performance measurements | Performance measurements | Performance measurements |
| In-class questions | Spot checks of student responses during group practice | Quiz responses |
The learning environment described by this lesson plan snippet signals that a central goal of class knowledge building is production of correct answers. Successfully achieving all specified learning objectives requires performing defined skills and/or recalling facts to arrive at a singular “correct answer”. Knowledge products produced by the class (e.g., balanced chemical equations) are constructed in order to be evaluated for correctness. Opportunities to practice performances introduced during lecture signals that repeated practice is a useful path toward achieving class knowledge construction goals. Decoupling skills and facts from the phenomena that lend these performances meaning (in-class and on assessments) messages that observable events are irrelevant; only producing a correct number/fact/drawing matters.
The fact that we claim messages about knowledge and knowing are embedded in curricula and assessments does not guarantee that students will experience these messages, nor that they will respond in certain, predictable ways. Indeed, we lack the data to infer which message(s) are likely to be most (or least) consequential to which students. Given the dynamic, context-sensitive nature of cognition, it is likely that answers to these questions will not be at-all straightforward (Russ, 2018). Our purpose here is to argue that curricula and assessments have the potential to convey messages about knowledge and knowing, not that a particular subset of these messages will be understood in a given way by students. Additionally, we claim that curriculum- and assessment-embedded messages have the potential to constrain the epistemologies students are likely to adopt. If all assessments, instructional interactions, and curricular components signal that quickly recalling a correct answer is the main goal of class, it is unlikely students will see the class as a space to construct and critique reasonable causal accounts for something they observed.
Centering teachers’ responses to epistemological messages
The vast majority of research on epistemologies in science learning has, justifiably, focused on students’ epistemologies (Hammer and Elby, 2002; Lising and Elby, 2005; Rosenberg et al., 2006; Hutchison and Hammer, 2010; Ke and Schwarz, 2020). This makes sense due to the field's goal of supporting students in developing epistemologies useful once they leave the classroom. However, given that teachers’ in-the-moment views on knowledge and knowing both affect and are affected by learners’ epistemologies, we claim that there is also great value in focusing on how teachers experience and respond to signals about knowledge and knowing. We expect teachers’ epistemologies to be affected by epistemological messages experienced from administrators, peers, students, parents, curricular materials and (likely) many other sources. Messages are likely to arise from interactions of several “sources”. For example, student groups interacting with curricular components may signal the value of certain knowledge products or processes to their instructor. Decisions made when enacting a course (e.g., how to respond to student thinking, whether to spend two classes practicing a skill) can be viewed as responses to epistemological messages teachers have experienced and negotiated.Fig. 2 describes our coarse model for teachers’ experiencing and responding to epistemological messages. Black circles represent possible sources of epistemological messages, black arrows of varying types represent different sorts of messages being “sent”. Potential interrelationships between sources are signified by double-headed blue arrows. Responses are indicated by red arrows and corresponding red boxes. This model is not meant to describe an exhaustive list of possible sources of such messages or possible responses. Likewise, the model does not describe how teachers experience and negotiate potentially conflicting messages. Teachers’ negotiation of epistemological messages (like students’) is almost certainly complex, dynamic, and impacted by many contextual factors (Jaber et al., 2022). Exploring this negotiation process is an interesting area for future research.
 | ||
| Fig. 2 Coarse-grained model for teachers experiencing and responding to epistemological messages from several sources. These sources are likely interrelated (signified by the double-headed blue arrows). For example, we expect messages to be sent by interactions of students with curricular components. Black arrows of different sorts are meant to represent varying epistemological messages. Red arrows are meant to represent responses (described in red boxes). | ||
Curricula-embedded messages can affect teacher epistemologies
We theorize that messages about knowledge and knowing embedded in curricula and assessments can affect the epistemologies teachers adopt in the classroom, which in turn can impact students’ epistemologies. Elements of the course described earlier (Table 1) have the potential to signal the value of correct numerical, factual or drawn responses to teachers. Likewise, a curriculum overstuffed with a series of topics might message that chemistry class should be a place where we cover vast swathes of content. Curricula and assessments a teacher adopts and adapts have the potential to constrain the epistemological messages this teacher is likely to send to students. If no part of the curriculum allows for construction and critique of causal accounts for phenomena, it is unlikely class will be seen (by students or teachers) as a place where making sense of phenomena is something one does. Accordingly, curricula of this general type are likely to act as impediments to realizing learning environments that position students as doers of science. It is important to note that specific curricular and assessment features do not guarantee coherent messages on knowledge and knowing will be conveyed in-class. It is possible to focus an activity meant to engage students in construction and use of models on listing vocabulary words (Kuhn and Pease, 2008; Berland and Hammer, 2012a; McNeill et al., 2016; Gouvea and Passmore, 2017). However, in the absence of curricula that (at least attempt) to foreground epistemologies for science, it is likely the “school game” of memorization and recall will win out as the main goal of science class (Lemke, 1990).Here, we surface the formerly tacit epistemological commitments that shaped a set of transformed curricular materials we created for high school chemistry (Stowe et al., 2019b, 2019a).
Methods
This research was approved by the university's Institutional Review Board as an investigation into how high school chemistry learning environments can engage students in sensemaking. All methods were in compliance with the university's policies on ethics. Informed consent was obtained for all participants prior to participation.Course context
This study occurred in the context of a year-long enactment of the high school curriculum High School Chemistry, Life, the Universe, and Everything (HS-CLUE). HS-CLUE was created by a team of teachers and researchers from the undergraduate general chemistry curriculum Chemistry, Life, the Universe and Everything (CLUE; Cooper and Klymkowsky, 2013; Stowe et al., 2019a). To generate the HS-CLUE suite of curricular materials our team of high school chemistry teachers and education researchers removed parts of the CLUE curriculum that were beyond the scope of a high school course (e.g., coupled reactions, molecular orbital theory) and added supplementary materials where CLUE did not fully support the expectations laid out by the NGSS. After the initial draft was completed, the teachers on our team piloted the materials in their classrooms. From this point, the team of teachers and researchers followed a regular cycle of development and refinement, using findings from analyses of teacher reflections to guide continued development, as shown in Fig. 3. This study was conducted during the 3rd cycle of curricular implementation. | ||
| Fig. 3 HS-CLUE design cycle. | ||
The HS-CLUE suite of curricular materials included 10 units addressing topics specified in Table 2. As noted previously, each of these topics is meant to be connected by a small set of powerful explanatory ideas. That is, there is no dedicated “energy” chapter in CLUE because energy is used as part of causal accounts in virtually all chapters. Each unit contained a teacher guide and any accompanying materials needed for each lesson (e.g., student worksheets, presentation slides, links to web resources). For our purposes, a lesson constituted a discrete activity or set of activities communicated in the teacher guide as belonging together. So, an “individual lesson” in the curricular materials does not necessarily constitute a single day's class time. For example, in the Atoms unit (Unit 1) the second lesson is a laboratory investigation that likely requires multiple class periods to complete while the sixth lesson involves a presentation with seven slides that would easily be addressed within a single class period. The materials were intended to support two semesters worth of learning experiences for an introductory high school chemistry course.
| Unit | Core topic(s) | Number of lessons |
|---|---|---|
| 1 | Atoms | 22 |
| 2 | Atomic interactions | 8 |
| 3 | Nuclear chemistry | 4 |
| 4 | Electrons | 7 |
| 5 | Emergent properties | 10 |
| 6 | Intermolecular forces | 13 |
| 7 | Thermochemistry | 10 |
| 8 | Solutions | 9 |
| 9 | Chemical reactions | 13 |
| 10 | Kinetics and equilibrium | 12 |
Teacher reflections
Characterizing teachers’ sense of valued knowledge products and processes over the course of a two-semester enactment is a non-trivial undertaking. Most work examining epistemologies focuses on either very small (e.g., interactions during a single class) or very large (e.g., before and after a course) timescales (Perkins et al., 2007; Barbera et al., 2008; Russ and Luna, 2013; Ding and Mollohan, 2015; Ke and Schwarz, 2019). Macro-level pre-post surveys cannot capture the dynamics of participant epistemic cognition and so were too low resolution for our purposes. Micro-level analyses of in-the-moment epistemologies, while able to beautifully capture how participants’ views on knowledge and knowing change one minute to the next, are far too labor intensive to be practical over the course of a semester. Here, we split the difference between these two types of analyses by eliciting evidence of teachers’ epistemologies via weekly reflective logs. Logs of this sort are a common tool used to characterize implementation of learning materials (Ball, 1999; Rowan and Correnti, 2009; Harris et al., 2015) and, as we shall see, contain useful information about the tacit epistemologies that shape responses.| Code (aim) | Description (the goal of the experience is…) | Fleiss's Kappa |
|---|---|---|
| True or mostly true beliefs | …acquiring knowledge that approximates or approaches truth | 1 |
| Understanding | …forming connections between ideas and seeing how ideas fit together | 0.71 |
| Construction of explanations | …using knowledge to generate an account of how or why a characterized/specified event occurs | 0.97 |
| Avoiding false beliefs | …avoiding knowledge that is false when acquiring beliefs | 1 |
Coding was divided amongst three raters (AS, TK, CS) so that each week's reflections were coded by two members of the research team. The three raters conducted frequent checks of code applications. Any disagreements in code applications were discussed, resulting in minor modifications to the code descriptions, when necessary. Coding continued until complete agreement was reached. In addition, reliability was measured with the inclusion of a fourth rater (RLS), who was a member of the original HS-CLUE development team. For the repeated reliability measurement, three raters per log entry were provided roughly 15% of the reflections to code. A Fleiss's kappa was calculated to determine agreement among the three raters. Table 3 shows the calculated values for each category, each of which indicate high reliability between raters (Cohen, 1960). From these codes, we were able to infer the epistemological aims surfaced when teachers reflecting on “what worked” in their local context.
Our four codes are unpacked in detail below:
Teachers may aim for students to acquire true (or mostly true) beliefs. These sorts of aims are accomplished by gaining knowledge that aligns with ideas that the teacher accepts as true (Chinn et al., 2011). When signalling the value of this aim, a teacher may mention “covering” or “addressing” factual constructs in the context of the learning environment. For example, when learning about particle motion, a teacher with the epistemological aim of acquiring true beliefs may design opportunities for their students to gather facts about particles.
Related to the epistemic aim of true beliefs is the aim of avoiding false beliefs. Although a teacher could achieve their goal of avoiding false beliefs by gaining true beliefs, teachers designing activities with the epistemological aim of avoiding false beliefs will purposefully scaffold learning to ensure that students do not “gain misconceptions” or “get the wrong idea” (Chinn et al., 2011). For example, a teacher may express that they “do not plan to explicitly teach Bohr's model this year since it results in many misconceptions about electron behavior.”
The epistemological aim of understanding moves beyond the acquisition of ideas by seeking to support students in connecting ideas and/or seeing how ideas “fit together” (Chinn et al., 2011). These sorts of aims address questions such as “How does X relate to Y?”. Continuing the previous example about particle motion, a chemistry teacher may design opportunities for students to drop dye into beakers containing water at different temperatures. By exploring the system in this example, students may relate particulate motion to temperature.
The final aim included in our coding scheme is construction of explanations. This epistemological aim is addressed when teachers design opportunities for their students to generate an account of how or why a phenomenon occurs (Chinn et al., 2011). Aims of this type address questions such as “Why does X relate to Y?” Using the previous example, a teacher may ask students to explain their observations after conducting the experiment of dropping dye into beakers at different temperatures. The epistemological aim builds from the aim of understanding by asking students to go beyond recognizing a relationship to explaining how/why something occurs.
We view the knowledge construction aims expressed by teachers as responses to epistemological messages they have experienced and negotiated. The findings from characterizing teachers’ goals (described in Results and discussion) motivated a follow-up analysis of the messages embedded within the HS-CLUE curriculum. In particular, we wondered whether the tacit epistemological messages embedded in the curricular materials aligned with the stated goals of our curriculum development program (i.e., supporting molecular-level sensemaking).
Curricular materials
| Category | 0 | 1 | 2 |
|---|---|---|---|
| Use of phenomena | Teacher focuses on skills or content without a connected, relevant phenomenon OR students do not engage with phenomenon introduced in class | Phenomenon used as hook OR example for students to work with but not as a driver of goals or activities across the entire lesson | The phenomenon is the thing that attention is consistently centered on (i.e., the thing investigated) throughout the course of the lesson |
| Opportunities for 3D learning | Only cross-cutting concepts present | Cross-cutting concepts and scientific and engineering practices present | 3-Dimensional (cross-cutting concepts, scientific and engineering practices, and disciplinary core ideas present) |
| Use of student ideas | Lesson is mostly students receiving information (via teacher talk, reading, and/or videos) rather than constructing or making sense of ideas | Student activity is consistently guided by teacher or materials such that teacher and/or materials guide both “what to do” and “how to think” | Students are actively constructing their understanding during the lesson. Students may interact with materials and teacher for guidance but interaction guides “what to do” not “how to think” |
| Coherence | There is no attempt to connect ideas in lessons to past or future lessons | The teacher plays the main role in connecting the lesson to past and future lessons | Students are the primary ones making sense of how the lesson is connected to past or future lessons |
The following four learning environment features were characterized as part of this analysis: the use of phenomena, opportunities for 3-dimensional learning, the use of student ideas, and coherence. The modified coding scheme for this study is presented in Table 4, with each category described in detail below. Recall that “lessons” here are units of instruction bounded by the bulleted teacher guide for HS-CLUE. Accordingly, a worksheet might be a “lesson” as might a scaffolded activity. We adopted this approach to materials analysis because it created curricular units we could analyze. We do not claim that lessons are of equivalent length or were emphasized to the same extent in-class.
Each code category is described in further detail below:
To characterize the HS-CLUE curricular materials, individual lessons were extracted from the teacher guides provided and coded using a scheme modified from Lowell and colleagues (Lowell et al., 2020) shown in Table 4. Coding was divided amongst three raters (AS, TK, MD) so that each lesson was coded by two members of the research team. The three raters conducted frequent checks of code applications to ensure dependability of the analysis. Any disagreements in code applications were discussed, resulting in minor modifications to the code descriptions, when necessary. Once all lessons were coded, a Cohen's kappa was calculated to judge agreement, resulting in a kappa value of 0.84 (high agreement; Cohen, 1960). In addition, reliability was measured again with the inclusion of a fourth rater (RLS). For the repeated reliability measurement, three raters per lesson were provided roughly 15% of the lessons to code. A Fleiss's kappa was calculated for each category to determine agreement among the three raters. Table 5 shows the calculated values for each category, each of which indicate high reliability between raters (Cohen, 1960).
| Category | Fleiss's Kappa |
|---|---|
| Use of phenomena | 0.72 |
| Opportunities for 3D learning | 0.82 |
| Use of student ideas | 0.75 |
| Coherence | 1.00 |
The categories in our scheme describe features of the curricular materials that, we contend, enable us to infer tacit messages about knowledge and knowing embedded within and across lessons. For example, learning environments that signal the importance of epistemology for science would require (at minimum) opportunities for students to grapple with causes for phenomena, and construct their own explanations of observations they make. Without specific guidance in the curricular materials for supporting these sorts of opportunities, it is unlikely that the materials would convey (to students or teachers!) that science classrooms are places for making sense of the world.
Importantly, the extent to which a curriculum signals the importance of science work that is meaningful and purposeful to students is not characterized by any single category in our coding scheme. That is, epistemological messages emerge from a constellation of curricular features. To infer messages about knowledge and knowing embedded in HS-CLUE lessons, we first grouped lessons by similar code applications. For example, lessons with observed codes of Phenomenon = 0; 3-Dimensional = 0; Use of Student Ideas = 1; Coherence = 1 were grouped together and lessons with observed codes of Phenomenon = 2; 3-Dimensional = 2; Use of Student Ideas = 2; Coherence = 2 were group together. After lessons were grouped, the research team theorized about the epistemological messages that might be communicated by lessons described by a particular cluster of codes. To illustrate potential epistemological messages conveyed within the lesson groupings, we generated several “lesson exemplars” which are discussed in further detail below; however, first we continue our discussion on teachers’ responses to aggregate messages they encountered in the learning environment (which includes messages sent by curricular materials).
Results and discussion
Epistemological goals communicated by the teachers
Table 6 summarizes the results from our coding of all teacher reflections. Table 8 in Appendix B further elaborates on the frequency of aims expressed within each unit. We opted not to perform statistical tests for significant differences between aims, since the teachers’ knowledge construction goals are very likely informed by many more message sources than just our curricular materials. Instead, we employed the teacher reflections as more of a litmus test, reflecting knowledge construction aims we assumed (in part) to be emergent from negotiation of messages communicated by the HS-CLUE curricular materials.| Acquiring true beliefs | Acquiring understanding | Constructing explanations | Avoiding false beliefs | Total |
|---|---|---|---|---|
| 97 | 18 | 15 | 2 | 132 |
Of the 132 goals communicated by teachers in their reflections, there was an overwhelming signal for epistemological goals aligned to acquiring true beliefs. Teacher reflections mostly emphasized the need to “cover” or “get through” the content. For example, one teacher shared that they “need to cover periodic trends to finish this unit.” Some teachers mentioned that the desire to “get through” the content created tensions in the learning environment. For example, “[Students] struggled until we had covered all of the [intermolecular forces of attraction] and were able to put all three types in front of them so they could see the differences in the polarity of the molecules. It was difficult for them to just link nonpolar to [London Dispersion Forces], polar to [dipole–dipole], and polar with HNOF (Lonepair) to H-bonds. It wasn’t until the end of the week that we could put that information together.” Teachers’ emphasis on “getting through” the content implies that chemistry class is viewed a place where vast swathes of content are covered.
Upon seeing an overwhelming emphasis on acquiring true beliefs in teacher reflections, we asked ourselves: How do the materials communicate the purpose of engaging in scientific endeavors? The teachers in this study are knowledgeable in their trade and do stellar work within their classrooms. These teachers attend regular professional developments, engage in learning communities, and meaningfully contributed to the design of the HS-CLUE materials. This paper should in no way be perceived as speaking to the detriment of the teachers. We believe that the teachers are merely communicating “what works” as they approached the limit of what the transformed curricular materials could support in their context. Burke (1966) and Wertsch (1998) refer to this limit as a “terministic screen,” meaning that teacher considerations about the materials intended use speak to limitations inherent within the design of the materials (Brown, 2009). The design of curricular materials needs to consider teachers’ decision making around enactment (Davis et al., 2016) to “open space” for students to figure out phenomena with agency (Ko and Krist, 2019; Lowell et al., 2020; Chen, 2022). If teachers experience strong messages from the curricular materials that chemistry class is a place where students should “take in” vast amounts of content while working to reproduce canonically “correct answers”, then this may shape their day-to-day epistemological aims (and how they design the learning environment). Remember, all teacher participants in this study volunteered to overhaul their course to align with HS-CLUE. Accordingly, we expect our study population to find curriculum-embedded epistemological messages especially consequential. We therefore read the overwhelming aim of getting “through content to obtain correct answers” as reflecting, to some degree, the valued knowledge products and processes baked into the HS-CLUE curriculum. This left us wondering what features of the curricular materials were potentially sending these sorts of messages.
Epistemological messages embedded within curricular materials
To describe features of curricular materials that could, when taken together, support inferences about embedded epistemological messages, each lesson was coded according to the scheme presented in Table 4. The results of this coding process can be found in Table 9 of Appendix C. Each category supported interpretation of the epistemological messages embedded within the HS-CLUE materials. For example, the way a lesson uses a phenomenon constrains available epistemological aims (Table 2). If all lesson activities are directed toward refining some sort of skill (e.g., Lewis structure drawing) this lesson does not have the potential to message the value of plausible mechanisms for phenomena (as there are no phenomena motivating the work). In analyzing our data, we recognized regular, salient features of curricular materials when coding results were gathered into groupings. For example, examining the use of phenomena in tandem with opportunities for 3-dimensional learning (i.e., opportunities to engage in the practices of scientists) lets us narrow the plausible messages about the process of knowledge construction and the character of the knowledge product (Miller et al., 2018; Russ, 2018; Ko and Krist, 2019). Additionally, the category use of student ideas speaks to the opportunities students have to communicate their ideas, express their thinking, and address questions they have about the learning environment. Examining how lessons use student ideas as well as ways in which they were coherent let us infer messages about the nature of epistemological agency afforded to students.We constructed three exemplars to represent the range of epistemological messages embedded in the HS-CLUE curricular materials (Fig. 4). These exemplars define a continuum from a class that typifies the “classroom game” (i.e., emphasizes acquisition of facts and skills; Lemke 1990) to a class with the potential to support student sensemaking. Assuming that the teacher crafting each learning environment in Fig. 4 strictly followed the guidance in the curricular materials, there would be observable differences in the sort of epistemological messages conveyed to students – and the teacher (Berland et al., 2016; Russ, 2018). For example, Sage on the Stage lessons (Phenomenon – 0; 3-Dimensional – 0; Use of Student Ideas – 1; Coherence – 1) would signal that class is a place where one should evaluate students’ recall of facts and skill performance. By contrast, Figuring Out Phenomena lessons (Phenomenon – 2; 3-Dimensional – 2; Use of Student Ideas – 2; Coherence – 2) would message that class is a place where students should be supported in constructing causal accounts consistent with what they have observed. Some exemplars better represent HS-CLUE than others. For example, 39 of 108 lessons coded were Sage on the Stage lessons while only one was a Figuring Out Phenomena lesson.
 | ||
| Fig. 4 Exemplars, derived from HS-CLUE materials, that describe a continuum from a class that typifies the “classroom game” to a class with the potential to support student sensemaking. | ||
Sage on the stage
The most common type of lesson (39 out of 108 lessons) was described by the following codes: Phenomenon – 0; 3-Dimensional – 0; Use of Student Ideas – 1; Coherence – 1. Curricular materials for these lessons supported teachers in communicating canonical information to students, as shown in Exemplar 1 of Fig. 4. The materials in this grouping encourage a lecture format focused on disaggregated facts and skills, so we have aptly titled this exemplar Sage on the Stage. The curricular materials for this sort of lesson might include presentation slides, guides for facilitating discourse, and prompts for students to respond to. Since the lessons were teacher-centered, they had less guidance on attending to student thinking relative to Exemplars 2 and 3 in Fig. 4. Additionally, Sage on the Stage lessons contained few to no opportunities for students to establish connection(s) to other learning activities. Similarly, the student guides typically did not give students opportunities to purposefully integrate science knowledge and activities. Instead, students were tasked with confirming knowledge as presented by the teacher. We claim that Sage on the Stage lessons are unlikely to signal the importance constructing plausible mechanisms to make sense of a phenomenon one cares about. Without explicit connection to a phenomenon, these curricular materials focus on learning the products of science, tasking students with accumulating facts, theories, and ideas established by others. These experiences may message to students that the products of scientific endeavors (e.g., models) are certain, and fixed forms of knowledge rather than tools used to convey understanding that are created, iteratively revised, and applicable within limited contexts (Schwarz and White, 2005; Sensevy et al., 2008). These materials may also message that there is no need to engage in the practices of scientists to gather those facts, since students have few to no opportunities to engage in scientific practices. Overall, the teacher-centered design and lack of opportunities for students to have any say over what facts they learn or why they learn said facts conveys that chemistry class is about answer-making, not sensemaking.Cookbooks for confirmation
To represent a lesson that describes a midpoint on the continuum between the “classroom game” and a sensemaking focused class, we crafted an exemplar described by the following set of codes: Phenomenon – 1; 3-Dimensional – 1; Use of Student Ideas – 1; Coherence – 1. This exemplar represents 2 of the 108 lessons. Lessons in this grouping gave students the opportunity to conduct a laboratory investigation to confirm course content. Phenomena were present in these activities, but they were used as hooks or examples to demonstrate an idea or scientific principle, not the focus of class sensemaking. As Exemplar 2 illustrates, students and teachers in this sort of lesson are focused on performing skills and following recipes, not explaining something they saw and wondered about. Others have referred to these sorts of materials as “hands on but not minds on”, meaning that students are “going through the motions” of science activities to be evaluated by a teacher (rather than figure out a perplexing phenomenon, Furtak and Penuel, 2019). Cookbooks for Confirmation lessons often include guidance on what students should do and observe in the lab, aides for supporting students through difficult calculations, and information on what a “correct” knowledge product would look like. Although students have the potential to engage in tasks that mimic the structure of scientific practices (e.g., analyzing and interpreting data they collect), they are doing so to please the teacher more so than to figure out a phenomenon. This sort of work is appropriately regarded as engagement in a pseudopractice as students are mimicking the structure of a science practice (modeling) without being motivated by a goal of figuring out causal accounts (Kuhn and Pease, 2008; Berland and Hammer, 2012b; Manz, 2015; McNeill et al., 2016; Manz et al., 2020). These performative approaches to engaging in scientific practices can undermine adoption of epistemology for science if they are presented merely as desirable skills taught as universal structures and rules (Ford, 2005; Miller et al., 2018; Ko and Krist, 2019; Haverly et al., 2020; Manz et al., 2020). Features crafted into the design of Cookbooks for Confirmation materials likely reinforce messages that “doing science” involves a linear process in which the outcome of any scientific endeavor is a product that is certain and unchangeable – as opposed to iterative, flexible, and limited in scope (Chen, 2022).Figuring out phenomena
To describe what a class with the potential to engage students in sensemaking might look like, we created an exemplar that is described by the following codes: Phenomenon – 2; 3-Dimensional – 2; Use of Student Ideas – 2; Coherence – 2. Only one lesson of this sort was embedded in the HS-CLUE suite of materials. Materials for this lesson explicitly connected students’ activities to constructing a causal account for a focal phenomenon. The corresponding Exemplar is therefore entitled Figuring out Phenomena. As shown in Fig. 4, the teacher in Exemplar 3 is guiding the students to use their current understandings to inform an explanation of the phenomenon, indicating that students’ prior experiences are valued in this context. One student appears to be grappling with the implications of their data, hinting that the goal of Exemplar 3 is not a response that exactly aligns with canon but an evidence-based explanation of the observed phenomenon that seems plausible to the classroom community. The curricular materials for this lesson guided the students through the experiment while leaving room for reflection on observations, provided the teacher with support for opening space for students to use their personal experiences, and situated figuring out the phenomenon as the central goal of knowledge building. The role of the phenomenon as something to be figured out gives students an opportunity to engage in (more) purposeful science investigation while conducting the sorts of activities that scientists engage in (Penuel et al., 2019; Manz et al., 2020). Additionally, the materials for this lesson provide students with opportunities to use their ideas and observations when figuring out the phenomenon, including items that encourage students to establish connections to other lessons within the unit. Exemplar 3 in Fig. 4 illustrates some of these features, showing the students interpreting their data to figure out a phenomenon. However, this snapshot does not capture the multitude of contextual affordances that would truly tip the student into a sensemaking frame (Odden and Russ, 2019b).Accumulated messages from HS-CLUE
The vast majority of HS-CLUE curricular materials would seem to support learning environments as depicted in Exemplars 1 and 2. Roughly 55% of lessons (59 out of 108) did not focus substantial attention on phenomena or 3-dimensional learning opportunities (i.e., received codes of Phenomenon – 0 and 3-Dimensional – 0), making it unlikely that the learning environments derived from them would support enactments that positioned students as doers of science (Manz et al., 2020). These materials likely acted to reinforce the commonly enacting teachers’ expressed epistemological aim of acquiring true beliefs by means of “getting through the content.” For example, one teacher shared, “Students also worked on the scientific skill of modeling by drawing Rutherford models of specific isotopes (e.g., carbon-12) and ions. Some students struggle with modeling isotopes and ions but with practice most catch on.” In this example, students are generating representations for the purpose of working towards a more canonically correct representation of common isotopes. Here, drawing a “correct” picture is the goal of the activity, not simplifying a system to enable explanation of a phenomenon. For construction of simplified system representations to be purposeful, model construction and use needs to be situated within a phenomenon and taken up by the classroom community as part of an ensemble of activities aimed at figuring out that phenomenon (Berland and Hammer, 2012b; Ryu and Sandoval, 2012; Ford, 2015; Manz et al., 2020). However, as we have observed here, the HS-CLUE materials did not consistently engage students in integrating a range of science knowledge and activities to unpack plausible mechanisms for an observable event. The observed alignment between epistemological messages embedded in the HS-CLUE materials and the teachers’ aims for learning speaks to the importance of developers consistently reflecting on the signals their materials send about valued knowledge products and processes. As we stated at the outset, we were unaware of our tacit epistemological commitments during our initial development efforts!Implications and future work
Supporting students in purposefully weaving together science knowledge and activities to make sense of perplexing phenomena is a demanding task that can be enabled by appropriately designed curricular materials (Miller et al., 2018; Furtak and Penuel, 2019; Ko and Krist, 2019; Haverly et al., 2020). Such materials should include opportunities to surface and meaningfully build on student ideas related to phenomena they have experienced and foreground connections between ideas that are productive from the students’ perspective rather than alignment with a narrow vision of canon. Curricula matter, in part, because they convey tacit messages to students and teachers about what it means to know and learn science. These messages have the potential to influence students’ and teachers’ interpretations of the learning environment (Tomanek et al., 2008; Haverly et al., 2020; Schafer and Yezierski, 2021). Epistemological commitments need to be surfaced and reflected on during curricular development. Additionally, classroom enactments should be analyzed for alignment to tacit epistemological commitments embedded within the curricular materials used to construct the learning environment. Learning environments that oversimplify the process of scientific investigation can reinforce messages that science mainly involves producing “correct answers” to please the teacher (Koslowski, 1996; Ford, 2005; Duschl, 2008) (Manz et al., 2020). Based on our analysis, it is likely that the HS-CLUE materials reinforced these sorts of messages, encouraging adoption of epistemology of science.How then, should we interpret the results of prior work (which showed that HS-CLUE students constructing more sophisticated explanations for a phenomenon than peers enrolled in other sorts of courses, Stowe et al., 2019b)? The foundation for HS-CLUE was the undergraduate CLUE curriculum, which is purposefully built around the use of a small set of powerful explanatory ideas to explain a host of phenomena (Cooper and Klymkowsky, 2013). We expect that HS-CLUE students, like CLUE students, responded to the curriculum-embedded message that correct explanations included connection of one or more core ideas to the cause for focal phenomena. That is, HS-CLUE students were better supported in constructing explanations that, to an expert, were more reasonable than those constructed by their peers in other learning environments. This is not a bad thing. We would be remiss if we did not note that CLUE is clearly a huge step forward relative to the scattershot topics and skills that characterize many general chemistry courses. CLUE students build a much more interconnected and useful command of fundamental chemistry ideas than is the norm in general chemistry (as demonstrated by work from the Cooper, Underwood, and Stowe groups among others) (Becker and Cooper, 2014; Cooper et al., 2015; Ralph et al., 2022). Indeed, it is difficult to imagine how one could cultivate epistemology for science in a lecture-based class in which hundreds of students are enrolled. However, at the high school level, affordances of the learning environment (including class size) enable us to consider how we might empower students with agency and authority over their knowledge construction goals and how they want to go about addressing these goals.
One such design feature useful for supporting epistemology for science involves focusing class attention on experiencing, wondering about, and figuring out phenomena over an extended period. Lessons that involved making sense of phenomena were more likely to make use of students’ ideas and explanations as well as support students in establishing coherence between activities. Stated differently, such lessons had the capacity to open space for students to chart their own knowledge construction path (Ko and Krist, 2019). Although obviously not a one-size-fits-all solution, our findings imply that a productive first step when developing or analyzing curricular materials is to examine how phenomena are used throughout the curricular sequence. This recommendation is consistent with many recent calls to focus science class on understanding aspects of experience (rather than learning about many facts and skills, Weizman et al., 2008; Ko and Krist, 2019; Ke and Schwarz, 2020). Curricula with the capacity to position students as doers of science are often built around unit-level “anchoring phenomena” that are figured out through a series of investigation-driven model refinements (Anderson et al., 2018; Edelson et al., 2021). Surfacing our formerly tacit epistemological commitments enabled our team to reflect on the sorts of epistemic aims and reliable processes that should guide development of NGSS-aligned curricula. We contend that a similar reflective process would be invaluable to all curriculum designers in the K-16 STEM education space. Continuing with our iterative design cycle, a central commitment in our design process will be to make space for students to take up activities (such as modeling and argumentation) as part of a larger enterprise of figuring out phenomena. As we continue to develop and refine curricular materials, we will investigate the utility of those materials for supporting epistemologies for science.
A looming question from this work is how enactment of transformed curricular materials might signal the value of epistemology for science, in manners that are consequential to students, over an extended timescale. Over the course of a single semester implementing HS-CLUE curricular materials, teachers on our team consistently shared goals aligned with acquiring true beliefs. However, teachers also shared goals aligned to developing explanations and building understanding. Clearly teachers possess productive ideas about knowledge and knowing that could be recruited in instructional contexts! We are yet to understand the extent to which curricular materials and accompanying professional learning opportunities can affect teacher (and student) stances toward what “doing science” entails. The backdrop against which this work occurs is a relatively stable set of epistemological norms conveyed through schooling in general and school science in particular (Rosenberg et al., 2006; Hutchison and Hammer, 2010). Accordingly, we expect shifting students and teachers toward adopting epistemology for science to be challenging indeed!
Limitations
Investigating curriculum-embedded epistemological messages in materials meant to support a two-semester course necessitated a coarse-grained analysis. A more fine-grained analysis (e.g., characterizing messages embedded in dialogue during a single lesson) could reveal more nuanced information about valued knowledge products and processes. Additionally, we recognize that teachers grapple with messages from a variety of sources. We only attended to one source of epistemological messages in this study (i.e., curricular materials) and cannot guarantee that this source was of especial consequence to enacting teachers. It is possible (and likely!) that teachers experienced other epistemological messages that they regard as more consequential than those embedded in curricula (e.g., teachers may pay more attention to observations of students or state-mandated goals). Other studies may examine the influence of curricular materials as compared to other sources of epistemological messages.Conclusions
The findings of this study suggest that messages about knowledge and knowing embedded in curricular materials may influence the epistemologies teachers adopt across moments of instruction. Further, the analytic process we used to surface tacit epistemologies woven throughout HS-CLUE has the potential of being broadly useful to other curriculum developers, teachers and researchers. The work described here was somewhat reactionary in nature. That is, we saw that teachers enacting HS-CLUE commonly expressed the goal of accumulating true beliefs in weekly reflections and sought to understand how (or whether) the curriculum signaled the importance of these sorts of goals. A modification of the four-category coding scheme developed by Lowell and colleagues was able to describe features of the HS-CLUE curriculum that, when taken together, enabled inferences about valued knowledge products and processes the materials likely message as important (Lowell et al., 2020).One stated goal for this suite of curricular materials was to support students in using molecular-level reasoning to explain phenomena they observed (Stowe et al., 2019a). Earlier studies indicated that the transformed materials were making progress toward this goal (Stowe et al., 2019b). However, the analysis detailed here revealed that, within and across lessons, the reason students were to construct explanations, models etc. was for these to be evaluated for correctness. That is, the purpose of engaging in science activities was to get the “correct answer”, not to unpack causes for phenomena of interest to the classroom community. Lessons in our materials that had the potential to signal the value of epistemology for science were found to have ratings closer to 2 for each category described in Table 5. This is not meant to communicate that 2s in each category always lead to the best lesson. Teachers and curriculum designers should weigh the embedded messages in their learning environments (and the tools used to construct the learning environment) against the goals the learning community is working towards.
Author contributions
Conceptualization: AGLS, TMK, RLS; data curation: AGLS, TMK; formal analysis: AGLS, TMK, CES, MYD, RLS; investigation: AGLS, RLS; supervision: RLS; visualization: AGLS, RLS; writing – original draft: AGLS, TMK, RLS; writing – review and editing: AGLS, TMK, CES, MYD, RLS.Conflicts of interest
There are no conflicts to declare.Appendix A: teacher demographic information
Information on the demographics and teaching experience of teachers who enacted HS-CLUE during the 2019–2020 academic year. Data was self-reported in response to prompts from the HS-CLUE developers. Years teaching HS-CLUE were reported at the end of the 19–20 academic year.| Teacher | Location | Degree | Degree field | State | Years teaching | Teaching chemistry | Teaching HS-CLUE |
|---|---|---|---|---|---|---|---|
| 1 | Rural | MS | Biology | IL | 25 | 5 | 2 |
| 2 | Suburban | BS | Chemistry | PA | 20 | 20 | 3 |
| 3 | Suburban | Med | Chemistry | PA | 12 | 7 | 2 |
| 4 | Suburban | MEd | Biology | MO | 10 | 3 | 2 |
| 5 | Suburban | BS | Chemistry | MI | 10 | 10 | 3 |
| 6 | Suburban | MEd | Psychology | OH | 6 | 3 | 2 |
| 7 | Suburban | MS | General Science | KS | 19 | 19 | 2 |
| 8 | Rural | MAT | Chemistry | MI | 12 | 12 | 3 |
| 9 | Suburban | MAT | Biology | MI | 26 | 17 | 4 |
Appendix B: summary of epistemological aims coding results
Counts of codes describing epistemological aims embedded in teachers’ responses to weekly reflections. Note that a given reflection may be described by more than one code and not all log entries enabled inferences about epistemological aims. The “take home message” of this coding was that acquiring true beliefs was far and away the most common epistemological aim expressed by enacting teachers.| Week | Epistemological aim | |||
|---|---|---|---|---|
| True beliefs | Understanding | Construction of explanations | Avoiding false beliefs | |
| 1 | 2 | 0 | 1 | 0 |
| 2 | 10 | 0 | 0 | 1 |
| 3 | 6 | 0 | 0 | 0 |
| 4 | 4 | 1 | 0 | 0 |
| 5 | 3 | 3 | 1 | 0 |
| 6 | 2 | 2 | 1 | 0 |
| 7 | 7 | 2 | 1 | 0 |
| 8 | 6 | 0 | 1 | 0 |
| 9 | 4 | 0 | 2 | 0 |
| 10 | 6 | 4 | 2 | 0 |
| 11 | 11 | 0 | 2 | 1 |
| 12 | 9 | 2 | 1 | 0 |
| 13 | 12 | 1 | 0 | 0 |
| 14 | 10 | 2 | 2 | 0 |
| 15 | 5 | 1 | 1 | 0 |
| Total | 97 | 18 | 15 | 2 |
Appendix C: summary of materials coding results by grouping
Matrix depicting the code clusters that described lessons in each of the 10 HS-CLUE units. The total number of lessons across the curriculum described by a code cluster is shown in the right-most column. Note that codes were mutually exclusive – that is a given lesson could only be described by one cluster of codes.| Grouping | Unit | Total | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| P-3D-SI-C | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
| 0-0-0-1 | 4 | 1 | 2 | 0 | 2 | 1 | 0 | 1 | 1 | 0 | 12 |
| 0-0-1-0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 3 |
| 0-0-1-1 | 5 | 3 | 1 | 4 | 4 | 8 | 3 | 2 | 5 | 4 | 39 |
| 0-0-1-2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| 0-0-2-1 | 2 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 3 |
| 0-0-2-2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| 0-1-1-1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 3 |
| 0-2-1-1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
| 1-0-0-1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 |
| 1-0-1-0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
| 1-0-1-1 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 4 | 3 | 11 |
| 1-0-2-1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| 1-1-1-1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 2 |
| 1-1-2-1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 |
| 1-2-1-1 | 1 | 0 | 0 | 0 | 0 | 2 | 1 | 0 | 0 | 0 | 4 |
| 2-0-1-1 | 1 | 1 | 0 | 0 | 0 | 1 | 2 | 0 | 0 | 2 | 7 |
| 2-0-2-1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 3 |
| 2-0-2-2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
| 2-1-2-1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 2 |
| 2-2-1-1 | 0 | 2 | 0 | 2 | 0 | 1 | 0 | 1 | 0 | 0 | 6 |
| 2-2-1-2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
| 2-2-2-1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 2 |
| 2-2-2-2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
| 108 | |||||||||||
Acknowledgements
We would like to thank the fantastic teachers who helped develop and enacted HS-CLUE. We would also like to thank Rosemary Russ for thoughtful dialogue that contributed to our understanding of epistemological messaging. Support for this work was provided by the U.S. National Science Foundation under Grant No. DRL-2003680.References
- Anderson C. W., de los Santos E. X., Bodbyl S., Covitt B. A., Edwards K. D., Hancock J. B., et al., (2018), Designing educational systems to support enactment of the Next Generation Science Standards, J. Res. Sci. Teach., 55(7), 1026–1052.
- Ball D. L., (1999), New tools for research on instruction and instructional policy: A web-based teacher log by center for the study of teaching and policy center for the study of teaching and policy, Educ. Res., 1999(December).
- Barbera J., Adams W. K., Wieman C. E. and Perkins K. K., (2008), Modifying and validating the Colorado Learning Attitudes about Science Survey for use in chemistry, J. Chem. Educ., 85(10), 1435.
- Becker N. M. and Cooper M. M., (2014), College chemistry students’ understanding of potential energy in the context of atomic-molecular interactions, J. Res. Sci. Teach., 51(6), 789–808.
- Berland L. and Hammer D., (2012a), Students’ framings and their participation in scientific argumentation, in Khine M. S. (ed.), Perspectives on scientific argumentation, Dordrecht: Springer Netherlands, pp. 73–93.
- Berland L. K. and Hammer D., (2012b), Framing for scientific argumentation, J. Res. Sci. Teach., 49(1), 68–94.
- Berland L. K., Schwarz C. V., Krist C., Kenyon L., Lo A. S. and Reiser B. J., (2016), Epistemologies in practice: Making scientific practices meaningful for students, J. Res. Sci. Teach., 53(7), 1082–1112.
- Brown M. W., (2009), The teacher-tool relationship: Theorizing the design and use of curriculum materials, in Remillard J. T., Herbel-Eisenmann B. A. and Lloyd G. M. (ed.), Mathematics Teachers at Work: Connecting Curriculum Materials and Classroom Instruction, New York, NY: Routledge, pp. 17–36.
- Burke K., (1966), Language as symbolic action: Essays on life, literature, and method, Berkeley, CA: University of California Press.
- Chase W. G. and Simon H. A., (1973), Perception in chess, Cogn. Psychol., 4, 55–81.
- Chen Y. C., (2022), Epistemic uncertainty and the support of productive struggle during scientific modeling for knowledge co-development, J. Res. Sci. Teach., 59(3), 383–422.
- Chi M. T., Feltovich P. J. and Glaser R., (1981), Categorization and representation of physics problems by experts and novices, Cogn. Sci., 5, 121–152.
- Chinn C. A. and Malhotra B. A., (2002), Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks, Sci. Educ., 86(2), 175–218.
- Chinn C. A., Buckland L. A. and Samarapungavan A., (2011), Expanding the dimensions of epistemic cognition: Arguments from philosophy and psychology, Educ. Psychol., 46(3), 141–167.
- Cohen J., (1960), A coefficient of agreement for nomial scale, Educ. Psychol. Meas., 1, 37–46.
- Cooper M. and Klymkowsky M., (2013), Chemistry, life, the universe, and everything: A new approach to general chemistry, and a model for curriculum reform, J. Chem. Educ., 90(9), 1116–1122.
- Cooper M. M., Williams L. C. and Underwood S. M., (2015), Student understanding of intermolecular forces: A multimodal study, J. Chem. Educ., 92(8), 1288–1298.
- Cooper M. M., Kouyoumdjian H. and Underwood S. M., (2016), Investigating students’ reasoning about acid–base reactions, J. Chem. Educ., 93(10), 1703–1712.
- Davis E. A., Janssen F. J. J. M. and Van Driel J. H., (2016), Teachers and science curriculum materials: where we are and where we need to go, Stud. Sci. Educ., 52(2), 127–160.
- Ding L. and Mollohan K. N., (2015), How college-level introductory instruction can impact student epistemological beliefs, J. Coll. Sci. Teach., 44(4), 19–27.
- DiSessa A., (1993), Toward an epistemology of physics, Cogn. Instr., 10(2), 105–225.
- Duschl R., (2008), Science Education in Three-Part Harmony: Balancing Conceptual, Epistemic, and Social Learning Goals, Rev. Res. Educ., 32(1), 268–291.
- Edelson D. C., Reiser B. J., McNeill K. L., Mohan A., Novak M., Mohan L., Affolter R., McGill T. A. W., Buck Bracey Z. E., Noll J. D., Kowalski S. M., Novak D., Lo A. S., Landel C., Krumm A., Penuel W. R., Van Horne K., González-Howard M. and Suárez E., (2021), Developing research-based instructional materials to support large-scale transformation of science teaching and learning: The approach of the OpenSciEd middle school program, J. Sci. Teach. Educ., 32(7), 780–804.
- Engle R. A., (2006), Framing interactions to foster generative learning: A situative explanation of transfer in a community of learners classroom, J. Learn. Sci., 15(4), 451–498.
- Ford M. J., (2005), The game, the pieces, and the players: Generative resources from two instructional portrayals of experimentation, J. Learn. Sci., 14(4), 449–487.
- Ford M. J., (2015), Educational implications of choosing “practice” to describe science in the next generation science standards, Sci. Educ., 99(6), 1041–1048.
- Furtak E. M. and Penuel W. R., (2019), Coming to terms: Addressing the persistence of “hands-on” and other reform terminology in the era of science as practice, Sci. Educ., 103(1), 167–186.
- Gouvea J. and Passmore C., (2017), ‘Models of’ versus ‘Models for’: Toward an agent-based conception of modeling in the science classroom, Sci. Educ., 26(1–2), 49–63.
- Hammer D. and Elby A., (2002), in Hofer B. K. and Pintrich P. R. (ed.), On the Form of a Personal Epistemology, Mahwah, New Jersey: Erlbaum.
- Hammer D., Elby A., Scherr R. E. and Redish E. F., (2005), Resources, framing, and transfer, in Mestre J. (ed.), Transfer of Learning: Research and Perspectives, Greenwich, CT: Information Age Publishing, pp. 89–119.
- Harris C. J., Penuel W. R., D’Angelo C. M., DeBarger A. H., Gallagher L. P., Kennedy C. A., et al., (2015), Impact of project-based curriculum materials on student learning in science: Results of a randomized controlled trial, J. Res. Sci. Teach., 52(10), 1362–1385.
- Haverly C., Calabrese Barton A., Schwarz C. V. and Braaten M., (2020), “Making space”: How novice teachers create opportunities for equitable sense-making in elementary science, J. Teach. Educ., 71(1), 63–79.
- Hofer B. K. and Pintrich P. R., (1997), The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning, Rev. Educ. Res., 67(1), 88–140.
- Hutchison P. and Hammer D., (2010), Attending to student epistemological framing in a science classroom, Sci. Educ., 94(3), 506–524.
- Inouye M., Houseal A. and Gunshenan C., (2020), Beyond the hook: What is a phenomenon and how is it used? Sci. Teach., 59–63.
- Irving P. W., Martinuk M. S. and Sayre E. C., (2013), Transitions in students’ epistemic framing along two axes, Phys. Rev. Spec. Top. – Phys. Educ. Res., 9(1), 1–11.
- Jaber L. Z., Dini V. and Hammer D., (2022), “Well that's how the kids feel!”—Epistemic empathy as a driver of responsive teaching, J. Res. Sci. Teach., 59(2), 223–251.
- Jiménez-Aleixandre M. P., Bugallo Rodríguez A. and Duschl R. A., (2000), “Doing the lesson” or “doing science”: Argument in high school genetics, Sci. Educ., 84(6), 757–792.
- Kang H., Windschitl M., Stroupe D. and Thompson J., (2016), Designing, launching, and implementing high quality learning opportunities for students that advance scientific thinking, J. Res. Sci. Teach., 53(9), 1316–1340.
- Kararo A., Colvin R., Cooper M. M. and Underwood S. M., (2019), Predictions and constructing explanations: An investigation into introductory chemistry students’ understanding of structure–property relationships, Chem. Educ. Res. Pract., 20, 316–328.
- Ke L. and Schwarz C. V., (2019), Using epistemic considerations in teaching: fostering students’ meaningful engagement in scientific modeling, in U. Z. Belzen A., Kruger D. and Van Driel J. (ed.), Towards a competence-based view on models and modeling in science education, Switzerland: Springer International Publishing, pp. 181–199.
- Ke L. and Schwarz C. V., (2020), Supporting students’ meaningful engagement in scientific modeling through epistemological messages: A case study of contrasting teaching approaches, J. Res. Sci. Teach., (58), 335–365.
- King P. M. and Kitchener K. S., (1994), Developing reflective judgement: Understanding and promoting intellectual growth and critical thinking in adolescents and adults, Jossey-Bass.
- Ko M. L. M. and Krist C., (2019), Opening up curricula to redistribute epistemic agency: A framework for supporting science teaching, Sci. Educ., 103(4), 979–1010.
- Koslowski B., (1996), Theory and evidence: The development of scientific reasoning, The MIT Press.
- Kuhn D., (1999), A developmental model of critical thinking, Educ. Res., 28(2), 16–46.
- Kuhn D. and Pease M., (2008), What needs to develop in the development of inquiry skills? Cogn. Instr., 26(4), 512–559.
- Lemke J., (1990), in Green J. (ed.), Talking science: language, learning, and values, Norwood, NJ: Ablex Publishing Corporation.
- Lising L. and Elby A., (2005), The impact of epistemology on learning: A case study from introductory physics, Am. J. Phys., 73(4), 372–382.
- Lowell B. R., Cherbow K. and McNeill K. L., (2020), Redesign or relabel? How a commercial curriculum and its implementation oversimplify key features of the NGSS, Sci. Educ., 1–28.
- Manz E., (2015), Resistance and the development of scientific practice: designing the mangle into science instruction, Cogn. Instr., 33(2), 89–124.
- Manz E., Lehrer R. and Schauble L., (2020), Rethinking the classroom science investigation, J. Res. Sci. Teach., 57(7), 1148–1174.
- McNeill K. L. and Berland L., (2017), What is (or should be) scientific evidence use in k-12 classrooms? J. Res. Sci. Teach., 54(5), 672–689.
- McNeill K. L., González-Howard M., Katsh-Singer R. and Loper S., (2016), Pedagogical content knowledge of argumentation: Using classroom contexts to assess high-quality PCK rather than pseudoargumentation, J. Res. Sci. Teach., 53(2), 261–290.
- McNeill K. L., GonzÁlez-Howard M., Katsh-Singer R. and Loper S., (2017), Moving beyond pseudoargumentation: Teachers’ enactments of an educative science curriculum focused on argumentation, Sci. Educ., 101(3), 426–457.
- Miller E., Manz E., Russ R., Stroupe D. and Berland L., (2018), Addressing the epistemic elephant in the room: Epistemic agency and the next generation science standards, J. Res. Sci. Teach., 55(7), 1053–1075.
- Minstrell J., (1992), Facets of students’ knowledge and relevant instruction, in Research in physics learning: Theoretical issues and empirical studies, pp. 110–128.
- National Research Council, (1999), How People Learn: Brain, Mind, Experience, and School, in Bransford J. D., Brown A. L. and Cocking R. R. (ed.), Washington, DC: National Academy Press.
- National Research Council, (2010), Preparing Teachers: Building Evidence for Sound Policy, Washington, DC: The National Academies Press.
- National Research Council, (2012), A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas, Washington, DC: The National Academies Press.
- National Research Council, (2013), in Pellegrino J. W. and Hilton M. L. (ed.), Education for life and work: Developing transferable knowledge and skills in the 21st century, Skills C. on D. D. L. and 21st C., Washington, DC: The National Academies Press.
- NGSS Lead States, (2013), Next Generation Science Standards: For States, by States (Appendix F – Science and Engineering Practices), in Achieve, Inc. behalf twenty-six states partners that Collab. NGSS, (November), pp. 1–103.
- Odden T. O. B. and Russ R. S., (2019a), Defining sensemaking: Bringing clarity to a fragmented theoretical construct, Sci. Educ., 103(1), 187–205.
- Odden T. O. B. and Russ R. S., (2019b), Vexing questions that sustain sensemaking, Int. J. Sci. Educ., 41(8), 1052–1070.
- Osborne J., (2014), Teaching scientific practices: Meeting the challenge of change, J. Sci. Teach. Educ., 25(2), 177–196.
- Penuel W. R., Turner M. L., Jacobs J. K., Horne K. and Sumner T., (2019), Developing tasks to assess phenomenon-based science learning: Challenges and lessons learned from building proximal transfer tasks, Sci. Educ., 103(6), 1367–1395.
- Perkins K. K., Barbera J., Adams W. K. and Wieman C. E., (2007), Chemistry vs. physics: A comparison of how biology majors view each discipline, in AIP Conference Proceedings, pp. 53–56.
- Phillips A. M. L., Watkins J. and Hammer D., (2018), Beyond “asking questions”: Problematizing as a disciplinary activity, J. Res. Sci. Teach., 55(7), 982–998.
- Ralph V. R., Scharlott L. J., Schwarz C. E., Becker N. M. and Stowe R. L., (2022), Beyond instructional practices: Characterizing learning environments that support students in explaining chemical phenomena, J. Res. Sci. Teach., 59(5), 841–875.
- Redish E. F., (2004), A Theoretical Framework for Physics Education Research: Modeling Student Thinking.
- Reiser B. J., (2013), What Professional Development Strategies Are Needed for Successful Implementation of the Next Generation Science Standards? in Invit. Res. Symp. Sci. Assess., (September), 1–22.
- Reiser B. J., Novak M. and Mcgill T. A. W., (2017), Coherence from the students’ perspective: Why the vision of the Framework for K-12 Science requires more than simply “combining” three dimensions of science learning, pp. 1–11.
- Rosenberg S., Hammer D. and Phelan J., (2006), Multiple epistemological coherences in an eighth-grade discussion of the rock cycle, J. Learn. Sci., 15(2), 261–292.
- Rowan B. and Correnti R., (2009), Studying reading instruction with teacher logs: Lessons from the study of instructional improvement, Educ. Res., 38(2), 120–131.
- Russ R. S., (2014), Epistemology of science vs. epistemology for science, Sci. Educ., 98(3), 388–396.
- Russ R. S., (2018), Characterizing teacher attention to student thinking: A role for epistemological messages, J. Res. Sci. Teach., 55(1), 94–120.
- Russ R. S. and Luna M. J., (2013), Inferring teacher epistemological framing from local patterns in teacher noticing, J. Res. Sci. Teach., 50(3), 284–314.
- Russ R. S., Scherr R. E., Hammer D. and Mikeska J., (2008), Recognizing mechanistic reasoning in student scientific inquiry: A framework for discourse analysis developed from philosophy of science, Sci. Educ., 92(3), 499–525.
- Russ R. S., Coffey J. E., Hammer D. and Hutchison P., (2009), Making classroom assessment more accountable to scientific reasoning: A case for attending to mechanistic thinking, Sci. Educ., 93(5), 875–891.
- Ryu S. and Sandoval W. A., (2012), Improvements to elementary children's epistemic understanding from sustained argumentation, Sci. Educ., 96(3), 488–526.
- Schafer A. G. L. and Yezierski E. J., (2021), Investigating how assessment design guides high school chemistry teachers’ interpretation of student responses to a planned, formative assessment, J. Chem. Educ., 98(4), 1099–1111.
- Scherr R. E. and Hammer D., (2009), Student behavior and epistemological framing: Examples from collaborative active-learning activities in physics, Cogn. Instr., 27(2), 147–174.
- Schwarz C. V. and White B. Y., (2005), Metamodeling knowledge: Developing students’ understanding of scientific modeling, Cogn. Instr., 23(2), 165–205.
- Schwarz C. V., Passmore C. and Reiser B. J. (ed.), (2017), Helping students make sense of the world, Arlington, VA: NSTA Press.
- Sensevy G., Tiberghien A., Santini J., Laubé S. and Griggs P., (2008), An epistemological approach to modeling: Cases studies and implications for science teaching, Sci. Educ., 92(3), 424–446.
- Shar K., Russ R. and Laverty J., (2020), Student epistemological framing on paper-based assessments, Phys. Rev. Phys. Educ. Res., 16(2), 20101.
- Sherin B., (2006), Common sense clarified: The role of intuitive knowledge in physics problem solving, J. Res. Sci. Teach., 43(6), 535–555.
- Stowe R. L., Herrington D. G., McKay R. L. and Cooper M. M., (2019a), Adapting a core-idea centered undergraduate general chemistry curriculum for use in high school, J. Chem. Educ.
- Stowe R. L., Herrington D. G., McKay R. L. and Cooper M. M., (2019b), The impact of core-idea centered instruction on high school students’ understanding of structure-property relationships, J. Chem. Educ.
- Tomanek D., Talanquer V. and Novodvorsky I., (2008), What do science teachers consider when selecting formative assessment tasks? J. Res. Sci. Teach., 45(10), 1113–1130.
- Weizman A., Shwartz Y. and Fortus D., (2008), The driving question board, Sci. Teach., 33–37.
- Wertsch J. V., (1998), Mind as action, New York, NY: Oxford University Press.
| This journal is © The Royal Society of Chemistry 2023 |