View PDF VersionPrevious ArticleNext Article
 
DOI: 10.1039/D0RP00313A (Paper) Chem. Educ. Res. Pract., 2021, 22, 431-442

Changes in teaching beliefs of early-career chemistry faculty: a longitudinal investigation†

Maia Popova a, Annika Kraft b, Jordan Harshman c and Marilyne Stains *b
aUniversity of North Carolina at Greensboro, Greensboro, North Carolina, USA. E-mail: mstains@virginia.edu
bUniversity of Virginia, Charlottesville, Virginia, USA
cAuburn University, Auburn, Alabama, USA

Received 15th October 2020 , Accepted 7th January 2021

First published on 27th January 2021

Abstract

Literature at the secondary level has demonstrated a tight interconnectedness between one's beliefs about teaching and learning and one's instructional practices. Moreover, this research indicates that personal and contextual factors influence beliefs and that growth and changes in beliefs are most notable during the early years of one's teaching experience. Despite the substantial influence of teaching beliefs on educational decisions, very little research has been conducted at the post-secondary level in both characterizing and monitoring changes in beliefs over time of early-career faculty members. This study aims to fill this gap by investigating (1) the changes over two and half years in the beliefs of early-career chemistry professors in the United States, and (2) patterns between changes in beliefs and personal and contextual factors as defined in the Teacher-Centered Systemic Reform Model. Nine faculty were interviewed using the modified Luft and Roehrig's Teaching Beliefs Interview protocol in Fall 2016/Spring 2017 and then again in Spring 2019. Combination of constant-comparative analysis and cluster analysis were utilized to characterize faculty beliefs after each data collection cycle. Faculty also completed four surveys over the course of this longitudinal study. These surveys were analyzed to identify personal and contextual factors that could relate to changes in faculty beliefs over time. Overall, the participants expressed more unique beliefs about teaching and learning during the second interview. Despite this increase, the substance and the message of the beliefs remained fairly similar to the beliefs expressed during the first interview, which suggests that beliefs do not change as an artifact of teaching experience. Four of the faculty demonstrated a desirable shift to student-centered thinking, while three did not change and two shifted toward teacher-centered. Analysis of the survey data revealed that access and use of chemical education research journal and researchers, repeated opportunities to teach the same course, and instructor's continued learning efforts with respect to teaching were more pronounced among faculty who shifted toward student-centered thinking.

Introduction

Teaching beliefs have been defined as “tacit, often unconsciously held assumptions about students, classrooms, and the academic material to be taught” (Kagan, 1992, p. 65). The empirical evidence presented in the educational literature, particularly at the secondary level, indicates that beliefs that instructors hold about teaching and learning affect their instructional practices (Pajares, 1992; Feyzioğlu, 2012; Enderle et al., 2014; Dolphin and Tillotson, 2015; Wong and Luft, 2015; Southerland et al., 2016; Gibbons et al., 2018, Şen and Sarı, 2018). Characterizing instructors’ beliefs, their connections to instructional practices and factors that influence them is thus critical to advancing instructional change.

Current studies on teaching beliefs, which have mostly been conducted at the secondary level (Fletcher and Luft, 2011; Luft et al., 2011; Wong and Luft, 2015), fall mainly into three categories: characterization of beliefs (Hora, 2014; Mavhunga and Rollnick, 2016; Chapman and McConnell, 2018; Lee, 2019; Pratt and Yezierski, 2019), measure of the impact of instructional reforms on participants’ beliefs (Mattheis and Jensen, 2014; Moore et al., 2015; Pelch and McConnell, 2016; Czajka and McConnell, 2019; Lee, 2019), and exploration of the relationship between beliefs and practice (Addy and Blanchard, 2010; Dolphin and Tillotson, 2015; Czajka and McConnell, 2016; Douglas et al., 2016; Şen and Sarı, 2018; Popova et al., 2020). Most studies aiming to characterize teaching beliefs place beliefs on the continuum from teacher-centered to student-centered. Teacher-centered beliefs generally reflect the idea that students learn from listening to an instructor teach (beliefs that support didactive mode of instruction), whereas student-centered beliefs reflect the ideas that students learn when conceptually engaged with the content and when interacting with their peers to construct understanding (beliefs that support active forms of instruction) (Luft and Roehrig, 2007). Transitional beliefs represent the midpoint along this continuum. Several of these studies have explored change in beliefs over time but through cross-sectional samples rather than following longitudinally the same group of instructors. This is likely due to the understandable difficulty associated with collecting and analyzing longitudinal data, but the absence of longitudinal data prevents knowledge generation about how instructors change over time.

Research has demonstrated the importance of characterizing the changes over time of the beliefs of novice or inexperienced instructors since these beliefs are still developing and more malleable than those of experienced instructors (Luft, 2001). For example, Wong and Luft (2015) followed 35 beginning secondary science instructors over the course of five years and they found that instructors with more student-centered beliefs were more likely to persist in teaching as their career. Two longitudinal studies demonstrated the influence of a lack of continuous professional development on new teachers’ beliefs about teaching and learning. Luft and colleagues (Luft et al., 2011) followed 98 beginning secondary science teachers over the course of two years to investigate the changes in their beliefs, pedagogical content knowledge (PCK), and teaching practices after exposure to different professional development programs. They found that most of the teachers’ beliefs were more student-centered during the first year, while they were provided on-going support by the professional development programs, but shifted toward more instructor-centered in the second year, during which no support was provided. Similarly, Fletcher and Luft (2011) followed five prospective secondary science instructors that participated in a university preparation program over the course of three years and found that their participants held more student-centered beliefs during the program, but shifted to instructor-centered thinking in their first year of teaching. To our knowledge, few studies have investigated changes in new STEM faculty members’ beliefs about teaching and learning as they are gaining teaching experience.

Although one may assume that secondary teachers and faculty members hold similar beliefs, their experiences and working contexts are vastly different. For example, Fletcher and Luft (2011) emphasized that each participant in their study held a unique set of beliefs because beliefs are shaped by the individual experiences, backgrounds, and training of each educator. Moreover, several contemporary empirical models on instructional change, including the Interconnected Model of Instructor Professional Growth (Clarke and Hollingsworth, 2002) and the Teacher-Centered Systemic Reform Model (Gess-Newsome et al., 2003), highlight the influence of contextual factors (e.g., environment) on instructors’ beliefs about teaching and learning. Characterizing changes in beliefs over time of new STEM faculty and factors that could have influenced these changes would be extremely valuable to identify support mechanisms and professional development programs that would help develop and sustain student-centered beliefs among new generations of faculty members. This study starts to address this need by answering the following research questions:

1. How do the teaching beliefs of early career chemistry faculty members change over time?

2. To what extent do personal and contextual factors relate to changes observed over time in a sample of early career chemistry faculty members’ beliefs?

Theoretical framework

The overall system of beliefs that an individual holds is composed of many beliefs that develop over time. Earlier beliefs in this network are held more strongly, are resistant to change, and have a great influence on the processing and recalling of the stored information, as they serve as a filter through which new phenomena is interpreted (Pajares, 1992). In contrast to knowledge, beliefs often lack an internal consistency with the overall system of beliefs that one holds because of their subjectivity. Beliefs are subjective by nature because they are based on evaluation, judgement, bias, and generalizations drawn from personal experiences. Despite this, some have argued that beliefs are much more influential than knowledge when explaining behavior (Nespor, 1987).

Models for instructional reforms have emphasized the roles that personal factors can have in shaping one's beliefs but also the influence of beliefs on instructional practices (Clarke and Hollingsworth, 2002; Gess-Newsome et al., 2003). In this study, we focus on the Teacher-Centered Systemic Reform Model (TCSR, Fig. 1) because this model places instructor beliefs (along with instructor knowledge) at its core under the label teacher thinking. It connects beliefs to not only personal factors (i.e., demographic profile, types and years of teaching experience, nature and extent of instructors’ preparation to teach, and nature and extent of instructors’ continued learning efforts) and instructor's practices but also to contextual factors (i.e., cultural context, school context, department and subject area context, and classroom context). The TCSR Model was developed by Woodbury and Gess-Newsome (2002) from their synthesis of the perspectives on reform in secondary education research. The model illustrates “interdependent elements of reform” which reflects the view of education as a system (Woodbury and Newsome, 2002). Several factors (personal, contextual, teacher thinking, and practice) are interconnected in this model to explain the multi-faceted systemic nature of education and the need to consider these interconnected factors when a change/reform is to be implemented. One of the main assumptions of the TCSR model is that change in instructional practices will result in better student outcomes. As previous studies have shown (Andrews et al., 2011), adoption of new instructional practices does not always result in enhanced student outcomes since faculty often adapt the innovation to fit their context and in doing so remove critical elements that made the innovation effective (Stains and Vickrey, 2017).


image file: d0rp00313a-f1.tif
Fig. 1 Teacher-centered systemic reform model (TCSR). Reproduced with permission from Gess-Newsome et al., 2003.

Methods

Data collection and sample

Thirteen assistant chemistry professors participated in semi-structured, think-aloud interviews (Drever, 1995; Patton, 2002) and provided surveys over the course of two and half years (IRB Number: 20151115802 EX). The timeline for data collection is presented in Fig. 2.
image file: d0rp00313a-f2.tif
Fig. 2 Timeline describing the collections of surveys and interviews for the participants involved in this study.

The first set of interviews was collected in Fall 2016 and Spring 2017. Faculty members were recruited while attending the Cottrell Scholars Collaborative New Faculty Workshop (CSC NFW, Baker et al., 2014). All faculty participants were from Master's and Doctoral Universities according to the Carnegie's classification (Center for Postsecondary Research). A code number was created for each participant to protect their identity. The modified Instructor Belief Interview (TBI) protocol was used to elicit their beliefs about teaching and learning (Luft and Roehrig, 2007). Although this interview protocol was developed within the K-12 context, it has been used in higher education contexts (Czajka and McConnell, 2016; Chapman and McConnell, 2018; Czajka and McConnell, 2019). The protocol was modified because two questions in the original protocol elicited redundant responses, “How do your students learn science best?” and “How do you maximize student learning in your classroom?” For this reason, we excluded the former question. Details pertaining to the first interview data collection procedures have been previously reported (Popova et al., 2020). What follows is a detailed description of the second interview data collection procedures which strongly resembles the first interview data collection process.

In Spring 2019, these thirteen chemistry professors were invited to participate in a second interview that utilized the same modified Instructor Belief Interview (TBI) protocol (the interview protocol can be found in the ESI†). All but four (P1, P2, P3 and P13) of the original thirteen participants agreed to participate in the second interview. These nine faculty were located across eight different states in the US, spanning all four of the regional divisions recognized by the US Census Bureau (U.S. Department of Commence): 3 universities in the South, 3 in the West, 2 in the Northeast, and 1 in the Midwest. Because this population was geographically diverse, multimedia-based programs (e.g., Skype, Zoom) were used to interview the faculty participants. An audio recorder was used to capture the data. Demographic details for the sample are shown in Table 1. Note that chemistry course or level (undergraduate or graduate) taught by faculty members changed for four (P6, P8, P9, P11) of the participants going from the first to second interview (e.g., P6 taught an undergraduate biochemistry course in Fall 2016 and a graduate biochemistry course in Spring 2019). Each faculty received a $50 gift card for participating in the second interview.

Table 1 Descriptive demographics for the sample. Note that G stands for a graduate course and U stands for an undergraduate course
ID Gender First interview Second interview
Year teaching Chemistry course Year teaching Chemistry course
P4 M First Bioanalytical, G Third Bioanalytical, G
P5 F Second Inorganic, U Fourth Inorganic, U
P6 M Second Biochemistry, G Fourth Biochemistry, U
P7 F Third Biochemistry, U Fifth Biochemistry, U
P8 F Second Analytical, G Fourth Analytical, U
P9 F Third Inorganic, U Fifth General, U
P10 M First Biochemistry, U Third Biochemistry, U
P11 F First Organic, G Third Organic, U
P12 F Second Analytical, U Fourth Analytical, U

Faculty members also completed several surveys (pre, post workshop, as well as 1- and 3-year following participation in the workshop). One of the goals of the surveys was to capture faculty characteristics within factors of the TCSR model; of particular interest for this study were the questions targeting personal and contextual factors as these could relate to the changes in beliefs over time (for survey items used in this study, see ESI†).

Data analysis

Interviews. Analysis of the second interview data followed the same steps as the analysis of the first interview data previously described in details (Popova et al., 2020), but the general process will be outlined here. Once transcribed verbatim, the interviews were read to identify quotes that communicated beliefs about teaching and learning (i.e., not descriptions of what chemistry professors are doing in their classrooms, but explanations of why they are doing it). All of the identified quotes were uploaded to NVivo 12 to be stored, organized, and deductively coded (Patton, 2002; Creswell, 2003; Bazeley and Jackson, 2013) using a code book that captured faculty participants’ Beliefs about Students, Beliefs about Content, and Beliefs about How Students Learn. This code book was developed while analyzing the first interview data from our faculty participants (Popova et al., 2020). Three coders used the rubric to code the first interview data and, after multiple debriefing sessions, they achieved a 100% interpretive convergence (Saldaña, 2013). The team coding, or what Saldaña describes as “analyst triangulation,” ensured reliability and credibility of the results, as frequent debriefing sessions helped the researchers address any biases and assumptions brought to the interpretative analysis (Saldaña, 2013; Pandey and Patnaik, 2014). After ensuring a complete agreement on the collective coding of the first interview data, the first author independently coded the second interview data. Any beliefs previously not expressed during the first interviews were captured to augment the original code book. The first author met weekly with the second and fourth authors to discuss the process of coding (especially any differences between the first and second sets of interviews). The discussions during these periodic meetings helped further the confidence in the rigor of findings and establish credibility of the results (Saldaña, 2013). The final code book for the first and second interview data is shown in Table 2.
Table 2 Themes, categories, and codes that capture beliefs of early-career chemistry faculty. Note that the total number of instances for each theme is greater than the number of faculty in this study (N = 9) because one participant could express multiple beliefs and, therefore, be assigned to multiple codes within one theme
Category Code/belief First Second
Theme I: beliefs about students
Highlighting student differences Different students put in different level of effort 2 6
Different students possess different ability to grasp the material and the instructor does not aim to reach all students 2 1
Different students possess different ability to grasp the material, but with help students can get better 1 3
International students are reluctant to participate 1 0
Non-major students are intimidated by chemistry 1 0
Active learning is not uniformly beneficial for all students 0 2
General to all students Students need to assume responsibility for their learning 2 5
Humans have limited attention spans/working memory capacity 2 0
Students like lecturing 0 2
Students think they know more than they actually do 0 1
Students lack productive study habits 0 1
Students do not like active learning 0 1
Theme II: beliefs about how students learn
Mechanisms through which learning occurs Learn better by doing/thinking, not listening 5 3
By listening to the instructor 2 4
When making connections between concepts 2 1
When applying their knowledge 2 2
By paying attention 1 5
By repetition 1 5
When being conceptually engaged 1 4
When being conceptually challenged 1 0
When being reflective 0 2
When actively taking notes 0 1
Context in which learning occurs Can learn from each other 4 5
Can learn outside of class 2 1
Learn best with instructor's guidance 1 0
Learn best in a positive classroom environment 0 1
Theme III: beliefs about content
The goal is student understanding, not content coverage Focus on foundational concepts 3 5
Teaching too much content is bad for students 2 2
Curriculum is a flexible agenda 1 3
Selection of content to prepare students for their future Incorporating literature or authentic content 2 2
Real-world applications of what students learn 2 6
Exposing students to a broad range of topics 1 0
Content that will make students more interested 1 5
Curriculum is a fixed agenda 3 3

While it is informative to detect and contrast beliefs via constant comparative analysis, additional use of quantitative methods can enrich the evidence and enable research question one to be answered more deeply. Therefore, in order to characterize patterns across the different types of beliefs about teaching and learning expressed by our faculty participants, an agglomerative hierarchical cluster analysis was conducted. Following the same procedure as when performing the cluster analysis for the first interview data (Popova et al., 2020), the nine faculty who participated in the second interviews were clustered based on their Beliefs about Students, Beliefs about Content, and Beliefs about How Students Learn (capitalization indicates name of themes referred throughout the rest of the manuscript). By running a matrix coding query in NVivo, a table illustrating what beliefs were expressed by each participant was generated, in which “0” indicated absence of a belief and “1” indicated presence of a belief. This categorical, nominal data was uploaded to IBM SPSS 25 to perform the cluster analysis. The agglomerative procedure began with each participant representing an individual cluster and then successively merging clusters together until a hierarchy of nested groupings was created (Frades and Matthiensen, 2010). Since faculty were classified solely on the patterns in their beliefs and elevation of scores across variables of interest was not applicable, Pearson's correlation similarity measure was selected to measure the association between the variables (Clatworthy et al., 2005; Wilks, 2014). Cluster analysis results for the first interview data have been previously reported (Popova et al., 2020) and results for the second interview data are shown below.

Surveys. Surveys collected information capturing faculty characteristics and institutional environments aligned with the personal and contextual factors within the TCSR Model. Personal factors examined included faculty sex, teaching experience, familiarity with evidence based instructional practices (EBIPs), and continued learning (seen as attendance in professional development). Contextual factors included institution, department, and course contexts (e.g., Carnegie classification, teaching load, type of course taught, and types of teaching-related resources available/accessed). Of note, only questions in the survey that had a minimum 50% response rate within each cluster were included in the analysis (questions that did not meet this threshold are provided in the ESI†). Survey responses within these factors were compared across patterns identified from the interview data to explore any potential relations that may explain any changes seen in beliefs over time. Due to the small sample size, no inferential analyses were conducted and only qualitative comparisons were made.

Results

Changes in the types of beliefs about teaching and learning

Overall, participants expressed a higher number of beliefs about teaching and learning during the second interviews (Table 2). During the first interviews faculty articulated an average of 6 unique beliefs (lowest observation equaled to 1 unique belief, whereas the highest observation equaled to 10) while during the second interviews, they expressed an average of 9 unique beliefs (lowest observation equaled to 5 unique beliefs, whereas the highest observation equaled to 11). Fig. 3 illustrates these changes for each faculty participant.
image file: d0rp00313a-f3.tif
Fig. 3 Number of beliefs about teaching and learning expressed by each participant during the first and second interviews.

In general, faculty participants expressed similar beliefs in both interviews. When new beliefs were articulated in the second interviews, they were not prevalent. For example, several new, mostly idiosyncratic beliefs were expressed about students: “students like lecturing” (n = 2), “active learning is not uniformly beneficial for all students” (n = 2), “students think they know more than they actually do” (n = 1), “students lack productive study habits” (n = 1), and “students do not like active learning” (n = 1). Three new beliefs were also expressed about how students learn: students learn “when being reflective” (n = 2), “when actively taking notes” (n =1), and students “learn best in a positive classroom environment” (n = 1). No new beliefs were expressed about content.

In respect to the prevalence of the beliefs that were articulated during both the first and second interviews, few noticeable changes were observed for Beliefs about Students, Beliefs about How Students Learn, and Beliefs about Content (Table 2). In respect to Beliefs about Students, two beliefs from the first interviews were expressed more prominently in the second interviews: “different students put in different level of effort” (from n = 2 to n = 6) and “students need to assume responsibility for their learning” (from n = 2 to n = 5). During the first interviews, the following codes were the most prevalent under the category of “mechanisms through which learning occurs,” which is a category under Beliefs about How Students Learn (Table 2): students learn better “by doing/thinking, not listening” (n = 5), “by listening to the instructor” (n = 2), “when making connections between concepts” (n = 2), and “when applying their knowledge” (n = 2). During the second interviews, however, the most prevalent beliefs under this category became “by paying attention” (n = 5), “by repetition” (n = 5), “when being conceptually engaged” (n = 4), and “by listening to the instructor” (n = 4). In respect to the Beliefs about Content, two beliefs that were not very prevalent during the first interviews were expressed with much greater frequency: teaching about “real-world applications of what students learn” (from n = 2 to n = 6) and selecting “content that will make students more interested” in the subject (from n = 1 to n = 5).

Types of belief systems in the second interviews

The cluster analysis was performed on the second interviews based on the identified codes under Beliefs about Students, Beliefs about How Students Learn, and Beliefs about Content. The analysis was conducted using Pearson's correlation similarity measure and the same cluster solution was obtained with Centroid Linkage and Ward's method, which suggests stability of the obtained cluster solution (a dendrogram illustrating the results of the analysis can be found in the ESI†). The qualitative examination of 2-, 3-, and 4-cluster solutions to identify the most homogeneous profiles of beliefs supported a 2-cluster solution as the solution that offers a clear interpretation of unique characteristics of each individual cluster. Additionally, to further ensure stability of the cluster solution, the cluster analysis was replicated several (7) additional times, every time mixing the order in which the objects existed in the database (Brandriet and Bretz, 2014; Harshman et al., 2017). The same cluster solution was obtained every time. The cluster analysis was used to examine the data for some key patterns that might not be readily identified using qualitative techniques only. While plausible, we do not present evidence or claim that these groupings will be observed in the broader professor population.

Shown in Table 3 are the comparative demographics for the two clusters. Table 4 depicts the prominent features of each cluster. Below, is a description of the patterns in the belief profiles of the two identified clusters across the three themes: beliefs about Students, Beliefs about How Students Learn, and Beliefs about Content.

Table 3 Descriptive demographics for each cluster
Demographic variables Cluster 1 n = 6 Cluster 2 n = 3
Sex
Female 4 2
Male 2 1
Course level taught
Graduate 1 0
Undergraduate 5 3
Year teaching
Third 1 2
Fourth 3 1
Fifth 2 0

Table 4 Patterns in the belief profiles of each cluster
Cluster label Beliefs about students Beliefs about how students learn Beliefs about content
1. Student-centered & consistent (n = 6) • Students possess different ability to grasp the material, but with help they can get better (n = 3) • Students can learn from each other (n = 5) • Teach about real-world applications of what students learn (n = 5)
• Content that will make students more interested (n = 4)
2. Instructor-centered & consistent (n = 3) • Students put in different level of effort (n = 3) • Students learn by paying attention (n = 3) • Incorporate literature or authentic content (n = 2)
• Students learn by listening to the instructor (n = 3) • Curriculum is a fixed agenda (n = 2)

Cluster 1: Student-centered and consistent beliefs (n = 6). In respect to the Beliefs about Students, half of the faculty in this cluster (n = 3) showed growth mindset beliefs, stating that students possess different ability to grasp the material, but with help they can improve. For example, when explaining why he assigns students to work in groups, P6 explained: “There is always a difference in proficiency. Some students catch things very quickly. Some students may take longer. So they kind of teach each other. Biochem is based on gen chem and organic chemistry concepts and not all of them are proficient in all of those concepts… When I get students to talk, some of them are better at organic and they will say – this is aldol condensation. Less proficient ones can ask why.” As is evident from this quote, P6 believes that students can teach each other and the “less proficient” students can improve in their understanding of concepts.

When it comes to the Beliefs about Content, most faculty in this cluster discussed the importance of connecting the content to students’ lives either by incorporating content that will make students more interested in the subject (n = 4) or by including real-world applications of what students learn (n = 5). This is evident in the quote of P8: “So my view is, hopefully if they see how this matters in medicine, and environmental science, and forensics, et cetera, they become more excited about it and want to really understand the techniques in the material better. Um, so a lot of examples that I try to give are related to those applications of the material.”

Finally, in respect to the Beliefs about How Students Learn, most (n = 5) noted that students can learn from each other: “Instructors tend to think that they are very useful when lecturing, but I don’t think that's true. I think that students tend to learn what they need from each other” (P8) and “Students learn not only from me, but also from peers. For example, the final assignment is group presentations… Students do the peer evaluation, peer review, they give each other some feedback. And what I always say to the students – learn from your peers” (P4). Thus, having considered the patterns in the beliefs across the three themes, this cluster was assigned the label “student-centered and consistent beliefs.”

Cluster 2: Instructor-centered and consistent beliefs (n = 3). In respect to the Beliefs about Students, all of the participants (n = 3) in this cluster discussed the issue that different students put in different levels of effort. For example, faculty explained that some students come to class unprepared – they do not do the assigned homework. For this reason, when it comes to the Beliefs about How Students Learn, the participants in this cluster stated that students learn by paying attention in class (n = 3) and by listening to the instructor (n = 3), because when students come to class unprepared, lecture is the most effective approach to get them “on the same page.” This is evident in the quote of P12: “For some students, for example those who don’t do the readings, who are lacking motivation, they’re struggling in the active learning teaching style… I feel that when I give lectures a little bit more, even those who didn’t do their homework, they didn’t do the reading or watch those videos, they will still have a basic idea about how this instrument works.” P11 also discussed that students learn well from an authority figure: “When they take this class, they don’t have much organic chemistry background. So I think it's nice for them to actually have someone to be essentially authority figure and then tell them these are things that are, that we should learn in this class.”

In respect to the Beliefs about Content, the faculty emphasized the importance of incorporating literature or authentic content (n = 2). For example, P12 stated: “Of course I will give them some other, uh, introductions about the modern techniques, for example some [inaudible word] and Raman spectroscopy, which are not covered in ACS, but uh, they're used a lot in research.” Finally, most faculty in this cluster (n = 2) saw curriculum as a fixed agenda, meaning that they would not alter the pace of content coverage: “I won't skip any chapters, um, for this class because I think everything is important. Um, especially when you have a second semester to take, if you miss one chapter it's gonna probably cause some issues in second semester, um, or even down the line, in the class” (P11). Having considered the patterns in the beliefs across the three themes and the fact that no participant stated that students can learn from each other (instead all believed that students learn best by paying attention and listening to the instructor), this cluster was assigned the label “instructor-centered and consistent beliefs.”

Changes in the types of belief systems over time

Cluster analysis of the first interview data grouped faculty into four distinct clusters (Fig. 4). “Student-centered and consistent beliefs” cluster believed that students learn best by doing/thinking instead of listening and that students learn best from interactions with their peers. “Transitional and consistent beliefs” cluster believed that students learn best from both paying attention to the instructor and from interacting with each other. “Instructor-centered and inconsistent beliefs” cluster believed that students learn best by doing/thinking instead of listening. At the same time, they also believed that students learn best by listening to the instructor lecture. Additionally, faculty in this cluster believed that depth of content coverage promotes conceptual understanding, while simultaneously believing that curriculum is a fixed agenda. Since these instructors simultaneously held competing ideas, their beliefs were labeled as inconsistent. Finally, “limited number of beliefs cluster” expressed very few beliefs in comparison to the faculty in the other three clusters and there were no patterns in their beliefs to characterize them on the continuum from instructor- to student-centered (Popova et al., 2020). Fig. 4 illustrates how participants changed in the cluster assignment from the first to second interview.
image file: d0rp00313a-f4.tif
Fig. 4 Changes in cluster membership between the first and second interviews.

As can be seen, after about two and a half years, some participants remained in their original clusters (i.e., P8 and P9 in the cluster of “student-centered beliefs” and P10 in the cluster of “instructor-centered beliefs”). Note that although when analyzing both the first and second interview data we identified a cluster of “instructor-centered beliefs”, this cluster slightly changed over time. Participants who fell under this cluster in the first interviews expressed not only instructor-centered beliefs, but also contradicting beliefs (i.e., stating that students learn better by doing/thinking instead of listening, but at the same time noting that students learn best when listening to the instructor). No such inconsistencies were identified in the beliefs profile of this cluster when analyzing the data from the second interviews.

All faculty who were initially in the cluster of “limited number of beliefs” shifted to other clusters, indicating a better ability on their part to describe their beliefs. During the first interviews, these faculty articulated an average of 3 unique beliefs (lowest observation equaled to 1 unique belief, whereas the highest observation equaled to 6) while during the second interview they expressed an average of 10 unique beliefs (lowest observation equaled to 9 unique beliefs, whereas the highest observation equaled to 11). P11 shifted to “instructor-centered beliefs;” P5 and P7 shifted to “student-centered beliefs.”

Two faculty showed desirable shifts from the first to second interviews: from “transitional beliefs” to “student-centered” for P4 and from “instructor-centered” to “student-centered” for P6. In the second interview, P4 no longer mentioned beliefs such as “students need broad exposure to a variety of topics” and “students learn by paying attention,” whereas P6 did not mention beliefs such as “curriculum is a fixed agenda” and “students learn by listening to the instructor.” P4's belief that “students learn from each other” remained constant from the first to second interview, whereas it was first mentioned by P6 in the second interview.

However, a shift from “student-centered” to “teacher-centered” was observed from P12. Going from the first to second interview, this participant no longer mentioned the following beliefs: “students can learn from each other,” “students learn with instructor's guidance” when engaged in activities, “too much content is bad for students,” and “depth of content coverage promotes conceptual understanding.” P12's belief that “different students put in different level of effort” remained constant from the first to second interview. However, the beliefs “students learn by listening to the instructor” and “by paying attention in class” were mentioned by P12 only in the second interview.

Relationship between changes in faculty beliefs and their personal and contextual factors

Our second research question aimed to explore factors that the TCSR model identifies as potential influencers on beliefs about teaching and learning. We define change in belief group as the shift between clusters of beliefs over time. Four faculty moved towards the “student-centered” cluster from the first to second interview and were denoted as the “Shifted to Student-Centered” group (n = 4). Some faculty remained in the same cluster thus showed no change and were denoted as the “Did Not Change” group (n = 3). Two faculty moved to “instructor-centered” thinking which comprised the “Shifted to Instructor-Centered” group (n = 2).

Participants completed several surveys as part of the workshop evaluation where faculty were originally recruited. These surveys collected information from within the factors in the TCSR Model and were analyzed to potentially explain changes in faculty thinking over time. This analysis focused on survey items determined to fit under the personal and contextual factors as these were inherent to the instructors themselves or their institutional context. The list of survey items corresponding to each factor and how each group of faculty (i.e., those who shifted toward student-centered, those who did not change and those who shifted toward instructor-centered) answered these items is presented in ESI.† Below we describe the items that indicated distinct patterns among the three group of faculty (Table 5).

Table 5 Patterns found within personal and contextual factors in changes of faculty beliefs over time
Factors Shifted to student-centered (N = 4) Did not change (N = 3) Shifted to instructor-centered (n = 2)
Contextual Broader cultural context Faculty in this group consult JCE papers more often than faculty in the other two groups (3/4 consulted 1–2 times per semester) Most faculty in these groups consult JCE less frequently than those in the other group (1 and 0 consulted 1–2 times per semester, respectively)
Department context In comparison to faculty in the other two groups, most (3/4) were able to consult CER faculty about teaching and did so about 1–2 times per year. 3/5 of faculty in these two groups did not have a CER colleague in their department and, therefore, were unable to ask them for advice on teaching.
Course context 3/4 did not change course contexts (i.e. same subject and course level) 3/5 had some changes (level and/or subject) in course taught between the first and second interview
Personal Nature and extent of instructor's preparation to teach Faculty in this group had the least exposure to EBIPs as students. Overall, faculty in this group had moderate experience with EBIPs as students. Faculty in this group had the most experience with EBIPs as students.
Nature and extent of instructor's continued learning efforts Very few faculty in this group attended any professional development prior to the CSC NFW. Faculty's familiarity with EBIPs prior to the CSC NFW hardly overlapped with those they experienced as a student, so it is unclear where or how these faculty learned about the additional EBIPs. Very few faculty in this group attended any professional developments prior to the CSC NFW. For most faculty in this group, over half of the EBIPs they were familiar with overlapped with those they experienced as a student, so most of their EBIPs knowledge came from their experience as students. But it is unclear where or how these faculty learned about the additional EBIPs. All faculty in this group had attended professional development prior to the CSC NFW. The majority of the EBIPs this group was familiar with overlapped with those they experienced as students. Even with attending professional development it seems these faculty did not gain much knowledge/experience of new EPIPs.
However, after attending the CSC NFW, they have the highest percent attendance of other professional developments.

In respect to the contextual factors, we found some similarities and a few interesting differences among the three change belief groups (see Table 5 and ESI† for more details). At the institutional context level, faculty of all groups were employed at a variety of institution classification types according to Carnegie Classification. In the broader cultural context, when it came to whether faculty sought advice to improve their teaching, those who “Shifted to Student-Centered” consulted papers from the Journal of Chemical Education more often than the faculty in the other two groups (data based on the year 3 survey). Additionally, the majority of faculty in the “Shifted to Student-Centered” group were able to consult Chemistry Education Research (CER) faculty within their department about teaching and did so about 1–2 times per year (data based on the year 3 survey). In contrast, most of the faculty in the other two groups did not have a CER colleague in their department and, therefore, were unable to ask CER specialists for advice on teaching. Due to the small sample size we make no claims that this is the reason why the “Shifted to Student-Centered” group displayed the desirable shift to more student-centered thinking. At the same time, it is interesting to note that perhaps the presence of these positions impacted individual faculty or department cultures.

As this was a longitudinal study and faculty may teach a variety of courses over time, we noted that five faculty indicated teaching different courses during the first and second interview. Examining change in beliefs by course context revealed some interesting findings. The majority of faculty in the “Shifted to Student-Centered” group did not change course contexts between the first and second interviews, meaning that these faculty taught the same course discipline and at the same level (graduate or undergraduate). This might explain their shift to more student-centered beliefs, as they had an opportunity to teach the same course several times and likely had more time to reflect on their practices, learn about students’ difficulties with content and test strategies to improve their learning. In contrast, less than half of the faculty in the other two groups taught the same course context between the first and second interviews. In fact, faculty in the “Did Not Change” group showed the most variety of change within course context, yet their beliefs remained unchanged, indicating that for these faculty their beliefs were seemingly independent from their course context. Future studies with larger sample size should further explore the extent to which repeated experiences in teaching the same course lead to shifts toward student-centered beliefs.

Across the personal factors investigated from those outlined in the TCSR Model, we saw some similarities between the change belief groups (see Table 5 and ESI†). Demographic evaluation showed that faculty in each change belief group contained a variety of teaching experience and nearly even sex representation. Most faculty within each group continued professional development between the first and second interviews (between Fall 2016/Spring 2017 and Spring 2019). The “Shifted to Student-Centered” group indicated that they had very little experience with EBIPs when they themselves were students. In addition, very few faculty in the “Shifted to Student-Centered” group attended any professional development prior to the CSC NFW. However, after attending this workshop, they participated in a higher number of other professional development opportunities in comparison to the faculty in the other two groups. These survey results highlight that in comparison to the faculty in the “Did Not Change” and “Shifted to Instructor-Centered” groups, faculty in the “Shifted to Student-Centered” group actively sought professional development. This is one potential reason that explains their shift to more student-centered beliefs over the course of this longitudinal study.

Limitations

While this study shed light on the changes over time in the beliefs about teaching and learning of early career chemistry faculty, the small sample size is its primary limitation. Having examined longitudinal changes in the beliefs of only nine faculty participants, we do not claim generalizability of our findings. The small sample size also does not allow for the use of cluster analysis in a predictive manner. Therefore, we used cluster analysis solely in an exploratory fashion, to allow for a deeper qualitative examination of patterns in the belief systems of the research participants. Although we examined several factors described in the TCSR Model, we could not explore several due to small sample size and did not explore faculty instructional practices. This was also due to the fact that nearly half of our faculty participants taught different courses during the longitudinal data collection.

Conclusions and implications

This study sought to identify how teaching beliefs of early-career chemistry professors change over time. Overall, the number of beliefs about teaching and learning increased from the first to second interview. This difference is particularly evident for faculty who were initially in the cluster of “limited number of beliefs” and who shifted to other clusters, indicating a better ability on their part to describe not only what they are doing in their classrooms, but also why they are doing it.

Despite the overall increase in the number of the articulated beliefs, the substance and the message of the beliefs remained fairly similar to the beliefs expressed during the first interviews about two and half years ago as suggested by almost identical code books from the first to second interviews. This implies that the sophistication of beliefs did not change as an artifact of additional teaching experience. Both, during the first and second interviews, participants expressed a range of beliefs, some better aligned with the literature on best practices of teaching and learning and some worse; however, most of the articulated beliefs lacked in depth. Additionally, the few new idiosyncratic beliefs that were expressed during the second interviews (e.g., “students like lecturing,” “active learning is not uniformly beneficial for all students,” and “students learn when actively taking notes”) were more instructor-centered in their nature (i.e., beliefs that support the transmission model of learning). This might be attributed to the complexities of the educational process and how numerous constraints and pressures affect instructor's thinking. This finding is similar to the finding of Fletcher and Luft (2011), who reported that the five prospective secondary science instructors in their study reverted to more traditional beliefs once in the classroom in their first year of teaching. They explained that some beliefs are held very tightly and remain consistent, whereas others can drastically change when exposed to the complexities of classroom teaching.

Although this study explored belief systems of a fairly homogeneous sample of participants, there was a noticeable variation in the sophistication of faculty's beliefs. These results highlight the need for instructional reform facilitators to recognize the diversity of beliefs present within a somewhat homogeneous group of instructors and differentiate the learning experiences accordingly. Based on the TCSR Model, various factors could be responsible for this variability: contextual factors (e.g., cultural, school, and classroom contexts) as well as personal factors (e.g., previous experiences as students and nature and extent of instructor's continued learning efforts) (Gess-Newsome et al., 2003). We found one aspect within the personal factors explored in this study that may have led faculty to change their thinking over time: more faculty who shifted toward student-centered beliefs had participated in professional development experiences after the CSC NFW. Additionally, we found a few notable differences within the contextual factor between change belief groups that may relate to change seen in faculty beliefs over time. In particular, the presence of a CER colleague in the department and faculty consultation of papers published in the Journal of Chemical Education were associated with faculty who shifted toward student-centered beliefs. Andrews and colleagues’ (2016) study found discipline-based education researchers (DBERs) to be agents of change in their departments. This may explain in part the change seen for those faculty who “Shifted to Student-Centered” as the majority had access to and reached out to CER faculty in their department to seek advice on their teaching.

A third of the sample in our study held on to their instructor-centered beliefs throughout the study, which also highlights the need for a continuous professional development that will challenge and cultivate beliefs that are better aligned with reform-based instructional practices. One participant held instructor-centered beliefs over the entire course of this study. As suggested by Pajares (1992), one key characteristic of beliefs is that the older beliefs are held more strongly and are resistant to change. This might suggest that without any professional development opportunities that challenge their thinking, beliefs of this instructor might remain instructor-centered over the course of their entire career. This is particularly concerning in the light of the results from a recent nationwide scale study that identified that chemistry instructors who held instructor-centered beliefs employed lecture-based teaching styles, whereas instructors whose classrooms reflected a reformed environment held student-centered beliefs (Gibbons et al., 2018).

To gain deeper, more generalizable insights, future research should aim to reproduce this study with a larger sample of faculty and monitor their beliefs over a longer period of time. It is also critical to identify how the changes over time in the beliefs of chemistry faculty are aligned with changes in their instructional practices, as well as the impact of these beliefs and practices on student learning outcomes. Finally, as Fang noted (1996, p. 59), the community needs to engage with the practically more important concern of understanding “how instructors apply their theoretical beliefs within the constraints imposed by the complexities of the classroom life.”

Conflicts of interest

There are no conflicts of interest to declare.

Acknowledgements

This work is supported by the National Science Foundation CAREER 1552448, 2021491. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

References

  1. Addy T. M. and Blanchard M. R., (2010), The problem with reform from the bottom up: Instructional practises and instructor beliefs of graduate teaching assistants following a reform-minded university instructor certificate programme, Int. J. Sci. Educ., 32(8), 1045–1071 DOI:10.1080/09500690902948060.
  2. Andrews T. M., Leonard M. J., Colgrove C. A. and Kalinowski S. T., (2011), Active learning not associated with student learning in a random sample of college biology courses, CBE–Life Sci. Educ., 10(4), 394–405.
  3. Andrews T. C., Conaway E. P., Zhao J. and Dolan E. L., (2016), Colleagues as change agents: How department networks and opinion leaders influence teaching at a single research university, CBE–Life Sci. Educ., 15(2), 1–17.
  4. Baker L. A., Chakraverty D., Columbus L., Feig A. L., Jenks W. S., Pilarz M., Stains M., Waterman R. and Wesemann J. L., (2014), Cottrell Scholars Collaborative New Faculty Workshop: Professional development for new chemistry faculty and initial assessment of its efficacy, J. Chem. Educ., 91, 1874–1881 DOI:10.1021/ed500547n.
  5. Bazeley P. and Jackson K., (2013), in Seaman J. (ed.), Qualitative Data Analysis with NVivo, 2nd edn, SAGE Oublications Ltd.
  6. Brandriet A. R. and Bretz S. L., (2014), Measuring meta-ignorance through the lens of confidence: Examining students’ redox misconceptions about oxidation numbers, charge, and electron transfer, Chemistry Education Research and Practice, 15, 729–746 10.1039/c4rp00129j.
  7. Center for Postsecondary Research, (n.d.). Carnegie Classification of Institutions of Higher Education, Retrieved May 24, 2019, from http://carnegieclassifications.iu.edu.
  8. Chapman L. A. Y. and McConnell D. A., (2018), Characterizing the Pedagogical Beliefs of Future Geoscience Faculty Members: a Mixed Methods Study, Innovative Higher Educ., 43(3), 185–200 DOI:10.1007/s10755-017-9416-9.
  9. Clarke D. and Hollingsworth H., (2002), Elaborating a model of instructor professional growth, Teach. Instr. Educ., 18, 947–967.
  10. Clatworthy J., Buick D., Hankins M., Weinman J. and Horne R., (2005), The use and reporting of cluster analysis in health psychology: A review, Br. J. Health Psychol., 10, 329–358 DOI:10.1348/135910705X25697.
  11. Creswell J. W., (2003), in Laughton C. D. (ed.), Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2nd edn, Sage Publications.
  12. Czajka C. D. and McConnell D., (2016), Situated instructional coaching: a case study of faculty professional development, Int. J. STEM Educ., 3(10), 1–14 DOI:10.1186/s40594-016-0044-1.
  13. Czajka C. D. and McConnell D., (2019), The adoption of student-centered teaching materials as a professional development experience for college faculty, Int. J. Sci. Educ., 41(5), 693–711 DOI:10.1080/09500693.2019.1578908.
  14. Dolphin G. R. and Tillotson J. W., (2015), “Uncentering” instructor beliefs: The expressed epistemologies of secondary science instructors and how they relate to instructor practice, Int. J. Environ. Sci. Educ., 10(1), 21–38 DOI:10.12973/ijese.2015.228a.
  15. Douglas J., Powell D. N. and Rouamba N. H., (2016), Assessing graduate teaching assistants’ beliefs and practices, J. Excellence Coll. Teach., 27(3), 35–61.
  16. Drever E., (1995), Using semi-structured interviews in small-scale research. A instructor's guide, The SCRE Centre.
  17. Enderle P., Dentzau M., Roseler K., Southerland S., Granger E., Hughes R., Golden B. and Saka Y., (2014), Examining the Influence of RETs on Science Instructor Beliefs and Practice, Sci. Educ., 98(6), 1077–1108 DOI:10.1002/sce.21127.
  18. Fang Z., (1996), A review of research on instructor beliefs and practices, Educ. Res., 38(1), 47–65.
  19. Feyzioğlu E. Y., (2012), Science instructors’ beliefs as barriers to implementation of constructivist-based education reform, J. Baltic Sci. Educ., 11(4), 302–317.
  20. Fletcher S. S. and Luft J. A., (2011), Early career secondary science instructors: A longitudinal study of beliefs in relation to field experiences, Sci. Educ., 95(6), 1124–1146 DOI:10.1002/sce,20450.
  21. Frades I. and Matthiensen R., (2010), Overview on Techniques in Cluster Analysis, in Matthiesen R. (ed.), Bioinformatics Methods in Clinical Research, Humana Press, pp. 81–107 DOI:10.1007/978-1-60327-194-3.
  22. Gess-Newsome J., Southerland S. A., Johnston A. and Woodbury S., (2003), Educational Reform, Personal Practical Theories, and Dissatisfaction: The Anatomy of Change in College Science Teaching, Am. Educ. Res. J., 40(3), 731–767 DOI:10.3102/00028312040003731.
  23. Gibbons R. E., Villafañe S. M., Stains M., Murphy K. L. and Raker J. R., (2018), Beliefs about learning and enacted instructional practices: An investigation in postsecondary chemistry education, J. Res. Sci. Teach., 55(8), 1111–1133.
  24. Harshman J., Yezierski E. and Nielsen S., (2017), Putting the R in CER: How the statistical program R transforms research capabilities, ACS Symposium Series, American Chemical Society, pp. 65–90 DOI:10.1021/bk-2017-1260.ch006.
  25. Hora M. T., (2014), Exploring faculty beliefs about student learning and their role in instructional decision-making, Rev. Higher Educ., 38(1), 37–70 DOI:10.1353/rhe.2014.0047.
  26. Kagan D., (1992), Implication of Research on Instructor Belief, Educ. Psychol., 1, 65–90.
  27. Lee S. W., (2019), The Impact of a Pedagogy Course on the Teaching Beliefs of Inexperienced Graduate Teaching Assistants, CBE Life Sci. Educ., 18, 1–12 DOI:10.1187/cbe.18-07-0137.
  28. Luft J. A., (2001), Changing inquiry practices and beliefs: The impact of an inquiry-based professional development programme on beginning and experienced secondary science instructors, Int. J. Sci. Educ., 23(5), 517–534.
  29. Luft J. A. and Roehrig G. H., (2007), Capturing Science Instructors’ Epistemological Beliefs: The Development of the Instructor Beliefs Interview, Electron. J. Sci. Educ., 11(2), 38–63, http://ejse.southwestern.edu.
  30. Luft J. A., Firestone J. B., Wong S. S., Ortega I., Adams K. and Bang E., (2011), Beginning secondary science instructor induction: A two-year mixed methods study, J. Res. Sci. Teach., 48(10), 1199–1224 DOI:10.1002/tea.20444.
  31. Mattheis A. and Jensen M., (2014), Fostering improved anatomy and physiology instructor pedagogy, Adv. Physiol. Educ., 38, 321–329 DOI:10.1152/advan.00061.2014.-Despite.
  32. Mavhunga E. and Rollnick M., (2016), Instructor- or Learner-Centred? Science Instructor Beliefs Related to Topic Specific Pedagogical Content Knowledge: A South African Case Study, Res. Sci. Educ., 46, 831–855 DOI:10.1007/s11165-015-9483-9.
  33. Moore T. J., Guzey S. S., Roehrig G. H., Stohlmann M. S., Park M. S., Kim Y. R., Callender H. L. and Teo H. J., (2015), Changes in Faculty Members’ Instructional Beliefs while Implementing Model-Eliciting Activities, J. Eng. Educ., 104(3), 279–302 DOI:10.1002/jee.20081.
  34. Nespor J., (1987), The role of beliefs in the practice of teaching, J. Curric. Stud., 19, 317–328.
  35. Pajares M. F., (1992), Instructors’ Beliefs and Educational Research: Cleaning Up a Messy Construct, Rev. Educ. Res., 62(3), 307–332 DOI:10.3102/00346543062003307.
  36. Pandey S. C. and Patnaik S., (2014), Establishing reliability and validity in qualitative inquiry: A critical examination, J. Dev. Manage. Stud. XISS, 12(1), 5743–5753. https://www.researchgate.net/publication/266676584.
  37. Patton M. Q., (2002), Qualitative Research & Evaluation Methods, 2nd edn, Thousand Oaks, CA: Sage Publications, Inc.
  38. Pelch M. A. and McConnell D. A., (2016), Challenging instructors to change: A mixed methods investigation on the effects of material development on the pedagogical beliefs of geoscience instructors, Int. J. STEM Educ., 3(5), 1–18 DOI:10.1186/s40594-016-0039-y.
  39. Popova M., Shi L., Harshman J., Kraft A. and Stains M., (2020), Untangling a complex relationship: Teaching beliefs and instructional practices of assistant chemistry faculty at research-intensive institutions, Chem. Educ. Res. Pract., 21, 513–527 10.1039/c9rp00217k.
  40. Pratt J. M. and Yezierski E. J., (2019), “You Lose Some Accuracy When You’re Dumbing it Down”: Teaching and Learning Ideas of College Students Teaching Chemistry through Outreach, J. Chem. Educ., 96(2), 203–212 DOI:10.1021/acs.jchemed.8b00828.
  41. Saldaña J., (2013), in Seaman J. (ed.), The Coding Manual for Qualitative Researchers, 2nd edn, Sage Publications Inc.
  42. Şen Ö. F. and Sarı U., (2018), From Traditional To Reform-Based Teaching Beliefs and Classroom Practices of Elementary Science Instructors, Int. J. Innovation Sci. Math. Educ., 26(6), 76–95.
  43. Southerland S. A., Granger E. M., Hughes R., Enderle P., Ke F., Roseler K., Saka Y. and Tekkumru-Kisa M., (2016), Essential Aspects of Science Instructor Professional Development: Making Research Participation Instructionally Effective, AERA Open, 2(4), 1–16 DOI:10.1177/2332858416674200.
  44. Stains M. and Vickrey T., (2017), Fidelity of implementation: An overlooked yet critical construct to establish effectiveness of evidence-based instructional practices, CBE–Life Sci. Educ., 16(1), rm1.
  45. U.S. Department of Commence, (n.d.). United States Census Bureau, Retrieved May 24, 2019, from https://www.census.gov.
  46. Wilks D. S., (2014), Cluster Analysis, A Concise Guide to Market Research, Springer-Verlag Berlin Heidelberg, pp. 273–324 DOI:10.1016/B978-0-12-385022-5.00015-4.
  47. Wong S. S. and Luft J. A., (2015), Secondary Science Instructors’ Beliefs and Persistence: A Longitudinal Mixed-Methods Study, J. Sci. Instr. Educ., 26, 619–645 DOI:10.1007/s10972-015-9441-4.
  48. Woodbury S. and Gess-Newsome J., (2002), Overcoming the Paradox of Change without Difference: A Model of Change in the Arena of Fundamental School Reform, Educ. Policy, 16(5), 763–782.

Footnote

† Electronic supplementary information (ESI) available. See DOI: 10.1039/d0rp00313a
This journal is © The Royal Society of Chemistry 2021