Investigating student perceptions of transformational intent and classroom culture in organic chemistry courses

Abstract

Within chemistry education, there are various curricular and pedagogical approaches that aim to improve teaching and learning in chemistry. Efforts to characterize these transformations have primarily focused on student reasoning and performance, and little work has been done to explore student perceptions of curricular and pedagogical transformations and whether these perceptions align with the transformational intent. To complement our previous work on the Organic Chemistry, Life, the Universe, and Everything (OCLUE) curriculum, we developed this exploratory study to determine if students had perceived the goals of the transformation. As in our previous research on OCLUE, we compared perceptions between OCLUE and a more traditional organic chemistry course. Using inductive and deductive qualitative methodologies, we analyzed student responses to three open-eneded questions focused on how students perceived they were expected to think, what they found most difficult, and how they perceived they were assessed. The findings were classified into three superodinate themes: one where students perceived they were expected to learn things as rote knowledge, such as memorization (“Rote Knowledge”), another where students perceived they were expected to use their knowledge (“Use of Knowledge”), and responses that used vague, generalized language, were uninformative, or did not address the questions asked (“Other”). Students in these two courses responded very differently to the open-ended questions with students in OCLUE being more likely to perceive they were expected to use their knowledge, while students in the traditional course reported rote learning or memorization more frequently. As the findings evolved, our interpretations and discussions were influenced by sociocultural perspectives and other cultural frameworks. We believe this approach can provide meaningful insights into transformational intent and certain features of classroom cultures.

---

Introduction

Chemistry education research (CER) has led to the development of a number of undergraduate course transformations with the goal of improving teaching and learning in chemistry (Talanquer and Pollard, 2010; Cooper and Klymkowsky, 2013; Sevian and Talanquer, 2014; Cooper et al., 2019; McGill et al., 2019). These transformations have been characterized and supported by research on student performance and reasoning within the contexts of these courses (Banks et al., 2015; Becker et al., 2016; Cooper et al., 2016; Crandell et al., 2019, 2020; Noyes and Cooper, 2019; Houchlei et al., 2021; Talanquer, 2021); however, little work has been done to explore student perceptions of what they think they are doing. That is, there is scarce research on student perceptions of what is valued in courses and whether these perceptions align with transformational goals.

Within the CER and science education literature, there are many studies exploring student perceptions within the affective domain of learning and student experiences across entire courses or programs (Bauer, 2005, 2008; Galloway and Bretz, 2016; Galloway et al., 2016; Flaherty, 2020a). For example, longitudinal studies such as Talking About Leaving (Seymour and Hewitt, 1997) and Talking About Leaving Revisited (Thiry et al., 2019) have leveraged student perceptions and found that students perceive competitive, unsupportive class cultures in many of their STEM courses, including chemistry. According to students in these studies, the class cultures, in conjunction with many other factors, ultimately contributed to their decision to switch out of their STEM majors (Seymour and Hewitt, 1997; Thiry et al., 2019). More recently, studies have explored student perceptions of their chemistry courses following the shift to online instruction during the 2020–2022 COVID-19 pandemic (Ramachandran and Rodriguez, 2020).

Outside of the affective domain, research on student perceptions of learning has also been common. For example, one study explored how students interpreted structure, property, and function relationships across biology and chemistry. The authors found that while students could discuss structure and properties in the context of both courses, students had more difficulty discussing function in the context of chemistry (Kohn et al., 2018). Such work is supported by previous research that has found that students may miss crucial information during instruction that could aid their understanding which causes them to not perform as well as they intended despite their success in earlier chemistry courses (Anderson and Bodner, 2008). In the context of undergraduate laboratories, work with course-based undergraduate research experiences (CUREs) found that students demonstrated gains in their perceived knowledge, experience, and confidence with specific research-related abilities. The authors concluded that such perceptions could help instructors with course evaluation and assessment design (Irby et al., 2020).

Student perceptions have also been leveraged to better understand how students engage in critical thinking. For example, Scott studied student perceptions of critical thinking after they completed a technology course where debate was employed as a pedagogical tool and found that students perceived their critical thinking abilities had been enhanced (Scott, 2008). Similarly, Hammersley-Fletcher and Hanley used student perceptions to explore the ways that international students in the UK viewed critical thinking and concluded that students thought that certain approaches associated with critical thinking silenced their voices (Hammersley-Fletcher and Hanley, 2016). Finally, in a study investigating student, teaching staff, and employer perceptions of the definition of critical thinking in chemistry communities, Danczak and colleagues found that definitions across the groups differed and that students perceived “critique”, “objectivity”, and “problem-solving” were all components of critical thinking (Danczak et al., 2017).

All of these examples highlight the robust and insightful nature of student perceptions in chemistry education, making them a significant area of research. In a study related to this work, co-author AAF employed constructivist grounded theory to investigate student perceptions of the structure and development of scientific knowledge within the transformed organic chemistry course discussed here. After interviewing twelve students in the transformed course, the findings indicated that students perceived memorization of content was not as effective as being able to reason, that students needed to critique information by interrogating prior knowledge, and that students recognized differences in explaining how and why chemical phenomena occur, among others (Flaherty, 2020b). Though this initial study was influential for our work discussed here, it's important to note that it asked fundamentally different research questions and was not comparative. That is, it was focused on student perceptions of the structure and development of scientific knowledge and focused exclusively on students in the transformed course without comparison of their perceptions to students in other organic chemistry environments. Regardless, the findings pushed us to pursue this line of inquiry further.

Although student perceptions had been leveraged in a variety of ways, to our knowledge, they had not been used to further assess transformation efforts and to ascertain whether student understanding of course goals aligned with instructor expectations. Therefore, we found student perceptions of expectations and what was valued to be a significant area of study for four reasons: (1) it complemented our previous research of our transformational efforts at our institution (Crandell et al., 2019, 2020; Houchlei et al., 2021); (2) it afforded another perspective and way to characterize our transformation that did not focus on student reasoning; (3) it allowed us to explore alignment between our transformational intent, expectations, and student perceptions (and ascertain whether there was misalignment); and (4) considering our transformational efforts were informed by research on how people learn and think (National Research Council, 2000, 2012b, 2012a; National Academies of Sciences, 2018), this study would enable us to investigate if student perceptions of what was expected and valued in courses aligned with the evidence base on effective ways of doing and thinking.

With these motivations in mind, we embarked on this exploratory study. However, as we will explain more later, the study evolved as we interfaced with and interpreted the data. Influenced by the Talking About Leaving studies (Seymour and Hewitt, 1997; Thiry et al., 2019), we were reminded that student perceptions can be used to provide insights on elements of the classroom culture. Studies have shown that alignment between course goals and classroom practices can lead to a more productive learning experience and engagement with scientific practices (Sandoval et al., 2019). Furthermore, we recognized that certain classroom norms communicate implicit and explicit messages to students about how to participate, think, and practice (Becker et al., 2013; Chang and Song, 2016; Reinholz and Apkarian, 2018). Considering that we were interested in knowing what students perceived they were doing in these courses and whether they aligned with our transformational intent, our interpretations and discussions evolved to consider the classroom cultures of the two organic chemistry courses in this study.

Our previously published research compared student performance on a variety of tasks, including constructing causal mechanistic explanations and the use of mechanistic arrows, across a two semester sequence of a transformed organic chemistry course (Crandell et al., 2019, 2020; Houchlei et al., 2021). As a result, we have insights on student thinking and their approaches to such tasks. Therefore, we opted to engage in this exploratory study where we investigated student perceptions of two organic chemistry courses (including our transformed course) that, in our opinion and from our previous research, employed different approaches to teaching and learning. One of the courses was transformed using three-dimensional learning (National Research Council, 2012a; 3DL4US, n.d.), and the other embodied a more traditional approach to organic chemistry (as further discussed below). Considering that our previous studies afforded us insights into how students responded to different types of organic chemistry tasks, the work presented here attempted to characterize the course experiences from student perspectives. Our motivations for this work were driven by an interest in complementing this previous work and to characterize our transformation efforts from a different perspective. Just as some CER scholars have argued that student perceptions can inform assessment design (Irby et al., 2020), we assert that student perceptions of what is expected and valued can inform course design and transformational efforts. Furthermore, this study enabled us to explore alignment of student perceptions of what they are expected to do, what is valued, our transformational intent, and the evidence base on effective ways of doing and thinking. As we will note later, given the research on the role that alignment between classroom practices, course goals, and norms of participation and practice have within the classroom culture, we ultimately discuss and situate this work within a sociocultural perspective that is informed by culture scholars (Vygotsky, 1978; Rogoff, 1990; John-Steiner and Mahn, 1996; Carlone et al., 2011; Becker et al., 2013; Chang and Song, 2016; Schein and Schein, 2016; Reinholz and Apkarian, 2018; Sandoval et al., 2019; Zotos et al., 2020; Petterson et al., 2022).

In order to gather this data, we needed an instrument that would help us capture student perceptions in a robust way while minimizing external influences on student responses. Although there are a number of previously developed instruments for use in higher education that address student perceptions, expectations, and other affective states, none of them met the needs of this study; therefore, we opted to develop our own. Our instrument involved three open-ended questions which will be discussed in more detail later. These questions specifically target student perceptions of how they were expected to think in organic chemistry, what they found most difficult in the course, and how they perceived they were assessed. However, first, we find it important to review some of these instruments to justify the development of our own.

Previously published instruments

Many of the previously published instruments we reviewed relied on the use of Likert or semantic differential scales where students responded to prompts developed by researchers. One of the first wide-scale uses of Likert-scale instruments in higher education was the Maryland Physics Expectations (MPEX) survey which was developed by Redish and co-workers (Redish et al., 1998). The MPEX later led to the development of the corresponding survey for chemistry known as the CHEMX (Grove and Bretz, 2007). Both the MPEX and the CHEMX have students respond to closed-ended questions on a agree-disagree Likert-scale and are designed to gather information on student assumptions, beliefs, and cognitive expectations within physics and chemistry. According to Redish, cognitive expectations refer to students “expectations about their understanding of the process of learning [physics] and the structure of [physics] knowledge rather than about the content of physics itself.” The CHEMX survey has a similar guiding philosophy. Both surveys compare student responses to expert responses, and it is notable that students appear to become less “expert-like” in their expectations and understanding of how science is done over the course of two semesters of introductory physics and chemistry. The authors of these surveys ascribe this apparent regression to how the content of these introductory courses is structured and how they are taught.

The Colorado Learning Attitudes about Science Survey (CLASS), is also a Likert-scale instrument developed for physics (Adams et al., 2005) and adapted for chemistry (Barbera et al., 2008) and biology (Semsar et al., 2011). The CLASS instruments are primarily focused on gathering information from students on their beliefs and attitudes about learning within the specific discipline, the content of the discipline, the structure of the disciplinary knowledge, and connections to the “real world”. In contrast to the MPEX and CHEMX, the CLASS asks about the discipline in general while the MPEX and CHEMX instruments probe student beliefs about a specific course. Just as with the MPEX and CHEMX, results from the CLASS are reported as how well they align with expert-responses, and typically there is no “improvement”. That is, there is no movement to more expert-like responses over a general chemistry sequence. However, these three instruments differ in that the MPEX and the CHEMX cluster responses using confirmatory factor analysis while the CLASS clusters according to exploratory factor analysis.

Although not immediately related to expectations, other instruments have been developed to explicitly measure student attitudes. Some instruments, such as the Chemistry Attitudes and Experiences Questionnaire (CAEQ) (Dalgety et al., 2003) and the Attitude towards the Subject of Chemistry Inventory (ASCI) (Bauer, 2008) have utilized a semantic differential format where students respond on a scale where the extremes include polar opposite adjectives. In the case of the ASCI, the structure of the instrument begins with a sentence stem such as “Chemistry is…” and then students respond to the sentence stem by rating their response on a 7-point semantic differential scale where the extremes represent aforementioned polar opposite adjectives such as “easy/hard”, “comprehensible/incomprehensible”, and “tense/relaxed”, among others. The ASCI has been further developed, producing the ASCIv2 (Xu and Lewis, 2011) and the ASCIv3 (Rocabado et al., 2019).

While the use of Likert- and semantic differential-scale instruments allow for quick diagnostics and analysis of the data, the questions within these instruments may prompt students to respond in a certain way, do not allow for students to state their experience in their own words, and students may not be given the opportunity to volunteer information that they deem as most important or relevant to their experience. These restrictions signify that more open-ended questions coupled with qualitative methodologies could be helpful discovering themes that capture a more accurate picture of student experience. Though some of the items in previous instruments were investigating similar ideas as we are here, the potential for prompting inherent in the questions and the lack of opportunities for students to use their own words may not accurately capture student perceptions, beliefs, or attitudes. Furthermore, qualitative approaches to investigate perceptions, expectations, and other constructs is scarce (Flaherty, 2020a), and we believed this to be a great opportunity to explore student perceptions of their organic chemistry courses in an open-ended way.

With this said, previously published instruments in CER and other fields tended to address how students experienced a course or whole discipline and did not align with our study objectives. Our goal was rather different in that we were interested in how students perceived course/instructor expectations and what was valued. Since we wanted to use students’ own words and perceptions to guide our investigation, minimize prompting in the questions, and use qualitative methodologies to analyze the responses, we opted to use our own instrument. Such an approach allowed for a combination of inductive and deductive coding, highlighted student voices, and provided students the opportunity to identify what they believed to be most important and relevant to their experiences.

Purpose

As noted throughout, the purpose for this study was to complement our previous work on student reasoning in these courses and characterize our transformational efforts further. This was coupled with the goal of generating insights on whether student perceptions of what they were doing and what was valued aligned with our transformational intent and the underlying theories of learning in the transformed course. Considering that previously published instruments were not appropriate given our exploratory goals and interests, we opted to use our own instrument. As we engaged with the data, we began to address the areas of interest through the lens of classroom culture. Therefore, the research questions that guided our work included: (1) in what ways do student perceptions of valued ways of doing and thinking align with the transformational intent; (2) How do elements of the course culture impact student perceptions of what is valued?

Theoretical framework

The work presented here began as an exploratory project; yet, as our findings began to take shape, we started to interpret and discuss the findings in terms of the classroom cultures. As we analyzed the data, we noted how certain classroom structures and practices, norms of participation, and messages about what was valued informed student perceptions (Becker et al., 2013; Chang and Song, 2016; Reinholz and Apkarian, 2018). That is, we saw student responses speaking to interpretations of course expectations, perceptions of valued ways of doing and practicing, and the influence these expectations and ways of doing had on course difficulty. Therefore, our interpretations of this exploratory work drew upon sociocultural perspectives and studies (Vygotsky, 1978; Rogoff, 1990; John-Steiner and Mahn, 1996; Carlone et al., 2011; Zotos et al., 2020; Petterson et al., 2022), as well as other culture-related frameworks (Schein and Schein, 2016; Reinholz and Apkarian, 2018). We will speak more to this framework at the beginning of the discussion after we have presented the results. Our rationale for this intentional writing decision is to highlight the initial exploratory nature of this study and how our analysis and interpretations evolved over time. By using student perceptions as a proxy for elements of organic chemistry classroom cultures, we aim to complement our previous research on student reasoning in the context of these courses and demonstrate how student perceptions of what they were expected to do, what was most difficult, and how they were assessed can be insightful for the development and enactment of chemistry courses.

Considering that this study uses student perceptions of what is expected and valued and ways of practicing, it is important to acknowledge that we (and students, for that matter) have assumptions and ideas about what it means to know and do. Broadly, our epistemological beliefs are informed by constructivist and sociocultural views of learning where we believe that students construct their own knowledge and are influenced by the contexts in which learning occurs and the interactions they have (Vygotsky, 1978; Bodner, 1986; Rogoff, 1990; John-Steiner and Mahn, 1996; National Research Council, 2000; Carlone et al., 2011; National Academies of Sciences, 2018; Zotos et al., 2020; Petterson et al., 2022). With this, we also ascribe to the resources perspective which asserts that students have knowledge that is connected in various ways which may or may not be activated when prompted depending on their knowledge structure and how the task is scaffolded. Furthermore, it is acknowledged that the resources students have may be more or less productive on a given learning task which offers a way to understand how students are connecting and applying concepts (Hammer, 2000). With all of this said, we ascribe to the idea that people learn best when they are in environments that provide them consistent opportunities to apply and use their knowledge (and resources). Considering that three-dimensional learning engages students in scientific practices around fundamental ideas in chemistry, it resonates with our epistemological beliefs and is the foundation for our course transformations as will be discussed in the Methods section (National Research Council, 2012a; 3DL4US, n.d.). Coupled with our previous work on student reasoning in the context of the courses in this study, we acknowledge these beliefs and previous research influenced our analysis and interpretations of student perceptions.

Methods

Context: transformed and traditional organic chemistry courses

This research took place in the context of two types of organic chemistry courses: transformed and traditional. Both courses were taught at a large research-intensive midwestern university in the United States. The transformed course used the Organic Chemistry, Life, the Universe and Everything (OCLUE) curriculum (Cooper et al., 2019) which uses the framework of three-dimensional learning to support knowledge in use. That is, it emphasizes core ideas, scientific practices, and crosscutting concepts as discussed in A Framework for K-12 Science Education (National Research Council, 2012a). In OCLUE, ideas are introduced and linked to the chemistry core ideas of Structure–Property relationships, Bonding and Interactions, Energy, and Change and Stability in the context of scientific practices (Cooper et al., 2017). In particular, the development and use of models and explanations is combined with mechanistic reasoning to support students as they explain how and why organic phenomena occur. OCLUE students are routinely asked to construct mechanistic explanations for phenomena such as acid–base reactions (Crandell et al., 2019), nucleophilic substitutions (Crandell et al., 2020), mechanisms for electrophilic addition and other reactions (Houchlei et al., 2021), thermodynamic and kinetic control, and solvent effects.

Lectures in OCLUE are somewhat interactive. New topics are introduced by having students discuss what they already know, clicker questions are posed, students are encouraged to discuss the answers, and occasional group activities are incorporated (for example, groups build molecular models and compare them together). Students work in groups in OCLUE recitation sections to complete scaffolded worksheets which include a mixture of three-dimensional and more traditional questions, such as draw a reaction mechanism or determine the identity of an unknown compound from spectroscopic data. Homework is assigned twice a week for credit upon completion rather than accuracy to encourage students to try and practice without penalty and also includes three-dimensional prompts similar to the recitation activities. Therefore, a considerable proportion of a students’ grade in OCLUE is determined by participating, practicing, and trying with “good faith effort” and explaining how and why something happens, thus allocating a smaller proportion of the students’ grade to high stakes testing. Examinations in OCLUE employ a mixture of multiple-choice and open-response items, some of which mirror traditional questions in an organic chemistry course (such as predicting products and drawing mechanisms). However, frequently students are asked to provide an explanation of how and why a given chemical phenomena is occurring with about 50% of the points on exams focusing on having students use core ideas in the context of scientific practices.

In contrast, traditional courses are usually organized by functional group. Rather than connecting a few types of reaction mechanisms to the core ideas, a traditional course tends to treat each type of reaction and functional group separately. By agreement between all instructors, the same topics are covered, the course is primarily taught in a traditional expository lecture format. While students can (and do) ask questions, there is no expectation of peer interactions either in the lecture or in the recitation sections. Instead, the recitation sections for the traditional course typically consist of a quiz, followed by a question-and-answer period, or another short lecture from the graduate teaching assistant. Students may complete online homework, typically multiple-choice questions, however the homework is not completed for a grade. The examinations consist of open-response items where students must fill in the reactant, reagent, or products, draw a mechanism for a reaction, or design a synthesis, and are typical high stakes summative assessments. These items are similar to those that we have found are prevalent in sophomore organic chemistry courses, and our prior analysis of these items indicates that students are typically not required to explicitly show evidence of reasoning, but rather can they answer questions by recall or pattern recognition (Stowe and Cooper, 2017).

It's worth noting that the overall assessment strategies for the two courses also differ significantly. In OCLUE, between 45–50% of the points are allocated through formative assessment strategies. That is, group work in recitation and homework are not graded for accuracy but on completion with a “good faith effort”. The rest of the overall grade in OCLUE comes from three mid-terms and a final exam. In contrast, in the traditional sections all of the points towards the class grade come from summative exams (midterms and final). This difference may have significant consequences for students since there is emerging evidence that allocating parts of the course grade to completion of formative assessments is a more equitable strategy that can address differences in outcomes among various demographic groups (Tashiro and Talanquer, 2021).

In summary, the two types of courses cover the same material, but they have different pedagogical approaches, course requirements, and approaches to assessments. Given that the purpose of this study was to complement previous research by characterizing our transformational efforts from the student perspective and to generate insights on how student perceptions aligned with our transformational intent and subsequent theories of learning, we found this to be an informative study. As will be discussed in more detail later, our interpretive frame of classroom culture clarifies these purposes by helping us acknowledge that the implicit and explicit messages sent by the course and instructors communicate what are valued ways of knowing and doing. Alignment between classroom practices, course goals, messages instructors send, and the interpretations of those messages by students as well as classroom norms have been shown to be important for engaging students in learning practices (Becker et al., 2013; Chang and Song, 2016; Schein and Schein, 2016; Sandoval et al., 2019).

Participants

The study took place in the Spring semester of 2018 in organic chemistry II. Therefore, this course was entirely in-person and was completed before the 2020–2022 COVID-19 pandemic moved classes online. The total number of participants in this study was 852 undergraduate students. Six-hundred and four students were enrolled in a traditional organic chemistry course and 248 students were enrolled in OCLUE. Both are large enrollment courses taught in lecture sections of 200–300 students that meet for approximately three hours per week. Each student is also enrolled in a one-hour recitation section of about 30 students that is taught by a graduate teaching assistant. Students are not aware of the differences in the two courses before they enroll, and the demographics and academic background of the students in each course section are similar (Crandell et al., 2020). Students answered the three questions in our instrument for extra credit in each class, and all students were informed of their rights as research participants in accordance with the institutional review board. Participant demographics are included in Table 1 which is the demographic breakdown of all students enrolled in organic chemistry II of the Spring 2018 semester at the university in this study that was provided by the university registrar. From the demographics breakdown it can be noted the majority of students were life sciences majors and white. We have previously not noted major differences in demographics between the two types of courses.

Table 1 Participant demographics

Gender		First-generation		Transfer
Female	693	Yes	202	Yes	152
Male	310	No	801	No	851
Total	1003	Total	1003	Total	1003

---

Major		Ethnicity
Life Sciences	733	American Indian/Alaskan Native	1
Lab Sciences	61	Asian (non-hispanic)	91
Physical Sciences	24	Black or African–American (non-hispanic)	66
Engineering	4	Hawaiian/Pacific Islander (non-Hispanic)	1
Animal Sciences and Veterinary	58	Hispanic	37
Food and Nutritional Sciences	36	International	49
Social Sciences	27	Not reported	11
Other	60	Two or more races (non-hispanic)	37
		White (non-hispanic)	710
Total	1003	Total	1003

---

Design of the instrument questions

To generate a manageable dataset for the 852 students in our study, our instrument included three open-ended questions. While similar instruments consist of many focused questions, our first goal was to ask open ended questions so that students could respond in their own words. Second, we wanted to minimize prompting; that is, we wanted to avoid using highly specific questions that might make students respond a certain way. Third, we wanted to ask a few questions that addressed different but related aspects of the course that could be answered in a few sentences at most and enable us to collect data from small or large courses. Finally, we wanted to pose questions that were accessible and understandable to students. The question design occupied a useful analytic middle ground. That is, it was not as constrained as a quantitative questionnaire, yet it could capture insightful, rich responses from many students without conducting time-consuming interviews.

The first question stated: “If you met a student who is thinking about enrolling in (traditional or OCLUE) organic chemistry next year, how would you describe the ways students are expected to think about reaction mechanisms in organic chemistry?” By mentioning mechanisms, we intended to scaffold student responses and help them reflect on course expectations. Furthermore, there is ample research to show that students have great difficulty with thinking about mechanisms and often resort to memorization as a way to succeed (Bhattacharyya and Bodner, 2005). This question also related to our previous work where we have shown that students in OCLUE are more likely to engage in causal mechanistic reasoning, to use mechanisms appropriately, and are significantly more likely than traditional peers to correctly predict products for unknown reactions (Houchlei et al., 2021).

The second question was the following: “What would you tell them is the most difficult thing about organic chemistry?” Previously published instruments often asked students about the difficulty of the overall course or specific content, and we thought this question would provide students the opportunity to identify aspects that they deemed most difficult without constraining their response. The CER literature has detailed various aspects of organic chemistry that students have difficulty with and by investigating student perceptions on the most difficult aspect of the course in this open-ended way, we can gather insights into which facets of a course students struggle with the most, such as a certain way of thinking, a course policy, or instruction in general.

The third question was “How would you describe to them what is assessed in organic chemistry?” This question was designed to elicit if students perceived that assessments aligned with how they perceived they were expected to think in the course. It is well recognized that assessments send strong messages to students about what is valued in a course (Momsen et al., 2013; Stowe et al., 2021). Considering that the approaches to summative and formative assessments were quite different between the two courses in this study, we believed this question would be insightful.

The design of these questions aligned with our goals of the study. We were interested in complementing our previous research on student reasoning and characterizing OCLUE from the student perspective which all three questions would address. In addition, we were interested in investigating whether student perceptions aligned with our course goals and expectations and the underlying theories of learning that informed the OCLUE curriculum. We anticipated that questions 1 (expectations of thinking) and 3 (assessment) would be an open-ended way to explore this alignment while question 2 (most difficult thing) provided additional information on how the enactment of the course impacted difficulties encountered by students.

Data collection

Student responses were collected in the form of a homework activity assigned through the beSocratic homework system (Bryfczynski, 2010; beSocratic, 2020). For both types of courses, students were given extra credit and an entire week to complete the questions. After the due date, the data was exported out of beSocratic into an Excel file and then responses were deidentified to protect the anonymity of students. Before beginning analysis on the data, the responses were blinded and mixed up so that the coders did not know which course the response were from (either traditional or OCLUE).

Data analysis

The data collected for this study was analyzed with an inductive thematic analysis approach (Thomas, 2006) that allowed us to establish an analytical framework that we then applied deductively to the rest of the data. This form of data analysis facilitates the emergence of research findings from themes within the data without being restrained by structured methodologies (Boyatzis, 1998; Thomas, 2006). Unlike a grounded theory methodology, which produces theory, or phenomenology, which produces a description of lived experiences, inductive data analysis produces themes or categories which are relevant to the research objectives identified (Thomas, 2006). The purposes of inductive data analysis involve (i) condensing text data into a brief, summary format, (ii) establishing links between the research objectives and the summary findings, and (iii) developing a model or theory about the underlying structure of experiences that are evident in the data (Thomas, 2006). Our analysis began with inductive thematic analysis which allowed us to form categories that were prominent in the data and develop a codebook. Although this study sought to complement our previous work, it's important to note that the language used to describe and name categories was pulled from student responses; that is, although our thinking about our categories may have been influenced by previous work, we attempted to use student words and perspectives to guide our analysis and name our categories. After our codebook was revised and developed, it was applied to the remainder of the data. That is, our analysis began inductively and proceeded to a deductive analysis once our codebook was developed (Merriam and Tisdell, 2016).

Inductive thematic analysis was deemed most suitable in the beginning because: (1) we wanted categories to emerge from student experiences at first to guide our analysis and highlight their voices; (2) we had highly open-ended questions; and (3) our initial stance toward this project was exploratory in nature and we believed beginning with an inductive approach was appropriate. Therefore, analysis was conducted as described by Thomas (Thomas, 2006). First, the raw data files were formatted to promote ease of comprehension. Second, we familiarized ourselves with the nature of the data by reading the student responses. Third, categories were identified and defined based on actual phrases or meanings in specific text segments. Finally, each category was continually revised based on the ongoing analysis of data. To establish the codebook, responses from 248 traditional students and all 248 OCLUE students were analyzed. Taken together these 496 students answered the three open-ended questions mentioned earlier, yielding 1488 responses across all three questions, and representing over 58% of the total data. Two of the authors (RSB and AAF) went through several rounds of independent coding of the 1488 responses, developing and revising the codebooks for each question each time. Upon settling on a semi-finalized codebook, the authors then calculated percent agreement and found they had an 86.1% agreement. After discussing the coding discrepancies and sharpening the code dimensions to yield the finalized codebook, the authors settled on a 99.4% agreement. With such a high percent agreement, the authors concluded that any additional measure of inter-rater reliability would not be necessary. The remaining set of data was then split in half between RSB and AAF and coded to yield the full set of analyzed data (all 2556 responses). Throughout the coding process, mutually exclusive codes were identified and used. The decision to use mutually exclusive coding was based on the following reasons:

(1) The overall majority of the responses could only be categorized by a single code.

(2) An analysis by author RSB using non-mutually exclusive coding yielded almost identical overall patterns. This is provided in the ESI,† Fig. S1–S3.

(3) The use of mutually exclusive codes allowed for quicker and more efficient coding of the 2556 responses.

Results

The three open-ended questions of our instrument were analyzed separately, and a separate codebook was developed for each question. The results section will report on the nature of these codes.

Question 1: expectations of thinking

As a reminder, the first question asked: “If you met a student who is thinking about enrolling in (traditional or OCLUE) organic chemistry next year, how would you describe the ways students are expected to think about reaction mechanisms in organic chemistry?” Responses were classified into six categories and outlined in Table 2. As we have noted throughout, one of the motivations behind this study was to complement our previous work on student reasoning. Therefore, our thinking and approach to analysis may have been informed in some way by this previous work; however, we reiterate that descriptions and naming of categories were based on student perspectives or language they chose to use in their responses.

Table 2 Codebook for question 1: perceptions of the expectations of thinking

Code	Dimensions	Example quotes
Apply and reason	Student responses that include one or more of the following dimensions in regard to their perceptions of how they are expected to think about mechanisms: (1) understanding “why” a reaction proceeds; (2) the use of knowledge, specifically with the use of fundamental or basic ideas; (3) the transfer of knowledge to new problems; (4) making connections between concepts, especially in order to apply them; (5) making predictions in order to solve a problem	Traditional: “I would tell them not to memorize them but to actually think through each of them and the reasoning behind why what happens, happens.”
		OCLUE: “Should expect to understand the molecular interactions of reactions and WHY these occur.”
Identify and describe	Student responses that include one or more of the following dimensions in regard to their perceptions of how they are expected to think about mechanisms: (1) understanding the “what” and “how” reactions proceed, particularly without mentioning the use of knowledge to understanding “why” reactions proceed; (2) mentions understanding at the scalar or one scalar below levels, particularly through the recognition of concepts such as polarity and electronegativity and their significance to understanding; (3) when a student explicitly mentions understanding the mechanism instead of memorization; (4) when a student mentions “differentiating” between reactions with any further explanation; (5) responses includes a discussion of forces, charges, or stabilization	Traditional: “They have to think about the polarity of bonds and the nature of atoms when reacting with other atoms in regard to electronegativity and polarity”
		OCLUE: “The reaction mechanism are meant to show the transfer of electrons from one compound/atom to another. This helps show how these reactions occur.”
Apply heuristics	Student responses that include one or more of the following dimensions in regard to their perceptions of how they are expected to think about mechanisms: (1) focuses on the approach to solving problems rather than thinking about a problem; (2) mentions explicit use of arrow pushing without mentioning how knowledge is used to engage in the formalism; (3) provide descriptive statements without causal or mechanistic knowledge such as “negatives attack positives” or “source goes to sink”; (4) mention of identifying patterns and trends without expanding on the significance of identifying these patterns and trends; (4) explicit mention of the movement or flow of electrons without further explanation of how the movement or flow of electrons influence reactions	Traditional: “think about it in terms of Nu- attacks E +”
		OCLUE: “they need to think about mechanistic arrows as the movement of electrons from a source to a sink”
Memorization	Student responses that include one or more of the following dimensions regarding their perceptions of how they are expected to think about mechanisms: (1) memorization, remembering, recalling, or regurgitation of reactions, products, reagents, and/or mechanisms; (2) knowing reactions, products, reagents, and/or mechanisms, particularly with no explicit mention of understanding the “what”, “how”, or “why” a reaction proceeds	Traditional: “Memorize, memorize, memorize.”
		OCLUE: “memorize what reacts with what”
Generalities	Student responses that include one or more of the following dimensions in regard to their perceptions of how they are expected to think about mechanisms: (1) the total absence of any chemistry in the response; (2) thinking of reactions and/or mechanisms as “puzzles”; (3) thinking of reactions and/or mechanisms on a “step-by-step” basis; (4) using generic and unclear descriptors for thinking such as thinking critically, conceptually, creatively, thoroughly, or differently, particularly if the student does not expand on what they mean; (5) stating general facts about reactions and mechanisms such as “reactants go to products” or that there are many mechanisms for a given reaction; (6) seeing organic chemistry as a new or different language; (7) mentioning that organic chemistry focuses on the details; (8) mentioning that mechanisms are “like a story”	Traditional: “You have to think about would they would benefit from most if they reacted.”
		OCLUE: “You need to think rationally rather than memorize.”
Not applicable	Student responses that include one or more of the following dimensions in regard to their perceptions of how they are expected to think about mechanisms: (1) the response does not answer the question, such as organic chemistry or mechanisms are “easy” and “straightforward”; (2) when a student provides no response at all; (3) when the response is unclear or interpretation difficult, such as when students say “you must understand/know the material”; (4) mention of actions students must do in organic chemistry such as “study a lot”; (5) mention of exam and/or course aspects such as exam difficulty and the challenging nature of organic chemistry; (6) when a student is venting about the course, professor, or other aspects relevant to the course	Traditional: “To do the practice problems in the text book and make flash cards.”
		OCLUE: “good”

---

The “Apply and Reason” and the “Identify and Describe” categories differ with respect to whether students noted the significance of knowing why a mechanism occurs. For example, if a student mentioned the existence of forces and stabilization in their response, this was coded as “Identify and Describe.” However, if the student mentioned the existence of forces and stabilization, and then expanded their response to include a discussion of how this helps explain why a reaction happens, then the response was coded as “Apply and Reason.” While “Apply and Reason” responses were considered more sophisticated than “Identify and Describe”, we acknowledge the complexity and potential understanding exhibited in “Identify and Describe” responses.

Although we did not set out to develop a hierarchical model of categories during the analysis, the progression from “Memorization” to “Apply and Reason” does suggest a greater degree of sophistication in students’ perceptions of how they were expected to think about organic chemistry mechanisms. Though it is possible that a student can memorize a heuristic, the “Apply Heuristics” and “Memorization” categories were considered separately because using a heuristic does require some form of application that simply memorizing does not. The “Generalities” and the “Not Applicable” categories were also identified and included in the analysis. The “Generalities” category included responses where students explained the need to think about organic chemistry mechanisms using general or vague terms such as “critically”, “conceptually”, “creatively”, “thoroughly”, or “differently”; that is, they did not appear to have developed a specific vocabulary for what they were doing. Also included in this category were responses which referred to the need to think about mechanisms in a step-by-step manner, solving puzzles, or telling stories since these responses often did not expand on what was meant. Responses that were answering the question but seemed to not refer to chemistry concepts were also included here because we believed they were answering the question, but their meaning and context was uncertain. The “Not Applicable” category included instances when students did not give any response at all, or when their response was unclear or unrelated to the question posed.

The analysis of student responses from the OCLUE and traditional courses revealed differences in the perceptions of expectations of thinking regarding reaction mechanisms. As noted in Fig. 1, more OCLUE students perceived the need to engage in more sophisticated ways of thinking about mechanisms than students in the traditional course. For example, 30.6% (n = 76) of OCLUE students perceived that they were expected to apply what they knew to navigate their way through new and unforeseen problems and to provide a reason for why these mechanisms proceed. This compares to 13.2% (n = 80) of students from the traditional course perceiving the same. For the category “Identify and Describe”, 21.4% (n = 53) of OCLUE students and 17.4% (n = 105) of students from the traditional course perceived this expectation. While the number of students from both types of courses who perceived the need to apply heuristics were quite similar (12.9%, n = 78, of the traditional students and 10.9%, n = 27, of OCLUE students), there was a much larger difference in extent to which students perceived they had to memorize material. Of the traditional student cohort, 20.9% (n = 126) of students perceived the need to memorize information on organic chemistry mechanisms with just 2.8% (n = 7) of OCLUE students perceiving the same. Finally, more students from the traditional course (24.5%, n = 148) used general terms to describe how they were expected to think about organic chemistry mechanisms than OCLUE students (16.1%, n = 40). The “Not Applicable” category included instances when students did not give any response at all (and there were very few of these responses across all three questions), or when their response was unclear or unrelated to the question posed. More students from the OCLUE course (18.1%, n = 45) had responses categorized to the “Not Applicable” category than students from the traditional course (11.1%, n = 67).

Fig. 1 Percentages of student responses in each category/code for Question 1.

Question 2: most difficult thing

The second question asked, “What would you tell them is the most difficult thing about organic chemistry?” Detailed explanations of each category as well as associated examples of student responses can be found in Table 3.

Table 3 Codebook for Question 2: perceptions of the most difficult thing about organic chemistry

Code	Dimensions	Example quotes
Apply and reason	Student responses that include one or more of the following dimensions in regard to their perceptions of what is the most difficult thing about organic chemistry: (1) understanding “why” a reaction proceeds; (2) the use of knowledge, specifically with the use of fundamental or basic ideas; (3) the transfer of knowledge to new problems; (4) making connections between concepts or piecing/linking concepts together, especially in order to apply them; (5) making predictions in order to solve a problem	Traditional: “Understanding why mechanism happen the way they do”
	OCLUE: “Realizing that you are not going to memorize every reaction, you just need to worry about patterns and reasons why things happen a certain way”
Identify and describe	Student responses that include one or more of the following dimensions in regard to their perceptions of what is the most difficult thing about organic chemistry: (1) understanding the “what” and “how” reactions proceed, particularly without mentioning the use of knowledge to understanding “why” reactions proceed; (2) mentions understanding at the scalar or one scalar below levels, particularly through the recognition of concepts such as polarity and electronegativity and their significance to understanding; (3) when a student explicitly mentions understanding the mechanism instead of memorization; (4) when a student mentions “differentiating” between reactions with any further explanation; responses includes a discussion of forces, charges, or stabilization. NOTE: responses that simply mention “knowledge” or “understanding” do not receive this code	Traditional: “For me, it was rotating molecules around in my head and understanding how each reaction condition affects the products.”
		OCLUE: “Understanding how each reagent indicates different mechanisms between structures.”
Specific topic	Student responses that include one or more of the following dimensions regarding their perceptions of what is the most difficult thing about organic chemistry: (1) listing off specific topics, particularly with no reference to understanding or approaches utilized. Most common specific topics mentioned include mechanisms, acid–base reactions, learning objectives, naming, synthesis, and spectroscopy; (2) explicit mention of “concepts” without expanding on what they mean (i.e., “understanding concepts” or “knowing a mechanism”)	Traditional: “the synthesis problems”
		OCLUE: “The most difficult thing is CNMR and HNMR so if you can learn that you can learn anything”
Memorization and/or workload	Student responses that include one or more of the following dimensions in regard to their perceptions of what is the most difficult thing about organic chemistry: (1) memorization, remembering, recalling, or regurgitation of reactions, products, reagents, and/or mechanisms; (2) knowing reactions, products, reagents, and/or mechanisms, particularly with no explicit mention of understanding the “what”, “how”, or “why” a reaction proceeds; (3) mention of the large amount/volume of material, the large amount of studying, and/or the amount of time the course requires; (4) explicit mention of feeling overwhelmed with the course; (5) mention of difficulty with keeping up with the class	Traditional: “The memorization of content. Its a lot of information.”
		OCLUE: “The most difficult thing about organic chemistry is how many mechanism you have to know. It can get a bit overwhelming, but if you try to practice once a day, and keep up with your notes then it won't be as bad.”
Personal, course, and/or exam aspects	Student responses that include one or more of the following dimensions regarding their perceptions of what is the most difficult thing about organic chemistry: (1) mention of personal action and/or behaviors that a student must have such as “staying motivated” or “staying on top of the material” or “being patient”; (2) mention of how a student must regulate actions and behaviors to complete the course; (3) when a student discusses aspects of the course or exams such as overall difficulty or time allotted to an exam	Traditional: “the most difficult part is holding yourself accountable to continue studying throughout the semester”
		OCLUE: “The most difficult part is the self discipline that is required in order to make sure you learn everything that is being offered to you in this course.”
Not applicable	Student responses that include one or more of the following dimensions in regard to their perceptions of what is the most difficult thing about organic chemistry: (1) the response does not answer the question; (2) when a student provides no response at all; (3) when the response is unclear or interpretation of the response is difficult, such as when students say “understanding the material”; (4) when the response falls into no other category; (5) when a student is venting about the course, professor, or other aspects relevant to the course	Traditional: “nothing”
		OCLUE: “Literally all of it”

---

The “Apply and Reason”, “Identify and Describe”, and “Memorization” categories align to previous explanations of the same categories in question 1 (expectations of thinking). The only category identified that was unique to the responses to this question was the “Personal, Course, and/or Exam Aspects” category. Responses for this category typically reported personal actions or behaviors such as “staying motivated”, “staying on top of the material”, or “being patient” as well as referring to facets of the course (i.e., the professor and grading schemes) and exams (i.e., format). These types of responses received their own category due to their prevalence in the data, unlike in question 1 (expectations of thinking) where there were so few of these types of responses that they were assigned to the “Not Applicable” category. In question 1 (expectations of thinking), “Memorization” had an entire category of its own; however, for question 2 (most difficult thing) many students, particularly in the traditional course, coupled their perception of memorization with a large workload that was “overwhelming” or included a “high speed of coverage”. Initially for this question, there were separate “Memorization” and “Workload” categories, but since it was difficult to determine whether to classify these responses separately as “Memorization” or “Workload” we decided to combine the codes together as it still allowed us to make a broad comparison of the two courses.

However, to further explain our rationale for the combination of “Memorization” and “Workload”, 355 out of 604 responses in the traditional course discussed “Memorization”, “Workload”, or both. Over 25% of the traditional students mentioned “Memorization” and “Workload” simultaneously. Given this sizable chunk of data in one of the courses mentioned both together, we opted to combine them especially given that the narrative we were interpreting did not change based on combining the two categories and it allowed for noting broad themes and patterns across all 852 responses.

Two other categories, namely “Specific Topic” and “Not Applicable” were also identified. The “Specific Topic” category included responses which listed discrete specific topics that students found difficult. Throughout these responses, students did not make any reference to ways of thinking used to interpret the content associated with these topics. The common topics mentioned by students included “acid–base reactions”, “naming”, “synthesis”, and “spectroscopy”. In contrast to question 1 (expectations of thinking), the “Not Applicable” category for question 2 (most difficult thing) did not include references to the course or instructor as they were coded separately, but it did include instances when students did not give any response at all or when the response was unclear or unrelated to the question posed.

The analysis of student responses from the OCLUE and traditional courses again revealed differences in how students perceived the difficulty of learning organic chemistry. As shown in Fig. 2, more OCLUE students perceived that more sophisticated ways of thinking such as “Apply and Reason” and “Identify and Describe” were the most difficult thing about learning organic chemistry compared to students in the traditional course. For example, 16.5% (n = 41) and 16.1% (n = 40) of OCLUE students perceived that the most difficult aspects of learning organic chemistry were applying and reasoning and identifying and describing, respectively. This compares to the 2.6% (n = 16) and 5.1% (n = 31) for students in the traditional course for those same categories. More students from the traditional course listed specific topics (18.4%, n = 111) and the memorization and/or workload aspect (58.8%, n = 355) as the most difficult part of learning compared to OCLUE students (17.7%, n = 44, and 17.7%, n = 44, respectively).

Fig. 2 Percentages of student responses in each category/code for Question 2.

Question 3: assessment

The third and final question asked, “How would you describe to them what is assessed in organic chemistry?” Detailed explanations of each category as well as associated examples of student responses can be found in Table 4.

Table 4 Codebook for Question 3: perceptions of what was assessed

Code	Dimensions	Example quotes
Apply and reason	Student responses that include one or more of the following dimensions regarding their perceptions of what is assessed: (1) understanding “why” a reaction proceeds; (2) the use of knowledge, specifically with the use of fundamental or basic ideas; (3) the transfer of knowledge to new problems; (4) making connections between concepts or piecing concepts together, especially to apply them; (5) making predictions to solve a problem	Traditional: “you don't just memorize; you understand why they are made like that so you can apply it to other reactions”
		OCLUE: “We were expected to know WHY things were happening, not just what was going on but the driving force behind those reactions”
Identify and describe	Student responses that include one or more of the following dimensions in regard to their perceptions of what is assessed: (1) understanding the “what” and “how” reactions proceed, particularly without mentioning the use of knowledge to understanding “why” reactions proceed; (2) mentions understanding at the scalar or one scalar below levels, particularly through the recognition of concepts such as polarity and electronegativity and their significance to understanding; (3) when a student explicitly mentions understanding the mechanism instead of memorization; (4) when a student mentions “differentiating” between reactions with any further explanation. (5) responses include a discussion of forces, charges, or stabilization. NOTE: responses that simply mention “knowledge” or “understanding” do not receive this code	Traditional: “You need to know how to classify and name molecules, know characteristics like acidity and aromaticity, and mostly know how bonds are formed and broken in different situations using different molecules.”
		OCLUE: “You are required to think about reactions more about how electrons are moved in a system rather than what the begining and end products are. you have to know the steps of how to get there.”
Specific topic	Student responses that include one or more of the following dimensions regarding their perceptions of what is assessed: (1) listing off specific topics, particularly with no reference to understanding or approaches utilized. Most common specific topics mentioned include mechanisms, acid–base reactions, learning objectives, naming, synthesis, and spectroscopy; (2) explicit mention of “concepts” without expanding on what they mean (i.e., “understanding concepts” or “knowing a mechanism”)	Traditional: “there is naming, mechanism, nmr, lots of reactions, and some bonus questions”
		OCLUE: “Different types of reactions and the classifications of structures.”
Memorization	Student responses that include one or more of the following dimensions regarding their perceptions of what is assessed: (1) memorization, remembering, recalling, or regurgitation of reactions, products, reagents, and/or mechanisms; (2) knowing reactions, products, reagents, and/or mechanisms, particularly with no explicit mention of understanding the “what”, “how”, or “why” a reaction proceeds	Traditional: “The majority of the exams are memorization of the reactions.”
		OCLUE: “Mechanisms and if you can memorize 20 different types of problems with the same molecule everytime.”
Exam format and aspects	Student responses that include one or more of the following dimensions regarding their perceptions of what is assessed: (1) the course materials leveraged on the exam such as lecture notes, homework, practice exams, and/or recitation materials; (2) the format of the exam, such as stating the types of questions on the exam (i.e., multiple-choice, or short answer); (3) the length, time, or fairness of the exam	Traditional: “You need to go to lecture and take notes, because the exams cover pretty closely what we cover in lecture.”
		OCLUE: “your ability to do them as fast as possible since the exam were only 50 minutes and crammed with material”
Not applicable	Student responses that include one or more of the following dimensions in regard to their perceptions of what is assessed: (1) the response does not answer the question; (2) when a student provides no response at all; (3) when the response is unclear or interpretation of the response is difficult, such as when students say “your understanding of the material”; (4) when the response falls into no other category; (5) when the response focuses on student actions (i.e., “be sure to study hard”); (6) when a student is venting about the course, professor, or other aspects relevant to the course and the response focuses on the course or the professor such as frustrations they have with the course or professor	Traditional: “Everything”
		OCLUE: “don’t take 3 other intens classes with it”

---

Codes such as “Apply and Reason”, “Identify and Describe”, and “Memorization” were explained in previous questions. A further three categories, namely that of “Specific Topic”, “Exam Aspects”, and “Not Applicable” were also identified and included in the analysis. The “Specific Topic” category is similar to the category in question 2 (most difficult thing) and included responses where students listed discrete specific topics as what gets assessed in organic chemistry. Once again, throughout these responses, students did not make any reference to ways of thinking used to interpret the content associated with these topics. The common topics mentioned by students here included “synthesis reactions”, “naming”, “NMR”, and “spectroscopy”. The “Exam Aspects” category included responses that referred to the format, length, time, and/or fairness of the exams in response to what gets assessed. These responses were noted in question 2 (most difficult thing), but were subsumed into the “Personal, Course, and/or Exam Aspects” category. Responses which noted perceptions of what course materials are typically assessed (such as lecture notes, homework, practice exams, and/or recitation materials) were also included in the “Exam Aspects” category. The “Not Applicable” category included instances when students did not give any response at all, or that their response was entirely unclear and unrelated to the question posed.

The analysis of student responses from the OCLUE and traditional courses to this question revealed differences in how students perceived what gets assessed in organic chemistry. As shown in Fig. 3, more OCLUE students perceived that more sophisticated ways of thinking were assessed in their course compared to students in the traditional course. For example, in relation to “Apply and Reason” and “Identify and Describe”, 35.5% (n = 88) and 12.5% (n = 31) of OCLUE students perceived these modes of thinking were assessed, respectively. This compared with 4.6% (n = 28) and 6.5% (n = 39), respectively, for students in the traditional course. More students from the traditional course listed specific topics (52.2%, n = 315) and memorizing information (11.8%, n = 71) as how they perceived they were assessed compared to OCLUE students (12.9%, n = 32, and 2.0%, n = 5, respectively). However, more OCLUE students noted responses coded to other categories such as “Exam Aspects” (27%, n = 67) and “Not Applicable” (10.1%, n = 25) than students from the traditional course (18.7%, n = 113, and 6.3%, n = 38, respectively).

Fig. 3 Percentages of student responses in each category/code for Question 3.

Superordinate themes

While the analysis of the open-ended student responses was conducted separately for the three questions, each yielded similar results. To note broader trends across the results and more easily communicate and discuss the findings, we grouped the results into three superordinate themes for each question. In the first theme, which we refer to as “Use of Knowledge”, are “Apply and Reason” and “Identify and Describe”. The second theme encompasses student responses that are more rote, formulaic, or surface level, do not imply ways of thinking but rather the idea that topics must be memorized, or that students must refer to rote methods used to think through problems. This theme is therefore called “Rote Knowledge” and includes categories like “Memorization”, “Apply Heuristics”, and “Specific Topic”. Categories such as “Generalities”, “Personal, Course, and/or Exam Aspects”, “Exam Aspects”, and “Not Applicable” captured responses that did not answer the question or were vague and uninformative. We refer to this theme as “Other” for our purposes. By condensing the codes in this way, we believe it is easier to see patterns in responses as related to how knowledge is used in these courses. In all three questions we saw marked differences between the “Use of Knowledge” and the “Rote Knowledge” themes for OCLUE and traditional students, while the responses coded as “Other” were more similar across the two cohorts. Because responses coded as “Other” were, in general, not specific enough to make inferences about the course culture and concomitant types of thinking required we will not discuss them in detail here. We opted to focus on how knowledge was used in our superordinate themes to better complement our previous research on student reasoning in the context of these two courses, and it seemed to be the most prevalent and overarching way to organize our analysis based on how students were responding to the questions. For question 1 (expectations of thinking), around 50% of OCLUE students believed that they were expected to reason with or use their knowledge in the context of drawing mechanisms, while 15% believed that this process was a more rote procedure. This split was more equal for traditional students with around 30% in “Use of Knowledge” and around 34% in “Rote Knowledge” as noted in Fig. 4. Responses from question 1 (expectations of thinking) such as “You’re expected to not memorize the reactions but understand why moelcules [sic] react the way they do so you can draw your own reactions” and “You think about where electrons are going and what it's bonding with and why it bonds with one thing over another” were classified as “Apply and Reason” because they include the idea that students must not only use their knowledge to do something, but also explain or understand why the phenomenon occurs. There is a subtle distinction between “Apply and Reason” responses and those that were classified as “Identify and Describe”. For example, one “Identify and Describe” response states: “Energy flow, electron flow, ect. You need to be able to understand how electrons are moving and see relationships throughout the year.” This response focuses more on the “how” a reaction happens rather than the “why”, and it highlights the need to identify relationships; though it does not mention knowing why, it still implies the use of knowledge. In contrast, responses such as “you need to memorize all reactions given to you in lectures!” which was classified as “Memorization” and “they need to think about mechanistic arrows as the movement of electrons from a source to a sink”, (coded as “Apply Heuristics”) indicate that students have not moved towards the use of knowledge but implies they are using rote procedures.

Fig. 4 Percentage of student responses in each superordinate theme for Question 1.

For question 2 (most difficult thing) there was an even more marked difference between the two cohorts as noted Fig. 5. OCLUE students were evenly split on what aspects of the course they perceived as more difficult, whereas almost 80% of the traditional students believed that the focus on “Rote Knowledge” was the most difficult. Here, we recall back to Fig. 2 where it can be noted that more students in the traditional course perceived that the memorization, workload, or the workload involved in memorizing a large amount of material was what made the course difficult. For example, one OCLUE student's response categorized as “Use of Knowledge” stated the following: “The most difficult thing in orgo [sic] is the mechanism and understanding where and how different molecules attack each other. If you know them well then it makes writing reactions easier”. This student highlights that when you understand the behaviors of different molecules, then this can make writing reactions more approachable. On the other hand, an example from the traditional course categorized as “Rote Knowledge” noted that: “There is a large amount of material that we have to know and memorize”. In this case, the student is not only perceiving a large workload, but they also perceive they are expected to memorize the material.

Fig. 5 Percentage of student responses in each superordinate theme for Question 2.

The differences in perceptions was continued in responses to question 3 (assessment; Fig. 6), where once again a majority (48%) of OCLUE students perceived an emphasis on the use of knowledge in course assessments, whereas 64% of traditional students perceived that they were being assessed on memorization and rote knowledge. For example, one OCLUE student's perspective on this question was the following: “Your knowledge not only of what is taught in class but your ability to apply it to various situations. Also, you [sic] knowledge of the CONCEPTS [sic] and underlying themes is heavily assessed.” Here, the student describes how OCLUE assesses the student's ability to transfer concepts from one problem to another and to identify underlying themes which correlated with “Use of Knowledge”. On the other hand, responses correlated with “Rote Knowledge” included: “how well you can memorize the reactions” and “the exams mainly test reactions and naming of molecules”. Here the responses in the traditional course cluster around memorization and the focus on discrete, specific topics, such as knowing reactions and nomenclature, rather than ways of thinking.

Fig. 6 Percentage of student responses in each superordinate theme for Question 3.

To further determine if the differences between OCLUE and traditional cohorts in the qualitative data was supported statistically, we conducted a Pearson's chi-square test of independence using an alpha of 0.05 within SPSS 27 (SPSS, 2020) with the data organized into superordinate themes. According to the chi-square tests, the analysis for each question yielded statistically significant results at an alpha of 0.05 where p < 0.001 for each question. Since all Pearson chi-square tests came back significant, we decided to run post-hoc analyses to further illustrate which theme(s) were primary drivers for statistical significance in the initial chi-square tests. From the post-hoc analyses we found that the “Use of Knowledge” and “Rote Knowledge” themes were strong primary drivers for significance in each question. All of the calculations and a more in-depth write up of these analyses can be found in the supplemental materials (Tables S1 and S2, ESI).

Discussion

Interpreting the findings through the lens of classroom cultures

The findings highlighted clear differences in the ways that students perceived knowledge use in the organic chemistry courses in this study. As noted throughout, the two organic chemistry courses had different pedagogical underpinnings; that is, the courses were designed, enacted, and assessed in different ways. Our previous research on student reasoning has demonstrated that students in OCLUE are better able to engage in causal mechanistic reasoning and retain this ability longer than students in traditional courses (Crandell et al., 2019, 2020). Therefore, we were aware of what students were doing; however, with this study, we wanted to know if students were aware of what they were doing. In other words, we wanted to know if students perceived the intent of the transformation, and we did not want to make assumptions without conducting this study. In the beginning, our goals were exploratory. We aimed to complement our previous work and characterize our transformational efforts further from the student perspective. This also allowed us to generate insights on how student perceptions aligned with course goals and expectations, our transformational intent, and the theories of learning that informed our course design. As we began analysis, we sought a way to further make sense of the findings.

Across all three questions, more students in OCLUE had perceptions aligned with the use of knowledge while student perceptions in the traditional course aligned more with rote knowledge. As we noted these differences in student perceptions, our interpretations and discussions often centered on the classroom cultures of learning in each organic chemistry course. Certainly, learning is a social and cultural activity that is dependent on the context in which it occurs (Vygotsky, 1978; Rogoff, 1990; Calabrese Barton et al., 2008; Carlone et al., 2011). Therefore, our discussion and interpretation of our findings can be situated within sociocultural perspectives (Vygotsky, 1978; Rogoff, 1990; John-Steiner and Mahn, 1996; Carlone et al., 2011; Zotos et al., 2020; Petterson et al., 2022) and is informed by other scholars who have conceptualized culture (Schein and Schein, 2016; Reinholz and Apkarian, 2018). Since the term “culture” can take on a variety of meanings, we find it important to provide a working definition prior to discussing our findings.

To begin, it is important to note that “no one view of culture… represents a thorough and complete understanding” (Parsons and Carlone, 2013). However, throughout this discussion when we refer to culture, we are referring to a micro-level culture, or a subculture, that exists in the context of these organic chemistry classrooms, as opposed to macro-level cultures which represent larger entities such as ethnic groups, nations, and international organizations (Schein and Schein, 2016; Thoman et al., 2017). Aside from sociocultural perspectives, our view of culture draws heavily on Reinholz and Apkarian's four frames for systemic change (Reinholz and Apkarian, 2018) and Schein and Schein's framework for organizational culture (Schein and Schein, 2016). Reinholz and Apkarian's four frames include structures, symbols, people, and power which exhibits overlap with Schein and Schein's framework of artifacts, espoused beliefs and values, and taken for granted assumptions, both of which inform our working definition and are further explained in our working definition below.

For us, our working definition of culture includes a constellation of visible structures and artifacts which encompass the visible course policies, course practices, expectations, and assessments, among other features. These structures are given meaning by an underlying system of symbols that include beliefs, values, and assumptions. Socializing mechanisms in a context enculturate people by encouraging them to adopt the symbols and participate in or interact with the structures and artifacts. These socializing mechanisms are mediated by people and power that directly and indirectly impact how people talk, act, and think (Rogoff, 1990; Miller and Goodnow, 1995; Lemke, 2001; Gutiérrez and Rogoff, 2003; Nasir and Hand, 2006; Calabrese Barton et al., 2008; Schein and Schein, 2016; Reinholz and Apkarian, 2018; Deng et al., 2021). While our findings cannot speak to all frames (structures/artifacts, symbols, people, and power), this definition helps us suggest potential explanations for our findings.

For this study, we found the cultural frames of structures/artifacts and symbols most useful particularly because most student responses were related to these frames given the questions asked. Structures, or artifacts, within a classroom could be elements such as the practices used, the learning and assessment tasks, and the established norms. That is, they are the visible features of the culture that are informed by the underlying symbols that give them meaning. The symbols could include the implicit and explicit messages that students receive and interpret that communicate valued ways of knowing and doing (Schein and Schein, 2016; Reinholz and Apkarian, 2018). The other frames mentioned by Reinholz and Apkarian, such as people and power are important, but were difficult to address with this data. Therefore, we aimed to use the frames of structures/artifacts and symbols to discuss how students perceived they were expected to practice learning and what was valued, both of which will be linked to elements of their respective cultures of learning. Other studies have used sociocultural perspectives to explore different classroom cultures and found that when the use of certain practices, such as argumentation, align with the course goals, then students engage more productively in the practice (Sandoval et al., 2019) while others have demonstrated how classroom norms (and their interpretation) can impact how students respond to learning tasks (Becker et al., 2013; Chang and Song, 2016). Thus, this suggests that better alignment between course goals, the practices students engage in, and clear and universally understood norms can lead to a more productive learning experience. Therefore, by investigating student perceptions of what is valued through the lens of classroom cultures we can help identify potential mismatches between instructor expectations and what students are doing that may perturb learning.

Question 1: expectations of thinking

Question 1 (expectations of thinking) was included in our instrument for three main reasons: (1) it helped us address one of our research questions regarding the alignment of student perceptions with transformational intent; (2) it related to our previous work on student reasoning in these courses (Crandell et al., 2019, 2020; Houchlei et al., 2021); and (3) it was inspired by previous research that found students in organic chemistry often resorted to memorization (Bhattacharyya and Bodner, 2005). As noted in Fig. 4, more OCLUE students perceived they were expected to use their knowledge while more students in the traditional course perceived they were expected to rely on rote knowledge. Earlier we noted the most salient differences between the two courses in this study, and we highlighted that OCLUE consistently encourages students to construct scientific explanations and arguments about how and why something happens. In the context of question 1 (expectations of thinking), these enacted practices in the course were also noted in student perceptions. That is, the expectations and emphasis on the use of knowledge were perceived by many students, indicating that student perceptions were at least partially aligning with the transformation goals (Cooper et al., 2019).

Constructing explanations and engaging in argumentation are important classroom practices in OCLUE (National Research Council, 2012a; Cooper et al., 2019; 3DL4US, n.d.; Flaherty, 2020b). Their incorporation coupled with the expectation students will engage in them act as structural features of the overarching culture. It has been suggested that by implementing the practice of constructing explanations that students will have a better idea of how scientific knowledge is developed (McNeill et al., 2017) and that argumentation can move the focus of learning away from memorization (Berland and McNeill, 2010). Certainly, this data corroborates this claim. Structural features of the classroom culture, such as the incorporation and consistent use of scientific practices, may have helped students in OCLUE perceive expectations of how to think on a deeper level relative to students in the traditional course.

Question 2: most difficult thing

Question 2 (most difficult thing) was incorporated into the study for two reasons: (1) previously published instruments asked about course difficulty; and (2) we believed it would be insightful to know about what aspects of a course students found to be most difficult in case it needed to be addressed. For example, if most students found a course policy to be more difficult than a way of thinking or content, then we would have viable feedback in order to address this.

For question 2 (most difficult thing), an overwhelming majority of students in the traditional course perceived that “Rote Knowledge” (such as memorization, the workload, or the workload associated with memorizing) was the most difficult part of the course (as noted in Fig. 5). In contrast, OCLUE students had a more even distribution of perceptions of which facets were most difficult, though more OCLUE students perceived that the “Use of Knowledge” was the most difficult aspect when compared to the traditional course. The OCLUE curriculum was designed in such a way to discourage rote memorization of content (Cooper et al., 2019), and far fewer students in OCLUE perceived memorization and workload as being the most difficult aspect of the course when compared to students in the traditional course. This highlights that student perceptions in OCLUE exhibit alignment with the transformational intent and implies that memorization and workload are stronger driving influences or forces within the culture of the traditional course.

Students have perceived that organic chemistry requires a great deal of memorization (Moran, 2013) which has also been noted as an approach that students take on organic chemistry exams (Webber and Flynn, 2018). Furthermore, instruments focused on gathering student perceptions have sought to collect information on whether students are memorizing in their courses (Grove and Bretz, 2007), yet, considering the previous exploration of the association of memorization with organic chemistry, a course centered on rote knowledge is almost certainly not the intent of the instructors. In a qualitative study on student reasoning in organic chemistry Anderson and Bodner (Anderson and Bodner, 2008) found that students did not appreciate that mechanisms were used to understand how and why phenomena occur despite the fact that this was the intent of the instructor in that course. In the same study, students in their interviews stated they wanted to understand the material on a deeper level but also mentioned that this was difficult given the volume and pace of the material (Anderson and Bodner, 2008), a perception we noted in our study for the students in the traditional course. That is, structural components of the traditional classroom culture, such as the amount of material covered, and pace of coverage may coalesce with perceived expectations to drive the perception that students need to memorize large amounts of material.

If instructors want students be able to explain how and why chemical reactions happen, the findings from Anderson and Bodner and our study make it clear that the purpose of mechanisms needs to be made explicit and leveraged consistently throughout the course and that courses need to slow down and connect content back to fundamental principles so that students can develop a robust understanding which may not have been clear in the traditional course. Both points are addressed in OCLUE by leveraging the scientific practices, crosscutting concepts, and core ideas and is further evidenced by the shift in perceived difficulties of students in the course toward use of knowledge, relative to students in the traditional course.

Question 3: assessment

Question 3 (assessment) was used in this study for three reasons: (1) considering that research has shown that assessment practices send strong messages to students about what is valued (Momsen et al., 2013; Stowe et al., 2021), we saw this as a useful question for ascertaining what students perceived are valued ways of doing and knowing; and (2) from our previous work and observations of the two types of courses in this study, we have known them to have different assessment approaches and wanted to explore student perceptions of these two approaches.

Finally, the responses for question 3 (assessment) yielded similar patterns to question 1 where, in general, OCLUE students perceived they were assessed more on their “Use of Knowledge” while students in the traditional course perceived they were assessed more on “Rote Knowledge” such as memorization and discrete, specific topics (as can be seen in Fig. 6). It is important to reiterate that assessments play a large role in the culture of a learning environment and send strong messages about valued ways of thinking and participating in the course (Snyder, 1973; Crooks, 1988; Entwistle, 1991; Scouller and Prosser, 1994; Scouller, 1998; Momsen et al., 2013; Stowe et al., 2021). Within OCLUE, much work is done to ensure alignment between learning goals, expectations, and assessments with regard to the use of knowledge. Student perceptions imply that this transformational goal may be accomplished (at least partially) since 52% of students in OCLUE perceived they were expected to use their knowledge and 48% perceived they were assessed on their use of knowledge.

In terms of culture, assessments act as one mechanism through which instructors reflect what is valued in the learning culture and what students are expected to do. As shown in this study, these messages can be perceived by students and influence how they participate in learning. If the goal of the learning environment is to engage students in reasoning and disciplinary practices, then the culture and instructor expectations must support that goal (Bain et al., 2020), as Cooper and Stowe note: “…it is important for students to receive and respond to the message that both knowledge and the ways that knowledge is used are crucial aspects of learning chemistry,” (Cooper and Stowe, 2018). Instructor expectations and assessments are intricately linked, and the ways in which courses communicate expectations, emphasize particular ways of doing, and place value on those ways of doing (by assessing them) become structures and symbols of the learning culture. The alignment of expectations and assessment in OCLUE, as noted in student perceptions, was an important component of the transformation effort to ensure that what was expected of students was valued in the form of points on assessments.

The impact of the transformed classroom culture

We set out to investigate whether student perceptions of what they were expected to do and what was valued aligned with the transformational intent of OCLUE and the theories of learning that informed it. We saw this study as complementing our previous research on student reasoning in the context of OCLUE and a traditional organic chemistry courses. Though the study does address these aims, it continued to evolve throughout data analysis. To make sense of the data and situate it within the literature, we discussed the results through the lens of elements of the classroom cultures. That is, the differences noted between student perceptions in these two organic chemistry courses could be attributed to the structures/artifacts (i.e., expectations, learning task design, assessment design, etc.) and the symbols (i.e., the intentional and/or unintentional valuing of certain ways of doing) within the course that are supported by the instructors. The difference between the design and enactment of these two types of courses not only have impacts on how students reason (as shown in our previous research), but it also impacts how students perceive they are to engage in doing and learning organic chemistry which may reflect classroom norms of engagement and learning that are more or less aligned with the disciplinary practice (Becker et al., 2013; Schein and Schein, 2016; National Academies of Sciences, 2018; Reinholz and Apkarian, 2018; Sandoval et al., 2019).

When considering the culture of a classroom, it becomes important to also consider the ways that culture socializes people. Instructional practices that focus on rote memorization and solving exercises will likely not introduce students to the authentic disciplinary culture nor encourage them to engage in “science-as-practice” (Nasir and Hand, 2006; Stroupe, 2014). Instead, if students are immersed in an environment where they are encouraged to use their knowledge, particularly with unfamiliar problems, and given the chance to make mistakes and learn from them, then students may develop perceptions of learning which are more aligned with authentic disciplinary ways of thinking (Brown et al., 1989). Furthermore, if a class culture's goals align with the practices students are expected to engage in, then it can lead to more productive engagement and learning (Sandoval et al., 2019). By leveraging scientific practices in the context of fundamental core ideas, instructors can shift the culture of learning to expect, emphasize, and value the use of knowledge and provide students a route to connect their knowledge and make sense of a phenomena rather than relying on memorization (Cooper, 2015).

One of the goals of our study was to explore the alignment between student perceptions and instructor expectations. Previous research has found that organic chemistry instructors do not list rote memorization as an important facet of learning organic chemistry (Duis, 2011); yet, students in the traditional organic chemistry in this study largely perceived they were expected to and assessed on their ability to memorize. We imagine that the goal of the instructor was not to have students rely solely on rote memorization. Therefore, there seems to be a disconnect and misalignment between what the instructor values and expects students to do and their learning and assessment task design (Stowe and Cooper, 2017). That is, though instructors may expect and value students to use their knowledge, students are still able to complete prompts and learning tasks by memorizing the material. In a recent interview study on student perceptions of “critical thinking” in organic chemistry courses, some students mentioned they saw memorization as an “easier” method to achieve the results they wanted (a better grade) and that they were accustomed to memorizing in school (Bowen and Cooper, manuscript in preparation). Thus, this highlights that more attention should be given to the questions and prompts being asked of students and that learning and assessment task design be intentional and reflective.

While there are a variety of ways to engage students in meaningful learning, pedagogical approaches such as those informed by A Framework for K-12 Science Education and three-dimensional learning have advocated for engaging students in authentic disciplinary practices in science (National Research Council, 2012a; Laverty et al., 2016; Matz et al., 2018; 3DL4US, n.d.). From these perspectives, learning involves introducing students to the disciplinary cultures of science by engaging them in the practices that scientists actually use, such as constructing explanations and using models to predict and explain. Our previous work on OCLUE, along with the findings here, demonstrate how three-dimensional learning can impact student performance and communicate clear expectations and values that are explicit for students and align with more expert-like practice.

Certainly, there are many factors at work in a learning culture, and our study did not, and could not, address them all. However, what we have highlighted is how instructor expectations, whether implicit or explicit, along with what is emphasized in a course and on assessments are related to the overarching classroom culture and how these features send strong messages about how people should think and practice. While our previous work on student reasoning was certainly insightful, we needed evidence to better understand if students perceived what they were doing aligned with the goals of the course. Put simply, the enactment of a course, the elements of its culture, such as instructor expectations, emphasis, and valued of ways of doing influence how students participate in learning and more attention should therefore be given to these influences when designing and enacting instructional practice.

Limitations

To begin, our three open-ended questions, though interesting to us, were not all-encompassing. Though the questions were open-ended enough to provide students the opportunity to comment on their instructors, we did not have a question directly asking students about the role of the instructor on their perception. Additionally, the large number of responses allocated to the “Other” theme is in part due to the “Generalities” category which was applied when students used vague, generalized language that was unclear. For example, many students mentioned they had to think “critically” but did not elaborate on what that entailed. It could be the case that some students did not have the vocabulary to explain what they meant and might not have been able to be more precise in their explanation because they have not been exposed to the notion of using knowledge to predict and explain. Studies are underway to identify scaffolded approaches to help students answer the questions we intended and clarify future responses. However, finding the right level of scaffolding takes time, as this approach can “over-prompt” students, which is not desirable in the context of these studies.

A third limitation is that our analytic approach relied on interpretation of written student perceptions. With any qualitative study, we must acknowledge that our interpretations are our own and contextual. Through the use of multiple coders, a unified codebook, and multiple cycles of coding and revisions, we aimed to characterize student perceptions the best way we could through noting broad and communicable themes across these two different organic chemistry course experiences. Finally, it could be that these perceptions may not be stable over time. Like other affective constructs, perceptions are subject to social and cultural influences and therefore may change throughout the semester; however, we do believe perceptions offer a snapshot into the student experience and worth considering in our transformation efforts.

Implications

For teaching implications, our approach to exploring student perceptions could be useful to instructors who are interested in how their courses are being perceived by students. It is clear from the findings that students in the traditional organic chemistry course perceived that they must memorize a great deal of material, yet it is highly unlikely that instructors intended for this to be the case. Studies on what organic instructors believe is important do not mention rote memorization (Duis, 2011), and the need for taking organic chemistry is often supported by the assertion that it fosters forms of critical thinking and problem solving (Stowe and Cooper, 2017). If anything, these results imply that if instructors desire students to use and apply their knowledge and recognize what they are doing, these expectation should be clear, emphasized, and valued by having students engage in these practices on course assignments and assessments.

With regard to research implications, the use of our instrument occupies a methodological middle ground in that it is less constrained than previously published quantitative instruments, and it does not take as much time as conducting interviews, yet the instrument still provides rich descriptions of student perceptions. By studying perceptions in this way, we can provide valuable insight into how students are engaging with curricula and learning environments without relying on our assumptions, and more qualitative studies on student perceptions of learning and affective states would help expand the CER literature base.

With all of this said, various cultural frameworks informed our interpretation and communication of the results. We posit that more work should be done within the realm of classroom cultures in chemistry courses. Culture influences how students talk, think, and act, and research focused on characterizing the different classroom cultures that support learning and foster student engagement would be productive and insightful.

Future directions

This study was exploratory in nature; therefore, we are attempting to rework the language of the questions in an attempt to minimize the number of responses in the “Other” theme while still ensuring that we are not prompting students. Furthermore, since many students used general or vague terms, such as “critical thinking”, to describe their experience, an interview study is planned to explore what students mean in the context of these two courses. We are also expanding this work to other courses such as introductory chemistry and biology to determine how robust the instrument and data analysis are in different contexts. Another side to transformational efforts are instructor expectations and intent. Therefore, we are currently discussing plans to investigate instructor perceptions of what they want students to do. The most time-consuming component of this project was the data analysis. We are currently working with the Automated Analysis of Constructed Response (AACR) tool (Automated Analysis of Constructed Response, n.d.) to train machine learning algorithms to automatically analyze data and categorize according to the codebooks established in this study. Preliminary results are encouraging, and we believe that there is great potential to use this approach as a supplement to classroom data gathering about teaching and learning. However, it's important to note that we do not support the use of this tool for faculty “evaluation” but rather for faculty development. Finally, more work needs to be done to understand the stability of affective constructs, including perceptions. While little work has been done in this area within science education, it is important to note that these constructs may be subject to change based on a variety of social and cultural factors. Therefore, in future studies we are planning to do multiple data collections in a single course. By investigating the perceptions of students within a course over time, some evidence could be provided into what factors of course design will assist in helping keep perceptions as stable as possible so that reliable measurements can be obtained.

Conclusions

This study was designed to investigate student perceptions in the context of transformed and traditional organic chemistry courses. The idea was that this study would complement our previous work on student reasoning and inform our transformation efforts. Using three open-ended questions and inductive thematic analysis we noted significant differences on what students perceived they were expected to do, what was most difficult, and how they were assessed in the transformed and traditional courses. Our interpretation of the findings led us to discuss these differences in the context of the classroom cultures. Overall, we noted that more OCLUE students perceived that the use of knowledge was expected and assessed while more students in the traditional course perceived that memorization was expected and assessed alongside discrete, specific topics. Differences in what students perceived the most difficult aspect of organic chemistry was noted where students in OCLUE perceived the use of knowledge as being most difficult. Using various frameworks, we discussed how the underlying cultures of these classrooms communicated expectations, emphasized the use of knowledge, and valued the use of knowledge differently. Student perceptions acted as a valued feedback mechanism about how course enactments were being experienced and perceived. Therefore, by using student perceptions as a proxy for elements of the classroom culture, we aimed to offer insights into the design and enactment of these courses.

Conflicts of interest

There are no conflicts to declare.

Notes and references

1. 3DL4US, (n.d.), Three-Dimensional Learning for Undergraduate Science, https://3dl4us.org.
2. Adams W. K., Perkins K. K., Dubson M., Finkelstein N. D. and Wieman C. E., (2005), The design and validation of the colorado learning attitudes about science survey, AIP Conf. Proc., 790(April 2015), 45–48.
3. Anderson T. L. and Bodner G. M., (2008), What can we do about “Parker”? A case study of a good student who didn’t “get” organic chemistry, Chem. Educ. Res. Pract., 9(2), 93–101.
4. Automated Analysis of Constructed Response, (n.d.), Home, https://beyondmultiplechoice.org.
5. Bain K., Bender L., Bergeron P., Caballero M. D., Carmel J. H., Duffy E. M., et al., (2020), Characterizing college science instruction: The three-dimensional learning observation protocol, PLoS One, 15(6).
6. Banks G., Clinchot M., Cullipher S., Huie R., Lambertz J., Lewis R., et al., (2015), Uncovering chemical thinking in students’ decision making: A fuel-choice scenario, J. Chem. Educ., 92(10), 1610–1618.
7. Barbera J., Adams W. K., Wieman C. E. and Perkins K. K., (2008), Modifying and validating the Colorado learning attitudes about science survey for use in chemistry, J. Chem. Educ., 85(10), 1435–1439.
8. Bauer C. F., (2005), Beyond “student attitudes”: Chemistry self-concept inventory for assessment of the affective component of student learning, J. Chem. Educ., 82(12), 1864–1870.
9. Bauer C. F., (2008), Attitude towards chemistry: A semantic differential instrument for assessing curriculum impacts, J. Chem. Educ., 85(10), 1440–1445.
10. Becker N., Rasmussen C., Sweeney G., Wawro M., Towns M. and Cole R., (2013), Reasoning using particulate nature of matter: An example of a sociochemical norm in a university-level physical chemistry class, Chem. Educ. Res. Pract., 14(1), 81–94.
11. Becker N., Noyes K. and Cooper M., (2016), Characterizing students’ mechanistic reasoning about london dispersion forces, J. Chem. Educ., 93(10), 1713–1724.
12. Berland L. K. and McNeill K. L., (2010), A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts, Sci. Educ., 94(5), 765–793.
13. beSocratic, (2020), Home page, https://besocratic.com/home.
14. Bhattacharyya G. and Bodner G. M., (2005), “It Gets Me to the Product”: How students propose organic mechanisms, J. Chem. Educ., 82(9), 1402–1407.
15. Bodner G. M., (1986), Constructivism: A theory of knowledge, J. Chem. Educ., 63(10), 873.
16. Bowen R. and Cooper M., (n.d.), Investigating student perceptions of critical thinking in organic chemistry, manuscript in preparation.
17. Boyatzis R. E., (1998), Transforming qualitative information: Thematic analysis and code development, Sage.
18. Brown J. S., Collins A. and Duguid P., (1989), Situated cognition and the culture of learning, Educ. Res., 18(1), 32–42.
19. Bryfczynski S. P., (2010), BeSocratic: An Intelligent Tutoring System for the Recognition, Evaluation, and Analysis of Free-Form Student Input, Dissertation, Clemson, South Carolina, US: Clemson University.
20. Calabrese Barton A., Tan E. and Rivet A., (2008), Creating hybrid spaces for engaging school science among urban middle school girls, Am. Educ. Res. J., 45(1), 68–103.
21. Carlone H. B., Haun-Frank J. and Webb A., (2011), Assessing equity beyond knowledge- and skills-based outcomes: A comparative ethnography of two fourth-grade reform-based science classrooms, J. Res. Sci. Teach., 48(5), 459–485.
22. Chang J. and Song J., (2016), A case study on the formation and sharing process of science classroom norms, Int. J. Sci. Educ., 38(5), 747–766.
23. Cooper M. M., (2015), Why ask why? J. Chem. Educ., 92(8), 1273–1279.
24. Cooper M. and Klymkowsky M., (2013), Chemistry, life, the universe, and everything: A new approach to general chemistry, and a model for curriculum reform, J. Chem. Educ., 90(9), 1116–1122.
25. Cooper M. M. and Stowe R. L., (2018), Chemistry education research – from personal empiricism to evidence, theory, and informed practice, Chem. Rev., 118(12), 6053–6087.
26. Cooper M. M., Kouyoumdjian H. and Underwood S. M., (2016), Investigating Students’ reasoning about acid–base reactions, J. Chem. Educ., 93(10), 1703–1712.
27. Cooper M. M., Posey L. A. and Underwood S. M., (2017), Core ideas and topics: Building up or drilling down? J. Chem. Educ., 94(5), 541–548.
28. Cooper M. M., Stowe R. L., Crandell O. M. and Klymkowsky M. W., (2019), Organic chemistry, life, the universe and everything (OCLUE): A transformed organic chemistry curriculum, J. Chem. Educ., 96(9), 1858–1872.
29. Crandell O. M., Kouyoumdjian H., Underwood S. M. and Cooper M. M., (2019), Reasoning about reactions in organic chemistry: Starting it in general chemistry, J. Chem. Educ., 96(2), 213–226.
30. Crandell O. M., Lockhart M. A. and Cooper M. M., (2020), Arrows on the page are not a good gauge: Evidence for the importance of causal mechanistic explanations about nucleophilic substitution in organic chemistry, J. Chem. Educ., 97(2), 313–327.
31. Crooks T. J., (1988), The impact of classroom evaluation practices on students, Rev. Educ. Res., 58(4), 438–481.
32. Dalgety J., Coll R. K. and Jones A., (2003), Development of chemistry attitudes and experiences questionnaire (CAEQ), J. Res. Sci. Teach., 40(7), 649–668.
33. Danczak S. M., Thompson C. D. and Overton T. L., (2017), “What does the term critical thinking mean to you?” A qualitative analysis of chemistry undergraduate, teaching staff and employers’ views of critical thinking, Chem. Educ. Res. Pract., 18(3), 420–434.
34. Deng J. M., McMunn L. E., Oakley M. S., Dang H. T. and Rodriguez R. S., (2021), Toward sustained cultural change through chemistry graduate student diversity, equity, and inclusion communities, J. Chem. Educ.
35. Duis J. M., (2011), Organic chemistry educators’ perspectives on fundamental concepts and misconceptions: An exploratory study, J. Chem. Educ., 88(3), 346–350.
36. Entwistle N. J., (1991), Approaches to learning and perceptions of the learning environment, High. Educ., 22(3), 201–204.
37. Flaherty A. A., (2020a), A review of affective chemistry education research and its implications for future research, Chem. Educ. Res. Pract., 21(3), 698–713.
38. Flaherty A. A., (2020b), Investigating perceptions of the structure and development of scientific knowledge in the context of a transformed organic chemistry lecture course, Chem. Educ. Res. Pract., 21, 570–581.
39. Galloway K. R. and Bretz S. L., (2016), Video episodes and action cameras in the undergraduate chemistry laboratory: Eliciting student perceptions of meaningful learning, Chem. Educ. Res. Pract., 17(1), 139–155.
40. Galloway K. R., Malakpa Z. and Bretz S. L., (2016), Investigating affective experiences in the undergraduate chemistry laboratory: Students’ perceptions of control and responsibility, J. Chem. Educ., 93(2), 227–238.
41. Grove N. and Bretz S. L., (2007), CHEMX: An instrument to assess students’ cognitive expectations for learning chemistry, J. Chem. Educ., 84(9), 1524–1529.
42. Gutiérrez K. D. and Rogoff B., (2003), Cultural ways of learning: Individual traits or repertoires of practice, Educ. Res., 32(5), 19–25.
43. Hammer D., (2000), Student resources for learning introductory physics, Am. J. Phys., 68(S1), S52–S59.
44. Hammersley-Fletcher L. and Hanley C., (2016), The use of critical thinking in higher education in relation to the international student: Shifting policy and practice, Br. Educ. Res. J., 42(6), 978–992.
45. Houchlei S. K., Bloch R. R. and Cooper M. M., (2021), Mechanisms, models, and explanations: Analyzing the mechanistic paths students take to reach a product for familiar and unfamiliar organic reactions, J. Chem. Educ., 98(9), 2751–2764.
46. Irby S. M., Pelaez N. J. and Anderson T. R., (2020), Student perceptions of their gains in course-based undergraduate research abilities identified as the anticipated learning outcomes for a biochemistry CURE, J. Chem. Educ., 97(1), 56–65.
47. John-Steiner V. and Mahn H., (1996), Sociocultural approaches to learning and development: A Vygotskian framework, Educ. Psychol., 31(3–4), 191–206.
48. Kohn K. P., Underwood S. M. and Cooper M. M., (2018), Connecting structure–property and structure–function relationships across the disciplines of chemistry and biology: Exploring student perceptions, CBE—Life Sci. Educ., 17(2).
49. Laverty J. T., Underwood S. M., Matz R. L., Posey L. A., Carmel J. H., Caballero M. D., et al., (2016), Characterizing college science assessments: The three-dimensional learning assessment protocol, PLoS One, 11(9), 1–21.
50. Lemke J. L., (2001), Articulating communities: Sociocultural perspectives on science education, J. Res. Sci. Teach., 38(3), 296–316.
51. Matz R. L., Fata-Hartley C. L., Posey L. A., Laverty J. T., Underwood S. M., Carmel J. H., et al., (2018), Evaluating the extent of a large-scale transformation in gateway science courses, Sci. Adv., 4(10), eaau0554.
52. McGill T. L., Williams L. C., Mulford D. R., Blakey S. B., Harris R. J., Kindt J. T., et al., (2019), Chemistry unbound: Designing a new four-year undergraduate curriculum, J. Chem. Educ., 96(1), 35–46.
53. McNeill K. L., Berland L. K. and Pelletier P., (2017), Constructing explanations, in Helping Students Make Sense of the World Using Next Generation Science and Engineering Practices, Schwarz C., Passmore C. and Reiser B. J. (ed.), NSTA Press, pp. 205–227.
54. Merriam S. B. and Tisdell E. J., (2016), Qualitative Research: A Guide to Design and Implementation, Jossey-Bass.
55. Miller P. J. and Goodnow J. J., (1995), Cultural practices: Toward an integration of culture and development, New Dir. Child Adolesc. Dev., 1995(67), 5–16.
56. Momsen J., Offerdahl E., Kryjevskaia M., Montplaisir L., Anderson E. and Grosz N., (2013), Using assessments to investigate and compare the nature of learning in undergraduate science courses, CBE—Life Sci. Educ., 12(2), 239–249.
57. Moran B., (2013), How to get an A- in organic chemistry, N. Y. Times.
58. Nasir N. S. and Hand V. M., (2006), Exploring sociocultural perspectives on race, culture, and learning, Rev. Educ. Res., 76(4), 449–475.
59. National Academies of Sciences E. &. Medicine, (2018), How People Learn II, The National Academies Press.
60. National Research Council, (2000), How People Learn I: Brain, Mind, Experience, and School: Expanded Edition, The National Academies Press.
61. National Research Council, (2012a), A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas, The National Academies Press.
62. National Research Council, (2012b), Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering, The National Academies Press.
63. Noyes K. and Cooper M. M., (2019), Investigating student understanding of London dispersion forces: A longitudinal study, J. Chem. Educ., 96(9), 1821–1832.
64. Parsons E. C. and Carlone H. B., (2013), Culture and science education in the 21st century: Extending and making the cultural box more inclusive, J. Res. Sci. Teach., 50(1), 1–11.
65. Petterson M. N., Finkenstaedt-Quinn S. A., Gere A. R. and Shultz G. V., (2022), The role of authentic contexts and social elements in supporting organic chemistry students’ interactions with writing-to-learn assignments, Chem. Educ. Res. Pract., 23, 189–205.
66. Ramachandran R. and Rodriguez M. C., (2020), Student perspectives on remote learning in a large organic chemistry lecture course, J. Chem. Educ., 97(9), 2565–2572.
67. Redish E. F., Saul J. M. and Steinberg R. N., (1998), Student expectations in introductory physics. Am. J. Phys., 66(3), 212–224.
68. Reinholz D. L. and Apkarian N., (2018), Four frames for systemic change in STEM departments, Int. J. STEM Educ., 5(1), 1–22.
69. Rocabado G. A., Kilpatrick N. A., Mooring S. R. and Lewis J. E., (2019), Can we compare attitude scores among diverse populations? An exploration of measurement invariance testing to support valid comparisons between black female students and their peers in an organic chemistry course, J. Chem. Educ., 96(11), 2371–2382.
70. Rogoff B., (1990), Apprenticeship in Thinking: Cognitive Development in Social Context, Oxford University Press.
71. Sandoval W. A., Enyedy N., Redman E. H. and Xiao S., (2019), Organising a culture of argumentation in elementary science, Int. J. Sci. Educ., 41(13), 1848–1869.
72. Schein E. H. and Schein P. A., (2016), Organizational Culture and Leadership, 5th edn, Jossey-Bass.
73. Scott S., (2008), Perceptions of students’ learning critical thinking through debate in a technology classroom: A case study, J. Technol. Stud., 34(1), 39–44.
74. Scouller K., (1998), The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay, High. Educ., 35, 453–472.
75. Scouller K. M. and Prosser M., (1994), Students’ experiences in studying for multiple choice question examinations, Stud. High. Educ., 19(3), 267–279.
76. Semsar K., Knight J. K., Birol G. and Smith M. K., (2011), The Colorado learning attitudes about science survey (CLASS) for use in biology, CBE--Life Sci. Educ., 10(3), 268–278.
77. Sevian H. and Talanquer V., (2014), Rethinking chemistry: A learning progression on chemical thinking, Chem. Educ. Res. Pract., 15(1), 10–23.
78. Seymour E. and Hewitt N. M., (1997), Talking About Leaving: Why Undergraduate Leave the Sciences, Westview Press.
79. Snyder B., (1973), The Hidden Curriculum, The MIT Press.
80. SPSS, (2020), IBM Corp, https://www.ibm.com/analytics/spss-statistics-software.
81. Stowe R. L. and Cooper M. M., (2017), Practicing what we preach: Assessing “critical thinking” in organic chemistry, J. Chem. Educ., 94(12), 1852–1859.
82. Stowe R. L., Scharlott L. J., Ralph V. R., Becker N. M. and Cooper M. M., (2021), You are what you assess: The case for emphasizing chemistry on chemistry assessments, J. Chem. Educ., 98(8), 2490–2495.
83. Stroupe D., (2014), Examining classroom science practice communities: How teachers and students negotiate epistemic agency and learn science-as-practice, Sci. Educ., 98(3), 487–516.
84. Talanquer V., (2021), Multifaceted chemical thinking: A core competence, J. Chem. Educ., 98(11), 3450–3456.
85. Talanquer V. and Pollard J., (2010), Let's teach how we think instead of what we know, Chem. Educ. Res. Pract., 11, 74–83.
86. Tashiro J. and Talanquer V., (2021), Exploring Inequities in a traditional and a reformed general chemistry course, J. Chem. Educ., 98(12), 3680–3692.
87. Thiry H., Weston T. J., Harper R. P., Holland D. G., Koch A. K., Drake B. M., et al., (2019), in Talking about Leaving Revisited, Seymour E. and Hunter A.-B. (ed.), Springer.
88. Thoman D. B., Muragishi G. A. and Smith J. L., (2017), Research microcultures as socialization contexts for underrepresented science students, Psychol. Sci., 28(6), 760–773.
89. Thomas D. R., (2006), A general inductive approach for analyzing qualitative evaluation data, Am. J. Eval., 27(2), 237–246.
90. Vygotsky L., (1978), Mind in Society, The Harvard University Press.
91. Webber D. M. and Flynn A. B., (2018), How are students solving familiar and unfamiliar organic chemistry mechanism questions in a new curriculum, J. Chem. Educ., 95(9), 1451–1467.
92. Xu X. and Lewis J. E., (2011), Refinement of a chemistry attitude measure for college students, J. Chem. Educ., 88(5), 561–568.
93. Zotos E. K., Moon A. C. and Shultz G. V., (2020), Investigation of chemistry graduate teaching assistants’ teacher knowledge and teacher identity, J. Res. Sci. Teach., 57(6), 943–967.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: 10.1039/d2rp00010e

---