Incorporating concept development activities into a flipped classroom structure: using PhET simulations to put a twist on the flip

Abstract

Implementation of the flipped classroom approach into STEM courses has been popularized in the last decade and has generally been reported to improve student performance outcomes. In a flipped classroom setting, students typically first encounter course content in the online format and subsequently engage in some form of active learning during the in-person class meetings. Although the flipped classroom approach can promote increased student engagement and provide an opportunity to apply content encountered in the classroom, this structure does not generally give students opportunities for discrete concept development prior to the application phase of learning. In an effort to build concept development activities into a flipped classroom structure, five learning cycle activities were implemented in a large enrollment first-term general chemistry course that has previously implemented the flipped classroom design. Four of these learning cycle activities incorporated PhET simulations as part of the exploration phase of learning, and all five activities were facilitated during the in-person class meetings to initiate the learning cycle. The activities were designed to help students explore models and engage in concept development. The application phase of the learning cycle was facilitated by flipped classroom modules or in-person classroom activities that included whole-class questioning coupled with collaborative think-pair-share discussion. Performance gains in conceptual understanding were evaluated by employing a one-group, pre-post-post research design. Non-parametric Friedman's tests indicate a significant main effect across time for each concept development activity, and post hoc Wilcoxon signed rank tests indicate the post-test and final exam scores are significantly higher than the pre-test scores for each activity (p < 0.001 for each pre-post and pre-final pairwise comparison). The findings reported herein demonstrate that concept development activities can be successfully integrated with flipped classroom modules and the combination of the introductory learning cycle activities and flipped classroom application activities led to knowledge gains that persisted through the end of the course. In total, creating this type of blended learning environment appears to help students achieve understanding of core general chemistry concepts.

---

Introduction

Studies on the efficacy of the flipped classroom course structure have grown steadily over the last decade and have generally shown that this classroom intervention leads to an improvement of student learning outcomes relative to traditional didactic lecture (Jensen et al., 2015; Casselman et al., 2019; Naibert et al., 2020). Implementations of flipped classrooms vary, but typically some portion of the traditional in-class lecture is moved to an online learning space followed by the facilitation of active learning in the subsequent live classroom period. The type of classroom activities that are facilitated during the in-person instruction are wide ranging, but generally include some form of collaborative group learning (Weaver and Sturtevant, 2015), formal peer-led learning (Liu et al., 2018), or whole-class questioning with think-pair-share discussion (Flynn, 2015). Regardless of the exact nature of the face-to-face learning activity, the in-class time is commonly structured to help students build upon the content encountered in the asynchronous pre-class learning environment (Naibert et al., 2020).

Although the flipped classroom structure has been identified as a possible way to reduce cognitive load relative to the delivery of content in a traditional lecture format (Seery, 2015), the initial steps of concept development are not well-understood or deliberately supported by the usual pre-class content. This is likely because students typically only have a listening and/or note-taking role in the online learning space. Thus, there is an opportunity and a need to build concept development into the flipped classroom design. This could be achieved by including an in-class activity prior to the asynchronous online learning module, in which exploration and concept development are facilitated through sense-making of phenomena, models, and data. This modification is aligned with a pedagogical method known as the learning cycle (Lawson and Karplus, 2002), which includes exploration, term introduction or concept development, and application phases of instruction. As mentioned above, the flipped classroom already has a strong application component in which the in-class activities build upon the pre-class online content. However, the exploration and concept development phases cannot be readily integrated into pre-class asynchronous learning activities, and to our knowledge incorporating preliminary concept development into a flipped classroom structure has not been reported. There are reports of incorporating POGIL (Process Oriented Guided Inquiry Learning) activities in the in-person portion of the flipped classroom (Hibbard et al., 2016; Canelas et al., 2017). Though POGIL involves collaborative group learning that aims to facilitate conceptual understanding through exploration, explanation, and sense-making processes (Moog and Spencer, 2008), the studies reported by Hibbard et al. and Canelas et al. were built on flipped classroom structures in which the POGIL activities were completed after the pre-class learning assignments.

In the present study, learning cycle activities that promoted exploration and concept development were implemented as introductory in-class activities prior to standard flipped classroom modules, which were then followed by in-class activities centered on whole-class questioning and think-pair-share group learning. The learning cycle activities were developed for five foundational learning objectives in the first-term general chemistry curriculum, four of which employed PhET simulations to engage students in the exploration phase of the learning cycle. PhET interactive simulations‡ were integrated in the concept development activities, because they can provide immediate feedback to users and are designed to support student connection-making and focus on active knowledge construction (D’Angelo et al., 2014).

Theoretical frameworks for learning

The flipped classroom approach is built upon theoretical frameworks that support student learning in both the pre-class online and in-person classroom environments. With respect to the pre-class online learning, Sweller's cognitive load theory has been previously identified to be one of the foundational frameworks linked to the impact of the flipped classroom (Seery, 2015). Cognitive load theory states that because a learner has limited working memory capacity and short-term memory, excessive levels of cognitive load can interfere with meaningful learning (Sweller, 1988). The rigid structure of traditional lecture settings can limit students’ ability to process and understand new information at their own pace. In the flipped classroom, some of the traditional lectures are replaced with pre-class online activities that can be completed by students with fewer time constraints and opportunities to reflect on their learning, ultimately leading to reduced cognitive load. Numerous studies on the efficacy of flipped classroom have been published in the chemistry education literature in recent years. A meta-analysis of evidence-based instructional practices in undergraduate chemistry courses concluded flipped classroom implementations generally lead to positive outcomes with respect to student performance (positive effect sizes ranging from small to medium were observed in the 15 studies included in the analysis), and therefore it should be considered for use as an evidence-based instructional practice that promotes student success (Rahman and Lewis, 2019).

Because the in-person classroom activities associated with flipped classroom implementations are often built on some form of collaborative group learning, this aspect of the flipped classroom is broadly grounded in Vygotsky's social constructivism framework (Vygotsky, 1978). Social constructivism emphasizes the importance of the collaborative nature of learning, and posits that cognitive functions are closely linked to social interactions. Classroom exercises such as peer-led team learning, POGIL, and even less formal think-pair-share activities are partially rooted in social constructivism due to the dependency upon collaborative work that supports learning in these interventions. The impact of social constructivism on the flipped classroom assumes collaborative group learning is embedded in the in-person classroom activities, and it should be noted that the opportunities for group interactions may indeed vary across different flipped classroom implementations.

Von Glasersfeld has proposed that constructivism is a framework in which knowledge is constructed in the mind of the learner from experience, and the function of cognition is to organize such experiences (Von Glasersfeld, 2001). This view of constructivism can be explicitly connected to classroom instruction through the learning cycle instructional model (Lawson and Karplus, 2002). The learning cycle engages students in iterative progressions of exploration of phenomena, concept invention/term introduction, and application. The learning cycle is particularly germane to our thinking about the flipped classrooms, because as noted above, the in-class active learning that follows the pre-class modules is typically an application activity. The learning cycle is notable for its student-centeredness and the opportunities it affords for students to build knowledge from experiencing phenomena in the exploration phase and making sense of the findings from the exploration in the concept invention/term introduction phases. It is important to consider that a teacher-centered lecture is antithetical to the experienced-focused exploration phase of the learning cycle. Applying the constructivist framework underlying the learning cycle, the current study aims to help students develop conceptual knowledge through the exploration and concept invention activities that take place in class prior to more typical flipped classroom structures.

Methodological framework

To better integrate the constructivist approach into the in-class activities, it is necessary to engineer a classroom experience in which students engage in concept development within an existing flipped classroom framework. Such engineering requires an iterative process of activity design, evaluation, and redesign until desired outcomes are achieved. Design-based research (Cobb et al., 2003; Wang and Hannafin, 2005) was employed by Minshall and Yezierski to iteratively design and revise a learning cycle activity for general chemistry on bond making and breaking (Minshall and Yezierski, 2021). This work served as proof-of-concept for the learning efficacy of the intervention and informed how analogous activities could be designed and implemented for the current study. With respect to evaluating the efficacy of newly designed activities, a one-group pre-post-post test design was employed. This quasi-experimental approach allowed for the effects of the activities to be detected while controlling for prior knowledge (pretest).

Four of the activities described here incorporated PhET simulations as part of the exploration phase of the learning cycle. The use of PhET simulations as a way to promote investigative inquiry and foster sense-making in chemistry instruction has been previously reported (Lancaster et al., 2013), and the use of the PhET Atomic Interactions simulation§ within the exploration phase in a concept development activity was described by Minahall and Yezierski (Minshall and Yezierski, 2021). The implementation of the Atomic Interactions activity followed a quasi-experimental design in which each new version of the activity was tested to evaluate the quality of the material in term of students’ learning outcomes. The present study builds upon this work, with the aim to further extend the learning benefits of interactive simulations to other foundational concepts in the general chemistry curriculum.

Three new activities that incorporated PhET simulations were used in addition to the previously reported Atomic Interactions activity, and one additional activity used a static POGIL model to engage students in concept exploration (Moog and Farrell, 2002). Students worked through the learning cycles in small and collaborative groups in class before interacting with the flipped classroom modules or in-person instruction that incorporated whole-class questioning with think-pair-share collaborative learning. The learning cycle activities provided students the opportunity to engage in concept development by making observations within the simulation/model, respond to concept development questions, engage in group discussion, verbally articulate their thinking, and receive feedback on that thinking from peers. The students then built on that conceptual foundation as they progressed through the subsequent flipped classroom modules and whole-class questioning activities. Five learning objectives addressed in most first-term general chemistry courses were chosen and targeted for the learning cycle activities, because the authors have routinely observed that students often struggle with core concepts related to these five learning objectives (see Table 1). These five topic areas are included in the General Chemistry Anchoring Concepts Content Map (Holme and Murphy, 2012), suggesting these are pertinent to most undergraduate general chemistry courses.

Table 1 List of learning cycle activities and PHET simulations used in present study

Learning cycle activity title	Version and year of PhET simulation	Learning cycle activity sheet and teaching notes^a	Learning objectives students will be able to:
a The activity sheets, pre/post/post test items, and detailed teaching notes are available to download from the PhET website.
Nuclear Atom	N/A	Nuclear atom POGIL activity (Moog and Farrell, 2002).	(1) Identify the atomic number and mass of atoms for different elements;
			(2) Identify the importance of atomic number in identifying atoms of different elements;
			(3) Identify different isotopes of the same element; and
			(4) Use the total number of protons and electrons to determine if ions are present and determine the charge of the ion.
Coulomb's Law	1.0.9, 2019‡‡	https://phet.colorado.edu/en/contributions/view/5909	(1) Predict how electrostatic attraction/repulsion changes as a function of distance between charged particles;
			(2) Predict how electrostatic attraction/repulsion changes as a function of the magnitude of charge between charged particles; and
			(3) Describe the concept of electrostatic attraction/repulsion in preparation for applying it to atoms.
Atomic Interactions	1.1.0, 2019§	https://phet.colorado.edu/en/contributions/view/5860	(1) Predict how the potential energy of a bond changes with changing interatomic distance;
			(2) explain how attractive and repulsive Coulombic forces impact the potential energy of a chemical bond; and
			(3) Predict if bond making and bond breaking events are exothermic or endothermic.
Molecular Shape	1.2.8, 2019§§	https://phet.colorado.edu/en/contributions/view/5910	(1) Use Lewis structures to determine the number of bonded atoms and number of non-bonding electron pairs on the central atom of a molecular structure;
			(2) Predict the molecular geometry and molecular shape of structures using the VSEPR model; and
			(3) Predict if the real bond angle is approximately equal to or less than the ideal bond angle for different types of molecular shapes.
Molecular Polarity	1.0.15, 2019¶¶	https://phet.colorado.edu/en/contributions/view/5911	(1) Explain the difference between a bond dipole and a molecular dipole;
			(2) Use electronegativity and knowledge of molecular shape to predict if molecules possess a net molecular dipole or not; and
			(3) Use electrostatic potential maps to explain if a net molecular dipole exists or not.

---

The theoretical and methodological frameworks described above were integrated within a first quarter general chemistry course to address the following research questions:

1. How can the use of introductory learning cycle activities foster conceptual learning for five foundational concepts in a first-term undergraduate general chemistry course?

2. How will knowledge gains (if realized) persist over the course of the term?

3. How can analyses of incorrect student responses provide insight about lingering inaccurate chemical ideas?

Methods

Setting and sample

The learning cycle activities were implemented in a first-quarter general chemistry course (CHEM 001A) taken by all science majors at a large research-intensive public university that is designated as a Hispanic-serving institution (HSI). The topics addressed in this course included: scientific practices and measurement; atomic structure; electronic structure of the atom; chemical bonding and molecular structure; compounds and the mole, chemical reactions, and stoichiometry. The course met over 10 weeks, twice each week for 80 minutes (total enrollment = 231 students), and associated recitation sections met once each week for 50 minutes (30–40 students per recitation section). The details of the course structure are provided in the course syllabus, which is provided in Appendix 1 (ESI†).

Students were informed through a verbal consent process that their performance on the pre- and post-test assessments would be evaluated for research purposes (under approved IRB protocol HS-10-135). They were also informed that all pre/post/post test data would be reported in aggregate form (i.e., all data would be reported in a way that would not reveal individual student identities) and that they could request to have their test data excluded at any time. During the course of the study no students requested to have their data excluded from the analysis.

Learning cycle activities, flipped modules, and in-class activities

Five important concept areas in first-term general chemistry were chosen for learning cycle activities. The Atomic Interactions activity by Minshall and Yezierski was used as previously described (Minshall and Yezierski, 2021), and authors EJY and JFE co-wrote the other four learning cycle activities. Four out of five activities were designed based on the available PhET simulations (Table 1), and each was completed by students in an 80 minute class period. A previous study revealed some students encountered difficulty relating electrostatic forces and potential energy (Minshall and Yezierski, 2021). Thus, the Coulomb's Law activity was implemented prior to the Atomic Interactions activity with the aim to build that prerequisite knowledge.

To help students develop the preliminary concepts independently while engaging with the activities, they were divided into ad-hoc groups of 3–4 students to explore the PhET simulations (Table 1) and complete the activity sheets. The students were instructed to write their answers on the worksheet. The course instructor and two graduate student teaching assistants were available to assist and answer questions as needed, and the instructor had the simulation running on the classroom projector. If points of confusion arose regarding the using the simulation, the instructor would demonstrate parts of the simulation for the class; however, students needed to make their own observations and document results. The activity sheets and detailed teaching notes for the activities that included the PhET simulations, along with the activities sheet and teaching notes are available online (see Table 1). The Nuclear Atom activity is a POGIL worksheet that includes a static drawing in lieu of a PhET simulation (Moog and Farrell, 2002).

After the introductory learning cycle activities, students engaged in either a flipped classroom module or an in-person classroom structure that employed whole-class questioning with think-pair-share collaborative group learning (see Fig. 1). A detailed description of the flipped classroom modules and whole-class questioning activities, and how these exercises were timed throughout the term is provided in Appendix 2 (ESI†). For the flipped classroom modules, students were given 3–4 days to watch instructor-created videos and complete the associated online quiz that was due prior to the in-class application exercises. The online videos also included embedded questions that students completed in the Playposit system.¶ For the flipped module in-class application exercises, the instructor briefly reviewed the material from the online videos and introduced the topic to be discussed that day in the in-class worksheet. Students then worked in collaborative groups of 3 or 4 while completing the worksheet, and they had access to the instructor and two graduate student teaching assistants if clarification was needed. Student work was evaluated by converting the free response questions to multiple choice format, and students submitted answers using the classroom polling system (PollEverywhere||). The instructor then used the last 10–15 minutes of the class period to review any questions in which a large proportion of students submitted incorrect responses and concluded by summarizing the main learning objectives. For some topics, it was determined that students should engage with in-class application exercises in the classroom period immediately after the learning cycle activity (see Appendix 2, ESI†). No online pre-class learning modules were included in these instances, and students applied concepts developed during the learning cycle activities in the subsequent class meeting via instructor-led discussion and whole-class questioning that included think-pair-share collaborative group learning. Analogous to the learning cycle activities, student responses to the whole-class questions were submitted using the classroom polling system. The course syllabus provided in Appendix 1 (ESI†) describes how the performance in the in-class polling factored into the overall course grade.

Fig. 1 Representative examples for how the concept development activities and flipped/hybrid modules were administered. The full schedule of all concept development activities and flipped/hybrid classroom modules is provided in Appendix 1 (ESI†), and the detailed description of the classroom and online learning activities is provided in Appendix 2 (ESI†) (• blue boxes denote learning cycle activities; • green boxes denote application of concepts using whole-class questioning/think-pair-share; • and yellow boxes denote flipped classroom modules).

Evaluation and test instruments

Evaluation of the efficacy of the learning cycle activities was carried out using a one-group pre-post-post quasi-experimental design. Students were instructed by email and in-person during the class meeting 2–3 days prior to the activity to bring a laptop or other electronic device capable of running the iLearn/Blackboard quiz system** on the day of the activity. Students were also informed that free laptops and tablets were available from the campus library. On the day of the activity, students took the pre-test individually and without using any online resources. Students were informed that the pretest would not count toward their course grade, but they were encouraged to do their best work. Based on the instructor's in-class observations, no obvious collaboration took place. The pre-test in iLearn was made unavailable to the students immediately after the pre-test submission. At the end of the activity, worksheet questions that addressed the overall learning objectives of the activity were converted to clicker questions, and the instructor facilitated the input of these questions (e.g., scaffolded activity questions 6b and 6c from the Atomic Interactions activity were used as in-class clicker questions§). The instructor then facilitated a class discussion in which students reported out answers to the activity questions, and the most important concepts or learning objectives were highlighted. To evaluate changes in knowledge, students completed an activity post-test by the end of the day subsequently to completing the activity in class. Students were informed that the post-test was counted for extra credit and instructed not to collaborate with other students (note: the extra points available for all post-test questions constituted less than 1% of the total available course points). The post-test in iLearn was left open for viewing to students for the remainder of the quarter, but students could not alter their answers after the test submission. All the pre- or post-test questions from the iLearn quiz administrations were embedded into the final exam. The final exam review guide encouraged students to review all the in-class activities for the final exam, but the students were not informed that the pre-/post-test questions would be included on the final exam.

Authors EJY and JFE co-wrote all the test items and came to consensus in confirming the content validity of the final set of test items for each activity. The single administration reliability of the test items was evaluated by calculating Cronbach's alpha for the set of test items for each activity, and the ability of the individual items to discriminate between high and low performers was estimated by calculating the item-total correlation values for the items (see Appendix 3, Tables 1 and 2, ESI†). These analyses suggest the scores on items across administrations for each activity were correlated to one another and generally appeared to discriminate between high- and low-performing students. However, the length of the tests only ranged from 2–5 items, making it difficult to arrive at definitive conclusions regarding the reliability of the tests (Tavakol and Dennick, 2011). To gain additional insight about how well the test items measured changes in student knowledge, Sankey diagrams were used to track responses at each time point of test, for each individual item (see Appendix 5, Fig. 1 and 2, ESI†). These data indicate the tests detected clear changes in student knowledge for all the activities except the Coulomb's Law activity, for which the students appeared to demonstrate a high level of a priori knowledge. In total, the item analyses and reliability data suggest the tests were acceptable measures of student knowledge for these five conceptual domains. A detailed summary and explanation of the item analysis and single administration test reliability are provided in Appendix 3 (ESI†).

Table 2 Friedman test statistics and effect size measurements. A non-parametric Friedman Test was conducted to measure the learning gains across three testing time points for each activity, and Kendall's concordance coefficient (W) was calculated to determine the effect size

Activity	Friedman test statistics			W
	χ ^2	df	p
a W = 0.1 indicates a small effect, W = 0.3 indicates a medium effect, and W > 0.5 indicates a large effect.
Nuclear atoms	216.515	2	<0.001	0.558
Coulomb's law	72.460	2	<0.001	0.187
Atomic interaction	249.213	2	<0.001	0.642
Molecular shape	156.071	2	<0.001	0.402
Molecular dipole	134.978	2	<0.001	0.348

---

Fig. 2 The total number of students who correctly answered the different number of test items for pretest (blue), posttest (orange), and final exam (grey). These data are shown for: (a) the combined test items for all activities; (b) Nuclear Atom test items; (c) Coulomb's Law test items; (d) Atomic Interactions test items; (e) Molecular Shape test items; and (f) Molecular Dipole test items.

Statistical analyses

To evaluate the impact of the learning cycle activities on student learning, inferential statistical analyses were conducted on student scores on the tests across three time points. All statistical analyses were conducted on SPSS (Statistics Package for the Social Sciences) Statistics 26 software package.†† Because of the non-parametric nature of the test data, a Friedman's test was employed to determine if student performance changed significantly over the three testing points for each activity assessment. A Kendall's concordance test was used to estimate the effect size of the test scores for each activity. To determine at which testing points significant changes in performance occurred, a post hoc Wilcoxon signed ranked test was used to evaluate pair-wise comparisons among scores (pre-test, post-test, and final exam) for each activity. The level of significance for the pair-wise comparisons was adjusted using a Bonferroni correction to control the family-wise error rate.

Results and discussion

Conceptual learning outcomes

To answer the first two research questions that aimed to determine how the learning cycle activities foster conceptual learning and if knowledge gains persisted over time, student performance was evaluated for each of the five activities at three time points. The general performance gains for the five activities in pre-test, post-test, and the final exam are shown in Fig. 2. Fig. 2a shows the distribution of the number of correct responses (or score) in all activities across the three time points. Out of the 19 test questions, the average score improved from the pre-test (11) to both the post-test (16) and final exam (17). The results suggest that knowledge gains were observed right after the learning cycle activity, and this knowledge was sustained through the final exam. Fig. 2b–f show the distribution of correct responses for each activity across the three time points. A cursory examination of the distributions of scores shows an immediate overall improvement for all five learning cycle activities as evidenced by the increase in number of students who answered all the test items correctly on the post-test (see Fig. 2b–f). The number of students who answered all the test items correctly on the final exam was equal to or greater than that observed on the post-test for all the activities except the Molecular Shapes learning cycle activity, in which there was a decrease in the total number of students who answered all the test items correctly on the final exam relative to the post-test.

Because the Atomic Interactions was previously developed and evaluated by Minshall and Yezierski (Minshall and Yezierski, 2021), it was of interest to compare the results of the current study to this previous report. Additionally, the learning objectives for the Atomic Interactions activity include having students correctly identify the energy changes associated with chemical bond forming and breaking processes. Since students often incorrectly associate bond breaking with an exothermic energy change (Boo, 1998), it was also desirable to determine if the efficacy of this activity would translate to a large enrollment course. As shown in Fig. 2d, approximately 73% of the students incorrectly answered the pre-test questions regarding whether bond breaking and bond making events are exothermic or endothermic. However, most students successfully demonstrated improved performance after the post-test (94% of students answered both test items correctly; see Fig. 2d) and retained this conceptual knowledge on the final exam (97% of students answered both test items correctly; see Fig. 2d). It is noteworthy that a higher percentage of students correctly answered the questions on the final exam using the same activities than the previous study carried out by Minshall and Yezierski, in which 82% of the 34 students were able to able to identify and distinguish between exothermic and endothermic processes (Minshall and Yezierski, 2021). The results from this current study are highly encouraging, as the sample is almost 6 times the size of previous study (231 verse 34 students). It suggests that the activities can be implemented in a large enrollment course without sacrificing student performance gains. This observation might be explained by the implementation of the Coulomb's Law activity‡‡ prior to the Atomic Interactions activity in the present study. Because the knowledge of attractive and repulsive electrostatic forces is required to explain how the potential energy of a chemical bond changes as a function of interatomic distance, we speculate that including the Coulomb's Law activity might have primed students to make better sense of their observations during the Atomic Interactions activity.

To determine if the changes in student performance over time were statistically significant, a non-parametric Friedman test was conducted to evaluate differences among the total number of correct test items at the three time points for each activity. The Friedman's Chi-square value (χ²) for all activities was found to be significant (p < 0.001), indicating the learning cycle activities had a significant effect on student performance (Table 2). The Kendall's Coefficient of Concordance (W) was used to measure the effect size of each activity. A W value of 0.5 or above indicates large differences among the three time points. Most activities have medium or large effect sizes; only the Coulomb's Law activity had a small effect size with this group of students (W = 0.187). The smaller effect size observed for the Coulomb's Law activity is possibly explained by the fact that the concepts addressed in this activity ended up being less challenging than originally expected (see Appendix 5, Fig. 1a–d, ESI†), as the students appeared to have a high level of a priori knowledge on electrostatic attraction/repulsion.

Post hoc pairwise comparisons of the non-normally distributed test scores were conducted using a Wilcoxon-signed rank test (Table 3) to determine if the student performance was significantly different at the three testing points. The effect size (r) for each pairwise comparison was calculated using previous published method (Tomczak and Tomczak, 2014), and the cutoffs for r are as follows: 0.1 (small effect), 0.3 (medium effect), and above 0.5 (large effect) (Cohen, 1988). Students’ test scores improved from the pre-test to both the post-test and final exam, for all five activities (for all post-test/pre-test and final exam/pre-test pairwise comparisons; p < 0.001). The learning cycle activities yield large effect sizes for test performance for the Atomic Interactions and Molecular Shape activities, medium effect sizes for the Nuclear Atom and Molecular Dipole activities, and a small effect size for the Coulomb's Law activity (see post-test/pre-test pairwise comparisons in Table 3). The combination of the learning cycle activities and subsequent flipped modules/in-class exercises yield a medium to large effect for all five conceptual domains (see final exam/pre-test pairwise comparisons in Table 3).

Table 3 A Wilcoxon signed-rank test (Z) was conducted to evaluate the average differences of two test timepoints for each learning cycle activity. The effect size (r) for each pairwise comparison was also calculated to determine the effect of the activity between the two timepoints

Activity	Pair-wise score comparison	Z	p (2-tailed)^a	r
a Significance level adjusted using a Bonferroni correction (α: 0.05/3 = 0.017). b The interpretation guidelines for r are as follows: 0.1 (small effect), 0.3 (medium effect), and above 0.5 (large effect).
Nuclear Atom	Post-test/pre-test	8.597	<0.001	0.44
	Final exam/pre-test	10.296	<0.001	0.52
	Final exam/post-test	4.101	<0.001	0.21
Coulomb's Law	Post-test/pre-test	3.488	<0.001	0.18
	Final exam/pre-test	7.804	<0.001	0.40
	Final exam/post-test	4.767	<0.001	0.24
Atomic Interaction	Post-test/pre-test	11.402	<0.001	0.58
	Final exam/pre-test	11.708	<0.001	0.59
	Final exam/post-test	1.762	0.078	0.09
Molecular Shape	Post-test/pre-test	10.222	<0.001	0.52
	Final exam/pre-test	8.131	<0.001	0.41
	Final exam/post-test	5.269	<0.001	0.27
Molecular Dipole	Post-test/pre-test	8.972	<0.001	0.46
	Final exam/pre-test	8.932	<0.001	0.45
	Final exam/post-test	0.340	0.734	0.02

---

Elucidating persistent inaccurate thinking

To address the third research question that asked how analyses of the student responses might inform changes for future implementations of the activities, Sankey diagrams were used to identify the common inaccurate ideas that appeared to persist after students engaged in the learning cycle interventions. Sankey diagrams that depict which responses were selected for each time point of testing were also created for the test items in which more than 10% of the students selected incorrect answers on the post-test and/or final exam (see Fig. 3 and Appendix 5 Fig. 1, ESI†). It should be noted that the Sankey diagrams only include the students who completed all three tests (90% or more of the study sample, depending on the test). Tracking which item distractors were selected at each phase of testing provided an opportunity to identify which specific inaccurate ideas generally persisted among the participants. Four test items from the Molecular Shape and Molecular Dipole activities were observed to reveal the most notable areas of inaccurate thinking (see Fig. 3 and Table 4).

Fig. 3 Three-node Sankey diagrams display the distributions of students correctly (green) or incorrectly (red) answer the question along with the answer choices at three time-points: pre-pre-test (left node), post-post-test (middle node), and the final Exam (right node).

Table 4 Four questions from Molecular Shape (MS) and Molecular Dipole (MD) activities had the highest incorrect responses illuminating the common areas of inaccurate thinking among students

MS Q2. Which would you predict to have the strongest repulsive force to a nearby electron pair?

A. A non-bonding electron pair

B. A bonding electron pair

C. These have the same repulsive force

MS Q3. Which would you expect to have the smallest bond angle?

A. NH₃

B. CH₄

C. H₂O

D. These would all have the same bond angle.

MD Q3. Which explains why water molecules are polar and carbon dioxide molecules are non-polar?

A. Water molecules have a non-linear molecular shape

B. Carbon dioxide has non-polar bonds

C. Both molecules have polar bonds

D. Hydrogen and oxygen have a large electronegativity difference

MD Q4. Which molecule has a net molecular dipole?

A. OCl₂

B. Cl₂

C. CO₂

D. BH₃

---

In Question 2 of the Molecular Shape test, the desired conceptual learning outcome is that an unbonded electron pair gives rise to the strongest repulsive force to a nearby bonded electron pair. For the question, “Which would you predict to have the strongest repulsive force to a nearby electron pair?”, the most common distractor in the pre-test was the answer “a bonding electron pair.” This was likely because students recognized electrostatic repulsion occurs between bonded atoms, but they neglected to consider the effect of non-bonding electrons on the bond angle between the bonded atoms. Though the number of students who chose this answer decreased from 69 to 12 from pre- to post-test administration, 48 students reverted back to the “a bonding electron pair” distractor on the final exam (see Fig. 3a). Question 3 of the Molecular Shape activity also evaluated students’ understanding of the impact of non-bonding electron pairs on molecular shape. The most common distractor in the pre-test for this item was the answer “CH₄,” likely because students associated smaller bond angles with molecules in which more atoms were bonded to the central atom. The number of students who chose this answer decreased from 114 to 26 from pre- to post-test administration, yet 65 students again chose this common distractor on the final exam (Fig. 3b). Overall, most students appeared to make knowledge gains with respect to identifying the impact of non-bonding electron pairs on the electron pair geometries and shapes of molecules. However, given the fact over 10% of the students reverted back to the inaccurate thinking for these two test items as described above, practitioners might consider refining the Molecular Shapes activity to mitigate this reversion in future implementations. Because some students appear to neglect the impact of the non-bonding electron pairs, the activity could guide students to draw out the molecules with all bonding atoms and non-bonding electro pairs (this could be added into Table 2 of the Molecular Shapes activity§§). If students were asked to draw the molecules in this way, and also to use arrows to identify where electrostatic repulsion occurs within the molecule, this might lead to longer lasting knowledge gains for this particular concept.

In Question 3 of the Molecular Dipole activity, the desired learning outcome was for students to understand the relationship between molecular structure and molecular dipoles. In this particular question, students were required to choose the response that explains why water molecules are polar and carbon dioxide molecules are non-polar. Although the number of students who chose the most common distractor decreased from 81 to 39 from pre- to post-test administration, 40 students still answered incorrectly on the final, and over 10% of students reverted from the correct answer on the post-test to the most common distractor on the final exam. The most common distractor stated “hydrogen and oxygen have a large electronegativity difference.” Although this statement is accurate, it does not explain why water is polar molecule and carbon dioxide is not. The students who chose this answer likely only considered the electronegativity difference between the atoms in the bonds in identifying molecular dipoles, without considering how the three-dimensional structure plays a role in giving rise to molecular polarity. For Question 4 in the Molecular Dipoles activity, the number of students who chose the most common distractor decreased from 63 to 20 from the pre- to post-test administration. Though only 33 students chose this distractor on the final exam, over 10% of students who chose this answer on the final reverted back to this common distractor after answering correctly on the post test. Because the most common distractor for this question was “BH₃,” we speculate students who chose this answer did so based on comparing its structural formula to the other answer choices, rather than carefully evaluating differences in electronegativity and/or molecular geometry (BH₃ was unique among the answer choices in having three atoms bonded to a central atom; see Appendix 4, ESI†). The critical thinking questions in the Molecular Dipole activity explicitly ask students to identify what properties are required for a molecule to possess a net dipole.¶¶ Therefore the observation that over 10% of students continued to incorrectly identify which molecules have net dipoles and/or why net dipoles are present was surprising. In future implementations of the activity, instructors might consider including a post-activity discussion highlighting that the molecular formula should not be used as a heuristic to identify the presence of net molecular dipoles, and that simply determining the presence of polar bonds is not sufficient in identifying molecular dipoles. Instructors might also consider having students draw out the net dipole vectors in addition to the electron density maps, which could help reinforce how both electronegativity differences and molecular shape determine if a net molecular dipole is present.

Discussion

Knowledge gains and the constructivist theory

The pre-post-post test results suggest the incorporation of introductory, in-person activities built upon the constructivist theory of learning within a flipped classroom structure can be successfully implemented in an existing flipped classroom structure. The one-group quasi-experimental design does not allow for the explicit determination of cause and effect between the specific learning interventions and observed knowledge gains, however we can speculate about how the classroom activities described herein may have impacted the performance on the post-test and final exam test items. Because the post-test was administered after the in-class learning cycle concept development activities, the preliminary knowledge gains observed on the post-test for all five conceptual domains are likely attributed to the introductory concept development activities (see post-test/pre-test pairwise comparisons in Table 3). After the introductory learning cycle activities, students completed flipped modules and/or engaged in whole-class questioning/think-pair-share exercises that were intended to fulfill the application phase of the learning cycle. Thus, the observed gains on the final exam test items are possibly linked to the overall combination of introductory concept development activities and flipped module/in-class application exercises (see final exam/pre-test pairwise comparisons in Table 3). Even though the post-test/pre-test and final exam/pre-test pairwise comparisons were statistically significant for all conceptual domains, the effect sizes and final exam/post-test pairwise comparisons suggest additional growth may have occurred after the introductory concept development learning cycle activity for three of the five conceptual domains (see Table 3). The time between completing the learning cycle activities and the final exam ranged from nine weeks to three weeks. This difference in time lag may have resulted in different levels of long-term knowledge retention for the different activities. However, the fact at least three weeks passed between the learning cycle activities and final exam suggests the application of the concepts in the flipped modules and in-class exercises led to notable retention of knowledge gains.

These observed knowledge gains can be viewed within the context of the constructivist theory of learning. As described in the introduction, Vygotsky's theory of constructivism posits that understanding comes from making meaningful connections through experience between prior knowledge and new knowledge (Vygotsky, 1978). The in-class learning cycle activities allowed students to explore the simulations in a guided fashion, and they constructed informal and general ideas about the subject individually built upon the foundation of previous learning. Vygotsky's theory states that language plays a fundamental role in shaping meanings. Because students wrote down their observations and personal interpretation of the simulations, and then subsequently engaged in group discussions to further extend their concept development, learning was supported by social constructivist principles. Additionally, a cycle of exploration, concept development, and concept application was created by combining the introductory learning cycle activities with the flipped modules/in-class exercises. This suggests the overall knowledge gains observed for the five conceptual domains can also be rooted in Von Glasersfeld's view of constructivism (Von Glasersfeld, 2001). Students were given the opportunity to construct new conceptual knowledge from experience (the exploration and concept development phases in the introductory learning cycle activity), then apply this new knowledge in the online flipped module exercises and in-class collaborative group learning exercises. The totality of this learning experience appears to have to led to both short term and longer-term conceptual understanding for students.

Limitations of the study

Because this design-based study did not include a “teaching as usual” control group comparison, this might be viewed as a research limitation. However, previous research has provided compelling evidence about the positive impact of learning cycle activities on student performance gains and demonstrated the general efficacy of learning cycle activities (Musheno and Lawson, 1999; Escalada et al., 2004; Rodriguez et al., 2020), therefore, creating a teaching as usual quasi-experimental control group that purposefully excluded the learning cycle activities was not necessary or prudent.

The short duration of the tests for each activity and the fact these measures of student knowledge were only administered to one sample may have limited the rigor of the reliability analysis; however, the item-total correlations and the apparent ability of the tests to detect gains in knowledge suggest the tests were adequate measures of student knowledge. It is also noted the pre-test performance did not impact the students’ grades, whereas the post-test questions counted as extra credit and the assessment questions were integrated as normal questions within the final exam. Because the students did not have the same grade incentive on the pre-test, this could have biased the improved performance on the post-test and final exam. Unfortunately, this limitation is difficult to overcome in a class-based quasi-experimental design, as it is generally not desired to have the pre-test scores impact student grades. The same set of items of were administered three times. Although this lends reliability to the results, it could limit the validity in measuring student knowledge (i.e., students could just be remembering the question and answer). However, this validity limitation is minimized because students were not provided with the answers nor were alerted that the same items would be on subsequent tests. Finally, the test questions were designed by the authors to evaluate conceptual knowledge and determine if students continued to adopt previously observed inaccurate chemical ideas. Despite being designed to probe conceptual understanding, the multiple-choice format of the questions may have limited the ability to fully capture the products of student sense-making and conceptual thinking. Future studies could involve creating items that include some free response questions and/or other methods that more effectively elicit and document student sense-making and conceptual understanding.

It is uncertain that the success of the learning outcomes in this study can be translated to other institutional settings. With the challenges of implementing the activities in a large enrollment course that had a high percentage of first-generation students, significant and positive learning outcomes were observed for each learning cycle activity in the present study. This suggests the inferential statistical claims made regarding performance gains in the present study should generalize to other undergraduate introductory/general chemistry courses. Though the activities need to be implemented in other institutions with different student populations to confirm this notion, the fact the performance gains reported here for the Atomic Interactions activity appear to be comparable to those reported by Minshall and Yezierski (Minshall and Yezierski, 2021) further suggest the learning cycle activities can be successfully implemented in institutional settings with disparate student populations.

With respect to the implementation of the PhET-based learning cycle activities, access to devices could be a potential limitation at some institutions. If laptop or tablet loan programs are not available, instructors could modify the implementation to allow students to share devices. If enough students have access to devices such that at least one student per group can run the PhET simulation, this should allow instructors to facilitate the learning cycle activities. If access to laptops or tablets is even more limited, instructors might consider running the simulation as a demonstration on the classroom projection system, and students could then make the necessary observations and progress through the scaffolded questions.

Conclusions and implications

Adopting the learning cycle activities allowed for students to actively engage in concept development and participate in the skill development within a flipped classroom environment. The learning cycle activities were seamlessly integrated into an existing course that included flipped classroom modules. The present study demonstrated flipped classroom modules coupled with in-class learning cycle activities that incorporated PhET simulations have increased the immediate and longer-term performance gains in a large classroom setting. More specifically, it appears the introductory learning cycle activities led to immediate increases in conceptual knowledge for all five categories of learning objectives, and the application of these concepts in subsequent flipped modules and/or in-class activities with whole-class questioning and think-pair-share collaborative group learning seems to have allowed students to retain or build upon that conceptual knowledge through the end of the course. Although more students appeared to struggle with selected concepts in the Molecular Shape and Molecular Dipole learning objectives, future implementations of the activities can be modified to more explicitly highlight the concepts in these two activities for which students persisted to demonstrate inaccurate thinking. This design research approach should not only yield improved performance outcomes for the activities described herein, but it also holds promise to act as a model for a scholarly approach to high-quality materials development more broadly in chemistry education.

Using simulations as a means for students to generate observations in learning cycle activities within a non-laboratory classroom setting is a promising approach to integrate concept development activities in higher education introductory chemistry courses. It is hoped the current study, in conjunction with the previous report from Minshall & Yezierski (Minshall and Yezierski, 2021), demonstrates how students can improve conceptual understanding when the learning cycle approach is used with simulation-based activities and will inspire instructors from a variety of institutional settings to engage their students in this approach. With respect to the use of the flipped classroom in higher education chemistry, educational studies generally provide only a vague description of coupling online video tutorials with some form of in-class active learning. To our knowledge no previous reports describe including an in-class concept development activity prior to the cycle of online/in-person learning. We hope the current study gives classroom practitioners a roadmap for how concept development activities can be incorporated into a flipped classroom structure, as well as a more complete picture of what is actually involved in the pre-class online learning and in-class active learning environments that produced the performance gains described herein. If instructors were to put a twist on the flip as described here, they would be creating a blended learning environment that actively engages students in a new way and likely helps students achieve understanding of core general chemistry concepts.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

We acknowledge the University of Colorado-Boulder PhET Interactive Simulations resource site for creating these valuable open-access instructional tools. The authors also acknowledge YuChen Xiao for helping with the facilitation of the in-class activities and the general chemistry students who participated in our study.

References

1. Boo H. K., (1998), Students' understandings of chemical bonds and the energetics of chemical reactions, J. Res. Sci. Teach., 35(5), 569–581.
2. Canelas D. A., Hill J. L. and Novicki A., (2017), Cooperative learning in organic chemistry increases student assessment of learning gains in key transferable skills, Chem. Educ. Res. Pract., 18(3), 441–456.
3. Casselman M. D., Atit K., Henbest G., Guregyan, C., Mortezaei K. and Eichler J. F., (2019), Dissecting the flipped classroom: Using a randomized controlled trial experiment to determine when student learning occurs, J. Chem. Educ., 97(1), 27–35.
4. Cobb P., Confrey J., DiSessa A., Lehrer R. and Schauble L., (2003), Design experiments in educational research, Educational Researcher, 32(1), 9–13.
5. Cohen J., (1988), Statistical Power Analysis for the Behavioral Sciences, 2nd edn, New York, NY: Lawrence Erlbaum Associates, chap. 3.
6. D’Angelo C., Rutstein D., Harris C., et al., Simulations for STEM learning: systematic review and meta-analysis, Menlo Park, CA: SRI International; 2014.
7. Escalada L. T., Rebello N. S. and Zollman D. A., (2004), Student explorations of quantum effects in LEDs and luminescent devices, Physics Teacher, 42(3), 173–179.
8. Flynn A. B., (2015), Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French, Chem. Educ. Res. Pract., 16 (2), 198–211.
9. Hibbard L., Sung S. and Wells B., (2016), Examining the effectiveness of a semi-self-paced flipped learning format in a college general chemistry sequence, J. Chem. Education, 93(1), 24–30.
10. Holme T. and Murphy K., (2012), The ACS Exams Institute undergraduate chemistry anchoring concepts content map I: General Chemistry, J. Chem. Education, 89(6), 721–723.
11. Jensen J. L., Kummer T. A. and Godoy P. D. D. M., (2015), Improvements from a flipped classroom may simply be the fruits of active learning, CBE—Life Sciences Education, 14, 1–14.
12. Lancaster K., Moore E. B., Parson R. and Perkins K. K., (2013), Insights from using PhET's design principles for interactive chemistry simulations, ACS Symposium Series, 1142, 97–126.
13. Lawson A. E. and Karplus R., (2002), The learning cycle, A love of discovery, Dordrecht: Springer, pp. 51–76.
14. Liu Y.; Raker J. R. and Lewis J. E., (2018), Evaluating student motivation in organic chemistry courses: moving from a lecture-based to a flipped approach with peer-led team learning, Chem. Educ. Res. Pract., 19(1), 251–264.
15. Minshall B. L. and Yezierski E. J., (2021), Data-driven activity reform: employing design research to improve scaffolding and concept development, Chem. Educ. Res. Pract., 22(1), 136–145.
16. Moog R. and Farrell J., (2002), Chemistry: a guided inquiry, NY: John Wiley & Sons.
17. Moog R. S. and Spencer J. N. (ed.), (2008), Process oriented guided inquiry learning, Washington, DC: American Chemical Society, vol. 994.
18. Musheno B. V. and Lawson A. E., (1999), Effects of learning cycle and traditional text on comprehension of science concepts by students at differing reasoning levels, J. Res. Sci. Teaching, 36(1), 23–37.
19. Naibert N., Geye E., Phillips M. M. and Barbera J., (2020), Multicourse Comparative Study of the Core Aspects for Flipped Learning: Investigating In-Class Structure and Student Use of Video Resources, J. Chem. Educ., 97(10), 3490–3505.
20. Rahman M. T. and Lewis S. E., (2019), Evaluating the evidence based instructional practices in chemistry through meta-analysis, J. Res. Sci. Teach., 57, 765–793.
21. Rodriguez J. M. G., Hunter K. H., Scharlott L. J. and Becker N. M., (2020), A review of research on process oriented guided inquiry learning: Implications for research and practice, J. Chem. Educ., 97(10), 3506–3520.
22. Seery M. K., (2015), Flipped learning in higher education chemistry: emerging trends and potential directions, Chem. Educ. Res. Pract., 16(4), 758–768.
23. Sweller J., (1988), Cognitive load during problem solving: Effects on learning, Cognitive science, 12(2), 257–285.
24. Tavakol M. and Dennick R., (2011), Making sense of Cronbach's alpha, JIAMSE, 2, 53–55.
25. Tomczak M. and Tomczak E., (2014), The need to report effect size estimates revisited. An overview of some recommended measures of effect size, Trends in Sport Sciences, 21(1), 19–25.
26. Von Glasersfeld E., (2001), The radical constructivist view of science, Foundations of Science, 6(1), 31–43.
27. Vygotsky L., (1978), Mind in Society, London: Harvard University Press.
28. Wang F. and Hannafin M. J., (2005), Design-based research and technology-enhanced learning environments, Educ. Technol. Res. Develop., 53(4), 5–23.
29. Weaver G. C. and Sturtevant H. G., (2015), Design, implementation, and evaluation of a flipped format general chemistry course, J. Chem. Educ., 92(9), 1437–1448.

---

Footnotes

† Electronic supplementary information (ESI) available. See DOI: 10.1039/d1rp00086a

‡ PhET Interactive Simulations. From https://phet.colorado.edu/.

§ PhET Atomic Interactions simulation. From https://phet.colorado.edu/en/simulation/atomic-interactions.

¶ Playposit is available at: https://go.playposit.com/.

|| PollEverywhere is available at: https://www.polleverywhere.com/.

** iLearn/Backboard quiz system. From https://www.blackboard.com/teaching–learning/learning-management.

†† IBM Corp. Released 2019. IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY: IBM Corp.

‡‡ PhET Coulomb's Law simulation. From https://phet.colorado.edu/en/simulation/coulombs-law.

§§ PhET Molecular Shapes simulation. From https://phet.colorado.edu/en/simulation/molecule-shapes.

¶¶ PhET Molecular Dipole simulation. From https://phet.colorado.edu/en/simulation/molecule-polarity.

---