Performance of underprepared students in traditional versus animation-based flipped-classroom settings

Abstract

Student performance in a flipped classroom with an animation-based content knowledge development system for the bottom third of the incoming first year college students was compared to that in a traditional lecture-based teaching method. 52% of these students withdrew from the traditionally taught General Chemistry course, compared to 22% in a similar course taught in a flipped classroom teaching method. Of the students who persisted in the course and obtained a grade, there was an increase in A's and B's as well as an increase in D's and F's for students taught in a flipped classroom teaching method when compared to those in the traditional setting. When the course that was initially taught in a flipped classroom method reverted to a traditional teaching method, students in that course generally performed worse than students who were in a traditionally taught course all throughout.

---

Introduction

While there are many variants of the flipped classroom teaching method, in general, implementing a flipped classroom involves transposing the content delivery to a time prior to the classroom treatment and a space outside of the lecture hall. The classroom space is then reserved for activities that are designed to reinforce the previously viewed content. In practice, this means that the flipped classroom teaching method usually involves, in succession: (1) a system for delivering content to the students prior to the lecture hall treatment, (2) a method for encouraging or inducing students to learn from the delivered content, and (3) an in-class treatment that emphasizes discussion or application of the content, eschewing lectures or further content delivery and other forms of direct instruction; the treatment acting as a formative assessment. The most commonly used method for content delivery is a video-recorded lecture, with the video recording being disseminated electronically. Inducements to learn from the video-recordings can be as straightforward as ensuring that the students understand that going through the video-lecture is necessary in order to perform well in the upcoming lecture hall activity, or it could come in more structured and formal processes such as graded assignments that need to be completed prior to the lecture hall activity. The lecture hall activity is usually built around some form of formative exercise that is meant to have the students apply the knowledge they had gained in perusing the video-lecture. In this sense, the teaching method has been called “flipped” since the content, the lecture, is delivered outside of the classroom, and prior to the lecture hall activity, and the “homework” is instead done in the classroom.

A survey of the literature on the flipped classroom teaching method (Logan, 2015; Seery, 2015; Zainuddin and Halili, 2016; DeLozier and Rhodes, 2017) revealed several variants of the classroom activity: quizzes, teacher-led discussions, worksheets to be completed individually, student-group activities, merging the flipped classroom teaching method with Peer-Led Team Learning (PLTL) (Robert et al., 2016), with Just In Time (JIT) teaching methods (Muzyka, 2015) and implementation of “productive failure” designs (Song and Kapur, 2017). Most of the content delivery, however, involved some variant of a printed text reading assignment (Lenczewski, 2016) and, more commonly, the viewing of a video-recorded lecture or screencast (Seery, 2015). The video-recordings were variants of an instructor being recorded while delivering the content in a lecture format. We found one instance (Fitzgerald and Li, 2015) where the Prezi presentation platform, a more robust form of slide presentation, was used, and Ryan and Reid (2016) used a voice-over PowerPoint video captured in Camtasia.

Our previous work on student-centered, self-guided learning showed that a Flash-based animation (Gregorius, 2010) was better at allowing students to develop content knowledge for and by themselves. We found that students at different levels, grades 3 and 4, high school, and first year college students, could effectively teach themselves specific content using carefully calibrated animations and simulations (Gregorius, 2005a, 2005b; Gregorius et al., 2010a, 2010b). Students who developed content knowledge in this way performed better on assessments than their counterparts who learned from printed texts, traditional lectures, or other forms of direct instruction (Gregorius, 2011). We wondered if these animations and simulations would be an effective content delivery system in a flipped classroom teaching method. We decided to collect data on student performance in a flipped classroom wherein the students developed content knowledge from animations and simulations rather than from direct instruction through video-recorded lectures.

A survey of the literature on the use of the flipped classroom teaching methods in chemistry courses showed generally positive outcomes. There were many reports on the effectiveness of a flipped classroom teaching and learning method: in High School Advanced Placement (AP) Chemistry (Schultz et al., 2014), in General Chemistry (Hibbard et al., 2016), specifically with low achieving students in General Chemistry (Ryan and Reid, 2016), with both the upper third and lower third of the students in General Chemistry (Butzler, 2015), and in Organic Chemistry (Rossi, 2015; Mooring et al., 2016; Shattuck, 2016). There were also some reports in which no statistically significant difference in student performance between those taught in the traditional lecture and textbook system versus the flipped classroom system was observed (Weaver, 2015), sometimes within the same article reporting on the benefits of the flipped classroom teaching method and for a specific student population or chemistry topic (Butzler, 2015; Ryan and Reid, 2016; Shattuck, 2016).

At the author's institution, we had a somewhat unique opportunity in that the lower performing third (to as much as the lower half) of all students taking the first semester general chemistry course could be recognized and segregated from the main body of students. The chemistry department determined that the mathematics component of the SAT (formerly an acronym for Scholastic Aptitude Test, later Scholastic Assessment Test, and now simply SAT), the standardized general knowledge test that is widely used as a college entrance examination in the United States of America, could be used as a metric to predict student success in the first semester course of the general chemistry program. It was observed that 70% of students who failed to achieve a grade of C or higher, and were therefore not allowed to continue on to the second semester of the two-semester general chemistry program, had mathematics SAT (MSAT) scores lower than 580 (for reference, the MSAT scores are normalize to range from 200 to 800). The author's home institution decided to develop an alternative program for these lower scoring students, expanding the first semester of the traditional two-semester general chemistry program into a one year, two-semester course, and – with the second half of the traditional two-semester program – producing an alternative three-semester general chemistry program. Incoming first year students were then placed in either the standard two-semester program or the alternative three-semester program based on a 580 cutoff MSAT score. It is important to state here that while a student's individual MSAT score was in no way a good predictor of that student's final grade in the standard or alternative general chemistry program, the MSAT score could predict pass/fail, and the cutoff was used as a way to place students in the program where they would have a good chance of success. Students were given the metrics data, informed of the rationale for their placement, and then were allowed to opt-in or out of the standard and alternative programs.

While the three-semester alternative program had some success (approximately 50% of the students placed in the program passed the first and second part of three semester program and were allowed to progress to the third semester – despite the metric that indicated that they would not succeed in the equivalent first semester of the standard, two-semester, general chemistry program), we wanted to see if this lower performing segment of students would perform better with alternative teaching methods. The premise was that these underperforming students tend to not do well in traditional teaching settings, but could possibly do well enough in alternative teaching methods that were aligned with our understanding of learning. We already observed a student performance boost when the traditional print-textbook was replaced with Flash-based animations and simulations (Gregorius, 2011, 2013). We wanted to see if a flipped classroom teaching method would further improve student performance and pass rates.

Study setting and method

All experiments were performed in compliance with the relevant laws and institutional guidelines. The study framework and experiment procedures were discussed with the author's home institution's Institutional Review Board (IRB) prior to proceeding with the study. The experiments and study structure were approved by the IRB. The committee agreed that no harm would come to the participants in the study, their learning would not be compromised, and their final grades, despite any interventions done in the course of the study, would still be an appropriate reflection of their effort and learning.

The study was conducted every fall semester over a four-year period on two sections of the first course of the three-semester alternative college-level general chemistry program. On average, each section would begin with 33 students with an additional 12 students per section opting into the three-semester program after doing poorly in the first exam in the two-semester program (an option which we allowed). The statistics on MSAT and Verbal and Critical Reading SAT (SATL) scores for students finishing the program and getting a grade over the four-year study are shown in Table 1. A breakdown of the MSAT and SATL scores by year in the study is shown in Table 2. It can be seen that the student aptitude and preparedness as measured by the SAT is consistent from year to year. We felt confident that student performance across the years of the study could be compared.

Table 1 MSAT and SATL scores of students in the 4-year study

	Students finishing the program
	Starting in the program		Transferring in
	MSAT	SATL	MSAT	SATL
Average	502	509	571	536
Median	505	510	580	550
Std dev	50	61	59	72
High value	580	750	670	710
Low value	300	330	350	390

---

Table 2 Yearly average and median MSAT and SATL scores

	Students finishing the program
	Starting in the program				Transferring in
	MSAT		SATL		MSAT		SATL
Year	Average	Median	Average	Median	Average	Median	Average	Median
1	504	510	515	500	574	590	521	550
2	502	500	514	520	526	550	590	610
3	503	500	512	510	584	590	536	545
4	496	500	507	510	573	570	520	525

---

The course in this study covered the first five chapters of a typical college/university general chemistry textbook: (1) an introduction to matter, units and measurements, and dimensional analysis, (2) the basics of atoms, ions, ionic and molecular compounds, and binary inorganic nomenclature, (3) fundamentals of empirical and molecular formulas, chemical equations, and stoichiometry, (4) a treatment of aqueous solutions and reactions in aqueous solutions, including precipitation, acid/base, and oxidation reactions, as well as the concepts of concentration and the molarity concentration unit, and (5) a treatment of thermochemistry: first law of thermodynamics, enthalpy, calorimetry (constant volume and constant pressure), Hess's law, and enthalpies of formation.

There were four summative assessments: three midterm exams and a final exam. The first midterm exam covered the first two topics described previously. The second midterm exam covered the third topic and the first half of the fourth, covering aqueous solutions and reactions but not molarity. The third midterm exam covered the remainder of the fourth topic and the final topic, thermochemistry. The final exam was comprehensive. The midterm and final exams were structured so as to capture the true knowledge of the students. A multiple-choice section, similar in structure to the American Chemical Society (ACS) paired-questions exam (ACS, 2005) had a balance of concept-based and application-based questions. We also tried to have to have both “easy” and “more involved” question over the same topic. An easy concept question on writing and balancing chemical equations might take the form excerpted in Fig. 1. A more involved variant of a concept question over the same topic might take the form shown in Fig. 2. On the other hand, an easy applied question on the same topic might take the form: “When the chemical equation for the combustion of C₃H₈O is balanced, the sum of all the coefficients is …” We also included an open response section. Students were required to show their work on calculations or write essays/draw figures in response to these types of questions. 50% of the midterm exam score came from this open response section.

Fig. 1 Example of an Easy concept question on chemical equations.

Fig. 2 Example of a More Involved concept question on chemical equations.

The traditional/standard two-semester general chemistry program was structured such that students who did poorly in the first midterm exam in that course could transfer to the alternative three-semester program after the first exam. The first exam performance in the two-semester program was by far the best indicator of potential success or failure in the general chemistry program. Our metrics indicated that ninety percent of students who failed the first midterm exam in the standard two-semester program were likely to withdraw or get a grade of D or lower in that course. Allowing these students, who passed through the 580 MSAT cut-off filter, the option of transferring to the slower-paced three-semester program offered a greater chance for success. This meant that of the three midterm assessments in the three-semester program, the first was given to students whose MSAT scores were below the 580 cut-off and who opted to start in the alternative program, while the second and third midterms included students who were above the cut-off or who opted out of the alternative program but did poorly in the first exam of the two-semester program.

Hoping to get a clearer picture of student performance relative to the teaching method, we decided to further differentiate the second and third midterm assessment by reverting to a traditional lecture and textbook for the third topic coverage. As such, we had two semesters worth of student performance data in a traditional teaching method in the three-semester program, two semesters worth of data of students below the MSAT cut-off in a flipped classroom teaching method for the first topic and midterm exams, two semesters of data for a flipped classroom teaching method with a mix of students below the MSAT cut-off and students above the MSAT cut-off but who performed poorly in their first exam in the standard two-semester program taught in a traditional lecture and textbook system and who then transferred to the three-semester alternative program, and we had assessment data on the same mix of students taught in the traditional lecture and textbook method for the third topic in the alternative program.

The traditional lecture teaching method followed the following protocol: a lecture, direct instruction over the current topic, following the sequence in the textbook (Brown et al., 2015) used by the entire department, prescription of exercises taken from the textbook or the online, electronic assignment system allied with the textbook, in-class group work activity after a block of content had been covered and serving as a formative assessment, a quiz serving as a short summative assessment, and when the entire section had been covered in this way, a midterm exam, as described previously, covering the section topic.

The flipped classroom teaching method followed the following protocol: assignment of an animation module covering the upcoming topic, a worksheet, which served as a learning guide, was coupled to the animations. Screenshots of a typical animation module are provided in Fig. 3, and a more thorough treatment of the design principles of the animations is provided elsewhere (Gregorius, 2008, 2010, 2013; Gregorius et al., 2010b). An excerpt of the accompanying worksheet/study guide is provided in Fig. 4. Students were expected to use the worksheet to ensure the proper development of content knowledge. In the classroom, at least two days after the release of the appropriate animations and worksheets, a series of pre-developed questions, similar in coverage and focus to the worksheets, were given in sequence. This “questions document” served as a structured treatment of the content without the process devolving into direct instruction or a lecture. Students answered the series of questions – with the instructor eschewing actually providing the answers to the questions but guiding the critique of the answers provided by students. The answers that the students agreed upon would be noted on the questions document, and later, when the entire questions document was completed by the students, the questions document would be published through the class's web course tool and serve as class notes and a statement of content understanding. This procedure was viewed as allowing students to take ownership of the content knowledge, and also served to prod the students to go through the animations and worksheet prior to the classroom activity. After a block of content was completed, similar to the traditional teaching method described previously, group work activities, a quiz, and a midterm exam followed.

Fig. 3 Screenshots of the animation module for stoichiometry: mole concept. Modules usually (a) begin with a statement of learning objectives, followed by (b) a menu of content topics navigating to (c and d) a treatment – often highly visual of the topic (always with voice annotation/narration and closed-captioning), and usually with (e) content understanding self-checks. The module usually ends with (f) a programmatically generated exercise of randomly generated numbers and question wordings.

Fig. 4 Excerpts from the worksheet accompanying the mole concept flash animation.

The students' time allocation for the course was expected to be nearly equivalent for the traditional and flipped classrooms. While more study time would have to be put in to peruse the animations in the flipped classroom, this would be partially offset by the absence of assigned homework or graded assignments. This expectation was supported by formal surveys and informal student reports. There was no significant difference in the hours spent per week studying for the course between the two teaching methods. The average and median study time per week in the flipped classroom was 2.2 h and 2 h, respectively, while it was 1.9 h and 2 h in the traditional setting. The content coverage and class hours spent was identical in both settings.

Student performance over two semesters in both the traditional and flipped classroom settings was monitored. The final grades and midterm exam performances are reported here and serve as indicators of the potential effectiveness of the flipped classroom teaching method.

Results and discussion

Apart from the aforementioned likelihood of passing the two-semester general chemistry program if the student had a MSAT score higher than 580, there is no correlation between the student's MSAT or SATL scores with their final grade. No statistically significant relationship (p < 10⁻⁴⁸) could be found between the student performance in the flipped classroom and traditional setting to the student's MSAT or SATL score.

The final overall grade distribution, including students who withdrew from the course and received a grade of W, is shown in Fig. 5a. What stands out is the percent of students who withdrew from the course in the traditional setting: 52% of 208 students versus 22% of 154 students in the equivalent flipped classroom setting. At first glance, with a higher percentage of students in the flipped classroom setting completing the course, it would seem that the students in the flipped classroom outperformed their counterparts in the traditional setting.

Fig. 5 Overall performance: traditional vs. flipped. (a) All students and (b) students with recorded grades only.

If our objective were to have as many students pass the first semester of the alternative three-semester course, than it would seem that a flipped classroom was a more effective teaching method than the traditional approach. Informal interviews and polling of students in the flipped classroom pointed out that students felt that they had “a good chance of making it”, that the material “made more sense”, that they “could always go back” and review the material – all of which contributed to students deciding to stay in the course. Other investigators have reported similar findings on student preference for the flipped classroom teaching method (Fitzgerald and Li, 2015; Seery, 2015; Weaver, 2015), and of students having a sense of confidence in their grasp of the material when presented in this way (Schultz et al., 2014; Hibbard et al., 2016; Shattuck, 2016).

However, if the objective was to enhance the learning of students (as measured by performance in exams), we might look to the performance of only the students who remained in the course and received a grade. If the students who eventually withdrew were taken out of the analysis, a somewhat different picture appears. The grade distribution for just the students who took the final exams and had a grade recorded is shown in Fig. 5b. There appears to be no significant difference in the percent distribution of students through the range of letter grades. It would seem that the strongest effect of the flipped classroom intervention is in the percent of students who successfully completed the course, rather than a better performance for those who did complete the course. As such, we wanted to delve deeper into the student performance of those who did complete the course in the hope of getting a clearer picture of student performance under the two teaching methods.

The performance in the first midterm exam for only the students who completed the course in both the traditional and flipped classroom teaching methods is summarized in Fig. 6a. There were n = 84 in the combined 4 sections over two semesters of the traditional teaching method, and n = 85 in the flipped classroom teaching method. The average and median scores for the traditional settings were 68 and 65, respectively, and 64 and 66 for the flipped classroom. It would seem, from the median and average scores, that there is no difference in student performance when subjected to the two different teaching methods. However, it can be seen from the graph of grade distributions that there was a greater percentage of students in the extremes in the flipped classroom setting. There were 4% of students getting A's in the traditional setting, while there were 11% in the flipped classroom. On the other hand, there were 12% getting D's in the traditional versus 21% in the flipped. This seems to point out that within the group of underprepared students (those who came into their first year of college with a MSAT of 580 or lower), there is a small subgroup that might thrive in a flipped classroom setting, but there is also a small subgroup that would perform worse when compared to those in the traditional teaching method.

Fig. 6 Student performance in the first and second midterm exam (counting only the students who finished the course). (a) Students below the 580 MSAT cut-off, and (b) students in (a) mixed with students who transferred in.

A similar pattern can be seen for the student performance in the second midterm exam, when students transferring from the two-semester program had been incorporated into the study groups. This is summarized in Fig. 6b. There were n = 100 in the combined 4 sections over two semesters of traditional teaching method, and n = 119 in the flipped classroom teaching method. Average and median scores for the traditional settings were 62 and 63, respectively, and 63 and 65 for the flipped classroom. Again, it would seem, from the median and average scores, that there is no difference in student performance when subjected to the two different teaching methods. However, the same trend of students in the flipped classroom either doing better or worse than their counterparts in the traditional setting can be seen. 1% of students in the traditional setting received A's, while 8% did so in the flipped classroom. On the other hand, there were 18% getting F's in the traditional versus 26% in the flipped classroom setting.

This trend of the flipped classroom being more effective for the upper bracket and worse for the lower bracket of students in this study can be further highlighted when the transfer student performance was separated from the overall student performance in this study. Fig. 7 separates the performance in the second midterm exam of students above and below the 580 MSAT cut-off; Fig. 7a shows the data for students who started the semester in the three-semester program, while Fig. 7b shows that of students who transferred into the alternative program after doing poorly in the first exam of the two-semester program. It can be seen from these graphs that, again, there is a wider spread of student performance in a flipped classroom setting, with more students performing better and worse than their traditional counterparts, and that this difference in overall performance is independent of whether the students started the semester in the three-semester, flipped classroom program or not.

Fig. 7 Student performance for the second midterm exam, for (a) students who started in the alternative program, and (b) students who opted-in after the first midterm exam in the standard two-semester program.

The third midterm exam was over content that was handled in a traditional teaching system for both groups in this study; the expectation was that there would be no difference in student performance between the two groups. However, it can be clearly seen in the graph shown in Fig. 8 that the students who had developed content in the flipped classroom teaching method performed poorer than their counterparts in the traditional setting when the teaching method reverted from flipped classroom to traditional teaching methods. Splitting the performance comparison into students who started in the alternative program (Fig. 8b) from students who transferred in after the first exam (Fig. 8c) showed that the poor performance for students who started in a flipped classroom setting and reverted to a traditional lecture setting for the third content and exam was independent of whether they started in the alternative program or transferred in midway through the semester. It seems that after a student had been studying in a flipped classroom setting for a certain period, going back to a traditional teaching/studying method had an adverse effect on the student's performance.

Fig. 8 Performance in the third midterms. For (a) all students, (b) students who started in the alternative program, and (c) students who opted-in.

Summary and conclusions

The data seem to indicate that students entering the first semester of the first year general chemistry program and coming in with poor preparation as indicated by their performance in their MSAT and SATL college entrance examination were more likely to remain in the program and receive a grade if a flipped classroom teaching method were used. This was in conjunction with the students’ reports that they found the content more accessible, understood the material better, could review the material with more ease, and believed that they had a good chance of success in this teaching method. Considering that 70% of the students who started in this alternative three-semester program were expected to fail in the standard two-semester general chemistry program, and 90% of the students who transferred in from the two-semester program into the alternative program were also expected to fail in the two-semester program, having 78% of these students complete the course when a flipped classroom teaching method was used, compared to 48% in the traditional teaching method, clearly points to the merits of a flipped classroom teaching method. If we only count the students who received a high enough grade in order to be allowed to proceed to the second semester of the three-semester program (a grade of C– or higher), then a 62% pass-through performance in a flipped classroom teaching method versus a 38% pass-through in the traditional teaching method again highlights the merits of a flipped classroom approach to teaching.

When the performance of only the students who completed the course was analyzed, it appeared that there were subgroups of students who either performed better or worse in the flipped classroom setting when compared to equivalent students in the traditional teaching method. This seemed to indicate that while students may get a sense of confidence in their grasp of the content in a flipped classroom setting, enough to put effort into the course and complete it, this sense did not readily translate to increased learning (as measured by performance in exams) for some of these students. The exam performance average and median scores for students who completed the flipped classroom course and for those who completed the traditional lecture-based course were similar enough to suggest that the level of learning for both groups in the study was comparable.

However, the average and median scores mask the difference in the spread or distribution of letter grades. The shape of the distribution curves were sufficiently different that it could be noted that there were more higher scoring as well as low scoring students in the flipped classroom. This could mean that a flipped classroom teaching method might have been too much for a small subset of students, who worked to stay in the course but could not perform well. We may need to be aware that not all students will thrive in this teaching method. On the other hand, this could mean that since the retention rate is higher in the flipped classroom setting, and considering that the students in this study are generally underprepared for college-level study, the shape of the letter grade distribution in the flipped classroom might have been skewed toward lower letter grades with the inclusion of more underperforming students. In this case, the flipped classroom student performance curve might actually represent higher learning outcomes. Further study needs to be made in order to determine if – higher retention rates not withstanding – an animation-based flipped classroom allows for higher learning outcomes, and/or the development of better understanding of particular topics.

Another important finding of this study was that while these underprepared students could readily adjust to a no-lecture, animation-based flipped classroom teaching method, after a certain amount of exposure to this method – the length of one content topic or chapter – reverting to a traditional teaching method was shown to have an adverse effect on student performance. This seems to indicate that, for these types of students, using multiple and very different teaching methods, especially if the novel teaching method is used for a prolonged period before switching to a traditional teaching method, would have a deleterious effect on student performance.

Contrary to the preconception among some instructors that the flipped classroom teaching method will not be effective for underprepared students, this study seems to indicate that even underprepared students, the bottom third of incoming first-year college students, will perform well in the flipped classroom. While there will be some students who will be adversely affected by, and have difficulty adjusting to, this novel teaching method, this effect is offset by the percentage of students who persist in the program, and the percentage of students who perform better in the flipped classroom setting despite coming in underprepared.

Implications for practice

Inquiry-focused, self-guided animations and simulations with guide worksheets provide a viable alternative procedure for providing content in a flipped classroom teaching method. Using worksheet-supported animations and simulations in a flipped classroom setting further emphasize learning as a process of developing, rather than absorbing, content and skills. While a video-recorded lecture is still primarily a direct instruction teaching method focused on delivering content, the animations and simulations of the type used in this investigation provides an environment for the student to develop content for himself or herself. The structure of the flipped classroom teaching method mitigates against any misconceptions a student might develop while going through the first phase of the process since the next, immediate activity is designed to be formative and corrective. Animations and simulations of the type used here have already been shown to lend itself to better student content development (Gregorius, 2005a, 2010, 2013; Gregorius et al., 2010a, 2010b), and the evidence here suggests that it is suited to a flipped classroom teaching method.

While the teaching method used here had a significant impact on student retention even among underprepared students, there is concern that, based on the performance of students who did complete the course, learning was not particularly enhanced for all students. Some students thrived in the teaching method while others fared worse than their counterparts in a traditional, lecture-based course. It may be necessary to temper our expectations of what this teaching method can allow for our students.

It is important to note that, should instructors choose to adopt this teaching system, there is data here suggesting that switching back to a traditional lecture-based teaching method after using the flipped classroom method for at least one unit or book chapter can have an adverse effect on student performance in later chapters. Rather than using the method for specific content or switch back and forth between different teaching methods, it might be better to develop an entire semester as a flipped classroom.

Conflicts of interest

There are no conflicts to declare.

Notes and references

1. ACS Exam Institute, (2005), General Chemistry Exam – Paired Questions, First Term|ACS Exams.
2. Brown T. L., LeMay H. E. J., Bursten B. E., Murphy C. J., Woodward P. and Stoltzfus M. E., (2015), Chemistry: The Central Science, 13th edn, Upper Saddle River, NJ: Pearson Prentice Hall.
3. Butzler K. B., (2015), ConfChem Conference on Flipped Classroom: Flipping at an Open Enrollment College, J. Chem. Educ., 92(9), 1574–1576.
4. DeLozier S. and Rhodes M. G., (2017), Flipped Classrooms: A Review of Key Ideas and Recommendations for Practice, Educ. Psychol. Rev., 29(1), 141–151.
5. Fitzgerald N. and Li L., (2015), Using Presentation Software To Flip an Undergraduate Analytical Chemistry Course, J. Chem. Educ., 92(9), 1559–1563.
6. Gregorius R. Ma., (2005a), Various Learning Environments and Their Impact on Student Performance, Part II: PowerPoint versus Flash–based Self–Instruction, Chem. Educ., 10, 78–81.
7. Gregorius R. Ma., (2005b), Various Learning Environments and Their Impact on Student Performance, Part I: Traditional versus PowerPoint and WebCT Augmented Classes, Chem. Educ., 10, 72–77.
8. Gregorius R. Ma., (2008), An eBook in Flash to Support Inductive Learning, Newsl. Conf. Chem. Educ. CONFCHEM.
9. Gregorius R. Ma., (2010), Good Animations: Pedagogy and Learning Theory in the Design and Use of Multimedia, in Belford R. E., Moore J. W. and Pence H. E. (ed.), Enhancing Learning with Online Resources, Social Networking, and Digital Libraries, ACS Symposium Series, American Chemical Society, pp. 167–190.
10. Gregorius R. Ma., (2011), Student Performance in Various Learning Protocols, J. Coll. Sci. Teach., 40, 101–111.
11. Gregorius R. Ma., (2013), Linking Animation Design and Usage to Learning Theories and Teaching Methods, in Pedagogic Roles of Animations and Simulations in Chemistry Courses, ACS Symposium Series. American Chemical Society, pp. 77–96.
12. Gregorius R. Ma., Santos R., Dano J. B. and Guitierrez J. J., (2010a), Can Animations Effectively Substitute for Traditional Teaching Methods? Part II: Potential for Differentiated Learning, Chem. Educ. Res. Pract., 11, 262–266.
13. Gregorius R. Ma., Santos R., Dano J. B. and Guitierrez J. J., (2010b), Can Animations Effectively Substitute for Traditional Teaching Methods? Part I: Preparation and Testing of Materials, Chem. Educ. Res. Pract., 11, 253–261.
14. Hibbard L., Sung S. and Wells B., (2016), Examining the Effectiveness of a Semi-Self-Paced Flipped Learning Format in a College General Chemistry Sequence, J. Chem. Educ., 93(1), 24–30.
15. Lenczewski M. S., (2016), Scaffolded Semi-Flipped General Chemistry Designed To Support Rural Students' Learning, J. Chem. Educ., 93(12), 1999–2003.
16. Logan B., (2015), Deep Exploration of the Flipped Classroom before Implementing, J. Instr. Pedagog., 16, 1–12.
17. Mooring S. R., Mitchell C. E. and Burrows N. L., (2016), Evaluation of a Flipped, Large-Enrollment Organic Chemistry Course on Student Attitude and Achievement, J. Chem. Educ., 93(12), 1972–1983.
18. Muzyka J. L., (2015), ConfChem Conference on Flipped Classroom: Just-in-Time Teaching in Chemistry Courses with Moodle, J. Chem. Educ., 92(9), 1580–1581.
19. Robert J., Lewis S. E., Oueini R. and Mapugay A., (2016), Coordinated Implementation and Evaluation of Flipped Classes and Peer-Led Team Learning in General Chemistry, J. Chem. Educ., 93(12), 1993–1998.
20. Rossi R. D., (2015), ConfChem Conference on Flipped Classroom: Improving Student Engagement in Organic Chemistry Using the Inverted Classroom Model, J. Chem. Educ., 92(9), 1577–1579.
21. Ryan M. D. and Reid S. A., (2016), Impact of the Flipped Classroom on Student Performance and Retention: A Parallel Controlled Study in General Chemistry, J. Chem. Educ., 93(1), 13–23.
22. Schultz D., Duffield S., Rasmussen S. and Wageman J., (2014), Effects of the Flipped Classroom Model on Student Performance for Advanced Placement High School Chemistry Students, J. Chem. Educ., 91(9), 1334–1339.
23. Seery M. K., (2015), Flipped learning in higher education chemistry: emerging trends and potential directions, Chem. Educ. Res. Pract., 16(4), 758–768.
24. Shattuck J. C., (2016), A Parallel Controlled Study of the Effectiveness of a Partially Flipped Organic Chemistry Course on Student Performance, Perceptions, and Course Completion, J. Chem. Educ., 93(12), 1984–1992.
25. Song Y. and Kapur M., (2017), How to Flip the Classroom – “Productive Failure or Traditional Flipped Classroom” Pedagogical Design? J. Educ. Technol. Soc., 20(1), 292–305.
26. Weaver G. C., (2015), Design, Implementation, and Evaluation of a Flipped Format General Chemistry Course, J. Chem. Educ., 92(9), 1437–1448.
27. Zainuddin, Z. and Halili S. H., (2016), Flipped Classroom Research and Trends from Different Fields of Study, Int. Rev. Res. Open Distrib. Learn., 17(3), 313–340.

---