Evaluating student motivation in organic chemistry courses: moving from a lecture-based to a flipped approach with peer-led team learning

Abstract

Academic Motivation Scale-Chemistry (AMS-Chemistry), an instrument based on the self-determination theory, was used to evaluate students’ motivation in two organic chemistry courses, where one course was primarily lecture-based and the other implemented flipped classroom and peer-led team learning (Flip–PLTL) pedagogies. Descriptive statistics showed that students in both courses were more extrinsically motivated and their motivation moved in negative directions across the semester. Factorial multivariate analysis of covariance revealed a main effect of pedagogical approach. Students in the Flip–PLTL environment were significantly more motivated toward chemistry at the end of the semester while controlling for the motivation pre-test scores; however, there was no evidence for a sex main effect or an interaction effect between sex and pedagogical approach. Correlation results revealed variable relationships between motivation subscales and academic achievement at different time points. In general, intrinsic motivation subscales were significantly and positively correlated with student academic achievement; Amotivation was negatively correlated with academic achievement. The findings in this study showed the importance of Flip–PLTL pedagogies in improving student motivation toward chemistry.

---

Introduction

Students struggle with organic chemistry because organic chemistry is challenging for students to learn (Lynch and Trujillo, 2011; Bodner et al., 2017). For example, students had difficulties with mechanisms (Bhattacharyya, 2014) and had limited understanding of what curved-arrow formalism means (Bhattacharyya and Bodner, 2005). Students are often intimidated and anxious when entering organic chemistry courses. There is a widely recognized problem with attrition and student learning in the course (Tien et al., 2002). In exploration of motivation in organic chemistry courses, motivation has shown associations with student academic achievement (Black and Deci, 2000; Lynch and Trujillo, 2011), sex differences in the associations (Lynch and Trujillo, 2011), and increased students’ perceptions of intrinsic motivation over the semester in organic chemistry courses implementing peer-led team learning (PLTL) (Black and Deci, 2000).

Motivation

Motivation refers to the desire to act. Motivation has been found to be one of the important factors to improve students’ persistence in science, technology, engineering, and mathematics (STEM) and to promote better academic achievement (Guay et al., 2010; PCAST, 2012; Griffin et al., 2013; Kusurkar et al., 2013; Taylor et al., 2014; Sturges et al., 2016). Motivation is a complex multidimensional construct ranging from amotivation (no motivation), to extrinsic motivation, and finally intrinsic motivation, along a self-determination continuum based on self-determination theory (SDT) (Deci and Ryan, 2000; Ryan and Deci, 2000). When students are amotivated, they lack the desire to perform any activity. For these students, it is necessary to provide incentives to extrinsically motivate them for learning to occur (Deci and Ryan, 2000). Extrinsic motivation means that people do an activity because of its consequences, such as rewards or punishments. Extrinsic motivation differs from intrinsic motivation: when students enjoy an activity and find the activity inherently satisfying, this is defined as intrinsic motivation (Ryan and Deci, 2000). People are differently motivated between tasks due to the reasons for doing the activity and the level of satisfaction experienced when engaging in the activity.

Motivation is closely associated with social environments. Social contexts either support or thwart the natural tendencies toward engagement (Vansteenkiste et al., 2006). When students take a consumerist approach to higher education, there is a shift from intrinsic to extrinsic motivation (Labaree, 1999). In a learning environment, where students are provided with meaningful rationales for doing the learning activities and given opportunities to interact with one other, and the instructors are encouraging and respecting of students, intrinsic motivation can be promoted (Black and Deci, 2000; Lepper and Henderlong, 2000; Chirkov and Ryan, 2001; Vansteenkiste et al., 2006; Su and Reeve, 2011; Reeve, 2012; Vaino et al., 2012; Jang et al., 2016a, 2016b). For example, Black and Deci (2000) found that organic chemistry students, who attended full-class lectures and randomly assigned PLTL groups in a small, eastern university in the U.S., had a positive but nonsignificant change on the enjoyment and interest, which is regarded as a measure of intrinsic motivation in the Intrinsic Motivation Inventory (McAuley et al., 1989), by the end of a semester. In another study, basic and high school students in Estonia had higher levels of intrinsic motivation when context-based modules were implemented in chemistry courses (Vaino et al., 2012).

Motivation may decrease over time (Zusho et al., 2003; Nilsson and Warrén Stomberg, 2008; Brouse et al., 2010; He et al., 2015). A decline in students’ motivational levels (i.e., self-efficacy, task value) was found for college students enrolled in introductory chemistry courses in a large Midwestern University in the U.S. over the course of a semester (Zusho et al., 2003). Motivation also decreased during a single term course for nursing students in Sweden (Nilsson and Warrén Stomberg, 2008). Freshmen students from Canada had higher levels of intrinsic and extrinsic motivation compared with seniors in college courses (Brouse et al., 2010). Students in an electrical engineering course in the western U.S. also had a decline in motivation that was associated with a decrease in exam scores (He et al., 2015). Researchers have also found that students’ perceptions of other affective variables, such as attitude and self-concept, changed across a course (Chan and Bauer, 2015). The findings from previous literature suggest that motivation in chemistry courses may decrease over time.

Sex differences in motivation levels have been reported. Female college students usually reported higher motivation levels than males (Vallerand et al., 1992; Cokley et al., 2001; Spittle et al., 2009; Brouse et al., 2010; Smith et al., 2010; Eymur and Geban, 2011). Females are under-represented in STEM fields (Ong et al., 2011; Villafane et al., 2014). It is important to investigate females’ motivation levels toward science because sex has been found as an important factor on academic achievement in science (Osborne et al., 2003). In a U.S. general chemistry course, females were found to be less motivated toward chemistry than male students (Liu et al., 2017). To date, sex differences in motivation in organic chemistry courses have not been explored; therefore, we were curious about how motivation may differ at the end of the semester by sex taking into considerations of students’ initial motivation levels.

Mixed results were reported in the literature regarding associations between motivation and academic achievement; one of the possibilities for such variable findings is the use of multiple definitions of motivation and achievement measures across studies. Cerasoli et al. (2014) reviewed 40 years of publications finding an average correlation coefficient of 0.24–0.31 between intrinsic motivation and academic achievement for college-aged samples. In chemistry, Jurisevic et al. (2012) found that Slovenian and Polish vocational and technical high school students with higher motivational scores on intrinsic motivation, regulated motivation, and controlled motivation scored higher on their tests of knowledge of visible spectrometry. In an introductory organic chemistry course, students’ motivation scores as measured by interest/enjoyment, were positively correlated with students’ academic achievement expressed by average grades of four in-term exams and final course grade (Black and Deci, 2000). In contrast, two studies in Slovenia found weak evidence that intrinsic motivation was positively associated with chemistry achievement for elementary students (Devetak et al., 2009) and for first-year pre-service primary school teachers (Jurisevic et al., 2008). Associations between extrinsic motivation and academic achievement are also mixed (Lynch and Trujillo, 2011; Liu et al., 2017). For example, organic chemistry students’ extrinsic goal, a type of motivation, was significantly and negatively correlated with academic achievement for females but the correlation was not significant for males (Lynch and Trujillo, 2011). In general chemistry, extrinsic motivation toward chemistry was not significantly correlated with exam scores at the beginning of the semester but positively correlated with exam scores at the end of the semester (Liu et al., 2017).

Academic motivation scale-chemistry

Based on self-determination theory, Academic Motivation Scale (AMS) (Vallerand et al., 1992) was developed and used in many contexts to examine student motivation toward education. Academic Motivation Scale-Chemistry (AMS-Chemistry), which measures student motivation toward chemistry in a specific chemistry course, a type of situational level of motivation (Vallerand, 1997; Vallerand and Ratelle, 2002), was recently developed by modifying the items in AMS (Liu et al., 2017). AMS-Chemistry measures amotivation, three types of extrinsic motivation (external regulation, introjected regulation, and identified regulation), and three types of intrinsic motivation (to know, to accomplish, and to experience).

Validity (including content, response process, internal structure, and relationships to other variables) and internal consistency reliability psychometric evidence (Arjoon et al., 2013) has been collected for AMS-Chemistry in the context of a general chemistry course (Liu et al., 2017). Results suggested that the AMS-Chemistry scores were valid and reliable in general chemistry and could be interpreted in terms of the seven motivational subscales, females were significantly less motivated than males, and motivation and its association with exam scores varied across a course (Liu et al., 2017). This previous work suggests that the AMS-Chemistry is a potential instrument to identify potential motivational movement along the self-determination continuum across a course. In this study, AMS-Chemistry was used to evaluate a pedagogical change in organic chemistry courses over a course of semester.

PLTL and flipped classroom

In the past few decades, active learning pedagogies have been extensively developed and implemented in college settings, leading to a number of positive outcomes compared with traditional classroom experiences, as has been documented in several reviews and meta-analyses (Prince, 2004; Freeman et al., 2014; Warfa, 2016; Wilson and Varma-Nelson, 2016; Apugliese and Lewis, 2017). Observed outcomes included increased academic achievement (Prince, 2004; Freeman et al., 2014; Warfa, 2016; Wilson and Varma-Nelson, 2016; Apugliese and Lewis, 2017), increased student retention (Wilson and Varma-Nelson, 2016), higher attitudes toward STEM disciplines (Brandriet et al., 2011; Chonkaew et al., 2016; Vishnumolakala et al., 2017), reduced attrition rates (Henry, 2017), and benefits for students who actively responded to questions in class on a regular basis and those who did not (Obenland et al., 2013). Self-determination theory (SDT) hypothesizes that in more autonomous learning environments, such as often found to be associated with active learning environments, students feel they have obtained the desired effects and outcomes and developed more internalized, or intrinsic, motivation (Deci and Ryan, 2000; Reis et al., 2000; Ryan and Deci, 2000). Without internalization of motivation, people won’t persevere through difficulty (Ryan and Deci, 2000; Prince, 2004). Research has explored the impact active learning pedagogies have on achievement (Prince, 2004; Freeman et al., 2014; Warfa, 2016; Wilson and Varma-Nelson, 2016; Apugliese and Lewis, 2017) and motivation (Black and Deci, 2000; Prince, 2004; Obenland et al., 2013; Abeysekera and Dawson, 2015; Cicuto and Torres, 2016). For example, biochemistry students in an active learning environment were found generally motivated and their motivation scores were higher than or equal to those in other courses, which suggested that the active learning environment had a positive impact on students’ motivation (Cicuto and Torres, 2016).

Peer-led team learning (PLTL) is an educational reform to actively engage students in learning through small-group work led by students who successfully completed the target course (Gosser and Roth, 1998; Wilson and Varma-Nelson, 2016). Peer leaders are trained to guide and engage students to facilitate collaboration, ask students to explain their work and seek group consensus, and point students to resources (Tien et al., 2002; Hockings et al., 2008; Drane et al., 2014; Robert et al., 2016). The impact of PLTL includes improved pass rates (Mitchell et al., 2012), increased student achievement (Lewis and Lewis, 2005a, 2008; Hockings et al., 2008; Popejoy and Asala, 2013; Drane et al., 2014; Carlson et al., 2016; Robert et al., 2016), and increased retention in STEM degree programs (Lewis, 2011; Mitchell et al., 2012; Popejoy and Asala, 2013; Drane et al., 2014; Lewis, 2014). PLTL has been extensively implemented and evaluated in first-semester general chemistry (Lewis and Lewis, 2005a, 2005b, 2008; Hockings et al., 2008; Lewis, 2011; Popejoy and Asala, 2013; Drane et al., 2014; Lewis, 2014; Chan and Bauer, 2015; Carlson et al., 2016). The implementation has been extended to other more advanced college chemistry courses (Tien et al., 2002; Lyle and Robinson, 2003; Wamser, 2006; Arrey, 2012; Mitchell et al., 2012; Lewis, 2014; Robert et al., 2016). There are fewer implementations of PLTL in organic chemistry courses (Tien et al., 2002; Lyle and Robinson, 2003; Wamser, 2006; Arrey, 2012; Robert et al., 2016). PLTL has showed positive outcomes in organic chemistry courses (Tien et al., 2002; Lyle and Robinson, 2003; Wamser, 2006; Arrey, 2012; Robert et al., 2016). For example, students experiencing PLTL workshops had increased performance and retention compared with students experiencing traditional recitation sessions in first semester organic chemistry courses (Tien et al., 2002). Higher success rates and persistence for students who participated in the PLTL organic chemistry workshops were reported (Wamser, 2006). Moreover, in an intensive course implementing PLTL, students performed better than students in courses with traditional semester format (Arrey, 2012).

Studies on PLTL mainly focus on cognitive learning (Lewis and Lewis, 2005a, 2008; Hockings et al., 2008; Popejoy and Asala, 2013; Drane et al., 2014; Carlson et al., 2016; Robert et al., 2016); fewer studies have focused on affect (Gafney and Varma-Nelson, 2008). Chan and Bauer (2015) studied students’ attitude, self-concept, and motivation in general chemistry taught with PLTL; they found that students had negative changes on their attitude and self-concept with small to medium effect sizes; they did not observe any significant differences in traditional and PLTL environments.

The flipped classroom is an instructional strategy developed to engage students in the course material before and during class in ways different from a traditional classroom (Seery, 2015). In flipped classrooms, students are required to study resources, e.g., recorded lectures, videos, tutorials, textbooks, worksheets, etc. before coming to class. Content is moved outside of class to allow for more time to do other activities during class, such as problem-solving sessions, small-group work, or classroom discussions (Morrison, 1976; Smith, 2013; Fautch, 2015; Flynn, 2015; Seery, 2015; Eichler and Peeples, 2016; Henry, 2017). Most research has focused on how well flipped classroom pedagogies improve cognitive learning. In general, researchers have reported increased or comparable outcomes compared with traditional teaching methods (Love et al., 2013; Jensen et al., 2015; Weaver and Sturtevant, 2015; Hibbard et al., 2016; Robert et al., 2016; Ryan and Reid, 2016; Shattuck, 2016; Gregorius, 2017). The attitude towards study materials (in particular videos or tutorials) has been considered (Fautch, 2015; Flynn, 2015; Jensen et al., 2015; Eichler and Peeples, 2016; Reid, 2016). Students responded that out-of-class videos used in the studies were effective and helpful (Seery, 2015; Hibbard et al., 2016; Reid, 2016; Shattuck, 2016). Recently researchers found that students in an organic chemistry that utilized flipped classroom and PLTL pedagogies had positive changes on attitudes towards chemistry (Mooring et al., 2016). A recent study found that general chemistry students, taught by flipped classroom pedagogy in a historically Black college and university for women in the southeast U.S., had positive motivation perceptions toward chemistry as displayed by high scores on intrinsic motivation, career motivation, self-determination, self-efficacy, and grade motivation subscales (Hibbard et al., 2016). However, in this study very few students (n = 27) took the motivation instrument. Moreover, motivational perceptions are rarely studied in classrooms with both PLTL and flipped classroom implemented and integrated.

In PLTL and flipped classroom environment, students could develop more intrinsic motivation. In PLTL environment, students have more chance to interact with their group members and peer leaders, and the activities are well designed, and peers and instructors are autonomy supportive. Therefore, according to the SDT, students can have their basic needs of autonomy, competence, and relatedness met, which will promote intrinsic motivation (Deci and Ryan, 2000; Ryan and Deci, 2000). A similar argument was made that flipped approaches might improve student motivation (Abeysekera and Dawson, 2015). The study materials that students are required to study in a flipped classroom are carefully selected, or created, and scaffolded, which helps student learn and therefore is good for students’ competence. The students can choose when to and how often to study the materials, which can meet student autonomy need. Again when students’ basic needs are met, we can promote student intrinsic motivation. When we integrate flipped classroom and PLTL in the same course, the instructor has more class time for learning activities in small groups because part of the content is moved outside of the lecture; students have more options and study resources in and outside of classroom. We, therefore, expect that the dual implementation of flipped classroom and PLTL pedagogies (Flip–PLTL) will have a positive effect on student motivation.

In this study, flipped classroom and peer-led team learning (Flip–PLTL) were integrated and implemented in organic chemistry by author JRR as a means to increase student motivation. Author JRR accomplished this goal through small group work during lecture periods. Students in the Flip–PLTL course interacted with peer-leaders and author JRR during small group work time. Peer leaders were trained to serve as facilitators of learning rather than expert sources of information. To examine the effect of the Flip–PLTL pedagogy, motivation toward chemistry was measured at the beginning and end of lecture-based and Flip–PLTL courses, both taught by author JRR, using the AMS-Chemistry instrument (Liu et al., 2017).

Research questions

In this study, motivation will be studied in a lecture-based course and a Flip–PLTL course using the AMS-Chemistry instrument (Liu et al., 2017). The following research questions will be addressed:

1. How does AMS-Chemistry function in two differently taught organic chemistry courses?

2. What is the student motivation status at the beginning and end of a semester in organic chemistry courses?

3. How do changes on motivation differ by pedagogical approaches and by sex?

4. How is motivation associated with student academic achievement in the organic chemistry courses?

Methods

Research context

This study was conducted in two first-semester courses of a yearlong organic chemistry course, which were taught a year apart by author JRR at a large research-intensive university in the southeast United States. Author JRR's course was one of the five sections of first semester organic chemistry course in Fall 2014 and Fall 2015. Students in this course learn about the three-dimensional chemical structures, naming the chemical structures, simple functional group transformations, and chemical reactions. Many of the students who are enrolled in this course are premedical or biomedical science majors; most students take the course to satisfy a major requirement. The course consisted of two 75 minute lecture periods and a 50 minute recitation period weekly for 15 weeks. Lectures were held in a large classroom (∼300 seats); recitations were held in small classrooms (∼35 students). While the two courses covered the same material, the two courses were taught with different pedagogical approaches (i.e. lecture-based pedagogy and Flip–PLTL pedagogy).

The first course was taught with a lecture-based pedagogy. Author JRR utilized a classroom response system (i.e., clicker) to promote discussion in pairs, small groups, and amongst the classroom. Author JRR was interactive and dynamic, posing questions and eliciting answers from the students throughout the weekly lecture periods. Attendance in lecture was tied to the classroom response system, of which points were awarded for both attendance and answer correctness. Recitation sessions were structured by teaching assistants; students had the opportunity to ask questions about difficult content, see additional worked examples, and request assistance with homework and suggested problems from the textbook. There were four during-the-term examinations and a final cumulative examination.

The second course was taught with a hybrid of lecture, flipped, and peer-led team learning (PLTL) pedagogies (Flip–PLTL). Author JRR continued to utilize a classroom response system, as with the first course, to promote discussion and learning in one of two lecture periods each week. In addition, students were assigned American Medical Association Commissioned Khan Academy videos (via YouTube) to watch before the second lecture period each week. The videos were approximately eight minutes long on average and addressed course learning objectives. We have anecdotal evidence to suggest that, as with most implementations of flipped-classroom pedagogies, there is great variability in engagement with the video content; we lack an observational strategy to monitor student engagement with the video content.

For the second lecture period, students completed a worksheet with the assistance of peer leaders (i.e., students who had successfully completed the course with an A or better). Worksheets had between four and eight items each, with some items having multiple parts particularly for topics including nomenclature, translating structural representations, and predicting the products of reactions; worksheet items were developed by author JRR and modeled after “end-of-the-chapter” items that are typical of most postsecondary second-year level organic chemistry textbooks. There was approximately one peer leader for every 12 to 16 students in the course. Author JRR provided little lecture during the second lecture period other than to explain clicker questions. Again, attendance during these two lecture periods was tied to the classroom response system; points again were awarded for attendance and correctness. Recitation sessions continued to be teaching assistants led. There were four during-the-term examinations. The ACS Organic Chemistry First-Term Examination (2014) was used as the final examination for the second course.

Author JRR started the project with a broad array of instructional experience using multiple pedagogical methods including using classroom response systems, flipped-classroom and small-group pedagogies. For the lecture-based pedagogy, this was the first instance of author JRR teaching the particular course at the research setting. For the hybrid pedagogy, this was the first instance of author JRR specifically combining flipped-classroom and peer-led team learning pedagogies.

Data were collected in accordance with University of South Florida Institutional Review Board (IRB) applications: Improving Gateway Chemistry Courses (Pro00017861) and Pathways to Success. The study activities (in the lecture-based course) in Pathways to Success constituted program evaluation and did not meet the USF definition of human subject research. Students were provided with a description of the studies prior to participation in the survey. Completion of the survey was deemed by the USF IRB as consent to participate in the study.

Instrument

Academic Motivation Scale-Chemistry (AMS-Chemistry) (Liu et al., 2017) was used to measure student motivation toward taking an organic chemistry course in this study. Sample items for each scale are displayed in Table 1. Students responded to 28 possible reasons for being enrolled in the target chemistry course. A five-point Likert scale ranging from “1” (not at all) to “5” (exactly) was used to show the degree of agreement with each reason.

Table 1 Sample items of AMS-chemistry

Subscales	Sample item
Amotivation	I don’t know; I can’t understand what I am doing taking chemistry courses.
External regulation	Because without having taking chemistry I would not find a high-paying job later on.
Introjected regulation	To prove to myself that I am capable of succeeding in chemistry.
Identified regulation	Because taking chemistry will enable me to enter a job market in a field that I like.
To experience	For the enjoyment I experience when I think about the world in terms of atoms and molecules.
To accomplish	For the satisfaction I feel when I work toward an understanding of chemistry.
To know	Because study chemistry allows me to continue to learn about many things that interest me.

---

Exam 1 was used as the first achievement measure. Exam 4 was used as the second achievement measure. The two exams were out of 100 points. In addition, Final Exam (cumulative and out of 150 points) and Final Score (the final course grade in percentages) were used to examine if motivation is associated with students’ overall academic achievement.

Data collection and participants

The AMS-Chemistry was administered as paper-and-pencil test in the target organic chemistry courses. The students were given 10 minutes during lecture to complete the survey. Students received a small amount of bonus points towards their final exam score for participating in the study; these points amount to less than 0.25% of the student's final course grade. We acknowledge that such a reward system may amplify a student's extrinsic motivation; however, the small percentage of a student's final course grade represented by the bonus points dampens the potential overall effect of such a reward for participation. Data were collected at two time points during lecture time in Fall 2014 and Fall 2015 semesters.

There were 257 students enrolled in the first course taught by lecture-based instructional pedagogy in 2014, coded as the “Lecture-Based” course. The pre-semester data collection (Pre1) was during the second week of class before Exam 1 in 2014. After checking for missing data and careless responses (e.g. the same response for all the 28 items), 235 students’ responses (93% of all the responses) were used for data analysis. The post-semester data collection in 2014 (Post1) was during the last (15th) week of class following Exam 4, and all the 224 students who responded to the survey had complete responses (response rate = 87%), which were used for data analysis. Both Pre1 and Post1 datasets were used to answer the first research question. In addition, a matched-pair sample (n = 204), who had compete responses for the two administration of AMS-Chemistry, was used to examine the status of student motivation, effects of pedagogical approach and sex, and association with academic achievement (Research Questions 2–4). For the 204 students, 61.3% were females; 51.5% were White, 17.2% were Hispanic or Latino, 7.8% were Black or African American, and 15.2% were Asian; 10.3% were sophomores, 43.1% were juniors, and 44.6% were seniors; 55.4% of the students were biomedical science or health related majors, 26.0% were biology or biology related majors, and only 4% were chemistry or chemistry engineering majors; the average SAT math and verbal scores for this class were 592.9 and 573.4. The demographic information and SAT backgrounds for students who were enrolled in the course (n = 257) are displayed in Appendices 1 and 2. The overall pass rate (C or above) in this course was 73.5%.

In 2015, there were 240 students enrolled in the course, coded as “Flip–PLTL”. The pre-semester data (Pre2) was collected during the fourth week of class after Exam 1. The post data (Post2) was collected during the 14th week after Exam 4. Nine students had incomplete responses and one student had no identifier, which resulted in 217 students’ responses for Pre2 with a response rate of 90%. In the Post2 dataset, all 190 students who responded to the survey had complete responses for data analysis, which resulted a response rate of 79%. Pre2 and Post2 datasets were used to answer Research Question 1. A matched-pair sample (n = 166), with compete responses for the two administration of AMS-Chemistry in the Flip–PLTL course, was used to answer Research Questions 2–4. For the 166 students, 63.9% were females; 44% were Whites, 22.3% were Hispanic or Latino, 6.6% were Black or African American, and 20.5% were Asian; 9.6% were sophomore, 37.3% were juniors, and 53% were seniors; 60.8% of the students were biomedical science or health related majors, 22.2% were biology-related majors, and only 3% were chemistry or chemistry engineering majors; the average SAT math and verbal scores were 601.9 and 585.1. The demographic information and SAT backgrounds for students who were enrolled in the course are displayed in Appendices 1 and 2. The overall pass rate (C or above) in this course was 95.4%.

Data analysis

Collected data for this study were analyzed using different statistical analyses. First, the scores of the AMS-Chemistry from Pre1, Post1, Pre2, and Post2 were analyzed to evaluate the internal structure validity of the instrument through confirmatory factor analysis (CFA) in Mplus 5.2. A Comparative Fit Index (CFI) greater than 0.90 is considered as an acceptable fit (Cheng and Chan, 2003). A Root Mean Square Error of Approximation (RMSEA) smaller than 0.08 is considered as a reasonable fit (Browne and Cudeck, 1992; MacCallum et al., 1996). A standardized root mean squared residual (SRMR) smaller than 0.10 is considered as an acceptable fit to the data (Hu and Bentler, 1995). In summary, we used the following cut-off values as an evaluation of a reasonable model fit beyond the chi-square test statistic: RMSEA < 0.08, SRMR < 0.10, CFI > 0.90.

The internal consistency of the seven subscales was assessed using Cronbach's alpha coefficients through SPSS 22.0; a benchmark of 0.7 is suggested for research purposes (Murphy and Davidshofer, 2005). SPSS 22.0 was also used for descriptive statistics of the items and subscales, correlation studies, and multivariate analysis of covariance. To compare whether the correlation coefficients were significantly different in the two courses, two-tailed Z-tests for independent correlations coefficients were conducted (Glass and Hopkins, 1970) while using Bonferroni procedures to control for the family-wise type-1 errors (Holm, 1979). A factorial multivariate analysis of covariance (MANCOVA) (Stevens, 2002) at an alpha level of 0.05 and post hoc comparison (Bonferroni approach) (Holm, 1979) at an alpha level of 0.007 (0.05/7) were conducted to examine the main effects of sex and pedagogical approach as well as the interaction effect on the seven types of motivation at the end of the semester, with pre-test motivation scores as covariates to eliminate the effect of any existing pre-test differences on the results, in particular because data were collected at different times at the beginning of semester.

Results and discussion

Internal structure validity and internal consistency reliability

To answer the first research question, how does AMS-Chemistry function in two differently taught organic chemistry courses, the internal structure validity and internal consistency reliability of AMS-Chemistry were examined to evaluate how the instrument functioned in the two courses. Most AMS-Chemistry items were normally distributed with skewness and kurtosis between −2 and +2; therefore, a maximum-likelihood method of estimate is employed for the confirmatory factor analysis (CFA). The sample size at each time point was within the range of 190–235 and therefore appropriate for CFA (Brown, 2006). A seven-factor model was examined. Fit indices for each administration are listed in Table 2; CFI values were between 0.91 and 0.93, RMSEA values were between 0.068 and 0.077, and SRMR values were between 0.060 and 0.064. Similar to the findings in a general chemistry course (Liu et al., 2017), the seven-factor model, rooted in SDT, showed reasonable fit to the data for each of the four data collections.

Table 2 Confirmatory factor analysis fit indices of the seven-factor model in two organic chemistry courses

Dataset	n	χ ^2	Df	p-Value	CFI	RMSEA	SRMR
Pre1	235	709.79	329	<0.001	0.91	0.070	0.061
Post1	224	672.27	329	<0.001	0.93	0.068	0.062
Pre2	217	657.15	329	<0.001	0.93	0.068	0.064
Post2	190	695.74	329	<0.001	0.92	0.077	0.060

---

Evidence of the internal consistency of the AMS-Chemistry was examined by using Cronbach's alpha coefficients. As shown in Table 3, the Cronbach's alpha coefficients were between 0.72 (identified regulation) and 0.90 (to accomplish) for the Pre1 dataset and between 0.78 (identified regulation) and 0.92 (to accomplish) for the Post1 dataset. The coefficients ranged from 0.78 (identified regulation) to 0.93 (to accomplish) for the Pre2 dataset. The alpha coefficients were from 0.84 (identified regulation) to 0.93 (to accomplish) for Post2 dataset. The results were similar to those in the general chemistry course (Liu et al., 2017), and showed acceptable internal consistency (Murphy and Davidshofer, 2005).

Table 3 Cronbach's alpha coefficients for the four datasets in two organic chemistry courses

Subscale	Pre1 n = 235	Post1 n = 224	Pre2 n = 217	Post 2 n = 190
Amotivation	0.77	0.87	0.85	0.86
External regulation	0.87	0.86	0.89	0.89
Introjected regulation	0.88	0.88	0.89	0.91
Identified regulation	0.72	0.78	0.78	0.84
To experience	0.87	0.86	0.84	0.89
To accomplish	0.90	0.92	0.93	0.93
To know	0.85	0.88	0.87	0.90

---

The CFA results together with Cronbach's alpha coefficients suggested that motivation can legitimately be explored by calculating the means for each subscale of the AMS-Chemistry instrument.

Motivation status

To answer the second research question, what is the student motivation status at the beginning and end of a semester in organic chemistry courses, the means and standard deviations of students’ motivation subscale scores were presented. Response scales range from 1 “not at all” to 5 “exactly”. A higher score for extrinsic motivation subscales means that students are more extrinsically motivated to take this organic chemistry course. The same goes for the intrinsic motivation subscales. A higher score for amotivation would mean that students are more lack of any motivation. Descriptive statistics for students with complete responses at the beginning and end of two organic chemistry courses are displayed in Appendix 3. Subscales were approximately normally distributed, with skewness and kurtosis in the range of ±1, except for amotivation subscale. While the descriptive statistics in Appendix 3 were like those who have responded to AMS-Chemistry twice over a semester (Table 4), our focus herein is only on the matched dataset in both courses.

Table 4 Mean and standard deviations of motivation scores and the changes over a semester in two organic chemistry courses

Course	Subscale	Pre M (SD)	Post M (SD)	M difference (effect size)
Lecture-based (n = 204)	Amotivation	1.41 (0.63)	1.85 (1.00)	0.44 (0.53)
	External regulation	3.77 (0.97)	3.74 (0.97)	−0.03 (0.03)
	Introjected regulation	3.48 (0.99)	3.14 (1.08)	−0.34 (0.33)
	Identified regulation	4.09 (0.73)	3.79 (0.89)	−0.29 (0.36)
	To experience	2.67 (1.10)	2.52 (1.10)	−0.15 (0.14)
	To accomplish	3.19 (1.06)	2.97 (1.13)	−0.21(0.20)
	To know	3.34 (0.99)	3.11 (1.03)	−0.23 (0.23)
Flip–PLTL (n = 166)	Amotivation	1.36 (0.66)	1.56 (0.80)	0.20 (0.27)
	External regulation	3.62 (1.05)	3.46 (1.03)	−0.15 (0.14)
	Introjected regulation	3.44 (1.01)	3.25 (1.04)	−0.19 (0.19)
	Identified regulation	3.86 (0.87)	3.64 (0.93)	−0.22 (0.24)
	To experience	2.59 (0.91)	2.61 (1.01)	0.02 (0.02)
	To accomplish	3.13 (1.05)	2.96 (1.04)	−0.17 (0.16)
	To know	3.26 (0.94)	3.07 (0.99)	−0.19 (0.20)

---

In both courses, students scored higher on extrinsic motivation subscales. For example, at the beginning of the semester in the Lecture-Based course, the means on extrinsic motivation subscales ranged from 3.48 (introjected regulation) to 4.09 (identified regulation), while the intrinsic motivation subscales ranged from 2.67 (to experience) to 3.34 (to know). In both courses, students scored higher on amotivation and lower on extrinsic and intrinsic motivation subscales at the end of the semester. For example, in the Lecture-Based course, students scored 1.41 on amotivation at the beginning of the semester, while at the end of the semester in the same course, the mean on amotivation increased to 1.85 and the extrinsic motivation subscales ranged from 3.14 (introjected regulation) to 3.79 (identified regulation), and the intrinsic motivation subscales were from 2.52 (to experience) to 3.11 (to know).

Higher scores on extrinsic motivation subscales suggest that students were more extrinsically motivated in both the Lecture-Based and Flip–PLTL courses. In particular, students were motivated because they may have identified the value of organic chemistry to their future careers, suggested by the highest mean on identified regulation subscale. We speculate the followings are the possible reasons that students were more extrinsically motivated in the two courses: Most students were medical science majors and no one in the course took the course as elective; the students took the course for entrance graduate school and profession studies, and the students needed to get “A” to be competitive. Students’ higher scores on extrinsic motivation subscales were consistent with the speculation that students may have taken a consumerist approach to higher education (Labaree, 1999). Furthermore, according to SDT, when most tasks are not inherently interesting and satisfying and that learning requires a lot of repetitive practices, students need some extrinsic stimulus to perform learning activities (Ryan and Deci, 2000). Comparable results were found in general chemistry (Liu et al., 2017). As students were more extrinsically motivated, researchers have speculated that instructors can use course policy to influence students’ extrinsic motivation, e.g. through policy regarding attendance, in-class assignments, and other activities, but it is hard to influence students’ intrinsic motivation (Maurer et al., 2012, 2013; Sturges et al., 2016).

When closely examining the motivation scores in each course, we found that the magnitude of changes on motivational scores was different over the course of a semester. In the Lecture-Based course, amotivation scores increased by 0.44 on average with a medium effect size (Cohen's d: small = 0.20, medium = 0.50, large = 0.80) (Cohen, 1988). This suggests that motivation levels decreased over a semester. All other motivation scores decreased with decreases ranging from 0.03 (external regulation) to 0.34 (introjected regulation) with small to medium effect sizes (d = 0.03 to 0.36). This suggests that students were less motivated by external reasons and internal satisfactions over a course of a semester. In the Flip–PLTL course, a similar trend was noticed, but the effect sizes were smaller (d = 0.02 to 0.27). The decline in motivation was consistent with prior literature that motivation changed over time and decreases in motivation occurred in particular contexts (Zusho et al., 2003; Nilsson and Warrén Stomberg, 2008; Brouse et al., 2010; He et al., 2015; Sturges et al., 2016; Liu et al., 2017). There are many possible reasons that students were demotivated. For example, Nilsson and Warrén Stomberg (2008) found that unstimulating curriculum, negative attitudes towards the studies, bad life situations, and difficulties with concepts were among the reasons for the decrease in motivation. Under-studied factors, such as fatigue (Smets et al., 1995) and exhaustion (Karatepe and Tekinkus, 2006), may be another possible reason that students were demotivated over time.

Sex and course effects

To answer the third research question, how do changes on motivation differ by pedagogical approaches and by sex, a factorial MANCOVA was conducted (Stevens, 2002; Nakajima and Freesemann, 2013). Motivation has been found to relate with math ability (Ablard and Lipschultz, 1998; Leaper et al., 2012). In addition, math ability is important in organic chemistry because the subject of organic chemistry requires diverse skills, including mathematical skills (Carpenter and McMillan, 2003) and one main type of the problems in organic chemistry with regard to content is mathematical problems (Graulich, 2015). We, therefore, first examined if students differed on their math ability (i.e., SAT-Math score) in the two courses and if SAT-Math was an appropriate covariate for a factorial MANCOVA. We found that students’ SAT-Math differed by 9.1 points in the two courses for students in the matched datasets. Two one-tailed independent t-tests (Schuirmann, 1987; Lewis and Lewis, 2005b) showed that the two courses were not equivalent based on SAT-Math: t₁ = 2.26 > 1.29, t₂ = 0.62 < 1.29 at an alpha level of 0.10. However, assumption tests showed that there was no evidence for a significant relationship between SAT-Math and the dependent variables: Λ = 0.977, F(7,285) = 0.948, p = 0.47. Therefore, SAT-Math was excluded as a covariate for the factorial MANCOVA.

The purpose of the factorial MANCOVA test was to examine the effects of sex and pedagogical approach on student motivation post-test scores, with pre-test motivation scores as covariates to eliminate the effect of any existing pre-test differences on the results. In this analysis, post scores of the seven motivation subscales were the dependent variables. Sex and pedagogical approach were the independent variables, and the pre-test scores of the seven types of motivation were the covariates. The interaction effect of sex and pedagogical approach was also examined. Results (see Table 5) did not provide evidence for a significant main effect of sex or an interaction effect for sex and pedagogical approach, suggesting females and males scored similarly at the end of the semester in the Lecture-Based and Flip–PLTL courses, while controlling for the pre-test motivation scores. Our results were different from prior literature, which found that female college students tended to have higher levels of motivation than males in various contexts (Vallerand et al., 1992; Cokley et al., 2001; Spittle et al., 2009; Brouse et al., 2010; Smith et al., 2010; Eymur and Geban, 2011). The results also differed from previous findings in which females reported being less motivated toward chemistry than males in a first-semester general chemistry course (Liu et al., 2017). Due to our easily accessible convenience sample in a single institution, these results regarding a lack of sex differences in self-reported motivation levels should be interpreted with caution when comparing with the literature.

Table 5 Factorial MANCOVA results for main and interaction effects of sex and pedagogical approach on student motivation

Effect	Λ	F(7,353)	p	Partial η^2
Sex	0.975	1.291	0.254	0.025
Pedagogical approach	0.928	3.906	<0.001	0.072
Sex × pedagogical approach	0.985	0.792	0.594	0.015

---

The factorial MANCOVA indicated a significant main effect of pedagogical approach after adjusting for the covariates: Λ = 0.928, F(7,353) = 3.906, p < 0.001. This suggests that the students scored significantly differently on the set of motivation variables at the end of the semester taught by different pedagogical approaches, while controlling for motivation pre-test scores. Based on the multivariate findings, univariate analyses of variance were done for each motivation subscale. Table 6 summarizes the significance levels for each of the variables. Results showed that students scored similarly on all the intrinsic and extrinsic motivation subscales, but students taught with the Flip–PLTL pedagogical approach scored significantly lower on amotivation at an alpha level of 0.007 while controlling type-1 error (Holm, 1979). Students in the two courses were similar based on their demographics (e.g. sex), when they enrolled in the course they knew which professor was going to teach but they were not aware of pedagogical changes; therefore, we speculate that the differences on student motivation could be due to the different pedagogical approaches. This suggests that different pedagogical approaches can differently affect student motivation (Ryan and Deci, 2000). Perhaps only one semester is insufficient for students to show gains in intrinsic motivation or differences in extrinsic motivation. Self-determination theory suggests that if basic needs of autonomy, competence, and relatedness are met, students can develop intrinsic motivation (Deci and Ryan, 2000; Ryan and Deci, 2000). We speculate that in a Flip–PLTL classroom, students have freedom regarding when to and how often to explore the study materials, students have more chance to interact with their peers, and the instructor and peer-leaders function more like facilitators in class; therefore, like other active learning pedagogies (Black and Deci, 2000; Prince, 2004; Obenland et al., 2013; Abeysekera and Dawson, 2015; Cicuto and Torres, 2016), Flip–PLTL pedagogies can have a positive effect on student motivation given long enough time according to theory (Deci and Ryan, 2000; Ryan and Deci, 2000; Abeysekera and Dawson, 2015).

Table 6 Univariate analysis of main effect of pedagogical approach on seven motivation subscales of AMS-Chemistry

Dependent variables	F(1,359)	p	Partial η^2
Amotivation	11.735	0.001	0.032
External regulation	4.884	0.028	0.013
Introjected regulation	3.81	0.052	0.011
Identified regulation	0.06	0.806	<0.001
To experience	3.195	0.075	0.009
To accomplish	0.564	0.453	0.002
To know	0.085	0.771	<0.001

---

Motivation and academic achievement

To answer the fourth research question, how is motivation associated with student academic achievement in the organic chemistry courses, the associations between motivation and academic achievement were studied in the two courses. Four academic achievement measures were used: Exam 1, Exam 4, Final Exam, and Final Score (the final course percentages). Matched samples were used for the correlation study in order to compare the associations in the two courses. Intrinsic and extrinsic motivation subscales had no significant correlations with Exam 1; however, amotivation was negatively correlated with Exam 1 with a small effect size (r: small = 0.1, medium = 0.3, large = 0.5) (Cohen, 1988) at the beginning of both courses (Tables 7 and 8).

Table 7 Association of motivation with academic achievement in the Lecture-Based course (n = 204)

Subscales	Exam 1^a	Final exam^a	Final score^a	Exam 4^b	Final exam^b	Final score^a
* Significant at 0.05 level (two-tailed test). ** Significant at 0.01 level (two-tailed test).a With pre-motivation scores.b With post-motivation scales.
Amotivation	−0.15*	−0.11	−0.13	−0.35**	−0.33**	−0.38**
External regulation	−0.01	0.00	0.02	0.00	−0.02	0.04
Introjected regulation	−0.08	0.05	0.02	0.13	0.16*	0.18**
Identified regulation	−0.06	0.05	0.04	0.22**	0.19**	0.24**
To experience	0.06	0.18**	0.18*	0.25**	0.24**	0.29**
To accomplish	0.08	0.20**	0.17*	0.26**	0.27**	0.31**
To know	0.06	0.17*	0.17*	0.28**	0.30*	0.32**

---

Table 8 Association of motivation with academic achievement in the Flip–PLTL course (n = 166)

Subscales	Exam 1^a	Final exam^a	Final score^a	Exam 4^b	Final exam^b	Final score^b
* Significant at 0.05 level (two-tailed test). ** Significant at 0.01 level (two-tailed test).a With pre-motivation scores.b With post-motivation scales.
Amotivation	−0.20*	−0.20**	−0.28**	−0.19*	−0.21**	−0.23**
External regulation	−0.03	−0.01	−0.01	−0.07	−0.03	−0.06
Introjected regulation	0.03	0.07	0.02	0.12	0.09	0.06
Identified regulation	−0.03	−0.00	0.04	0.11	0.06	0.02
To experience	0.02	0.12	0.08	0.17*	0.15	0.13
To accomplish	0.12	0.20*	0.16*	0.18*	0.15	0.13
To know	0.08	0.19*	0.15	0.20**	0.17*	0.13

---

For the Lectured-Based course, at the beginning of the semester intrinsic motivation subscales were significantly and positively correlated with Final Exam and Final Score (see Table 7); correlations ranged from 0.17 to 0.20 with small effect sizes. At the end of the semester, intrinsic motivation subscales were significantly and positively correlated with Exam 4, Final Exam, and Final Score: r = 0.24–0.32, with medium effect sizes; introjected regulation and identified regulation also had positive correlations with the exam grades, with r values ranging from 0.16 to 0.24.

For the Flip–PLTL course, amotivation was significantly and negatively correlated with exam grades (see Table 8). At the beginning of the semester, both to know and to accomplish subscales were significantly and positively correlated with Final Exam; the to accomplish subscale was significantly and positively correlated with Final Score. At the end of the semester, to know and to accomplish subscales were significantly and positively correlated with Exam 4; only the to know subscale was significantly and positively correlated with Final Exam. There was no evidence for significant correlations between extrinsic motivation subscales and academic achievement throughout the semester. Two-tailed Z-tests of independent correlation coefficients (Glass and Hopkins, 1970) did not provide evidence for differences in correlation coefficients between the two courses (p > 0.05 for each of the two-tailed Z-tests), suggesting the association trend between motivation and academic achievement was similar in the two courses.

The variable correlations over time were consistent with the findings in general chemistry course (Liu et al., 2017). Negative correlations between amotivation and academic achievement suggested that it would be beneficial to stimulate all forms of self-determined motivation toward chemistry. Extrinsic motivation had no significant correlations with academic achievement in the Flip–PLTL course, suggesting that while providing some minor rewards could help student extrinsic motivation, merely relying on extrinsic rewards may not be enough for students to achieve. This was consistent with a hypothesis that students had difficulty keeping up studies if they were primarily concerned with grades (Lynch and Trujillo, 2011), a key indicator of high extrinsic motivation. Positive correlations between intrinsic motivation subscales and students’ exam grades were consistent with prior findings in the literature (Black and Deci, 2000; Jurisevic et al., 2008; Devetak et al., 2009; Cicuto and Torres, 2016; Liu et al., 2017), which suggest that students who were more intrinsically motivated toward chemistry had better achievement (Lynch and Trujillo, 2011). Intrinsic motivation had a stronger association with student academic achievement. Therefore, we suggest instructors to take efforts to promote student intrinsic motivation. For example, instructors can provide choices to students and rationales for assignments. Instructors can also acknowledge students’ feelings and give them a sense of independence. Moreover, instructors can provide students’ positive feedback to help them be confident and grow in their abilities, as well as help them to have a sense of belonging by encouraging collaborations in small groups. According to SDT, when students’ basic needs of autonomy, competence, and relatedness are met, intrinsic motivation development is supported (Deci and Ryan, 2000; Ryan and Deci, 2000).

Conclusion

The purpose of the present study is to examine the effect of pedagogical approaches by using AMS-Chemistry to evaluate motivational perceptions before and after the implementation of flipped classroom and PLTL (Flip–PLTL). To achievement this purpose, we first gathered psychometric evidence and suggested that the AMS-Chemistry can be used to measure motivation in the target organic chemistry courses that utilized different pedagogies. AMS-Chemistry scores were used to evaluate the effectiveness of Flip–PLTL on student motivation; results suggest that AMS-Chemistry has potential to evaluate the impact of other research-based instructional pedagogies on motivation. Results suggest that students are less amotivated in a Flip–PLTL instructional environment; we conclude that the reformed pedagogy is having the intended effect on motivation. Additional, we observe that one semester is not long enough to observe meaningful positive changes in the extrinsic and intrinsic subscales of the AMS-Chemistry; a longitudinal study over multiple semesters would lend evidence to support claim that the AMS-Chemistry data are invariant across time and can be used to articulate changes across time.

Limitations

This study has several limitations, which suggest that other researchers should be cautious when comparing our results with findings in other contexts. First, the samples were conveniently taken from the two target courses and from courses at a single institution. The inferential results may be due to the particular sample and unreflective of causal associations between changes in motivation and instructional environments. Second, the sample size was too small for measurement invariance testing (Xu et al., 2016) and limited the power of the statistical analysis reported. Third, results were primarily based on quantitative data. Fourth, there was no data to show the fidelity of the implementation of PLTL and flipped classroom. In the future, qualitative studies should be conducted to triangulate these findings. For example, interviews could uncover which aspects of flipped classroom and PLTL motivate students toward chemistry and how the curriculum, fatigue, and exhaustion may have influenced motivation.

Implications

Our findings have multiple implications for the chemistry education community. Our work found some positive effect of Flip–PLTL on student motivation; therefore, we encourage faculty to implement such a pedagogical approach to their instructional practices. According to SDT, when instructors support the autonomy of students, students develop more intrinsic motivation and identified regulation (Deci and Ryan, 2000; Ryan and Deci, 2000). For example, autonomy-supportive interventions could be used to help faculty develop this instructional technique (Su and Reeve, 2011). Reeve (2009) has suggested instructors to use informative and permissive language while communicating with students, to acknowledge and accept students’ feelings; such communication techniques enhance the support of autonomy. Instructors should provide students with optimally challenging tasks and tools necessary for success, resources for self-paced learning, and positive feedback, be respectful, and create opportunities for students to interact with each other (Kusurkar et al., 2011; Su and Reeve, 2011; Jang et al., 2016a, 2016b); these behaviors also meet students basic needs of autonomy, competence, and relatedness, therefore support the development of intrinsic motivation according to SDT (Deci and Ryan, 2000; Ryan and Deci, 2000; Orsini et al., 2015). When students are motivated, more positive outcomes are expected, for example, better academic achievement and persistence in STEM fields (PCAST, 2012).

Scores from AMS-Chemistry can be interpreted using different frameworks based on different mini-theories in SDT. First, using the seven-factor model, we can show nuances of motivation and potential movement along the self-determination continuum based on the organismic integration theory. Second, with the support of alternative models, the data collected from AMS-Chemistry can also be interpreted in terms of autonomous and controlled motivation using causality orientations theory, another mini-theory in SDT (Deci and Ryan, 2000, 2008; Ryan and Deci, 2000; Jurisevic et al., 2008; Guay et al., 2010; Kusurkar et al., 2013), which can be used to produce simplified student motivation profiles and enhance our understanding of SDT in college chemistry courses.

Conflicts of interest

There are no conflicts to declare.

Appendices

Appendix 1: the demographics of students enrolled in the courses

	“Lecture-Based”	Flip–PLTL
n	257	240
Females	158 (61.5%)	152 (63.3%)
Males	99 (38.5%)	88 (36.7%)
White	123 (47.9%)	109 (45.4%)
Hispanic or Latino	44 (17.1%)	54 (22.5%)
Asian	40 (15.6%)	47 (19.6%)
Black or African American	27 (10.5%)	14 (5.8%)
Senior	117 (45.5%)	120 (50%)
Junior	110 (42.8%)	90 (37.5%)
Sophomore	26 (10.1%)	29 (12.1%)
Post Bachelor	3 (1.2%)	1 (0.4%)
Freshman	1 (0.4%)	0
Biomedical Sciences	105 (40.9%)	125 (52.1%)
Cell and Molecular Biology	25 (9.7%)	22 (9.2%)
Integrative Animal Biology	18 (7.0%)	18 (7.5%)
Health Sciences	32 (12.5%)	16 (6.7%)
Chemistry/Chemical Engineering	11 (4.3%)	10 (4.2%)

Appendix 2: examination of the academic background of all the students enrolled in two courses

	Lecture-Based		Flip–PLTL
	SAT_Q	SAT-V	SAT_Q	SAT-V
n	208	208	192	192
M	590.0	570.8	607.2	587.8
SD	74.4	79.5	84.6	87.8

Appendix 3: descriptive statistics of student motivation based on all students’ responses

Data collection	Subscales	M	SD	Sk	Ku
Note: Sk = skewness, Ku = kurtosis.
Pre1 n = 235	Amotivation	1.47	0.67	1.73	3.10
	External regulation	3.79	0.96	−0.79	0.00
	Introjected regulation	3.53	0.99	−0.43	−0.25
	Identified regulation	4.05	0.75	−0.71	−0.19
	To experience	2.65	1.11	0.24	−0.94
	To accomplish	3.18	1.05	−0.07	−0.83
	To know	3.33	0.99	−0.11	−0.84
Post1 n = 224	Amotivation	1.85	1.00	1.19	0.63
	External regulation	3.74	0.98	−0.83	0.04
	Introjected regulation	3.15	1.10	−0.23	−0.92
	Identified regulation	3.78	0.90	−0.63	−0.28
	To experience	2.50	1.10	0.35	−0.83
	To accomplish	2.95	1.13	−0.14	−0.86
	To know	3.10	1.03	−0.08	−0.79
Pre2 n = 217	Amotivation	1.34	0.62	2.53	7.78
	External regulation	3.63	1.03	−0.65	−0.28
	Introjected regulation	3.50	1.00	−0.4	−0.51
	Identified regulation	3.91	0.86	−0.82	0.54
	To experience	2.65	0.96	0.33	−0.43
	To accomplish	3.19	1.05	−0.06	−0.8
	To know	3.33	0.94	0.13	−0.64
Post2 n = 190	Amotivation	1.56	0.82	1.74	2.82
	External regulation	3.42	1.05	−0.51	−0.25
	Introjected regulation	3.25	1.05	−0.32	−0.45
	Identified regulation	3.61	0.95	−0.53	−0.27
	To experience	2.62	1.00	0.21	−0.67
	To accomplish	2.98	1.04	−0.16	−0.55
	To know	3.09	0.98	−0.16	−0.64

References

1. Abeysekera L. and Dawson P., (2015), Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research, High. Educ. Res. Dev., 34(1), 1–14.
2. Ablard K. E. and Lipschultz R. E., (1998), Self-regulated learning in high-achieving students: Relations to advanced reasoning, achievement goals, and gender, J. Educ. Psychol., 90(1), 94–101.
3. Apugliese A. and Lewis S. E., (2017), Impact of instructional decisions on the effectiveness of cooperative learning in chemistry through meta-analysis, Chem. Educ. Res. Pract., 18(1), 271–278.
4. Arjoon J. A., Xu X. and Lewis J. E., (2013), Understanding the state of the art for measurement in chemistry education research: Examining the psychometric evidence, J. Chem. Educ., 90(5), 536–545.
5. Arrey L. N., (2012), Organic chemistry: Intensive format or traditional format, Summer Academe: A Journal of Higher Education, 6, 37–45.
6. Bhattacharyya G., (2014), Trials and tribulations: student approaches and difficulties with proposing mechanisms using the electron-pushing formalism, Chem. Educ. Res. Pract., 15(4), 594–609.
7. Bhattacharyya G. and Bodner G. M., (2005), “It gets me to the product”: How students propose organic mechanisms, J. Chem. Educ., 82(9), 1402–1407.
8. Black A. E. and Deci E. L., (2000), The effects of instructors' autonomy support and students' autonomous motivation on learning organic chemistry: A self-determination theory perspective, Sci. Educ., 84(6), 740–756.
9. Bodner, G. M., Ferguson, R., and Çalimsiz, S., (2017), Doing the research that informs practice: A retrospective view of one group's attempt to study the teaching and learning of organic chemistry, Chem. – Asian J., 12(13), 1413–1420.
10. Brandriet A. R., Xu X., Bretz S. L. and Lewis J. E., (2011), Diagnosing changes in attitude in first-year college chemistry students with a shortened version of Bauer's semantic differential, Chem. Educ. Res. Pract., 12(2), 271–278.
11. Brouse C. H., Basch C. E., LeBlanc M., McKnight K. R. and Lei T., (2010), College students' academic motivation: Differences by gender, class, and source of payment, Coll. Q., 13(1), n1.
12. Brown, T. A., (2006), Confirmatory factor analysis for applied research, 1st edn, Guilford Publications, Inc.
13. Browne M. W. and Cudeck R., (1992), Alternative ways of assessing model fit, Sociol. Methods Res., 21(2), 230–258.
14. Carlson K., Turvold Celotta D., Curran E., Marcus M. and Loe M., (2016), Assessing the impact of a multi-disciplinary peer-led-team learning program on undergraduate STEM education, Teach. Learn. Pract., 13(1), 5.
15. Carpenter, S., and McMillan, T., (2003), Incorporation of a cooperative learning technique in organic chemistry, J. Chem. Educ., 80(3), 330–332.
16. Cerasoli C. P., Nicklin J. M. and Ford M. T., (2014), Intrinsic motivation and extrinsic incentives jointly predict performance: A 40 year meta-analysis, Psychol. Bull., 140(4), 980–1008.
17. Chan J. Y. K. and Bauer C. F., (2015), Effect of peer-led team learning (PLTL) on student achievement, attitude, and self-concept in college general chemistry in randomized and quasi experimental designs, J. Res. Sci. Teach., 52(3), 319–346.
18. Cheng S. T. and Chan A. C. M., (2003), The development of a brief measure of school attitude, Educ. Psychol. Meas., 63(6), 1060–1070.
19. Chirkov V. and Ryan R. M., (2001), Parent and teacher autonomy support in Russian and U.S. adolescents: Common effects on well-being and academic motivation, J. Cross. Cult. Psychol., 32(5), 618–635.
20. Chonkaew P., Sukhummek B. and Faikhamta C., (2016), Development of analytical thinking ability and attitudes towards science learning of grade-11 students through science technology engineering and mathematics (STEM education) in the study of stoichiometry, Chem. Educ. Res. Pract., 17(4), 842–861.
21. Cicuto C. A. T. and Torres B. B., (2016), Implementing an active learning environment to influence students’ motivation in biochemistry, J. Chem. Educ., 93(6), 1020–1026.
22. Cohen J., (1988), Statistical power analysis for the behavioral sciences, Hillsdale, NJ: Erlbaum.
23. Cokley K. O., Bernard N., Cunningham D. and Motoike J., (2001), A psychometric investigation of the Academic Motivation Scale using a United States sample, Meas. Eval. Couns. Dev., 34(2), 109–119.
24. Deci E. L. and Ryan R. M., (2000), The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior, Psychol. Inq., 11(4), 227–268.
25. Deci E. L. and Ryan R. M., (2008), Self-determination theory: A macrotheory of human motivation, development, and health, Can. Psychol., 49(3), 182–185.
26. Devetak I., Lorber E. D., Jurisevic M. and Glazar S. A., (2009), Comparing Slovenian year 8 and year 9 elementary school pupils' knowledge of electrolyte chemistry and their intrinsic motivation, Chem. Educ. Res. Pract., 10(4), 281–290.
27. Drane D., Micari M. and Light G., (2014), Students as teachers: Effectiveness of a peer-led STEM learning programme over 10 years, Educ. Res. Eval., 20(3), 210–230.
28. Eichler J. F. and Peeples J., (2016), Flipped classroom modules for large enrollment general chemistry courses: A low barrier approach to increase active learning and improve student grades, Chem. Educ. Res. Pract., 17(1), 197–208.
29. Eymur G. and Geban O., (2011), An investigation of the relationship between motivation and academic achievement of pre-service chemistry teachers, Egit. Bilim., 36(161), 246–255.
30. Fautch J. M., (2015), The flipped classroom for teaching organic chemistry in small classes: Is it effective? Chem. Educ. Res. Pract., 16(1), 179–186.
31. Flynn A. B., (2015), Structure and evaluation of flipped chemistry courses: Organic & spectroscopy, large and small, first to third year, English and French, Chem. Educ. Res. Pract., 16(2), 198–211.
32. Freeman S., Eddy S. L., McDonough M., Smith M. K., Okoroafor N., Jordt H. and Wenderoth M. P., (2014), Active learning increases student performance in science, engineering, and mathematics, Proc. Natl. Acad. Sci. U. S. A., 111(23), 8410–8415.
33. Gafney L. and Varma-Nelson P., (2008), PLTL and the goals of higher education, in Peer-led team learning: Evaluation, dissemination, and institutionalization of a college level initiative, Dordrecht: Springer Netherlands, pp. 97–102.
34. Glass G. V. and Hopkins K. D., (1970), Statistical methods in education and psychology, Englewood Cliffs, NJ: Prentice-Hall.
35. Gosser D. K. and Roth V., (1998), The workshop chemistry project: Peer-led team-learning, J. Chem. Educ., 75(2), 185–187.
36. Graulich N., (2015), The tip of the iceberg in organic chemistry classes: how do students deal with the invisible? Chem. Educ. Res. Pract., 16(1), 9–21.
37. Gregorius R. M., (2017), Performance of underprepared students in traditional versus animation-based flipped-classroom settings, Chem. Educ. Res. Pract., 2017, 18(4), 841–848.
38. Griffin R., MacKewn A., Moser E. and VanVuren K., (2013), Learning skills and motivation: Correlates to superior academic performance, Bus. Educ. Accredit., 5(1), 53–65.
39. Guay F., Ratelle C. F., Roy A. and Litalien D., (2010), Academic self-concept, autonomous academic motivation, and academic achievement: Mediating and additive effects, Learn. Individ. Differ., 20(6), 644–653.
40. He W., Gajski D., Farkas G. and Warschauer M., (2015), Implementing flexible hybrid instruction in an electrical engineering course: The best of three worlds? Comput. Educ., 81, 59–68.
41. Henry R. M., (2017), Engaging participation and promoting active learning through student usage of the internet to create notes for general chemistry in class, J. Chem. Educ., 94(6), 710–716.
42. Hibbard L., Sung S. and Wells B., (2016), Examining the effectiveness of a semi-self-paced flipped learning format in a college general chemistry sequence, J. Chem. Educ., 93(1), 24–30.
43. Hockings S. C., DeAngelis K. J. and Frey R. F., (2008), Peer-led team learning in general chemistry: Implementation and evaluation, J. Chem. Educ., 85(7), 990–996.
44. Holm S., (1979), A simple sequentially rejective multiple test procedure, Scand. J. Stat., 6(2), 65–70.
45. Hu L. and Bentler P. M., (1995), Evaluating model fit, London: Sage.
46. Jang H., Kim E. J. and Reeve J., (2016a), Why students become more engaged or more disengaged during the semester: A self-determination theory dual-process model, Learn. Instr., 43, 27–38.
47. Jang H., Reeve J. and Halusic M., (2016b), A new autonomy-supportive way of teaching that increases conceptual learning: Teaching in students' preferred ways, J. Exp. Educ., 84(4), 686–701.
48. Jensen J. L., Kummer T. A. and Godoy P. D. d. M., (2015), Improvements from a flipped classroom may simply be the fruits of active learning, CBE Life Sci. Educ., 14(1), 1–12.
49. Jurisevic M., Glazar S. A., Pucko C. R. and Devetak I., (2008), Intrinsic motivation of pre-service primary school teachers for learning chemistry in relation to their academic achievement, Int. J. Sci. Educ., 30(1), 87–107.
50. Jurisevic M., Vrtacnik M., Kwiatkowski M. and Gros N., (2012), The interplay of students' motivational orientations, their chemistry achievements and their perception of learning within the hands-on approach to visible spectrometry, Chem. Educ. Res. Pract., 13(3), 237–247.
51. Karatepe O. M. and Tekinkus M., (2006), The effects of work-family conflict, emotional exhaustion, and intrinsic motivation on job outcomes of front-line employees, Int. J. Bank Mark., 24(3), 173–193.
52. Kusurkar R. A., Croiset G. and Ten Cate O. T. J., (2011), Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory, Med. Teach., 33(12), 978–982.
53. Kusurkar R. A., Ten Cate T. J., Vos C. M. P., Westers P. and Croiset G., (2013), How motivation affects academic performance: A structural equation modelling analysis, Adv. Health. Sci. Educ., 18(1), 57–69.
54. Labaree D. F., (1999), How to succeed in school without really learning: The credentials race in American education, Yale University Press.
55. Leaper C., Farkas T. and Brown C. S., (2012), Adolescent girls’ experiences and gender-related beliefs in relation to their motivation in math/science and English, J. Youth Adolesc., 41(3), 268–282.
56. Lepper M. R. and Henderlong J., (2000), Turning “play” into “work” and “work” into “play”: 25 years of research on intrinsic versus extrinsic motivation, in Intrinsic and extrinsic motivation: The search for optimal motivation and performance, Harackiewicz C. S. J. M. (ed.), San Diego, CA, US: Academic Press, pp. 257–307.
57. Lewis S. E., (2011), Retention and reform: An evaluation of peer-led team learning, J. Chem. Educ., 88(6), 703–707.
58. Lewis S. E., (2014), Investigating the longitudinal impact of a successful reform in general chemistry on student enrollment and academic performance, J. Chem. Educ., 91(12), 2037–2044.
59. Lewis S. E. and Lewis J. E., (2005a), Departing from lectures: An evaluation of a peer-led guided inquiry alternative, J. Chem. Educ., 82(1), 135–139.
60. Lewis S. E. and Lewis J. E., (2005b), The same or not the same: Equivalence as an issue in educational research, J. Chem. Educ., 82(9), 1408–1412.
61. Lewis S. E. and Lewis J. E., (2008), Seeking effectiveness and equity in a large college chemistry course: An HLM investigation of peer-led guided inquiry, J. Res. Sci. Teach., 45(7), 794–811.
62. Liu Y., Ferrell B., Barbera J. and Lewis J. E., (2017), Development and evaluation of a chemistry-specific version of the academic motivation scale (AMS-Chemistry), Chem. Educ. Res. Pract., 18(1), 191–213.
63. Love B., Hodge A., Grandgenett N. and Swift A. W., (2013), Student learning and perceptions in a flipped linear algebra course, Int. J. Math. Educ. Sci. Technol., 45(3), 317–324.
64. Lyle K. S. and Robinson W. R., (2003), A statistical evaluation: Peer-led team learning in an organic chemistry course, J. Chem. Educ., 80(2), 132–134.
65. Lynch D. J. and Trujillo H., (2011), Motivational beliefs and learning strategies in organic chemistry, Int. J. Sci. Math. Educ., 9(6), 1351–1365.
66. MacCallum R. C., Browne M. W. and Sugawara H. M., (1996), Power analysis and determination of sample size for covariance structure modeling, Psychol. Methods, 1(2), 130–149.
67. Maurer T., Allen D., Gatch D. B., Shankar P. and Sturges D., (2012), Students’ academic motivations in allied health classes, Internet J. Allied Health Sci. Pract., 10(1), 6.
68. Maurer T. W., Allen D., Gatch D. B., Shankar P. and Sturges D., (2013), Students’ academic motivations in three disciplines, J. Scholarship Teach. Learn., 13(5), 77–89.
69. McAuley E., Duncan T. and Tammen V. V., (1989), Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: A confirmatory factor analysis, Res. Q. Exercise Sport, 60(1), 48–58.
70. Mitchell Y. D., Ippolito J. and Lewis S. E., (2012), Evaluating peer-led team learning across the two semester general chemistry sequence, Chem. Educ. Res. Pract., 13(3), 378–383.
71. Mooring S. R., Mitchell C. E. and Burrows N. L., (2016), Evaluation of a flipped, large-enrollment organic chemistry course on student attitude and achievement, J. Chem. Educ., 93(12), 1972–1983.
72. Morrison D. F., (1976), Multivariate statistical methods, p. 252.
73. Murphy K. R. and Davidshofer C. O., (2005), Psychological testing: Principles and applications, 6th edn, Upper Saddle River, NJ: Prentice Hall.
74. Nakajima M. A. and Freesemann K. W., (2013), Help-seeking behaviors among athletic training students in the clinical education setting: A pilot study, Athlet. Train. Educ. J., 8(4), 115–123.
75. Nilsson K. E. and Warrén Stomberg M. I., (2008), Nursing students motivation toward their studies – a survey study, BMC Nurs., 7(1), 1–7.
76. Obenland C. A., Munson A. H. and Hutchinson J. S., (2013), Silent and vocal students in a large active learning chemistry classroom: Comparison of performance and motivational factors, Chem. Educ. Res. Pract., 14(1), 73–80.
77. Ong M., Wright C., Espinosa L. L. and Orfield G., (2011), Inside the double bind: A synthesis of empirical research on undergraduate and graduate women of color in science, technology, engineering, and mathematics, Harvard Educ. Rev., 81(2), 172–208.
78. Orsini C., Evans P., Binnie V., Ledezma P. and Fuentes F., (2015), Encouraging intrinsic motivation in the clinical setting: Teachers' perspectives from the self-determination theory, Eur. J. Dent. Educ., 20, 102–111.
79. Osborne J., Simon S. and Collins S., (2003), Attitudes towards science: A review of the literature and its implications, Int. J. Sci. Educ., 25(9), 1049–1079.
80. Popejoy K. and Asala K. S., (2013), A team approach to successful learning: Peer learning coaches in chemistry, J. Coll. Sci. Teach., 42(3), 18–23.
81. President's Council of Advisors on Science and Technology PCAST, (2012), Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics, Report to the President, Washington, DC: Executive Office of the President.
82. Prince M., (2004), Does active learning work? A review of the research, J. Eng. Educ., 93(3), 223–231.
83. Reeve J., (2009), Why teachers adopt a controlling motivating style toward students and how they can become more autonomy supportive, Educ. Psychol., 44(3), 159–175.
84. Reeve J., (2012), A self-determination theory perspective on student engagement, in Handbook of research on student engagement, Christenson, S., Reschly, A. and Wylie, C., (ed.), New York: Springer Science + Business Media, pp. 149–172.
85. Reid S. A., (2016), A flipped classroom redesign in general chemistry, Chem. Educ. Res. Pract., 17(4), 914–922.
86. Reis H. T., Sheldon K. M., Gable S. L., Roscoe J. and Ryan R. M., (2000), Daily well-being: The role of autonomy, competence, and relatedness, Pers. Soc. Psychol., B, 26(4), 419–435.
87. Robert J., Lewis S. E., Oueini R. and Mapugay A., (2016), Coordinated implementation and evaluation of flipped classes and peer-led team learning in general chemistry, J. Chem. Educ., 93(12), 1993–1998.
88. Ryan M. D. and Reid S. A., (2016), Impact of the flipped classroom on student performance and retention: A parallel controlled study in general chemistry, J. Chem. Educ., 93(1), 13–23.
89. Ryan R. M. and Deci E. L., (2000), Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being, Am. Psychol., 55(1), 68–78.
90. Schuirmann D. J., (1987), A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability, J. Pharmacokinet. Biopharm., 15(6), 657–680.
91. Seery M. K., (2015), Flipped learning in higher education chemistry: Emerging trends and potential directions, Chem. Educ. Res. Pract., 16(4), 758–768.
92. Shattuck J. C., (2016), A parallel controlled study of the effectiveness of a partially flipped organic chemistry course on student performance, perceptions, and course completion, J. Chem. Educ., 93(12), 1984–1992.
93. Smets E. M. A., Garssen B., Bonke B. and De Haes J. C. J. M., (1995), The multidimensional fatigue inventory (MFI) psychometric qualities of an instrument to assess fatigue, J. Psychosom. Res., 39(3), 315–325.
94. Smith J. D., (2013), Student attitudes toward flipping the general chemistry classroom, Chem. Educ. Res. Pract., 14(4), 607–614.
95. Smith K. J., Davy J. A. and Rosenberg D. L., (2010), An examination of the validity of the Academic Motivation Scale with a United States business student sample, Psychol. Rep., 106(2), 323–341.
96. Spittle M., Jackson K. and Casey M., (2009), Applying self-determination theory to understand the motivation for becoming a physical education teacher, Teach. Teach. Educ., 25(1), 190–197.
97. Stevens J. P., (2002), Applied multivariate statistics for the social sciences, Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc.
98. Sturges D., Maurer T. W., Allen D., Gatch D. B. and Shankar P., (2016), Academic performance in human anatomy and physiology classes: A 2 year study of academic motivation and grade expectation, Adv. Physiol. Educ., 40(1), 26–31.
99. Su Y.-L. and Reeve J., (2011), A meta-analysis of the effectiveness of intervention programs designed to support autonomy, Educ. Psychol. Rev., 23(1), 159–188.
100. Taylor G., Jungert T., Mageau G. A., Schattke K., Dedic H., Rosenfield S. and Koestner R., (2014), A self-determination theory approach to predicting school achievement over time: The unique role of intrinsic motivation, Contemp. Educ. Psychol., 39(4), 342–358.
101. Tien L. T., Roth V. and Kampmeier J. A., (2002), Implementation of a peer-led team learning instructional approach in an undergraduate organic chemistry course, J. Res. Sci. Teach., 39(7), 606–632.
102. Vaino K., Holbrook J. and Rannikmäe M., (2012), Stimulating students' intrinsic motivation for learning chemistry through the use of context-based learning modules, Chem. Educ. Res. Pract., 13(4), 410–419.
103. Vallerand R. J., (1997), Toward a hierarchical model of intrinsic and extrinsic motivation, in Advances in experimental social psychology, Zanna M. P. (ed.), vol. 29, pp. 271–360.
104. Vallerand R. J., Pelletier L. G., Blais M. R., Briere N. M., Senecal C. and Vallieres E. F., (1992), The Academic Motivation Scale: A measure of intrinsic, extrinsic, and amotivation in education, Educ. Psychol. Meas., 52, 1003–1017.
105. Vallerand R. J. and Ratelle C. F., (2002), Intrinsic and extrinsic motivation: A hierarchical model, in Handbook of self-determination research, Deci E. L. and Ryan R. M. (ed.), Rochester: The Univerisity of Rochester Press, pp. 37–60.
106. Vansteenkiste M., Lens W. and Deci E. L., (2006), Intrinsic versus extrinsic goal contents in self-determination theory: Another look at the quality of academic motivation, Educ. Psychol., 41(1), 19–31.
107. Villafane S. M., Garcia C. A. and Lewis J. E., (2014), Exploring diverse students' trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry, Chem. Educ. Res. Pract., 15(2), 114–127.
108. Vishnumolakala V. R., Southam D. C., Treagust D. F., Mocerino M. and Qureshi S., (2017), Students' attitudes, self-efficacy and experiences in a modified process-oriented guided inquiry learning undergraduate chemistry classroom, Chem. Educ. Res. Pract., 18(2), 340–352.
109. Wamser C. C., (2006), Peer-led team learning in organic chemistry: Effects on student performance, success, and persistence in the course, J. Chem. Educ., 83(10), 1562–1566.
110. Warfa A.-R. M., (2016), Using cooperative learning to teach chemistry: A meta-analytic review, J. Chem. Educ., 93(2), 248–255.
111. Weaver G. C. and Sturtevant H. G., (2015), Design, implementation, and evaluation of a flipped format general chemistry course, J. Chem. Educ., 92(9), 1437–1448.
112. Wilson S. B. and Varma-Nelson P., (2016), Small groups, significant impact: A review of peer-led team learning research with implications for STEM education researchers and faculty, J. Chem. Educ., 93(10), 1686–1702.
113. Xu X., Kim E. S. and Lewis J. E., (2016), Sex difference in spatial ability for college students and exploration of measurement invariance, Learn. Individ. Differ., 45, 176–184.
114. Zusho A., Pintrich P. R. and Coppola B., (2003), Skill and will: The role of motivation and cognition in the learning of college chemistry, Int. J. Sci. Educ., 25(9), 1081–1094.

---