E. K.
Faulconer
*,
J. C.
Griffith
*,
B. L.
Wood
,
S.
Acharyya
and
D. L.
Roberts
Embry-Riddle Aeronautical University, 600 South Clyde Morris Boulevard, Daytona Beach, FL 32114, USA. E-mail: faulcone@erau.edu; griff2ec@erau.edu
First published on 18th December 2017
While the equivalence between online and traditional classrooms has been well researched, very little effort has been expended to do such comparisons for college level introductory chemistry. The existing literature has only one study that investigated chemistry lectures at an entire course level as opposed to particular course components such as individual topics or exams. Regarding lab courses, only one study is available and it involves moderating variables that are largely uncontrolled. In this work, we compared the student pass rates, withdrawal rates, and grade distributions between asynchronous online and traditional formats of an introductory chemistry lecture as well as its associated lab course. The study was based on the 823 university records available for the 2015–2016 academic year. Student pass and withdrawal rates between the two modes were quite similar and did not appear to be statistically significant. However, grade distributions for both the lecture and lab differed between the two learning modes, showing significant statistical associations. Online students were more likely to earn As in both lecture and lab while traditional in-person students were more likely to earn Cs or Ds. Further research should include replication of this study with a larger data set. Additionally, this study should be repeated in three to five years to determine if advances in course design, standardization and delivery platforms further reduce or eliminate differences between learning modes. Future studies should also use qualitative tools for a better understanding of why students fail or withdraw from courses.
An early meta-analysis of research dating from 1928 to 1998 consisting of 355 research reports and papers concluded there was no significant difference in learners’ success between traditional instruction and instruction employing technology (Russell, 2001). Subsequent meta-analysis studies have supported this conclusion (Cavanaugh et al., 2004; Zhao et al., 2005; Jahng et al., 2007; Lundberg et al., 2008). However, other meta-analysis studies have found a significant difference between online and traditional instruction (Shachar and Neumann, 2003; Allen et al., 2004; Bernard et al., 2004; Sitzmann et al., 2006; Williams, 2006).
Beyond investigating variations in learner outcomes between the two modalities, studies also show mixed results for the correlation between failure rates and course modality. In a study of 20677 enrollments, a statistically significant relationship was found between the failure rate and the modality for English and Math courses, but not for Economics and Humanities courses (Griffith et al., 2014). A large scale study of community college enrolments also supported the occurrence of higher failure/withdraw rates in online courses (Jaggars et al., 2013). The failure rate of Math and English developmental courses was particularly high in relation to traditional courses (Jaggars et al., 2013). Furthermore, low GPA students have been shown to be more likely to withdraw from an online course than a face-to-face course (Jaggars et al., 2013). Griffith et al. (2014) also established a correlation between grade distribution and modality for English, Math, Economics, and Humanities courses. Online students have been shown to be less likely to persist in their educational path to attain a degree (Jaggars et al., 2013).
A meta-analysis has revealed important moderating factors in distance education (Lou et al., 2006). The pedagogical methods employed (instructor-directed vs. collaborative) explain a significant amount of the variance in outcomes for synchronous and asynchronous distance education courses. The influence of the media used in the distance education was puzzling, with some media affecting student achievement positively while others affected achievement negatively. However, Sitzmann et al. (2006) found no statistically significant difference between the delivery media investigated.
Student attitudes towards online learning versus traditional learning is also well addressed in the literature. A 2002 meta-analysis found that students marginally prefer traditional format over distance education (Allen et al., 2002). Students report lower level of instructor presence in online courses, resulting in the mindset that they have to “teach themselves”; students also report preference for “easier” classes online and “harder” classes in a traditional format (Jaggars, 2014).
Despite the unanswered questions and concerns raised in the literature regarding online education compared to traditional education, the chemistry education community is increasing focus on the blended and online modality for both chemistry lecture and lab (Pienta, 2013; Gould, 2014). Distance learning in a chemistry lab can either be a virtual experience through a platform such as LateNiteLabs or hands-on through mail-order lab kits through companies such as eScience Labs. In some cases, the development of virtual labs has been financially supported by the US Department of Education's Fund for the Improvement of Postsecondary Education (Carnevale, 2003).
Although hundreds of studies exist that compare traditional and online modalities in higher education, very limited research is available to compare these two approaches for college level introductory chemistry. The Colorado Department of Higher Education performed a comparison of community college science classes, including chemistry (Colorado Department of Higher Education, 2012). In their study, students self-selected into the course format of their choice between Fall 2007 and Fall 2009 (n = 4585 with 2395 enrolled in the online format and 2190 enrolled in the traditional format). When looking at the entire data set (chemistry, biology, and physics), students enrolled in the Online courses typically had a higher GPA and more credit hours but the traditional course resulted in a higher average grade in the course. This same trend was mirrored when isolating the data solely to chemistry enrolments. The authors suggest these differences are open to interpretation and suggest future research. However, it is unclear if there were laboratory components to the courses that were considered in this study.
A 2013 study compared exam performance for students enrolled in general chemistry lecture for non-majors in the online and traditional modalities (Gulacar et al., 2013). The authors concluded that students enrolled in the online format outperformed traditional students for exam questions that fall into the “remember” category of Bloom's taxonomy. However, there was not a statistically significant difference for questions at the analysis level. The cause of the differences at lower levels is open to interpretation: online instruction may promote better memorization of facts or students that excel at memorization may gravitate towards the online modality. When comparing student performance on individual chemistry topics between the modalities, there was no difference in mastery based on modality.
The comparison of online labs to traditional labs is less expansive than the comparison of lecture courses. One study comparing student performance between an online and traditional engineering lab demonstrated increased conceptual understanding, a more positive attitude, and shorter completion time for the online lab compared to the traditional lab (Javidi, 2005). A review of recent (post-2005) empirical studies found that student learning outcome are achieved at equal or higher rates in non-traditional labs compared to traditional laboratory environments, with the majority of studies reviewed focusing on content knowledge outcomes using quizzes or other summative assessments (Brinson, 2015). Research focusing specifically on comparing outcomes by modality for chemistry labs is even more limited. A comparison of an individual electrochemistry lab activity between the two formats in a second year general chemistry course showed no significant difference between the learning modes in regards to content knowledge and development of hands-on skills (Hawkins and Phelps, 2013). Another study demonstrated that engagement with virtual lab manipulations was the best predictor of performance on a traditional summative assessment on stoichiometric calculations (Yaron et al., 2010). The only study to compare chemistry laboratory outcomes for an entire term found no significant difference in student outcomes between modalities (Casanova and Civelli, 2006). However, this study had largely uncontrolled moderating factors, with the online and traditional courses occurring at different institutions and the nature of the laboratory experiences varying between modality.
The purpose of this study was to critically compare student performance in an introductory chemistry course and lab based on learning mode. The learning modes compared were the traditional face-to-face format and the online format. We are seeking to answer the following question: does the mode of instruction of a first-year general chemistry class impact failure/withdrawal rate or grade distribution?
Hypotheses:
Ha1. Pass rates and learning mode (online or in person) are associated in an introductory chemistry lecture.
Ha2. Pass rates and learning mode (online or in person) are associated in an introductory chemistry laboratory.
Ha3. Grade distribution and learning mode (online or in person) are associated in an introductory chemistry lecture.
Ha4. Grade distribution and learning mode (online or in person) are associated in an introductory chemistry laboratory.
Ha5. Withdrawal rates and learning mode (online or in person) are associated in an introductory chemistry lecture.
Ha6. Withdrawal rates and learning mode (online or in person) are associated in an introductory chemistry laboratory.
A total of 370 students registered for the introductory chemistry lecture course and 453 registered for the introductory chemistry lab in the time span covered in this study. Overall, 11 students withdrew from the lecture and 25 withdrew from the lab leaving a total of 359 and 428, respectively. Those registrations were divided between in-person traditional course offerings and online. Chemistry course registrations in the traditional lecture setting totalled 261 students. With six withdrawals, 255 students completed the traditional lecture course. The in-person traditional lab was taken by 351 students. With twenty withdrawals, 331 students completed the traditional laboratory course. Online student registrations totalled 109 for the introductory lecture course. With five withdrawals, 104 students completed the online lecture course. Online student registrations totalled 102 for the lab course. With five withdrawals, 97 students completed the online laboratory course.
The lecture and lab courses operate from the same master course outlines, which specifies the course description, goals, and learning outcomes. Furthermore, the same textbook was used at both the residential campus and the online campus, which standardized the content of the course across modalities. Instructors at both campuses do have a degree of academic freedom and thus were able to design their own summative assessments. Common summative lecture assessments for both traditional and online chemistry included quizzes and tests. In the lab course, summative assessments in the traditional lab often included lab reports and a lab practical exam. The online lab course did not include a lab practical exam. Each campus has control over the specific lab topics covered if the topics are in support of topics covered in lecture. This resulted in a 72% alignment of topics between the two campuses. Differences in the classroom environments due to the instructor's teaching skills and pedagogy choices are impossible to control and still arrive at a meaningful n value. Additionally, it is impossible to control for the moderating factor of variations between cohort, term to term.
The researchers used a causal-comparative research model. All data were treated as nominal. Six hypotheses were testing using the Chi square test of independence and Fishers Exact test, as appropriate (α = 0.05) (Gay et al., 2006).
Learning mode | Chemistry lecture | Chemistry lab | ||||
---|---|---|---|---|---|---|
P | F | Total | P | F | Total | |
In person | 220 (86.2%) | 35 (13.7%) | 255 | 312 (94.2%) | 19 (5.7%) | 331 |
Online | 94 (90.3%) | 10 (9.6%) | 104 | 90 (92.8%) | 7 (7.2%) | 97 |
Total | 314 (87%) | 45 (13%) | 359 | 402 (94%) | 26 (6%) | 428 |
Grade distributions based on modality are shown in Table 2. A higher percentage of online students earned an A in the chemistry lecture. Students earned B grades at a similar rate. Traditional in-person lecture students achieved more Cs, Ds or Fs. Online students also earned more As (74.2% vs. 53.2%) in the labs, however a higher percentage also earned F grades (7.2% vs. 5.7%).
Grade | A | B | C | D | F | Total | |
---|---|---|---|---|---|---|---|
Lecture | In person | 50 (19.6%) | 90 (35.3%) | 59 (23.4%) | 21 (8.2%) | 35 (13.7%) | 255 |
Online | 38 (36.5%) | 37 (35.6%) | 17 (16.4%) | 2 (1.9%) | 10 (9.6%) | 104 | |
Lab | In person | 176 (53.2%) | 91 (27.5%) | 31 (9.4%) | 14 (4.2%) | 19 (5.7%) | 331 |
Online | 72 (74.2%) | 15 (15.5%) | 2 (2.1%) | 1 (1%) | 7 (7.2%) | 97 |
Little difference was seen in withdrawal rates between traditional in-person students and online students (Table 3). Traditional lecture students withdrew at a 2.3% rate vs. a 4.6% rate for online students. The withdrawal rates for the chemistry lab were also relatively similar, with traditional students withdrawing at a rate of 5.7% compared to 4.9% of online laboratory students.
Completed | Withdrew | Total | Chi-square (p value) | Fishers exact test | ||
---|---|---|---|---|---|---|
Lecture | In person | 255 (97.7%) | 6 (2.3%) | 261 (100%) | ||
Online | 104 (95.41%) | 5 (4.59%) | 109 (100%) | — | p = 0.312 | |
Lab | In person | 331 (94.3%) | 20 (5.7%) | 351 (100%) | ||
Online | 97 (95.1%) | 5 (4.9%) | 102 (100%) | 0.096 (p = 0.757) | — |
Variable | DF | χ 2 value | p-Value | |
---|---|---|---|---|
Ha1. Failure rate chemistry lecture | 1 | 1.138 | — | 0.286 |
Ha2. Failure rate lab | 1 | 0.287 | — | 0.592 |
Ha3. Grade distribution chemistry lecture | 4 | 15.839 | — | 0.003* |
Ha4. Grade distribution lab | 4 | 14.771 | — | 0.001* |
Ha5. Withdrawal rate chemistry lecture | — | — | Fisher's exact test | 0.312 |
Ha6. Withdrawal rate lab | 1 | 0.096 | — | 0.757 |
Interestingly, the grade distributions for both lectures and labs showed a significant difference. There is evidence to support the alternate hypothesis that there is an association between grading distributions and modality of course delivery when comparing online vs. traditional in-person course grades. A higher percentage of online students earned As in both the lecture and lab as shown on Table 4. Traditional-in person students tended to earn more Cs and Ds than online students.
The analysis of withdrawal rate across modality for the lecture course yielded a Chi-square result of 1.396 (p = 0.2374) and a warning of low expected cell sizes. Therefore, the data was re-evaluated using Fisher's Exact test (a more conservative test) (Gould and Ryan, 2012). The withdrawal rates for both lecture and lab did not show enough evidence to conclude there was an association between the rates and the learning modality.
Grade | A | B | C | D | F | |
---|---|---|---|---|---|---|
Lecture | In person n = 255 | 50 (19.6%) | 90 (35.3%) | 59 (23.4%) | 21 (8.2%) | 35 (13.7%) |
Online n = 104 | 38 (36.5%) | 37 (35.6%) | 17 (16.4%) | 2 (1.9%) | 10 (9.6%) | |
Chi sq. p value | 6.586 (p = 0.010*) | 0.0012 (p = 0.972) | 1.361 (p = 0.243) | 4.431 (p = 0.035*) | 1.138 (p = 0.286) | |
Lab | In person n = 331 | 176 (53.2%) | 91 (27.5%) | 31 (9.4%) | 14 (4.2%) | 19 (5.7%) |
Online n = 97 | 72 (74.2%) | 15 (15.5%) | 2 (2.1%) | 1 (1%) | 7 (7.2%) | |
Chi sq. p value | 3.3966 (p = 0.065) | 3.7144 (p = 0.054) | 5.008 (p = 0.025*) | – Fishers exact test (p = 0.2082) | 0.287 (p = 0.592) |
Disparity in the grade distributions between traditional in person students and online students were noted in this study. Data showed a statistically significant association between grade distribution and learning mode for both the chemistry (p = 0.003) and lab courses (p = 0.001). Further analysis showed that (in this study) traditional in person chemistry lecture students tended to get fewer As and more Ds than their online counterparts. In person lab students also tended to get more Ds than online students. Previous research has also noted differences in grade distribution (Griffith et al., 2014).
Interestingly, students enrolled in online and traditional in-person courses withdrew from either the chemistry lecture or lab at similar rates. Previous literature has shown higher withdrawal rates from online courses (Jaggars and Bailey 2010; Jaggars et al., 2013; Cochran et al., 2014). Some researchers have theorized that students in traditional in person classes might withdraw at a lower rate due to the social support from other students and the structure of going to class at a given time (Metz, 2002; Wilcox et al., 2005; Bawa, 2016). However, this social support factor was not evident in this study.
While this study did not demonstrate a significant difference in withdrawal rate by modality, a qualitative study could improve understanding of why students fail or withdraw. Previous studies suggested we would find significant differences by modality, which could be attributed to peer support. A future qualitative study could further probe the peer support theory.
This journal is © The Royal Society of Chemistry 2018 |