View PDF VersionPrevious Article
 
DOI: 10.1039/C7RP00173H (Paper) Chem. Educ. Res. Pract., 2018, 19, 392-397

A comparison of online and traditional chemistry lecture and lab

E. K. Faulconer *, J. C. Griffith *, B. L. Wood , S. Acharyya and D. L. Roberts
Embry-Riddle Aeronautical University, 600 South Clyde Morris Boulevard, Daytona Beach, FL 32114, USA. E-mail: faulcone@erau.edu; griff2ec@erau.edu

Received 8th September 2017 , Accepted 18th December 2017

First published on 18th December 2017

Abstract

While the equivalence between online and traditional classrooms has been well researched, very little effort has been expended to do such comparisons for college level introductory chemistry. The existing literature has only one study that investigated chemistry lectures at an entire course level as opposed to particular course components such as individual topics or exams. Regarding lab courses, only one study is available and it involves moderating variables that are largely uncontrolled. In this work, we compared the student pass rates, withdrawal rates, and grade distributions between asynchronous online and traditional formats of an introductory chemistry lecture as well as its associated lab course. The study was based on the 823 university records available for the 2015–2016 academic year. Student pass and withdrawal rates between the two modes were quite similar and did not appear to be statistically significant. However, grade distributions for both the lecture and lab differed between the two learning modes, showing significant statistical associations. Online students were more likely to earn As in both lecture and lab while traditional in-person students were more likely to earn Cs or Ds. Further research should include replication of this study with a larger data set. Additionally, this study should be repeated in three to five years to determine if advances in course design, standardization and delivery platforms further reduce or eliminate differences between learning modes. Future studies should also use qualitative tools for a better understanding of why students fail or withdraw from courses.

Introduction

As technology advances, the implementation of technology in higher education also increases. An increasing number of universities are offering online courses and enrolments in online courses are on the rise (Online Learning Consortium, 2016). In fact, 2012 was declared the “Year of the MOOC” (massive open online course) by the New York Times (Pappano, 2012). With this increase comes the question of equivalency of the modes.

An early meta-analysis of research dating from 1928 to 1998 consisting of 355 research reports and papers concluded there was no significant difference in learners’ success between traditional instruction and instruction employing technology (Russell, 2001). Subsequent meta-analysis studies have supported this conclusion (Cavanaugh et al., 2004; Zhao et al., 2005; Jahng et al., 2007; Lundberg et al., 2008). However, other meta-analysis studies have found a significant difference between online and traditional instruction (Shachar and Neumann, 2003; Allen et al., 2004; Bernard et al., 2004; Sitzmann et al., 2006; Williams, 2006).

Beyond investigating variations in learner outcomes between the two modalities, studies also show mixed results for the correlation between failure rates and course modality. In a study of 20[thin space (1/6-em)]677 enrollments, a statistically significant relationship was found between the failure rate and the modality for English and Math courses, but not for Economics and Humanities courses (Griffith et al., 2014). A large scale study of community college enrolments also supported the occurrence of higher failure/withdraw rates in online courses (Jaggars et al., 2013). The failure rate of Math and English developmental courses was particularly high in relation to traditional courses (Jaggars et al., 2013). Furthermore, low GPA students have been shown to be more likely to withdraw from an online course than a face-to-face course (Jaggars et al., 2013). Griffith et al. (2014) also established a correlation between grade distribution and modality for English, Math, Economics, and Humanities courses. Online students have been shown to be less likely to persist in their educational path to attain a degree (Jaggars et al., 2013).

A meta-analysis has revealed important moderating factors in distance education (Lou et al., 2006). The pedagogical methods employed (instructor-directed vs. collaborative) explain a significant amount of the variance in outcomes for synchronous and asynchronous distance education courses. The influence of the media used in the distance education was puzzling, with some media affecting student achievement positively while others affected achievement negatively. However, Sitzmann et al. (2006) found no statistically significant difference between the delivery media investigated.

Student attitudes towards online learning versus traditional learning is also well addressed in the literature. A 2002 meta-analysis found that students marginally prefer traditional format over distance education (Allen et al., 2002). Students report lower level of instructor presence in online courses, resulting in the mindset that they have to “teach themselves”; students also report preference for “easier” classes online and “harder” classes in a traditional format (Jaggars, 2014).

Despite the unanswered questions and concerns raised in the literature regarding online education compared to traditional education, the chemistry education community is increasing focus on the blended and online modality for both chemistry lecture and lab (Pienta, 2013; Gould, 2014). Distance learning in a chemistry lab can either be a virtual experience through a platform such as LateNiteLabs or hands-on through mail-order lab kits through companies such as eScience Labs. In some cases, the development of virtual labs has been financially supported by the US Department of Education's Fund for the Improvement of Postsecondary Education (Carnevale, 2003).

Although hundreds of studies exist that compare traditional and online modalities in higher education, very limited research is available to compare these two approaches for college level introductory chemistry. The Colorado Department of Higher Education performed a comparison of community college science classes, including chemistry (Colorado Department of Higher Education, 2012). In their study, students self-selected into the course format of their choice between Fall 2007 and Fall 2009 (n = 4585 with 2395 enrolled in the online format and 2190 enrolled in the traditional format). When looking at the entire data set (chemistry, biology, and physics), students enrolled in the Online courses typically had a higher GPA and more credit hours but the traditional course resulted in a higher average grade in the course. This same trend was mirrored when isolating the data solely to chemistry enrolments. The authors suggest these differences are open to interpretation and suggest future research. However, it is unclear if there were laboratory components to the courses that were considered in this study.

A 2013 study compared exam performance for students enrolled in general chemistry lecture for non-majors in the online and traditional modalities (Gulacar et al., 2013). The authors concluded that students enrolled in the online format outperformed traditional students for exam questions that fall into the “remember” category of Bloom's taxonomy. However, there was not a statistically significant difference for questions at the analysis level. The cause of the differences at lower levels is open to interpretation: online instruction may promote better memorization of facts or students that excel at memorization may gravitate towards the online modality. When comparing student performance on individual chemistry topics between the modalities, there was no difference in mastery based on modality.

The comparison of online labs to traditional labs is less expansive than the comparison of lecture courses. One study comparing student performance between an online and traditional engineering lab demonstrated increased conceptual understanding, a more positive attitude, and shorter completion time for the online lab compared to the traditional lab (Javidi, 2005). A review of recent (post-2005) empirical studies found that student learning outcome are achieved at equal or higher rates in non-traditional labs compared to traditional laboratory environments, with the majority of studies reviewed focusing on content knowledge outcomes using quizzes or other summative assessments (Brinson, 2015). Research focusing specifically on comparing outcomes by modality for chemistry labs is even more limited. A comparison of an individual electrochemistry lab activity between the two formats in a second year general chemistry course showed no significant difference between the learning modes in regards to content knowledge and development of hands-on skills (Hawkins and Phelps, 2013). Another study demonstrated that engagement with virtual lab manipulations was the best predictor of performance on a traditional summative assessment on stoichiometric calculations (Yaron et al., 2010). The only study to compare chemistry laboratory outcomes for an entire term found no significant difference in student outcomes between modalities (Casanova and Civelli, 2006). However, this study had largely uncontrolled moderating factors, with the online and traditional courses occurring at different institutions and the nature of the laboratory experiences varying between modality.

The purpose of this study was to critically compare student performance in an introductory chemistry course and lab based on learning mode. The learning modes compared were the traditional face-to-face format and the online format. We are seeking to answer the following question: does the mode of instruction of a first-year general chemistry class impact failure/withdrawal rate or grade distribution?

Hypotheses:

Ha1. Pass rates and learning mode (online or in person) are associated in an introductory chemistry lecture.

Ha2. Pass rates and learning mode (online or in person) are associated in an introductory chemistry laboratory.

Ha3. Grade distribution and learning mode (online or in person) are associated in an introductory chemistry lecture.

Ha4. Grade distribution and learning mode (online or in person) are associated in an introductory chemistry laboratory.

Ha5. Withdrawal rates and learning mode (online or in person) are associated in an introductory chemistry lecture.

Ha6. Withdrawal rates and learning mode (online or in person) are associated in an introductory chemistry laboratory.

Experimental

Participants

Final course grades were gathered for first year general chemistry enrolments at Embry-Riddle Aeronautical University (ERAU) for August 2015 to July 2017 through the institutional database, with no unique student identification (including name or numerical identifiers) obtained or used in this study. The study was deemed exempt by ERAU's Internal Review Board. ERAU was ranked number 1 in Best Online Bachelor's Programs in 2016 and 2017 according to the US News and World Report. In 2017, the projected enrolments show 15% of classes meeting in a traditional lecture modality while 85% of the University's enrolments in non-traditional modalities. At this time, the chemistry course is offered through traditional in-person classroom and online asynchronous modalities. Aggregate data containing a total of n = 823 enrolments were reviewed for the equivalent general chemistry lecture and lab courses across the online and residential campuses.

A total of 370 students registered for the introductory chemistry lecture course and 453 registered for the introductory chemistry lab in the time span covered in this study. Overall, 11 students withdrew from the lecture and 25 withdrew from the lab leaving a total of 359 and 428, respectively. Those registrations were divided between in-person traditional course offerings and online. Chemistry course registrations in the traditional lecture setting totalled 261 students. With six withdrawals, 255 students completed the traditional lecture course. The in-person traditional lab was taken by 351 students. With twenty withdrawals, 331 students completed the traditional laboratory course. Online student registrations totalled 109 for the introductory lecture course. With five withdrawals, 104 students completed the online lecture course. Online student registrations totalled 102 for the lab course. With five withdrawals, 97 students completed the online laboratory course.

Procedure

The independent variable in this study was the course modality, with two categories: in-person (traditional), and online (non-traditional). The online lab courses were executed through LateNiteLabs simulation platform. As an overt indicator of student performance, the dependent variables measured were student failure rate, grade distribution, and student withdrawal rate.

The lecture and lab courses operate from the same master course outlines, which specifies the course description, goals, and learning outcomes. Furthermore, the same textbook was used at both the residential campus and the online campus, which standardized the content of the course across modalities. Instructors at both campuses do have a degree of academic freedom and thus were able to design their own summative assessments. Common summative lecture assessments for both traditional and online chemistry included quizzes and tests. In the lab course, summative assessments in the traditional lab often included lab reports and a lab practical exam. The online lab course did not include a lab practical exam. Each campus has control over the specific lab topics covered if the topics are in support of topics covered in lecture. This resulted in a 72% alignment of topics between the two campuses. Differences in the classroom environments due to the instructor's teaching skills and pedagogy choices are impossible to control and still arrive at a meaningful n value. Additionally, it is impossible to control for the moderating factor of variations between cohort, term to term.

The researchers used a causal-comparative research model. All data were treated as nominal. Six hypotheses were testing using the Chi square test of independence and Fishers Exact test, as appropriate (α = 0.05) (Gay et al., 2006).

Results and discussion

Descriptive statistics

Pass and failure rates for students who took the introductory chemistry course in a traditional in-person format and students who took the same course online are presented in Table 1. Pass rates between traditional in-person courses and online courses did not differ greatly between the two campuses in either the lecture or lab. Overall, approximately 87% of all students who took the introductory chemistry lecture passed, with online students showing a slightly higher pass rate. Approximately 94% of all students who took the laboratory course passed, with in-person students showing a slightly higher pass rate.
Table 1 Pass (P) and failure (F) rates based on learning modes, excluding withdrawals (α = 0.05)
Learning mode Chemistry lecture Chemistry lab
P F Total P F Total
In person 220 (86.2%) 35 (13.7%) 255 312 (94.2%) 19 (5.7%) 331
Online 94 (90.3%) 10 (9.6%) 104 90 (92.8%) 7 (7.2%) 97
Total 314 (87%) 45 (13%) 359 402 (94%) 26 (6%) 428

Grade distributions based on modality are shown in Table 2. A higher percentage of online students earned an A in the chemistry lecture. Students earned B grades at a similar rate. Traditional in-person lecture students achieved more Cs, Ds or Fs. Online students also earned more As (74.2% vs. 53.2%) in the labs, however a higher percentage also earned F grades (7.2% vs. 5.7%).

Table 2 Grade distribution by modality, excluding withdrawals (α = 0.05)
Grade A B C D F Total
Lecture In person 50 (19.6%) 90 (35.3%) 59 (23.4%) 21 (8.2%) 35 (13.7%) 255
Online 38 (36.5%) 37 (35.6%) 17 (16.4%) 2 (1.9%) 10 (9.6%) 104
Lab In person 176 (53.2%) 91 (27.5%) 31 (9.4%) 14 (4.2%) 19 (5.7%) 331
Online 72 (74.2%) 15 (15.5%) 2 (2.1%) 1 (1%) 7 (7.2%) 97

Little difference was seen in withdrawal rates between traditional in-person students and online students (Table 3). Traditional lecture students withdrew at a 2.3% rate vs. a 4.6% rate for online students. The withdrawal rates for the chemistry lab were also relatively similar, with traditional students withdrawing at a rate of 5.7% compared to 4.9% of online laboratory students.

Table 3 Withdrawal rated by modality (α = 0.05)
Completed Withdrew Total Chi-square (p value) Fishers exact test
Lecture In person 255 (97.7%) 6 (2.3%) 261 (100%)
Online 104 (95.41%) 5 (4.59%) 109 (100%) — p = 0.312
Lab In person 331 (94.3%) 20 (5.7%) 351 (100%)
Online 97 (95.1%) 5 (4.9%) 102 (100%) 0.096 (p = 0.757) —

Inferential statistics

Statistical tests of the hypotheses are shown in Table 4. Differences in the pass/failure rate by modality for both the lecture and lab were shown to be statistically insignificant, with respective p-values of 0.286 and 0.5924.
Table 4 Test results for hypotheses (α = 0.05)
Variable DF χ 2 value p-Value
Ha1. Failure rate chemistry lecture 1 1.138 — 0.286
Ha2. Failure rate lab 1 0.287 — 0.592
Ha3. Grade distribution chemistry lecture 4 15.839 — 0.003*
Ha4. Grade distribution lab 4 14.771 — 0.001*
Ha5. Withdrawal rate chemistry lecture — — Fisher's exact test 0.312
Ha6. Withdrawal rate lab 1 0.096 — 0.757

Interestingly, the grade distributions for both lectures and labs showed a significant difference. There is evidence to support the alternate hypothesis that there is an association between grading distributions and modality of course delivery when comparing online vs. traditional in-person course grades. A higher percentage of online students earned As in both the lecture and lab as shown on Table 4. Traditional-in person students tended to earn more Cs and Ds than online students.

The analysis of withdrawal rate across modality for the lecture course yielded a Chi-square result of 1.396 (p = 0.2374) and a warning of low expected cell sizes. Therefore, the data was re-evaluated using Fisher's Exact test (a more conservative test) (Gould and Ryan, 2012). The withdrawal rates for both lecture and lab did not show enough evidence to conclude there was an association between the rates and the learning modality.

Post hoc testing on grade distributions

Research hypotheses for failure rates and withdrawal rates were not supported in this study. Grade distributions did show an association between learning modes and grades however. As shown in pairwise comparisons in Table 5, online students were almost twice as likely to earn an A as traditional in-person lecture students (36.5% vs. 19.6% respectively, p = 0.1). The same trend was seen in the laboratory course, where online students earned As at a higher proportion than traditional in-person students (74% vs. 53%) although not to a statistically significant degree (α = 0.05). The proportion of Bs were similar between modalities in the chemistry lecture. Traditional in-person laboratory students received a higher proportion of Bs than online students although associations were not statistically significant. In-person students earned a higher proportion of Cs and Ds in both the chemistry lecture and lab. Chemistry lab student grades showed a significantly higher proportion of Cs for in person students (9.4% than online (2.1% − p = 0.025)). In person attendees earned D grades at significantly higher rates than online students in the chemistry lecture class (p = 0.035).
Table 5 Pairwise comparison of passing grades by modality (α = 0.05)
Grade A B C D F
Lecture In person n = 255 50 (19.6%) 90 (35.3%) 59 (23.4%) 21 (8.2%) 35 (13.7%)
Online n = 104 38 (36.5%) 37 (35.6%) 17 (16.4%) 2 (1.9%) 10 (9.6%)
Chi sq. p value 6.586 (p = 0.010*) 0.0012 (p = 0.972) 1.361 (p = 0.243) 4.431 (p = 0.035*) 1.138 (p = 0.286)
Lab In person n = 331 176 (53.2%) 91 (27.5%) 31 (9.4%) 14 (4.2%) 19 (5.7%)
Online n = 97 72 (74.2%) 15 (15.5%) 2 (2.1%) 1 (1%) 7 (7.2%)
Chi sq. p value 3.3966 (p = 0.065) 3.7144 (p = 0.054) 5.008 (p = 0.025*) – Fishers exact test (p = 0.2082) 0.287 (p = 0.592)

Discussion

The results of this study mirror previous studies where minimal or no difference in pass rates between online learners and traditional in-person students were found (Russell, 2001; Lou et al., 2006; Hrastinski, 2008). In this study, students passed introductory chemistry lectures and laboratories at similar rates regardless of learning mode.

Disparity in the grade distributions between traditional in person students and online students were noted in this study. Data showed a statistically significant association between grade distribution and learning mode for both the chemistry (p = 0.003) and lab courses (p = 0.001). Further analysis showed that (in this study) traditional in person chemistry lecture students tended to get fewer As and more Ds than their online counterparts. In person lab students also tended to get more Ds than online students. Previous research has also noted differences in grade distribution (Griffith et al., 2014).

Interestingly, students enrolled in online and traditional in-person courses withdrew from either the chemistry lecture or lab at similar rates. Previous literature has shown higher withdrawal rates from online courses (Jaggars and Bailey 2010; Jaggars et al., 2013; Cochran et al., 2014). Some researchers have theorized that students in traditional in person classes might withdraw at a lower rate due to the social support from other students and the structure of going to class at a given time (Metz, 2002; Wilcox et al., 2005; Bawa, 2016). However, this social support factor was not evident in this study.

Conclusions

The goal of this study was to explore if general measures of student outcomes (pass rate, grade distribution, and withdrawal rate) indicated a significant difference between modalities. Because of the differences discovered between modalities, a follow-up study is warranted that implements a set of standardized assessments to measure if students are achieving specific learning outcomes similarly by modality. One goal of course standardization across modalities is to make the content delivery equitable, no matter how a student chooses to take a course; any variations in student mastery of learning objectives should be explored. Differences in mastery of learning objectives by modality may help explain the differences seen in grade distribution in this study.

While this study did not demonstrate a significant difference in withdrawal rate by modality, a qualitative study could improve understanding of why students fail or withdraw. Previous studies suggested we would find significant differences by modality, which could be attributed to peer support. A future qualitative study could further probe the peer support theory.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

We would like to acknowledge Mitsouko “Mickey” Munoz (Business Intelligence Analyst) and Tarji Kinsey (Data Reporting Manager) for collecting the raw data for analysis. We would like to thank Paulo Jiminez for formatting the images.

References

  1. Allen M., Bourhis J., Burrell N. and Mabry E., (2002), Comparing student satisfaction with distance education to traditional classrooms in higher education: a meta-analysis, Am. J. Distance Educ., 16(2), 83–97.
  2. Allen M., Mabry E., Mattrey M., Bourhis J., Titsworth S. and Burrell N., (2004), Evaluating the effectiveness of distance learning: a comparison using meta-analysis, J. Commun., 54402–54420.
  3. Bawa P., (2016), Retention in online courses: exploring issues and solutions – a literature review, Sage Open, 6(1), 1–11.
  4. Bernard R. M., Abrami P. C., Lou Y., Borokhovski E., Wade A., Wozney L., Wallet P., Fiset M. and Huang B., (2004), How does distance education compare with classroom instruction? A meta-analysis of the empirical literature, Rev. Educ. Res., 74(3), 379–439.
  5. Brinson J., (2015), Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: a review of the empirical research, Comput. Educ., 87, 218–237.
  6. Carnevale D., (2003), The Virtual Lab Experiment.
  7. Casanova R. S. and Civelli J. L., (2006), Distance learning: a viable alternative to the conventional lecture-lab format in general chemistry, J. Chem. Educ., 83(3), 501.
  8. Cavanaugh C., Gillan K. J., Kromrey J., Hess M. and Blomeyer R., (2004), The effects of distance education on K-12 student outcomes: a meta-analysis, Learning Point Associates/North Central Regional Educational Laboratory.
  9. Cochran J., Campbell S. M., Baker H. M. and Leeds E. M., (2014), The role of student characteristics in predicting retention in online courses, Res. High. Educ., 55(1), 27–48.
  10. Colorado Department of Higher Education, (2012), Online versus Traditional Learning: A Comparison Study of Colorado Community College Science Classes, Colorado Department of Higher Education.
  11. Gay L. R., Mills G. E., and Airasian P. W., (2006), Educational Research: Competencies for analysis and applications, 8th edn, Upper Saddle River, New Jersey: Pearson Education, Inc.
  12. Gould J., (2014), Online education: the rise of virtual labs.
  13. Gould R. N. and Ryan C. N., (2012), Introductory Statistics, 1st edn, Boston, MA: Pearson.
  14. Griffith J., Roberts D. and Schultz M., (2014), Relationship between grades and learning mode, J. Am. Business Rev., 3(1), 81–88.
  15. Gulacar O., Damkaci F. and Bowman C., (2013), A comparative study of an online and face-to-face chemistry course, J. Interact. Online Learn., 1227–1240.
  16. Hawkins I. and Phelps A., (2013), Virtual laboratory vs. traditional laboratory: which is more effective for teaching electrochemistry, Chem. Educ. Res. Pract., 14(4), 516–523.
  17. Hrastinski S., (2008), Asynchronous and synchronous e-learning, Educ. Quart., (4), 51–55.
  18. Jaggars S. S., (2014), Choosing between online and face-to-face courses: community college student voices, Am. J. Distance Educ., 28(1), 23–28.
  19. Jaggars S. S. and Bailey T., (2010), Effectiveness of fully online courses for college students: Response to a department of education meta-analysis, Community College Research Center.
  20. Jaggars S. S., Edgecombe N. and Stacey G. W., (2013), What we know about online course outcomes, Community College Research Center.
  21. Jahng N., Krug D. and Zhang Z., (2007), Student achievement in online distance education compared to face-to-face education, Eur. O. D. L., 10(1).
  22. Javidi G. (2005), A comparison of traditional physical laboratory and computer simulated laboratory experiences in relation to engineering undergraduate students conceptual understandings of a communication systems topic, University of South Florida.
  23. Lou Y., Bernard R. M. and Abrami P. C., (2006), Media and pedagogy in undergraduate distance education: a theory-based meta-analysis of empirical literature, Educ. Technol. Res. Dev., 54(2), 141–176.
  24. Lundberg J., Castillo-Merino D. and Dahmani M., (2008), Do Online Students Perform Better than Face-to-face students? Reflections and a Short Review of some Empirical Findings, 1st edn, Editorial UOC.
  25. Metz G. W., (2002), Challenges and changes to tinto's persistence theory.
  26. Online Learning Consortium, (2016), Babson study: distance education enrollment growth continues, http://https://onlinelearningconsortium.org/news_item/babson-study-distance-education-enrollment-growth-continues-2/.
  27. Pappano L., (2012), The Year of the MOOC.
  28. Pienta N., (2013), Online Courses in Chemistry: Salvation or Downfall.
  29. Russell T., (2001), The No Significant Difference Phenomenon: As reported in 355 research reports, summaries, and papers, 5th edn, IDECC, North Carolina State University.
  30. Shachar M. and Neumann Y., (2003), Differences between traditional and distance education academic performances: a meta-analytic approach, Int. Rev. Res. Open Distrib. Learn., 4(2), 153.
  31. Sitzmann T., Kraiger K., Steward D. and Wisher R., (2006), The comparative effectiveness of web-based and classroom instruction: a meta-analysis, Pers. Psychol., 59(3), 623–664.
  32. Wilcox P., Winn S. and Fyvie-Gauld M., (2005), ‘It was nothing to do with the university, it was just the people’: the role of social support in the first-year experience of higher education, Stud. High. Educ., 30(6), 707–722.
  33. Williams S. L., (2006), The effectiveness of distance education in allied health science programs: a meta-analysis of outcomes, Am. J. Distance Educ., 20(3), 127–141.
  34. Yaron D., Karabinos M., Lange D., Greeno J. G. and Leinhardt G., (2010), The ChemCollective – virtual labs for introductory chemistry courses, Science, 328(5978), 584–585.
  35. Zhao Y., Lei J., Yan B., Lai C. and Tan H. S., (2005), What makes the difference? A practical analysis of research on the effectiveness of distance education, Teach. Coll. Rec., 107(8), 1836–1884.
This journal is © The Royal Society of Chemistry 2018