Measuring the effectiveness of online preparation videos and questions in the second semester general chemistry laboratory

Abstract

Chemistry laboratory experiences provide students the opportunity to engage all three domains of learning: psychomotor, cognitive and affective. However, they are often stressful environments where students are expected to quickly learn new laboratory techniques, and collect data in a short amount of time. In principle, providing additional preparation activities should help students be better prepared to successfully complete the lab. These activities should lead to more meaningful interactions with the lab instructor and better performance on lab outcomes. In this study, we report the usefulness and effectiveness of online preparation activities for students that include video lectures demonstrating the labs that the students will participate in, and preparation questions that mimic data analysis for the lab. These online prelab activities were implemented in the second semester general chemistry laboratory at a large Hispanic serving institution in the southwestern United States. Over three semesters, students enrolled in this course were surveyed using the Meaningful Learning in the Laboratory Instrument (MLLI) to assess their lab expectations as well as author generated post-lab surveys to assess the usefulness of the prelab activities. Additionally, lab instructors were surveyed on their perception of the efficacy of the additional preparation activities. Findings suggest that both students and instructors agree that having access to these materials as a part of a portfolio of resources, including the lab manual, help them better prepare for the lab. Although students’ expectations on the cognitive domain decreased after a semester of instruction, questions related to comfort with lab equipment show improvements in the affective domain for students with access to the additional preparation activities. Lastly we found that both students and instructors see a lot of value and benefits in having these types of prelab activities available as a way to help prepare students for the upcoming laboratory sessions. In general, the potential benefits that prelab activities had on students outweigh the modest effort to create these materials.

---

Introduction

Undergraduate laboratories are integral part of most STEM majors including biology, physics and chemistry. Science curricula is centered around the idea that students learned best or have a more meaningful learning experience when they are involved in their learning process. An undergraduate laboratory is designed to help students make connections between concepts learned in lecture and hands-on activities where students develop their psychomotor skills while reinforcing the concepts. Since chemistry is a central part of any STEM curriculum, students will be exposed to the undergraduate chemistry laboratory during their first or second year of college. Therefore it is important to study and assess the undergraduate chemistry laboratories as they will have an impact in students’ learning experience early in their college career. When focusing on the undergraduate chemistry laboratories, different positive student outcomes have been hypothesized about the laboratory experience. Some of these outcomes are learning communication, social and technical skills, engaging students in scientific inquiry, and helping students learn chemistry concepts and develop problem solving skills (Hofstein and Lunetta, 2004; Högström et al., 2010; DeKorver and Towns, 2015; Jolley et al., 2016). However, in many instances, students feel overwhelmed by the different tasks that they need to accomplish before, during, and after a laboratory experience. This overwhelmed feeling comes as a result of a cognitive overload that students experience and has been identified to originate from different sources, such as the laboratory manual, direct instruction, unfamiliar equipment and materials, technical skills, theoretical background, and time management (Reid and Shah, 2007). Students are expected to process all this information and master the skills needed to successfully complete each task required in the laboratory. One of the problems with this expectation is that most of the time, students are not adequately prepared and they spend more time worrying about the technical details and following a procedure than understanding the underlying concepts of the laboratory tasks.

Prelaboratory preparation reduces the amount of information students are exposed to at the beginning of the laboratory (Reid and Shah, 2007; Jolley et al., 2016) helping them to successfully complete the lab (Schmidt-McCormack et al., 2017). Prelaboratory materials should help students reduce cognitive overload and therefore gain a deeper engagement with the concepts covered during the experience (Teo et al., 2014; Box et al., 2017; Schmidt-McCormack et al., 2017). According to Jolley et al. (2016), prelaboratory material should “facilitate students' engagement and encourage effective preparation.” Previous studies have found that prelaboratory materials have increased students' understanding of the theory and practical work, as well as enhancing their experience in the laboratory session by providing students with opportunities to become familiar with the experimental design and procedures before getting to the laboratory (Johnstone et al., 1994; Rollnick et al., 2001; Jolley et al., 2016).

Recently the value and the goals of the laboratory for students' learning have been debated in chemical education (Bretz, 2019). Some research literature and researchers suggest the importance of the laboratory experience for students and the potential benefits of prelaboratory preparation for them (Reynders et al., 2019; Sansom and Walker, 2019; Seery, 2020). However, there is a wide call in the field to present evidence that the lab courses help students learn chemistry and develop the necessary skills to be a chemist that justify the need in the curriculum (Bretz, 2019; Reynders et al., 2019; Sansom and Walker, 2019; Seery, 2020). One of the problems is the way that learning in the lab is measured and how we measure the skills gained in the lab. Despite all the debate on laboratory courses, lab courses are still offered throughout the chemistry curriculum from lower level courses such as general chemistry to upper division courses such as physical chemistry. Therefore, it is important to assess these laboratory courses and find ways to help students succeed and learn through those experiences. There are very few studies that focus on assessing learning in the lab and even less studies on how prelaboratory materials help students to have a better experience (Seng and Mohamad, 2002; Vician and Charlesworth, 2003; Chittleborough et al., 2007; Alsharif and Henriksen, 2009; Jones, 2010; Jolley et al., 2016; Schmidt-McCormack et al., 2017).

There are different types of prelaboratory materials that have been used in the laboratory setting (Agustian and Seery, 2017). One of the most common includes students completing a writing-based prelab activity in which they answer a set of questions related to theory and procedure. Another common prelab activity is to have students copy the objectives and procedure in their lab notebook. However, in a time where technology is an important educational tool, online materials supplementing lectures have been more frequently introduced in courses. In general, students seem to have a favorable attitude toward the use of online materials to supplement lectures and laboratories (Seng and Mohamad, 2002; Vician and Charlesworth, 2003; Alsharif and Henriksen, 2009; Jolley et al., 2016). Also, previous studies have reported that students feel more prepared when online prelab activities exist (Chittleborough et al., 2007; Jones, 2010). In a biology setting, one study reported that students felt more confident about the tasks required in the laboratory experience after being exposed to online prelab materials that contained visual representations of the procedure (Jones, 2010). Those same students reported more motivation and a better understanding of what was required during the lab session. In a recent study with chemistry students in an introductory analytical chemistry lab, (Jolley et al., 2016) found that video demonstrations and prelab online quizzes helped students with their perceived level of preparedness although no improvement in their academic performance was found.

In the Agustian and Seery (2017) review, they found three main rationale for prelaboratory activities: to introduce chemical concepts, to introduce laboratory techniques, and to address affective dimensions. For this study, we focus on prelab activities to introduce laboratory techniques. Specifically, in this paper we will discuss the effectiveness of a set of online preparation activities that were developed and implemented for the second semester general chemistry laboratory course. These preparation activities include video lectures demonstrating the labs that the students will participate in, and preparation questions that mimic data analysis for the lab. While there was some effort to cover theoretical topics, the focus of the preparation materials was so that students would feel comfortable working with the lab equipment and have the confidence to perform the calculations they would need to complete once they collect their data. Since there is only so much work that can be shifted outside of the lab, experimental details were prioritized. This work was to determine if this focus has an impact on student performance as measured by their expectations and perception of the preparation materials’ usefulness. The specific research questions guiding this study are:

1. What are students’ perceptions on the usefulness of the prelab activities including prelab questions and videos as well as other resources?

2. What are students’ laboratory expectations and how these expectations change after a semester of instruction?

3. What are the instructors’ insights about students’ benefits of using the prelab activities in the laboratory?

Methods

Participants

Students enrolled in the second-semester general chemistry laboratory in a four-year public master university in the southwestern part of the United States participated in this study. The university is classified as a Hispanic-Serving Institution (HSI). Students were asked to participate in the research and consent were obtained following the Institutional Review Board (IRB) guidelines and approval. Data were collected from Spring 2017, Fall 2017 and Spring 2018. Spring 2017 data involved a quasi-experiment where three sections participated in the study, but only one section had access to the preparation activities. Fall 2017 and Spring 2018 data involved four lab sections each semester and all of them had access to the preparation activities. Instructors teaching the laboratory during these semesters participated in the study as well.

Laboratory structure

The laboratory for the second semester general chemistry in our institution is part of the lecture course. It consists of two-three hours lab per week; where, one day students work in a wet lab and the other day in a virtual lab. The virtual lab consists of students performing virtual experiments on laptops together in a classroom. For the wet labs, a total of seven experiments are performed during the semester. Three of the wet experiments, the decomposition of methyl orange, the standardization of a sodium hydroxide solution, and the synthesis and characterization of aspirin, require students to write a formal laboratory report. These formal lab reports include an Introduction, Methods, Results, Discussion and Conclusion section. For the remaining four laboratories only a provided worksheet needs to be completed by the students. These worksheets are scaffolded to provide students the opportunity to analyze, interpret and reflect on the lab experience. The questions are short answers. Given that the three identified labs were more complicated and required formal lab reports, more time is spent on performing the experiments and working on these reports.

Preparation activities

The preparation activities produced for the lab involved both videos and questions posed on our Moodle-based learning management system (LMS). The preparation videos demonstrate how to use the lab equipment, what their data would look like, safety information, and proper lab techniques. Video lectures were only created for the decomposition of methyl orange and standardization of sodium hydroxide labs. These video lectures are particularly important for these complicated labs since students will have never performed these experiments before and have limited time to complete their data collection. The videos are hosted on YouTube and an example screenshot of one of the videos can be seen in Fig. 1.

Fig. 1 An example screenshot of the lab demonstration videos. The full playlist is available at https://www.youtube.com/playlist?list=PL_j40xIfCA31rCkGEuXNm5nSYdlDmIlVY.

The preparation activities were written for every wet lab session on our LMS. One example preparation question is shown in Fig. 2. The preparation questions focused on practicing the calculations that students would need to complete to analyze their data. They were instructed to write their solutions in their lab manual so that they would all have a template to work from in the lab for their collected data. For example, for the Standardization of Sodium Hydroxide lab, the preparation assignment led students through the process of initial determination of the concentration of NaOH from the mass of NaOH added to a measured amount of water, as well as refining that measurement by calculating the concentration from a titration against a weighed sample of potassium hydrogen phthalate, and finally determining the relative average deviation from several measurements of the concentration via titration. Each student had a unique set of questions available to them through the LMS and they were allowed to attempt the problems as many times as necessary in order to get the correct solution. To ensure that students completed the preparation assignment 10% of the score for the deliverable of the experiment was based on this preparation activity. Furthermore, instructors would not allow them to start that day's experiment without completing it.

Fig. 2 An example of the preparation questions for the Standardization of Sodium Hydroxide lab written into the local LMS designed to mirror the data analysis that students will encounter in the lab.

Surveys

Post-lab surveys. The post-lab surveys were developed by the authors and were administered to the students online through Qualtrics after every major lab report was submitted. The post-lab surveys were given to assess students’ perceptions on the effectiveness of the provided materials for their preparation for the lab. Students were asked about the usefulness of resources available to them and how prepared they felt to complete the lab experiment. Surveys developed by the authors were also administered to lab instructors who used the prelab materials to gain insights about the benefits they perceive the students get from interacting with the materials. Both surveys can be found in the ESI.†

MLLI. The Meaningful Learning in the Laboratory Instrument (MLLI) developed by (Galloway and Bretz, 2015) was used to assess students’ expectations in the lab in the cognitive and affective domains. The MLLI was administered online via Qualtrics at the start and end of the lab course.

Data analysis

Data from the MLLI survey was analyzed quantitatively using descriptive statistics. Only students who completed both, pre and post MLLI, were included on the analysis. Negatively stated items were recoded (Galloway and Bretz, 2015). Skewness and kurtosis were used to assess the normality of the results. Paired t-test was used for pre-post test comparison and to determine if the difference between pre and post scores were significant. Standardized effect size (Cohen's d) for paired samples was calculated to examine how meaningful are the differences observed. Cohen's d was used for effect size and values of 0.20 (small), 0.50 (medium), and 0.80 (large) were used to interpret the magnitude of the effect size (Cohen, 1988).

Data from post-lab surveys were analyzed qualitatively by looking at patterns obtained from students' and instructors’ responses. Frequency tables with patterns from the survey data were used to summarize these patterns.

Lab reports were also collected during the Spring 2017 semester for the three lab sections that participated in the study. Lab reports were graded by each instructor for their students’ grades; however, we wanted to determine if there was any difference in lab reports components when students had access to prelab materials. students’ lab report components were then evaluated using a rubric developed by the authors and the lab coordinator. The rubric was used to score each component of the lab report and each section in the rubric had specific details about each component and the lab experience. The scores were done independently by two raters. After scoring the lab reports, a comparison between two groups (lab section with access to prelab activities and lab section with no access) were performed to determine if there was a significant difference in any of the lab report components. A total of 45 lab reports were scored by the two graders, 13 reports from the lab section with access to prelab activities and 32 from the lab sections with no access. An independent sample t-test was used for significance testing. Assumptions for independence, normality of the scores distributions and equal variance were checked and no violation for the assumptions were found.

Analysis of quantitative and qualitative data was performed using SPSS v.25.

Results

Student perceptions

Spring 2017. In Spring 2017, three sections participated in the study (45 students). One section had access to the additional preparation activities (small section of 13 students). The other two sections did not have access to the additional preparation activities (each section had 16 students). The students in these three lab sections were all a part of the same lecture section of the course meaning that they all had the same lecture instructor. These three lab sections each had their own instructor who was different from the lecture instructor. The lab section who had access to the additional preparation activities was taught by one of the co-authors of this manuscript (MNG). All three sections were surveyed twice during the semester using post-lab surveys. These post-lab surveys have a similar set of questions but were administered after two different laboratory experiences. Both post-lab surveys were analyzed and findings are summarized below and in Fig. 3. After analysis, it was observed that the sections that did not have the preparation activities reported that the lab manual was the most helpful resource they used to prepare for the lab. The proportion of students reporting the lab manual are

Fig. 3 Percentage of students who mentioned the lab manual, the TA, the preparation questions, and the preparation videos in the survey question: “Of the provided materials for the experiment, which materials were the most helpful for you as you completed the tasks?” after each of the two labs with formal lab report deliverables (Kinetics of Methyl Orange and the Standardization of NaOH) in Spring 2017. Student responses from those with access to the preparation materials are separated from those without the preparation materials.

• During the first post-lab survey (after the decomposition of methyl orange experiment), 54% of respondents mentioned the lab manual

• During the second post-lab survey (after the standardization of a sodium hydroxide solution experiment), 72% of respondents mentioned the lab manual.

In the lab section provided with the additional preparation activities (videos and questions), the students reported that these activities were the most helpful resources they used to prepare for the lab. The proportion of students reporting the preparation activities are

• During the first post-lab survey, 54% of the students mentioned the preparation videos; while 38% mentioned the preparation questions as the most useful resource.

• During the second post-lab survey (after the standardization of a sodium hydroxide solution experiment), 42% of the students mentioned the preparation videos; while 38% mentioned the preparation questions.

• In both of these post-lab surveys, the lab manual was only mentioned 15%, and 33% of the time respectively.

Based on the post-lab survey data, it appears that students rely mostly on a lab manual to prepare for the lab experience, but if provided with additional resources, such as the videos and questions, they will find it useful since these preparation activities could help them to visualize what they will accomplish in the lab and practice how they will analyze their data. These results are visualized in Fig. 3. Even though it is very exciting that the students that had access to the preparation activities found them useful, the percentages are based on a very small group of students, 13 and 12 students responses to the first and second post-lab survey, respectively; thus the percentages and the differences seems bigger than they actually are. Therefore it was important to see if more students had access to the preparation activities if they would also find them useful, so we collected more data in the following semester (discussed later: Fall 2017 and Spring 2018).

This more robust set of preparation activities seems to help students when writing Abstract, Introduction, and Conclusion sections of their formal lab report. Based on the common rubric, for the Abstract section students with access to the preparation activities scored 2.92 ± 0.35 out of 4 possible points and students without access to the activities 2.19 ± 0.15. For the Introduction section written by students with access to the additional preparation activities were scored as 5.85 ± 0.48 out of 8 possible points while the students that did not have access scored 4.94 ± 0.27. Additionally, the Conclusion section written by students with access to the additional preparation activities were scored as 3.00 ± 0.30 out of 4 possible points while the students that did not have access scored 2.03 ± 0.20. However, after performing an independent t-test for each of the components, we observed that there was not a significant difference after adjusting the p value to 0.008. The scores for the lab report component are reported in Table 1.

Table 1 Average scores and standard deviation of the mean from the Kinetics of Methyl Orange lab reports from students in Spring 2017 broken down by section. What each section is scored out of is indicated beside the section heading

	Without PM (N = 32)	With PM (N = 13)
Abstract (/4)	2.19 ± 0.15	2.92 ± 0.35
Introduction (/8)	4.94 ± 0.27	5.85 ± 0.48
Methods (/4)	3.41 ± 0.14	3.38 ± 0.27
Results (/8)	7.41 ± 0.43	7.85 ± 0.80
Discussion (/8)	4.25 ± 0.23	4.46 ± 0.51
Conclusion (/4)	2.03 ± 0.20	3.00 ± 0.30

---

Fall 2017. In Fall 2017, 68 students completed the post-lab survey from several lab sections. The post-lab survey was given at the end of the semester. All the lab sections had access to the additional preparation activities, however, it was up to the instructors to decide how much they wanted to use them. In the post-lab survey we asked students to rank the materials that they found the most useful with regard to their preparation for the lab and the results are presented in Fig. 4a. 57% of respondents ranked the instructor as the most important resource (average rank of 1.74) followed by 31% of students reporting that the lab manual was the most useful (average rank of 2.26). The third most useful resource was the preparation videos even though they were not universally applied. This was followed by the lecture materials, the prelab questions and friends.

Fig. 4 (a) The average ranking students assigned to the lab manual, instructor, preparation videos, and preparation questions according to their usefulness in helping them prepare for the lab where 1 was most useful. (b) Percentage of students who mentioned the lab manual, the instructor, the preparation questions, and the preparation videos in response to the question “Please discuss why you felt that the provided material was enough for you to complete the tasks.” These responses were collected in surveys administered to students in the Fall 2017 (n = 68) and Spring 2018 (n = 71) semesters.

When the students were asked to clarify why they made their top two rankings, they responded by saying that the instructor was able to clearly explain what they needed to accomplish in the lab and answer any questions that they had while the lab manual provided a step-by-step set of instructions that was easy to follow. For example, one student remarked:

“The lab instructor is very helpful in helping us understand what is the purpose of the lab and what are our goals. The lab manual has all the steps listed and gives all the specific amounts.”

while another commented:

“[The instructor] explains everything in detail and is very knowledgeable. The Lab Manual works well for me because I like to follow detailed instructions as I go.”

The preparation videos were ranked third highest with 7% of students saying it was the most important resource in their preparation (average rank of 4.00). In most cases, the students’ comments focused on how the videos complemented the lab manual and the instructor by visualizing what both were describing. For example, one student commented that the lab manual and the preparation videos:

“were the most useful because [the] lab manual gave a step by step procedure of what was needed to be done. YouTube Videos help visually how to do things.”

Another student commented:

“I think that the lab instructor provides a well preparation of telling the students on what is going to be on the lab and its safety hazards. The lab instructor would guide the students through any calculations or procedures that maybe the lab manual has failed to have the students understand. I think that the YouTube videos help prepare for the lab too because there are students who are visual learners and that they might want to know how the lab would be executed by seeing how the lab is played out, from there, students might be able to understand the lab more and become more prepared before coming to class.”

Based on these responses, it suggests that the preparation videos complement a reference document and a strong instructor.

What is interesting about the top three rankings is that in a separate question when students were asked an open-ended question on why they felt the provided material was enough to complete the tasks a majority of the students identified the preparation videos as a major component. This is illustrated in Fig. 4b. In total, 54% of students commented on how the preparation videos were helpful frequently citing that being able to visualize the procedure as extremely useful. For example, one student commented:

“The youtube videos allowed me to visualize what needed to be done in the lab. Because I was able to visualize it, I was able to use my time effectively and efficiently during class.”

The lab manual was still mentioned by 43% of students and the lab instructor was mentioned by 22% of respondents. This demonstrates that even though students recognize having a strong instructor and clear lab manual are important to complete the lab, they value having a visual demonstration of the lab as a part of their preparation.

The preparation questions administered through the LMS featured much less prevalently in the student responses. When students were asked to rank the usefulness of each intervention, the preparation questions ranked 5th out of 8 options (average rank of 5.24; lecture materials was ranked 4th). Some students who mention the prelab questions acknowledge their value:

“All of the prelab questions on Titanium were applicable to the concepts and calculations that needed to be carried out in the actual lab itself. Because there were unlimited attempts I felt like the purpose of it was to make sure I actually understood what was going on, not to just give me busy work for points. Lastly, seeing parts of the lab demonstrated via the youtube videos, I felt confident going into lab knowing how to carry out the different procedures.”

However, the vast majority of the responses that discussed the prelab questions said that they poorly ranked them because they did not use them. Given that they were not universally administered by instructors this is probably a significant contributor for the lower ranking of these materials.

Spring 2018. In Spring 2018, 71 students completed the lab surveys from four different lab instructors. All the lab sections had access to all the additional preparation materials through provided LMS templates. In the survey we again asked students to rank the materials that they found the most useful with regard to their preparation for the lab and the results are presented in Fig. 4a. The ranking of number 1 votes is as follows: 52% of students ranked the lab manual as the most useful (average rank of 2.14), 17% of students ranked the lab instructor as the most useful (average rank of 3.01), 11% of students found their friends the most useful resource (average rank of 3.89), 10% of students ranked the preparation videos as the most useful (average rank of 4.03) and finally 4% of students ranked preparation questions as the most useful (average rank of 4.41).

Students were asked to explain why those resources were the most useful and most students think that the lab manual provided detailed information about the experiment; while instructors were always willing to help and clarify any doubts during the experiment. For example, one student remarked

“the lab manual was very easy to read and going through the background and procedure provided clear instructions as to what was expected for the following lab”

while another student commented:

“The lab manual is well written so that the student is prepared, but if not the instructor is more than willing to explain the topics and the procedure of the lab if the student does not understand.”

In both cases, we can see why students thought the lab manual and instructor were the most helpful. A surprising finding was that the third most useful resource was friends. Most students explained that they rely on friends when they have questions or for clarification since they feel more comfortable. One student expressed this idea very clearly,

“since friends taking the same class are learning the same material, if I get confused on a concept or how to perform a task, I could ask them for clarification. I also find them easier to approach than lab instructors due to convenience and familiarity.”

The Spring 2018 survey also included the separate, open-ended question to discuss why they felt the provided material was enough for them to complete the tasks. Again, the number one mentioned item was the preparation videos which is illustrated in Fig. 4b. In total, 32% of students commented on how the preparation videos were helpful frequently again citing that being able to visualize the procedure as extremely useful. For example, one student commented:

“The resources provided to explain the procedures of this lab experiment were extremely helpful because the explanations were delivered in different form. The best form that helped me complete the tasks were the videos on Youtube because I could see exactly what needed to be done.”

What is interesting about this student response is that they ultimately ranked the lab manual as the most useful material to prepare for the lab. In this separate question, the lab manual was also mentioned by 27% of students and the lab instructor was mentioned by 20% of respondents. This demonstrates that even though students recognize having a strong instructor and clear lab manual are important to complete the lab, they value having a visual demonstration of the lab as a part of their preparation.

4% of students also ranked the lecture materials as the most useful, but the average ranking was 5.14 implying that there were a couple of students who found the lecture materials useful, but a larger fraction did not compared to the preparation questions. Students expressed in various instances that the lecture for the course was not “aligned” to the lab. Sometimes the concepts covered in lab have not yet been presented in lecture. One student summarized this point as:

“Most of the time the lecture materials did not correspond to the lab of the time. either the professor would not talk about the lab or would be behind and not cover lab techniques until after the lab was completed.”

In addition, we asked students what improvements they would suggest for the lab. Some of the students suggested giving more example problems/calculations before the preparation questions and giving feedback when the answer was wrong. Other students expressed that lecture and lab should be better aligned, for example one student explained that

“an improvement would be that we should align prelab to lecture more coherently because sometimes I have trouble doing the prelab because I barely or didn't even learn the new material in lecture.”

Student expectations

Students’ expectations were assessed at the beginning and at the end of the semester to determine if their lab expectations were met after a semester of instruction. Students lab expectations were measured using the MLLI and only students that completed both surveys were used in the comparison. Table 2 presents the results by scales for the Spring 2017 data set. The largest and significant difference can be observed for the cognitive scale with a medium effect size (Cohen's d = −0.58). Interestingly, the difference is negative which means that students’ expectations are not met at the end of the semester. For the cognitive/affective scale the difference is negative but smaller (Cohen's d = −0.10) and for the affective, the difference is smaller with an effect size of 0.01. Both of those differences are not significant. The same trend is observed for the Spring 2018 data set which is also included in Table 2.

Table 2 Students expectations from MLLI surveys in Spring 2017 (N = 20) and Spring 2018 (N = 34)

Spring 2017
Scales	Mean		Difference	Cohen's d effect size
	Pre	Post
The asterisk (*) denotes that the difference is significantly different at p < 0.02. The double asterisk (**) denotes a decrease in scores.
Cognitive	70.8	61.3	−9.5*	−0.58**
Affective	62.8	62.6	0.2	0.01
Cog/Aff	61.6	59.4	−2.2	−0.10**

---

Spring 2018
Scales	Mean		Difference	Cohen's d effect size
	Pre	Post
Cognitive	71.8	62.6	−9.2*	−0.73**
Affective	55.7	56.8	1.1	0.058
Cog/Aff	55.6	48.6	−7.0	−0.45**

---

This drop in students’ expectations after a semester of lab instruction was also found in previous research studies where a decrease in students’ expectations in the post MLLI surveys were observed (Galloway and Bretz, 2015; Galloway et al., 2015; Schmidt-McCormack et al., 2017). This drop was expected to happen since students at the beginning of the semester usually have higher expectations; however, once the semester get busy, students mostly get overwhelmed with the new materials and start to worry about the technical details and forget the actual concepts behind the presented laboratory tasks.

For Spring 2017 data, one interesting finding was that students that had access to the prelab activities had a significant difference in their expectations when asked about being confused about how instruments work. In Fig. 5, we can observe that students with the prelab activities (intervention) scored significantly lower; which meant that students were less confused when working with instruments than they expected. For students that did not have access to the prelab activities, the difference between pre and post was not significantly different; which also suggests that students were still confused when working with the instruments even after performing the experiment. This suggests that the prelab activities help students in their technical skills for the lab experiments.

Fig. 5 Results in the Spring 2017 data set for question 6 on the MLLI: …to be confused about how the instruments work. This is a Cognitive domain coded question where the scale is the percent the student is in agreement with the statement.

Instructor surveys

The preparation materials were developed by two of the co-authors (SA and MNG) and the lab coordinator. Besides MNG and the lab coordinator, none of the surveyed instructors who implemented the preparation materials were involved with creating them. Given that each instructor is provided the latitude to approach presenting the lab course material however they choose, it is expected that there will be a range of opinions on how effective these preparation materials are as some instructors would have their own preferences on what/how to present the learning objectives.

The instructors surveyed unanimously agreed that the value of both the preparation questions and preparation videos was that the students whose sections had access to the interventions generally came to the lab better prepared. When asked about the value of the additional preparation activities, the TAs responded that the students demonstrated a greater degree of understanding of the experimental procedure and data analysis. Furthermore, one instructor commented on how preparation questions helped them with follow up questions from the students since they could refer back to those questions and quickly address the student's gap in knowledge. The same instructor also remarked that students also took more ownership over their learning of course materials. This probably stems from being given an unlimited number of attempts for each problem which allows for students to keep trying until they get the question correct. The instructors surveyed had a positive perception of the effectiveness of the materials on student preparation.

The instructor survey also asked TAs about how the additional preparation activities benefit them personally and for new TAs to the course. The majority of instructors identified that the preparation materials also help them prepare for the lab session by introducing what the day's activity will look like and remind them on how to do the data analysis. Another instructor also remarked that they found the materials to help them prepare a more effective in-lab lecture given that the students would have already been exposed to the material. These comments indicate that the preparation materials not only prepare students, but also the instructors, new, or otherwise, to be effective in the lab.

Finally, when the instructors were asked about improvements to the preparation materials, they indicated that the preparation material needs to be focused and to the point. They remarked that if there were too many preparation questions that the students would tend to feel frustrated having to complete all of them. Furthermore, the learning management system uses a tolerance to determine if a student's answer is close enough to the correct answer to be graded as a correct solution. Sometimes students will follow the correct procedure to calculate the answer, but still generate a final answer that is outside of the tolerance for the correct solution. This could arise from rounding their number mid-calculation. This leads to further frustration for the students given that a human evaluator would be more forgiving. Getting these tolerances right takes some time but the automated nature of the grading process is important for the student to get immediate feedback so that they can continue to work towards a correct solution. Communication between the instructors and the lab coordinator who manages the question bank can help alleviate these issues.

Discussion

In regards to the resources typically offered in a general chemistry lab setting, it is clear that the students appreciated having a written reference document as well as having someone to help them interpret and understand lab related materials. Even when additional preparation activities are offered, students still greatly rely on the resources that are traditionally offered. Regardless of if additional preparation activities are offered, lab manuals should be continually refined and lab instructors be trained so that students have access to high-quality manuals and instructors to help foster a positive and well-structured experience.

It is clear, however, that the additional preparation activities are well received by instructors and students. The additional preparation activities can help to reduce the anxiety students face when needing to perform complicated tasks under a time constraint. As one student reported:

“For me, the Youtube videos helped out the most because it eased the anxiety of working with the expensive equipment; Usually I would get nervous if I would damage the equipment by not handling it carefully.”

This is important so that students feel comfortable approaching laboratory topics and equipment so that they have the confidence to successfully engage with the material.

The results from the Spring 2017 lab report analysis indicates that the additional preparation activities have a positive effect on providing students an effective overview of the lab as well as describing the background topics and main results to be reported from the lab. The students are instructed to write in the introduction of their lab reports the purpose of the lab, any background information related to the chemistry topics explored in the lab, and the equipment they will use. Additionally, in their conclusions they are instructed to restate the purpose of the lab and present the main results that demonstrate that this purpose was achieved. Finally, in their abstracts, students are asked to write the purpose of their work, how they measured their results, the final result and link that result back to the purpose of the lab. The analysis of the lab reports indicate that students who had access to the additional preparation activities were able to better articulate these topics in their report. While it is possible that there is an instructor related effect given that there were three different instructors involved, it is likely that the intervention materials still played a role. This is based on how the section with the additional preparation activities showed a much lower difference in scores from the control sections by the graders for the other sections of the formal lab report (Methods, Results, Discussion). This is interesting because the preparation videos and preparation questions showed what their results from the experiment might look like, and mimic what they would do in the data analysis. It would follow that there should be a difference in the students’ ability to present their results and discuss the meaning of their results. However, these results indicate that other interventions are probably necessary to help students with these aspects of the laboratory experience since the additional preparation activities did not appear to change the student's ability to think critically about what their results mean.

Additionally, having these materials does not prevent lab instructors from doing a prelab lecture to enhance student understanding. It is never envisioned that these additional preparation activities will replace lab TAs; they are only meant to enhance student preparation leading into the lab. However, one thing that these preparation materials can help with is standardizing the prelab lecture so that all students receive the same information. This may help students with instructors who do not provide the necessary direction since they are supplemented by these additional preparation activities to help them complete their lab work.

Furthermore, the second semester general chemistry course at this institution is offered as a coupled lecture and lab course. This creates issues with synchronization between the lecture and lab where students are supposed to learn material in the lecture and then apply it in the lab. Frequently, the lecture falls behind the lab schedule so students are faced with performing labs on topics they have yet to cover in the lecture. Our results documented this student frustration. The additional preparation activities can help solve this issue by providing basic instruction on the topics necessary to complete the lab in addition to an overview of the lab. This approach has been reported by Schmidt-McCormack et al. (2017) for analytical labs where students are doing rotation of lab experiments and prelab lectures by the instructors are not feasible. Although in our case, all students are performing the same experiment, students, similar to the Schmidt-McCormack et al. (2017) study, have the opportunity to have more autonomy over their learning, be better prepared to complete their lab, and have a better understanding and perception about their lab experience.

Finally, one obstacle to the effective use of these enhanced materials is having the support of the TAs to utilize them. Without instructor buy-in, students will not be compelled to use them thereby reducing their influence. This seems to be especially true for the preparation questions as students tend to gravitate to watching the videos on their own, however, they are less inclined to perform math based questions that simulate data analysis prior to performing the lab. One way that was shown to be effective towards a 100% completion rate of the preparation questions is to require that they successfully complete all the questions prior to being able to start the lab. This allows students the freedom to work on this activity prior to the lab, or get help in real time during the lab with the instructor's help while the other students set up. The value of the preparation questions is that students then have a completely accurate template to follow in order to analyze their data which dramatically helps them complete the lab on time.

Conclusion

This study discusses the effectiveness and usefulness of using preparation activities, online videos and preparation questions, to help students be better prepared during their laboratory experience in a second semester general chemistry laboratory. Data was collected across three different semesters and students’ perceptions on the usefulness of the prelab resources and their lab expectations were assessed. Three research questions guided this study. For the first research question, we assessed students’ perception on the usefulness of prelab activities and resources available to them. We establish that students found the prelab activities useful when they had accessed to them, but they still found lab manual and instructors to be a reliable source when preparing for their lab experience. For the second research question, students’ expectations were assessed at the beginning and at the end of the semester using the MLLI survey. Findings suggested that students’ expectations decreased in the cognitive and cognitive/affective domain, but they increased slightly in the affective domain. The last research question assessed instructor insights about students’ benefits of using the prelab activities. Instructors believed that students using the prelab activities were better prepared for the lab experience as well as they observed that students demonstrated a greater degree of understanding of the experimental procedure and data analysis.

In general, we found that both students and instructors see value and benefits in having these types of materials available for them as a way to help prepare students for the upcoming laboratory sessions. If students are better prepared, they will most likely understand better the concepts behind the experience and will be able to successfully complete the lab. These potential benefits highlight the importance of providing students with resources that they can use and rely on when preparing and working on the laboratory. At the end, the time to plan, film, edit the lecture videos as well as create preparation questions on the LMS in most cases are minimal compared to the value they generate over multiple years in a course that serves a large number of students.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

The authors would like to thank Beena Matthew, the second semester general chemistry lab coordinator at CSUF, for her help with making some of the preparation videos, building lab report rubrics and for having the flexibility and patience for us to perform this work during live lab sessions. The authors would also like to thank Dr Julia Chan (CSUF) for her insightful comments while preparing the manuscript.

References

1. Agustian H. Y. and Seery M. K., (2017), Reasserting the role of pre-laboratory activities in chemistry education: a proposed framework for their design, Chem. Educ. Res. Pract., 18, 518–532.
2. Alsharif N. Z. and Henriksen B., (2009), Electronic integration of prerequisite course content, Am. J. Pharm. Educ., 73(8), 150.
3. Box M. C., Dunnagan C. L., Hirsh L. A. S., Cherry C. R., Christianson K. A., Gibson R. J., Wolfe M. I. and Gallardo-Williams M. T., (2017), Qualitative and quantitative evaluation of three types of student-generated videos as instructional support in organic chemistry laboratories, J. Chem. Educ., 94(2), 164–170.
4. Bretz S. L., (2019), Evidence for the importance of laboratory courses, J. Chem. Educ., 96(2), 193–195.
5. Chittleborough G. D., Treagust D. F. and Mocerino M., (2007), Achieving greater feedback and flexibility using online pre-laboratory exercises with non-major chemistry students, J. Chem. Educ., 84(5), 884.
6. Cohen J., (1988), Statistical power analysis for the behavioral sciences, 2nd edn, Erlbaum: Hillsdale, NJ.
7. DeKorver B. K. and Towns M. H., (2015), General chemistry students’ goals for chemistry laboratory coursework, J. Chem. Educ., 92(12), 2031–2037.
8. Galloway K. R. and Bretz S. L., (2015), Development of an assessment tool to measure students’ meaningful learning in the undergraduate chemistry laboratory, J. Chem. Educ., 92(7), 1149–1158.
9. Galloway K. R., Malakpa Z. and Bretz S. L., (2015), Investigating affective experiences in the undergraduate chemistry laboratory: Students’ perceptions of control and responsibility, J. Chem. Educ., 93(2), 227–238.
10. Hofstein A. and Lunetta V. N., (2004), The laboratory in science education: Foundations for the twenty-first century, Sci. Educ., 88(1), 28–54.
11. Högström P., Ottander C. and Benckert S., (2010), Lab work and learning in secondary school chemistry: The importance of teacher and student interaction, Res. Sci. Educ., 40(4), 505–523.
12. Johnstone A., Sleet R. and Vianna J., (1994), An information processing model of learning: Its application to an undergraduate laboratory course in chemistry, Stud. High. Educ., 19(1), 77–87.
13. Jolley D. F., Wilson S. R., Kelso C., O’Brien G. and Mason C. E., (2016), Analytical thinking, analytical action: Using prelab video demonstrations and e-quizzes to improve undergraduate preparedness for analytical chemistry practical classes, J. Chem. Educ., 93(11), 1855–1862.
14. Jones B. D., (2010), An examination of motivation model components in face-to-face and online instruction, Electron. J. Res. Educ. Psychol., 8(3), 915–944.
15. Reid N. and Shah I., (2007), The role of laboratory work in university chemistry, Chem. Educ. Res. Pract., 8(2), 172–185.
16. Reynders G., Suh E., Cole R. S. and Sansom R. L., (2019), Developing student process skills in a general chemistry laboratory, J. Chem. Educ., 96(10), 2109–2119.
17. Rollnick M., Zwane S., Staskun M., Lotz S. and Green G., (2001), Improving pre-laboratory preparation of first year university chemistry students, Int. J. Sci. Educ., 23(10), 1053–1071.
18. Sansom R. and Walker J. P., (2019), Investing in laboratory courses, J. Chem. Educ., 96(2), 193–195.
19. Schmidt-McCormack J. A., Muniz M. N., Keuter E. C., Shaw S. K. and Cole R. S., (2017), Design and implementation of instructional videos for upper-division undergraduate laboratory courses, Chem. Educ. Res. Pract., 18, 749–762.
20. Seery M. K., (2020), Establishing the laboratory as the place to learn how to do chemistry, J. Chem. Educ., 97(6), 1511–1514.
21. Seng L. and Mohamad F. S., (2002), Online learning: Is it meant for science courses? Internet High. Educ., 5(2), 109–118.
22. Teo T., Tan K. C. D., Yan Y. K., Teo Y. C. and Yeo L. W., (2014), How flip teaching supports undergraduate chemistry laboratory learning, Chem. Educ. Res. Pract., 15, 550–567.
23. Vician C. and Charlesworth P., (2003), Leveraging technology for chemical sciences education: an early assessment of webct usage in first-year chemistry courses, J. Chem. Educ., 80(11), 1333–1337.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: 10.1039/d0rp00240b

---