Ian Hawkins*a and
Amy J. Phelpsb
aWelch College, 3606 West End Avenue, Nashville, TN 37205, USA. E-mail: ihawkins@welch.edu; Fax: +1 615-844-5004; Tel: +1 615-844-5262
bMiddle Tennessee State University, Department of Chemistry, PO Box 68, Murfreesboro, TN 37132, USA. E-mail: amy.phelps@mtsu.edu; Fax: +1 615-898-5182; Tel: +1 615-898-2077
First published on 22nd July 2013
The use of virtual laboratories has become an increasing issue regarding science laboratories due to the increasing cost of hands-on laboratories, and the increase in distance education. Recent studies have looked at the use of virtual tools for laboratory to be used as supplements to the regular hands-on laboratories but many virtual tools have not been used as replacements. In order to understand the effects of virtual laboratory methods as well as ascertain the effectiveness of our normal lab procedures, we used a virtual versus hands-on scenario for an electrochemistry laboratory. We separated our General Chemistry II Lab students into two groups. Each group completed an electrochemistry lab either through our normal hands-on format or with the use of a virtual lab animation created by the Iowa State Education Group. Pre- and post-test data for conceptual and factual understanding were collected as well as a post-test hands-on setup of an electrochemical cell. There were no significant differences in scores on either the pre-test, post-test, or the hands-on setup test. However, individual item analysis on the tests revealed a significant difference in the use of the salt bridge during the post-test and the hands-on setup activity.
Recent changes and developments in educational delivery particularly in the area of technology have led to questions regarding how best to design chemistry instruction (Casanova et al., 2006). Since distance education and online schools have become a prominent option at both the high school and especially college levels, new ways to teach and deliver content have become inevitable. This has led some to consider computer simulations as viable options for laboratory education. The use of computers to enhance or even replace certain laboratory activities is not new, and many laboratories use computers at a number of different levels to both analyze and even create data (Perone, 1970; Davis et al., 1973; Wilkins, 1975; Cavin et al., 1978; Krause, 1988; Jones, 2000). However, there has been skepticism regarding the use of these tools as a replacement for hands-on laboratory experiences and both the American Chemical Society and the College Board have decided to endorse the hands-on laboratory activities as preferred over the virtual laboratories for chemistry majors at the college level and for high school Advanced Placement classes respectively (Dillon, 2006; American Chemical Society, 2011; College Board, 2013). Any attempt to compare hands-on laboratories and virtual laboratories are stymied due to the lack of sufficient research in the area of virtual or computer simulated labs as well as traditional hands-on laboratory (Hofstein and Lunetta, 2004; Ma and Nickerson, 2006; Honey and Hilton, 2011). Also many of the research studies look at different purposes and different content making it difficult to deduce the overall effectiveness of simulations as a whole (Woodfield et al., 2004; Josephsen and Kristensen, 2006; Rodrigues, 2007, 2011). Many simulations have also been created to deal with specific problems in laboratory and lecture such as misconceptions and connections between the particulate, macroscopic, and symbolic worlds in the laboratory (Sanger and Greenbowe, 1997a, 1997b, 1999; Akaygun and Jones, 2013). Recent studies, using new computer simulations mainly as supplements, have shown some evidence that these labs can be helpful in increasing lecture test scores, enhancing students' attitudes, improving preparedness for hands-on lab, and strengthening conceptual knowledge (Yaron et al., 2003; Woodfield et al., 2004; Woodfield et al., 2005; Liu, 2006; Dalgarno et al., 2009). They have been studied as both pre-lab preparations and post-lab reviews (Woodfield et al., 2004; Burewicz and Miranowicz, 2006; Supasorn et al., 2008; Limniou et al., 2009). Since many of these virtual simulations are done in addition to the normal hands-on labs, these improvements may be mainly due to more time on task. A review of several studies using virtual labs in different science classes indicated that the results were varied in regards to the efficacy of virtual labs. The lack of a consistent message when looking at virtual laboratories is difficult to achieve at least in part because of the different goals that each research project focused on (Ma and Nickerson, 2006). One of the primary concerns regarding virtual laboratories was the inability to teach laboratory techniques and one of the advantages of virtual laboratories appeared to be the ability to help students properly understand the concepts in chemistry by allowing students to visualize the particulate nature of chemistry.
One study highlighting the use of virtual lab simulations as a tool to teach chemistry concepts comes from the lab of Dr Thomas J. Greenbowe at Iowa State University. Several of Dr Greenbowe's students have worked to understand the issues and misconceptions students have with electrochemistry. Dr Michael Sanger conducted several research projects as part of Dr Greenbowe's research group to identify student misconceptions and determine what language and ideas in the instructional materials and textbooks that contributed to the students' misconceptions (Sanger and Greenbowe, 1997a, 1997b, 1999). Electrochemistry is difficult due to complex terminology and the lack of students' connections between the macroscopic, sub-microscopic, and the symbolic natures of the concept. In the 1990's, Greenbowe's research group developed some simple animations to help students understand and view the particulate (sub-microscopic) aspect of chemistry and its implications for electrochemistry (Greenbowe et al., 1995; Greenbowe, 1997). A few years later Burke and Liu continued the work with animations and used them to increase students' conceptual knowledge as well as understand the psychological differences between computer simulations and hands-on laboratories. (Burke et al., 1998; Liu, 2005). These simulations were used in lecture to help students visualize the particulate view of matter and were successful in increasing students' performance on quizzes designed to test their concept knowledge (Sanger and Greenbowe, 1997a, 1997b). Later these animations and others were developed for web access so that students could view these out of class. Many of these virtual simulations were designed with worksheets to help the students work through the concepts. These lab simulations covered several areas of electrochemistry including: Redox reactions, voltaic cells, concentration cells, and electrolysis cells (Chemical Education Group Iowa State University, 2005). Using several of these animations, we put together a virtual lab to help develop students' understanding of the concepts of electrochemistry including how to set-up a voltaic cell.
In order to test the efficacy of this virtual lab as a replacement for the hands-on laboratory normally used in General Chemistry II, students were randomly separated into two groups based on their laboratory section to perform either the hands-on lab on electrochemistry or the electrochemistry virtual lab simulation. The students were given both a pre- and post-test on conceptual knowledge and a final test on the hands-on setup of a voltaic cell.
Several weeks prior to the laboratory, students were given information about the research project and each student included in the study signed an informed consent. Six of the lab sections were chosen to complete the virtual lab. Each lab section completed either the virtual lab or the hands-on lab. The experimental group completed the virtual simulations in the departmental computer lab. Each virtual lab student turned in a lab report consisting of the worksheets. All students were required to complete the lab they were assigned to after completing a pre-test and providing some demographic information and a survey of their opinions regarding technology. The students completed the assigned laboratory in groups of two or three. These were the same lab groups they had been working in throughout the semester. The laboratory teaching assistants (TAs) were present during their laboratories as usual. Two weeks later, the students completed the normal final exam for the laboratory portion of the course and were given the post-test (same as the pre-test) after completing the final exam. Students were also given a hands-on assessment at the end of this lab exam time where they were asked to set up an electrochemical cell with the maximum possible voltage with the materials they were given. In order to minimize lack of effort on the pre- and post-tests, extra credit was given to individuals for completing the pre-test and for correct answers on the post-test. The pre- and post-tests were designed with 5 questions (Table 1). These questions focused on the zinc and copper reaction because both the hands-on and virtual labs used this cell as their model. Even though there were slight differences in the labs the material on the pre- and post-test was dealt with in the two laboratory activities.
Pre- and post-test questions |
1. You have a solution of copper ions (Cu2+) and when you place a strip of solid zinc in the solution you notice a copper color appearing on the piece of zinc. |
(a) Which metal is being oxidized? |
(b) Which metal is being displaced? |
2. A voltaic cell is created using zinc metal and zinc ions in one half cell and copper metal and copper ions in the other half-cell. |
(a) Which metal is the anode? |
(b) Write each half reaction and the overall reaction. |
(c) Draw a diagram of the voltaic cell and label the solutions, the electrodes, and the flow of electrons. |
3. What would happen to the reaction above if I increased the concentration of the zinc ion in the overall equations above? |
4. If we took the voltaic cell of zinc and copper from above and we added an external source of electricity, what would eventually happen as we increased the voltage of the external source? |
Hands-on activity question |
Using the supplies given to you (iron metal, copper metal, tin metal, copper sulfate, iron chloride, U-tube, soaked filter paper, anode label, and cathode label), construct a voltaic cell that would have the greatest voltage and label the cathode and anode. When you are done I will give you a voltmeter to test your results. |
In order to evaluate the students' answers to the pre- and post-tests, points were assigned to the individual parts of each question. For the first three questions, students were given one point for the correct answer. In the written equations, students were given points for correct reactants, correct products, correct states of matter symbols, correct direction of reaction, and having electrons on the proper side of the equation. For the overall reaction, the points were given similarly but one point was given for correctly canceling out the electrons in the equation. In the voltaic cell diagram, students earned points for having the correct labels on each electrode and each solution, having two separate solutions, having the solutions and metals matched correctly, having a salt bridge, having a wire connecting the metal electrodes, and labeling the correct direction of the electron movement in the wire. The equilibrium question was graded on a scale where two points were given to the students who identified the correct shift in equilibrium, one and a half points to students who identified one change that would occur but did not mention shift in equilibrium, one point for students who mentioned equilibrium but did not mention the direction of shift or students who had contradictory statements, a half of a point to students who shifted in the wrong direction, and zero points for all other answers. The electrolysis question was awarded one point for mentioning that the reaction is reversed when the power source overcame the cell voltage. The grading was designed to be as objective as possible and the tests were scored as one large set without regard for which lab group they participated in. The hands-on task was graded on five parts including anode cathode labels, correct metals and solutions, and the correct voltage determination. Using this point distribution each person could score thirty-one points on the pre- and post-test and five points for the hands-on task. In the end, data was collected for 84 students who completed the virtual lab, and 85 students who completed the hands-on lab.
Since our questions proved reliable, we proceeded to compare the pre-tests scores from each group. The t-test revealed that there was no significant difference between the pre-test scores for virtual lab students (N = 84, M = 9.04, SD = 6.60) and for hands-on lab students (N = 85, M = 9.77, SD = 6.77), t(167) = 7.12, p = 0.478 (Table 2) indicating that the groups were not significantly different in there understanding of electrochemistry prior to participating in the laboratory activities. The groups were then compared using a repeated measures ANOVA to see if the groups significantly increased their scores on the test from pre to post (Fig. 1). Students' scores were significantly higher on the post-test (virtual M = 16.84, SD = 6.69, hands-on M = 17.03, SD = 6.51) than the pre-test F(1,167) = 241.427, p < 0.001 (Table 2, TIME). This significant difference indicates that the students in both laboratories gained in their understanding of electrochemistry.
Measures | Virtual | Hands-on | t-Test | |||
---|---|---|---|---|---|---|
Mean | N | Mean | N | T-score | Sig. | |
a α = 0.05. | ||||||
Pre-test | 9.04 | 84 | 9.77 | 85 | 0.712 | 0.478 |
Post-test | 16.84 | 84 | 17.03 | 85 | 0.190 | 0.849 |
Hands-on | 2.71 | 84 | 3.14 | 79 | 1.702 | 0.091 |
Post–pre | 7.80 | 84 | 7.26 | 85 | −0.555 | 0.579 |
Repeated measures anova | |||
---|---|---|---|
F | Sig. | ||
Time × lab | All four groups | 0.309 | 0.599 |
Lab | Virtual vs. hands-on | 0.264 | 0.608 |
Time | Post vs. pre | 241.43 | <0.001 |
Fig. 1 Pre-test (1) and post-test (2) scores of hands-on labs (blue) and virtual labs (green). |
Notice, however, that while the scores did significantly increase they still scored about 60% on the average, only scoring 16–17 out of 27, which is not the increase one might hope for in their students. However, students in the two lab groups were not significantly different on their post-test scores t(167) = 0.190, p = 0.849 (Table 2) when compared to each other. Each group significantly increased their scores between the pre-test and post-test similarly as indicated by the graph in Fig. 1. The repeated measures ANOVA also showed that there was not a significant difference between the gains of both groups between the pre- and post-tests F(1,167) = 0.309, p = 0.599 (Table 2, time × lab). This was also confirmed by the t-test results of the post-tests minus the pre-tests for each lab group t(167) = −0.555, p = 0.579 (Table 2). The lab groups were also not significantly different from one another when averaging all the test scores for both pre-tests and post-tests (Table 2, lab).
Since the students in lab were separated into different lecture groups, we wondered if the lecture group had any effect on their pre- and post-test scores. We ran an ANOVA to compare the differences that might arise based on the lecture section (Fig. 2). The lecture groups had significantly different scores for both the pre-test, F(5,163) = 7.673, p < 0.001, and post-test, F(5,163) = 8.199, p < 0.001. This difference could be due to the fact that students in the various sections have different levels of prior knowledge about electrochemistry. Since lab is carried over the course of a week, there are students who would have had lectures on electrochemistry prior to taking the pre-test and other students who would not have had any formal electrochemistry presentation prior to participating in the laboratory. This is one explanation for the differences in the pre-test scores. However when you look at the differences in gains, it is interesting to note that some lecture groups show significantly higher gains than others (see Fig. 2) indicating that the lecture group a student is in could have a profound impact on a student's understanding of electrochemistry. While this is interesting data it clearly needs further investigation to completely explain these observed differences.
Fig. 2 Pre-test (1) and post-test (2) scores of the lecture sections (colors). |
One of the criticisms of virtual labs is the fact that they do not teach laboratory techniques and manipulative skills well. We included the hands-on set-up to see if there was a difference in the two laboratory methods in teaching manipulative (hands-on) skills although this lab does not require extensive techniques. We decided to have the students set up of a voltaic cell with the materials given (Sn, Fe, and Cu metals, Cu and Fe solutions, filter paper, U-tube, anode and cathode labels, and voltmeter). Students were graded on correct metals used to create the highest voltage, the use of a salt bridge, correctly placing the anode and cathode labels, placement of voltmeter leads on metals, and correct positive voltage reading. Hands-on activity scores were not significantly different between the virtual (N = 84, M = 2.71, SD = 1.65) and the hands-on (N = 79, M = 3.14, SD = 1.53) laboratory groups t(161) = 1.702, p = 0.091 (Table 2). Also there was no significant difference in the hands-on activity scores between the lecture groups F(5,157) = 0.910, p = 0.476. Since hands-on activity scores would have been significant at the 90% confidence level we decided to run t-tests on the individual items (Table 3) to see if we could identify the potential difference. The students who were in the hands-on laboratory group were significantly more likely to correctly use the salt bridge in the hands-on activity (virtual 66.7% correct, hands-on 91.1%), t(167) = 4.016, p < 0.001. Since the individual t-tests revealed something when applied to the individual hands-on activity items, we ran t-tests on the individual items of the pre- and post-test (Table 3). The use of a salt bridge in the written diagram of the voltaic cell had a t-test that was significant (t(167) = 2.087, p = 0.039). Mantel–Haenzsel chi-square analysis showed similar results to the individual item t-tests as well. The use of the salt bridge was the only significant difference between lab groups on the assessments we used. As students performed the hands-on activity in our presence we began to notice some differences between the two lab groups. First we gave the students two options for a salt bridge, a U-tube and filter paper. The hands-on lab used filter paper for the salt bridge during the original laboratory whereas the virtual lab only saw a U-tube in the simulation. Those virtual students who used a salt bridge in the hands-on assessment used the U-tube as their salt bridge 93% of the time, compared to hands-on students who used the U-tube only 50% of the time. Secondly, we noticed that the hands-on lab students kept placing the leads of the voltmeter into the solutions rather than on the electrodes. Only 34% of hands-on students placed the voltmeter leads into the solution versus 15% of virtual lab students. In trying to understand the reason for placing the metal leads into the solutions, we realized that the hands-on students had been given such small metal samples during the instructional lab that they had to place the metals around the leads down into the solutions or they fell into the solutions due to the small size of the electrodes, so many believed that was necessary.
Post-test measures | t-Test | |
---|---|---|
T-score | Sig. | |
Correct anode | 1.692 | 0.093 |
Reaction equations | ||
Zn in half | −0.365 | 0.716 |
Zn2+, e− in half | −0.949 | 0.344 |
Zn half states | −0.245 | 0.807 |
Zn e− correct side | −0.553 | 0.581 |
Zn half direction | 1.177 | 0.241 |
Cu in half | 0.428 | 0.670 |
Cu2+, e− in half | −0.325 | 0.746 |
Cu half states | −0.061 | 0.951 |
Cu e− correct side | 0.694 | 0.489 |
Cu half direction | 0.704 | 0.482 |
Overall Cu & Zn | 0.148 | 0.882 |
Overall states | −0.565 | 0.573 |
Overall direction | 0.182 | 0.856 |
No e− in overall | 0.530 | 0.597 |
Overall direction | −0.150 | 0.881 |
Diagram | ||
Cu2+ solution | 0.552 | 0.582 |
Cu metal | −0.509 | 0.611 |
Zn2+ solution | −0.159 | 0.874 |
Zn metal | −1.107 | 0.270 |
Metals in solution | −1.356 | 0.177 |
Two solutions | 0.843 | 0.400 |
Metal ions correct | −1.030 | 0.305 |
Salt bridge | 2.085 | 0.039 |
Wire connection | −0.047 | 0.963 |
e− flow | −0.538 | 0.591 |
Equilibrium | ||
Identified equ. | 1.201 | 0.231 |
Hands-on | ||
Solutions and metals | 1.010 | 0.314 |
Salt bridge | 4.016 | <0.001 |
Anode, cathode | 1.390 | 0.166 |
Leads on metals | −9.84 | 0.327 |
Positive voltage | 0.959 | 0.339 |
In this study, we looked at the use of virtual labs as a replacement to our normal hands-on lab, to determine the efficacy of virtual labs to help students learn the concepts and set-up of materials of electrochemical cells. We were using a comparison to our own labs and these results are only valid in relation to this context. Also the use of electrochemistry lab as a measurement tool for technique learning is limited since the set-up of materials to form voltaic cells is not necessarily indicative of technique learning, but this did provide some data regarding the ability to learn hands-on concepts in a virtual environment. Many virtual pre-lab exercises involve teaching students the proper way to use machines and chemical apparatus in the lab. With these goals and limitations in mind, the study showed that there was no significant difference between the two lab teaching methods when it came to the pre- and post-test data which focused primarily on conceptual understanding of the electrochemical cells. One of the main goals of lab is to teach content knowledge but according to this study the methods used here did not affect the scores on the tests. When comparing lecture sections to one another, however, we did find significant differences between the pre- and post-test scores, but not in the hands-on activity scores. This may indicate that the lecture is the main place where students learn conceptual and factual knowledge. We did not ask any descriptive questions regarding lab materials such as the color of the solutions or the metals. It may be that laboratories play an important role in providing this descriptive information but since we did not assess them on this we were unable to say anything about the development of this knowledge. The virtual lab was designed to include accurate macroscopic representations regarding colors of solutions and colors of metals. More research needs to be conducted to discover what uses or methods are the best for these goals.
When comparing the hands-on task, we did not find a significant difference between either the lab groups or the lecture groups. However, the p-value was significant at the 0.10 alpha; we decided to do an item analysis on the pre- post- and hands-on activity test. The only items that showed a significant difference between the lab groups was the placement of a salt bridge in their hands-on voltaic cell activity and the correct addition of a salt-bridge to the cell diagram. The virtual lab students left the salt bridge out more than the hands-on students in the hands-on activity. This may be due to the fact that the simulation did not require the students to place the salt bridge in the cell but placed it for them in the simulation, whereas in the hands-on lab students had to place the salt bridge correctly to get any results. In fact in the hands-on laboratory, they are asked to make a measurement without the salt bridge specifically to help them understand the importance of the salt bridge. The inclusion of a salt bridge in the pre- and post-test diagrams of the cell also revealed a difference in the two groups of students and the t-test was significant at the 95% confidence interval level. If the simulation was redone to include the placement of the salt bridge we may see different results. It is possible that merely including this consideration in the virtual lab would close this gap in performance.
In observing the labs and specifically grading the hands-on task, we noticed an abundance of students putting the voltmeter leads into the solution on the metals. This was cause for concern and we looked at the data to determine if one group placed the leads into the solution more than the other. The data revealed that more students in the hands-on lab placed their leads in the solutions, but this difference did not lead to a significant difference in the item analysis for this question. One explanation for the difference was that in the hands-on laboratory, we used such small pieces of metal that it became impossible for our students to attach the electrodes to them without having the leads in the solutions as well. This led to corrosion of the electrodes and may have led students to believe that proper placement of the electrodes was in the solution. This idea may give us some insight into proper technique training. While it is believed that hands-on labs teach technique better than virtual labs, techniques can be improperly taught if students are not observed and corrected during the lab and if the materials given to the students are not proper for the tasks at hand. In order to have students do the labs with less cost and time we may be sacrificing the teaching of technique due to our shortcuts. This idea could also be seen in the use of different types of salt bridges used in the two labs. The hands-on lab used soaked filter paper, whereas, the virtual simulations and the textbooks use U-tubes. These subtle differences may not be well understood by the students and so misconceptions would occur by not understanding the role the equipment plays in the facilitation of the laboratory experiments.
Virtual laboratories are a tool that, when used properly can be very beneficial. The difficulty is knowing when they are appropriate for a particular situation. In order for us to identify the strengths and weaknesses of virtual laboratories, we need to have clearly stated goals and conduct controlled research in these areas that focus on the agreed upon goals. This study showed that a virtual lab simulation was just as good as the normal hands-on general chemistry laboratory at teaching concepts and voltaic cell set-up in electrochemistry. More research needs to be done to determine virtual laboratories efficacy as a replacement for more traditional hands-on laboratory experiences.
This journal is © The Royal Society of Chemistry 2013 |