K. Christopher
Smith
* and
Valeria
Alonso
Department of Chemistry, University of Texas Rio Grande Valley, 1201 W. University Drive, Edinburg, TX 78539, USA. E-mail: kenneth.smith@utrgv.edu
First published on 6th November 2019
In this study a survey was developed to investigate students’ engagement during general chemistry laboratory sessions. Aspects of engagement surveyed included cognitive, behavioral, and emotional engagement, and the survey items were focused on activities during the pre-laboratory introduction, laboratory procedures, and data collection. Exploratory factor analysis of the results was conducted to determine the various underlying factors in the survey, and the scores of the general chemistry laboratory students along these underlying factors were compared. The findings supported the various dimensions of engagement reported in the literature.
While such educational goals are indeed worth striving for, Hofstein and Lunetta (1982) noted that there was significant overlap between objectives stated for laboratory experiences and objectives outlined for science courses themselves; a potential conclusion drawn from this overlap could be that student outcomes associated with laboratory experiences could be achieved through sciences courses themselves. As such, Hofstein and Lunetta (1982) highlighted the need for defined goals for laboratory work taking into account the opportunities potentially afforded by laboratory instruction.
Another important point raised by Hofstein and Lunetta (1982) was that researchers had not yet convincingly or comprehensively investigated the relationships between the teaching laboratory experience and student learning, especially in contrast to other forms of instruction. As such, it could be difficult to defend the effectiveness and the role of the teaching laboratory in science curricula. Some twenty years later, Hofstein and Lunetta (2004) noted that these concerns were still valid, despite the centrality of the teaching laboratory in science education.
Additional calls have been made for researchers to investigate the effects of the laboratory experience on student learning, most recently by Bretz (2019). She noted that undergraduate chemistry teaching laboratories are labor intensive and costly, and that such resources could come under scrutiny from university administrators facing increasingly strained budgets. Bretz (2019) highlighted the need to have arguments and data available to justify the presence of the standard chemistry teaching laboratory. Over the past fifteen years, researchers have been making more concerted and consistent efforts towards investigating student learning in the chemistry teaching laboratory. As the introduction to a themed collection on student learning in the chemistry teaching laboratory in this journal, Hofstein and Mamlok-Naaman (2007) wrote that the themed collection would hopefully provide insight into learning in the chemistry teaching laboratory. In addition, a number of research groups have been investigating student learning, and in particular meaningful learning, in the chemistry teaching laboratory.
Faculty goals for chemistry laboratory work have also been examined within the context of meaningful learning, with a focus on cognitive, affective, and psychomotor domains. Bretz et al. (2013) conducted a qualitative study on faculty goals for chemistry laboratory work and coded the faculty-identified goals as cognitive, affective, or psychomotor. They reported six cognitive goals: connecting the lecture to the laboratory, connecting the laboratory to everyday life, emphasizing conceptual understanding in the laboratory, connecting laboratory content to other science and mathematics fields, engaging in critical analysis, and communicating with the scientific community. Affective goals included laboratory connections to the real world, gaining independence, and engaging in collaboration. Lastly, psychomotor goals included using laboratory equipment and learning laboratory techniques. There were, however, differences in emphases on some of the general chemistry laboratory goals among faculty in departments which had successfully obtained federal funding geared towards laboratory curricular improvement, compared to faculty in departments which had not obtained such funding, similar to the findings reported by Bruck et al. (2010). Faculty in departments with such funding emphasized critical thinking and connecting lecture to laboratory more, while it was suggested that faculty in departments without such funding emphasized collaboration more.
Students are a critical component of the educational enterprise, and it is important to consider students’ goals for laboratory work, especially since educators and students may hold mismatched goals (Hofstein and Lunetta, 2004). Fittingly, student goals for general chemistry laboratory work have also been investigated within the context of meaningful learning. DeKorver and Towns (2015) collected general chemistry student-identified goals pertaining to specific general chemistry experiments students conducted, and coded the goals among the cognitive, affective, and psychomotor domains. They reported that students’ most prevalent goals focused on the affective domain, in terms of finishing quickly (positive), finishing late (negative), earning good grades (positive), and making mistakes (negative). DeKorver and Towns (2015) also reported that some students stated learning techniques as a goal (psychomotor), but did not take advantage of opportunities to practice or try techniques, while some students would follow the steps in the procedure without thought or understanding, with the intent to understand the material at a later time. This last result was also reported by Galloway and Bretz (2016), and by Galloway et al. (2016), who reported that some students were influenced to make this choice because they perceived a lack of control in the structure of their laboratory course, so they disengaged cognitively and focused only on carrying out the procedures. DeKorver and Towns (2015) noted that these actions led to a separation of psychomotor tasks from the cognitive domain, that the affective goal of finishing quickly was in conflict with cognitive and psychomotor goals, and that there were indeed areas of misalignment between student goals and faculty goals. In a related vein, Flaherty et al. (2017) have also reported on the misalignment between the perceptions of general chemistry students and laboratory demonstrators (similar to graduate teaching assistants) on laboratory demonstrators’ responsibilities for addressing aspects of meaningful learning in the general chemistry laboratory. Flaherty et al. (2017) wrote that graduate students (whether graduate teaching assistants or laboratory demonstrators) are integral to the enculturation of undergraduate students into the scientific community, as similarly noted by Smith and Sepulveda (2018); as such, it is important to focus on goals of students, faculty, and teaching assistants, in relation to laboratory work.
These studies have certainly helped to define educator and student goals for general chemistry laboratory work, and within the context of meaningful learning. The results can help inform educators on which domains of meaningful learning are being more or less attended to by educators and students, and can help guide the development of general chemistry laboratory curricula.
Galloway and Bretz also reported on the results of the MLLI administered to general chemistry laboratory students in a local study (Galloway and Bretz, 2015b), as well as to undergraduate chemistry laboratory students, including general chemistry laboratory students, in a national study (Galloway and Bretz, 2015c). They reported that their results might indicate that students intrinsically hold the affective domain to be more important than the cognitive domain within the context of learning in the laboratory (Galloway and Bretz, 2015b), and that even if students report an overall positive learning experience in the laboratory, it is possible that they may have also had some negative affective experiences, which could hinder the quality of their meaningful learning (Galloway and Bretz, 2015c). Galloway and Bretz called for chemistry laboratory curricula to attend more to the incorporation of the affective domain, beyond only a focus on group work and real world connections (Galloway and Bretz, 2015b, 2015c); in addition, it has been noted that the development of chemistry laboratory curricula should focus on various affective dimensions, and not focus only on student interest (Galloway et al., 2016). These studies have contributed greatly to shedding light on the presence and integration of cognitive, affective, and psychomotor domains in the general chemistry laboratory.
Within the context of science laboratory work specifically, few articles focused on exploring aspects of cognitive, affective, and psychomotor domains have appeared in the literature. Bowen (1999) developed the Chemistry Laboratory Anxiety Instrument (CLAI), which examined student anxiety in college freshman and sophomore chemistry laboratory courses. He identified five dimensions of anxiety: working with chemicals, using equipment and procedures, collecting data, working with other students, and having adequate time. Barrie et al. (2015) developed the Advancing Science by Enhancing Learning in the Laboratory (ASELL) Student Laboratory Experience (ASLE) survey instrument, which focused on students’ experiences, including aspects of relevance and interest, in mostly college freshman science laboratory courses. The ASLE instrument was meant to be administered to students engaged in various laboratory experiments to provide feedback to laboratory instructors to improve the laboratory experiments. They found three dimensions of the students’ experiences: motivators, assessment, and resources. In addition, Sadler et al. (2011) developed the Laboratory Instructional Practices Inventory (LIPI), an observation protocol focused on student engagement and discourse in college biology, chemistry, and physics laboratory courses. The LIPI protocol was also meant to capture data from various laboratory experiments to provide feedback for the improvement of the laboratory experiments. Sadler et al. defined various types of student engagement being observed, including disengagement (students doing something other than the laboratory work), passive engagement (students receiving information), task-oriented engagement (students working on laboratory tasks or protocols), and epistemic engagement (students being cognitively engaged). These studies have all contributed to the knowledge base on aspects of cognitive, affective, and psychomotor domains in science curricula.
A useful framework for considering student engagement is found in the seminal review article written by Fredricks et al. (2004), in which they thoroughly discussed the concept of engagement in school settings. They defined three types of engagement: behavioral engagement, emotional engagement, and cognitive engagement. While they presented various definitions of each of these forms of engagement, each type of engagement had a definition that was most relevant to chemistry laboratory work, which is the focus of this article. The most relevant definition of behavioral engagement centered on students’ involvement in academic tasks and learning, and was characterized by behaviors such as effort, persistence, and attention. Similarly, the most relevant definition of emotional engagement focused on students’ affective reactions in the academic environment, such as interest, boredom, and anxiety. Likewise, the most relevant definition of cognitive engagement involved students’ psychological investment in academic tasks, as well as their psychological effort geared towards academic tasks, such as learning, understanding, and mastering. It is important to note that these dimensions of engagement align with the domains of meaningful learning; cognitive engagement aligns with the cognitive domain, emotional engagement with the affective domain, and behavioral engagement with the psychomotor domain. Furthermore, the cognitive and behavioral aspects of engagement focus on students’ efforts along those dimensions, which aligns with one of the stipulations for meaningful learning to occur, namely that the learner must consciously choose to integrate the new material into the prior knowledge (Ausubel, 1963). In contrast, the emotional aspect of engagement focuses on students’ affective reactions.
Within this context of student engagement, several published studies do provide for a measure of student engagement in science laboratory activities, largely in the dimension of emotional engagement. The Meaningful Learning in the Laboratory Instrument (MLLI) developed by Galloway and Bretz (2015a) measured the extent of affective reactions. The MLLI also measured cognitive aspects, but more in terms of the degree of cognitive aspects present, as opposed to cognitive effort. The Chemistry Laboratory Anxiety Instrument (CLAI) developed by Bowen (1999) also measured the extent of affective reactions. The Laboratory Instructional Practices Inventory (LIPI) developed by Sadler et al. (2011) did capture aspects of engagement observed, but it did not measure the effort associated with behavioral or cognitive engagement, and it did not measure the extent of affective reactions.
Several other published studies do include reports of positive student engagement associated with chemistry laboratory activities, but without the use of instruments such as the MLLI (Galloway and Bretz, 2015a) or the CLAI (Bowen, 1999). Laredo (2013) reported on the implementation of problem-based undergraduate general chemistry laboratories. She reported that students were engaged, and more so than when conducting traditional general chemistry laboratory experiments. These conclusions about engagement were based on student feedback which indicated that the laboratory activities were enjoyable and made them think. Irby et al. (2018) reported on the use of a virtual general chemistry laboratory module to investigate students’ engagement with macroscopic, submicroscopic, and symbolic representations of chemistry, in comparison to the wet general chemistry laboratory. They measured engagement by comparing students’ dialogue relevant to macroscopic, submicroscopic, and symbolic representations of chemistry, as students worked through the two different laboratory formats. In addition, a number of published laboratory experiments have indicated positive student engagement outcomes. For example, Smith and colleagues have reported that student feedback indicated that students enjoyed chemistry laboratory experiments involving cranberry-apple juice (Edionwe et al., 2011) and acetaminophen in gel capsules (Smith and Cedillo, 2014), and that students found an inquiry-based lab on acid conductivity to be engaging (Smith and Garza, 2015). These articles do report on aspects of engagement, but generally without the use of an instrument or an observation protocol, and without a focus on all the dimensions (behavioral, emotional, and cognitive) of engagement.
In a recent article, Nyutu et al. (2019) investigated student engagement in individual undergraduate microbiology laboratories. They cited the framework of engagement described by Fredricks et al. (2004) but opted to use the Laboratory Instructional Practices Inventory (LIPI) developed by Sadler et al. (2011), since it was specific to science. We were also interested in examining student engagement in individual laboratory experiments, within the context of the general chemistry laboratory specifically, and with a consideration of all three types of engagement presented by Fredricks et al. (2004). No instrument currently exists to measure this student engagement in general chemistry laboratory experiments, so we saw the need to develop such an instrument. In addition, during general chemistry laboratories there are typically several different types of activities: a pre-laboratory talk by the instructor or teaching assistant, engaging in the laboratory procedures, and collecting data and observations. We were interested in examining the various types of student engagement during these different types of activities during the laboratory. Furthermore, we were interested in measuring and comparing students’ engagement in various laboratory experiments. In light of these various considerations, our research question was: How do students’ behavioral, emotional, and cognitive engagement vary and compare across various laboratory experiments in university general chemistry laboratory courses?
The next section of the survey included forty-six Likert-scale items with response options of Strongly Disagree, Disagree, Agree, and Strongly Agree. Nadler et al. (2015) reported that the strongest validity and reliability are exhibited by Likert-scale items with 4- to 7-point scales, and we chose to use a 4-point scale with our Likert-scale items. Furthermore, we did not include a midpoint because we wanted participants to select a stance instead of simply selecting the midpoint (Nadler et al., 2015). In addition, we opted to use a Likert-scale as opposed to a slider scale because slider scales have labels only at the ends of the scales, potentially leading to participants having various interpretations of response categories, whereas the Likert-scale has each response category labeled, resulting in common interpretations of the labels by participants (Weijters et al., 2010). The Likert-scale items focused on the various parts of a typical general chemistry laboratory experiment: a pre-laboratory talk by the instructor or teaching assistant, engaging in the laboratory procedures, and collecting data and observations. In addition, there were items involving behavioral, emotional, and cognitive engagement associated with each different part of a typical general chemistry laboratory experiment, as well as several items on general engagement during laboratory experiments.
After the survey was constructed, as a measure of face validity of the survey, the survey was shared with several undergraduate students who had recently taken general chemistry courses and there were no major suggested changes to the survey items. The survey was administered to students enrolled in General Chemistry for Engineers Laboratory courses, as well as the first semester (General Chemistry Laboratory I) and second semester (General Chemistry Laboratory II) freshman general chemistry laboratory courses viaQualtrics (2017). It should be noted that these chemistry laboratory courses all offered traditional laboratory experiments, which would typically be described as Level 0: Confirmation, following Bruck et al. (2008). The laboratory experiments occupy a two hour and forty minute time slot, with a teaching assistant or instructor assigned to each laboratory classroom with a maximum of 22 students. During the laboratory experiments, students typically work in pairs to collect data and fill in data tables, while the laboratory report has students follow prescribed calculations and data analysis. During the semester in which survey data were collected, there were 9 sections of the general chemistry for engineers course (with 193 students total), 33 sections of the general chemistry laboratory I course (with 685 students total), and 9 sections of the general chemistry laboratory II course (with 182 students total).
Participation in the survey by students was voluntary, and students received a 5-point bonus credit on their final exam for completing the survey. The survey was administered just once, several weeks before the end of the semester. In total, 651 students accessed the survey, while 616 of these students completed the survey. As one check on the quality of the data, we tallied the number of students who completed the survey and who had responded in exactly the same way (either Strongly Disagree, Disagree, Agree, or Strongly Agree) to all of the Likert-scale items, which was a possible indication that they were not really paying attention to the items and were simply entering responses. There were 2 students who responded Agree and 3 students who responded Strongly Agree, respectively, to all of the Likert-scale survey items; these data were excluded from the analysis, and as such, our preliminary survey data analysis was based on the data of the remaining 611 students. At the end of the survey, students had an opportunity to give open-ended feedback on the survey, with this feedback used as a further measure of face validity of the survey. 123 of the students indicated that they had no feedback to give, while 18 students indicated that it was a good survey, based on it being thorough, detailed, well-organized, answered easily, and reflective of the various aspects of the chemistry laboratory experiment experience. Two students indicated that they found the survey questions somewhat repetitive, while four students indicated that they would have preferred a neutral option in the Likert-scale response options. In addition, there was no student feedback indicating doubt or confusion about the meaning of any of the survey items.
Data analysis (described in the following sections) of the results of the preliminary survey led to eighteen of the forty-six survey items being dropped, yielding a final survey consisting of twenty-eight items. As a measure of the response process validity associated with the final survey, eleven general chemistry students were interviewed individually to gauge their understanding of the various survey items. Each student was interviewed on either nine or ten of the survey items, with each survey item being addressed by either three or four students. In each student's interview, each survey item presented to them was printed out on a slip of paper and presented to them one at a time. Each student was asked to read aloud each survey item, and to describe in their own words what the survey item meant to them. Students’ interpretations of the survey items were well aligned with the intended meanings of the survey items. In addition, as a result of the interviews, several small changes were made to the wording of some of the survey items. One such change involved the words “excited” and “intrigued” in some of the survey items; students indicated that such words were too enthusiastic, and that the word “interested” would be more fitting, which reinforced the importance of such item wording (Galloway and Bretz, 2015a; Galloway et al., 2016).
As a measure of the content validity associated with the final survey, three instructors who regularly teach general chemistry laboratory courses (two of them with doctoral degrees in chemistry and one of them with a master's degree in chemistry) individually reviewed the final survey items. In reviewing the survey items they were asked to indicate the type of engagement (behavioral, emotional, or cognitive) associated with each survey item. There was total agreement on the type of engagement on twenty-two of the survey items, while there was partial agreement (two out of three of the reviewers) on five of the survey items, and no agreement amongst the reviewers on one survey item. The reviewers were also asked to indicate the extent to which they perceived each individual item to be representative of the type of engagement that they selected for each item, according to the following scale: 1 = not representative, 2 = minimally representative, 3 = moderately representative, and 4 = strongly representative. All reviewers rated the representativeness of all items as either 3 (moderately representative, 22.6% of the total items) or 4 (strongly representative, 77.4% of the total items). Finally, the reviewers were asked to comment on the clarity of the items, and they all indicated that the items were clear.
The final survey was administered in a subsequent semester to students enrolled in General Chemistry for Engineers Laboratory courses, as well as the first semester (General Chemistry Laboratory I) and second semester (General Chemistry Laboratory II) freshman general chemistry laboratory courses viaQualtrics (2017). During the semester in which survey data were collected, there were 6 sections of the general chemistry for engineers course (with 143 students total), 26 sections of the general chemistry laboratory I course (with 482 students total), and 17 sections of the general chemistry laboratory II course (with 375 students total). Participation in the survey by students was voluntary, and students received a 5-point bonus credit on their midterm exam for completing the survey. The survey was administered just once, near the middle of the semester. In total, 296 students accessed the survey, while 269 of these students completed the survey. As one check on the quality of the data, we tallied the number of students who completed the survey and who had responded in exactly the same way (either Strongly Disagree, Disagree, Agree, or Strongly Agree) to all of the Likert-scale items, which was a possible indication that they were not really paying attention to the items and were simply entering responses. There was 1 student who responded Agree and 1 student who responded Strongly Agree, respectively, to all of the Likert-scale survey items; these data were excluded from the analysis. In addition, there were 4 students who responded Agree or Strongly Agree to all the items, even though some of the items were reverse worded; these data were also excluded from the analysis. As such, our final survey data analysis was based on the data of the remaining 263 students.
The average age of the students taking the final survey was 20.4 years. The distribution of students by gender, course, classification, and major are shown in Table 1; one student did not submit their laboratory course.
Number of students | % of students | ||
---|---|---|---|
Gender | Female | 165 | 62.7 |
Male | 98 | 37.3 | |
Course | Gen. Chem. Eng. Lab. | 13 | 4.9 |
Gen. Chem. Lab. I | 154 | 58.6 | |
Gen. Chem. Lab. II | 95 | 36.1 | |
Classification | Freshman | 82 | 31.2 |
Sophomore | 78 | 29.7 | |
Junior | 70 | 26.6 | |
Senior | 33 | 12.5 | |
Major | Biology | 127 | 48.3 |
Nursing | 38 | 14.4 | |
Chemistry | 16 | 6.1 | |
Mechanical Engineering | 10 | 3.8 | |
Civil Engineering | 7 | 2.7 | |
Computer Engineering | 5 | 1.9 | |
Psychology | 5 | 1.9 | |
Pre-Professional | 2 | 0.8 | |
Other | 53 | 20.2 |
We then conducted exploratory factor analysis on the data in IBM SPSS Statistics 23 (2015). We also conducted reliability analyses of the various factors, and comparisons of means of the various groups on the various factors as well as the factors combined. Exploratory factor analysis is a statistical technique used to reduce a large number of variables in a set of data into a smaller set of factors, with each of the factors being related to a latent construct (Taherdoost et al., 2014), which cannot be directly measured. We chose to conduct exploratory factor analysis on our survey data to determine which survey items loaded onto various factors, and to determine the nature of the various factors. Also, exploratory factor analysis provides evidence of construct validity (Thompson and Daniel, 1996), allowing us to provide evidence of construct validity for our survey, which is a Likert-scale self-reporting survey. Exploratory factor analysis was appropriate for our preliminary survey data as our number of survey participants surpassed the generally recommended number of 300 survey participants for exploratory factor analysis (Taherdoost et al., 2014); we also conducted exploratory factor analysis on our final survey data, as the number of our survey participants was approaching 300. In addition, our ratio of preliminary survey participants to survey items was close to 13.3:1, which surpassed the minimum recommended ratio of 10:1 (Costello and Osborne, 2005); our ratio of final survey participants to survey items was 9.4:1, which was approaching the ratio of 10:1. We followed the best practices outlined by Costello and Osborne (2005) in conducting exploratory factor analysis of our data.
To determine the reliabilities of the various factors, internal consistencies were measured by Cronbach's α, with values greater than 0.7 considered acceptable (Nunnally, 1978). To explore difference in survey scores among the different groups (gender, course, classification), analysis of variance (ANOVA) tests were conducted.
After the initial exploratory factor analysis was conducted on the final survey data, one item was dropped as a result of cross-loading on factors associated with behavioral engagement and emotional engagement in data collection. Apart from this item, there were only two items remaining associated with behavioral engagement in data collection, and they were both low-loading items, so they were dropped.
The Kaiser–Meyer–Olkin Measure of Sampling Adequacy (KMO), which evaluates the degree to which items are correlated, generally has a minimum value of 0.7 considered as adequate, with higher values approaching the maximum value of 1 being preferred (Taherdoost et al., 2014). Our final survey data value for the Kaiser–Meyer–Olkin Measure of Sampling Adequacy (KMO) was 0.894. In addition, our p-value for Bartlett's test of sphericity, which tests the hypothesis that the correlation matrix is an identity matrix, was <0.001; this result indicated that the correlation matrix was not an identity matrix and that the items were indeed correlated.
The number of final survey factors retained from the exploratory factor analysis was determined using the scree test (Costello and Osborne, 2005). This test involves examining the plot of eigenvalue versus factor number, and looking for a break or bend in the plot where the curve flattens out. The number of factors retained would be the number of data points above the break or bend. Fig. 1 shows the scree plot.
As a further gauge for the number of factors to retain from the exploratory factor analysis, we determined the scree test acceleration factors (which are a function of eigenvalue), which emphasize the point on the scree plot at which the slope changes abruptly (Raîche et al., 2013). The scree test acceleration factor plot graphs the scree test acceleration factor as a function of factor number from the second to the penultimate eigenvalue. This graph allows for an examination of where the break or bend in the scree plot occurs, enabling us to make a decision on the number of factors to be retained. Fig. 2 shows the scree test acceleration factor plot.
Based on the scree plot and the scree test acceleration factor plot, as well as a consideration of Kaiser's criterion which recommends that all factors with eigenvalues greater than one be retained, six factors were retained. The six factors and corresponding survey items are shown in Table 2.
Factors and items | Factor loadings | |||||
---|---|---|---|---|---|---|
F1 | F2 | F3 | F4 | F5 | F6 | |
Factor 1 (F1): Cognitive Engagement in Data and Overall | ||||||
I put a lot of effort into understanding and evaluating the data/observations as I was collecting them during the lab | 0.59 | −0.01 | −0.01 | 0.10 | 0.09 | −0.03 |
I put a lot of effort into thinking about techniques to help me accurately record the data from the instruments, equipment, glassware, and my observations | 0.72 | −0.06 | −0.01 | 0.01 | 0.08 | 0.02 |
I put a lot of effort into planning how I would collect all the data/observations before the end of the lab | 0.74 | 0.03 | −0.02 | 0.05 | 0.01 | 0.04 |
I tried hard to connect the lab experiment to the theory from lecture | 0.76 | 0.00 | 0.05 | −0.04 | −0.12 | 0.00 |
I put a lot of effort into thinking about real world applications and connections to the lab experiment | 0.59 | 0.12 | 0.04 | −0.09 | 0.11 | −0.06 |
I tried hard to understand the lab concepts rather than just memorizing them | 0.72 | −0.06 | −0.02 | −0.03 | −0.01 | 0.11 |
Factor 2 (F2): Negative Emotional Engagement in Lab Procedures | ||||||
I felt nervous about using the glassware during the lab procedures | 0.03 | 0.75 | −0.05 | 0.01 | −0.11 | −0.12 |
I felt unsure when following the lab procedures | 0.00 | 0.66 | 0.08 | −0.01 | 0.07 | 0.14 |
I felt insecure about using the lab equipment and instruments during the lab procedures | −0.07 | 0.71 | 0.11 | −0.05 | 0.01 | 0.14 |
I felt nervous when using and/or preparing the chemicals during the lab procedures | 0.03 | 0.87 | −0.12 | 0.07 | 0.03 | −0.04 |
Factor 3 (F3): Positive Emotional Engagement in Lab Procedures | ||||||
I felt interested when using the lab equipment and instruments during the lab procedures | −0.04 | 0.06 | 0.82 | 0.06 | −0.05 | −0.01 |
I found it interesting to use and/or prepare the chemicals during the lab procedures | −0.01 | −0.14 | 0.75 | 0.12 | 0.01 | 0.08 |
I found it interesting to use the glassware during the lab procedures | −0.01 | −0.02 | 0.87 | −0.08 | 0.07 | 0.00 |
I felt interested when following the laboratory procedures | 0.14 | 0.06 | 0.74 | −0.01 | −0.06 | −0.10 |
Factor 4 (F4): Behavioral Engagement in Lab Procedures | ||||||
I participated fully when using the lab equipment and instruments during the lab procedures | −0.05 | −0.03 | −0.10 | 0.73 | 0.07 | 0.09 |
I put a lot of effort into following the lab procedures properly | −0.03 | 0.01 | 0.17 | 0.56 | 0.06 | 0.00 |
I participated fully in using and/or preparing the chemicals during the lab procedures | 0.02 | −0.10 | 0.09 | 0.76 | −0.05 | 0.02 |
I participated fully when using the glassware during the lab procedures | 0.04 | 0.16 | −0.03 | 0.79 | −0.06 | −0.11 |
Factor 5 (F5): Cognitive Engagement in lab Procedures | ||||||
I put a lot of effort into understanding the design of the lab procedures in terms of how different parts of the lab procedures were related to one another | 0.10 | −0.04 | −0.03 | 0.13 | 0.63 | 0.08 |
I tried hard to understand why specific lab equipment and instruments were used during the lab procedures | 0.08 | −0.04 | −0.09 | 0.01 | 0.81 | 0.00 |
I put a lot of effort into understanding why specific chemicals were used and/or prepared during the lab procedures | 0.04 | 0.01 | 0.02 | −0.03 | 0.87 | −0.03 |
I tried hard to understand why specific glassware was used during the lab procedures | 0.16 | 0.03 | 0.12 | −0.07 | 0.64 | −0.12 |
Factor 6 (F6): Negative Emotional Engagement in Data Collection | ||||||
I felt insecure about the data/observations I was collecting during the lab | 0.07 | −0.05 | −0.06 | 0.02 | −0.09 | 0.86 |
I felt unsure about accurately recording the data from the instruments, equipment, glassware, and my observations | −0.02 | 0.07 | −0.02 | −0.02 | 0.05 | 0.80 |
I felt worried about collecting all the data/observations before the end of the lab | 0.00 | 0.21 | 0.09 | −0.01 | 0.01 | 0.56 |
Factors 1 and 5 dealt with cognitive engagement, Factors 2, 3, and 6 dealt with emotional engagement, and Factor 4 dealt with behavioral engagement. These types of engagement aligned with the various types of engagement described by Fredricks et al. (2004), but within the context of the general chemistry laboratory.
Factor 1, focused on cognitive engagement, included items associated with collecting and understanding data, and understanding the lab overall. Factor 5, also focused on cognitive engagement, included items associated with the laboratory procedures. These results indicated that students experienced cognitive engagement in laboratory procedures differently than cognitive engagement in data collection.
Factor 2 dealt with negative emotional engagement in the laboratory procedures, Factor 3 dealt with positive emotional engagement in the laboratory procedures, and Factor 6 dealt with negative emotional engagement in data collection. These results indicated that students experienced negative emotional engagement in laboratory procedures differently than negative emotional engagement in data collection. These results also showed that negative emotional engagement in laboratory procedures and positive emotional engagement in laboratory procedures are different factors, which supported the work of Galloway and Bretz (2015a, 2015b, 2015c, 2015d), who reported students experiencing either contributions to meaningful learning (positively worded items) or inhibitions against meaningful learning (negatively worded items).
Factor 4 focused on behavioral engagement in laboratory procedures. Overall, laboratory procedures was the aspect of the laboratory experience most represented among the various dimensions of engagement, with factors involving negative emotional engagement in laboratory procedures, positive emotional engagement in laboratory procedures, behavioral engagement in laboratory procedures, and cognitive engagement in laboratory procedures.
Survey items | Cronbach's α |
---|---|
Factor 1 (Cognitive Engagement in Data and Overall) | 0.85 |
Factor 2 (Negative Emotional Engagement in Lab Procedures) | 0.84 |
Factor 3 (Positive Emotional Engagement in Lab Procedures) | 0.88 |
Factor 4 (Behavioral Engagement in Lab Procedures) | 0.82 |
Factor 5 (Cognitive Engagement in Lab Procedures) | 0.88 |
Factor 6 (Negative Emotional Engagement in Data Collection) | 0.83 |
These results indicated that the internal consistencies of the various factors all surpassed the benchmark Chronbach's α value of 0.7.
Survey items | Scores |
---|---|
a The average survey score was significantly different at the p = 0.05 level. b The items in Factors 2 and 6 were reverse scored. | |
Factor 1 (Cognitive Engagement in Data and Overall) | 3.19 |
Factor 2 (Negative Emotional Engagement in Lab Procedures)b | 3.01 |
Factor 3 (Positive Emotional Engagement in Lab Procedures) | 3.55a |
Factor 4 (Behavioral Engagement in Lab Procedures) | 3.76a |
Factor 5 (Cognitive Engagement in Lab Procedures) | 3.23 |
Factor 6 (Negative Emotional Engagement in Data Collection)b | 3.07 |
Entire survey | 3.30 |
The average survey scores on Factors 3 and 4 were significantly different from all average survey scores at the p = 0.05 level. In addition, the average scores on Factors 2 and 6 were not significantly different, the average scores on Factors 1 and 6 were not significantly different, and the average scores on Factors 1, 5, and the entire survey were not significantly different, all at the p = 0.05 level.
These results indicated that overall, Factor 4 (Behavioral Engagement in Laboratory Procedures) had the highest average score. The behavioral engagement dimension is the dimension most associated with “doing” the laboratory experiment, and these results indicated that the highest level of engagement were associated with “doing” the laboratory work.
Factor 3 (Positive Emotional Engagement in Laboratory Procedures) had a significantly higher average score than Factor 2 (Negative Emotional Engagement in Laboratory Procedures). This result indicated that on average students agreed more strongly with the positive emotional engagement in laboratory procedures items in comparison to disagreeing less strongly with the negative emotional engagement in laboratory procedures items.
Survey items | Laboratory experiment | |||||
---|---|---|---|---|---|---|
Gen. Chem. Eng. Lab. acids (N = 40) | Gen. Chem. Eng. Lab. volumetric (N = 8) | Gen. Chem. Lab. I limiting (N = 25) | Gen. Chem. Lab. I copper (N = 102) | Gen. Chem. Lab. II factors (N = 29) | Gen. Chem. Lab. II law (N = 58) | |
a The average survey score was significantly different at the p = 0.05 level. | ||||||
Factor 1 | 3.30 | 3.13 | 3.06 | 3.22 | 3.10 | 3.18 |
Factor 2 | 3.05 | 3.03 | 2.95 | 2.93 | 3.09 | 3.11 |
Factor 3 | 3.69 | 3.31 | 3.49 | 3.62 | 3.58 | 3.40 |
Factor 4 | 3.79a | 3.78 | 3.72 | 3.83a | 3.90a | 3.57a |
Factor 5 | 3.33 | 2.97 | 3.14 | 3.29 | 3.10 | 3.20 |
Factor 6 | 3.03 | 2.88 | 3.16 | 3.02 | 3.22 | 3.13 |
Entire survey | 3.37 | 3.19 | 3.24 | 3.32 | 3.32 | 3.26 |
The results indicated that there were only few significant differences between the average survey scores on any of the six laboratory experiments at the p = 0.05 level. On Factor 4 (Behavioral Engagement in Lab Procedures), students had a significantly lower average score in the general chemistry II rate law laboratory experiment compared to several other laboratory experiments, indicating that students were less engaged on the behavioral dimension in that particular laboratory experiment.
For further statistical context, the statistical power and effect size of the various ANOVA tests are presented in Table 6; these results may have contributed to the lack of significance found.
Survey items | Laboratory experiments | |
---|---|---|
Statistical power (%) | Effect size, η2 | |
Factor 1 | 30.9 | 0.02 |
Factor 2 | 28.1 | 0.02 |
Factor 3 | 77.7 | 0.05 |
Factor 4 | 98.9 | 0.10 |
Factor 5 | 40.0 | 0.02 |
Factor 6 | 29.5 | 0.02 |
Entire survey | 26.2 | 0.01 |
Laboratory experiments in real world contexts or inquiry contexts are suggested to be engaging and motivating for students (Barrie et al., 2015), with real world contexts pertaining particularly to the affective (emotional) domain (Bretz et al., 2013; Galloway and Bretz, 2015b). The survey instrument reported in this study can be used to examine the differences in cognitive, emotional, and behavioral engagement between laboratory experiments embedded in real world contexts and inquiry-based contexts, and laboratory experiments not embedded in such contexts.
2. I put a lot of effort into following the lab procedures properly.
3. I participated fully in using and/or preparing the chemicals during the lab procedures.
4. I participated fully when using the glassware during the lab procedures.
5. I felt interested when using the lab equipment and instruments during the lab procedures.
6. I found it interesting to use and/or prepare the chemicals during the lab procedures.
7. I found it interesting to use the glassware during the lab procedures.
8. I felt interested when following the laboratory procedures.
9. I felt nervous about using the glassware during the lab procedures.
10. I felt unsure when following the lab procedures.
11. I felt insecure about using the lab equipment and instruments during the lab procedures.
12. I felt nervous when using and/or preparing the chemicals during the lab procedures.
13. I put a lot of effort into understanding the design of the lab procedures in terms of how different parts of the lab procedures were related to one another.
14. I tried hard to understand why specific lab equipment and instruments were used during the lab procedures.
15. I put a lot of effort into understanding why specific chemicals were used and/or prepared during the lab procedures.
16. I tried hard to understand why specific glassware was used during the lab procedures.
17. I felt insecure about the data/observations I was collecting during the lab.
18. I felt unsure about accurately recording the data from the instruments, equipment, glassware, and my observations.
19. I felt worried about collecting all the data/observations before the end of the lab.
20. I put a lot of effort into understanding and evaluating the data/observations as I was collecting them during the lab.
21. I put a lot of effort into thinking about techniques to help me accurately record the data from the instruments, equipment, glassware, and my observations.
22. I put a lot of effort into planning how I would collect all the data/observations before the end of the lab.
23. I tried hard to connect the lab experiment to the theory from lecture.
24. I put a lot of effort into thinking about real world applications and connections to the lab experiment.
25. I tried hard to understand the lab concepts rather than just memorizing them.
This journal is © The Royal Society of Chemistry 2020 |