Using the laboratory to engage all students in science practices

J. P. Walker; V. Sampson; S. Southerland; P. J. Enderle

doi:10.1039/C6RP00093B

View PDF VersionPrevious ArticleNext Article

DOI: 10.1039/C6RP00093B (Paper) Chem. Educ. Res. Pract., 2016, 17, 1098-1113

Using the laboratory to engage all students in science practices†

J. P. Walker *, V. Sampson , S. Southerland and P. J. Enderle
Department of Chemistry, East Carolina University, Greenville, NC 27858, USA. E-mail: walkerjoi15@ecu.edu

Received 19th April 2016 , Accepted 17th August 2016

First published on 23rd August 2016

Abstract

This study examines the extent to which the type of instruction used during a general chemistry laboratory course affects students’ ability to use core ideas to engage in science practices. We use Ford’s (2008) description of the nature of scientific practices to categorize what student do in the laboratory as either empirical or representational. One approach to lab instruction, engages students in the empirical practices of science but in a traditional prescriptive manner designed to demonstrate and verify content. The second approach, Argument-Driven Inquiry (ADI), engages students in both the empirical and representational practices of science. A practical exam was used to compare student learning in each condition. The assessment targeted student ability to participate in specific scientific practices, including planning and conducting investigations, analyzing and interpreting data and arguing from evidence. Students who were taught with either ADI (N = 81) or Traditional (N = 76) had equivalent understanding of content based on the ACS-GCST exam, however the mean score on the practical exam was significantly higher for students in the ADI sections. Results also indicate that the mean scores on the practical exam were significantly higher in the ADI sections for all students including female students, under-represented minority (URM) students, and students with lower past academic achievement. In the traditional laboratory sections there was a significant difference in the mean scores on the practical exam for the URM student relative to the majority, which was not present in the ADI sections. However, the opposite was found for students with low past academic achievement; the mean score on the practical exam was significantly lower for the students in the ADI sections in comparison to the traditional sections.

Introduction

Over the past several decades there has been a steady decline worldwide in the number of students studying science, technology, and engineering (Hannover and Kessels, 2004). The movement away from science has been experienced in Australia, England, Canada, Denmark, France, Germany, India, Ireland, Japan, Korea, The Netherlands, Norway, New Zealand and the USA (Lyons, 2006). For example, in Germany, only about 10% of all students choose physics as their major subject and fewer than 10% choose chemistry (Chandrasena et al., 2014). Other studies have shown that when asked to rank school subjects according to personal preference, typically, language subjects are at the head of the list, while the sciences are at the far end (Chandrasena et al., 2014). In Australia, the number of Year 12 students studying any science subject has fallen from 94 per cent to just over 50 per cent in the past 20 years, and according to the available data enrollments are still falling (Chandrasena et al., 2014). This has resulted in a lack of qualified people for employment in science-related jobs. At universities in the USA, as many as half of the aspiring science majors, become discouraged and migrate to other disciplines before graduating. In addition, females and most minority groups in the USA continue to be under-represented in the physical sciences, engineering, and technology based upon their numbers in the overall undergraduate populations; as women and other minority students constitute 75% of the total college population, but only 45% of the STEM degrees earned (Villafañe et al., 2014; National Science Foundation, National Center for Science and Engineering Statistics, 2015). While issues of racial diversity may be more pronounced in the US, this failure to fully include a significant portion of the population in any country is cause for concern, as reflected in the concern for under-representation of women reported in the UK and Australia (Lyons, 2006).

There are several factors that can help explain why undergraduate STEM majors are migrating to other disciplines. High-achieving students, for example, often describe introductory science courses as uninspiring and low achieving students describe them as unwelcoming. High and low achieving students, as a result, report that they do not learn much from their introductory science courses (National Research Council, 2014). Many students will therefore switch majors in order to take courses that are more interesting or to be in courses where they can feel welcome and successful. Under-represented minority (URM) students face even more obstacles when they enroll in introductory science courses. These students are often confronted with a mismatch between their personal identity and their perceptions of the scientific community (Brown et al., 2012; Graham et al., 2013) and struggle to develop supportive academic peer relationships and campus community bonds (Espinosa, 2011; Wilson and Kittleson, 2013). To make matters worse, introductory science courses tend to be individual-rather than community-centered, which prevents URM students from developing strong ties with their peers, instructors, and the discipline (NRC, 2014). Major changes to undergraduate courses need to be made in order to keep students from leaving the various STEM majors. These courses, at a minimum, need to change in a way that not only gives students more opportunities to learn the knowledge and skills they need to be successful in a STEM major but also makes students feel like they are member of a community. One way to accomplish this goal is to change the nature of instruction that takes place in undergraduate science courses.

In this study, we attempt to address this issue by examining how two different approaches to instruction that were used in an undergraduate chemistry laboratory course affected undergraduate students’ ability to use core ideas and scientific practices to solve problems using a baseline comparison research design. Given the importance of making science more inclusive and current efforts to broaden the participation of a wider diversity of students in science, we take particular care to investigate how women, under-represented minority (URM) students, and low achieving students perform in each context (National Research Council, 2011; Graham et al., 2013). Students in the treatment group were taught using Argument-Driven Inquiry (ADI), an instructional model developed by the authors that is designed to integrate the learning of core ideas with inquiry, argumentation and writing (Walker et al., 2011). Students in the baseline comparison group were taught using a traditional laboratory approach, featuring carefully scripted investigations that emphasized data collection and data analysis in order to verify the topics addressed in a lecture course.

The ADI instructional model is designed to give a more central place to argumentation and the role of argument in the social construction of scientific knowledge while promoting inquiry. Fig. 1 outlines the seven steps of ADI that are designed to integrate the learning of scientific concepts and scientific practices in such a way that little explicit instruction is necessary, rather students gain proficiency through engagement in the laboratory investigations moving from investigation design, to data analysis and argument development, to argumentation sessions, and to a final written argument (Walker et al., 2011).


	Fig. 1 7-Steps for ADI (Adapted with permission from Walker et al. (2011). Copyright (2011) American Chemical Society).

The first step of the ADI instructional model is the identification of the task through discussion of the research question. This step is designed to provide students with a challenging problem to solve and to capture the students’ attention and interest. The second step of ADI, which is called the generation of data, is therefore situated by the research question. In this step of the model, students work in a collaborative group (3 or 4 students) in order to develop and implement a method (e.g., an experiment or analysis) to answer the research question provided in step 1. This provides students with an opportunity to learn how to design and conduct informative investigations and to learn to deal with the ambiguities of empirical work. The third step, production of a tentative argument, calls for students to craft an evidence-based argument in response to the research question. During the fourth step, an argumentation session, the small groups have an opportunity to share their arguments and to critique the arguments of other groups. In other words, argumentation sessions are designed to give students an opportunity to learn to critique the products (i.e., conclusions, explanations or arguments), processes (i.e., methods), and context (i.e., theoretical or empirical foundations) of an inquiry. The fifth step of the ADI instructional model is the creation of a written investigation report by individual students. This report gives the students an opportunity to share the goal of their investigation, the method they used, and their overall argument. The sixth step of the ADI instructional model is a double-blind group peer-review. This stage is designed to support the appropriation of evaluation criteria by students as well as engagement in the assessment practices embedded in the model. The seventh, and final, step of the ADI instructional model is the revision of the investigation report based on the results of the peer-review. Overall, ADI is designed to function as a short integrated instructional unit (NRC, 2005) and engages students in a sequence of activities that are intended to help students understand important concepts and participate in both the empirical and representational practices of science (Walker et al., 2011; Walker et al., 2012). The ADI instructional approach, therefore, provides opportunities for the social construction of knowledge through collaborative work inside the lab and for individual sense-making through writing, review and revision of laboratory reports. This approach thereby leverages both major spaces of learning while engaging deeply with important science content to frame students’ activity. The overarching outcome is the development of practices and habits-of-mind that are aligned with the scientific community. An example of an investigation from the student lab manual is available as Appendix A (ESI†).

The second method of instruction is best described as a traditional expository laboratory activity. The intent of the traditional method of instruction is to give students an opportunity to learn to collect data and draw conclusions based on the data they collect. To accomplish this goal, the traditional approach is prescriptive in nature, providing methods for data collection and analysis to the students in order to find a pre-determined result. Students are also not given an opportunity to share, critique, or revise arguments as part of an investigation. The traditional approach, which tends to be the ‘business-as-usual’ approach to lab instruction across the U.S., consists of five major stages (see Fig. 2). In the first stage, the instructor provides an overview of the content at the heart of the lab activity and describes the expected outcomes of the investigation. The instructor then presents the students with a concept to investigate and provides them with a procedure to follow in order to gather the data needed to answer the research question. Next, the students work in pairs to collect data and record their observations or measurements in data tables that are supplied to them. The next stage is data analysis wherein learners are guided by the instructor through the process of developing a conclusion based on data that is then compared to the expected or known value. In the final stage of the model, students communicate their findings and make connections to important scientific concepts or principles by writing a structured lab report or answering a set of summative questions. This approach is efficient in terms of addressing multiple topics with limited instructional time, but limits opportunities for students to participate in the representational scientific practices. In addition, this type of instructional approach emphasizes individual or cooperative group work inside the lab but does little to foster collaboration between students or the social construction of knowledge through the development of shared goals, norms, and epistemological commitments that are aligned with the scientific community. An example of an investigation from the student lab manual is available as Appendix B (ESI†).


	Fig. 2 Steps for traditional laboratory instruction.

Student learning in both settings was measured using a laboratory practical exam, which required students to use core ideas in chemistry to plan and carry out an investigation, to analyze and interpret data, and to generate a written argument. Student performance on the laboratory practical was collected in the traditional laboratory course during the spring semester (the baseline condition) and then a comparison set of exams was collected one year later following the implementation of ADI (the treatment condition).

Theoretical framework

This study is grounded in a social constructivist theory of learning; a perspective on learning founded on the basic assumption that knowledge cannot be transmitted directly from one person to another, but is actively built or “constructed” by the learner based on interaction with others (Vygotsky, 1978; Bransford et al., 1999). In science these interactions are embedded within a culture that is shaped by tools and symbols which are not found in nature, but are constructs of the scientific community (Hodson, 2008). In light of social constructivist views of learning, rather than being a place where students verify concepts taught in the lecture, the laboratory should provide a setting where students can interact directly with the material world, using the tools, data collection techniques, models and theories of science. The specific disciplines that underlie undergraduate level courses, such as physics, biology and chemistry, have established bodies of knowledge, a unique language, and rules for gathering evidence and evaluating results, broadly speaking science practices. A discipline specific laboratory course therefore, presents a unique platform for providing students an opportunity to participate in and develop proficiency in the scientific practices that are valued within a discipline.

Ford's (2008) description of the nature of scientific practices provides a lens to illuminate what students do when they participate in science practices within a classroom. Ford describes the “material practices” of science as having two distinct but complementary components. The first component are practices related to manipulating nature to study aspects of it and the second component are those practices that are related to “making nature’s behavior apparent” to peers (p. 408). The first component consists of practices such as conducting investigations, analyzing data and the like, and these are the practices that are very much a staple of science laboratory courses. The second component includes those practices that scientists engage in order to portray or represent nature in ways that will be convincing to others in the community. Ford suggests that laboratory results themselves do not directly produce scientific claims; rather they become the focus of the rhetorical efforts of the scientific community. This dual nature of the practices of science is important to recognize, as it places a premium on the role of both the natural world and the community in the enterprise of science.

From this perspective, a laboratory course should provide access to the practices of science that serve both the empirical and representational functions. For example, a laboratory course can provide a setting where students can interact directly with the material world to make sense of natural phenomena, using the tools, data collection techniques, models and theories of a discipline. In this venue, students can conduct investigations in conversation and collaboration with peers as well as more experienced members of that community. As students engage in conversations with others, they can draw on their expertise; explain, extend, and reflect on their own ideas; and gain exposure to the habits of mind exhibited by disciplinary experts (Sunal et al., 2004). Through these types of interactions, learners can encounter and explore ideas that are valued in a discipline, how these ideas are used, and the ways that new ideas are validated; that is, students learn what constitutes legitimate knowledge in a field (Linn and Burbules, 1993; Engle and Conant, 2002; Ford, 2008). As students engage in these types of activities they learn how to use the core ideas of a field to explain natural phenomena and how to develop new ideas in a manner that is consistent with the norms, discursive habits, and epistemological commitments of the discipline. Ford (2008) describes this process of learning how to use core ideas and how to develop new ideas in a field as developing a “grasp of practice” and argues that such an understanding is fundamental to learning science.

Review of the literature

Interventions that are designed to address the decline of students studying science have ranged from the development of new pedagogical approaches, to teacher and faculty professional development initiatives, to the establishment of programs that provide extra support for students (NRC, 2012, 2014; Chandrasena et al., 2014). Assessing the impact of these different inventions has been difficult for a multiple of reasons, including but not limited to, differences in contexts, variation in implementation, and the use of different research designs or measures. The authors of the discipline-based education research (DBER) report, which is based on comprehensive review of current research that focuses on teaching and learning of science at the undergraduate level, therefore call for more research that investigates how teaching affects learning in a discipline from a perspective that reflects the discipline’s priorities, worldview, knowledge, and practices (NRC, 2012). Given this recommendation, we focus our review of the literature on ways to improve the teaching and learning in chemistry laboratory courses using outcomes that are valued by chemistry educators. These outcomes include helping students understand the core ideas of chemistry (NRC, 2012), helping students learn how to participate in scientific practices (Ford, 2008), and bringing more, and a wider diversity, of individuals into the field (NRC, 2011; Chandrasena et al., 2014).

The general chemistry sequence is a foundational component of the undergraduate curriculum, not just for chemistry majors, but also for most science disciplines. The impact of general chemistry sequence in any STEM major on students’ ability to move forward in their chosen science discipline can be significant. A primary recommendation for science reform is for more inquiry-based instruction in science courses (National Research Council, 2005; Lyons, 2006; Chandrasena et al., 2014), and several methods of adapting the chemistry lecture courses have demonstrated the value of moving away from a straight lecture format. The positive impact of instructional practices like the SCALE-UP project, Process Oriented Guided Inquiry Learning (POGIL) and Peer Led Team Learning have been well documented in the literature (Gosser and Roth, 1998; Farrell et al., 1999; Lee, 2004; Lewis and Lewis, 2005). The potential for development of argumentation in lecture has been explored by researchers using Peer-led Guide Inquiry (PLGI) in introductory courses (Kulatunga et al., 2013) and POGIL in upper division courses (Moon et al., 2016).

While laboratory courses are a well-established element of the undergraduate science curriculum, research that examines what students learn from their experiences inside the laboratory has not been the focus of much research (NRC, 2012). The literature that is available indicates that the goal of most undergraduate science laboratory courses is to demonstrate or validate concepts that are introduced in a lecture course and to provide some training in proper laboratory technique (Reid and Shah, 2007). As with so many aspects of education, this goal has not changed in response to the decreased need for bench chemists, the increase demand for a wider diversity of chemists, and the realization that chemistry research requires highly specialized knowledge that can only be developed from strong grasp scientific practices (NRC, 2012). The laboratory investigations that are included in these courses are therefore designed to ensure that students get the “right” results and draw the “appropriate” conclusions from these results. Although these expository or “cookbook” labs can be highly efficient for introducing students to a wide range of topics over the course of a semester, students tend to learn little from these types of activities (Domin, 1999; Hofstein and Lunetta, 2004). Numerous educators, as a result, have called for major changes in the nature of laboratory activities and more research that examines what students actually gain from engaging in different types of laboratory activities (Hofstein and Mamlok-Naamon, 2007; National Research Council, 2012; Bruck and Towns, 2013).

Studies across several disciplines have demonstrated that laboratory instruction that offers students more opportunities to engage in science practices can improve learning outcomes, particularly those outcomes related to improved conceptual understanding of the content. Burke, Greenbowe and Hand (2006), for example, examined what students learn from participating in Science Writing Heuristic (SWH) laboratory exercises during a General Chemistry laboratory course. SWH laboratory exercises are designed so students have an opportunity to decide what parameters to investigate and then come to understand what they are doing and why they are doing it as part of the process. The results from this study indicate that students, who were enrolled in sections of the laboratory course with adequate implementation of SWH, had improved scores on a lecture exam question targeting conceptual understanding. This study, however, not did examine other learning outcomes such students’ ability to participate in scientific practices. It also did not examine the effect of the intervention on bringing more, and a wider diversity, of individuals into the field. Schroeder and Greenbowe (2008) investigated the impact of combining POGIL based lecture instruction with SWH laboratory instruction in an introductory organic course. This study focused on one topic, nucleophilic substitution, from one exam, however there was a clear distinction between the performance of students in traditional courses and those in the POGIL-SWH course. This type research is encouraging as far as it goes, but the impact on multiple topics and student ability to engage in scientific practices has not been evaluated by these researchers.

Etkina and colleagues (Etkina and Van Heuvelen, 2007) have developed and studied an approach to physics instruction called Investigative Science Learning Environment (ISLE). ISLE helps students learn physics by engaging in processes that mirror the activities of physicists when they construct and apply knowledge. These processes involve observing, finding patterns, building and testing explanations for the patterns, and using multiple representations to reason about physical phenomena. In ISLE, students are assessed for conceptual understanding, for problem-solving ability, and, most importantly, for their use of various scientific abilities. Traditional instruments such as the Force Concept Inventory (FCI) and the conceptual survey of Electricity and Magnetism demonstrated that gains by ISLE students where comparable to other reformed courses (Hestenes et al., 1992; Maloney et al., 2001). Several non-traditional assessment measures were employed to determine if ISLE students acquire abilities used in the practice of science and engineering (Etkina and Van Heuvelen, 2007). The impact of the physics instructional environment was evaluated for at-risk students (particularly minorities and women) and found significant improvement in retention of minority and female students compared to a traditional course, however this result was based on course grades which were determined differently in the ISLE courses (Etkina, 1999). In a comparison study, ISLE students who had actually planned and carried on an investigation were significantly more capable of transferring their abilities to novel experimental situations than students who did not have an opportunity to design their own investigation (Etkina et al., 2010). While ISLE has been extensively studied for 15 years, there are few studies directly comparing ISLE to traditional instructional practices. In addition, the existing studies seldom focus on the ability of all students, including at-risk groups, to participate in important scientific practices, such as analyzing and interpreting data, arguing from evidence, or communicating information in writing.

Cooper (1994) developed an innovative laboratory format for General Chemistry and Organic Chemistry Laboratories that exposed students to the process of scientific problem solving, emphasized collaborative work, and required students to communicate their results both orally and in writing. In this format, groups of 3 or 4 students work together on three multistep, open-ended projects over the course of a semester rather than a set of closed exercises that take one lab period to complete. The students devise their own experiments with assistance from teaching assistants rather than following scripted instructions from a lab manual. The effects of these laboratories on students have been very encouraging, particularly with regard to increased achievements and retention rates for females. In one study, for example, Cooper and Kerns (2006) found that a comparison of mean exam scores for females showed a consistent pattern in which females in the collaborative laboratories scored higher on the lecture examinations than their counterparts in conventional laboratories. In addition, the retention rate for females who were enrolled in the collaborative laboratories was greater than that of their counterparts in conventional laboratories. The effect for the females in the collaborative laboratories essentially made their performances and retention rates equal to the males in the class.

Inquiry and argumentation are complementary goals that make laboratory experiences more scientifically authentic and educative for students (Jimenez-Aleixandre, 2008; Osborne, 2010). Past research on ADI also indicates that there are numerous potential benefits associated with using an instructional approach that emphasizes student participation in scientific practices during an undergraduate General Chemistry I laboratory course. For example, in one study that compared General Chemistry I laboratory sections taught using ADI to traditional methods of instruction found improved student attitudes towards science, with a significant positive gender effect for female students (Walker et al., 2012). In two other studies of the General Chemistry I laboratory course, students enrolled in an ADI section make significant improvement in ability to write lab reports over time (Sampson and Walker, 2012; Walker and Sampson, 2013b) and showed significant growth in their ability to participate in scientific argumentation (Walker and Sampson, 2013a). These studies suggest that ADI is an instructional model that could address the national need for authentic laboratory experiences that engage students in identified science practices. These studies, however, did not focus on other important scientific practices, such as planning and carrying out investigations or analyzing and interpreting data so it is unclear how ADI affects students’ ability to participate in a wide range of practices.

Taken altogether, these studies of undergraduate science laboratory instruction point towards a sincere need to engage students in a wide range of scientific practices if we desire them to learn more than just the ideas of science. Further, laboratory instruction should include scaffolding mechanisms that provide students opportunities to make sense of the experiences with these practices and give them feedback on their understanding and ability. These studies provide evidence that students will not develop a grasp-of-practice that will enable them to plan and carry out investigations, analyze and interpret data, argue from evidence, and communicate their ideas to others if they are not given an opportunity to actually participate in them.

Objective of the study and research questions

The primary objective of this study was to examine how laboratory instruction that emphasizes students’ active participation in a wide range of the practices of science affects student learning. The ADI instructional model, which is the focus of this study, provides opportunities for students to participate in both aspects of scientific practices—those involved in getting “nature to speak” or the empirical practices and those representational practices involved in “representing nature’s voice” (Ford, 2008, p. 408). This instructional model was compared to a more traditional laboratory instructional approach that primarily focuses on the first sense of practice—those involved in the manipulation of nature.

Given our review of the literature and theoretical perspective, we decided to focus on students’ ability to use the core ideas that are presented in lecture and laboratory and scientific practices to explain a phenomenon or to solve problem as the primary outcome of interest. This outcome, we argue, should be the major focus of chemistry laboratory courses at the undergraduate level. The research questions that guided the design of this study, as a result, were:

(1) How do opportunities to participate in the empirical and representational practices of science during a general chemistry laboratory course affect students’ ability to use core ideas and scientific practice to explain a phenomenon or to solve a problem?

(2) How does the performance of women, underrepresented minority students, and low achieving students compare to the performance of their counterparts in this context?

Method

Research design

We used a baseline comparison research design (Christ, 2007; Kratochwill et al., 2010) to answer the two research questions. The traditional approach to instruction, which is designed to engage in students in the empirical practices of science but in a prescriptive manner, was used during the spring semester of 2011 (baseline condition) and the ADI approach, which engages students in both the empirical and representational practices of science, was used during the spring semester of 2012 (treatment condition). The laboratory course was integrated with a lecture course so the students learned the same core ideas in both contexts. The outcome measure, which was a practical exam, was administered at the end of each semester in order to compare student learning in each context. All student data was collected and analyzed in compliance with the Institutional Review Board (IRB) at Florida State University. Students were informed of the research and given the opportunity decline to participate. Students participating in the study signed an IRB approved informed consent document during the first laboratory meeting.

Participants

One hundred and fifty-seven students enrolled in a General Chemistry II Laboratory course at a community college in the southeastern United States agreed to participate in this study. The community college was an open admission, comprehensive two-year institution serving nearly 40 [thin space (1/6-em)]

000 students annually at a main campus and four service centers located throughout the college’s statutory service area. The college has a number of STEM programs that recruit from the state’s high schools, paying particular attention to high needs schools in the state’s rural areas. The college ranks first among the 28 community colleges in the state in terms of percentage of graduates entering the state’s University system. The majority of the students enrolled in the course were General Transfer students seeking to move to a 4 year institution after receiving their Associates of Arts (AA) Degree. Table 1 presents demographic data for the students who agreed to participate in the study and all the students enrolled in the course during the study.

Table 1 Participant demographics

N	Female students	URM^a students	Age mean (range)	GPA mean (range)
a Underrepresented minority; black, hispanic or multiracial.
157	73 (46.5%)	50 (31.8%)	21.5 (17–42)	3.12 (1.65–4.00)

Equivalence of the participants in each condition

In order to attribute differences in achievement to a method of instruction, it is important for the participants in each condition to be equivalent. One way to accomplish this task is to randomly select a group of individuals to participate in a study and then use a random method of assignment to place the participants into either a treatment group or a comparison group. This type of approach is used in experimental designs such as the randomized controlled trial or the randomized cluster control trial. In this study, however, we decided to use a baseline comparison design rather than a more traditional experimental design because the faculty at the community college elected to adopt and use the ADI instructional approach in all sections of the General Chemistry II laboratory course beginning in the Fall of 2011. It was therefore not possible to randomly assign sections of the course to different conditions. The random assignment of sections of the course to either a treatment or control condition within a single semester would also have done little to assure equivalence of the groups because the community college only offers 6 to 8 sections of the course each semester. In addition, using a baseline comparison design eliminates the need to control for crossover between students, student self-selection, and instructor interaction effects because every section is taught the same way during a semester. We therefore decided that the baseline comparison research design was the most appropriate one to use in this context given our objective and research questions.

The equivalence of students in each condition within a baseline comparison design is based on the assumption that students who enroll in a course at a specific institution are equivalent in terms of demographics and past achievement over a short period of time, in this case six semesters. In order to test this assumption, we first examined the distribution of female students across the two conditions (traditional laboratory instruction = baseline condition; ADI laboratory instruction = treatment condition). The distribution of students in each condition is provided in Table 2. We conducted a 2 × 2 contingency table analysis in order to determine if the differences observed in the proportion of females within the two conditions were significant or not. The two variables included in each analysis were condition and gender. The results of this test indicate that there was no significant difference in the proportion of females across the two conditions, χ² (1, 157) = 0.28, p = 0.60 (Fisher’s exact test, p = 0.63, two-tailed).

Table 2 Female and URM students in each condition

Condition	N	Male	Female	White, asian, unreported	URM
Traditional (baseline)	76	39 (51%)	37 (49%)	54 (71%)	22 (30%)
ADI (treatment)	81	45 (56%)	36 (44%)	53 (65%)	28 (35%)

We then examined the distribution of underrepresented minority students across the two conditions. In order to determine if the differences observed in the proportion of underrepresented minority students within the two conditions were significant or not, we conducted a second 2 × 2 contingency table analysis. The two variables included in each analysis were condition and majority or minority status (White, Asian, or unreported and underrepresented minority). The results of this test indicate that there was no significant difference in the proportion of underrepresented minority students across the two conditions, χ² (1, 157) = 0.57, p = 0.45 (Fisher’s exact test, p = 0.49, two-tailed).

Next, we compared the ages of the participants in each condition because there is often a wide range of students in terms of age who are enrolled in courses offered at community colleges. Table 3 provides the means and standard deviations of the age of the participants in each condition. We conducted an independent samples t-test to determine if the age of the participants in the two conditions was significantly different. This test met the assumptions of independence, normality, and homogeneity of variance. The test was not significant, F(148.7) = 0.05, p = 0.96. The result of this test indicates that the participants in the baseline condition and the participants in the treatment condition are therefore equivalent in terms of age as well.

Table 3 Comparison by age, GPA and ACS-GCST exam score

	Condition		t	df	p
	Traditional (N = 76)	ADI (N = 81)
	Mean (SD)	Mean (SD)
a Equal variance was not assumed.
Age	21.57 (3.40)	21.53 (4.48)	0.05	148.7^a	0.96
GPA	3.10 (0.45)	3.14 (0.55)	0.51	152.4	0.61
ACS-GCST	12.75 (4.46)	13.32 (4.33)	0.81	154	0.42

We then examined past academic achievement as a fourth indicator of equivalence between the two groups of participants. We used grade point average (GPA) as a measure of past academic achievement. Table 3 provides the means and standard deviations of GPA for the participants in each condition. We conducted an independent samples t-test to determine if the GPA of the participants in each condition were equivalent. This test met the assumptions of independence, normality, and homogeneity of variance. The test was not significant, F(152.4) = 0.51, p = 0.61. The result of this test suggests that the participants in the baseline condition and the treatment condition were equivalent in terms of past academic achievement.

Finally, we used the American Chemical Society General Chemistry (Conceptual) Second Term (ACS-GCST) Exam to ensure that the students in both conditions had an equivalent understanding of the core ideas that were presented during the lecture at the end of the semester. The ACS-GCST consists of 30 items that target the concepts of physical properties and intermolecular forces, phases and phase changes, kinetics, equilibrium, acid–base chemistry (including pH and buffers), and electrochemistry. The lecture addressed all six of the concepts during the Spring 2011 and Spring 2012 semesters. Table 3 provides the means and standard deviations of ACS-GCST exam score for the participants in each condition. We conducted an independent samples t-test to determine if the exam score of the participants in each condition were equivalent or not. This test met the assumptions of independence, normality, and homogeneity of variance. The test was not significant, t(154) = 0.811, p = 0.42. This test, as a result, indicates that the scores of the students in the baseline condition (M = 12.75) and the ADI condition (M = 13.32) on the ACS-GCST exam were equivalent. This result suggests that student understanding of the core ideas presented in the lecture at the end of each semester was the same across conditions.

Nature of instruction in each condition

The objective of the General Chemistry II Laboratory course is to provide students with an opportunity to conduct investigations focused on Core Ideas identified by the faculty as important for understanding chemistry; the nature of intermolecular forces, acid–base titration, electrochemistry, kinetics, chemical equilibrium, and buffers (see Table 4). The laboratory had six octagonal tables that accommodated up to four students for a total enrollment of 24 students per laboratory section. The laboratory sections were scheduled daily in 3 hour blocks.

Table 4 Comparison of laboratory investigations

a Technique instruction activity.
Intermolecular forces
Traditional	None
ADI	Structure determines properties: molecular models^a
ADI	Why do liquids evaporate at different rates?

Titration
Traditional	Determination of formula weight of unknown acid
	Computer-based graphical analysis
	pH titration curves for a weak acid/strong base
ADI	Standardization of base and statistical analysis^a
ADI	How much acetic acid is in vinegar?

Colligative properties
Traditional	Molar mass from freezing point depression
ADI	None

Electrochemistry
Traditional	The electrolytic cell
ADI	Preparation of standard solutions^a
ADI	Which metals will make the best battery?

Kinetics
Traditional	Kinetic study of a chemical reaction
ADI	Serial dilutions and Beer’s law^a
ADI	How fast does the crystal violet decolorize?

Equilibrium
Traditional	Determining equilibrium constants using spectroscopy
ADI	What factors affect chemical equilibrium?^a
ADI	What is the formation constant for FeSCN²⁺?

The ADI approach is designed to engage students in both the empirical and representational practices of science and to encourage them to use the core ideas of science to explain natural phenomena or to solve problems as members of a sense-making community. The scientific practices that students participate in during each ADI laboratory investigation are provided in Table 5. The students participated in 5 ADI investigations during the course of the semester (see Table 4).

Table 5 Opportunities to participate in scientific practices

Type of scientific practice	Opportunity for students to participate in a scientific practice
Type of scientific practice	Traditional	ADI
Empirical
Plan an investigation	No	Yes
Carry out an investigation	Yes	Yes
Analyze and interpret data	Yes	Yes

Representational
Construct an explanation	Yes	Yes
Argue from evidence	No	Yes
Share findings with peers	No	Yes
Evaluate and critique ideas	No	Yes

The week prior to each ADI investigation, students engaged in an activity that allowed for instruction in specific techniques that would be needed for the ADI investigation. For example, students were explicitly taught how to conduct a titration through the process of determining the molarity of the sodium hydroxide solution they would use to titrate vinegar. Each combination of technique instruction and investigation took about 200 minutes to complete and spanned several class periods. The students had an opportunity to participate in all 7 of the practices of science listed in Table 5 during each investigation. As described above, students in the ADI condition worked in teams of 3 to 4 to plan an investigation that answered the guiding question. The outcome of these investigations was designed to vary considerably which allowed students the opportunity to engage in argumentation in support of their results and to rationalize anomalous results. Students remained in the laboratory until all six groups had completed their investigation, constructed a preliminary argument and engaged in argumentation with other groups. Each of these investigation involved students in designing methods, analyzing and interpreting data, arguing from evidence, evaluating information and critiquing arguments constructed by others, and writing about their investigation outcomes.

In contrast, as shown in Table 5, students in the traditional labs completed 9 laboratory investigations. For these, the students were supplied with a step-by-step procedure to follow, a data table to complete, and a set of analysis questions to answer during each investigation. The outcome of each investigation was well defined and focused on ensuring that the students collected and analyzed data in the same way. The students worked in pairs during the traditional labs, but completed the analysis of the data and its write up individually. The write-up consisted of a combination of formal lab reports and answers to a set of analysis questions. Each investigation took about 150 minutes to complete and was finished in a single class period.

The students in the traditional labs, as a result, completed more investigations than the students in the ADI condition but spent less time on each investigation (although the total amount of time spent in lab across conditions was equivalent). In addition, the students in the traditional labs had an opportunity to participate in three of the practices of science listed in Table 5 during each investigation. Two of the practices, which are heavily privileged during traditional laboratory instruction, provided students with empirical experiences including conducting investigations and analyzing data collected from those efforts. With respect to the representational practices of science, students in the traditional condition were provided cursory experiences with constructing scientific explanations, often in response to a ‘Why?’ question following summative assessment questions.

All students were assessed using on-line pre-laboratory questions, class participation and individual lab reports, as well as the laboratory practical. The lab report formats are compared in Table 6. In addition, the examples of laboratory investigations provided as Appendices A and B, provide investigation specific details to guide student’s in writing their laboratory reports. In both instructional models, the laboratory reports were graded by the instructors using an established rubric and students were provided feedback in a timely manner.

Table 6 Comparison of laboratory report formats

Traditional lab reports

ADI lab reports

Title and purpose

Theory

Procedure

Data and calculations

Discussion

Conclusion

Section 1: What were you trying to explain and why?

Section 2: How did you go about your work and why?

Section 3: What is your argument?

All of the faculty, both fulltime and adjunct had taught the traditional labs for 5 to 20 years. Over the years, a set of instructional notes had been developed for each investigation and these were provided to new faculty and problematic aspects of each lab were highlighted further during the semester in weekly emails. The ADI instructional model was introduced over a three-year period in general chemistry I by two of the faculty in the study. New faculty were brought on-board initially through 10 weeks of professional development meetings during the semester, which resulted in a set of weekly instructional notes similar to those used in the traditional labs. The fulltime faculty participated as part of their service to the department and adjunct faculty were compensated at their standard hourly rate. Instructors with experience running ADI in general chemistry I were recruited for ADI implementation in the second semester course. Once ADI was the established curriculum, new instructors observed at least one three-week cycle of ADI taught by an experienced faculty member. This was sufficient for the new instructor to understand the methodology and then rely on the instructor notes and weekly emails highlights for the remaining laboratory investigations. During the first years of ADI implementation, the lead author observed laboratory instructors on a regular basis and established that this practice for faculty development and preparation was sufficient.

Data sources

We used a practical exam to measure how well the students were able to use core ideas in chemistry and the practice of science to explain phenomena or solve problems. The practical exam was developed and refined over several semesters prior to the baseline data collection primarily in traditional labs, but also in a single pilot ADI lab section. For several semesters, the exam was administered in selected laboratory sections. The students in these sections were encouraged to ask the instructor questions so the instructor would know what parts of the practical they did not understand. Through this process and based on students’ submitted answers, the laboratory practical was revised to ensure that students were using the target cognitive processes. All versions of the practical exam were submitted to faculty for expert review and changes were made based on their recommendations.

The final version of the practical exam included three sections (see Fig. 3). Each section included a specific task that targeted a specific core idea and a practice of science. The first task required each student to determine the molarity of a weak acid solution using titration with a strong base. This task targeted the students understanding of acid–base chemistry and the practice of planning and carrying out an investigation. The second task required each student to make observations about color change of a solution and temperature in order to identify a reaction as exothermic or endothermic. This task targeted the students’ knowledge of equilibrium and the practice of analyzing and interpreting data. In the third, and final task the students were given a data table with pH information for 0.1 M solutions of HCl, NaOH, HC₂H₃O₂, NH₄OH and a buffer. Using the original pH and changes in pH as a result of adding strong acid or base, the students were expected to identify the five solutions from a list and then use evidence to support their claims. This task targeted the students understanding of acid–base chemistry and the practice of arguing from evidence or the use of logical deduction and reasoning. Although each part of the practical exam was designed to target a specific practice, there was some overlap of practices between the three parts.


	Fig. 3 Laboratory practical exam.

Data analysis

In order to score the practical exam, two of the authors examined a data set collected from an earlier group of students to develop a rubric. The final version of the rubric is available in Appendix C (ESI†). The inter-rater reliably of the rubric, as measured by Cohen’s Kappa, is 0.84. Once the rubric was developed, the two scorers randomly divided the student papers from this study and scored each part of the practical exam. The assessments were blinded with respect to which laboratory condition they represented. In cases where student answers had a questionable fit with the rubric, the scorers would discuss the answer and come to consensus. An example of the practical exam for a student with the mean score (19) for the traditional sections is available in Appendix D (ESI†).

We then compared the practical exam scores from the students in each condition in order to answer the two research questions. To address question 1, we used an independent samples t-test to determine if student scores on the practical exam differed based on method of instruction. To address question 2, we conducted a series of 2 × 2 ANOVAs to evaluate the effect of method, demographics (sex, minority status, past academic achievement), and the interaction between method and demographics on student performance on the practical exam. We did not use a pre-test as a covariant in our analysis of the scores because the students in the baseline and treatment conditions were equivalent in terms of sex, minority status, and age composition as well as past academic achievement and content knowledge (see Table 3). In addition, we decided to only administer the practical exam at the end of each semester in order to eliminate a potential testing effect.

Results

The presentation of the results first describes the overall performance of the students in each condition on the practical exam. We then examine the performance of subsets of students in each condition. This practical exam, as noted earlier, was used to assess how well the students were able to use core ideas and science practices to explain a phenomenon or solve a problem.

Performance of all students on the practical exam based on type of instruction

We conducted an independent samples t-test to determine if student scores on the practical exam differed based on method of instruction. The independent variable for this test was method (traditional or ADI) and the dependent variable was the practical exam score. This test met the assumptions of independence, normality, and homogeneity of variance. The results presented in Table 7 indicate that the mean score of the students in the ADI sections of the course (M = 26.50) was significantly higher on the practical exam than the mean score of the students in the Traditional sections of the course (M = 19.07). An error bar graph that illustrates student performance on the practical exam as a function of method is provided in Fig. 4.

Table 7 Student performance on the practical exam

Practical exam	Condition				t	df	p	d
	Traditional		ADI
	M	SD	M	SD
a Significant at α = 0.016.
Section I	8.52	3.56	10.29	3.65	2.84	136	0.005^a	0.48
Section II	1.78	1.59	2.43	1.51	2.44	136	0.016^a	0.41
Section III	8.78	7.41	13.79	8.07	3.73	136	< 0.001^a	0.63
Total score	19.71	9.71	26.45	9.52	1.46	136	< 0.001	0.74


	Fig. 4 Practical exam scores.

Given the significant difference between the two groups, we conducted a series of independent sample t-tests to determine if student scores on each section of the practical exam (I, II, and III) differed based on method of instruction. We used the Bonferroni approach to control for Type-I error across the three comparisons and set alpha at 0.016 (0.05/3). The mean score and standard deviation on each section of the practical exam are provided in Table 7.

The result of the first test indicates that the mean score of the students in the ADI condition (M = 10.29) was significantly higher on section I of the practical than the mean score of the students in the traditional condition (M = 8.52). This section of the practical exam required the students to plan and carry out an investigation to determine the molarity of a weak acid solution using titration with a strong base. The result of the second test indicates that the mean score of students in the ADI condition (M = 2.43) was significantly higher on section II of the practical than the mean score of the students in the traditional condition (M = 1.78). This section of the practical exam required the students to identify a reaction as exothermic or endothermic based on their observations and knowledge of equilibrium (analyze and interpret data). The result of the third test indicates that the mean score of the students in the ADI condition (M = 13.79) was significantly higher on section III of the practical than the mean score of the students in the traditional condition (M = 8.78). This section of the practical exam required the students to first identify five different solutions based on the information available to them to support their claims using the data provided as evidence or to use logical reasoning and deduction. All effect sizes (see Table 7) calculated for these significant results reflect medium to large effects (0.5 < d < 0.8) using the scale described by Cohen's Kappa (Berry and Mielke, 1988). These effect sizes describe the magnitude of the significant differences between laboratory conditions.

Performance of females, URMs and Low GPA students

We conducted three different 2 × 2 ANOVAs to evaluate the effect of method, demographics, and the interaction between method and demographics on the mean score for students on the practical exam. The first test divided the participants into subgroups based on method (ADI or traditional) and sex (male or female). We divided the participants into subgroups based on method and minority status (White, Asian, and unreported or URM) for the second test. For the third, and final test, we divided the participants into subgroups based on method and past academic achievement (high or low GPA). To group the students based on past academic achievement, we used a median split of the students’ current GPA. All three tests met the assumptions of independence, normality, and homogeneity of variance.

The results of the first test indicated a significant main effect for method, F(1, 134) = 18.08, p < 0.001, partial η² = 0.12. The main effect of sex, F(1, 134) = 0.43, p = 0.52, and the interaction effect of method and sex, F(1, 134) = 0.02, p = 0.90, however, were not significant. These results indicate that the mean score on the practical exam for the students enrolled in the ADI sections was higher than the mean score for the students enrolled in the traditional sections regardless of their sex and that there was no difference in performance between males and females within each condition (see Fig. 5). The means and standard deviations for scores on the practical exam as a function of sex are provided in Table 8.


	Fig. 5 Practical exam scores by gender.

Table 8 Practical exam scores as a function of method and sex

Condition	Sex	N	Mean	SD
Traditional	Male	26	19.81	11.54
Traditional	Female	32	18.47	7.88
ADI	Male	44	26.91	9.04
ADI	Female	36	26.00	10.04

The results of the of the second test indicated a significant main effect for method, F(1, 134) = 20.58, p < 0.001, partial η² = 0.13 and minority status, F(1, 134) = 4.80, p = 0.03, partial η² = 0.04. The interaction effect of method and minority status, however, was not significant, F(1, 134) = 0.74, p = 0.39. The partial η² calculated for the significant result indicates a moderate to large effect size for this difference between the laboratory conditions and a small effect for minority status. The means and standard deviations for scores on the practical exam as a function of minority status are provided in Table 9. These results indicate that the mean score on the practical exam for the students in the ADI sections was higher than the mean score for the students in the traditional sections regardless of minority status (see Fig. 6). The mean score on the practical exam for the White, Asian, and non-reporting students was significantly better than the mean score for the underrepresented minority students in the traditional condition, t(56) = 2.02, p = 0.05, whereas the mean score for the URM students was not significantly different from the mean score for the White, Asian, and unreported students in the ADI condition, t(78) = 1.05, p = 0.30.

Table 9 Practical exam scores as a function of minority status

Condition	Minority status	N	Mean	SD
Traditional	White, asian, or non-report	43	20.53	9.76
Traditional	Underrepresented minority	15	14.87	8.11
ADI	White, asian, or non-report	52	27.37	9.35
ADI	Underrepresented minority	28	24.89	11.20


	Fig. 6 Practical exam score by minority status.

The results of the third test indicate a significant main effect for method, F(1, 134) = 19.51, p < 0.001, partial η² = 0.13 and past academic achievement, F(1, 134) = 10.34, p = 0.002, partial η² = 0.07. The interaction effect of method and past academic achievement, however, was not significant F(1, 134) = 0.27, p = 0.61. The partial η² calculated for the significant result indicates a large effect size for this difference between the laboratory conditions and a medium effect for past academic achievement. The means and standard deviations for scores on the practical exam as a function of past academic performance are provided in Table 10. These results indicate that the mean score on the practical exam for the students enrolled in the ADI sections of the course was higher than the mean score for the students enrolled in the traditional sections regardless of their past academic achievement. However, in the traditional section there was not a significant difference in the mean scores on the practical exam for high vs. low past academic achievement, t(58) = 1.788, p = 0.08, whereas in the ADI sections the mean score on the practical was significantly lower for the students with low past academic achievement, t(80) = 2.862, p = 0.005. The difference in performance of students on the practical exam within in each condition as a function of past academic achievement is illustrated in Fig. 7.

Table 10 Practical exam scores as a function of past academic achievement

Condition	Past academic achievement (GPA)	N	Mean	SD
Traditional	High	27	21.44	9.31
Traditional	Low	31	17.00	9.56
ADI	High	40	29.58	9.50
ADI	Low	40	23.43	9.72


	Fig. 7 Practical exam scores by past academic achievement.

Limitations and conclusions

All research designs involve tradeoffs and it is therefore important to discuss some of the decisions that we made when we designed this study. One such decision was to use the baseline comparison research design to compare the effect of instructional method on student learning during the same semester in consecutive years rather than using a clustered randomized control trial during a single semester. While the clustered randomized control trial is considered the gold standard for education research, there were issues and barriers associated with maintaining the integrity of this research design for this study since the students in the two conditions would share a lecture course and interact outside of the laboratory setting (Christ, 2007; Kratochwill et al., 2010). There were also, as noted earlier, several additional threats to the internal validity of this design in this context. These threats included, but were not limited to, the relatively small sample at the level of randomization (in this case laboratory sections), potential bias due to student self-selection, and instructor effects. We decided to use the baseline comparison research design, in part, because it enabled us to sidestep these issues since every section was taught using the same approach during a given semester. One major drawback of our choice to use the baseline comparison research design, however, is that we cannot assure the equivalence of the students in each condition through the use of random assignment.

Our conclusions therefore need to be viewed in light of several limitations. First, we are unable to rule out all potential alternative explanations for the observed differences in student performance on the practical exam due to the nature of the research design. Second, our choice of outcome (the ability to participate in scientific practices) and the instrument we used to measure it (a practical exam) provides us with only a limited picture of what students learned in each context. Our findings, as a result, may have differed if we choose to target and assess a different learning outcome that can be the focus of a laboratory course (e.g., conceptual, cognitive, psychomotor, epistemic, and affective). Third, in order to examine the performance of different demographic groups, the participant numbers get small, therefore, these results should be considered preliminary findings. However, it is important to stress that each analysis that we conducted met the assumptions required to use inferential statistics. Fourth, this study was conducted at a single institution, which further limits the generalizability of our findings. Fifth, half of the instructors that implemented ADI were adjunct faculty who had used the ADI instructional model in semesters prior to data collection, but with limited professional development. The way these instructors implemented ADI, as a result, may have reduced the potential total effect of this approach. Finally, as explained earlier we did not use a pre-test as a covariant in our analysis of the scores. With these limitations in mind, we will now provide our conclusions as tentative answers to our two research questions.

How do opportunities to participate in the empirical and representational practices of science during a general chemistry laboratory course affect students’ ability to use core ideas and scientific practice to explain a phenomenon or solve a problem?

Our results indicate that the students who had more opportunities to participate in the empirical and representation practices of science were able to use core ideas and scientific practices to explain a phenomenon or solve a problem better than students who only had an opportunity to participate in the empirical practices of science. These results, in other words, indicate that when it came to actually “doing” chemistry at the end of the semester, the students in the ADI condition performed better than their peers. In addition, the difference in scores on the lab practical and the observe effect sizes suggest that it is not the number of investigations that matters; it is what the students are expected to do during each investigation that affects learning. The students enrolled in the ADI sections of the course only participated in total of 6 investigations over the course of the semester but had an opportunity to participate in all 7 of the practices of science listed in Table 5 during each of these investigations. The students enrolled in the traditional lab sections, in contrast, completed a total of 9 laboratory investigations during the semester but only had an opportunity to participate in 3 of the 7 practices of science (see Table 5) during each one. In addition, the students enrolled in the traditional lab sections spent less time on each investigation than the students in the ADI condition (although the total amount of time spent in lab across conditions was equivalent).

It could be argued that the students in the traditional condition were unprepared by their instructional method for this type of assessment, yet this practical was not so technical to include specialized skills that only one group of students had an opportunity to learn. Indeed, the practical exam was developed using students in traditional laboratory sections and was adapted based on their performance. In addition, the scoring rubric allowed for equal points for students that used an argument framework or logical deduction on Part III. Therefore, it is reasonable to conclude that the observed difference in student performance on this measure speaks to the nature of student learning engendered by these two very different instructional approaches. Both groups of students, for example, had carried out a weak acid/strong base titration, but only the students in the ADI condition had planned and carried out an investigation on their own. Given this experience, the students in the ADI sections were more able to use their understanding of acid–base reactions and the practice of planning and carrying out an investigation at the end of the semester than the students who followed a prescribed procedure for conducting a titration. These findings resonate with studies of undergraduate laboratory instruction that demonstrate actual participation in scientific practices, both empirical and representational, can enhance students’ ability with those practices and their understanding of their function in science (Apedoe, 2008; Gormally et al., 2009; Etkina et al., 2010). Further, the comparison of these two instructional approaches in this study provides evidence that moving laboratory instruction towards more student-centered inquiry does not inhibit students’ learning of science concepts rather it expands their learning to include practical aspects of the scientific enterprise (Gormally et al., 2009; Etkina et al., 2010; Chen et al., 2014).

In considering the two types of practice framing this study, only ADI provided students the opportunity to participate in both the empirical and representational aspects of science. The traditional condition is typical of most laboratory curricula (Bruck and Towns, 2013), which as we outlined in Table 5 do not cover the full range of representational practices. If we are to help students use their content understanding to engage in the practices of science so they can actually do chemistry, then our findings suggest that more representational experiences are necessary. These representational practices serve two critical functions that have been emphasized by other studies involving undergraduate laboratory instruction. First, the cognitive work engendered by these practices affords students’ opportunities to reflect on their own sense making of science, which improves their understanding (Apedoe, 2008; Gormally et al., 2009). Second, by engaging in representational scientific practices, particularly argumentation, social interactions with other students and instructors provide a forum for feedback, which is critical to improving students’ science proficiency (Etkina et al., 2010).

How does the performance of women, underrepresented minority students, and low achieving students compare to the performance of their counterparts in this context?

The value of using of an instructional approach that engages students in a wide range of scientific practices is further supported when the data from different student groups were disaggregated. All students enrolled in the ADI sections of the course, including students from under-represented minority groups, females and students with low GPAs, outperformed their counterparts who were enrolled in the traditional sections. In addition, students from underrepresented minority groups and females in the ADI group performed as well as the students from majority groups and males in the ADI section of the course and outperformed the students from majority groups and males who were enrolled in the traditional sections. This finding is important because it suggests that authentic, interactive, and educative approaches to laboratory instruction that give all students an opportunity to participate in both empirical and representational practices of science, such as ADI, may help all students be more successful in chemistry.

This finding is also noteworthy because despite research in other disciplines to the contrary (Treisman, 1992) there persists a perception among science teachers and faculty that instructional methods that seek to engage students in authentic scientific practices, such as investigation design and argumentation, are beyond the abilities of low performing students (Sampson and Blanchard, 2012), and so science teachers often do not attempt to engage students in the disciplinary practices of science (Yore et al., 2003; Banilower et al., 2013). Indeed, research has demonstrated that the nature of instruction for the low performing students is often fundamentally different from and more simplistic than that of their higher performing peers (Yerrick et al., 1997; Yerrick, 2000), with ambitious instruction reserved for the elite, so that lower performing students are engaged in instruction that limits students’ input and contribution. While the overall finding was that the mean scores on the practical exam were higher for the students in the ADI sections, the increased gap between students with high past academic achievement and students with low past academic achievement warrants further investigation. In addition, faculty that implement non-traditional instruction in the laboratory should be cautious study if they assign grades on the curve, as students with lower past achievement might have a harder time achieving good grades, even if they are learning more than they might have in a traditional setting.

Perhaps, the most interesting result of this analysis is the closing of the achievement gap for the URM students in the ADI laboratory courses. These students are often marginalized in the traditional science classrooms (Banilower et al., 2013), something that may be due to false perceptions of role, lack of socialization, lack of explicit scaffolding for scientific discourses and practices, decreased socialization, or lack of an identity congruent with success in science. Graham and colleagues (2013) recently provided a framework focused on increasing the persistence of students, particularly those from underrepresented minorities, in STEM majors. This framework, developed from evidence drawn from relevant programs and studies, argues that to encourage persistence in STEM, students must learn science and develop an identity as a scientist. Two critical elements necessary for achieving the confidence and motivation to pursue a STEM degree entail the use of active learning strategies and involvement in learning communities. Warren et al. (2001), further suggest that given the opportunity to employ their everyday experiences and informal language these students who find themselves otherwise marginalized in school were able to create possibilities for both seeing and encountering phenomenon differently. Essentially offering alternative perspectives that were productive for them as well as typically academically successful students.

Implications

There is general agreement in the literature that the original role of the laboratory, which was to train future chemists for bench work, is no longer valid or useful (Reid and Shah, 2007; Bruck and Towns, 2013). Several chemistry educators have therefore called for research-based evidence of what students actually gain from engaging in laboratory activities in order to help faculty develop laboratory curricula and maintain institutional resources devoted to laboratory courses. This study attempted to address these issues by specifically defining the chemistry laboratory as a venue for students to learn and participate in the practices of science rather than as a place to demonstrate or verify concepts introduced in lectures or to learn basic laboratory technique. The students in the baseline (traditional) and the students in the treatment (ADI) groups had equivalent lectures and indeed gained equivalent conceptual understanding of the content at the end of the semester and both groups had an opportunity to use the same lab equipment. The distinction came in students’ ability to use the core ideas addressed in lecture and scientific practices to explain a phenomenon or to solve a problem. The ability to use content and equipment to solve a problem or to answer a question by engaging in scientific practices, we argue, is at the very core of understanding of chemistry and indeed an understanding of the nature of science in general. The ADI model may push all students to develop new ways of explaining and problem solving by scaffolding a learning environment that has explicit rules of engagement that levels the playing field, so to speak. By creating these interactive and educative learning spaces, students may have a chance to learn about science through the application of content matter through practice, increasing their self-confidence and motivation to identify as a scientist.

The results of the current study, in combination with other studies, establishes a substantial body of work that not only demonstrates that students may make significant improvement in their attitudes towards science and ability to participate in argumentation, design and carry out investigations, analyze and interpret data, and write in a scientific manner, when ADI is used in a Chemistry lab course. This research also demonstrates the inadequacy of traditional laboratory instruction to achieve these same goals. The possibility that ADI instruction could mediate these issues is very encouraging and holds promise for better meeting the needs of a wide range of science learners through non-traditional instructional methods.

It is important to note as well, that ADI is just one of several instructional models that have the potential to produce similar results in student ability to engage in the practice of science. There is a growing body of evidence indicating that transformation of the traditional expository laboratories can have benefits in terms of student learning and preparing students to become members of the discipline (Cooper and Kerns, 2006; Poock et al., 2007; Rudd et al., 2007; Schroeder and Greenbowe, 2008; Sampson and Walker, 2012; Sandi-Urena et al., 2012; Walker and Sampson, 2013a; Hunnicutt et al., 2015; Pabuccu and Erduran, 2016). Undergraduate science laboratory courses should therefore be viewed as a place for students to learn to engage in the practices of science in order to explain phenomena or solve problems rather than just a place to reinforce the conceptual outcomes that are already the focus of lecture courses. When viewed in this manner, the laboratory can serve a vital and necessary role in the undergraduate science curriculum. It may also play a more fundamental role in current efforts to keep more, and a wider diversity, of students in STEM.

Notes and references

Apedoe X. S., (2008), Engaging students in inquiry: tales from an undergraduate geology laboratory-based course. Sci. Educ., 92(4), 631–663. DOI: http://10.1002/sce.20254.
Banilower E. R., Smith P. S., Weiss I. R., Malzahn K., Campbell K. and Weis A., (2013), The 2012 National Survey of Science and Mathematics Education, Chapel Hill, NC: Horizon Research.
Berry K. J. and Mielke P. W., (1988), A Generalization of Cohen's Kappa Agreement Measure to Interval Measurement and Multiple Raters, Educ. Psychol. Meas., 48(4), 921–933.
Bransford J., Brown A. and Cocking R., (1999), How people learn: brain, mind, experience and school, Washington, DC: National Academy of Science Press.
Brown B., Henderson J. B., Gray S., Sullivan S., Donovan B., Patterson A. and Wagstaff W., (2012), The Black Scientists Project: An examination of success and access to science careers, Stanford, CA: Stanford University.
Bruck A. D. and Towns M., (2013), Development, implementation, and analysis of a national survey of faculty goals for undergraduate chemistry laboratory, J. Chem. Educ., 90(6), 685–693.
Burke K., Greenbowe T. and Hand B., (2006), Implementing the science writing heuristic in the chemistry laboratory, J. Chem. Educ., 83(7), 1032–1038.
Chandrasena W., Craven R., Tracey D. and Dillon A., (2014), Seeding Science Success: Relations of Secondary Students’ Science Self-concepts and Motivation with Aspirations and Achievement, Australian Journal of Educational & Developmental Psychology, 14, 186–201.
Chen S., Chang W.-H., Lai C.-H. and Tsai C.-Y., (2014), A Comparison of Students’ Approaches to Inquiry, Conceptual Learning, and Attitudes in Simulation-Based and Microcomputer-Based Laboratories, Sci. Educ., 98(5), 905–935. DOI: http://10.1002/sce.21126.
Christ T. J., (2007), Experimental control and threats to internal validity of concurrent and nonconcurrent multiple baseline designs, Psychol. Schools, 44(5), 451–459.
Cooper M., (1994), Cooperative chemistry laboratories, J. Chem. Educ., 71(4), 307–311.
Cooper M. and Kerns T., (2006), Changing the laboratory: effects of a laboratory course on students attitudes and perceptions, J. Chem. Educ., 83, 1356–1361.
Domin D., (1999), A review of laboratory instruction styles, J. Chem. Educ., 76(4), 543–547.
Engle R. A. and Conant F. R., (2002), Guiding principles for fostering productive disciplinary engagement: explaining an emergent argument in a community of learners classroom, Cognition Instruct., 20(4), 399–483.
Espinosa L., (2011), Pipelines and pathways: women of color in undergraduate STEM majors and the college experiences that contribute to persistence, Harvard Educ. Rev., 81(2), 209–240.
Etkina E., (1999), Lessons learned: a case study of an integrated way of teaching introductory physics to at-risk students at Rutgers University, Am. J. Phys., 67(9), 810. DOI: http://10.1119/1.19129.
Etkina E. and Van Heuvelen A., (2007), Investigative science learning environment–a science process approach to learning physics, Research-Based Reform of University Physics, 1. Retrieved from http://www.compadre.org/PER/per_reviews/media/volume1/ISLE-2007.pdf.
Etkina E., Karelina A., Ruibal-Villasenor M., Rosengrant D., Jordan R. and Hmelo-Silver C. E., (2010), Design and Reflection Help Students Develop Scientific Abilities: Learning in Introductory Physics Laboratories, J. Learn. Sci., 19(1), 54–98. DOI: http://10.1080/10508400903452876.
Farrell J., Moog R. and Spencer J., (1999), A guided inquiry general chemistry course, J. Chem. Educ., 76(4), 570–574.
Ford M. J., (2008), Disciplinary authority and accountability in scientific practice and learning, Sci. Educ., 92(3), 404–423.
Gormally C., Brickman P., Hallar B. and Armstrong N., (2009), Effects of Inquiry-based Learning on Students’ Science Literacy Skills and Confidence, International Journal for the Scholarship of Teaching and Learning, 3(2). DOI: http://10.20429/ijsotl.2009.030216.
Gosser D. and Roth V., (1998), The Workshop Chemistry Project: Peer-Led Team Learning, J. Chem. Educ., 75(2), 185–187.
Graham M., Frederick J., Byars-Winston A., Hunter A. and Handelsman J., (2013), Increasing persistence of college students in STEM, Science, 341(Sept. 27), 1455–1456.
Hannover B. and Kessels U., (2004), Self-to-prototype matching as a strategy for making academic choices. Why high school students do not like math and science, Learn. Instr., 14(1), 51–67. DOI: http://10.1016/j.learninstruc.2003.10.002.
Hestenes D., Wells M. and Swackhammer G., (1992), Force Concept Inventory, The Physics Teacher, 30, 141–158.
Hodson D., (2008), In Towards scientific literacy: A teachers’ guide to history, philosophy and sociology of science. Rotterdam, The Netherlands: Sense Publishers.
Hofstein A. and Lunetta V., (2004), The laboratory in science education: Foundations for the twenty-first century, Sci. Educ., 88(1), 28–54.
Hofstein A. and Mamlok-Naamon R., (2007), The laboratory in science education: the state of the art, Chem. Educ. Res. Pract., 8(2), 105–107.
Hunnicutt S. S., Grushow A. and Whitnell R., (2015), Guided-Inquiry Experiments for Physical Chemistry: The POGIL-PCL Model, J. Chem. Educ., 92(2), 262–268. DOI: http://10.1021/ed5003916.
Jimenez-Aleixandre M. P., (2008), Designing argumentation learning environments, in Erduran S. and Jimenez-Aleixandre M. P. (ed.), Argumentation in science education: Perspectives from classroom-based research, Dordrecht: Springer Academic Publishers, pp. 91–115.
Kratochwill T. R., Hitchcock J., Horner R. H., Levin J. R., Odom S. L., Rindskopf D. M. and Shadish W. R., (2010), Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
Kulatunga U., Moog R. S. and Lewis J. E., (2013), Argumentation and participation patterns in general chemistry peer-led sessions, J. Res. Sci. Teach., 50(10), 1207–1231. DOI: http://10.1002/tea.21107.
Lee V. S. (ed.), (2004), Teaching and learning through inquiry: a guidebook for institutions and instructors 1st edn, Sterling, Va: Stylus Pub.
Lewis S. and Lewis J., (2005), Departing from lectures: An evaluation of a Peer-Led Guided Inquiry alternative, J. Chem. Educ., 82, 135.
Linn M. C. and Burbules N. C., (1993), Construction of knowledge and group learning, in Tobin K. (ed.), The practice of constructivism in science education, Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, pp. 91–119.
Lyons T., (2006), Different Countries, Same Science Classes: Students’ experiences of school science in their own words. Int. J. Sci. Educ., 28(6), 591–613. DOI: http://10.1080/09500690500339621.
Maloney D. P., O’Kuma T. L., Hieggelke C. J. and Van Heuvelen A., (2001), Surveying students’ conceptual knowledge of electricity and magnetism, Am. J. Phys., 69(S1), S12. DOI: http://10.1119/1.1371296.
Moon A., Stanford C., Cole R. and Towns M., (2016), The nature of students’ chemical reasoning employed in scientific argumentation in physical chemistry, Chem. Educ. Res. Pract., 17(2), 353–364. DOI: 10.1039/C5RP00207A.
National Research Council., (2005), America’s Lab Report: Investigations in High School Science, Washington, DC: National Academy Press.
National Research Council., (2011), Expanding underrepresented minority participation: America’s science and technology talent at the crossroads, Washington, DC: National Academies Press.
National Research Council., (2012), Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. A report prepared by the Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research Board on Science Education, Division of Behavioral and Social Sciences and Education, (N. A. Press, Ed.). Washington, DC: National Research Council.
National Research Council., (2014), Undergraduate Chemistry Education: A Workshop Summary, Washington D.C.: National Academies Press. Retrieved from http://www.nap.edu/catalog/18555.
National Science Foundation, National Center for Science and Engineering Statistics., (2015), Women, Minorities, and Persons with Disabilities in Science and Engineering: 2015 (Special Report NSF No. 15–311). Arlington, VA. Retrieved from http://www.nsf.gov/statistics/wmpd/.
Osborne J., (2010), Arguing to learn in science: The role of collaborative, critical discourse, Science, 328(5997), 463–466.
Pabuccu A. and Erduran S., (2016), Investigating students’ engagement in epistemic and narrative practices of chemistry in the context of a story on gas behavior, Chem. Educ. Res. Pract., 17(3), 523–531. DOI: 10.1039/C6RP00011H.
Poock J., Burke K., Greenbowe T. and Hand B., (2007), Using the science writing heuristic in the general chemistry laboratory to improve students academic performance, J. Chem. Educ., 84(8), 1371–1378.
Reid N. and Shah I., (2007), The role of laboratory work in university chemistry, Chem. Educ. Res. Pract., 8(2), 172–185.
Rudd J., Greenbowe T., Hand B. and Legg M., (2007), Using the Science Writing Heuristic to improve students understanding of general equilibrium, J. Chem. Educ., 84(12), 2007–2011.
Sampson V. and Blanchard M., (2012), Science teachers and scientific argumentation: trends in views and practice, J. Res. Sci. Teach., 49(9), 1122–1148.
Sampson V. and Walker J. P., (2012), Learning to write in the undergraduate chemistry laboratory: the impact of argument-driven inquiry, Int. J. Sci. Educ., 34(10), 1443–1485.
Sandi-Urena S., Cooper M. and Stevens R., (2012), Effect of Cooperative Problem-Based Lab Instruction on Metacognition and Problem-Solving Skills, J. Chem. Educ., 89(6), 700–706. DOI: http://10.1021/ed1011844.
Schroeder J. D. and Greenbowe T. J., (2008), Implementing POGIL in the lecture and the Science Writing Heuristic in the laboratory—student perceptions and performance in undergraduate organic chemistry, Chem. Educ. Res. Pract., 9(2), 149–156. DOI: 10.1039/B806231P.
Sunal D. W., Wright E. L. and Day J. B. (Eds.)., (2004), Reform in undergraduate science teaching for the 21st century, Greenwich, CN: Information Age Publishing.
Treisman U., (1992), Studying Students Studying Calculus: A Look at the Lives of Minority Mathematics Students in College, Coll. Math. J., 23(5), 362–372. DOI: http://10.2307/2686410.
Villafañe S. M., Garcia C. A. and Lewis J. E., (2014), Exploring diverse students’ trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry, Chem. Educ. Res. Pract., 15(2), 114–127. DOI: 10.1039/C3RP00141E.
Vygotsky L. S., (1978), Mind in society. The development of higher psychological processes, Cambridge: Harvard University Press.
Walker J. P. and Sampson V., (2013a), Learning to argue and arguing to learn: Argument-driven inquiry as a way to help undergraduate chemistry students learn how to construct arguments and engage in argumentation during a laboratory course, J. Res. Sci. Teach., 50(5), 561–596.
Walker J. P. and Sampson V., (2013b), Using the laboratory to improve undergraduates’ science writing skills through meaningful science writing, peer-review and revision, J. Chem. Educ., 90(10), 1269–1274.
Walker J. P., Sampson V., Grooms J., Zimmerman C. and Anderson B., (2012), Argument-driven inquiry in undergraduate chemistry labs: the impact on students’ conceptual understanding, argument skills, and attitudes toward science, J. Coll. Sci. Teach., 41(4), 74–81.
Walker J. P., Sampson V. and Zimmerman C., (2011), Argument-driven inquiry: An introduction to a new instructional model for use in undergraduate chemistry labs, J. Chem. Educ., 88(10), 1048–1056.
Warren B., Ballenger C., Ogonowski M., Rosebery A. and Hudicourt-Barnes J., (2001), Rethinking diversity in learning science: the logic of everyday language, J. Res. Sci. Teach., 38, 529–552.
Wilson R. E. and Kittleson J., (2013), Science as a classed and gendered endeavor: persistence of two white female first-generation college students within an undergraduate science context. J. Res. Sci. Teach., 50(7), 802–825. DOI: http://10.1002/tea.21087.
Yerrick R., (2000), Lower track science students’ argumentation and open inquiry instruction, J. Res. Sci. Teach., 37(8), 807–838.
Yerrick R., Parke H. and Nugent J., (1997), Struggling to promote deeply rooted change: the “filtering effect” of teachers’ beliefs on understanding a transformational view of teaching science. Sci. Educ., 81, 137–157.
Yore L., Bisanz G. L. and Hand B. M., (2003), Examining the literacy component of science literacy: 25 years of language arts and science research, Int. J. Sci. Educ., 25(6), 689–725. DOI: http://10.1080/09500690305018.

Footnote

† Electronic supplementary information (ESI) available: Appendix A: example of traditiaonl laboratory investigation. Appendix B: example of argument-driven inquiry laboratory investigation. Appendix C: lab practical exam scoring rubric. Appendix D: example of student exam. See DOI: 10.1039/c6rp00093b