Participants' reflections two and three years after an introductory chemistry course-embedded research experience

Gabriela A. Szteinberg and Gabriela C. Weaver *
Purdue University, Department of Chemistry, 560 Oval Dr, West Lafayette, IN 47907, USA. E-mail: gszteinb@purdue.edu; gweaver@purdue.edu; Tel: +1 765-494-5280, +1 765-496-3055

Received 23rd August 2012 , Accepted 10th September 2012

First published on 10th October 2012


The Center for Authentic Science Practice in Education (CASPiE) is a course-embedded undergraduate research curriculum that aims to introduce authentic research experiences for students in their early college years. A cohort of students who were randomly assigned to the CASPiE laboratory sections was tracked during and two and three years after course enrollment. Students from the traditional laboratory sections were also tracked for control purposes. This paper presents students' perceptions of the laboratory course at the time of enrollment, based on their answers to an online survey, and a comparison to students' later responses to semi-structured oral interviews about their course involvement. We found that students' views of the curricula persisted over the years. CASPiE students thought the curriculum allowed them to learn how to do research and in turn gained confidence to do such work. In contrast, students from the traditional sections thought they did not learn the practice of research in their course. These and related comparisons are presented with evidence from survey results and interview vignettes. The implications of the CASPiE curriculum implementation and its effects over the long term are discussed.


Introduction and background

For over two decades, educational reform has called for inclusion of inquiry activities in the instruction of science, at all student age levels, to achieve scientific literacy (American Association for the Advancement of Science, 1990, 1993; National Research Council, 1996, 2007; Partnership for 21st Century Skills, 2009). Increasingly, decisions that affect people's everyday lives require them to be scientifically literate. Beyond that, the 21st century workforce will need technical skills, soft skills such as problem solving and teamwork, and scientific reasoning abilities in order to remain competitive. To achieve this goal of preparing students for these life and professional challenges, educational institutions need to provide opportunities that allow students to learn and practice inquiry approaches to science.

Historically, the teaching laboratory has played a major role in science education and has served as the main venue for students to practice scientific inquiry and develop skills to prepare them to be scientifically literate (Hofstein and Lunetta, 1982, 2004). In the laboratory, students have the opportunity to reproduce experiments and visualize concepts learned in a lecture course, become familiar with scientific equipment and techniques, and practice working in a group. Although there have been critiques about the effectiveness of the laboratory in achieving various teaching goals, along with the associated complications of cost and logistics (Baker and Verran, 2004), significant evidence exists to show that the laboratory is necessary in education and, when well implemented, students can learn greatly from their experiences and attain positive achievement goals (Matz et al., 2012; Roth, 1994; Tobin, 1990). There is no ideal way to perform laboratory activities, however. In fact, there are many types of laboratory instruction and the level of inquiry can vary as well (Bruck et al., 2008). The higher the level of inquiry, the more a student is responsible for their involvement in planning the laboratory work (Weaver et al., 2008). However, even at the highest level of inquiry, students do not always understand the nature of science or become more scientifically literate (Abd-El-Khalick et al., 1998; Lederman, 1992). Postulated reasons for this include that inquiry laboratory activities are still not implemented properly because consensus is yet to be established on what the best definition of inquiry is (Russell and Weaver, 2011); therefore, students still follow instructions as they would in a cookbook. Even though inquiry learning is definitely a recognized way to improve students' understanding of the nature of science, performance in science courses, and promotion of science interest in general (Hofstein and Lunetta, 2004; Hofstein and Mamlok-Naaman, 2007) the best ways to implement inquiry learning are still being developed and assessed.

Another place where students can engage in the process of scientific activity is through authentic research opportunities (Taraban and Blanton, 2008; Weaver et al., 2008). Undergraduate students who are involved in research become apprentices in a research group, typically at a university or research facility. The literature on research apprenticeships has shown that students and faculty agree that participants gain valuable skills such as learning research laboratory techniques, confidence with doing research work, feeling part of the scientific community, and soft skills such as giving presentations and writing scientific reports (Hunter et al., 2007; Seymour et al., 2004). In addition, students are more prone to stay in a science, technology, engineering, and mathematics (STEM) major, continue on to pursue graduate degrees in a STEM discipline, or choose a STEM-related career (Nagda et al., 1998; Russell et al., 2007; Slovacek et al., 2012). Given the beneficial results on students after they participate in an authentic research opportunity, it is of importance for the scientific community to provide such experiences for all students who may be interested in a STEM career.

At the college level, students typically take the opportunity to engage in research during their junior or senior years. Opportunities to do research may be part of a grant-based program such as a Research Experience for Undergraduates (REU), or they may be independent of such programs when students seek out an advisor to work in their laboratory during the academic year. Students who partake in this type of research experiences are usually a self-selected, high-achieving small percentage of the overall eligible undergraduate population (Sadler et al., 2010). Existing assessments of students doing research have focused on this student population, and therefore it is difficult to make a comparison with younger or lower achieving students who do not usually take part in research activities. These may be students who do not take the initiative themselves to seek out a research experience but who could potentially benefit from it.

As a way to expand research opportunities to students who are less likely to be engaged in research activities, the Center for Authentic Science Practice in Education (CASPiE) was created. CASPiE is a multi-institutional collaborative project that aims at providing course-embedded authentic research experiences for undergraduate students during their early years in college, specifically during their general and organic chemistry courses. This project was funded through the National Science Foundation. CASPiE provides undergraduate students with research opportunities at an early stage in their undergraduate college education (Weaver et al., 2006). Students get to work cooperatively on projects that will benefit actual researchers and their ongoing work.

Studies have been done to look at the effects of undergraduate research on factors such as student retention in STEM (Russell et al., 2007), attitudes (Freedman, 1997), and skills learned (Lopatto, 2004; Seymour et al., 2004). These studies were mostly done with students in their later college years, and with those who took the research experience as an extra load in their schedule or during the summer. It would be useful, therefore, to study the effects of early college course-embedded research experiences, such as the one that CASPiE provides, to examine the effects they may have on undergraduate students, especially in comparison to what is known about traditional research experiences. This study was done to understand the long-term outcomes of the CASPiE implementation.

Previous research on the CASPiE program has shown that it has been successful at delivering authentic research experiences for undergraduate students, engaging students from typically underrepresented populations such as women and students from ethnically diverse groups, and increasing student interest in science and chemistry, as well as their understanding of research methods (Scantlebury et al., 2011). CASPiE also seems to promote more sophisticated understanding of the nature of science, especially when students were asked to explain conceptions of theories and the process of the scientific method (Russell and Weaver, 2011). More recently, the longitudinal assessment on CASPiE has suggested that students from CASPiE graduate faster on average and perform better over the years when taking advanced chemistry courses (Szteinberg, 2012).

In this paper we present students' reflections on their CASPiE course experience right after and two and three years later. We compare these to students who took the traditional chemistry course. Having a longitudinal tracking of the students a few years after their participation in the course allows us to understand the potential long-term impact of the laboratory research format on students' subsequent academic and career choices. Our study also incorporates comparison to control groups with random assignment of participants. The longitudinal and control group features of this study have often been missing from other published studies of undergraduate research experiences.

Methodology

The information presented in this paper is part of a larger longitudinal study that was carried out to assess the impact of CASPiE on student performance in chemistry courses and retention in STEM majors and careers (Szteinberg, 2012). The main research question of the longitudinal study is: What are the long-term effects of the CASPiE program on students?

To help answer the major question, we were also interested in knowing how students felt about their CASPiE experience compared to students who were in the traditional laboratory. Therefore, there is a sub-question that will be answered with the qualitative data obtained: What do CASPiE and traditional students report to be positive and negative aspects of their laboratory course in relation to their educational and professional experiences in STEM?

The theoretical framework used for this portion of the study is situated cognition (Brown et al., 1989), which establishes that knowledge is developed within specific environments with characteristic cultures, norms, and even languages. Close engagement with other people in these environments provides cognitive apprenticeships, through which students are actively engaged in and responsible for completing tasks. The result is meaningful learning. In addition, cognitive apprenticeships allow learners to become experts in a subject field. The CASPiE curriculum allows students to learn through the mechanism of situated cognition.

The participants used for the study reported here were students from a large, land-grant, Midwestern University, who took the second-semester general chemistry course for STEM majors (such as engineering, nursing, technology, pre-medicine, and other natural and physical sciences). For the purposes of this paper and to protect anonymity of the participants, we will call this course CHEM1b. The participants took the course in the spring semester of 2007 (2006–2007 academic year). In that semester, all the students had the same lecture course but the population that comprised two lecture sections was randomly split into CASPiE and traditional laboratory sections. Each lecture section had 13–14 total sections of laboratory associated with it, for laboratory sections of approximately 20–24 students each. In total, there were 318 students in the lecture with traditional laboratories and 333 students in the lecture with CASPiE laboratories.

Students in the traditional laboratory course carried out experiments that were fully described in their laboratory manual. Most of the work assigned consisted of pre-laboratory questions that the students had to complete and turn in at the beginning of each session, in-lab work to be carried out by students in groups of two or four and where students took notes on what they did during the period and submitted carbon copies to the teaching assistant to prove their attendance, and a post-laboratory written report submitted on behalf of all group members that was due a week after the laboratory. Most of the activities required students to follow laboratory instructions, giving them no or minimal responsibility for deciding the topic, hypothesis, procedure or materials for each lab period.

The students in the CASPiE sections had five weeks of traditional laboratory work that was carried out as described above and used the same laboratory manual as the students in the traditional section. Following that, students had seven weeks of CASPiE laboratory work. The activities came from a module written for the CASPiE program: Phytochemical Antioxidants with Potential Health Benefits in Foods (Burgess, 2011). This module was written before the CASPiE implementation in 2007 and has been edited every year, therefore the publication year is more current at its website location. Students had three weeks of skill-building activities, where they learned about the laboratory techniques to quantify antioxidant content in food. After the introductory weeks, students had a week to design their research experiment after a literature search to ensure the authenticity of their novel experiments. Students were advised to pick a research question that involved investigating antioxidant quantification of a type of spice after receiving a treatment such as heating/cooking, freezing, or simulation of human digestion. The students were allowed to consult their teaching assistants or peer leaders about their experimental design throughout the research weeks. Peer leaders were incorporated into the program using the Peer-Led Team Learning (PLTL) model (Gafney and Varma-Nelson, 2007; Gosser et al., 2001) to provide assistance to students and to simulate the experience of a research group community in which younger members learn from their more experienced peers. CASPiE students were also required to submit pre-laboratory assignments where they presented their plans to work in the lab, along with recognition of hazards and waste disposal procedures. The in-lab work that students submitted consisted of notes taken during the experimental process, including raw data. This work was graded by the teaching assistants. After every laboratory, students submitted a post-laboratory written report that presented analysis of the data collected in the previous session, discussion of the meaning of the results, and plans for the following laboratory period. At the end of the CASPiE laboratory weeks the students presented their work in poster or paper format, as they would when presenting at a scientific conference or writing a journal article.

The two lecture sections were the same – they were both taught by the same two professors who shared the responsibility of teaching during the semester. The first instructor was a female, Associate Professor who taught during the first half of the semester, before the CASPiE laboratories began. The next professor was a male, at the rank of Professor, who started teaching when CASPiE laboratory activities initiated.

Students in both the CASPiE and traditional sections were asked to fill out a survey before and after the CASPiE weeks that asked them about their experiences in the laboratory. These CASPiE evaluation surveys were developed by the Evaluation and Assessment Center at Miami University of Ohio specifically for evaluation of the impacts of the CASPiE program. These self-report surveys probed students' views of various aspects of their laboratory instruction. The survey has been found to be reliable and valid on measuring the factors described (Scantlebury et al., 2011). With this survey, we were able to track CASPiE and traditional student changes in the dimensions of interest in chemistry and science, real life science, authentic lab practices, perceptions of learning through the lab, belief in chemistry knowledge, and collaborative learning in courses. Table 1 shows the total number of students who responded to the pre- and post-surveys.

Table 1 Total number of students who took the pre- and post-surveys during the spring 2007 semester
N Pre/Post CASPiE Traditional
Total Pre 307 294
  Post 299 253
Male Pre 187 174
  Post 185 145
Female Pre 120 120
  Post 114 108


The surveys were administered using a web-based survey tool, which allowed us to download and analyze the data rapidly. The surveys were given in the second week of the semester, and in the last week of the semester. The pre-survey asked students to respond about their most recent laboratory course which would have been the first semester chemistry course for most of the students who participated in this study. That course will be called CHEM1a.

As part of the longitudinal study, CASPiE and traditional students were sent another survey two and three years after their CHEM1b participation in 2007. In this survey students were asked about their research experiences and career choices after participation in the course, and they were invited to participate in an oral interview. For the survey participation the students were offered a compensation of entering their name into a drawing for three Amazon.com gift cards. The students who agreed to participate in the oral interviews were contacted individually to schedule interview appointments. The interview results will be described here but the longitudinal survey results appear elsewhere (Szteinberg, 2012). In total, 23 students who took the course in spring 2007 were interviewed, during the springs of 2009 and 2010. Table 2 shows the total number of CASPiE and traditional students who were interviewed.

Table 2 Total number of students who were interviewed
N CASPiE Traditional Total
Total 12 11 23
Male 6 5 11
Female 6 6 12


The interviews lasted between 20 and 40 min, and students were prompted to talk about their CHEM1b experiences and how those affected, if in any way, their academic and professional choices in the future. Students were interviewed by one of the researchers who was not present at the institution at the time of the course implementation. The students were compensated with $25 for their participation in the interviews.

The interviewer and assistant researchers transcribed the interviews verbatim. After transcription, the interviews were analyzed using the constant comparative method of grounded theory as described in Strauss and Corbin (1998). The answers to each interview question were combined and analyzed together, through qualitative content analysis (Mayring, 2000; Starosta, 1988) and open coding. After three researchers coded all the spring 2009 interviews, a revised combined coding scheme was developed. Coding of the interviews from spring 2010 helped to further refine the scheme by axial coding. All the interviews then underwent a final round of coding, by one of the authors with another research assistant, with the finalized coding scheme and saturation was reached (Miles and Huberman, 1994). Interrater reliability was carried out and a kappa coefficient of 0.84 was obtained, which is a high level of agreement (Ary et al., 2002).

The codes with the largest number of participant references were further analyzed to compare differences between CASPiE and traditional students. One of the authors counted how many students from each group had talked about the different codes and assertions were derived based on this comparison of the groups. The methodological framework used to analyze the interviews was phenomenography (Marton, 1981; Orgill, 2007; Patton, 2002). This framework allowed us to map out all the different lived experiences of the students in each laboratory format, without bias from outside parties such as the authors or participants other than the students of the study.

Results

The results presented below correspond to the pre- and post-surveys during the spring 2007 semester, while students were enrolled in the course, as well as the final longitudinal interview analysis. The results show that students' views on the course and the laboratory were sustained over the years.

The surveys were composed of statements to which students selected a level of agreement or disagreement on a scale. We used a 6-point Likert scale, with 1 = strongly disagree, 2 = disagree, 3 = barely disagree, 4 = barely agree, 5 = agree and 6 = strongly agree. For the CASPiE and traditional sections of CHEM1b, we were able to compare the pre and post student scores for each of the statements. For the pre and post spring 2007 surveys, independent samples t-tests were used to examine whether differences in the average response to each statement were statistically significant or not, between CASPiE and traditional students. Paired-samples t-tests were used when comparing pre–post differences within a section. Fig. 1 shows the statements for which there was a statistically significant difference in the post score between the CASPiE and traditional sections and for which the CASPiE section values were higher. Also indicated in this figure is any pre/post change that was statistically significant within a group (CASPiE or traditional) and the section that had this difference is represented by the letter in parentheses. The numbers at the beginning of each statement (1–4) represent the theme from the longitudinal interviews to which these survey statements are aligned (to be described in greater detail below): understanding and remembering laboratory work in CHEM1b, understanding the research process and gaining experience in authentic research, self-efficacy and sense of accomplishment, and applicability and relevance of laboratory work.


Survey items where CASPiE (C) students' post-score had statistically significant higher agreement than the traditional (T) students' post score. T or C inside parentheses means the pre- and post-scores were significantly different within the students' group (also shown with asterisks). Statements are preceded by the number corresponding to the longitudinal interview themes.
Fig. 1 Survey items where CASPiE (C) students' post-score had statistically significant higher agreement than the traditional (T) students' post score. T or C inside parentheses means the pre- and post-scores were significantly different within the students' group (also shown with asterisks). Statements are preceded by the number corresponding to the longitudinal interview themes.

Many of the items for which there was a pre/post increase in the CASPiE group showed a decrease in the traditional group, even though the two groups had approximately equal pre-survey scores on many of the items. In general, these items show a greater level of understanding and appreciation of relevance reported by the students in CASPiE sections than those in the traditional sections. Those statements tied to self-efficacy (“I believe I could accurately explain…”, “…motivated me to do well…”, “…made me realize I could do science research…”) also show significant gains for the CASPiE students, though they are decreasing for the students in traditional sections.

Fig. 2 also shows the statements for which there was a statistically significant difference in the post score between the traditional and CASPiE sections but, in this case, the values for the traditional section are higher in the post survey. The T and C letters in parentheses after each statement and the numbers preceding them have the same meaning as in Fig. 1 (i.e. statistically significant difference pre-to-post within a section and relationship to interview themes).


Survey items where traditional (T) students' post-score had statistically significant higher agreement than the CASPiE (C) students' post score. T or C inside parentheses means the pre- and post-scores were significantly different within the students' group. Statements are preceded by the number corresponding to the longitudinal interview themes, which are 5: Following directions or using creativity; 6: Course organization; and 7: Lab and lecture connection.
Fig. 2 Survey items where traditional (T) students' post-score had statistically significant higher agreement than the CASPiE (C) students' post score. T or C inside parentheses means the pre- and post-scores were significantly different within the students' group. Statements are preceded by the number corresponding to the longitudinal interview themes, which are 5: Following directions or using creativity; 6: Course organization; and 7: Lab and lecture connection.

These are items that show decreases for both groups from approximately equal starting values but the decreases tend to be larger for the CASPiE group. The first two items are consistent with the open-ended nature of the research-based projects in which students were in a position to develop their own experimental procedure, rather than following a rote set of instructions. The third statement may also be related to this and will also be discussed in greater detail below with respect to the training of teaching assistants for a first-time implementation of a research-based course. The last item reflects the inherent lack of a step-by-step link between the CASPiE experiments and the lecture topics. Because the CASPiE laboratory engages students in a single project over an extended period of time, the typical weekly alignment between the lab topic and the lecture topic is not present. As will be shown below, however, this does not create a detriment for the students in terms of performance.

The interview data that were collected two and three years after the course was taken result in seven main themes emerging, which align with the themes represented by the above survey data. It is significant that there is persistence over time in students' views of their laboratory experiences. Each of the themes will be discussed in detail below with respect to student quotes and connection to the survey data.

1. Understanding and remembering laboratory work

Fig. 1 shows the survey items preceded by a 1, in which CASPiE students in the spring 2007 semester believed they understood the work they were carrying out in the laboratory, and that they could explain it to other students and instructors. Two and three years later, when students were asked about what they thought was the most rewarding experience from their CHEM1b course and how the course compared to others, 42% of CASPiE students mentioned remembering and were able to explain the work they did when they were in the course, whereas only 18% of the traditional students explicitly remembered their lab work. The traditional students who remembered what they did were only able to describe techniques that they learned in the lab but not the ideas behind the experiments. Fig. 3 shows the comparison of CASPiE and traditional students' tendency to remember their laboratory work, as elicited by the longitudinal interviews. These numbers do not total to 100% for a given group because students were not explicitly prompted to talk about these experiments during the interviews but, rather, overall experiences that were rewarding.
Percent comparison of CASPiE and traditional students interviewed, who explicitly mentioned remembering or not remembering their laboratory work in CHEM1b.
Fig. 3 Percent comparison of CASPiE and traditional students interviewed, who explicitly mentioned remembering or not remembering their laboratory work in CHEM1b.

The quotes below show examples of the emerging ‘remembering’ theme when students were asked to compare their experiences in all the laboratory courses taken during college, and to describe the most rewarding experience learned from their CHEM1b lab. A CASPiE student describes remembering what was done in CASPiE because it was a very different experience compared to other courses:

“I remember doing CASPiE, I remember my experiment pretty clearly, I remember most of my results, it's just kind of … it's I guess one of those experiences that's different enough for me to remember for a while”

Two CASPiE students used examples of their own work when talking about their experiences in the CHEM1b lab:

“Well we expected when it cooked or when it froze the blueberries, that they would lose their [antioxidant] content, but it actually retained it, and they had more, which we didn't know why ‘cause it should have, we would think, the same amount or less instead of more, but so then we ran it…did it again and we got different numbers so it might just be our extraction techniques or something”

“I don't remember exactly what we had but… we wanna figure out what types of antioxidants within different foods would have a different effect on different people or whatever it was… it was nice to come into lab every week with a plan with a good group and to be able to sit down and figure it all out then at the end evaluate how well we did and if we were successful in what we wanted to do so that's what I enjoyed a lot”

These examples show that, after two or three years, the students still remembered the main ideas behind their laboratory activities in CASPiE. The second quote presented above also shows that this student gained a sense of accomplishment from doing the CASPiE laboratory work and this emergent topic will be discussed in more detail in a later section titled “Self-efficacy and sense of accomplishment”.

In contrast, when a traditional student was asked about how the CHEM1b course compared to other courses taken during college, the student mentioned remembering the techniques learned, but not the purpose of the laboratory:

“I remember what I did, I remember what happened, and I remember … like the experiment, but I don't remember the concept behind it, because sometimes I feel like either it's a little too simple for a concept that I don't catch it because it's almost, you know, just obvious, or … it just like, I don't remember it really being like … that connected to like the coursework being done. So that's why I didn't see a connection and then a lot of it just kind of forgot, like what it was about, and all we did which is doing experiment (sic), and get the lab report, and get the points, and leave with, like, that attitude, but I actually did try to wanna, like, learn it, so I remember some but not a lot”

The interview results show that CASPiE students remembered what they did in their CHEM1b lab over the years, and were still able to explain the main ideas behind their work. This is an indication of knowledge retention. We can infer that CASPiE students tended to have more memory of their laboratory activities because they assumed a greater sense of responsibility for them. This was a goal that we had hoped to achieve by having students design their own experimental procedures after lab period three based on the skills and techniques that they learned in lab periods one through three. In this sense, the CASPiE lab manual intentionally does not provide a set of procedures for those last laboratory periods, requiring the students to develop their own.

2. Understanding the research process and gaining experience in authentic research

CASPiE students gave higher post-course survey scores than students in the traditional labs to statements about having an understanding of the process of doing research (Fig. 1). CASPiE students also indicated at a higher level feeling that the activities in the lab related to research done by professional scientists (the number 2 in Fig. 1 precedes these statements). During the interviews two and three years later, this difference was still apparent. Only CASPiE students talked about feeling that they had learned how to do research, and that the experience introduced them earlier to research. In addition, a small percentage of CASPiE students talked about feeling confident to do research. None of the traditional students who were interviewed talked about having learned research during their CHEM1b laboratory. In fact, as can be seen in Fig. 1, traditional students' perception of having experienced research in CHEM1b decreased. Fig. 4 illustrates the percent of CASPiE students from the interviews who talked about learning the research process in their CHEM1b lab experience.
Percent of CASPiE students interviewed who mentioned learning the process of scientific research in their CHEM1b laboratory. No students from the traditional sections made any comments related to this topic.
Fig. 4 Percent of CASPiE students interviewed who mentioned learning the process of scientific research in their CHEM1b laboratory. No students from the traditional sections made any comments related to this topic.

A CASPiE student mentioned having experienced authentic research in the lab:

“I think it helped me grasp the idea, the meaning of a laboratory … of an authentic laboratory experience. In that … you come up with a question that you would like answered, and you do your best to answer it, with good … evidence. And that's what we did in CASPiE. We haven't really done that in other labs.”

Among the examples of students discussing that they learned about doing research in the lab is the following quote:

“Other than my main takeaway from this would be the CASPiE process, for me personally, and the way that I learned was extremely beneficial to… not only myself as a chemist but while I was in … it was interesting to do it that way, because I had never done research like that before and I think a lot of what I do now I kind of apply that same mentality, where you establish a goal… establish a set of how you're gonna … a set of procedures to accomplish that goal, that you can document your way throughout, and arrive at a conclusion, whether it be right or wrong, but it's well documented and you can learn from it”

In addition, another CASPiE student said that the most rewarding experience from CHEM1b was having learned research early on in college and contributing to authentic research:

“I guess it was just starting from the beginning, and starting from scratch, and getting the whole thing done. I guess, I hadn't even thought about it as a freshman and doing research or anything, or I thought about it but wasn't taking that step yet, and it was nice to do the lab and then at the end have that paper I guess where I did the research for another professor, so it was a good feeling to get it done, and get it turned in, and then know that I actually contributed”

This quote also shows that the student felt a sense of accomplishment from doing the laboratory work, and how important it is to provide access to research for students who may not otherwise consider it. The next section expands on the topic of sense of accomplishment.

3. Self-efficacy and sense of accomplishment

Along with discussing the confidence to do research, students talked about having or not having feelings of being like an actual scientist doing research. They also described whether they felt capable of doing such work, which is indicative of self-efficacy (Bandura, 1977), along with a sense of accomplishment. Fig. 1 shows survey statements preceded with a 3 where CASPiE students had a significantly higher agreement about feeling motivated and capable of doing authentic research. The second and third items in Fig. 1 also relate to students' feelings of self-efficacy with respect to discussing their own work. In the interviews, CASPiE students mentioned more often than traditional students having feelings of increased self-efficacy in doing their work in the laboratory. Traditional students mostly explicitly mentioned having no feelings of self-efficacy in doing research and laboratory work in general, although a few indicated having a sense of accomplishment from finishing their work and obtaining results. Fig. 5 illustrates these differences.
Comparison of CASPiE and traditional students interviewed who explicitly mentioned feelings of self-efficacy and sense of accomplishment from doing laboratory work in CHEM1b.
Fig. 5 Comparison of CASPiE and traditional students interviewed who explicitly mentioned feelings of self-efficacy and sense of accomplishment from doing laboratory work in CHEM1b.

A CASPiE student reflected on the experience in CHEM1b and remembered feeling like a chemist. Even though the student did not feel like the work would have a large impact, he still felt capable of employing what was learned about doing research:

“I did enjoy the research portion and, I guess, feeling like a chemist – someone in a lab who comes into work and does research that is potentially beneficial to mankind … which I guess on our level doing it in a classroom it's not going to be used for drug research or anything that's really going to help people but it was cool, I guess, to act like that and to kind of feel that if you could really enjoy, what you could do on a much larger and more thorough scale afterwards”

The fact that the student felt he had done work similar to what a professional chemist does is indicative of his confidence to carry out research and that, if presented with the opportunity to do it again, he feels capable of doing it because he has the experience and has enjoyed previously.

Two CASPiE students expanded on feeling ‘smart’ and capable of carrying out a project and writing about it because of the responsibility that needed to be taken to do the research work in CASPiE:

“We had to cite sources for our abstract and it was neat to see other research in the same thing we were looking at. We had to find it, other research papers, and cite them and use so many sources. So that was cool to see that kinda other people…this wasn't just a silly little research project. Other people were doing, like,…PhDs were doing the same thing that we're doing, so it kinda made me realize that I'm not in high school anymore. I'm taking, like, university classes, so I'm actually pretty smart to be able to figure out how to do it, so it was kinda like took it to the next level I guess”

“I didn't think I could like take on a research project, to be able to write a research paper and, at the time, I thought it was, like, so hard and I just was so frustrated ‘cause I didn’t think I was doing a good job and going, looking back on it was actually kind of, I mean, I think it was good, really good experience, it wasn't the best paper … that I've written but I think it really taught me to just understand that I can do things, like it's, ‘cause writing a paper is, is, when you're a freshman it seems so daunting you're just like ‘oh my goodness we have to write this long paper and cite it and read other scientific works' and also … in my other classes we were required to read scientific works and do an abstract and summarize them and stuff and, as part of our class, but I already had experience doing that with CASPiE as a freshman, because we had to cite other papers in our paper, in our final CASPiE paper report, so I already had experience reading through them and trying to sift through the information and so that was nice ‘cause I already felt like I was ahead in that area… confidence in myself, kind of, was something good that came out of it because, when I looked it back I was like ‘oh, I did this and not everyone was in it and not everyone could do it’”

A traditional student expressed that doing research required the use of creativity, especially when something does not work out and it is necessary to find an alternative pathway to carry out experiments. However, the student explained that he did not feel ready to do research work and being creative with it because he did not have enough experience:

“I’d say just because … I think that being creative can … uncover something great, like those mistakes … it's like, I don't think I would ever try to do anything like that, ‘cause I'd probably think ‘well, that's stupid’ (laughs) why I'd do something like that? But when in all reality it could be … like a mastermind idea, so I'd like to work towards maybe gaining the confidence of … being more creative”

This same student also expressed that he would have liked to be assigned to CASPiE and that he did not obtain a summer research position because he did not have previous experience, which CASPiE could have provided:

“I thought it [CASPiE] was pretty cool, and I thought I would have liked to have done it. I'm not sure how successful I would have been ‘cause I think they did … they took blueberries and saw what antioxidant potential they had and then they froze them and saw what potential they had for antioxidant, I think that's all that they did … I like the idea of something building over time but then in regular CHEM1b labs it was one lesson in three hours and then the next is a whole new different lesson, you know, maybe some of the techniques carried over but, like, the principles behind and the chemistry learned was completely different. I think it [CASPiE] is a good idea, I mean, like, something for me, like, I could at least put that in my resume or, you know, ‘cause like I think the main reason why I didn't get the research job this summer was because I had no experience, ‘cause that's, that's one of the questions, you know, what experience do you have in research”

Including this student, 46% of traditional students expressed during their interviews that they would have liked to have the opportunity take CASPiE to get research experience and its related benefits.

A theme that emerged throughout all of the interviews was about experiencing a sense of accomplishment. A CASPiE student commented that she remembered the work she did in her laboratory because students had a goal to work toward and they learned useful skills from the experience:

“I kinda liked the CASPiE lab because it [each lab period] built on each other. Like, at the beginning I think, I remember we learned techniques and just kinda did a normal lab. And then when we started our CASPiE project I felt like I was working toward something, as opposed to just doing a lab and being done. Like, it was nice because I felt like I was actually working towards a goal as opposed to just getting the points and being in the class”

Two other CASPiE students expressed the same feeling:

“There was an end result, which was really nice to have an end result, I like doing an experiment with a results that makes it seem like (laughs) there's a reason to be doing it and not just keep me busy, so I definitely felt it was the strongest lab that I had taken so far”

“The most rewarding was probably just that feeling of accomplishment because in a normal lab you're pretty much doing experiments that have been done over and over, they know what they want the outcome to be.. you know what they want the outcome to be.. you can almost match up your results to make sure that you're fitting to what they want you to learn for that lab session… whereas the CASPiE experience is a little more… is a lot more… you know, you were unsure of the outcome you didn't know what it was gonna be, you had to write a proposal on … or a summary of what happened and what you thought this meant and you were graded on that so that was I mean it was a different experience I guess I've never really done, that was the first time I'd done a lab experience where I didn't know what was supposed to happen and when it happened it was new to everyone so… it was a unique experience so I liked it … you're just done with it and you can see the results, and you get to, kind of, I mean there's no right or wrong answer, it's, you know, what did you learn and you kind of go with it and explain further about what you did, and what you think that means and how that can be applied to other aspects of science or whatever”

Two traditional students talked about having a sense of accomplishment because they enjoyed going through the process of obtaining results and working with a group of students:

“I think most of this stuff from the chemistry labs … is still useful to me … the fact that you had this experiment that you had to run, you had this project to complete in a certain amount of time, and you knew that you got it done, but you'd have to work with other people to do it … just like, the whole process of starting it, getting through it, everybody doing something and getting it done, and dividing up the work evenly and checking each other's work, is sort of more useful to me now than actually any of the actual chemistry”

In general, students in the CASPiE sections perceived their experiences as authentic research ones, and because of this exposure some of the students felt capable of putting in practice the research skills that they learned. In comparison, traditional students seemed to feel that, because they did not have the experience of being creative in the laboratory, they did not feel confident to do open-ended laboratory work unless they gained more practice in doing such activities. Traditional students also expressed that they could have benefitted from CASPiE if only they had been assigned to take it. Students in the CASPiE sections felt a sense of accomplishment from working on the CASPiE research because they felt they were working towards a goal besides the completion of it for a grade. Traditional students felt accomplished from going through the process of the experiment and getting it done.

4. Applicability and relevance of laboratory work

In the post-survey taken in spring 2007, CASPiE students gave significantly higher scores than the traditional students to items about the similarity of their laboratory work to authentic research being done by scientists and the relevance to their own lives (statements preceded by a 4 in Fig. 1). During the interviews, approximately the same proportion of CASPiE and traditional students stated that the lab work they did in CHEM1b had no apparent applicability or relevance to their lives. However, in the interviews only CASPiE students explicitly stated feeling that their laboratory work was applicable to their lives and other science work. Students also commented on the clarity of the purpose of their laboratory work. In this case, more traditional students explicitly mentioned feeling that there was no clear purpose to doing the lab work they carried out in CHEM1b, whereas only a few CASPiE students had the same feeling. In contrast, only CASPiE students explicitly mentioned feeling that the lab purpose was clear. Fig. 6 shows these comparisons.
CASPiE and traditional students' perceptions of the applicability/relevance and clarity of the purpose of their laboratory work in CHEM1b, based on explicit respondent comments during the longitudinal interviews.
Fig. 6 CASPiE and traditional students' perceptions of the applicability/relevance and clarity of the purpose of their laboratory work in CHEM1b, based on explicit respondent comments during the longitudinal interviews.

A CASPiE student mentioned understanding the application of an instrument that was used to analyze samples, and remembering that information later:

“That's right, it was HPLC, we used that to, like, find the concentrations, so I was like, it's not just a machine that … you use just for lab. [It] is, like, you could see what you could use it for in the real world so, I guess I really like that it was a contextual style, instead of just coming in, and then this is what, this is HPLC, this is what you do, you're done, then you leave. So it kinda, like, a lasting impact on what I thought about the machine and the techniques that we used”

Another CASPiE student talked about the relevance of the subject of the laboratory work to her life as well as how the techniques she learned are applicable to related research and can be used to inform people about the results:

“I mean I liked it … I thought the antioxidant thing was very applicable to our lives. There is such a big push for people to eat more antioxidants now, so doing the experiment was cool to be able to, like, go back to my family and, you know, tell them what I learned … I mean that, like, CASPiE is more applicable to my life … than my other, some of my other chem classes … definitely CASPiE was nice to put it in context, like, when you do an HPLC you can use this in practical terms of finding ascorbic acid in different treatments in fruit … and you can tell people “well, if you're afraid of vegetables or fruit going bad, freezing them or cooking them beforehand doesn't really change much nutritional value” so I really liked it because it had a meaning and it had a purpose, instead of just ‘learn this, and do this lab, and get your results, write a lab report, you don't really have a reason to do it but just do it' (laughs)”

In contrast, two traditional students mentioned not understanding the real-world application or purpose of their laboratory work and why it was important to do it:

“I feel like some of the labs weren't really, like, there wasn't a background information, like ‘oh why do we do this’. Like, there was (sic) a couple labs where, like, you made aspirin and, like, luminol and nylon, but, like, for other ones where you're just, like, basically, like, titrating, there wasn't really a said goal. They didn't tell you like what that actually is used for. I just would have liked some more information, like ‘oh, why are we doing this?’ Kind of thing…like, what goal are you going for? Like, what would you actually use this for in the future?”

“Feel like as if you went in and did it, and then left and nothing happened, and there was, like, no point to it”

The previous four sections described how students' responses to various survey items and the longitudinal interviews indicate that: the CASPiE module enabled students to remember the work they did in the laboratory, understand the process of scientific research, and perceive the relevance of chemistry to the real world. The following sections describe the topics of the survey items, accompanied by interview quotes, which are shown in Fig. 2.

5. Following directions or using creativity

As shown in Fig. 2, it can be seen that traditional students had a higher post-score than CASPiE students with regards to feelings of being successful when following explicit directions in the lab (statements preceded by a 5). When students were interviewed, more traditional students than CASPiE students commented on having to follow directions all the time. In contrast, CASPiE students more often mentioned feeling that they used creativity in their CHEM1b work. Fig. 7 shows this distinction.
Comparison of CASPiE and traditional students' perceptions of following directions and using creativity in the lab.
Fig. 7 Comparison of CASPiE and traditional students' perceptions of following directions and using creativity in the lab.

A CASPiE student describes the process of planning the research work:

“We were told that we had to like pick something and we had to find its antioxidant concentration, or something like that, I mean, so they gave us the general guidelines but we got to pretty much pick it from then on, and then we had, I think there were like four different procedures or testing that we had done before we started the CASPiE, and then we were allowed to pick from those what (sic) ones we wanted to do and run on our sample”

A student in the traditional class mentions the limited use of creativity in the CHEM1b lab work:

“The CHEM1b labs were pretty straightforward … the only times that I really had to use creativity were … for like post-lab questions, trying to figure out, “Oh, so this is what they mean, so we're going to answer it this way” and so my answer might not have necessarily been correct, but because I was creative in the way I approached it, it was still technically correct, so it might not have been what they were looking for, but … it was … sometimes, typically, right”

One outcome of engaging students in research is that they take on greater intellectual responsibility for their work. The fact that CASPiE students realized that they would not be successful in their CHEM1b laboratory by just following directions and that they indeed had to use creativity to complete their work is evidence that CASPiE provides opportunities for students to accomplish authentic research experiences.

6. Course organization

Traditional students thought their CHEM1b laboratory was organized, whereas CASPiE students perceived their lab activities to not be as organized as they could have been (as shown in survey statements preceded by a 6 in Fig. 2). At the time of the interviews, CASPiE students retained this perception, as shown in Fig. 8.
Comparison of CASPiE and traditional students' perceptions of the organization of their lab section.
Fig. 8 Comparison of CASPiE and traditional students' perceptions of the organization of their lab section.

A CASPiE student, who thought the organization was not very good, comments on what could have been done to ensure the course was better organized:

“Maybe the only thing I can think of is, kind of is given, (sic) an overview … maybe tell people at the beginning “Here are some things you're going to do,” because I don't really remember getting a whole overview … like, you're going to do your experiment, you're going to get your results back, and then you're going to try again and rerun it a couple times, and then you're going to want to …start writing your paper, and just kinda get things going”

In contrast, a traditional student mentions that the organization of the course was typical:

“I thought it was a very fair lab for the amount of material and what you learned. It was useful and I didn't hate it. It was fine. It was a typical laboratory experience after I've taken four years of labs now, so, typical lab experience.”

The organization of the CASPiE sections was affected by the fact that this was the first full-scale implementation of the curriculum at the institution, expanded from the previous year's pilot experience with a single section of self-selected students. There were logistical issues with the implementation of CASPiE that still needed to be addressed, and these issues caused frustration both for students and teaching assistants. Because this was the first time that the CASPiE approach had been used on a large scale at this particular institution, it is not surprising that there were organizational difficulties. Later implementations addressed the training and support for teaching assistants, including helping them learn how to guide the students and raising their awareness about the effect that their own attitudes have on the students in their lab sections.

7. Lab and lecture connection

In the survey during the spring 2007 semester, traditional students thought that the laboratory and lecture content were more in sync with each other in comparison to the reports by CASPiE students (as seen on survey statement preceded by a 7 in Fig. 2). When interviewed, 18% of traditional students talked about the lack of this connection, whereas only 8% of the CASPiE students did.

The CASPiE students that mentioned the lack of lab and lecture connection thought it was just not related to general chemistry content in general:

“As far as chemistry related, not so much. Yeah, not so much.”

While this might be expected from students engaged in the CASPiE sections because of the design of the multi-week research projects, there were also some comments of this nature from students in the traditional sections. For example:

“I never really felt I had a whole understanding of what was going on in the labs. A lot of times what you were learning in lecture didn't exactly coincide with when you learned it in lab, and so there was a lot of confusions as to what I was doing. And so I can follow lab procedures, like I followed the procedure right, my results would be fine, and I'd get all the points in the lab report but at the same time there was never the connection made between, like, what I was supposed to be learning and like what I was applying.”

There was a concern that because the CASPiE labs were not explicitly linked to the lecture, that the performance of these students in the lecture portion of the course would be negatively affected. We looked at the exam scores of students in the two sections to see if there was any impact. The exams were written primarily based on the material that is covered in lecture. Table 3 shows the exam and total course score for each section. At all times, the scores were not statistically significantly different when comparing CASPiE and traditional students.

Table 3 Average total scores for exams during the semester. There are no statistically significant differences between students in the CASPiE and traditional sections. The CASPiE research began after Exam II
Section Measure Total Exam I Exam II Exam III Exam IV
Trad. Avg. 879.39 121.21 103.79 98.16 190.00
  St. Dev. 178.37 22.80 28.36 27.61 51.84
CASPiE Avg. 860.65 118.32 99.54 94.96 182.54
  St. Dev. 201.04 27.08 29.57 27.12 54.86


Because the CASPiE laboratory experiments do not follow the lecture material in lock step, as the traditional labs attempt to do, we did not expect to see any increase in the exam scores as a result of participation in CASPiE. However, based on the analysis of exam scores shown in Table 3, the CASPiE experience also does not appear to have had a negative impact on their performance. As part of the larger longitudinal study of CASPiE, students' academic performance in chemistry was tracked and it was not different when comparing between groups over the years. However, for students who took courses up to the 300 level, the traditional students had a significant decrease in performance (Szteinberg, 2012). This further shows that implementing a research-based laboratory could be beneficial to train students early in research, while they learn general chemistry material from their lecture.

Discussion and implications

The results of the attitude surveys and interviews indicate that, at some level, the CASPiE curriculum was able to involve students in real research, providing them with experience in scientific process skills, such as designing experiments and using experimental evidence to draw conclusions. CASPiE also seemed to show students connections between the chemistry they are learning in school and chemistry that takes place in the real world. Overall it seemed to have a positive impact on students' attitudes about chemistry and the learning of chemistry.

The effects on CASPiE students were that they felt confident about being able to explain the work they did in the laboratory, they understood the application and relevance of that work in the scientific field and their lives, they felt motivated to learn because of their research, and they developed a greater understanding of the process of doing research. In contrast, traditional students perceived that their lab course provided explicit instructions to follow in the lab and therefore did not have to use any creativity to be successful in the course. These students also thought the lab experiments were well organized, and they thought the connection between the lab and lecture was more apparent than what the CASPiE students thought.

The effects on the CASPiE students show that CASPiE was successful at delivering authentic research experiences for the participants and, further, that this connection to research may have been a possible motivator for the students to seek more research opportunities. In addition, students indicated feeling a sense of accomplishment during their work in the CASPiE research. Motivation to learn and a positive attitude towards a course likely lead to higher achievement (Glynn and Koballa, 2007, Koballa and Glynn, 2007). The results on students' academic performance can be found elsewhere (Szteinberg, 2012) and are also part of another manuscript currently in preparation.

The effects that were perceived more by the traditional students can be explained. It is no surprise that the students thought they had to follow specific instructions, because the lab activities are set up that way. Students' perception of course organization also comes as no surprise, because the CASPiE implementation was rather new at the time, whereas the traditional implementation had been running for many decades. Finally, the connection between lab and lecture is a topic that has been researched before. Even though having concurrent enrollment in lab and lecture may enhance achievement (Matz et al., 2012), it is not explicitly necessary for them to be connected, as shown by the exam scores in Table 3 where CASPiE and traditional students' performance in the course was not significantly different.

The results described above demonstrate that students had a positive experience as part of the course-embedded research that CASPiE provided. The results are significant not only in showing the student-reported benefits of the program but also in that students' views of the positive aspects of the CASPiE program persist over time. While the positive and negative views of the traditional course also persist over time, students in that class have forgotten quite a bit of detail about what they did and why they were doing it. This paper shows the importance of having longitudinal studies to understand the long-term impacts of research experiences on participating students, which is often missing in the existing literature (Sadler et al., 2010). Research experiences have proven to be beneficial for students to have more hands-on experiences with authentic science practices, and may increase student motivation to learn, as well as increase retention in STEM fields. This study shows that the perceived benefits of doing course-embedded undergraduate research persist over the years and therefore may be worthwhile to include this structure in educational settings.

Limitations and future work

This study was carried out at a single institution. More studies can be done to look at other types of higher education settings, as well as other education levels. Currently, the CASPiE curriculum has been implemented at over 15 institutions over the United States, an institution in Australia, and at least three high schools. Additional studies can also examine the effects on students of different gender or different ethnic/racial groups.

Acknowledgements

The authors would like to thank the National Science Foundation for support for this research (CHE-0418902) as well as the Discovery Learning and Research Center. We thank individuals who helped with the analysis of the data: Kathy Quardokus, Jordan Bradford, Emily Dinoto, and Heather Houlton. We would also like to extend our thanks to those individuals that made the implementation of the CASPiE curriculum possible.

Notes and references

  1. Abd-El-Khalick F., Bell R. L. and Lederman N. G., (1998), The nature of science and instructional practice: Making the unnatural natural. Sci. Educ., 82(4), 417–436.
  2. American Association for the Advancement of Science, (1990), Science for all Americans. Retrieved 10/05/2009, 2009, from http://www.project2061.org/publications/sfaa/online/sfaatoc.htm.
  3. American Association for the Advancement of Science, (1993), Benchmarks for science literacy: A Project 2061 report. New York: Oxford University Press.
  4. Ary D., Jacobs L. C. and Razavieh A., (2002), Introduction to Research in Education (6th edn). Belmont, California: Wadsworth/Thomson Learning.
  5. Baker N. and Verran, J., (2004), The future of microbiology laboratory classes – wet, dry or in combination?. Nat. Rev. Microbiol., 2(4), 338–342.
  6. Bandura A., (1977), Self-efficacy: toward a unifying theory of behavioral change. Psychol. Rev., 84(2), 191–215.
  7. Brown J. S., Collins A., and Duguid P., (1989), Situated Cognition and the Culture of Learning. Educational Researcher, 18(1), 32–42. American Educational Research Association. Retrieved from http://www.jstor.org/stable/1176008.
  8. Bruck L., Bretz S. and Towns M. H., (2008), Characterizing the Level of Inquiry in the Undergraduate Laboratory. J. Coll. Sci. Teach., 38(1), 52–58.
  9. Burgess J., (2011), Phytochemical Antioxidants with Potential Health Benefits in Foods. Retrieved from http://stemedhub.org/resources/173.
  10. Freedman M. P., (1997), Relationship among laboratory instruction, attitude toward science, and achievement in science knowledge. J. Res. Sci. Teach., 34(4), 343–357.
  11. Gafney L., and Varma-Nelson P., (2007), Evaluating peer-led team learning: A study of long-term effects on former workshop leaders. J. Chem. Educ., 84, 535–539.
  12. Glynn S. M., and Koballa T. R. J. (2007), Motivation to learn in college science. In J. Mintzes and W. H. Leonard (ed.), Handbook of college science teaching. Arlington, VA: National Science Teachers Association Press.
  13. Gosser D., Cracolice M., Kampmeier J., Roth V., Strozak V., and Varma-Nelson P., (2001), Peer-Led Team Learning: A Guidebook. Upper Saddle River, NJ.
  14. Hofstein A., and Lunetta V. N., (1982), The role of the laboratory in science teaching: Neglected aspects of research. Rev. Educ. Res., 52(2), 201–217.
  15. Hofstein A., and Lunetta V. N., (2004), The laboratory in science education: Foundations for the twenty-first century. Sci. Educ., 88(1), 28–54.
  16. Hofstein A., and Mamlok-Naaman R., (2007), The laboratory in science education: the state of the art. Chem. Educ. Res. Prac., 8(2), 105–107.
  17. Hunter A. B., Laursen S. L. and Seymour E., (2007), Becoming a scientist: The role of undergraduate research in students' cognitive, personal, and professional development. Sci. Educ., 91(1), 36–74.
  18. Koballa T. R. J., and Glynn S. M., (2007), Attitudinal and motivational constructs in science learning. In S. K. Abell and N. G. Lederman (ed.), Handbook of Research on Science Education. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc., Publishers.
  19. Lederman N. G., (1992), Students' and teachers' conceptions of the nature of science: A review of the research. J. Res. Sci. Teach., 29(4), 331–359.
  20. Lopatto D., (2004), Survey of Undergraduate Research Experiences (SURE): First findings. Cell Biol. Educ., 3(4), 270–277.
  21. Marton F., (1981), Phenomenography – Describing conceptions of the world around us. Instr. Sci., 10, 177–200.
  22. Matz R. L., Rothman E. D., Krajcik J. S., and Banaszak Holl M. M., (2012), Concurrent enrollment in lecture and laboratory enhances student performance and retention. J. Res. Sci. Teach., 49(5), 659–682.
  23. Mayring P., (2000), Qualitative Content Analysis. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 1(2).
  24. Miles M. B. and Huberman A. M., (1994), Qualitative data analysis: An expanded source book. Thousand Oaks, CA: Sage Publications.
  25. Nagda B. A., Gregerman S. R., Jonides J., von Hippel W. and Lerner J. S., (1998), Undergraduate student–faculty research partnerships affect student retention. Rev. High. Educ., 22(1), 55–72.
  26. National Research Council, (1996), National Science Education Standards. Available from http://www.nap.edu/catalog.php?record_id=4962.
  27. National Research Council, (2007), Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future. Washington, DC.
  28. Orgill M., (2007), Phenomenography. In G. M. Bodner and M. Orgill (ed.), Theoretical frameworks for research in chemistry/science education. Upper Saddle River, NJ: Pearson Prentice Hall.
  29. Partnership for 21st Century Skills, (2009), 21st Century Skills MapScience. Tucson, AZ. Retrieved from http://www.21centuryskills.org.
  30. Patton M. Q., (2002), Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.
  31. Roth W.-M., (1994), Experimenting in a constructivist high school physics laboratory. J. Res. Sci. Teach., 31(2), 197–223..
  32. Russell C. B., and Weaver G. C., (2011), A comparative study of traditional, inquiry-based, and research-based laboratory curricula: impacts on understanding of the nature of science. Chem. Educ. Res. Prac., 12(1), 57–67.
  33. Russell S. H., Hancock M. P. and McCullough J., (2007), THE PIPELINE: Benefits of Undergraduate Research Experiences. Science, 316, 548–549.
  34. Sadler T. D., Burgin S., McKinney L., and Ponjuan L., (2010), Learning science through research apprenticeships: A critical review of the literature. J. Res. Sci. Teach., 47(3), 235–256.
  35. Scantlebury K., Li Y., and Woodruff S. B., (2011), Evaluation of CASPiE: Final report. Oxford, OH: Miami University.
  36. Seymour E., Hunter A. B., Laursen S. L. and DeAntoni T., (2004), Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Sci. Educ., 88(4), 493–534.
  37. Slovacek S., Whittinghill J., Flenoury L. and Wiseman D., (2012), Promoting minority success in the sciences: The minority opportunities in research programs at CSULA. J. Res. Sci. Teach., 49(2), 199–217.
  38. Starosta W. J., (1988), A National Holiday for Dr King?: Qualitative Content Analysis of Arguments Carried in the Washington Post and New York Times. J. Black Stud., 18(3), 358–378. Retrieved from http://www.jstor.org/stable/2784512.
  39. Strauss A., and Corbin J., (1998), Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.
  40. Szteinberg G. A., (2012), Long-term effects of course-embedded undergraduate research: The CASPiE longitudinal study. Doctoral thesis, Purdue University, West Lafayette, IN.
  41. Taraban R. and Blanton R. L., (2008), Creating effective undergraduate research programs in science: The transformation from student to scientist. New York, NY: Teachers College Press.
  42. Tobin K. G., (1990), Research on science laboratory activities. In pursuit of better questions and answers to improve learning. Sch. Sci. Math., 90, 403–418.
  43. Weaver G. C., Russell C. B. and Wink D. J., (2008), Inquiry-based and research-based laboratory pedagogies in undergraduate science. Nat. Chem. Biol., 4(10), 577–580.
  44. Weaver G. C., Wink D., Varma-Nelson P., Lytle F., Morris R., Fornes W., et al. (2006), Developing a new model to provide first and second-year undergraduates with chemistry research experience: Early findings of the Center for Authentic Science Practice in Education (CASPiE). Chemical Educator, 11, 125–129.

This journal is © The Royal Society of Chemistry 2013