Improving critical thinking via authenticity: the CASPiE research experience in a military academy chemistry course

Abstract

Course-based undergraduate research experiences (CUREs) can introduce many students to authentic research activities in a cost-effective manner. Past studies have shown that students who participated in CUREs report greater interest in chemistry, better data collection and analysis skills, and enhanced scientific reasoning compared to traditional laboratory activities. Though self-reports are informative, performance measures are needed to evaluate CURE effectiveness objectively. The present study examines whether a CURE implementation at the United States Military Academy (by the Center for Authentic Science Practice in Education [CASPiE]) affects students' self-reported perceptions or critical thinking test scores. Students reported significant increases in their perceptions of learning through the laboratory, authentic scientific laboratory practices and interest in chemistry when compared to previous chemistry courses with traditional laboratory activities. Results also showed a significant increase in critical thinking scores, moderated by student perception of the authenticity of the laboratory activities.

---

Introduction

Critical thinking and chemistry

Chemistry is an experimental science that encompasses both theoretical and practical training within its instruction. As practical skills are taught primarily in laboratory courses, most college-level chemistry courses include a significant laboratory component (Abraham et al., 1997). Unlike traditional laboratory activities that ask students to verify known chemistry relations, researchers have proposed having students conduct authentic research through course-based undergraduate research experiences (CUREs). These experiences afford students the opportunity to participate in a real research project throughout the semester. Students plan and execute experiments, collect data, and report results as a part of the laboratory course component.

Vital outcomes of chemistry courses include teaching students skills that are relevant for the chemistry field, including critical thinking – the focal point of many recent chemical education studies (Carmel and Yezierski, 2013; Uzuntiryaki-Kondakci and Capa-Aydin, 2013; Bruehl et al., 2015) and a target goal in academic and national educational standards (Olson and Loucks-Horsley, 2000; Osborne, 2014). Critical thinking is defined as creative thinking, problem solving, data interpretation/analysis, and communication (Stein et al., 2007).

In this study, we test whether course-based undergraduate research experiences (CUREs) within a chemistry context increase students' critical thinking skills for undergraduate students in a military academy. We further investigate possible reasons for those increases. We employ measures that probe students' ability to think critically before and after a CURE implementation at the United States Military Academy (USMA). Their performance on these measures were then added to self-report survey questionnaires to examine causal interpretability of changes.

As a result of this recent emphasis on improving critical thinking, curriculum developers have started altering goals of the laboratory component of many science curricula. These interventions aiming to improve critical thinking have focused on students' collaborative efforts and activities beyond traditional lectures. Gupta et al. (2015) scored laboratory reports to show that the Science Writing Heuristic improved students' critical thinking skills. Recent studies have also shown that learning interventions (e.g., active learning, Kim et al., 2013; Peer-Led Team Learning, Quitadamo et al., 2009) can improve critical thinking. When students face authentic situations that require collaboration to achieve project goals, they often interact with course materials critically. This is the foundation of this study. Students were given authentic research tasks to complete in teams that incorporated hypothesis development, data collection and presentation of findings. These goals presented a unique opportunity to examine whether authentic practices in the science laboratory affect students' critical thinking.

Course-based undergraduate research experiences

As a supplement to course lectures, laboratory activities often confirm known hypotheses by repeating standard experiments. These are verification laboratory activities and contrast with authentic research activities in which the outcome is unknown. As students follow step-by-step instructions during verification laboratory activities, these activities do not prepare students well for future research endeavors or jobs (Szteinberg and Weaver, 2013).

In many universities, research opportunities are restricted to a few students within their final semesters of undergraduate education, as reported by Canaria et al. (2012):

“It is not feasible to require undergraduate research as part of the degree program at a large university. The high student-to-faculty ratio makes it impossible to support all of these students in a research project.”

CUREs, however, enable many students to participate in authentic, research activities (Auchincloss et al., 2014). These experiences vary in nature but often include semester-long, authentic research experiences that sometimes give students the ability to be co-authors on research manuscripts (Gasper et al., 2012). Students participating in these experiences form groups and learn research techniques. They advance the research project by collecting data and analyzing them to complete a final project or presentation. These types of curricula offer authentic research experiences to students in entry-level classes. CUREs are less costly, larger-scale means of providing students with research experiences, compared to the traditional undergraduate research internship (which often requires many more research faculty to supervise the same number of students).

With suitable preparation, Wolkow et al. (2014) found that CUREs can be implemented successfully at both two-year and four-year institutions to yield positive student experiences. For example, United States Air Force Academy cadets in a CURE produced data for researchers and co-authored publications (Snellman et al., 2006).

CUREs have increased students' self-reported interest in chemistry, improved their data collection and analysis skills, and enhanced their scientific reasoning. These represent important components of critical thinking within science courses (DebBurman, 2002; Wood and Gentile, 2003; as reported by Weaver et al., 2008; Gasper and Gardner, 2013). Though successes involving CUREs have been reported, a meta-analysis of 60 studies of undergraduate research experiences by Linn et al. (2015) showed that most studies relied on students' self-reports. Linn and colleagues highlighted the need for “systematic, iterative studies with multiple indicators of success” (echoed by Brownell and Kloser, 2015). In their framework for CURE evaluation, Brownell and Kloser (2015) advocate measurement of affective outcomes “in concert with measurements of their competencies in scientific practices.” This study addresses this issue by providing both self-report analyses and objective critical thinking data to support the impact of CUREs on critical thinking.

CASPiE

The Center for Authentic Science Practice in Education (CASPiE) was developed with the goal of streamlining the procedure of giving students authentic scientific research experiences at universities. CASPiE is “a multi-institutional collaborative project that aims at providing course-embedded authentic research experiences for undergraduate students during their early years in college, specifically during their general and organic chemistry courses” (Szteinberg and Weaver, 2013). After a series of skill-building modules to teach skills relevant to the research, students form research groups in the classroom, approach data with different hypotheses, and present their results at the end of the semester.

Early CASPiE studies showed several benefits for students. For example, this teaching model challenges students to design their own experiments (Weaver et al., 2006). Also, the CASPiE experience can increase students' connections between science and everyday life and affect their future career choices (Weaver et al., 2008). Furthermore, studies of a CASPiE module on antioxidant capacities in foods increased the sophistication of students' views on the nature of science (Hoch et al., 2009; Russell and Weaver, 2011). Specifically, students gained a better understanding of the meaning of experiments and scientific theories.

Another study showed that student interest in science and understanding of research methods both increased after participating in CASPiE (Scantlebury et al., 2011). This study also showed that CASPiE particularly engaged female students and students from minority groups. In a three-year study, Szteinberg and Weaver (2013) showed that students who participated in CASPiE had a greater sense of accomplishment and greater perceived responsibility than before. They also remembered the main ideas of the laboratory work after one year. Furthermore, Pilarz (2013) showed that CASPiE helped high school students develop a stronger, scientific research community, in which they communicated and worked with one another and with their teachers. Lastly, Gasper and Gardner (2013) showed that a version of CASPiE with a biological focus in an undergraduate course helped increase students' critical thinking.

Study overview

This study examines the relationship between participation in the CURE chemistry course-based research experience and critical thinking. Additionally, the present study examines the impacts of the CURE on perceptions of (a) learning through the laboratory, (b) authenticity of scientific laboratory practices, and (c) interest in the chemistry/science of the course, when compared to previous chemistry courses with traditional laboratory activities.

Research questions:

(1) What is the impact of the CURE participation on critical thinking?

(2) What is the impact of CURE participation on perceptions of (a) learning through the laboratory, (b) authenticity of scientific lab practices, and (c) interest in the chemistry/science of the course when compared to a traditional chemistry course?

Hypothesis 1: participation in the CURE increases critical thinking.

Hypothesis 2: compared to their previous course, CURE students increase their:

• Perceptions of learning through the laboratory

• Perceptions of authenticity in scientific lab practices

• Interest in chemistry/science

Setting

This study was performed at the United States Military Academy (USMA) at West Point, NY. USMA students, known as cadets, are committed to five years of active duty service as Army officers upon graduation. Cadets at West Point currently are required to take (or validate) a full year of General Chemistry during their freshman year. Despite a typical admittance of roughly 1200 cadets per year, all classes at West Point have a class size of 19 cadets per instructor for a core course such as General Chemistry, and even fewer cadets normally attend upper level courses. Their “Thayer Method” teaching style resembles the Socratic method; lecture format is not permitted (Shell, 2002). The faculty at West Point is approximately one-third civilian and two-thirds military. Civilian faculty are PhD holders comparable to faculty at any major undergraduate institution, whereas the majority of military faculty serve at West Point for three years after earning a Master's degree, and in some cases officers serve a second three-year tour after earning a PhD. When not in graduate school or teaching at West Point, these officers are assigned to traditional military positions (infantry, transportation, combat engineers, etc.). Hence, these officers are familiar with the needs of the US modern fighting force, on which they can capitalize to teach cadets in a scientific field.

The participants in this study were 86 cadets enrolled in six sections of Advanced General Chemistry courses, all of which include laboratory work. These cadets scored high on a Chemistry exam administered during their first week at the Academy. Cadets either meet for an 80 minute class or attend a 2 hour lab every other day. The Advanced General Chemistry course sequence covers nearly the same material as the regular General Chemistry course sequence, but the material is compressed into fewer lessons to allow for trips or more extensive laboratory experiences. During the 2014 academic year, cadets in the Advanced General Chemistry course (CHEM 152) participated in the CURE during the second semester, which replaced several traditional laboratory activities and resulted in some compression of other course objectives into fewer lessons. No content was removed from the course.

Method

Participants

86 undergraduate cadets from USMA participated in this study and enrolled in the advanced general chemistry course (CHEM 151/152). They were 83% male and 17% female between the ages of 17 and 22 years (mean = 18.7).

Design

The experiment contains a within-subjects design testing pre/post differences. The within-subjects comparisons come from pre/post differences only associated with those students in the CASPiE program. These variables include the affective responses from a self-report survey. Scores on the critical thinking test were compared across the treatment semester for changes. Regression models then examined whether their perceptions of the CASPiE program accounted for the differences in pre- and post-test scores. Participants in the study were afforded the opportunity to opt-out of data collection. Protocols were approved through the Human Subjects Research Protection Program at USMA. Though the authors of this manuscript include the course instructors, de-identified data were collected and analysed by external collaborators.

Materials

To link cadet work with important on-going research, we asked these cadets to conduct research on a relevant military problem that may directly affect them upon graduation from West Point. The Waste-to-Energy research project selected for this CASPiE experience sought to tackle simultaneously two major challenges faced by the fighting force in the modern Army: (1) disposal of waste at Contingency Bases (CBs) or Forward Operating Bases (FOBs) and (2) provision of energy to generators used to power these locations. On a daily basis, a typical Army base in an undeveloped area can consume a staggering amount of fuel and water as well as generate over 1000 lbs of solid waste. This diverts significant manpower from mission operations to manage the delivery and security of these resources. Fuel costs are high while deployed, and delivering fuel to the end user requires overcoming several logistical and security challenges. In addition to fuel requirements, the predominance of open burning of solid wastes as an expedient disposal method has increased soldiers' health risks. Specifically, these soldiers might inhale smoke and particulates from incomplete combustion of a wide variety of wastes. While the Department of Defense has tried to limit the hazardous constituents being burned and has fielded small incinerators at some locations, the problem remains.

One proposal to alleviate these important problems is a gasification-based waste-to-energy technology, suitable for use in a CB. Gasification of solid wastes, to produce a syngas which can be burned in a diesel generator, eliminates the complications of trying to produce a liquid fuel from waste. However, the handling and processing of mixed wastes has proven difficult in currently available downdraft gasifiers. Researchers from the US Army Construction Engineering Research Laboratory, The Center for Environmental Science and Technology (CEST) at SUNY Cobleskill, and West Point, are working on a DoD-sponsored grant that will analyze the operation of a prototype unit and later design a deployable, rotary kiln gasification system. While gasifier syngas products could be transformed into liquid fuels, the overall intent of this research project is to design a rotary kiln, waste-to-energy, gasification system that will directly burn the syngas for fuel and:

• be easily deployable

• accept all types of solid wastes typically generated at a Contingency Base with minimal pre-processing or separation

• provide a syngas that will fuel a standard Army diesel generator at a net-positive energy balance, displacing a large fraction of fuel use.

A rotary kiln design was selected to allow for gasification of a wide variety of materials to simultaneously alleviate the challenges posed by waste and provide fuel on CBs (Cosper, 2014). A rotary kiln gasifier operates on the most straightforward possible mode of gasification: direct flaming pyrolysis. Solid and potentially liquid wastes are thermally converted to a flammable gas at temperatures above 800 °C to prevent char residue; inorganic ash is the only byproduct. The overall goal of the research project is to better understand how changes in the waste stream affect syngas characterization. Because the overall gasification project occurs at multiple locations, the cadets were assigned to focus on narrower aspects of the overall project. Cadets focused on the gas cleaning process, including selection of the scrubbing agent (liquid solvent or solid phase material).

The evaluative procedures for the CASPiE program consisted of a survey developed by the researchers that assessed student attitudes towards learning chemistry in the laboratory classes (Wink and Weaver, 2008). Students are asked to respond on a 5-point scale of agreement to a series of statements about their beliefs regarding chemistry and the course experience. The original study by Wink and Weaver (2008) measured self-reported gains when compared to the previous chemistry class that the student had taken. The survey was administered at the beginning and end of the treatment semester via an online survey distribution tool. Students received 10 points of extra credit for completing the survey each time and were given the option to complete an alternative activity for extra credit in lieu of participating in the evaluation.

The Critical-thinking Assessment Test (CAT) developed by Tennessee Technological University was used to show critical thinking changes across the entire academic year (Stein et al., 2007). The CURE implementation occurred only in the second semester. The test takes approximately 45 minutes (though students were given up to 60 minutes if needed) and contains 15 open-ended questions that put students in various scenarios to probe their critical thinking. The CAT was administered at the beginning of the spring semester (January 2014) and at the end of the second semester (May 2014). Tests were scored anonymously by a panel of instructors with an agreed upon rubric for each question. A person directly trained by the test developers led the scoring of the tests. Scores were then sent back to the developers to confirm that the scoring was valid and reliable. Scores were then tabulated and sent to the authors for further data analyses. Student participation in the CAT was completely voluntary, and they received no compensation. 77 of the total 86 students participated in all implementations of the self-report and CAT surveys.

Procedure

At the outset of the course (six weeks before the CASPiE modules), cadets were introduced to the research project and three possible research topics. Each instructor gave a concise presentation corresponding to his or her research area. A further description of each component of the course is described in the Appendix. These presentations gave cadets a background of the science involved within the research, the overall plan of the project, and some of the techniques involved. Two weeks before the beginning of the modules, cadets were shown the overview slides a second time and asked to select the set of modules that they would like to pursue. Next, the instructor of their selected module gave them introductory reading assignments. The three major components roughly corresponded to the three academic majors offered in the Department of Chemistry and Life Science at West Point: Chemistry (Analytical Chemistry), Life Sciences (Toxicology), and Chemical Engineering. This had the additional benefit of reducing the cadet-to-instructor ratio to an average of six cadets per instructor, rather than the normal 18 : 1 ratio typical at West Point. The smaller cadet-to-instructor ratio facilitated instructor–student interaction. Within each component of the course, cadets were organized into groups of 2–3. Each group worked together to formulate a unique hypothesis and collect data. Finally, each group presented a poster of their work at the end of the semester.

Data analyses

Confirmatory factor analyses (CFA) of the student responses to three sets of three survey questions self-report data yielded the corresponding three factors (see Table 1). A gender variable was added to each CFA specification for sufficient degrees of freedom.

Table 1 Description of items in factors from self-report survey

Factor	Survey item
Interest in Chemistry/Science	The lab experience made me more interested in chemistry.
	The lab experience made me more interested in science.
	The lab experience made me more interested in a science career.
Authentic Scientific Lab Practices	The lab experiences were very similar to real research.
	The lab experiences made me realize I could do science research in a real science laboratory (for instance, at a college or with a pharmaceutical company).
	The lab experiments presented real science to students, similar to what scientists do in real research labs.
Perceptions of Learning through Laboratory	I better understood the ideas of chemistry, in general, as a result of completing the experiments.
	I believe I could accurately explain a chemistry experiment from the course to other student.
	I believe I could accurately explain a chemistry experiment from the course (including the significance of the results) to my instructors.

---

Factor models were assessed for global model-data fit by for goodness-of-fit statistics. Good model fit is indicated by a failure to reject the null hypothesis as measured by fit indices. Good model fit is indicated by a root mean squared error approximation value less than 0.10, a Tucker-Lewis Index greater than 0.95, and a standardized root mean residual less than 0.08 (Hu and Bentler, 2009). STATA v14 Statistical Software was used to test the fit of the factor models in this study (StataCorp, 2015). This analysis was followed by a paired samples t-test of the weighted mean sum scores (DiStefano et al., 2009) from each standardized factor loading.

Linear regression tests whether several explanatory variables are simultaneously correlated with the target outcome of total critical thinking score. Specifically, it tests the hypothesis that each explanatory variable's regression coefficient differs from zero at a 95% confidence level (all else being equal). Pre-post gains for CAT and self-report surveys were analyzed by paired samples t-tests of significance at a 95% confidence level. A correlational matrix of these variables is presented in the Appendix (see Table A1). Effect sizes of these gains were assessed by Cohen's d.

Results

The results of the factor analyses were similar to those in the original CASPiE report (Wink and Weaver, 2008). Table 2 displays the emergent factors as well as their standardized root mean square residuals, root mean squared error approximations, Tucker-Lewis indices and global chi-squared values.

Table 2 Resulting factor loadings and pre-post changes

Weighted mean-sum scores analysis				Confirmatory factor analysis
Variable	Pre-mean	Post-mean	Mean-difference	SRMR	TLI	RMSEA	Chi-squared p-value
* = p < 0.05; ** = p < 0.01; *** = p < 0.001; SRMR is standardized root mean residual, TLI is Tucker–Lewis index, and RMSEA is root mean squared error approximation.
Perceptions of Learning through Laboratory	11.52	12.34	+0.821***	0.008	1.004	0.102	0.069
Interest in Chemistry/Science	11.24	12.40	+1.15***	0.012	1.005	0.000	0.491
Authentic Scientific Lab Practices	9.54	12.93	+3.39***	0.011	1.009	0.000	0.582

---

Differences in factor scores on the pre- and post-surveys showed significant increases in perceived authentic scientific laboratory practices, perceptions of learning through the laboratory, and interest in chemistry/science changes. These results suggest that after CURE participation, students perceived improvements in all three dimensions. A significant difference in perceived authenticity shows that students felt that the CURE was more authentic than previous laboratory courses. Gains in interest in chemistry shows that CURE participants feel as though they can pursue further education or work in chemistry after the academy.

Students also felt that they learned significantly more through the CURE than through earlier laboratory courses. Students scored higher on the CAT post-test after CASPiE participation than on the CAT pre-test (see Table 3, Model 1). This supports the hypothesis that course-based research experiences can improve students' critical thinking abilities. Further regression analyses explore other factors (specifically those related to demographic data and the self-report survey) and their relationship with this change across the treatment semester.

Table 3 Regression models predicting critical thinking score

	Model 1	Model 2	Model 3	Model 4	Model 5
	CAT (May)	+ Authenticity	+ Auth*May	+ Gender	+ Ethnicity
* = p < 0.05, ** = p < 0.01, *** = p < 0.001; all standard errors in parentheses.
CAT (May)	2.59*** (0.73)	1.93* (0.97)	−6.36 (4.02)	−5.90 (4.02)	−5.08 (4.04)
Authenticity		0.11 (0.16)	−0.13 (−0.66)	−0.12 (0.20)	−0.11 (0.20)
Authenticity*May			0.70* (0.33)	0.67* (0.33)	0.66* (0.33)
Gender				−0.24 (1.01)	−0.27 (1.01)
Ethnicity					0.30 (0.57)
Intercept	24.35*** (0.51)	23.36*** (1.58)	25.60*** (1.89)	25.59*** (1.88)	25.17*** (2.05)
Explained variance	0.08	0.07	0.10	0.10	0.11

---

The next series of models investigates whether several explanatory variables might account for the higher CAT score among CURE participants. While authenticity alone was not significant (Table 3, Model 2), its interaction with participation in the CURE was significant (Table 3, Model 3). As the survey asked students about their recent experience (previous semester), this result shows that authenticity significantly predicts CAT score after participating in the CURE program. Students who viewed the program as more authentic significantly gained in critical thinking after participating in the CURE program. Model 4 shows that this result is robust; after adding Gender and Ethnicity variables, the interaction between CURE participation and the May implementation of the CAT remains the only significant predictor.

Parallel regression models were run for both perceived learning and interest in science. Models 2–4 were run for each of the two factors to determine if the change in CAT score could be also attributed to changes in students' perception of learning or their interest in pursuing further education/work within the chemistry/scientific field. Models 2–4 showed no significant predictors of CAT score when incorporating interest in chemistry/science or perceived learning. The only series of models that statistically predicted changes in CAT score was the authenticity-based regression models displayed in Table 3. Results from the regression of interest in chemistry/science models as well as the regression of the perceived learning models can be found in the Appendix.

Discussion

Recent studies of course-based laboratory research aims to improve students' critical thinking and their preparation to become future scientists (Carmel and Yezierski, 2013; Bruehl et al., 2015). Course-based research studies have shown that students in such courses report improved critical thinking, including greater data collection and analysis skills, and enhanced scientific reasoning (Linn et al., 2015). Using pre- and post-tests and surveys, this study implemented a CURE via CASPiE and showed that afterwards, students perceived greater authenticity of scientific laboratory practices, more learning through the laboratory, and higher interest in chemistry/science, than before. After CASPiE, students showed higher critical thinking test scores than before, and this gain was higher for students who viewed the CURE activities as more authentic than the laboratory activities in the previous course. These three findings indicate positive educational outcomes associated with participating in the CASPiE research experience for this sample of military academy cadets. These findings are further discussed below.

The self-report results displayed significant increases after CURE participation. These results suggest that students felt that they learned more, gained more of an interest in chemistry/science, and did more authentic scientific work in the CASPiE experience than in a traditional laboratory class. They are also factors that are related to critical thinking increases in previous CURE implementations (Wink and Weaver, 2008).

Critical thinking gains across the semester were consistent with previous CASPiE research (Gasper and Gardner, 2013). Significant gains in total CAT score occurred for participants during the treatment semester. CURE participation was also related to a significant increase in students' perceived authenticity of the laboratory experience. Students who viewed the CURE activities as more authentic had higher increases in critical thinking (on the post-test compared to the pre-test). This suggests that the more authentic that students viewed the experience, the more impact that it had on their critical thinking. This relationship is fundamental to the CURE experience as the primary focus of any CURE is to give students an authentic perception of research. The results suggest a relationship between CURE participation, its authentic research activities and critical thinking gains for these military academy cadets.

The CASPiE implementation at USMA was successful in terms of both students' perceptions and their critical thinking. Given the value of critical thinking, perceptions of greater authenticity, greater learning and greater interest in chemistry for future endeavors both scientific and otherwise, these results extend the CURE literature in important ways.

A link between authentic practice and critical thinking is an important finding that can inform and improve student learning during chemistry courses. Engaging with the course content in an authentic way can help students think critically about possible outcomes or extensions of the research. Hence, instructors who develop authentic curricular materials might help students improve their critical thinking. Educators and curriculum designers can also use CUREs as a vehicle to deliver a real research experience to students and to systematize undergraduate research experiences. CUREs offer research experience to many undergraduate students at one time (Auchincloss et al., 2014). Incorporating such experiences into the curricula can streamline this process, making it easier and less expensive.

While CURE research has shown connections between authentic science practice and critical thinking (Gasper and Gardner, 2013), this relationship requires further investigation. This study adds to the body of CURE literature by showing that students who perceive laboratory activities as more authentic show greater critical thinking gains.

The next step for this implementation of CASPiE is to address issues reported by faculty instructors. They noted that this CURE course was very time-consuming and labor-intensive. The instructors spent a lot of time trying to set up laboratory times, experiments, and curricular material. They were not able to collaborate efforts much as the course incorporated three projects running concurrently. One possible way to reduce their time and effort is to change the curriculum from three projects to one project.

Limitations and future work

Some limitations of this research study include both design and implementation aspects. Cadets at military academies are engaged in an academic environment that is far different than most undergraduates in traditional universities. The results of this study are only generalizable to an extent. One cannot simply assume that increases in CAT scores for these students are indicative of increases that would occur in many other academic settings. This point is further illustrated by the lack of a true control/treatment experimental design. Though interactions in regression analyses are robust in telling of relationships between variables, a classical control-treatment experiment would demonstrate all outcomes of CUREs when compared to traditional settings.

The CURE program at USMA has continued to be implemented in second semester chemistry courses and is now the main method of educating all cadets in this level of general chemistry. This study further extends the sparse research literature on the effects of CUREs on military academy cadets. CASPiE studies have examined the effects of the program on several student outcomes, such as student views of the nature of science and affective gains (Weaver et al., 2008). This study in conjunction with Gasper and Gardner (2013) adds critical thinking to a target outcome of CURE. Some next steps in the evaluation of this type of instruction include considering other educational outcomes, such as creative thinking or a standardized content exam. Future studies can also measure longer-term critical thinking effects with a delayed post-test. Future research can also examine the applicability of content learned. The course-based research model of instruction gives students a deep understanding of research techniques and content that is specific to this project. This raises the question of what happens to that knowledge and whether students can apply it to other situations outside of the traditional curriculum.

Appendix

Chemical engineering

The largest number of cadets (42) selected the Chemical Engineering Group. This team worked to determine which scrubbing method (e.g. solid phase material or liquid solvent) was most effective at removing unwanted tars from the gas. Removal of tars is essential if the gasifier is to feed fuel to an Army power generation unit without causing undue wear-and-tear on the system. In order to accomplish this goal, cadets were first taught how to use ChemCAD modeling software and asked to read a number of papers related to gas scrubbing systems and tar removal. After using ChemCAD to reproduce the results of a related paper, the cadets were allowed to either continue with modeling or conduct hands-on experiments. The majority of cadets continued with hands-on work with a liquid solvent or solid-phase scrubbing material of their own selection and design (rice husks, propylene glycol, motor fuel, aerogel, etc.). Cadet groups developed a wide variety of different hypotheses. First, they tested the scrubbing material in ChemCAD and then compared their predictions to the actual results obtained through hands-on experimentation. The hands-on testing of scrubber solvents occurred using a gasifier the students designed and built as a team.

Life sciences

Cadets with a strong interest in the Life Sciences (31 cadets) selected the Toxicology Group. There is a valid concern that the syngas and waste tar created by the gasification process could be hazardous to humans and the environment. There is potential for exposure to the fumes of the solvent or the solvent itself during maintenance, storage, or disposal. As one step towards identifying any potentially negative health effects, cadets tested the solvent using the Ames assay (Ames, 1979). This simple assay identifies chemical substances that are mutagenic to a modified strain of Salmonella bacteria; the assay is relatively inexpensive and requires extensive use of both positive and negative controls as well as replicates, so the assay itself serves as an excellent teaching tool for experimental design. Cadets started with a harmless strain of E. coli to learn basic aseptic techniques. Next, cadets performed the Ames assay by exposing Salmonella typhimurium to a control substance or the polishing fluid (propylene glycol) that had been used to clean the gas generated by the trash. After performing two iterations of the Ames assay, cadets had enough preliminary data and experience to design their own experiment using appropriate positive and negative controls. The cadets' hypotheses varied substantially from group to group. Some cadets tested various concentrations of solvent, others tested multiple strains of the bacteria, and others exposed bacteria to either the solvent fumes or the actual gas produced by the system before and after scrubbing.

Table 4 Correlation matrix of key variables

	May	Authentic	AuthMay	Learning	LearnMay	Motivation	MotivMay	Gender	Ethnicity
May	1.00
Authentic	0.6165	1.00
AuthMay	0.9775	0.6998	1.00
Learning	0.2794	0.3672	0.3251	1.00
LearnMay	0.9897	0.6396	0.9809	0.3555	1.00
Motivation	0.2199	0.5476	0.2988	0.4107	0.2585	1.00
MotivMay	0.9480	0.6701	0.9659	0.3387	0.9574	0.4249	1.00
Gender	0.0034	0.1048	0.0105	−0.1741	−0.0117	0.1090	0.0061	1.00
Ethnicity	0.0017	−0.0374	0.0045	0.0160	−0.0022	0.0096	0.0091	0.610	1.00

---

Analytical chemistry

Several cadets (14 cadets) were interested in the Analytical Chemistry Group. These cadets worked in small groups of 2–4 cadets to analyze scrubbing oil for the presence of poly-aromatic hydrocarbons (PAHs) using a GC/MS. GC analysis is generally not one that supports the processing of oil samples. However, the developed method separated the non-polar fraction of the oils and saponified them so that they became aqueous (Mathison and Holstege, 2013). The goal of this project was to compare the GC results with a known standard to identify the different polycyclic aromatic hydrocarbons (PAHs) in the resultant oil. PAHs are known to be dangerous when inhaled, therefore it is important to identify if the gasification process is producing a dangerous environment for soldiers. The module began with an introduction to the GC instrument as well as some examples of what chromatographs look like and how to properly interpret them. After the initial instruction, the skill-building modules included the development of standard curves with known solutions. This allowed cadets to familiarize themselves with the process of identifying specific peaks on a graphical output from these types of experiments. The process of prepping the samples and placing them into the instrument queue also proved to be a vital learning experience for the ensuing project. Cadets were evaluated in a formative manner by their laboratory notebooks as well as their ability to interpret results alongside the instructor.

The skill-building laboratory sessions varied in content and depth across the modules, but all generally contained hands-on laboratory activities that mirrored the results of a research publication with the end goal of developing a standard method for proceeding. The toxicology group began by learning how to safely and aseptically handle bacterial cultures before learning to conduct the Ames assay using Salmonella typhimurium. The chemical engineering group began to create a ChemCAD model of an actual experimental publication to use as a baseline to compare results. The analytical Chemistry group created a standard reagent curve for a mix of poly-aromatic hydrocarbons. Following the skill-building modules, cadets engaged in a research collaboration meeting with other groups from different areas. They then began to develop their hypotheses for their own research project. These hypotheses were submitted and reviewed in an iterative process with the corresponding instructor. Students executed their planned experiments and adjusted as necessary based on their results. After completing their experimental work, a short poster training session provided each group of two to four cadets with sufficient guidance to create their own research poster. Cadets also visited poster sessions of upper class research cadets shortly before their own poster was due. CASPiE instructors reviewed each draft poster at least once and provided feedback on both design and content. Cadets concluded their CASPiE experience at a poster session attended by a wide audience of their peers, upper class cadets, instructors and senior leadership at West Point as well as outside guests, including the lead researchers of the project.

Acknowledgements

The authors would like to acknowledge the Center for Environmental Science and Technology at the State University of New York – Cobleskill for the allowing cadets to participate in the waste-to-energy gasification research project.

Notes and references

1. Abraham M. R., Craolice M. S., Graves A. P., Aldhamash A. H., Kihega J. G., Gal J. G. and Varghese V., (1997), The nature and state of general chemistry laboratory courses offered by colleges and universities in the United States, J. Chem. Educ., 74(5), 591–594.
2. Ames B. N., (1979), Identifying environmental chemicals causing mutations and cancer, Science, 204(4393), 587–593.
3. Auchincloss L., Laursen S., Brandchaw J., Eagan K., Graham M., Hanauer D., Lawrie G., McLinn C., Pelaez N., Rowland S., Towns M., Trautmann N., Varma-Nelson P., Weston T. and Dolan E., (2014), Assessment of course-based undergraduate research experiences: a meeting report, CBE-Life Sci. Educ., 13, 29–40.
4. Brownell S. E. and Kloser M. J., (2015), Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology, Stud. High. Educ., 40(3), 525–544.
5. Bruehl M., Pan D. and Ferrer-Vinent I. J., (2015), Demystifying the chemistry literature: building information literacy in first-year chemistry students through student-centered learning and experiment design, J. Chem. Educ., 92, 52–57.
6. Canaria J., Schoffstall A. M., Weiss D. J., Henry R. M. and Braun-Sand S. B., (2012), A model for introductory undergraduate research experience, J. Chem. Educ., 89, 1371–1377.
7. Carmel J. and Yezierski E., (2013), Research and teaching. Are we keeping the promise? Investigation of students' critical thinking growth, J. Coll. Sci. Teach., 42(5), 71–81.
8. Cosper S., (2014), Rotary Kiln Gasification of Solid Wastes for Base Camps. Retrieved August 1, 2016, from http://https://www.serdp-estcp.org/Program-Areas/Weapons-Systems-and-Platforms/Waste-Reduction-and-Treatment-in-DoD-Operations/WP-2211/WP-2211/(language)/eng-US.
9. DebBurman S. K., (2002), Learning how scientists work: experiential research projects to promote cell biology learning and scientific process skills, Cell Biol. Educ., 1, 154–172.
10. DiStefano C., Zhu M. and Mindrila D., (2009), Understanding and using factor scores: considerations for the applied researcher, Practical Assess., Res. Eval., 14(20), 1–11.
11. Gasper B. and Gardner S., (2013), Engaging students in authentic microbiology research in an introductory biology laboratory course is correlated with gains in student understanding of the nature of authentic research and critical thinking, J. Microbiol. Biol. Educ., 14(1), 25–34.
12. Gasper B. J., McCreight J. C., Banschbach K., Bustion A., Davis C., Divecha R., Donoho M., Elmore A. G., Garrison C. M., Glenn S., Goeman D. C., Haby M., Hooks T., Korman A. M., Kowal J., Kuschke S., Mellencamp J. E., Meyer M., Myers A. N., Nichols M. F., Pfeifer A., Porucznik A., Qu X., Ramos-Miller M., Reed R., Sagintayev A., Singel J. M., Smigh A., Valle M. E., Venderley A., Weber C. A., Zaffino A. J., Csonka L. and Gardner S. M., (2012), Isolation and preliminary characterization of amino acid substitution mutations that increase the activity of the osmoregulated ProP protein of salmonella enterica serovar typhimurium, DNA Cell Biol., 31(6), 956–967.
13. Gupta T., Burke K. A., Mehta A. and Greenbowe T. J., (2015), Impact of guided-inquiry-based instruction with a writing and reflection emphasis on chemistry students' critical thinking abilities, J. Chem. Educ., 92, 32–38.
14. Hoch M., Russell C. B., Steffen D. M., Burgess J. R. and Weaver G. C., (2009), Assessment of antioxidant capacities in foods: a research experience for general chemistry students, J. Chem. Educ., 86(5), 595–597.
15. Hu L. and Bentler P. M., (2009), Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives, Struct. Equ. Modeling, 6(1), 1–55.
16. Kim K., Sharma P., Land S. M. and Furlong K. P., (2013), Effects of active learning on enhancing student critical thinking in an undergraduate general science course, Innov. High. Educ., 38, 223–235.
17. Linn M. C., Palmer E., Baranger A., Gerard E. and Stone E., (2015), Undergraduate research experiences: impacts and opportunities, Science, 347(6222), 1261757.
18. Mathison B. and Holstege D., (2013), A rapid method to determine sterol, erythrodiol, and uvaol concentrations in olive oil, J. Agr. Food Chem., 61(19), 4506–4513.
19. Olson S. and Loucks-Horsley S., (2000), Inquiry and the national science education standards: a guide for teaching and learning, National Academy Press, Washington, DC.
20. Osborne J., (2014), Teaching critical thinking? New directions in science education, Sch. Sci. Rev., 95(352), 53–62.
21. Pilarz M., (2013), Implementation of a research-based lab module in a high school chemistry curriculum: a study of classroom dynamics, Doctoral thesis, West Lafayette, IN: Purdue University.
22. Quitadamo I. J., Brahler C. J. and Crouch G. J., (2009), Peer-led team learning: a prospective method for increasing critical thinking in undergraduate science courses, Sci. Educ., 18(1), 29–39.
23. Russell C. B. and Weaver G. C., (2011), A comparative study of traditional, inquiry-based, and research-based laboratory curricula: impacts on understanding of the nature of science, Chem. Educ. Res. Pract., 12, 57–67.
24. Scantlebury K., Li Y. and Woodruff S. B., (2011), Evaluation of CASPiE: final report, Oxford, OH: Miami University.
25. Shell A., (2002), The Thayer method of instruction at the United States Military Academy: a modest history and a modern personal account, Primus, 12(1), 27–38.
26. Snellman E., Krueger J. and Unangst E., (2006), Moving research into the classroom, J. Coll. Sci. Teach., 35(5), 32–36.
27. StataCorp, (2015), Stata Statistical Software: Release 14, College Station, TX: StataCorp LP.
28. Stein B., Haynes A., Redding M., Ennis T. and Cecil M., (2007), Assessing critical thinking in stem and beyond, in Iskander M. (ed.), Innovations in E-learning, Instruction Technology, Assessment and Engineering Education, New York, NY: Springer-Verlag New York, LLC, 1st edn, pp. 49–82.
29. Szteinberg G. and Weaver G., (2013), Participants' reflections two and three years after an introductory chemistry course-embedded research experience, Chem. Educ. Res. Pract., 14, 23–35.
30. Uzuntiryaki-Kondakci E. and Capa-Aydin Y., (2013), Predicting critical thinking skills of university students through metacognitive self-regulation skills and chemistry self-efficacy, Educ. Sci.: Theor. Pract., 13(1), 666–670.
31. Weaver G. C., Wink D., Varma-Nelson P., Lytle F., Morris R., Fornes W., Russell C. and Boone W. J., (2006), Developing a new model to provide first and second-year undergraduates with chemistry research experience: early findings of the center for authentic science practice in education (CASPiE), Chem. Educator, 11, 125–129.
32. Weaver G. C., Russell C. B. and Wink D. J., (2008), Inquiry-based and research-based laboratory pedagogies in undergraduate science, Nat. Chem. Biol., 4(10), 577–580.
33. Wink D. J. and Weaver G. C., (2008), Evaluation of the center for authentic science practice in education (CASPiE) model of undergraduate research. Workshop linking evidence to promising practices in STEM undergraduate education, Washington, DC: National Research Council, p. 14.
34. Wolkow T. D., Durrenberger L. T., Maynard M. A., Harrall K. K. and Hines L. M., (2014), A comprehensive faculty, staff, and student training program enhances student perceptions of a course-based research experience at a two-year institution, CBE-Life Sci. Educ., 13, 724–737.
35. Wood W. B. and Gentile J. M., (2003), Teaching in a research context, Science, 302(5650), 1510.

---