Cynthia A.
Stanich
*,
Michael A.
Pelch
,
Elli J.
Theobald
and
Scott
Freeman
Department of Chemistry, University of Washington, USA. E-mail: stanich@uw.edu
First published on 23rd May 2018
To help students who traditionally underperform in general chemistry, we created a supplementary instruction (SI) course and called it the STEM-Dawgs Workshops. These workshops are an extension of the Peer-led Team Learning (PLTL) SI. In addition to peer-facilitated problem-solving, we incorporated two components inspired by learning sciences: (1) training in research-based study skills, and (2) evidence-based interventions targeting psychological and emotional support. Here we use an explanatory mixed methods approach to measure the impact of the STEM-Dawgs Workshops, with a focus on four sub-populations that are historically underrepresented in Chemistry: underrepresented minorities, females, low-income students, and first-generation students. Specifically, we compared three groups of students in the same General Chemistry course: students in general chemistry and not the workshops (“Gen Chem students”), students in the workshops (“STEM-Dawgs”), and students who volunteered for the workshops but did not get in (“Volunteers”). We tested hypotheses with regression models and conducted a series of focus group interviews with STEM-Dawgs. Compared to the Gen Chem population, the STEM-Dawg and Volunteer populations were enriched with students in all four under-represented sub-populations. Compared to Volunteers, STEM-Dawgs had increased exam scores, sense of belonging, perception of relevance, self-efficacy, and emotional satisfaction about chemistry. URM STEM-Dawgs had lower failure rates, and exam score achievement gaps that impacted first-generation and female Gen Chem students were eliminated in the STEM-Dawg population. Finally, female STEM-Dawgs had an increased sense of belonging and higher emotional satisfaction about chemistry than women Volunteers. Focus groups suggested that successes came in part from the supportive peer-learning environment and the relationships with peer facilitators. Together, our results indicate that this supplementary instruction model can raise achievement and improve affect for students who are underrepresented in chemistry.
Students from underrepresented populations face an array of challenges due to the circumstances of their birth. Many are underprepared due to educational disadvantage or—in the case of women—stereotypes about their ability in quantitative fields. In addition, URM, low-income, and first-generation students are more likely to experience economic disadvantage, requiring them to work while in school and take on debt to pay for tuition and living expenses (Stephens et al., 2012). First-generation students are also entering a world that is unfamiliar to their family members, creating a cultural mismatch that can place them at risk (Orbe, 2008).
Researchers have developed interventions designed to address both the academic and the emotional and psychological obstacles that underrepresented students face. For example, courses with high structure—meaning that they require pre-class preparation, intensive active learning, and regular exam practice—can reduce achievement gaps for low-income and URM students because they increase deliberate practice and sense of community (Haak et al., 2011; Eddy and Hogan, 2014). Programs that offer comprehensive support in the form of financial aid, access to research opportunities, supplementary instruction (SI) in key courses, and mentoring have also shown dramatic benefits (e.g.Barlow and Villarejo, 2004; Shields et al., 2012; Slovacek et al., 2012; Hall et al., 2014). Finally, less-intensive interventions, often in the form of brief writing exercises or structured group discussions, have been beneficial in reducing achievement gaps caused by stereotype threat (Miyake et al., 2010; Jordt et al., 2017) or for promoting a more positive sense of belonging (Walton and Cohen, 2011).
Research has shown that bad experiences in general chemistry can be especially discouraging for URM and other at-risk students (Barr et al., 2008; De La Franier et al., 2016). As the first step in a comprehensive effort to redesign general chemistry at the University of Washington and improve the performance of underrepresented students, we designed and evaluated an SI workshop as a voluntary, 2-credit companion course for the initial offering in the 3-quarter, year-long general chemistry sequence. SI has a long track record of improving the performance of all students in an array of STEM courses (Arendale, 1994), and in some cases has produced disproportionate gains by underrepresented students (e.g.Fullilove and Treisman, 1990). In chemistry, several SI designs have increased student grades (Rath et al., 2012; Batz et al., 2015), with the Peer-led Team Learning (PLTL) model having a particularly large-scale impact (Gosser, 2011).
In almost every case, the core of successful SI in STEM courses has been cooperative group learning—often focused on working exam-like problems. Well-managed cooperative group learning can improve student performance in chemistry (Warfa, 2016) because it encourages students to be metacognitive about their problem solving—meaning that they become more aware of, and better-monitor, their own learning (Schraw et al., 2006; Sandi-Urena et al., 2012; Snyder and Wiles, 2015). Although some SI models employ graduate teaching assistants or professional tutors, peer-facilitated group learning—the PLTL model—has been particularly effective in chemistry (Becvar et al., 2008; Hockings et al., 2008; Snyder et al., 2016).
In addition to building on the strengths of peer-facilitated group practice, we wanted to design and test an SI model that would leverage recent experimental work on effective study strategies. In the absence of explicit training in study skills, many students respond to poor exam scores by doing more of the approaches that produced the initial, disappointing outcome (Stanton et al., 2015). Increased time on task can be helpful, but there is no guarantee that students are spending the additional study time wisely (Chan and Bauer, 2014; Ye et al., 2016). In contrast, making the benefit and “how-to” of evidence-based study skills explicit can promote changes to study behavior (Cook et al., 2013). Explicit practice of evidence-based study habits, in turn, can improve exam performance (Zhao et al., 2014); effective learning strategies can also improve affect through the positive emotions they engender (Pekrun et al., 2002, 2007).
In addition to PLTL and effective study practices, we also incorporated a third element in the SI model analyzed here: evidence-based interventions that provide psychological and emotional support. The purpose of these exercises was to reduce the impact of stereotype threat, imposter syndrome, test anxiety, and other issues that have a disproportionate impact on underrepresented students. Each of the interventions we chose has been shown to increase some aspect of student performance for disadvantaged students in STEM when used in isolation (citations for each are listed below). Many of these interventions show improvement for underrepresented students in STEM (Cohen et al., 2006; Walton and Cohen, 2007; Miyake et al., 2010; Harackiewicz et al., 2014; Yeager et al., 2014; Paunesku et al., 2015; Yeager et al., 2016; Jordt et al., 2017). The workshop that we designed is the first time that a large suite of psychological and academic interventions has been used in conjunction with the same student population. Additionally, we wanted to brand the course in a positive way, so we called it the “STEM-Dawgs workshop” and developed a logo that was printed on course materials and on a mobile phone carrying case that was given to participants.
We hypothesized that the novel combination of deliberate practice, study skills training, and psychological–emotional interventions, implemented in a weekly workshop that was run by peer facilitators, would provide cognitive and affective benefits to student populations that have historically underperformed in introductory chemistry courses. Our specific research questions were:
(1) What impact does the STEM-Dawgs workshop have on course performance in general chemistry as indexed by exam scores and failure rates? Does it disproportionally benefit students who are traditionally underrepresented in STEM?
(2) What impact does the workshop have on aspects of student affect that may be particularly important for URM, female, low-income, and first-generation students to be successful in general chemistry?
This work was conducted under the University of Washington Human Subjects Division application 52402.
The STEM-Dawgs workshop was a credit/no credit course, with grades based on completion of assignments and points earned on quizzes. Students were expected to earn at least 70% of the points in the course for credit. Nine STEM-Dawgs sections consisted of a maximum of 24 students each, led by two peer facilitators. Each week, workshop participants were assigned pre-class homework questions and a writing exercise. Prior to most workshop sessions and at other specified times, students did an evidence-based writing intervention designed to help them cope with psychological or emotional difficulties they might face in making the transition from high school to college. We chose these interventions because they have been shown to improve student performance, and especially outcomes for underrepresented students, in courses similar to general chemistry, as we explain in detail below. The writing interventions addressed the following barriers:
(1) Stereotype threat: students who are subject to stereotypes about their academic ability may underperform due to the cognitive load of coping with the stereotype, or may take criticism as an inaccurate perception of performance or evidence of not belonging. A values affirmation writing intervention that can mitigate stereotype threat in academic contexts (Cohen et al., 2006; Cohen et al., 2009; Miyake et al., 2010; Jordt et al., 2017) was assigned twice per quarter: at the start and prior to the second midterm. This assignment, called Values Affirmation, prompts students to select the most important traits about themselves from a list and to write a short essay explaining why they selected those particular traits. In addition, peer facilitators communicated that any critical feedback they gave was motivated by high standards, not racial bias (Cohen et al., 1999; Cook et al., 2013).
(2) Mindset: students who view intelligence as a fixed attribute can struggle in response to academic challenges (Dweck and Leggett, 1988; Blackwell et al., 2007). To reduce the impact of this issue, STEM-Dawgs participants did an online assignment that supports better performance by teaching students that the brain is like a muscle and requires exercise to get stronger (Blackwell et al., 2007; Paunesku et al., 2015; Yeager et al., 2016).
(3) Self-regulation: self-regulation of behavior, emotions, and impulse is important for academic success (Duckworth, 2011). Mental Contrasting with Implementation Intentions (MCII) is a fantasy-realization intervention designed to convert feasible goals into concrete actions. Early in the term, STEM-Dawgs participants implemented this approach by visualizing a wish, outcome, obstacle and plan (WOOP). The goal was to increase commitment to and confidence in future achievement (Duckworth et al., 2013).
(4) Test anxiety: anxiety over assessment can cause decreased levels of performance. A short expressive writing exercise done right before a test improves performance when taking a high-stakes exam (Ramirez and Beilock, 2011). This assignment prompts students to express their pre-exam emotions in writing. STEM-Dawgs participants were encouraged to do the assignment online, as close to the start of each exam as possible.
(5) Belonging: feeling accepted in a group has positive effects on health and achievement (Magnuson and Waldfogel, 2008; Meng et al., 2015). To support feelings of belonging in college, students in the workshop did a reading and writing exercise modeled on an intervention about the high school to college transition that has been shown to improve academic outcomes (Walton and Cohen, 2011; Yeager et al., 2016). In this intervention, students read messages from previous students about the challenges inherent in the transition from high school to college. STEM-Dawgs then summarized what they read and told their own transition story. After this writing assignment, STEM-Dawgs were shown a student's video describing her transition story and then discussed it with the peer-mentors.
(6) Value of learning: a self-transcendent reason for learning positively correlates with academic outcomes, and can be supported with an intervention, also assigned to the STEM-Dawgs, that prompts students to write about a social injustice and self-transcendent motivations for pursuing higher education (Yeager et al., 2014).
(7) Expectancy value: we assigned an intervention that can improve exam performance by increasing the perceived value of a course. The task consists of a short essay on how course content can be relevant to one's own life (Hulleman et al., 2010).
(8) Post-exam metacognition: after each exam, STEM-Dawgs completed a writing assignment directing them to examine their study skills. The students answered several open-ended questions regarding study methods and time-on-task, then read a list of suggested study techniques and designed a new study plan for the next exam (Stanton et al., 2015). This online writing assignment was due 10 days after each of the two midterms.
The schedule for these psychological interventions exercises is provided in Appendix 2.
Workshop sessions started with each student individually taking a 10-minute content quiz, working in a group of three to four to discuss the answers, and then potentially being chosen at random to explain their group's answer to the entire class. Next, the peer facilitators guided students in a discussion about the pre-class writing exercise—the psychological or emotional intervention—and the highlighted study skill for that week. After these discussions, students again worked in small groups to answer chemistry questions that required higher-order cognitive skills and that were relevant to the week's lecture material. During exam weeks, students individually worked on a practice exam, followed by small group work to confirm answers.
The full list of study skills and citations introduced in the course is provided in Appendix 2. During the discussion portion of each week's workshop, a study skill was presented through a worksheet that summarized the methods and results sections of a paper from the peer-reviewed literature. Students then participated in group discussions about why the study skill appeared to be effective and how they could apply the result in their own classes. Annotations about the study skills appeared in the student manual for the workshops and next to assigned problems where the skill was relevant, and peer facilitators were encouraged to point out specific skills as they were being practiced during in-class activities.
It is important to realize two things about these populations: (1) all students (Volunteers, STEM-Dawgs, and Gen Chem students) were enrolled in general chemistry, and (2) the STEM-Dawgs and Volunteers represent demographically distinct populations from the Gen Chem Students. As Appendix 1 shows, the STEM-Dawgs and Volunteers populations had a much higher representation of URM, EOP, first-generation, and female students, ranging from a 19–100% increase relative to the Gen Chem population.
Peer facilitator training began with a two-hour, pre-quarter overview session where the STEM-Dawg mission and techniques were made clear. New facilitators learned about (1) the mechanics of the weekly schedule, and (2) techniques that help facilitate positive group interactions during class. After the workshop's inaugural implementation, we included experienced peer facilitators in the training process, as many of the initial facilitators were eager to stay involved in the course. This development allowed us to remove the pre-quarter training session, because new mentors learned this background information through an apprenticeship with an experienced mentor in the classroom. All peer facilitators attended a weekly meeting to discuss the previous week's session and address tasks for the upcoming week, including a “just-in-time” review of relevant chemistry topics. Finally, peer facilitators would discuss the highlighted study skill for the upcoming class as well as the writing assignment, with the goal of developing strategies to facilitate effective class discussions.
The pre-post survey included a series of Likert-scale questions designed to measure four constructs; mindset, belonging, relevance, and self-efficacy. Mindset is measured through three questions such as, “You can learn new things, but you can’t really change your basic intelligence” (Blackwell et al., 2007); belonging through four questions such as, “I feel like I belong in this class” (Walton and Cohen, 2007); relevance via four questions such as “My class gives me useful preparation for what I plan to do in life” (Hulleman et al., 2010); and self-efficacy with four questions such as, “I can earn an A in this class” (Harackiewicz et al., 2014). The instrument is made available online by the Project for Education That Scales (PERTS, 2018). In this pre-post survey assignment, students also completed the 8-item Attitude Toward the Subject of Chemistry Inventory V2 (ASCI-V2; Bauer, 2008; Xu and Lewis, 2011) and the Chemical Concept Inventory (CCI; Mulford and Robinson, 2002). The CCI is a 22-item inventory of chemical knowledge and misconceptions, which we administered on a password-protected learning management system called Canvas. Students were given 30 minutes to attempt the questions, which were shown one at a time. We used this inventory as a control for chemical knowledge preparedness in the models described below. We used the PERTS and ASCI-V2 surveys in combination because they are intended to capture related but distinct constructs—how students feel about learning and the course (PERTS), and how they feel about chemistry (ASCI-V2). We specifically chose short surveys to measure the constructs of interest to minimize potential survey fatigue for our students. Factor analysis (described and shown below) confirms that the questions we chose successfully target the affect we were interested in.
We tested two possible random effects (section and term) by fitting six complex models, each of which included all possible fixed effects but a different random effects structure. Model 1 had term and section as random effects, Model 2 had just term as a random effect, Model 3 had just section as a random effect, Model 4 had section as a random effect and term as a fixed effect, Model 5 had term as a random effect and section as a fixed effect, and Model 6 had no random effects. We compared models using AICc, Akaike's information criterion with a correction for small sample size (Burnham and Anderson, 2002). AICc (and AIC) is a measure of how well a model fits, that relies both on the model maximum likelihood and the number of parameters in the model. Specifically, AICc penalizes model fit as additional parameters are added to the model. In this way, AICc is a relative measure that is particularly useful in comparing nested models (Burnham and Anderson, 2002). Final, best-fitting models had the lowest AICc values; if models differed in AICc by less than 2 units, the model fit was considered identical and the model with the fewest number of parameters was selected as better fitting (Burnham and Anderson, 2002). Starting models are described in association with each hypothesis, and final models are described in the Results.
The second step in our model selection process involved determining the best fitting model by selecting fixed effects. Again, we used backwards selection and assessed model fit by comparing AICc. Using the random effect structure that fit best (from above), we fit complex models that modeled the outcome of interest as a function of the pre-score aligned with the outcome of interest (e.g. a PERTS survey construct) and confounding variables that we predicted may affect the outcome. We used a stepwise approach to remove variables, starting with variables that explained the least amount of variation in the outcome. We removed variables until all the variables remained were significant or until the balance between AICc and the number of parameters was minimized.
To investigate the differential effects for students of interest (URM, female, first-generation, and low-income), we fit separate models that included an interaction between student of interest and Group (Gen Chem, STEM-Dawgs, or Volunteer). If in the full starting model the interaction did not help significantly predict the outcome, the interaction parameter was removed first.
As is typical in model selection procedures such as these, each model can be considered a distinct hypothesis being tested (Burnham and Anderson, 2002). For this reason, the p-values of best-fitting models are redundant and have not been specifically indicated. Instead, all parameters that were retained in the final model are interpreted with a magnitude and direction. All models were fit in R version 3.4.0 (R Core Team, 2017).
The survey data are highly bounded, like most survey data, and have a strong ceiling effect (see Appendix 3). A ceiling effect in survey data commonly occurs when some respondents who gave the highest response (6 in the case of PERTS and 7 in the case of ASCI-V2) would have responded even higher if they had been able to do so. This kind of bounding can be problematic when fitting a linear model because the model may predict values that exceed the highest possible value of the scale. In these cases, a value at or above the threshold (ceiling) will all be forced at the value of the threshold (ceiling). To be sure that this ceiling effect did not qualitatively impact our estimates, we fit censored regression models (also called Tobit models) using the censReg package in R (Henningsen, 2017).
Fitting censored regression models with censReg allows for specification of the upper and lower bound of the data (the ceiling and the floor respectively). In our case, our data did not approach the floor. Censored regression models account for ceiling (and floor) effects by modeling an uncensored latent outcome in place of the censored observed outcome (Henningsen, 2010).
Nonetheless, model selection with models that were fit using censored regression resulted in identical final, best fitting models. Comparing models fit with the censored regression to models fit with glm, the direction of the relationship was identical, but the strength changed (as is common in censored regression). For simplicity, output from the glm is reported here but the more numerically precise censored regression coefficients can be found in Appendix 4.
To provide a greater degree of confidence in the reliability of these factors, we calculated the Cronbach's α for both instruments (see Appendix 5). A value 0.7 or greater is generally considered sufficient internal consistency (Murphy and Davidshofer, 2005). To measure the validity based on internal structure, we measured the comparative fit index (CFI) and the standardized root mean square residuals (SRMR) for each instrument (see Appendix 4). The PERTS results were modeled as a four-factor survey and the factors were allowed to correlate. The ASCI-V2 were modeled as a two-factor survey and the factors were allowed to correlate. The CFI and SRMR were determined using the cfa function in the lavaan package in R (Rosseel, 2012) using the maximum likelihood estimator. Good fit values are CFI > 0.95 and SRMR < 0.08 (Hu and Bentler, 1999).
The PERTS, modeled as a four-factor survey tool, was found to have a good fit as assessed by both CFI and SRMR. The ASCI-V2, modeled as a two-factor survey tool, had a low CFI value, but the SRMR was within good-fit parameters. That result, along with the good Cronbach's α values and previous use of the ASCI-V2 as a two factor instrument (Brandriet et al., 2011; Xu and Lewis, 2011) encouraged us to interpret the ASCI-V2 as a two factor survey tool.
Coding was done by one of the co-authors (MP) and began with thorough readings of the transcripts and identification of segments of text in the Atlas.ti software package. Next, each identified segment of text was evaluated independently and characterized with a note. These notations allowed the researcher (MP) to characterize the explicit or implicit meaning of each quotation, which facilitated placing quotations into codes. This preliminary phase of coding led to several iterative and recursive rounds of combining and dividing codes based on the researcher's (MP) evolving interpretations of the data which allowed the analysis to determine the central components, or themes, of the student focus groups. Finally, the researcher used constant comparison to evaluate each quotation and determine if it was reasonable to assign the statement to multiple codes. While codes represent distinct concepts, statements from students are often complex and layered representations of their impressions of the companion course and general chemistry. To account for this complexity, students’ statements were often assigned multiple codes where appropriate. Consequently, this allowed the researcher to calculate a code co-occurrence matrix using the Atlast.ti software.
Thematic content analysis employing code co-occurrences used in this study is constructivist in its orientation and assumes a level of subjectivity in the statements made by focus group participants. In order to maintain consistency and depth in the analyses, it was important to constantly reflect upon and revise the qualitative analyses as the quantitative analyses and results evolved over the course of the study. We chose to address this by having a single researcher manage the analysis of all the focus group data instead of relying on the agreement of multiple raters. We acknowledge that interrater reliability can enhance the trustworthiness of some qualitative interpretations and is well-suited to other forms of content analysis. However, our design choice allowed a researcher (MP) experienced with the iterative and recursive nature of thematic analyses to ensure that the data were viewed from a consistent perspective throughout the study. This helped support the trustworthiness and consistency of the qualitative analysis and our choice to omit interrater agreement is consistent with other explanatory study designs in chemistry education (refer to Vishnumolakala et al., 2017).
There was a trend for first-generation students to underperform relative to non-first-generation students in General Chemistry. This gap was slightly reduced for STEM-Dawgs and the trend reversed in the Volunteer population—meaning that first-generation Volunteers did exceptionally well. All of these trends were small, however, even though the interaction term was retained in the final model. First-generation Gen Chem students show a statistically significant achievement gap of 0.08 standard deviations lower for first-generation relative to non-first generation students. Opposite of our expectations, first-generation Volunteers scored 0.49 standard deviations higher as Volunteers (see Appendix 6). No achievement gap was found between first-generation and non-first-generation STEM-Dawgs. Women, on the other hand, scored 0.15 standard deviations lower than men in General Chemistry, in contrast to finding no gender achievement gap in STEM-Dawgs or Volunteers (controlling for SAT, CCI, and Term, see Appendix 6).
Fig. 3 When accounting for CCI and chemistry lecture section, STEM-Dawgs (SD) show a higher score on the post-mindset survey, on average, than the Gen Chem Students (GC). |
Furthermore, we found that women Volunteers scored 0.9 points less than STEM-Dawgs and 1.1 points lower than the Gen Chem Students, on average. In contrast to the Gen Chem Students, there were gender gaps in belonging for the other two study populations, though the male–female gap was much less in the STEM-Dawgs population (1.4 points) than in the Volunteer population (1.8 points; Fig. 4B).
• Course Structure, which encompassed statements about the overall design of the course;
• Classroom Activities, which detailed events that took place during workshop sessions;
• Classroom Culture, which included interaction(s) between peer facilitators and students, as well as comments about the social aspects of the course;
• Affect, which comprised statements dealing with students’ attitudes or emotions about science, chemistry, or the workshop;
• Peer Facilitator, which consisted of all statements that directly or indirectly related to the undergraduate course leaders;
• Impact, which contained statements that dealt with the workshop's influence on students’ overall chemistry course experience or academic behavior.
Table 1 summarizes the number of quotations from focus group participants assigned to each of the six codes, and an example quotation from the focus groups that is characteristic of each code. Due to students’ statements often being qualified by multiple codes, the counts shown in Table 1 do not represent discrete bins of quotations but rather the number of times that a particular code was applied to statements from the focus group data.
Code | Number of quotations | Example quotation |
---|---|---|
Course structure | 17 | “It was basically an extra help class. I saw it as a support class to Chemistry 142.” |
Classroom activities | 21 | “The peer facilitators would split us into different chem[sic] groups based on our professor, so we would work together…” |
Culture | 33 | “…if you got it, then you could explain things to other people and if you didn’t get it, then people could explain things to you…” |
Affect | 24 | “Empathy made the class setting more comfortable.” |
Peer facilitators | 67 | “They worked together, I wasn’t sure if they were friends beforehand, but they seemed like friends.” |
Impact | 30 | “It made my learning easier, which made chemistry easier.” |
Peer facilitators were the most frequently discussed topic during focus groups. Statements from students about peer facilitators were diverse and ranged from positive statements like “it was still fresh in their minds [referring to their content knowledge]” to more critical statements such as “mentors [peer facilitators] were a little too friendly and lost authority.” Statements qualified by the Culture code resulted in the second highest numbers of coded statements. Student statements about culture were often reflective of the companion course being a much more friendly, close and group-oriented environment than their chemistry lab or discussion sections. For example, a student stated that “we would all just get the right answer anyway because there were so many people working on it together, we didn't have a problem about finding an answer.” Statements from the Impact code had the third highest occurrence among students’ talk. Students’ perceptions on the impact of the companion varied from explicitly positive statements to statements where they suggested no impact at all. However, some of the comments explicitly stating that it did not help would be associated with segments of talk that alluded to positive impacts of the companion course. For example, a student stated that “I don't think it impacted my grade at all. It impacted how I felt about the course, more positively.” In that statement it is clear the student did not think that the companion course improved their grades but they do seem to state that it improved their experience with chemistry. The Affect and Classroom Activities codes have roughly the same number of occurrences. Many of the statements coded as Affect, such as “I felt more comfortable asking questions” and “You could ask anything, whereas if you're in one of those 800-people classes with a professor, it's more intimidating to ask a question you may think is stupid…”, alluded to the students’ perceiving a greater degree of comfort with the companion course. Statements in the Classroom Activities code captured student talk about what actually happened during the companion course classes. Students would often talk about working in groups as well as working through the psychological interventions and study skills activities. The Course Structure code applied to the smallest number of statements made by students, but still provided insight into students’ perceptions of the companion course. Statements found within this code encompassed students’ frustration with their perception of how much work they felt they had to do in the workshop conflicting with the workshop's use of low-stakes assessments. For example, “I feel like I didn't try as hard on homework assignments because they weren't graded. Basically, if you show up, you get points.”
Code co-occurrences represent the level of association between codes (Table 2). For example, a student made the following comment about their peer facilitators: “they [peer facilitators] worked together. I wasn't sure if they were friends beforehand, but they seemed like friends. They got along really well.” This quotation was qualified by the Peer Facilitators code because the facilitators are the focus of the comment. Additionally, the quotation was also qualified by the culture code due to the student commenting how they perceived that the peer facilitators got along and how they worked together. Because this statement received a Peer Facilitator and Culture code this would mean that those codes are co-occurring. A value of 1 means that two codes always co-occur together while a value of zero means that two codes never co-occur on the same quotation. We interpreted that the higher the code co-occurrence value the greater the conceptual relationship between two codes. The three highest co-occurrences are the Affect, Culture, and Peer Facilitators codes. The highest code co-occurrence was between the Affect and Culture codes while the least co-occurring codes was between the Peer Facilitators code and Classroom Activities. While the peer facilitators in the workshop were in charge of facilitating classroom activities, it is important to note that the focus groups revealed a higher association between Peer Facilitator code and the Culture code, suggesting that peer facilitators had a stronger influence on the classroom culture of the companion course than did the classroom activities. The Impact code also more strongly associated with the Affect and Culture codes (Table 2), suggesting that the impacts of the companion course were less related to direct performance benefits and more associated with emotional aspects of chemistry.
1 | 2 | 3 | 4 | 5 | 6 | |
---|---|---|---|---|---|---|
(1) Course structure | N/A | |||||
(2) Classroom activities | 0.03 | N/A | ||||
(3) Culture | — | 0.15 | N/A | |||
(4) Affect | — | — | 0.16 | N/A | ||
(5) Peer facilitators | — | 0.02 | 0.12 | 0.06 | N/A | |
(6) Impact | — | — | 0.07 | 0.08 | — | N/A |
Exams | DFW | Mindset | Belonging | Relevance | Self-efficacy | Intellectual accessibility | Emotional satisfaction | |
---|---|---|---|---|---|---|---|---|
A/a: STEM-Dawgs > Volunteers; B/b: STEM-Dawgs > GenChem; c: STEM-Dawgs = Volunteers; d: STEM-Dawgs = GenChem; e: STEM-Dawgs < GenChem; f: STEM-Dawgs < Volunteers. | ||||||||
Overall | Ad | Ba | Ab | ab | Ad | Ab | ||
URM | Ad | ce | ab | |||||
EOP | ab | |||||||
FG | ef | ab | ||||||
Women | ae |
Improved exam scores and lower failure rates may be particularly important for underrepresented students who traditionally have been discouraged from pursuing medical school by poor performance in general chemistry (Barr et al., 2008), which is the goal of a large portion of our undergraduate STEM majors. It is notable that unlike Gen Chem Students, there was no achievement gap in exam scores for women and first-generation students in the workshop, and no achievement gap in failure rates for URM individuals who participated.
Data from the focus group interviews supports the hypothesis that increased course performance was due, at least in part, to the deliberate practice that occurred in the workshops (Haak et al., 2011). Many of the student comments coded as Classroom Activities focused on the active learning that occurred in the peer-facilitated sessions, and especially the group work. For example, students made comments such as “…since you had to discuss it in groups you were forced not to lie that you didn't get it…” and “they [peer facilitators] would prompt students to help each other in that way. They weren't necessarily teaching – students were teaching each other and helping each other, which I thought was super useful.” The focus on group work may also have facilitated a transfer of learning strategies between students with variable levels of academic skill and experience. For example, one student summarized the workshop's impact by saying “… (I) definitely think it helped, not because of the class itself, but just the people I met there. I met some really hardcore study geniuses, they dragged me along with them. That pulled me up better than I would've done alone.”
Although the workshops included explicit training in the challenge mindset required to overcome academic obstacles, we observed only a 0.5 point difference in Mindset among the three treatment groups. As Yeager et al., (2016) suggest, the benefits of a challenge mindset may now be so ingrained in the public consciousness that the intervention's efficacy is diluted. The other aspects of affect captured in the PERTS survey questions—belonging and self-efficacy—indicated that STEM-Dawgs had much better attitudes about the general chemistry course than Volunteers, and in the case of belonging, better than Gen Chem Students. Male workshop participants reported a much higher sense of belonging.
Focus group data suggest that these results could be due to the workshop establishing a more-positive classroom culture around the topic of chemistry than the traditional lecture or laboratory. Code co-occurrences suggest that the classroom culture was created by both the classroom activities and peer facilitators (Table 2); an insight summarized in the following quote: “I think the way it was led was more of a really organized study group of friends.” This statement suggests that students perceived the workshop to be a more comfortable learning environment than their laboratory or lecture sections. The workshop's culture was also perceived as a more-accepting environment, where students felt surrounded by individuals like themselves. “I think it made me more comfortable with chemistry, just being in the class and seeing that other people were struggling and knowing that when you come in with questions, you're not the only one that's struggling.” Gaining an “I’m not the only one” perspective can lead to a greater sense of belonging; students who feel like they belong are more likely to persist and perform well in STEM courses (Trujillo and Tanner, 2014; Hanauer et al., 2016).
The more-positive sense of belonging that we observed for female STEM-Dawgs versus Volunteers may be particularly important. Research in other fields suggests that women are much more “grade-sensitive” than men—meaning that they are less confident about being successful and less likely to persist than men who are performing at the same level (Rask and Tiefenthaler, 2008; Ellis et al., 2016). Further research should explore whether the higher sense of belonging declared by women in the workshops helps to mitigate the achievement gaps we observed in exam scores and helps them persist in STEM.
Many of the quotes in the Impact code also refer to the workshop's influence on self-efficacy, which has both emotional and academic benefits (Lewis et al., 2009; Sawtelle et al., 2012; Bjørnebekk et al., 2013; Chan and Bauer, 2014; Larson et al., 2015). For example, students made comments such as and “I'd go to lecture and be like [sic], ‘Oh, I learned that in [the workshop] yesterday.’ I could sit there and be like, ‘Okay,’ and take notes and understand. I learned it previously…That was nice,” and, “I felt more empowered in the class just because I was capable of showing I could do the problem.”
Similar to relevance, the workshop had no impact on the ASCI-V2 intellectual accessibility construct. This scale measures what students believe and know about how to study chemistry and did not change even though STEM-Dawgs had explicit training in research-based study skills. Indeed, gaps in the intellectual accessibility scale occurred for EOP versus non-EOP workshop participants—opposite the direction intended by the workshop's design. Did the initial exposure to research-based study skills negatively impact students who realized that their existing study skills are contraindicated by evidence? Would repeated exposure to study skills training produce a better outcome? With respect to the ASCI-V2 emotional scale, however, it was clear that STEM-Dawgs scored higher than both Gen Chem Students and the Volunteers. The workshop helped students feel better about studying chemistry—even though they may have been unsure of how to go about it.
The only other SI study in General Chemistry with a volunteer-control, by Chan and Bauer (2014), also used peer facilitators for collaborative group work in 50 or 80 minute supplementary meetings to work on material from that week's lectures. Similar to our study, they show slight decreases in affect over the quarter for the emotional satisfaction factor for the ASCI (as in Appendix 7). However, that study showed no difference in exam scores between men and women or between PLTL and non-PLTL students. The contrast between our results and those of Chan and Bauer (2014) suggest that adding study skill education and psychological interventions during SI may improve students’ grades.
PLTL studies that show impacts similar to ours suffer from concerns over student equivalence. For example, a workshop program for student volunteers used peer facilitators for 2 hours a week and showed a lowered rate of students earning D or F in chemistry if participating, as well as a larger increase in grades for minority volunteers compared to non-minority volunteers (Drane et al., 2005).
A more recent study of PLTL avoided this self-selection bias by comparing student outcomes in a traditional course design to a course that dropped one lecture session per week in general chemistry and substituted a required 50 minute PLTL session per week. This model moved PLTL out of the supplemental instruction format and into an element of the course required of all students. Parallel to our study, this data set showed improved retention of all students and a disproportionate improvement for underrepresented minority students (Lewis, 2011).
In addition to being empathetic and positive, peer facilitators were perceived as more relatable and approachable than graduate teaching assistants or professors. One student stated that “Having them be [sic] peers was really helpful in a lot of cases because they were like, ‘I remember when I took this. I didn't understand it either.’ So, you were like, ‘okay, I'm not alone in this’.” This result echoes a recent study showing that near-peer teachers in a medical course were perceived as more approachable than traditional teachers, and that the near-peers appeared to be more aware of learning outcomes and more invested in exam success (Tayler et al., 2015).
We also do not know how much of the program's efficacy was due to increased time-on-task, and whether the workshop will “travel.” Unlike widely implemented SI models like PLTL in general chemistry or the calculus (Triesman) workshops, the STEM-Dawgs model has only been tested at a single institution and in a single course. Collecting data on total time-on-task and testing this workshop model with other student populations are important topics for future research.
Our explanatory mixed methods design used focus group data to interpret results from the quantitative analyses. Consequently, we did not use interrater agreement to build trustworthiness, rather, we relied on a single researcher to ensure that data were consistently interpreted. We acknowledge that this can lead to a certain level of subjectivity in the interpretation of focus group data. Interpretations from focus groups are also limited by the sample size and non-purposeful sampling of interview participants. Specifically, they are subject to volunteer bias. In future studies, it would be helpful to explore key hypotheses in a prospective framework: that feelings of belonging and self-efficacy were particularly important aspects of the experience, and that the presence of empathetic and positive peer facilitators is critical to classroom culture.
Finally, although it appears that peer facilitators were a key design element in our workshop, we have yet to study the peer facilitators themselves. Previous work in chemistry has shown that students who mentor other students in a chemistry course show increased grades in a subsequent chemistry course (Amaral and Vala, 2009). Does working as a peer facilitator in the STEM-Dawgs model, and leading discussions about study skills and psychological issues, provide any additional benefit?
Fall | Winter | Spring | Total | ||
---|---|---|---|---|---|
STEM-Dawgs | Total # | 87 | 82 | 43 | 212 |
% URM | 20% | 39% | 35% | 30% | |
% EOP | 36% | 51% | 40% | 42% | |
% FG | 43% | 61% | 49% | 51% | |
% Women | 67% | 59% | 60% | 62% | |
Volunteers | Total # | 33 | 47 | 0 | 80 |
% URM | 12% | 28% | 0 | 21% | |
% EOP | 24% | 47% | 0 | 38% | |
% FG | 36% | 55% | 0 | 48% | |
% Women | 73% | 79% | 0 | 76% | |
Gen Chem | Total # | 1318 | 704 | 302 | 2324 |
% URM | 10% | 21% | 20% | 15% | |
% EOP | 16% | 28% | 31% | 22% | |
% FG | 30% | 40% | 41% | 34% | |
% Women | 52% | 56% | 52% | 53% |
Activity to be done before class | In-class activities | Study skill | |
---|---|---|---|
Week 1 | NO CLASS | ||
Week 2 (week of October 3rd) | • Course information | Suggestions for doing well on exams as told by previous students. | |
• Class introductions | Information sheet for use in study activities. | ||
• Rubber Band Blast | Study Skills: (Dunlosky et al., 2013) | ||
Week 3 | Writing exercise 1: low-stakes practice writing (online) | • Quiz 1 (Chapter 2 and 12) | Test enhanced learning, (Roediger et al., 2011) |
• Group work | |||
Week 4 | Writing exercise 2: WOOP (online) | • WOOP discussion | Productive failure, (Kapur and Bielaczyc, 2012) |
Online Mock Exam 1 | • Collaborative mock exam key | ||
Week 5 CHEM 142 EXAM 1 | Writing exercise 3: expressive writing to be done as close to your CHEM 142 exam as possible (online) | • Quiz 2 (Chapter 3) | Elaborative interrogation, (Woloshyn, Pressley and Schneider, 1992) |
• Group work | |||
Week 6 | Writing exercise 4: learning sciences (online) | • Quiz 3 (Chapter 3) | Interleaved practice, (Rohrer and Taylor, 2007) |
Post exam 1 metacognition | • Group work | ||
Week 7 | Writing exercise 5: first year transition (online) | • Videos from previous students & discussion | Distributed practice, (Richter and Gast, 2017) |
• Quiz 4 (Chapter 4) | |||
• Group work | |||
Week 8 | Writing exercise 6: the value of learning (online) | • Discussion of reasons for learning. | Self-explanation, (Berry, 1983) |
Online Mock Exam 2 | • Collaborative mock exam key | ||
Week 9 CHEM 142 EXAM 2 | Writing exercise 7: low-stakes writing II | NO CLASS | |
Writing exercise 8: expressive writing 2 to be done as close to your CHEM 142 exam as possible (online) | |||
Week 10 | Writing exercise 9: everyday chemistry (online) | • Sharing resources used for writing exercise 8 | Re-reading, (Rothkopf, 1968) Or sleep deprivation, (Pilcher and Huffcutt, 1996) |
Post exam 2 metacognition | • Quiz 5 (Chapter 15) | ||
• Group work | |||
Week 11 | Online practice final exam | • Collaborative mock final exam key | Concept mapping |
Suggestions for doing well on exams as told by previous students. | |||
Final exam week | Writing exercise 10: expressive writing 3 to be done as close to your CHEM 142 exam as possible (online) | • Group picture, swag |
Construct | Group of interest | Intercept | CHEM 142 | Vol | SOI | SOI* CHEM 142 | SOI*Vol |
---|---|---|---|---|---|---|---|
a Controlling for SAT, CCI, and Term. b Controlling for SAT, CCI, and CHEM 142 section. c Betas are log(odds) from a logistic regression. | |||||||
Exam | Overalla | −0.049 | −0.063 | −0.260 | — | — | — |
URMb | −0.057 | −0.071 | −0.254 | −0.088 | — | — | |
EOPb | −0.050 | −0.072 | −0.251 | −0.081 | — | — | |
First gen.a | −0.002 | −0.082 | −0.514 | −0.107 | 0.020 | 0.536 | |
Womenb | 0.078 | −0.071 | −0.235 | −0.146 | — | — | |
DFWc | Overallb | −3.335 | — | — | — | — | — |
URMb | −3.205 | −0.123 | −1.040 | −0.627 | 0.541 | 2.290 | |
EOPb | −3.005 | −0.356 | −0.778 | −0.861 | 0.903 | 1.367 | |
First gen.b | −3.163 | −0.380 | −0.760 | −0.367 | 0.841 | 1.117 | |
Womenb | −3.612 | 0.110 | −0.093 | 0.326 | — | — |
Construct | Group of interest | Regression | Intercept | Pre score | CHEM 142 | Vol | SOI | SOI: CHEM 142 | SOI: Vol |
---|---|---|---|---|---|---|---|---|---|
a Controlling for CCI and CHEM 142 section. b Controlling for SAT, CCI, and CHEM 142 section. | |||||||||
Mindset post | Overall | Lineara | 3.604 | 0.729 | −0.479 | −0.441 | — | — | — |
Censoreda | 2.617 | 0.822 | −0.585 | −0.547 | — | — | — | ||
URM | Lineara | 3.751 | 0.729 | −0.662 | −0.562 | −0.628 | 0.879 | 0.511 | |
Censoreda | 2.816 | 0.822 | −0.836 | −0.752 | −0.846 | 1.226 | 0.875 | ||
EOP | Lineara | 3.604 | 0.729 | −0.479 | −0.441 | — | — | — | |
Censoreda | 2.572 | 0.818 | −0.545 | −0.551 | 0.327 | — | — | ||
First gen | Lineara | 3.604 | 0.729 | −0.479 | −0.441 | — | — | — | |
Censoreda | 2.617 | 0.822 | −0.585 | 0.547 | — | — | — | ||
Women | Lineara | 3.604 | 0.729 | −0.479 | −0.441 | — | — | — | |
Censoreda | 2.617 | 0.822 | −0.585 | −0.547 | — | — | — | ||
Belonging post | Overall | Linearb | 8.540 | 0.557 | −0.443 | −1.110 | — | — | — |
Censoredb | 7.960 | 0.596 | −0.502 | −1.186 | — | — | — | ||
URM | Linearb | 8.540 | 0.557 | −0.443 | −1.110 | — | — | — | |
Censoredb | 7.960 | 0.596 | −0.502 | −1.186 | — | — | — | ||
EOP | Linearb | 8.540 | 0.557 | −0.443 | −1.110 | — | — | — | |
Censoredb | 7.960 | 0.596 | −0.502 | −1.186 | — | — | — | ||
First gen | Linearb | 8.540 | 0.557 | −0.443 | −1.110 | — | — | — | |
Censoredb | 7.960 | 0.596 | −0.502 | −1.186 | — | — | — | ||
Women | Linearb | 9.220 | 0.559 | −1.199 | −0.398 | −1.357 | 1.424 | −0.478 | |
Censoredb | 8.728 | 0.598 | −1.340 | −0.549 | −1.489 | 1.543 | −0.374 | ||
Relevance | Overall | Lineara | 4.681 | 0.696 | −0.517 | −0.785 | — | — | — |
Censoreda | 4.038 | 0.733 | −0.514 | −0.811 | — | — | — | ||
URM | Lineara | 4.435 | 0.697 | −0.265 | −0.507 | 1.016 | −1.113 | −1.186 | |
Censoreda | 3.799 | 0.734 | −0.268 | −0.511 | 0.952 | −1.099 | −1.266 | ||
EOP | Lineara | 4.681 | 0.696 | −0.517 | −0.785 | — | — | — | |
Censoreda | 4.038 | 0.733 | −0.514 | −0.811 | — | — | — | ||
First gen | Lineara | 4.681 | 0.696 | −0.517 | −0.785 | — | — | — | |
Censoreda | 4.038 | 0.733 | −0.514 | −0.811 | — | — | — | ||
Women | Lineara | 4.706 | 0.682 | −0.509 | −0.847 | 0.457 | — | — | |
Censoreda | 4.067 | 0.719 | −0.507 | −0.873 | 0.445 | — | — | ||
Self-efficacy | Overall | Linearb | 5.371 | 0.601 | −0.441 | −1.554 | — | — | — |
Censoredb | 4.880 | 0.637 | −0.544 | −1.641 | — | — | — | ||
URM | Linearb | 5.371 | 0.601 | −0.441 | −1.554 | — | — | — | |
Censoredb | 4.880 | 0.637 | −0.544 | −1.641 | — | — | — | ||
EOP | Linearb | 5.371 | 0.601 | −0.441 | −1.554 | — | — | — | |
Censoredb | 4.880 | 0.637 | −0.544 | −1.641 | — | — | — | ||
First gen | Linearb | 5.371 | 0.601 | −0.441 | −1.554 | — | — | — | |
Censoredb | 4.880 | 0.637 | −0.544 | −1.641 | — | — | — | ||
Women | Linearb | 6.445 | 0.573 | −0.476 | −1.503 | −1.004 | — | — | |
Censoredb | 6.037 | 0.607 | −0.581 | −1.583 | −1.082 | — | — | ||
ASCI Int. Sat. | Overall | Linearb | 5.791 | 0.577 | — | — | — | — | — |
Censoredb | 0.545 | 0.593 | — | — | — | — | — | ||
URM | Linearb | 5.791 | 0.577 | — | — | — | — | — | |
Censoredb | 0.545 | 0.593 | — | — | — | — | — | ||
EOP | Linearb | 5.936 | 0.576 | — | — | −0.588 | — | — | |
Censoredb | 5.700 | 0.592 | — | — | −0.637 | — | — | ||
First gen | Linearb | 5.692 | 0.574 | — | — | 0.393 | — | — | |
Censoredb | 5.442 | 0.590 | — | — | 0.411 | — | — | ||
Women | Linearb | 6.280 | 0.564 | — | — | −0.590 | — | — | |
Censoredb | 6.044 | 0.579 | — | — | −0.602 | — | — | ||
ASCI Emo. Sat. | Overall | Linearb | 7.852 | 0.562 | −0.540 | −1.248 | — | — | — |
Censoredb | 7.658 | 0.573 | −0.552 | −1.272 | — | — | — | ||
URM | Linearb | 7.852 | 0.562 | −0.540 | −1.248 | — | — | — | |
Censoredb | 7.658 | 0.573 | −0.552 | −1.272 | — | — | — | ||
EOP | Linearb | 7.852 | 0.562 | −0.540 | −1.248 | — | — | — | |
Censoredb | 7.658 | 0.573 | −0.552 | −1.272 | — | — | — | ||
First gen | Linearb | 7.852 | 0.562 | −0.540 | −1.248 | — | — | — | |
Censoredb | 7.658 | 0.573 | −0.552 | −1.272 | — | — | — | ||
Women | Linearb | 7.852 | 0.562 | −0.540 | −1.248 | — | — | — | |
Censoredb | 7.658 | 0.573 | −0.552 | −1.272 | — | — | — |
α pre | α post | CFI pre | CFI post | SRMR pre | SRMR post | ||
---|---|---|---|---|---|---|---|
PERTS N = 1690 | Mindset | 0.91 | 0.92 | 0.961 | 0.957 | 0.044 | 0.044 |
Belonging | 0.82 | 0.86 | |||||
Relevance | 0.83 | 0.85 | |||||
Self-efficacy | 0.90 | 0.90 | |||||
ASCI-V2 N = 1686 | Intellectual accessibility | 0.77 | 0.79 | 0.882 | 0.897 | 0.071 | 0.066 |
Emotional satisfaction | 0.72 | 0.75 |
Mindset | Belonging | Relevance | Self-efficacy | ||||||
---|---|---|---|---|---|---|---|---|---|
Pre | Post | Pre | Post | Pre | Post | Pre | Post | ||
STEM-Dawgs | All students | 14.3 ± 2.9 | 14.3 ± 3.2 | 18.7 ± 2.7 | 18.7 ± 3.3 | 19.8 ± 3.0 | 18.6 ± 3.6 | 17.8 ± 3.3 | 16.3 ± 4.6 |
Women | 14.2 ± 2.9 | 14.5 ± 2.9 | 18.6 ± 2.5 | 18.3 ± 3.5 | 20.4 ± 2.4 | 18.8 ± 3.6 | 17.1 ± 3.2 | 15.4 ± 4.6 | |
Men | 14.5 ± 2.9 | 14.0 ± 3.7 | 18.8 ± 3.1 | 19.6 ± 2.7 | 18.8 ± 3.7 | 18.3 ± 3.6 | 19.0 ± 3.4 | 18.0 ± 4.0 | |
First gen | 14.8 ± 2.5 | 14.9 ± 2.9 | 18.5 ± 2.8 | 18.6 ± 2.9 | 19.6 ± 3.3 | 18.5 ± 3.9 | 17.2 ± 3.6 | 15.8 ± 4.3 | |
Non-first gen | 13.9 ± 3.2 | 13.8 ± 3.4 | 18.8 ± 2.6 | 18.9 ± 3.7 | 20.1 ± 2.7 | 18.7 ± 3.4 | 18.4 ± 3.0 | 16.8 ± 4.8 | |
URM | 15.1 ± 2.0 | 14.7 ± 3.0 | 19.1 ± 2.5 | 19.1 ± 3.0 | 19.7 ± 3.0 | 19.4 ± 2.5 | 18.1 ± 2.8 | 16.4 ± 3.8 | |
Non-URM | 14.0 ± 3.1 | 14.2 ± 3.3 | 18.5 ± 2.7 | 18.6 ± 3.4 | 19.9 ± 3.0 | 18.3 ± 3.9 | 17.7 ± 3.6 | 16.3 ± 4.8 | |
EOP | 14.9 ± 2.6 | 14.8 ± 3.0 | 19.2 ± 2.1 | 18.8 ± 3.1 | 20.0 ± 2.8 | 18.8 ± 3.2 | 17.7 ± 3.2 | 15.6 ± 4.6 | |
Non-EOP | 14.0 ± 3.0 | 14.0 ± 3.3 | 18.3 ± 3.0 | 18.7 ± 3.5 | 19.7 ± 3.1 | 18.4 ± 3.9 | 17.8 ± 3.5 | 16.8 ± 4.5 | |
Volunteers | All students | 14.1 ± 2.9 | 13.9 ± 3.3 | 18.0 ± 3.2 | 17.3 ± 3.8 | 19.2 ± 3.5 | 17.9 ± 3.3 | 17.7 ± 3.6 | 14.3 ± 4.7 |
Women | 14.4 ± 2.8 | 13.9 ± 3.2 | 18.5 ± 2.9 | 17.0 ± 3.9 | 20.0 ± 2.5 | 18.2 ± 3.0 | 17.5 ± 3.7 | 13.7 ± 4.5 | |
Men | 13.1 ± 3.2 | 13.8 ± 3.7 | 16.5 ± 3.4 | 18.1 ± 3.3 | 16.6 ± 4.8 | 16.9 ± 4.0 | 18.3 ± 3.6 | 16.4 ± 5.0 | |
First gen | 14.0 ± 2.9 | 14.3 ± 3.1 | 17.7 ± 3.8 | 17.2 ± 4.2 | 18.7 ± 3.3 | 18.2 ± 3.5 | 16.8 ± 4.3 | 14.2 ± 5.1 | |
Non-first gen | 14.2 ± 3.0 | 13.5 ± 3.5 | 18.3 ± 2.6 | 17.3 ± 3.4 | 19.6 ± 3.6 | 17.6 ± 3.3 | 18.5 ± 2.8 | 14.5 ± 4.5 | |
URM | 14.5 ± 2.9 | 14.3 ± 2.9 | 17.4 ± 3.9 | 16.1 ± 5.2 | 19.0 ± 3.0 | 18.2 ± 3.4 | 16.0 ± 4.3 | 12.5 ± 5.9 | |
Non-URM | 14.0 ± 3.0 | 13.8 ± 3.4 | 18.2 ± 2.9 | 17.6 ± 3.2 | 19.2 ± 3.7 | 17.8 ± 3.4 | 18.2 ± 3.3 | 14.9 ± 4.2 | |
EOP | 14.4 ± 2.8 | 14.6 ± 2.9 | 17.5 ± 3.6 | 16.6 ± 4.4 | 19.1 ± 3.3 | 18.6 ± 3.7 | 16.6 ± 4.2 | 13.6 ± 5.7 | |
Non-EOP | 13.9 ± 3.0 | 13.4 ± 3.5 | 18.3 ± 2.9 | 17.7 ± 3.3 | 19.2 ± 3.7 | 17.4 ± 3.0 | 18.4 ± 3.1 | 14.8 ± 4.0 | |
Gen Chem | All students | 13.7 ± 3.2 | 13.2 ± 3.5 | 18.7 ± 2.5 | 18.4 ± 3.3 | 19.4 ± 2.9 | 17.7 ± 3.7 | 18.1 ± 3.3 | 16.3 ± 4.4 |
Women | 13.8 ± 3.1 | 13.3 ± 3.4 | 18.5 ± 2.4 | 18.3 ± 3.2 | 19.8 ± 2.7 | 18.2 ± 3.4 | 17.4 ± 3.4 | 15.4 ± 4.4 | |
Men | 13.5 ± 3.3 | 12.9 ± 3.6 | 18.9 ± 2.6 | 18.6 ± 3.4 | 18.8 ± 3.0 | 17.0 ± 4.0 | 18.9 ± 3.1 | 17.5 ± 4.1 | |
First gen | 14.0 ± 3.2 | 13.5 ± 3.5 | 18.7 ± 2.6 | 18.2 ± 3.3 | 19.5 ± 3.0 | 18.0 ± 3.5 | 17.9 ± 3.5 | 15.7 ± 4.5 | |
Non-first gen | 13.5 ± 3.2 | 13.0 ± 3.5 | 18.7 ± 2.5 | 18.5 ± 3.2 | 19.3 ± 2.8 | 17.6 ± 3.8 | 18.2 ± 3.3 | 16.5 ± 4.4 | |
URM | 14.2 ± 3.0 | 13.8 ± 3.4 | 18.8 ± 2.6 | 18.2 ± 3.6 | 19.7 ± 2.9 | 18.1 ± 3.9 | 18.2 ± 3.5 | 16.1 ± 4.4 | |
Non-URM | 13.6 ± 3.2 | 13.1 ± 3.5 | 18.7 ± 2.5 | 18.4 ± 3.2 | 19.3 ± 2.9 | 17.7 ± 3.7 | 18.1 ± 3.3 | 16.3 ± 4.4 | |
EOP | 14.3 ± 3.0 | 13.9 ± 3.5 | 18.6 ± 2.7 | 18.1 ± 3.6 | 19.6 ± 3.0 | 17.9 ± 3.8 | 17.8 ± 3.6 | 15.7 ± 4.5 | |
Non-EOP | 13.5 ± 3.2 | 13.0 ± 3.5 | 18.7 ± 2.5 | 18.5 ± 3.2 | 19.3 ± 2.8 | 17.6 ± 3.7 | 18.2 ± 3.3 | 16.4 ± 4.4 |
ASCI-V2 | |||||
---|---|---|---|---|---|
Intellectual accessibility | Emotional satisfaction | ||||
Student group | Pre | Post | Pre | Post | |
STEM-Dawgs | All students | 12.4 ± 3.9 | 12.4 ± 4.0 | 17.8 ± 3.6 | 17.8 ± 4.1 |
Women | 11.4 ± 3.5 | 11.8 ± 3.6 | 17.6 ± 3.7 | 17.4 ± 4.0 | |
Men | 14.2 ± 3.8 | 13.7 ± 4.5 | 18.0 ± 3.5 | 18.5 ± 4.2 | |
First gen | 12.0 ± 3.8 | 12.4 ± 4.3 | 17.3 ± 3.1 | 17.5 ± 3.9 | |
Non-first gen | 12.7 ± 4.0 | 12.5 ± 3.7 | 18.2 ± 4.0 | 18.1 ± 4.3 | |
URM | 11.9 ± 3.8 | 11.7 ± 4.4 | 17.8 ± 3.3 | 17.6 ± 3.7 | |
Non-URM | 12.6 ± 3.9 | 12.7 ± 3.8 | 17.7 ± 3.7 | 17.8 ± 4.3 | |
EOP | 11.6 ± 3.8 | 11.6 ± 4.5 | 17.7 ± 3.1 | 17.4 ± 3.7 | |
Non-EOP | 12.9 ± 3.9 | 13.0 ± 3.5 | 17.8 ± 3.9 | 18.0 ± 4.3 | |
Volunteers | All students | 11.9 ± 3.2 | 12.1 ± 4.0 | 17.4 ± 2.8 | 16.1 ± 4.0 |
Women | 11.9 ± 3.2 | 11.7 ± 4.0 | 17.4 ± 2.9 | 15.4 ± 3.9 | |
Men | 11.9 ± 3.3 | 13.1 ± 4.1 | 17.4 ± 2.7 | 18.1 ± 3.8 | |
First gen | 12.0 ± 3.4 | 12.2 ± 3.8 | 16.8 ± 3.3 | 16.2 ± 3.0 | |
Non-first gen | 11.8 ± 3.0 | 12.0 ± 4.3 | 17.8 ± 2.4 | 16.0 ± 4.7 | |
URM | 11.8 ± 3.3 | 10.8 ± 3.6 | 16.9 ± 3.1 | 15.0 ± 3.5 | |
Non-URM | 11.9 ± 3.2 | 12.4 ± 4.1 | 17.5 ± 2.8 | 16.4 ± 4.1 | |
EOP | 11.9 ± 3.8 | 11.5 ± 3.7 | 16.3 ± 3.3 | 15.6 ± 3.5 | |
Non-EOP | 11.9 ± 2.8 | 12.4 ± 4.3 | 18.0 ± 2.3 | 16.4 ± 4.3 | |
Gen Chem | All students | 13.1 ± 3.9 | 13.4 ± 4.3 | 17.8 ± 4.0 | 17.5 ± 4.4 |
Women | 12.4 ± 3.9 | 12.7 ± 4.3 | 17.4 ± 4.1 | 17.1 ± 4.5 | |
Men | 14.0 ± 3.7 | 14.4 ± 4.0 | 18.2 ± 3.9 | 17.9 ± 4.2 | |
First gen | 13.4 ± 3.9 | 13.5 ± 4.2 | 17.9 ± 4.0 | 17.2 ± 4.4 | |
Non-first gen | 13.0 ± 3.9 | 13.4 ± 4.3 | 17.1 ± 4.0 | 17.6 ± 4.4 | |
URM | 12.7 ± 4.4 | 12.6 ± 4.4 | 17.7 ± 4.6 | 16.9 ± 4.8 | |
Non-URM | 13.2 ± 3.8 | 13.6 ± 4.2 | 17.8 ± 3.9 | 17.6 ± 4.3 | |
EOP | 12.9 ± 4.2 | 12.5 ± 4.3 | 17.4 ± 4.4 | 16.6 ± 4.7 | |
Non-EOP | 13.2 ± 3.8 | 13.7 ± 4.2 | 17.9 ± 3.9 | 17.7 ± 4.3 |
This journal is © The Royal Society of Chemistry 2018 |