Exploring diverse students' trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry

Sachel M. Villafañe; C. Alicia Garcia; Jennifer E. Lewis

doi:10.1039/C3RP00141E

View PDF VersionPrevious ArticleNext Article

DOI: 10.1039/C3RP00141E (Paper) Chem. Educ. Res. Pract., 2014, 15, 114-127

Exploring diverse students' trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry

Sachel M. Villafañe , C. Alicia Garcia and Jennifer E. Lewis *
Department of Chemistry, University of South Florida, Tampa, FL, USA 33620. E-mail: jennifer@usf.edu

Received 2nd November 2013 , Accepted 19th December 2013

First published on 13th January 2014

Abstract

Chemistry self-efficacy has been defined as a student's beliefs about his or her own capability to perform a given chemistry task. These chemistry self-efficacy beliefs can be influenced by students' experiences in a course, and eventually, these beliefs could affect students' decisions to continue into STEM related-careers. In this study, we examined students' chemistry self-efficacy throughout a semester in a college preparatory chemistry course for science majors. Students' chemistry self-efficacy was measured five times during a semester using items from the Chemistry Attitudes and Experiences Questionnaire (CAEQ). A multilevel modeling analysis was performed with a growth curve model to examine changes in self-efficacy across the term, including the potential for differences by sex and race/ethnicity. Differences in expected CSE scores were noticeable at the beginning of the semester, but this gap was smaller by the end of the semester. Higher initial CSE scores and a negative trend as the term continued were observed for Black and Hispanic males when compared with White males. In contrast, the CSE trend for Hispanic females was found to be positive. The findings in this study showed the importance of measuring CSE beliefs at multiple time points and for students from demographic groups underrepresented in STEM fields, being alert to the potential for different CSE trajectories.

Introduction

Most students consider chemistry to be a difficult subject, even those students who say that they like it (Cousins, 2007). Introductory chemistry is a central part of any STEM major's curriculum; therefore students' success in the course is required to continue to more advanced courses. At most U.S. colleges and universities, a non-negligible percentage of students who attempt introductory chemistry drop the course and must either change to a non-STEM major or retake a class to remain in STEM. This situation translates into a continuous special interest in retaining and encouraging students to persist in introductory chemistry courses. The situation in other countries, as well, has pointed to declining enrollments in courses necessary for careers in STEM fields (Lyons, 2006).

Retaining students in STEM is of great interest and concern for educators and researchers. This interest and concern is even greater when we focus on females and underrepresented minorities such as Hispanics and Blacks (Hurtado et al., 2010; Bayer Corporation, 2012). Even though the number of females in STEM careers has been growing, they are still underrepresented as compared with males in upper-level STEM positions (Zeldin et al., 2008; Hill et al., 2010; Akkuzu and Akcay, 2012), and the situation is even worse for women of color (Ong et al., 2011). For Hispanics and Blacks of both sexes, representation in STEM and health professions in the U.S. is very low in comparison with the increasing population diversity in race/ethnicity (Perry et al., 2012), a state of affairs that has been termed an “enormous mismatch” (Bianchini, 2013). According to a special report from the National Science Foundation (NSF) about women, minorities, and persons with disabilities in science and engineering, the science workforce remains mainly White and male, with less than 10% of these careers occupied with females from minority backgrounds (National Science Foundation, and National Center for Science and Engineering Statistics, 2013). While issues of racial/ethnic diversity are hard to compare internationally, similar concerns about the under-representation of women in the STEM workforce have been expressed in the UK (Chimba and Kitzinger, 2010) and in Australia (Little and León de la Barra, 2009).

Given this situation, it continues to be important to study factors that could potentially influence the retention of students from underrepresented groups in STEM-related fields. Educators have tried different pedagogical approaches to help all students succeed in chemistry. Researchers have included both cognitive and non-cognitive domains when assessing these approaches, measuring such variables as attitude, motivation, prior chemistry knowledge, and prior math achievement, (Bauer, 2008; Lewis and Lewis, 2008; Seery, 2009; Cooper and Pearson, 2012; Scott, 2012). This research has begun to explain the role of both domains in student learning; however, the affective aspects of success in chemistry, such as self-efficacy, have not yet been thoroughly investigated. It has been suggested that, in a male-dominated workforce environment, self-efficacy can sustain females in their quest to persevere and succeed (Zeldin et al., 2008); the same principle may hold for female and underrepresented minority students in college chemistry courses.

Self-efficacy

The construct of self-efficacy emerged from Bandura's Social Cognitive Theory (Bandura, 1986, 1997), and it is an aspect of success that has gained the interest of science educational researchers (Pajares, 1996a; Britner and Pajares, 2001; Dalgety and Coll, 2006; Lawson et al., 2007; Zeldin et al., 2008). Self-efficacy has been defined as one's perception of his/her own ability to perform a specific task with a certain level of proficiency. This construct is relevant to student learning because, according to the theory, if a student does not feel able to do tasks necessary for learning a subject, he/she will try to avoid those tasks. Individuals with low self-efficacy will not only delay their attempts to accomplish the desired tasks, but may even give up after unsuccessful trials. On the other hand, many positive educational outcomes have been hypothesized for those individuals with high self-efficacy, such as longer perseverance to complete a task and higher achievement (Bandura, 1986; Britner and Pajares, 2001). The idea is that, the stronger a student's self-efficacy is, the more likely that the student will choose challenging learning tasks, will persist at them, and will eventually perform them successfully (Britner and Pajares, 2001; Pajares and Schunk, 2001; Margolis and McCabe, 2003; Zeldin et al., 2008). Rather than being a fixed element, self-efficacy is thought to be cyclical in nature, which means a student's self-efficacy changes with the experiences each student has (Dalgety and Coll, 2006). In general, self-efficacy will increase with positive experiences and decrease with negative experiences. One important aspect of the construct of self-efficacy is that it is task specific and context specific; therefore, students' perceptions of their ability to perform well on chemistry-related tasks in a particular course would be measured by their chemistry self-efficacy while taking that course.

At many universities, preparatory or bridging chemistry courses are designed for students intending to major in STEM who are not prepared or who do not feel prepared to enroll in a college general chemistry course. These courses provide students with some of the basic chemistry knowledge that will help them to be prepared to take general chemistry. It has been assumed that a well-designed preparatory chemistry course should lead to increases in chemistry content knowledge as well as in chemistry self-efficacy (Schmid et al., 2012; Youl et al., 2012). Therefore, it is of interest to examine the changes in students' chemistry self-efficacy throughout a semester of a preparatory college chemistry course.

Self-efficacy and sex differences in STEM

Since students with higher self-efficacy are thought to be more likely to persist in STEM careers, and females are currently underrepresented in many STEM fields (Zeldin et al., 2008), it is important to examine whether sex differences related to self-efficacy toward science exist. In general, many previous studies related to science self-efficacy have reported that males tend to have higher self-efficacy than females, although the females may be just as capable as the males to succeed in science (Pajares, 1996a, 2002; Lloyd et al., 2005; Dalgety and Coll, 2006; Hutchison-Green et al., 2008; Michaelides, 2008).

Considering chemistry specifically, in a multiple-time-point study with college chemistry students in New Zealand, Dalgety and Coll (2006) measured students' chemistry self-efficacy three times within one year using the self-efficacy component of the Chemistry Attitudes and Experiences Questionnaire (CAEQ). They reported that, overall, students were very confident about the tasks presented, but males had higher chemistry self-efficacy than females for certain chemistry tasks. Specifically, they found that males reported a greater degree of confidence in advanced-level skills, such as explaining chemistry content to others, than did females. Most of those differences were no longer observed by the end of the first semester or by the end of the academic year, but some new differences had emerged, still favoring males. Recent research in the area of chemistry self-efficacy includes the development of a new self-efficacy measure, the College Chemistry Self-Efficacy Scale (Uzuntiryaki and Aydin, 2009) and the use of Structural Equation Modeling for establishing a relationship between self-efficacy and anxiety (Aydin et al., 2010; Kurbanoglu and Akim, 2010), self-efficacy and attitude (Kurbanoglu and Akim, 2010), and self-efficacy and critical thinking (Uzuntiryaki-Kondakci and Capa-Aydin, 2013) for first year college chemistry students in Turkey, but none of this work looked specifically for differences between males and females. In another study with first-year chemistry students, Cook (2013) found that chemistry self-efficacy and attitude were significant predictors of students' intention to take future chemistry courses, still with no attention to potential differences between male and female students.

More broadly, looking at science self efficacy or engineering self-efficacy rather than chemistry self-efficacy, studies with middle school science students (Britner and Pajares, 2001; Kiran and Sungur, 2012) and college engineering students (Concannon and Barrow, 2012) have been mixed with respect to differences between males and females. Britner and Pajares (2001), using a science self-efficacy scale that measures students' confidence in earning a grade in their science class, found that females had a higher science self-efficacy than males. In a more recent study, Kiran and Sungur (2012) used the Sources of Self-Efficacy Scale (SSES) to assess middle school students' science self-efficacy in Turkey and found that the science self-efficacy for females and males did not differ. At the college level, in a recent study with engineering students, Concannon and Barrow (2012) found that there were not significant differences in the overall mean of self-efficacy for the females when compared with the males. This result contrasts somewhat with an earlier qualitative study by Hutchison-Green and co-workers (2008) investigating the impact of first-year engineering experiences on students' efficacy beliefs, in which males were found to be more likely to focus on positive experiences, while females were more likely to focus on negative experiences. Similarly, in a longitudinal and cross-institutional study with female engineering students in the U.S., Marra et al. (2009) found that, although females had increases in some aspects of self-efficacy over time, they experienced a decrease with respect to their feeling of inclusion, particularly for African-American females.

Since the results in previous literature about sex differences are mixed, with more recent work tending to show fewer differences, and there is no recent work investigating this issue for chemistry self-efficacy, it is salient to examine whether sex differences exist in our study. Understanding students' self-efficacy beliefs and whether there are differences by sex might help us to understand factors that could influence the level of representation of females in STEM fields, particularly females from underrepresented minority groups.

Self-efficacy and underrepresented minorities in STEM

Another aspect of self-efficacy that has been of interest in the last decade is the self-efficacy beliefs of underrepresented minorities in STEM fields. While some studies, as discussed previously, have focused on females in science and engineering (Zeldin and Pajares, 2000; Zeldin et al., 2008; Marra et al., 2009), fewer studies have focused on racial/ethnic minorities (Britner and Pajares, 2001; Perry et al., 2012). Britner and Pajares (2001), by assessing students' confidence in earning a grade in their science class, found that White students have a stronger science self-efficacy than African American students in middle school science. In another study with sixth graders, Perry et al. (2012) also found that students' confidence for completing a science course was lower for African American students than for White students. In the same study, Perry et al. examined the interaction between sex and race/ethnicity. They found that a significant interaction for African Americans and Latinos; where African American and Latino females have more confidence in their ability (self-efficacy) to complete science courses than males. No such comparison studies have been reported at the college level to date.

Gaps in the literature

As shown in the previous sections, studies in the area of science self-efficacy focusing on possible differences with respect to sex and underrepresented minority status have appeared within the last decade. Studies focusing specifically on chemistry self-efficacy at the college level (Dalgety and Coll, 2006; Uzuntiryaki and Aydin, 2009; Kurbanoglu and Akim, 2010; Cook, 2013; Uzuntiryaki-Kondakci and Capa-Aydin, 2013) as well as studies that look at students' self-efficacy beliefs in a longitudinal way (Dalgety and Coll, 2006; Marra et al., 2009) are still limited, however, although a growing interest has been observed in the last few years. Since there is ongoing interest to retain females and racial/ethnic minorities in STEM fields, more studies are needed in this area (Zeldin et al., 2008). Regarding racial/ethnic minority groups, only two studies have reported on the self-efficacy for these groups and both of them are with middle school students (Britner and Pajares, 2001; Perry et al., 2012), with no studies on college students or in chemistry courses. Thus, this study will address major gaps in the literature by examining students' chemistry self-efficacy during a semester in a college preparatory chemistry course with comparisons between different groups of students by sex and race/ethnicity.

Present study

The goal of the present study is to examine students' chemistry self-efficacy throughout a semester of a college preparatory chemistry course for science majors. Specifically, the aim is to use multilevel modeling (MLM) analysis to explore students' trends during the semester, focusing on possible differences by sex and race/ethnicity, while taking into consideration prior math achievement (SAT Math). The research questions guiding this study are:

(1) What are students' chemistry self-efficacy (CSE) beliefs at the beginning of the semester? What differences, if any, can be observed for groups underrepresented in STEM fields?

(2) What changes can be observed for students' chemistry self-efficacy (CSE) beliefs during the semester? What differences, if any, can be observed for groups underrepresented in STEM fields?

Method

Instruments

Chemistry self-efficacy (CSE). Students' chemistry self-efficacy was measured using items from a subscale of the Chemistry Attitudes and Experience Questionnaire (CAEQ) developed by Dalgety et al. (2003). The chemistry self-efficacy subscale consists of 17 items measuring different aspects of students' chemistry self-efficacy such as learning chemistry theory, applying chemistry theory, learning chemistry skills, and applying chemistry skills (Dalgety et al., 2003). For the purpose of this study, items related to laboratory skills were not used, since the course has no laboratory component. Five items assessing students' self-efficacy beliefs regarding applying chemistry knowledge were used for the present study. These items were chosen in alignment with one of the goals of the course, that students are expected to apply the chemistry knowledge they learn in the course; therefore these items were relevant for our purpose (see Appendix A for all the items). For example, one of the items was ‘Applying a set of chemistry rules to different elements of the Periodic Table’. Students were asked to rate how confident they felt about completing each chemistry-related task on a five-point scale between being Not confident to Totally confident. Higher numbers on the scale (4 or 5) indicate that students feel very confident to complete the given task; on the other hand, lower numbers (1 or 2) will indicate students were not so confident about completing the given task.

Prior math achievement. The quantitative part of the SAT was used as a measure of students' prior math achievement (SATM hereafter). The SAT is a college entrance exam in the U.S. that is typically administered during the last year of high school. The SATM scores range from 200 to 800, and they have been found to have high internal consistency coefficients (Cronbach's alpha = 0.92) (Ewing et al., 2005). SATM has been used as a predictor of student performance in science (Ewing et al., 2005) and, importantly for its role as a covariate here, its relationship with student chemistry performance has been well documented (Spencer, 1996; Wagner et al., 2002; Lewis and Lewis, 2007, 2008; Pyburn et al., 2013; Xu et al., 2013). Although other measures of math ability exist (Wagner et al., 2002; Pienta, 2003; Cooper and Pearson, 2012), SATM has been found to be an appropriate measure of students' prior math achievement and is used to place students in the preparatory chemistry course (see below); therefore it is relevant to include it as covariate in this study.

Data collection and participants

Participants were students enrolled in a preparatory chemistry course for science majors during Fall 2007 at a large southeastern public research university. Preparatory chemistry at this institution is a one-semester course recommended for science majors who have little or no exposure to secondary school chemistry (Heredia et al., 2012). The course is also recommended for students who have a low SATM score (lower than 550).

Students' self-efficacy in chemistry was measured five times during the semester using the CSE survey. Surveys were one of several options for students to receive attendance points (other options included responding to clicker questions, turning in a scantron, etc.). Students were informed that their participation was voluntary and that the course instructor would not be provided with individualized response data but rather with an aggregate result. The first CSE administration was during the first day of classes to assess students' self-efficacy beliefs before any formal instruction, and it will be referred to as CSE at time 0 or CSE0. The next four times the CSE was administered during class time, in the class immediately before each of the four exams to be consistent throughout the semester and to avoid having students base their survey responses on exam results. These will be referred to as CSE at time 1 to CSE at time 4 or CSE1 to CSE4, respectively.

A total of 409 students took the preparatory chemistry course in Fall 2007. For this study, only students who completed at least one survey and had demographic information available were included for further analyses. Table 1 presents the demographic information for these 384 participants. As presented in Table 1, there are 278 females (72.4%) and 106 males (27.6%) in our sample. Regarding race/ethnicity, 50.8% are White, 18.8% Hispanic, 18.8% Black, and 7.3% Asian.

Table 1 Demographics: number and percentage of students by sex and race/ethnicity (n = 384)

	No. of students	Percentage
Sex
Female	278	72.4
Male	106	27.6

Race/ethnicity
White (Not of Hispanic Origin)	195	50.8
Hispanic or Latina	72	18.8
Black (Not of Hispanic Origin)	72	18.8
Asian	28	7.3
Unknown	12	3.1
American Indian or Alaskan Native	3	0.8
Others	2	0.5

Data analysis

Descriptive statistics

Descriptive statistics were obtained using SAS 9.3. General trends and univariate normality for each variable (CSE item and composite scores and SATM scores) were assessed using descriptive statistics.

CSE psychometrics

Internal consistency reliability (Cronbach's alpha coefficient) for each of the CSE survey administrations was calculated using SAS 9.3. Internal consistency reliability will allow us to examine if the items in the CSE scale yield consistent scores. Although in the literature there are different cutoffs for Cronbach's alpha coefficient, as it depends on the test purpose (Murphy and Davidshofer, 2005), the most common cutoff reported to determine whether scores are sufficiently reliable is 0.7 (Cortina, 1993; Murphy and Davidshofer, 2005).

In terms of validity evidence (Arjoon et al., 2013), the internal structure of the scores was examined using Confirmatory Factor Analysis (CFA). CFA was performed using MPlus 5.2 to estimate how well the proposed model fits the data. The proposed model is a first-order model (1-factor solution), where the 5 items related to students' beliefs regarding their confidence toward applying chemistry theory to different tasks were set to correlate to each other. A Maximum Likelihood estimator was employed. Guidelines for fit statistics such as Comparative Fit Index (CFI) greater than 0.90 and Standardized Root-Mean Squared Residual (SRMR) less than 0.08 were considered, to determine if the model had a good fit to the data (Hu and Bentler, 1999; Cheng and Chan, 2003). The analyses allow us to determine whether the interpretation of the chemistry self-efficacy scores from the CSE survey can be done using a composite score from the scale (i.e. at the scale level) or if the interpretation should be done at the item level.

Multilevel modeling analysis (MLM)

Multilevel modeling (MLM) is a multivariate analysis that is an adaptation of traditional regression models. This analysis involves data at different levels with observations nested in larger units. It has many advantages when compared with simple linear regression or other multivariate analysis such as MANOVA (Osborne, 2000). The principal advantage of using MLM is that it allows working with observations that are not independent, for example observations from students who are in the same classroom or multiple observations of the same students over time. Since MLM takes into consideration the different levels of the data and relevant characteristics at each level, the effect estimates are less biased (Raudenbush and Bryk, 2002; Luke, 2004), than those for standard linear regression, making MLM a suitable analysis for this study.

MLM was performed to examine students' chemistry self-efficacy changes throughout a semester using Proc Mixed in SAS 9.3. A growth curve model was chosen to examine changes in time, including the potential for differences by sex and race/ethnicity. The changes in students' self-efficacy scores over time, on average by sex and race/ethnicity, were estimated by interpreting parameter estimates obtained from the analysis. The estimation method used for analysis is Restricted Maximum Likelihood (REML), which produces less biased random-effect estimates (i.e. variance components) than Maximum Likelihood (ML) (Raudenbush and Bryk, 2002; Luke, 2004).

In this study, the focus of the MLM analysis was the change in the chemistry self-efficacy scores; therefore, the outcome variable is represented by students' CSE scores at each time point. The growth curve model has two levels, chemistry self-efficacy observations over time nested within students. The first level in the model is represented by student CSE observations. One predictor, time, was added to Level 1. Time indicates when each CSE observation was recorded, for example, for the initial administration of CSE (CSE0), time is 0, second administration (CSE1), time is 1, etc.

Level 2 in the model corresponds to the student level, where relevant student characteristics are considered as predictors. Since we are interested in tracking changes of CSE by sex and race/ethnicity, students' sex and race/ethnicity were added to the model as Level 2 predictors. Also, students' prior math achievement as measured by the SATM scores was added to Level 2 as a covariate. This covariate will allow us to control for differences in students' SATM scores. This single MLM model was used to address the research questions that guide this study.

Results and discussion

Descriptive statistics

The chemistry self-efficacy (CSE) survey was administered five times during the semester. Descriptive statistics were performed on each of the CSE surveys and SATM scores to examine general trends in the data and to assess the univariate normality of each variable for the 384 students in the sample. As shown in Table 2, the sample size for each variable varies, since it indicates how many participants completed each of the surveys. This variation in sample size demonstrates that not all participants have complete sets of data. For further analysis, all participants will be included in the analysis, even with incomplete data sets, since MLM analysis can handle this situation and can estimate coefficients for these students (Raudenbush and Bryk, 2002; Gibson and Olejnik, 2003; Lewis and Lewis, 2008).

Table 2 Descriptive statistics for CSE and SATM for all students

	N	Mean	Std dev	Minimum	Maximum	Skewness	Kurtosis
CSE0	355	2.90	0.82	1	5	−0.01	0.04
CSE1	338	3.26	0.72	1	5	−0.07	−0.10
CSE2	290	3.35	0.80	1	5	−0.34	0.002
CSE3	277	3.32	0.84	1	5	−0.30	−0.12
CSE4	274	3.41	0.80	1	5	−0.35	0.33
SATM	334	511	57.2	330	760	0.40	1.78

The mean of the CSE scores ranged from 2.90 to 3.41. Univariate normality was assessed by examination of the skewness and kurtosis values for each variable. As presented in Table 2, each variable has absolute skewness values of less than 1 and absolute kurtosis values of less than 2. These values indicate that these variables are approximately normally distributed.

In general, a positive trend in the mean scores of chemistry self-efficacy can be observed. At the beginning of the semester, students' overall mean CSE score was 2.90, which indicates that students were not so confident in applying the chemistry theory to different tasks; however, the mean scores for later administrations, in general, indicate that students' confidence gradually increases. At the end of the semester, the overall CSE mean score was 3.41.

Table 1 presented the demographic distribution for the 384 students. For further analysis, only participants with SATM and classified as Asian, Black, Hispanic, and White were included. Other racial/ethnic categories were removed because their sample size is too small for multivariate analysis. After this adjustment, a total of 320 participants remained, with a demographic distribution and descriptive statistics similar to the initial sample. For the demographic information, there are 235 females (73.4%) and 85 males (26.6%), and, regarding race/ethnicity, 51.6% are White, 20.6% Hispanic, 19.7% Black, and 8.1% Asian. The descriptive statistics for this data set are presented in Table 3. The CSE mean ranged from 2.93 to 3.39. Univariate normality for this data set again follows an approximately normal distribution according to skewness and kurtosis values. Descriptive statistics for each item in each administration are presented in the appendix for both samples.

Table 3 Descriptive statistics for CSE and SATM (N = 320)

	N	Mean	Std dev	Minimum	Maximum	Skewness	Kurtosis
CSE0	297	2.93	0.83	1	5	0.09	0.02
CSE1	279	3.29	0.73	1	5	−0.09	0.02
CSE2	237	3.40	0.80	1	5	−0.48	0.31
CSE3	230	3.36	0.84	1	5	−0.34	0.004
CSE4	229	3.39	0.81	1	5	−0.33	0.40
SATM	320	511	57.4	330	760	0.40	1.82

CSE psychometrics

Internal consistency reliability was assessed using Cronbach's alpha coefficient. Cronbach's alpha for each CSE administration (CSE0-CSE4) ranged from 0.79 to 0.87. These values indicate good internal consistency reliability for the CSE scores. The covariance matrices for each administration are presented in the appendix. Confirmatory Factor Analysis (CFA) was performed on the CSE scores (covariance matrices) for each CSE administration. Fit indices for each administration indicate a reasonable fit for the 1-factor solution. The CFI index ranged from 0.88 to 0.96 and SRMR values ranged from 0.04 to 0.06. Therefore, it is reasonable to interpret the scores of the CSE surveys as measuring one construct, chemistry self-efficacy beliefs, or more specifically, students' belief regarding how confident they feel about applying chemistry knowledge to the tasks.

Multilevel modeling analysis (MLM)

Model evaluation

The assumptions for MLM were examined for the set of variables in the study. Boxplots and histograms were used to examine the distribution of Level 1 and Level 2 residuals (normality assumption). Both representations showed an approximately normal distribution of the residuals. Further descriptive statistics showed that the mean for the Level 1 and Level 2 residuals is 0, and the examination of the skewness and kurtosis suggest normal distributions. No outliers were identified in the analysis. An examination of the residuals in the scatterplot and the test of homogeneity of variance did not show evidence of violation for the homoscedasticity assumption for the set of variables in this study.

Unconditional model

An unconditional or baseline model was run to determine the degree of explained variance in chemistry self-efficacy scores between students. This model is called a baseline model because it has no predictors.

Level 1 (within students):


CSEscores_ti = π_0i + e_ti	(1)

Level 2 (between students):


π_0i = β₀₀ + r_0i	(2)

In this model as shown in eqn (1) and (2), CSEscores_ti is the CSE score obtained at time t by student i. The Level 1 equation's intercept (π_0i) represents the CSE mean score across all time points for student i. The Level 2 equation's intercept (β₀₀) represents the CSE grand mean across all time points for all students. The variability of CSE scores within student i is represented by e_ti and between students by r_0i. The variance components for the baseline model variability between and within students were obtained from this analysis. The Level 2 variance component (τ₀₀), or variance between students, is 0.21 and the Level 1 variance component, or variance within students (σ²), is 0.45. The intraclass correlation (ICC), or the ratio of between-group variance to total variance, was 0.32, indicating that 32% of the overall variation in student chemistry self-efficacy lies between students as shown in eqn (3). It is therefore reasonable to use an MLM analysis, since an ICC of 0.25 or above is recommended for these analyses (Kreft, 1996; Heinrich and Lynn, 2001).

ICC = ρ = τ₀₀/(τ₀₀ + σ²)


ICC = ρ = 0.21/(0.21 + 0.45) = 0.32	(3)

“Unconditional” model with time as predictor

A second model similar to the unconditional model was run, with the only difference the addition of a time predictor at Level 2 as shown in eqn (4). This model was used to determine the degree of explained variance at the student level, ICC, when the time predictor is added.

Level 1 (within students):


CSEscores_ti = π_0i + π_1i × (time)_ti + e_ti	(4)

Level 2 (between students):


π_0i = β₀₀ + r_0i	(5)


π_1i = β₁₀ + r_1i	(6)

In this model, CSEscores_ti represents the CSE score obtained by student i at time t. For the Level 1 equation, time was added as predictor to the model. Time is an ordinal variable that spans from 0 (representing first administration, CSE0) to 4 (representing the last administration, CSE4). The Level 1 equation's intercept (π_0i) represents the expected CSE score at time 0, while the slope (π_1i) describes the change in an individual's CSE score over time. The ICC was calculated for this model and 49% of the overall variation in student chemistry self-efficacy lies between students.

Full MLM model

A multilevel model was constructed as prescribed in eqn (7)–(9) to examine students' chemistry self-efficacy beliefs at the beginning of a college preparatory chemistry course and their changes, if any, throughout the semester.

Level 1 (within students):


CSEscores_ti = π_0i + π_1i × (time)_ti + e_ti	(7)

Level 2 (between students):

π_0i = β₀₀ + β₀₁ × (sex)_ti + β₀₂ × (ethnicity)_ti + β₀₃ × (SATM)_ti + β₀₄ × (sex × ethnicity)_ti + r_0i


π_1i = β₁₀ + β₁₁ × (sex)_ti + β₁₂ × (ethnicity)_ti + β₁₃ × (SATM)_ti + β₁₄ × (sex × ethnicity)_ti + r_1i	(8)

Combined equation:


CSEscores_ti = β₀₀ + β₀₁ × (sex)_ti + β₀₂ × (ethnicity)_ti + β₀₃ × (SATM)_ti + β₀₄ × (sex × ethnicity)_ti + β₁₀ × (time)_ti + β₁₁ × (sex × time)_ti + β₁₂ × (ethnicity × time)_ti + β₁₃ × (SATM × time)_ti + β₁₄ × (sex × ethnicity × time)_ti + r_0i + r_1i × (time)_ti + e_ti	(9)

In this model, the Level 1 equation (eqn (7)) is the same as in the previous model (eqn (4)) where time is added as predictor. Level 2 is defined using two equations to capture the effect that changes in student variables or characteristics (sex, race/ethnicity, SATM) can have on the slope and intercept of the Level 1 equation. These variables, sex, race/ethnicity (represented in the model as ethnicity), and SATM, were added to the model in each of the Level 2 equations. SATM scores were used as a covariate and grand-mean centered on 511. Four dummy variables were created to represent the eight sex and racial/ethnic groups using standard MLM procedures. The dummy variable for sex was defined using 0 for males as the reference group and 1 for females. The four racial/ethnic groups were defined in the model using 0 for the Whites as the reference group, and 1 for the other groups, Asians, Blacks, and Hispanics (see Table 4). Since both sex and race/ethnicity categories are categorical (defined by 0 and 1), parameters were reported in the unstandardized form.

Table 4 Dummy coding scheme, intercept, and slope for each group of students

Dummy code	Sex	Race/ethnicity			Intercept	Slope
Dummy code	D1	D2	D3	D4	Intercept	Slope
Male, White	0	0	0	0	β ₀₀	β ₁₀
Female, White	1	0	0	0	β ₀₀ + β₀₁	β ₁₀ + β₁₁
Male, Asian	0	1	0	0	β ₀₀ + β_02,A	β ₁₀ + β_12,A
Male, Black	0	0	1	0	β ₀₀ + β_02,B	β ₁₀ + β_12,B
Male, Hispanic	0	0	0	1	β ₀₀ + β_02,H	β ₁₀ + β_12,H
Female, Asian	1	1	0	0	β ₀₀ + β₀₁ + β_02,A + β_04,A	β ₁₀ + β₁₁ + β_12,A + β_14,A
Female, Black	1	0	1	0	β ₀₀ + β₀₁ + β_02,B + β_04,B	β ₁₀ + β₁₁ + β_12,B + β_14,B
Female, Hispanic	1	0	0	1	β ₀₀ + β₀₁ + β_02,H + β_04,H	β ₁₀ + β₁₁ + β_12,H + β_14,H

Another Level 2 predictor included in the model is the interaction between sex and race/ethnicity. This interaction at the Level 2 equation is included to examine whether students' initial CSE scores and their changes over time are different for females and males of the different racial/ethnic groups. Adding this necessary interaction makes the interpretation of the results for each group complex; working through the dummy variables as shown in Table 4 can be helpful to understand the interpretation of the parameter estimates. For the intercept, β₀₀ represents the expected CSE score at t = 0 for White males with the average SATM score. β₀₁ represents the difference in the intercept (initial CSE score) between a White male and a White female. β₀₂ has three estimates, one estimate for each comparison between each male of the racial/ethnic group (Asian, Black or Hispanic) and reference group, White males. The interaction term β₀₄ indicates the degree to which the sex difference changes across race/ethnicity, e.g., the sex difference for Hispanics is defined by two parameters, β₀₁ + β_04,H.

For the time slope, β₁₀ indicates the CSE score change over time unit for White males. The interpretation of time slope difference across groups using β₁₀, β₁₁, β₁₂, and β₁₄ follows the same pattern as for the intercept. For example, the slope for Hispanic males is defined by β₁₀ + β_12,H.

A pseudo effect size for MLM was calculated to determine the proportion of error variance explained by the set of predictors when compared to the “Unconditional” model with the time predictor. This effect size is analogous to R² in regression analysis and was calculated using Raudenbush and Bryk's (2002) approach. The result was that, for Level 2, 14% of the variance was explained by the set of additional predictors.

General results from full MLM model

The results of the full MLM model are shown in Table 5. Estimates for each fixed effect, e.g. intercepts and slopes, by sex and race/ethnicity are presented. Given that we are interested in the expected chemistry self-efficacy scores at the beginning of the semester and the changes in expected scores throughout the semester for the different groups (e.g. by sex and race/ethnicity), the results obtained from the full model were used to create a figure for ease of interpretation, as follows: (1) for each group, a linear equation was constructed to represent the modeled trend, (2) these equations were then used to obtain CSE scores at each time, and (3) the modeled trend for each group is presented in Fig. 1.

Table 5 Fixed and random effects for full MLM model

Symbol	Description	Sex	Race/ethnicity	Estimate	Standard error	Significance
a Statistically significant at p < 0.05.
Fixed effects
β ₀₀	Intercept			3.08	0.11	<0.0001^a
β ₁₀	Time			0.12	0.04	0.0010^a
β ₀₁	Sex	Female		−0.19	0.13	0.1456
β _02,A	Ethnicity		Asian	0.90	0.31	0.0036^a
β _02,B	Ethnicity		Black	0.44	0.24	0.0707
β _02,H	Ethnicity		Hispanic	0.58	0.22	0.0077^a
β ₀₃	SATM			0.0012	0.0008	0.1199
β _04,A	Sex × ethnicity	Female	Asian	−1.02	0.36	0.0050^a
β _04,B	Sex × ethnicity	Female	Black	−0.14	0.27	0.6155
β _04,H	Sex × ethnicity	Female	Hispanic	−0.54	0.25	0.0328^a
β _12,A	Time × ethnicity		Asian	−0.11	0.11	0.3028
β _12,B	Time × ethnicity		Black	−0.17	0.08	0.0407^a
β _12,H	Time × ethnicity		Hispanic	−0.20	0.07	0.0064^a
β ₁₁	Time × sex	Female		0.0090	0.04	0.8393
β ₁₃	Time × SATM			8.958 × 10⁻⁶	0.000263	0.9728
β _14,A	Time × sex × ethnicity	Female	Asian	0.15	0.12	0.2395
β _14,B	Time × sex × ethnicity	Female	Black	0.12	0.09	0.2078
β _14,H	Time × sex × ethnicity	Female	Hispanic	0.21	0.09	0.0151^a

Random effects
σ ² (e_ti)				0.3585	0.0195	<0.0001^a
τ ₀₀ (r_0i)				0.3152	0.0474	<0.0001^a
τ ₁₀ (r_1i)				−0.0480	0.0140	0.0006^a
τ ₁₁				0.0197	0.0057	0.0003^a


	Fig. 1 MLM trends for CSE scores by sex and race/ethnicity.

Examining the CSE score trajectories for students in a college preparatory chemistry course is important to determine if our expectation that the course should increase students' chemistry self-efficacy is reasonable. The students' overall CSE trajectories can first be examined using the raw data presented in Table 3. Students' overall CSE score at the beginning of the semester, time 0, is 2.93 while at the end of the semester, time 4, it is 3.39. The difference in CSE scores is 0.46 points. The effect size for this difference is 0.6, which is a medium effect size according to Cohen's d guidelines, 0.20 (small), 0.50 (medium), and 0.80 (large) (Cohen, 1988). This effect size indicates a meaningful increase in the students' overall CSE scores, which will validate our expectation for a college preparatory course. Although the actual CSE scores for our sample are lower than what Dalgety and Coll (2006) reported with their first year chemistry students, the effect size for the increase in CSE is consistent with their results that, after a year of first-year chemistry, students' CSE scores for most of the chemistry self-efficacy items increased with a small to medium effect.

Since one of the interests of this study is to examine the CSE trajectory for the underrepresented groups in our sample, focusing on the overall raw data is not enough. Results from the MLM analysis allow us to look at the modeled CSE trajectories for each group of students with statistical significance testing for group differences. As displayed in Fig. 1, the expected CSE scores at the beginning of the semester (time 0) of college preparatory chemistry and the CSE trajectory across the semester are not the same for all students. Therefore the results from the MLM analysis were used to describe the differences among the groups by sex and race/ethnicity at the beginning of the semester and their changes across the semester.

What are students' chemistry self-efficacy (CSE) beliefs at the beginning of the semester? What differences, if any, can be observed for groups underrepresented in STEM fields?

Students' CSE scores at the beginning of the semester were examined using the parameter estimates displayed in Table 5. The intercept for the full model (β₀₀) is 3.08, which indicates the expected chemistry self-efficacy score for a White male student of average SATM at time 0 (at the beginning of the semester). This score for White males indicates a relatively neutral belief regarding their confidence in applying chemistry knowledge at the beginning of the semester.

The fixed effect estimates (β₀₁, β₀₂) represent the expected differences in CSE scores by sex and race/ethnicity when compared to the reference group at time 0. The observed fixed effect (β₀₁) by sex (for females) at time 0 is −0.19, which means that, on average, White female students had a lower but not significantly different (p = 0.146) expected CSE score than White males. Even though, in general, the expected CSE scores were lower for most females of the different racial/ethnic groups in our study, our results for White females are consistent with recent studies in science (Kiran and Sungur, 2012) and engineering (Concannon and Barrow, 2012) self-efficacy, where those differences were not statistically significantly different.

Regarding race/ethnicity, statistically significant effects are observed for two groups, Asians (β_02,A) and Hispanics (β_02,H). For Asians, the effect is 0.90, which indicates that at time 0, for students of average SATM, the expected chemistry self-efficacy score for an Asian male student is 0.90 higher than for a White male student (p = 0.0036); while for Hispanics, the effect is 0.58, which indicates that at time 0, the expected chemistry self-efficacy score is 0.58 higher than for a White male student (p = 0.0077). From Fig. 1, we can see these differences in expected CSE scores at the beginning of the semester. For example, on average, the expected CSE score for a Hispanic male at the beginning of the semester is 3.66, while for a White male it is 3.08.

Significant effects are also observed for sex × ethnicity interactions for Asians (β_04,A) and Hispanics (β_04,H), which indicates that the effect of ethnicity on chemistry self-efficacy for these two groups is different for females and males. In both cases, the effect on CSE is negative, which signals that, on average, the Asian and Hispanic females in this study have a lower initial expected CSE score than the Asian and Hispanic males, respectively, when these differences are compared to the difference in expected score for males and females in the reference group. For Asians, the effect is −1.02 (p = 0.0050) and for Hispanics it is −.54 (p = 0.0328). A difference between Hispanic males and females also been reported by Perry et al. (2012); however, the direction of the effect is not the same. Perry et al. reported that Hispanic females in sixth grade were more confident in their ability to complete a science course than the Hispanic males, while our results reveal that at the beginning of a college preparatory chemistry the opposite is likely to be true for our sample. It is always important to remember that, for self-efficacy, the context is important; therefore, this reversal of results between 6th graders and college students points to the fact that we need more studies in similar contexts to be able to compare and draw conclusions.

The students' chemistry self-efficacy beliefs at the beginning of the semester in this preparatory chemistry course reflect that most of the female students were not that confident about applying chemistry knowledge. These differences may suggest that, although the majority of these students are expected to have very little exposure to chemistry, their beliefs have already been influenced, perhaps by their previous school learning experiences (Dalgety and Coll, 2006).

What changes can be observed for students' chemistry self-efficacy (CSE) beliefs during the semester? What differences, if any, can be observed for groups underrepresented in STEM fields?

CSE scores were tracked during the semester, and CSE trends by sex and race/ethnicity were compared with the reference group (White males). These trends capture the changes in CSE scores throughout the semester, and are displayed by the time variable added to the model. The time effect (β₁₀) is 0.12, which indicates the change in the expected CSE score for a White male of average SATM as time increases by one unit. For example, the expected CSE score for a White male student of average SATM at time 2 is 3.33; whereas at time 3, his CSE expected score is 3.45, both of which are above neutral for the CSE scale. These above neutral scores are not unreasonable, considering that CSE scores for chemistry majors were reported to be around 4 for most of the items in the original survey (Dalgety and Coll, 2006).

As presented in Table 5, the interactions between time and race/ethnicity have significant effects for two racial/ethnic groups, Blacks (β_12,B) and Hispanics (β_12,H). This interaction suggests that the changes in chemistry self-efficacy scores over time depend on the race/ethnicity of the student; therefore Black males' and Hispanic males' changes in CSE scores are significantly different when compared to White males' changes in CSE scores. The change of CSE scores over time for a Black male student of average SATM is −0.17 (p = 0.0407) and for a Hispanic male student is −0.20 (p = 0.0064). These negative changes denote that, in general, the CSE scores for Blacks and Hispanic males over time tend to decrease, while White males' CSE scores tend to increase. This trend for Black and Hispanic males is observed in Fig. 1. This finding demonstrates that, after experiencing a semester of college preparatory chemistry (time 4), Black and Hispanic males' confidence levels, although lower than at the beginning of the semester, are more similar to the confidence levels expressed by other groups of students in our sample. If this negative trend were to continue during general chemistry it would be a concern, however, since students with low chemistry self-efficacy may have less chance of staying in STEM-related fields.

To determine whether differences in CSE changes over time by sex vary across race/ethnicity, the interactions of sex and race/ethnicity with time were obtained. For this interaction, a significant effect can be seen for Hispanics. The effect is 0.21, which indicates that the change in self-efficacy over time follows a different trend for Hispanic males and females when compared to the trend for the reference group. The expected chemistry self-efficacy trend for a Hispanic female student with an average SATM is positive, similar to the White males' trend, which indicates that their expected CSE scores increase throughout the semester in contrast to the negative trend for Hispanic male students. Hispanic females became more confident in their ability to apply chemistry knowledge during the semester; thus for them, the experiences in the course appear to have had a positive impact on their chemistry self-efficacy beliefs.

From Fig. 1, we can also observe that the differences in expected CSE scores for each group are more noticeable at the beginning of the semester. When examining the CSE trends at the end of one semester of college preparatory chemistry, we can see that the gaps in expected CSE scores between groups have lessened as compared to the beginning of the semester. This finding is consistent with a previous study in college chemistry by Dalgety and Coll (2006), where they observed that most of the CSE differences by sex obtained at the beginning of the semester were no longer noticeable by the end of the academic year. This study takes the comparison a step further by revealing that self-efficacy gaps by racial/ethnic group also decreased after a semester of a college preparatory chemistry course.

In summary, we have seen that, by just focusing on the raw CSE data for all students, an overall increase in chemistry self-efficacy beliefs was observed by the end of the semester. However, the results from this study showed that this positive trajectory might not be the same for all students, since for Black and Hispanic males, a negative trend was observed. Although the trajectories are different, the expected CSE scores for these two groups at the end of the semester were very close to the CSE scores for the other groups. This finding suggests that perhaps, at the beginning of the semester, students (Blacks and Hispanic males) were overconfident about what chemistry-related tasks they were able to perform (Pajares, 1996b; Bandura, 1997; Britner and Pajares, 2001), but they became more realistic by the end of the semester. Bandura (1986) has argued that students need to have a strong belief in what they can accomplish, since this will help them to persist on a task, but, as Britner and Pajares (2001) have argued, “…how much confidence is too much confidence?” This situation opens up the question of the role of a preparatory chemistry course with respect to developing student self-efficacy.

Conclusion and implications

This study examined students' chemistry self-efficacy throughout a semester in a college preparatory chemistry course. The CSE scores were used to examine students' trends by sex and race/ethnicity through multilevel model analysis. The findings of this study were used to answer two research questions regarding students' CSE scores at the beginning of the semester and CSE trajectories across a semester by sex and race/ethnicity.

For the first research question, we examined students' expected CSE scores at the beginning of the semester by sex and race/ethnicity. At the beginning of the semester, the expected CSE beliefs differed by student demographic group. Most importantly, CSE score differences were not observed when males and females were compared in general; however, differences were observed only when taking into race/ethnicity into consideration as well. These findings highlight the value of acknowledging that the experiences of males and females can differ by racial/ethnic group in order to gain a better understanding of the role of self-efficacy in retention. Differences in self-efficacy can have an impact not only on students' persistence in introductory chemistry but also later on, for advanced coursework and progress within a STEM workforce that remains dominated by Whites and males. Continued research into these differences in self-efficacy at all levels is required to uncover the degree to which this factor is at play in preventing full development of the potential STEM workforce.

For the second research question, we examined CSE changes and trends during a semester by sex and race/ethnicity. Differences in students' CSE trends were observed for two underrepresented groups, Blacks and Hispanics. These findings suggest that, since chemistry self-efficacy is influenced by students' experiences related to the tasks presented, different groups of students would have experienced the course and its chemistry-related tasks in different ways. Some students may have had repeated positive experiences in the course, which helped them to build stronger CSE beliefs toward the end of the semester, while other students may have had repeated negative experiences that led to a decrease in their self-efficacy beliefs (Dalgety and Coll, 2006). At the beginning of the semester, when students have only prior experiences on which to base their self-efficacy beliefs, overconfidence could have been a factor for Black and Hispanic males. Being overconfident can be problematic, as students might feel that they do not need to make any effort to do well (Margolis and McCabe, 2003), which, for a challenging subject such as chemistry, can set up a negative trend in self-efficacy as the course progresses.

Chemistry self-efficacy could be a potential factor influencing students' achievement and retention in STEM careers (Bandura, 1986; Britner and Pajares, 2001; Ong et al., 2011). Although this study has not focused on students' achievement or retention in STEM, examining students' CSE in their first college chemistry course is highly relevant to those goals. This study is the first one, at this time, to focus on underrepresented groups in a college preparatory chemistry setting, which serves as a bridge for students to continue into general chemistry, a required course for STEM majors. This particular preparatory course has a higher proportion of underrepresented minority students than do later courses in the chemistry sequence, which makes it an important target for examination of the chemistry self-efficacy beliefs for these students. From this study, we have seen that our expectation that a college preparatory chemistry course should increase students' chemistry self-efficacy (Schmid et al., 2012; Youl et al., 2012) might not be true for all students. Given that there is a continuous interest in retaining students from underrepresented groups in STEM-related fields, it is important for researchers and educators to be aware of potential differences among these students when planning interventions or strategies to help students stay in STEM-related fields.

This study has limitations. First, the sample was drawn from a single semester of a college preparatory chemistry course at a particular institution; therefore, this model may work for this context but not apply to other situations. Second, the sample size for some of our racial/ethnic groups is small, which could limit the generalizability of the results. Third, the findings are based on quantitative data only, which limits the understanding of the differences in chemistry self-efficacy beliefs among the different groups of students. Also, race/ethnicity is a complex term, such that students in the same category may have quite different cultural backgrounds, which makes it difficult to draw conclusions regarding their differences. Triangulation of quantitative data with interviews and observations would be a helpful and important aspect of future studies in this area. Fourth, since self-efficacy is task-specific, topics covered during the semester could have a differential impact on students' self-efficacy beliefs. Further studies could include a comparison of self-efficacy beliefs associated with selected topics, to determine whether particular topics consistently challenge students' sense of self-efficacy.

Despite the limitations presented, this study has important implications for the chemistry education community. For researchers, it is important to replicate this study with additional samples. Developing a greater understanding of variability in self-efficacy trajectories for diverse students will help to enrich our understanding of this construct. Additionally, studies that specifically look at the relationship of chemistry self-efficacy and student performance in STEM courses over time are needed, again with attention to diversity. Some interesting work on comparative perceived ability vs. actual ranked exam performance over time in general chemistry has recently appeared (Pazicni and Bauer, 2014); although the study setting did not permit a full examination of diversity issues, female students were found to predict that their performance would be closer to the average performance than comparable male students.

For those of us in the classroom, it is very important to be aware of potential differences in the confidence that students may have when solving chemistry-related tasks, with respect to the influence of self-efficacy on persistence. Students with strong self-efficacy will be more likely willing to try a challenging task and persist until accomplishing it successfully (Britner and Pajares, 2001; Pajares and Schunk, 2001; Margolis and McCabe, 2003; Zeldin et al., 2008), but the relationship between expected outcomes and self-efficacy also means that repeated failure at challenging tasks can be expected to lower self-efficacy and dampen student willingness to try (Pajares, 1996b). Given the imperative to encourage students to persist in introductory chemistry courses and on to STEM-related careers, educators' decisions about how to structure their courses to meet the needs of diverse learners can be highly influential. For students who begin with a strong chemistry self-efficacy, it may be the case that attention to developing metacognitive abilities, as suggested in a recent study on the study behaviors of ethnically diverse organic chemistry students (Lopez et al., 2013) as well as in the previously mentioned study on perceived vs. actual general chemistry exam performance rank over time (Pazicni and Bauer, 2014) will be a promising direction. For those who begin with a lower self-efficacy, scaffolding tasks from easy to hard has been suggested as one way to provide the necessary support for growth (Margolis and McCabe, 2003). Regardless, because of the influence of students' own experiences on self-efficacy (Britner and Pajares, 2001; Dalgety and Coll, 2006), educators should strive to provide a variety of relevant and interesting tasks that are appropriate for progressive development of diverse students' skills and knowledge.