View PDF VersionPrevious ArticleNext Article
 
DOI: 10.1039/C6RP00101G (Paper) Chem. Educ. Res. Pract., 2016, 17, 878-892

Can they succeed? Exploring at-risk students' study habits in college general chemistry

Li Ye , Constantine Shuniak , Razanne Oueini , Jenay Robert and Scott Lewis *
Department of Chemistry, University of South Florida, Tampa, Florida 33620, USA. E-mail: slewis@usf.edu

Received 3rd May 2016 , Accepted 22nd May 2016

First published on 23rd May 2016

Abstract

A well-established literature base identifies a portion of students enrolled in post-secondary General Chemistry as at-risk of failing the course based on incoming metrics. Learning about the experiences and factors that lead to this higher failure rate is essential toward improving retention in this course. This study examines the relationship between study habits and academic performance for at-risk students in General Chemistry. Students who were in the bottom quartile of SAT math scores were identified as at-risk students. The study habits of General Chemistry students, both those identified as at-risk and those not identified were measured by text message inquiries. The text message asked ‘‘Have you studied for General Chemistry I in the past 48 hours? If so, how did you study?” twice a week throughout a semester. Student responses to the messages were used to calculate the frequency of studying throughout the term. The results from a multiple regression analysis showed that high frequency of studying could mitigate the difference between at-risk and non-at-risk students on final exam scores. Additionally, the quality of studying for six at-risk students was analyzed by student interviews in concert with their text message responses. The results indicated that the quality of studying is not necessarily linked to frequency of studying and both quality and frequency can play a role in at-risk students' academic performance. The results presented offer a path for at-risk students to succeed in General Chemistry and the methodology presented offers a potential avenue for evaluating future efforts to improve student success.

Introduction

Students who perform poorly in courses are a primary concern for education researchers and instructors. High attrition rates of 30% or more in STEM gateway courses during the initial years of college have been reported in different institutions throughout the country (Harris et al., 2004; Benford and Gess-Newsome, 2006; Gabriel, 2008; Griff and Matter, 2008; Gultice et al., 2015). General Chemistry is one of those initial courses that are generally perceived by college students as difficult (Tai et al., 2005). Failure rates of 50% and more in general chemistry courses have been reported by certain institutions (Chambers, 2005; Gafney and Varma-Nelson, 2008). High failure rates in general chemistry delay or prevent students from entering advanced courses in the course sequence since General Chemistry is required for most STEM majors. High failure rates may also cause decreasing confidence and morale for students and increasing costs for students and universities. At many institutions, without demonstrating satisfactory understanding of general chemistry knowledge, students cannot continue pursuit of any STEM fields. To aid in the improvement of student retention in General Chemistry, research literature has identified multiple student characteristics that relate to a higher likelihood of not succeeding in General Chemistry. Students who exhibit these characteristics are referred to as “at-risk”. This investigation seeks to better understand the actions and experiences of at-risk students with the intent that doing so will offer potential paths toward improving student retention in General Chemistry.

Identifying at-risk students

Prior research regarding identification of at-risk students in chemistry has been generally focused on two main student characteristics. A major component of research has been focusing on exploring cognitive characteristics of students, including standardized tests such as the SAT (e.g., Pickering, 1975, Spencer, 1996, Lewis and Lewis, 2007; Cracolice and Busby, 2015) and ACT (Carmichael et al., 1986), prior knowledge (Hailikari and Nevgi, 2010), high school GPAs (Carmichael et al., 1986) and diagnostic tests (Russell, 1994) as predictors to identify at-risk students. The second category of studies considers students' affective characteristics, for example, self-efficacy (House, 1995), attitude (Xu et al., 2013; Cracolice and Busby, 2015) and self-concept (Lewis et al., 2009; Chan and Bauer, 2014). Recent research provides evidence that the cognitive and affective characteristics describe unique factors in understanding chemistry performance (Lewis et al., 2009; Xu et al., 2013). In addition to the two main student characteristics, other predictors such as demographic information (Wagner et al., 2002; Tai et al., 2005), personality characteristics (House, 1995) and student metacognitive skills like self-evaluation (Potgieter et al., 2010) have also been reported in research articles.

Among those predictors, there is a long history of using SAT math score to predict students' academic performance in chemistry courses in the literature. SAT math is a component of the SAT, a standardized college-entrance test commonly administered in secondary school. SAT math is designed to measure quantitative reasoning including problem solving, modeling and algebraic structure (College Board, 2016). SAT math was strongly associated with student academic performance in chemistry; students who have low SAT math scores are more likely to have low academic performance in chemistry courses (e.g., Pickering, 1975; Spencer, 1996; Lewis and Lewis, 2007). The cut-off scores of SAT math used to determine at-risk students in chemistry were varied due to the diverse incoming abilities of students among universities. Lewis and Lewis (2007) examined a range of SAT math cut-offs and found that the bottom 25% to 35% of the sample by SAT math made approximately 65% to 75% correct predictions in describing a student as at-risk. Combined the research to date indicates that students who enter General Chemistry with low SAT math are disproportionately likely to not succeed in the course.

Helping at-risk students

Past research reported utilizing various methods to help at-risk students. Valentine et al. (2011) conducted a meta-analysis of past work on the effects of college retention programs aimed at helping at-risk students in higher education. In the meta-analysis, they defined at-risk students as academically underprepared and economically disadvantaged students; they found inconsistent effect sizes among 18 studies. The average effect size of these retention programs on at-risk students' various academic achievement outcomes (mainly GPA) was 0.07 and the range was from −0.61 to 0.93. The meta-analysis indicated that more comprehensive interventions led to more effective results. For example, an intervention that implemented a seminar designed to assist college adjustment, in conjunction with smaller classes and other activities like tutoring led to more positive effects on attendees (Hecker, 1995) than the smaller scale intervention that added a journal-writing component to an English composition class (Cohen Goodman, 1998).

In chemistry, the main practices that help improve at-risk students' success in chemistry have been reported as offering external remedial coursework (Meckstroth, 1974; Pickering 1975; Walmsley 1977; Bentley and Gellene 2005; Heredia et al., 2012), group activities (Mason and Verdel, 2001) or training programs for at-risk students (Shields et al., 2012; Hall et al., 2014). Remedial courses typically offer at-risk students lectures on preparatory chemistry content concurrently or consecutively with the regular lectures. Pickering (1975) reported providing a supplementary course for students who had SAT math scores of 610 or lower. This course taught students the solutions of diverse types of problems associated with the content students learned in the parallel chemistry lecture course. Results showed that students who attended the supplementary course had mean grades that were 0.29 (on a 4-point scale) higher than the comparable students who did not attend, and the difference was significant. Interestingly, Bentley and Gellene's (2005) study suggested that the effect of a remedial course depended upon students' SAT math scores. They offered multiple sections of an Introductory Chemistry course that aimed to teach vocabulary, concepts and problem solving skills for students scoring below 50% on chemistry placement test (CPT). Results showed that students with low scores on the CPT who took the Introductory Chemistry course finished with a grade in General Chemistry that was ¼ to ½ of a letter grade higher than their counterparts who did not take Introductory Chemistry. However this effect was only found for students with SAT math scores from 460 to 600, little or no effect was found for students below or above this range. The above studies focus on providing more repetition of course content for at-risk students. Similar to the general results suggested by the meta-analysis study, these articles report mixed effectiveness of these practices for at-risk students in chemistry. Even though positive effectiveness has been reported, since the above studies all used student grades as the outcome measures, the effectiveness could be partially attributable to how instructors assigned grades to students.

Other past work describes efforts to improve at-risk students' study skills through training programs or group activities. Hall and colleagues (2014) trained less prepared students who were in the bottom quartile of SAT math scores or lacked advanced placement (AP) courses in math and science by using a project called Science Advancement through Group Engagement (SAGE). SAGE was run concurrently with the regular lectures and implemented study group sessions focused on foundational chemistry knowledge with the aid of teaching assistants. SAGE also trained students with a self-regulated learning (SRL) approach. SRL encourages students to follow a study cycle of task analyses, planning, reflection and self-adjustment based on the value and meaning of their efforts. The results showed that the retention of SAGE participants was more than double that of the non-SAGE participants and historical group in chemistry sequence courses. By the fourth course, Organic Chemistry II, SAGE participants performed as well on final course grades as those students who had stronger high school backgrounds with more AP science and math courses and significantly higher SAT scores. Shields and colleagues (2012) implemented a transition program including extended-length recitations, peer-led team-learning (PLTL) study groups and peer-mentoring groups to help underprepared students who were in the bottom 25% of predicted scores based on ACT math, total of STEM AP test scores and online diagnostic scores. The study found the transition program helped the participants make significant gains in final general chemistry course scores that combined quiz scores, midterm and final exams in comparison to students who were in regular recitations only.

In both Hall et al. (2014) and Shields et al. (2012), at-risk students were trained with certain study skills or strategies and notable academic benefits were observed. However, neither of these investigations incorporated a measure of study habits so it is not possible to make a definitive claim that the interventions employed influenced student study habits. Another plausible explanation might be self-selection bias where participating students possessed higher motivation to succeed than the reference group from the onset of the study. Additionally, it has not been well established that the reason at-risk students struggle in General Chemistry is related to their study habits. That said, the notable benefits observed are cause for further investigation into the relationship between study habits and the academic success for at-risk students.

Effective study habits

Theoretical framework

The theoretical framework used in this investigation for describing the quality of participants' study habits was Sinapuelas and Stacy's Learning Approaches Framework for Chemistry (2015). This framework is built on an extensive body of research that describes learning approaches as surface or deep (Marton and Salijo, 1976, 1984; Biggs, 1987). Surface level learners tend to rely on techniques such as rote memorization of unconnected facts, reading and rereading resources provided by instructors, and relying on others for help when they experience difficulty with homework or other assignments. In contrast, deep level learners utilize techniques such as making connections between pieces of information based on overarching concepts, constructing their own supplemental content such as study guides or practice problems, and working through difficult problems collaboratively with peers. Additionally, surface learners tend to be extrinsically motivated by factors such as grades, whereas deep learners are intrinsically motivated to learn material for the satisfaction of gaining understanding and developing personal knowledge.

Sinapuelas and Stacy (2015) built on this model in an effort to characterize the study approaches of introductory, non-major college chemistry students. In this study, 61 students were interviewed at three time points throughout the semester. In the interview students were asked to describe the resources they used to prepare for exams and to elaborate on how they were used. The analysis of student responses led to the creation of the Learning Approaches for Chemistry framework that describes learning approaches in four hierarchical levels:

Level 1: gathering facts – students tend to memorize unrelated facts by scanning course materials, typically independently. Students do not monitor their own learning.

Level 2: learning procedures – students begin to make connections between pieces of information and try to work out practice problems. Students rely on others for answers, but they possess basic metacognitive skills such as assessing for procedural errors.

Level 3: confirming understanding – students evaluate and question data, form their own arguments, and work collaboratively with peers. Students assess their own knowledge based on their ability to justify and explain answers.

Level 4: applying ideas – students question data, try to use concepts to explain real-world phenomena, and act as “teachers” with their peers. Students possess advanced metacognitive skills such as assessing for gaps in conceptual understanding.

Levels 1 and 2 emphasize memorization, matching the description of surface level approaches; levels 3 and 4 emphasize content generation and application, matching the deep level approaches (Sinapuelas and Stacy 2015). Since this framework provides additional description to the surface-deep dichotomy and describes students' approaches while engaging in studying chemistry, this framework will be used to describe the quality of students' study habits in this work.

The role of study habits

Research studies have demonstrated an association between study habits and academic performance in different settings. Crede and Kuncel (2008) published a meta-analysis study and reported that the average correlations for study habits (measured by various study skill inventories) and college GPA in different disciplines was 0.33 with a range of 0.09 to 0.51. Additionally, the meta-analysis also found the relationships between study habits and cognitive ability measures like college admissions tests (e.g. SAT and ACT) were trivial, indicating study habits are independent of these cognitive ability measures. This result also suggested that it is possible that students can benefit from effective study habits regardless of their incoming ability.

Specific to General Chemistry, Chan and Bauer (2016) divided students into high, medium and low affective groups using cluster analysis on the results of a survey measuring attitude, self-concept and motivation in chemistry. Surveys, open-ended questions and interviews were used to investigate students' study strategies used in the lecture and when preparing for exams. Students in the high group reported understanding the notes they took in the lecture more frequently than the low group, and the low group relied more on others for help when preparing for exams, analogous to the surface level learning description in Sinapuelas and Stacy’ article (2015). In addition, answers to the open-ended questions showed that the high group tended to be more confident about their study strategies while students in the low group felt less confident about their strategies and planned on changing their current study strategies, suggesting that confidence and studying strategies are related constructs.

Ye et al. (2015a) examined students' study habits of General Chemistry students outside the class via inquires sent through text messages. Students were characterized based on the types and frequencies of studying reported in their text message responses. Using cluster analysis, three patterns of studying emerged: students who knowingly do not study (Cluster 1), students who study in addition to the mandatory course components such as reading the textbook or practicing problems (Cluster 2) and students who primarily describe mandatory course components such as doing homework assignments as studying (Cluster 3). These three groups were compared on the measures of final exam and revised two-factor Study Process Questionnaire (rSPQ) (Biggs et al., 2001), an instrument used to measure students' study process with two sub-scales of deep and surface approaches. The results of ANOVAs showed that students in Cluster 2 were significantly higher on the final exam than the other two clusters. Students in Cluster 1 were significantly higher on the surface approach than the other two clusters, and Cluster 2 was significantly higher on the deep approach compared to Cluster 1. These results indicate that frequency of studying relates to academic performance in General Chemistry though the sample was not delineated for at-risk students. In reviewing the literature, no research exploring the role of at-risk students' study habits in the context of post-secondary chemistry was identified.

Purpose of the study

The broad intent of this study is to better understand the frequency and quality of studying (herein referred to as study habits) of at-risk students as the habits relate to academic performance in General Chemistry. Improving our understanding of the role of study habits potentially offers a primary path toward improving student success in General Chemistry. Past research has shown that the quality and frequency of studying plays a role in student success when examined for a representative sample of students in General Chemistry (Ye et al., 2015b; Sinapuelas and Stacy, 2015). However, it is not known whether these relationships hold true for at-risk students, thus it is not known whether the study habits of at-risk students can explain the lower success observed for at-risk students; that is, whether at-risk students have appreciably different study habits than the rest of the population and whether this difference is responsible for the observed lower success rates. Accordingly, this research study has the goal to explore the study habits of at-risk students and their relationship to student success as well as to initiate an exploration into the characteristics of at-risk students who succeed well beyond their predicted performance. This research is guided by the following research questions:

1. What is the relationship between study habits (frequency) and academic performance in college General Chemistry for at-risk students as compared to the larger remaining General Chemistry cohort?

2. What are the effective and non-effective study habits (frequency and quality) of at-risk students in college General Chemistry and what additional factors may explain the study habits employed?

Methods

Research setting

This study was conducted at a large public research university in the southeastern United States. At the setting, there were four General Chemistry I classes. The class size of each class was between 200 and 240 students. The classes were coordinated, using a common textbook, syllabus and grading scheme. All classes used an online homework system and gave common tests at the same time. Students attended the regular lectures twice a week and problem-solving peer leading sessions (Gosser et al., 2001; Lewis, 2011) once a week. The textbook used for the classes was “Chemistry: A Molecular Approach” (Tro, 2014). Eight online homework assignments were assigned throughout the semester using the Sapling Learning online homework system. Tests consisted of three in-class midterm tests (15% each of their grades) and a cumulative final test (25%). The format of the tests consisted of multiple-choice questions developed by the instructors and a series of true/false questions. These true/false questions called Measure of Linked Concepts (MLCs),were developed by the researchers in this study. MLCs are an instrument used to promote students making connections within the course. For an MLC, students were provided a prompt that describes a chemistry situation such as “an aqueous solution of 0.1 M NH4Cl”. Students were then asked to determine the validity of a series of statements related to the prompt. The statements were deliberately planned to cover both recent content coverage and previous content in the course to emphasize the links across topics in the course (Ye et al., 2015a). Past tests were posted before each test for review purposes through an online course management system. In addition to the four tests, grades were also determined by three effort-based measures (10% each) including performance on in-class clicker questions used in the regular lecture setting, attendance and participation in peer leading sessions and the online homework assignments.

Data collection

The study utilized Experience Sampling Methodology (ESM), a research method that asks participants to self-report their actions or psychological state in their natural environment at certain times (Hektner et al., 2007). In ESM participants' experiences in the moment are recorded at multiple instances with the aid of technology. ESM has been described as systemic phenomenography as it focuses on participants' self-report of a construct, which can be either psychological states or actions that a participant observes or participates in. The strength of ESM is the systemic collection regarding a construct, which facilitates the exploration of patterns over time and can aid in establishing reliability of the responses. By measuring multiple times, ESM also has the advantage that it can rely on a much more proximal retrospective recollection as opposed to a singular measurement in the same time frame. In this study, ESM was used to assess students' study habits outside the General Chemistry I classroom via text messages.

The data collection spanned two semesters. First, in the spring semester, students were recruited from three of the four General Chemistry I classes on the first day of class. In the recruitment, the nature of the study was described to students. The participants would be asked to provide their cell phone numbers and would twice weekly receive a text message that asked the same question: “Have you studied for General Chemistry I in the past 48 hours? If so, how did you study?” The text messages would be sent at random times between 9 AM and 9 PM. Participants would be asked to reply to the message within 12 hours if possible. To encourage participation, students who replied to at least 80% of the text message inquires would be entered into a raffle for a $25 gift card at the end of the semester. The recruitment led to 301 students agreeing to participate in the study. The text message inquiry was sent out 28 times over the course of the semester. The text message responses from participants were collected and managed via a commercial online website. Student performance such as test scores, course grades, attendance and homework completion, along with demographics and SAT scores were collected from either university records or in-class records. Clickers were used for each class to record student attendance in the setting.

During the following fall semester, 28 at-risk students who replied to at least one quarter of the 28 text message inquires and were currently enrolled in General Chemistry II were invited via E-mail for a follow-up interview. Students who volunteered would be compensated with a $20 gift card. Six students volunteered and each was interviewed individually. The interviews covered three major themes: students backgrounds, e.g., major and prior chemistry coursework; elaboration on the study approaches reported through text messages, such as how the textbook was used; and questions that were related to students' approaches to learning, e.g., working with others, metacognition and affective factors. A complete list of interview questions can be found in the appendix. The interviews adopted a semi-structured approach. The lengths of the interviews ranged from 20 to 40 minutes. All data collection was carried out with the approval of the university's Institutional Review Board (IRB).

Procedures of identifying at-risk students

SAT math scores were used to determine at-risk students in this study based on established literature. Students were divided into two groups based on their SAT math score: those who were in the bottom quartile (SAT math < 515) in the sample were considered as at-risk students. The remaining cohort with higher SAT math will be referred to as non-at-risk students. To validate whether the method we used to classify at-risk students and non-at-risk students was appropriate, we compared the differences in academic performance in the course between the two groups. Table 1 lists the descriptive statistics for at-risk and non-at-risk students in the General Chemistry I course. Missing data in Table 1 was removed list-wise. As listed in Table 1, the at-risk students' average score on each test was lower than non-at-risk students with differences ranging from 6% to 12%. Class attendance and homework completion were comparable for the two groups. In addition, the average course GPA for at-risk and non-at-risk students were 2.41 and 2.78 respectively. The difference in GPA of 0.37 represents just over a partial letter course grade (0.33), such as the difference between C+ and B−. Analysis of course grade distributions show that at-risk students had 9% and 16% lower percentages of students earning grades of A and B, respectively. At-risk students also had a 20% higher percentage earning a grade of C. The course failure rates, which includes students who received a C− or lower in the course and therefore did not meet the minimum requirement to enter the next course in the sequence, was 10.5% for the at-risk students and was 5.8% for the non-at-risk students.
Table 1 Descriptive statistics of at-risk and non-at-risk students
Variables Non-at-risk At-risk Cohen's da
a Cohen's d = 0.2 (small), 0.5 (medium) and 0.8 (large).
N 384 153
SAT math (mean ± SD) 583 ± 52 476 ± 39 2.32
Test 1 (%) 71.2 ± 14.0 63.7 ± 14.1 0.53
Test 2 (%) 69.9 ± 15.9 63.1 ± 14.9 0.44
Test 3 (%) 52.8 ± 15.2 41.2 ± 14.1 0.79
Final test (%) 51.2 ± 15.2 44.8 ± 16.3 0.41
Attendance (%) 82.3 ± 19.4 81.0 ± 20.3 0.07
HW (%) 94.2 ± 14.9 95.8 ± 11.8 0.12
Course GPA 2.78 ± 0.76 2.41 ± 0.75 0.49

To determine whether the group differences were significant, MANOVA analysis and univariate follow-up tests were conducted on the variables of each test, attendance, homework and course GPA listed in Table 1. Results of the MANOVA showed the group difference in means on the set of outcome variables was statistically significant with α = 0.05, F(7,529) = 10.243, p < 0.001, Λ = 0.881, which means the proportion of variance in the combination of outcome variables that was accounted for by the grouping variable was 12%. The size of the multivariate effect was estimated to be medium (ω2c = 0.10). The results of univariate follow-up tests revealed statistically significant group differences for each test and course GPA but not attendance and homework completion. Effect size measured by Cohen's d for comparisons on each individual variable are also listed in Table 1.

In sum, at-risk students performed worse on each single test and final course grade than non-at-risk student in the General Chemistry I course, but the effort measures such as attendance and homework completion were comparable. At-risk students were displaying as much effort as non-at-risk students but achieving less on the tests. These results support the method of identifying at-risk students based on SAT math as appropriate.

Data analysis

Text messages were coded using an open-coding scheme. The open-coding scheme resulted in 16 categorical codes, representing the types of the study habits employed by students, for example, “Reviewed notes” or “Practiced problems”. Each text message response could receive multiple codes if multiple study habits were described. Each text response was also coded using dichotomous codes as either a study habit was used or not. With the dichotomous codes, the study percent outside the course was calculated for each student by the number of times the student reported studying divided by the number of responses. Ambiguous codes, for example “attended class,” were coded as missing as it didn't fit the definition of study habits in this study. Also, the code “do homework” was coded as missing because homework assignment were mandatory and the data suggests that nearly all students (over 90%) regularly completed the homework regardless of whether they reported it in their responses. In the subsequent interviews, students' discussions of the ambiguous codes study habits, for example, how they were involved in the peer leading sessions and how they performed the homework were retained as it provides relevant details pertaining to the quality of their study habits.

Course grades were converted into 4-point scale numbers for computing averages. For each student, percentage of attendance was calculated as the number of days the student recorded a clicker response divided by the maximum number of days the student could record a response in the semester. Homework completion in the course was measured using percentages of completion of homework. Instead of homework grades, percentages of the homework completion were used to measure student effort. In order to make the scales of the four outcome variables consistent, test scores were transferred into percentages.

To examine the relationship between study habits and academic performance in college General Chemistry for at-risk students as compared to the non-at-risk students, scatter plots showing relationship between study percent and final exam score for the two groups were constructed. To determine statistical significance, a multiple regression was conducted where SAT math score, study percent, and the interaction between study percent and SAT math score were used to predict students' final test scores. The reason for using final test scores in the regression model is that the study percentages represent studying across the entire term and the final exam was the only cumulative test.

The six interviews were transcribed verbatim using an open coding method. First, four chemical education researchers coded the transcripts independently; each person was assigned to code one to two distinct transcripts to describe all the themes present. The separate themes identified were compiled and the researchers discussed the similarities and differences among their themes to create a unified code list. Finally, two of the researchers coded the six transcripts independently based on the unified code list using NVivo 11.1.1 software. Upon completion of coding, disagreements between codes were discussed until consensus was reached.

Results

Descriptive statistics

To describe the relationship between study habits and academic performance for at-risk and non-at-risk, students who replied to the text message inquiries were considered. There were 122 students who replied to at least one quarter of the 28 text message inquires and these students were selected for the analysis as these students replied to a sufficient number of text messages to provide a picture of their studying habits. Using the aforementioned SAT math cut-off, 28 of these students were classified as at-risk and 94 were non-at-risk students. Descriptive statistics for these two groups are listed in Table 2.
Table 2 Descriptive statistics of at-risk and non-at-risk students in the selected sample
Variables Non-at-risk At-risk Cohen's da
a Cohen's d = 0.2 (small), 0.5 (medium) and 0.8 (large).
N 94 28
SAT math (mean ± SD) 586 ± 51 483 ± 28 2.50
Test 1 (%) 69.9 ± 15.1 62.4 ± 13.4 0.53
Test 2 (%) 72.0 ± 14.6 66.2 ± 11.2 0.45
Test 3 (%) 53.2 ± 15.3 42.0 ± 11.9 0.82
Final test (%) 49.1 ± 13.5 45.7 ± 15.8 0.23
Attendance (%) 83.1 ± 17.0 81.6 ± 21.0 0.08
HW (%) 94.0 ± 12.4 97.3 ± 6.2 0.34
Course GPA 2.76 ± 0.72 2.50 ± 0.61 0.39
Study percent (%) 46.6 ± 25.4 61.6 ± 29.3 0.55

First, noting that comparison between Tables 1 and 2 on the same variables shows that the selected sample and the broader population are very similar, supporting the ability of the sample to represent the population at least on the variables of interest. Second, the average study percent outside the class for at-risk students and non-at-risk students were 61.6% and 46.6%, respectively. It is interesting that the study percent for at-risk students was 15% higher than non-at-risk students, which means at-risk students reported studying more frequently outside the class than non-at-risk students in our setting.

For each student, the percent of text responses that used a particular study habit was calculated and the average for each study habit for the non-at-risk and at-risk students are presented in Table 3. For brevity, only the study habit codes that represent at least 5% of the text responses are shown. The data in Table 3 suggests that the study habits employed by the at-risk students did not differ from the non-at-risk students in terms of relative frequency; however, the at-risk students did employ each study habit at a higher rate.

Table 3 Common study habits by percent of text messages
Study habit Non-at-risk students (%) At-risk students (%)
Reviewed notes or PowerPoint 18.3 21.8
Reviewed the textbook 15.4 19.8
Online homework 13.1 18.0
Practiced problems 6.6 10.0
Previous exams or study guides 5.8 9.0

Study habits predicting academic performance

To describe the relationship between study habits and academic performance for at-risk and non-at-risk students, scatter plots (Fig. 1) and a linear regression were examined for each group separately. The scatter plots and regression analysis show little relationship between study percent and final exam score for non-at-risk students (R2 = 0.004), in contrast to a moderate relationship between study percent and final exam score for at-risk students (R2 = 0.291). This differential relationship indicates that the relationship between study percent and final exam score might be modified by SAT math.
image file: c6rp00101g-f1.tif
Fig. 1 Scatter plots showing correlation between the study percent and final exam for non-at-risk and at-risk students.

To further examine the differential relationship, a multiple regression model was run using SAT math, study percent, and the interaction between SAT math and study percent to predict final exam score. The interaction term was added to model the differential relationship. The multiple regression model is presented in Table 4 and suggests the linear best-fit equation of:

Final test = −0.339 + (0.00137 × SAT math) + (0.939 × Study percent) + (−0.00150 × SAT math × Study percent)

Table 4 Multiple regression model
Variables b Std. error Beta p-Values
Constant −0.339 0.243 0.166
SAT math 1.37 × 10−3 4.28 × 10−4 0.617 0.002
Study percent 0.939 0.383 1.798 0.016
Study × SAT math −1.50 × 10−3 6.89 × 10−4 −1.568 0.032

The prediction model statistically significantly predicts students' final test, F(3,118) = 5.39, p = 0.002, R2 = 0.121, with a medium effect size f2 = 0.14 (Cohen, 1988). All terms were significant (p < 0.05) except for the constant (Table 4).

The results indicate that both study percent and SAT math score are positively associated with final exam score. The interaction effect between study percent and SAT math is negative and significant, indicating that the differential relationship observed earlier is unlikely to be attributed to chance. The effect of study percent on final test score depends on students' incoming SAT math scores; in short, a high rate of studying can mitigate the impact of low incoming SAT math scores. To confirm that the results were also applicable for other students, the multiple regression model was also conducted on the participants who replied to the text messages at least once, with the same trend in results observed.

Fig. 2 shows a diagram plotted based on the regression equation. The lines represent the relationship between study percent and predicted final test scores when students have different SAT math scores using 50-point iterations in the range of 500 to 650 (representing the 15th to 93rd percentile in the sample). In general, higher SAT math score leads to a higher score on the final test. However, the differences caused by SAT math scores in performance on predicted final test scores for students change dramatically by frequency of studying for students. For at-risk students (math SAT < 515) the frequency of studying outside the class played a more important role in predicting final test scores than those with higher SAT math.


image file: c6rp00101g-f2.tif
Fig. 2 Study habits predicting final exam scores.

The differential relationship of frequency of studying with academic performance for different SAT math levels merits further study. One possible explanation is that students with different SAT math respond to earlier assessments in a different manner. For example, students with higher SAT math who perform well on the assessments throughout the term study at a relatively low frequency and continue to perform well on the final exam, possibly a result of seeing similar content in secondary school. However, when students of higher SAT math are not performing to their satisfaction, they respond by studying at a very high frequency. Students with lower SAT math may have an opposite relationship. Students with lower SAT math who perform well on early assessments may respond to the positive feedback by continuing to study at a high rate. However, if lower SAT students perform below their expectations, they may be discouraged and study less frequently as they do not expect to see a payoff from their efforts. This proposed explanation for the differential relationship essentially uses incoming SAT math scores and early academic performance as a proxy for students' self-efficacy. The role of self-efficacy in terms of study habits for at-risk students will be explored in the second research question.

Exploring at-risk students' study habits through interviews

Table 5 lists demographic information, SAT math, final test score, predicted final test score (from the above regression), final course grade from General Chemistry I and study approaches of the six interviewees. Among the interviewees, four groups of at-risk students were found based on their study habits. The sections below include: a thematic description of each group based on text message responses and the open coding of their interview transcripts as described previously. Pseudonyms are used to protect the identities of interviewees.
Table 5 (a) Information of the six interviewees. (b) Information of the six interviewees
(a)
Name Jack Ellie Mary Bella Lucy Ava
a Numbers listed in the parentheses mean the number of times the study approach was mentioned in the text message responses. b “i” means the study approach only was mentioned in the interview.
Gender Male Female Female Female Female Female
Race White Asian White Hispanic Hispanic Black
Major Environ. Science Physical Therapy Pre-Medical Sciences Biomedical Sciences Biomedical Sciences Biomedical Sciences
SAT math 510 460 430 510 490 510
Actual final test (%) 29 26 64 58 38 46
Predicted final test (%) 42 30 49 52 51 41
Final grade General Chemistry I C C A− B C+ B−
(b)
Name # Of text responses Study approaches (frequencya)
Jack 21 Homework (4), textbook (3), notes (2) and study in groups (ib)
Ellie 28 Homework (4), practice tests (1) and study in groups (1)
Mary 24 Notes (11), textbook (9), practice tests (6), study in groups (6), flash cards (3), practice problems (2), visit instructor (1)
Bella 25 Homework (14), textbook (5), notes (2), practice tests (2), practice problems (1), study in groups (i)
Lucy 27 Notes (18), homework (11), textbook (2), practice tests (1), online videos (1), study in groups (i) and visit instructor (i)
Ava 28 Homework (4), notes (4), textbook (3), practice tests (1), online videos (1), study in groups (i) and flash cards (i)

Jack & Ellie

Jack and Ellie were the two interviewees that knowingly did not study regularly according to their responses to the text message inquiries and interviews. Jack replied to 21 out of 28 text message inquiries of which ten of them reported not studying, while Ellie replied to all 28 text message inquiries and indicated not studying 23 times. In the text message responses, both Jack and Ellie reported limited types of study approaches, and they mostly reported using single study approaches and occasionally combined two study approaches. The interview data align with these remarks. Except for the pattern of not studying, the common theme for Jack and Ellie in the interviews was using study approaches at the surface level when they were asked to articulate their study habits.

Study approaches

Although Jack reported using the textbook to study three times via text message response, the way Jack used the textbook was cursory with little evidence of seeking meaning: “I didn't do much with the textbook honestly… I would just like, skimmed it” and “Going back in the book actually I don't even know… like at the end of chapters how they'll have like example questions… I never did those.” Jack described notes as a means to facilitate memorization, in line with a surface level approach “I just did the basics…reading over the notes, copying some of the notes just to help get a better memory of it… that's really all I did.” Jack did his homework with peers as it was easier to get help, if there were any hard questions, he could “just ask for help”, and he just wanted to “get the answer and move on”. He also relied on friends when he encountered unclear concepts; if his friends couldn't help he showed no signs of attempting problems on his own: “I would go to friends, ask for their help… a lot of the times they would do that for me kind of thing… If they didn't know I would honestly just skip over it.”

Ellie did all the homework assignments and tried to solve problems on the practice tests, but she spent a considerable amount of time stuck on problems:

“I go through the homework, I do the homework problems again… and then a test review I usually look at that and try to solve each problem... and then if I don't really get one problem, I sit there for like an hour, and I'm trying to like figure it out and I finally get it after five hours.” (Ellie)

Like Jack, Ellie also reported relying on friends in her studying: “In college, I don't really know anything on chemistry so I depend a lot on my peers” and “in General Chemistry, I had my friend like be there every step of the way and help me.” Ellie's feature of relying on others happened in different studying scenarios, for example, in the peer leading session:

“In the peer leading everyone is inputting their own ways of how to do it… but I like to get the right answer from the main person [peer leader], know I'm learning it right, rather than trying to figure it out and do it wrong and then I learn it wrong and remember it wrong.” (Ellie)

When she was not sure about a concept in her studying, she also still ended up seeking help from others: “I either like sat there and cried or I would go try to like find it online and see if they could explain it. And if I still didn't get it and I went to like one of my peers and I was like, ‘Hey, explain this to me’ and then after like a while I finally got it.” One anecdote in Ellie's interview further evidenced a surface approach to studying. When asked to nominate the most interesting things she learned in General Chemistry I, she responded: “The most interesting thing I learned… I don't know… it's bad but I don't really remember like what it was in General Chemistry, I learned.”

Metacognition

Jack and Ellie showed little reflection on their own study approaches. Jack thought homework and studying in groups were helpful and Ellie believed all the study approaches she used were helpful. When asked if he had plans to change his study approaches, Jack said he didn't have plans to change his study approaches: “I feel like my habits will probably like be the same… I guess like really the effort is like the main thing that was kind of my problem last semester.” Ellie asserted that she would do more independent work: “I needed to probably do more independent work because I'm kind of depending on peers now to help me.” For the text messages both showed contradictions in their reflection. Jack asserted the messages “reminded me to study and gave me more of a motivation to get it done” despite frequently indicating not studying; Ellie said text messages reminded her to study but added that the messages did not influence her studying: “It [the text message inquiries] didn't really influence me, it kind of reminded me I need to study”.

Affective factors

Both Jack and Ellie expressed low self-efficacy in chemistry. In the interview, Jack said: “I'm not that good in chemistry so my confidence for that just in general, is pretty low.” Ellie described herself as a slower learner in learning chemistry:

“I mean the concept of molality and molarity, I know this is kind of sad and embarrassing for me but it took me about a week to be able to distinguish the two. I'm a slow learner… I'm not confident because it'll take me forever to learn one topic.” (Ellie)

Interestingly, Ellie mentioned that she was more confident in the General Chemistry II course, a difference she attributes to having a different instructor. In terms of interest, both of them stated that were not interested in chemistry, “Like I'm a science fan. But chemistry, not so much” (Jack) and “not so much [interest] in chemistry.” (Ellie)

In summary, Jack and Ellie both describe surface approaches and low self-efficacy regarding chemistry, exhibited by the belief that they could not solve problems on their own, and a reliance on others as a coping strategy. In the Learning Approaches Framework for Chemistry, both Jack and Ellie provide indications of the first two levels by seeking to memorize facts and rely on others for answers. This could also explain the infrequent studying exhibited throughout the semester. As seen in Table 1, both Jack and Ellie finished with a C, the lowest possible grade available given the selection criteria of enrollment in the follow-on course.

Mary & Bella

Mary and Bella reported studying regularly according to their responses to the text message inquiries and interviews. Mary replied to 24 text message inquiries with 20 of them reporting studying and Bella replied to 25 text message inquiries with 24 of them reporting studying. The study approaches reported by Mary and Bella were much more diverse than those reported by Jack and Ellie. Bella tended to use one single approach at a time when studying, while Mary used multiple approaches. Here is an example of Mary's text message response: “Yes. Reviewed old test, practice problems, flash cards, read the book and reviewed notes”. Mary and Bella not only studied consistently, but also showed signs of a deep level of study approaches when they described their study approaches during the interview.

Study approaches

Mary used the textbook to clarify concepts that were not clear for her, along with example questions and practice problems in the textbook. “I like to use it [the textbook] to review the concepts in cases where her [the instructor's] notes weren't helpful, [I wanted to] see if I can get it explained a little bit better” She said she would “have them [problems in the textbook] totally worked out” and “repeated those by myself and compared”. For the homework, Mary would study beforehand and then do it as a quiz, and she used her notes to help when she got stuck. She always saved the last homework assignment for the days before the test and practiced problems on the homework assignment.

“The homework, is kind of like my self-quiz… so I study a little bit and then do that [homework], so if I can do on my own, and tell myself I am doing good, and then would I need help, I have my notes for it, that is the concept I start reviewing.”

She marked things she didn't understand when studying and brought them to the instructor in the office hour. “I didn't go every week, but during test week, I make an appointment to go there at least once, sit with her, review things that I circle or mark that I don't understand.”

In the interview, Bella didn't articulate many details about how she used each study approach, but her answers to some of the questions projected that she adopted deep level approaches. For example, she said that when she was unclear about concepts, she searched online or went to ask peers, but she would not rely on them, in contrast to Jack and Ellie. “When I don't understand a problem obviously I go to my peers but I don't rely just on them and just study with them. Like I'll study maybe twice with them and then as the exam approaches closer I'll just focus on studying alone.” In addition, like Mary, Bella asked her instructor for help after making her own attempts. “I would go to her office hours during that week [the week of exams]. And then I would take my practice exam with me and then ask whatever I have problems with”.

Another theme in common for Mary and Bella is both of them liked to help others when studying in groups, and they thought helping others could help themselves learn better. When describing the interaction with others in a group, Mary likes tutoring others because it helps her too: “I prefer to study in group, …I end up knowing more so I tutor, I am like tutoring everyone, it helps me cause I teach it I know it.” Bella described a mutually beneficial relationship between her and her peers. “If you have a problem, you can go to your peers… They can explain it to you, and they have a problem, you can help them, also when you are helping people, you are kind of learning yourself.” In summary, both Mary and Bella provided a description that matches Level 3 where students develop their own understanding and use interactions with others primarily to confirm their understanding.

Metacognition

Mary and Bella showed much more reflection on their study approaches compared to Jack and Ellie. Mary thought studying her notes, practicing problems and taking practice tests were helpful for her to study. One study habit that she particularly liked was tutoring others about chemistry concepts, she believed that helping others could help her learning and had a substantial influence on her study. “It is mostly me helping other people, [if my friends] need a concept just clear or something [I can help them], and that is huge for me, cause I teach, if you can do that, you got it, you would be fine… So that is a huge influence on my study habit, so I love it.” Bella thought lecture notes, peer leading sessions and group work were helpful. Bella expressed that she is studying more consistently in the General Chemistry II course instead of cramming: “[This semester] I am studying more ahead of time, and learning from last semester [that I should] not just leave it for the last week.” This reflection is particularly telling given Bella's consistent studying reported by the text messages in General Chemistry I. Either Bella planned to increase her frequency of studying beyond her consistent approach earlier, or her reflection of her studying consistency in General Chemistry I was not accurate. Bella also indicated she would keep most of her study approaches except for using the textbook more and trying to read through the whole chapter in the textbook.

For the text messages, both Mary and Bella thought they were reminders of studying. Mary kept track of her text message responses and she used them to reflect on her study habits. “I found it [the text message project] was helpful… try to improve myself by writing things down, like what I would do differently, you really see what you are doing instead of just doing it.” Bella also expressed that text messages helped her to study more. “It [the text message project] did help me a lot and like doing a lot of stuff after class and reading the notes. So it did remind me to do all that kind of stuff.”

Affective factors

Mary and Bella both showed higher self-efficacy with General Chemistry I and expressed interest in chemistry as a subject or certain topics in chemistry. “I used to hate chemistry and took General Chemistry 1 honestly and then I loved it all of a sudden, just worked out… I would say I really like molecular geometry… it made me fall in love with chemistry” (Mary), and “I'm not like overly confident, and I'm not like I'm scared or like nervous about it that much… I really like doing Lewis structures, I thought it was fun” (Bella).

Lucy

Lucy reported studying regularly, as she replied to 27 text message inquiries with 24 times reported that she was studying. However, the interview revealed that Lucy adopted a surface level study approach.

Study approaches

Lucy mentioned diverse study approaches in her studying, but for most of them, she used them in a surface level when asked to explain them. For example, in the interview, Lucy described herself as “not a textbook person” and “there was so much information and then there were things worded in ways that sometimes I'm like I don't understand what they're trying to say.” For the notes, she asserted that she would “read over them and in those notes there was practice problems and I always highlighted stuff.” For homework and practice tests, she just mentioned that she used these for practicing problems without further descriptions. Lucy did not like to interact with peers outside the class, because she mentioned that “most interaction I have is when we are working on like on clicker problems.” Like Jack and Ellie, Lucy described her role in groups as relying on others for building knowledge, as she describes group work: “I think that you get more knowledge on things that we are doing, you get help if you don't understand something.” Lucy described her study approaches mainly as gathering facts and relying on others, providing evidence of Level 1 and Level 2 learning approaches in the framework.

Metacognition

Lucy reflected on her study approaches and showed moderate metacognitive skills over her studying. Regarding plans to change, she said she actually has taken a different approach on learning General Chemistry II, do less cramming and study more consistently: “I guess that I am taking it step by step… last semester I would cram some information, now every day I review my notes and review the Power Points and try to do at least five practice questions before my exam.” She also said she would use more outside resources like tutoring and help from friends. Lucy described the text messages as motivation for her to study: “whenever I got the message or the text, I was like, oh snap, I need to study it, and then that would like motivate me to study”.

Affective factors

Lucy described herself as having average confidence in learning chemistry, but she has become more confident in the General Chemistry II course because of studying. Interviewer: “How confident are you in chemistry, in general?” Lucy: “On a scale of 1 to 10, I'd say a 6… maybe because I'm studying more than I was last semester.” She said she never reads about science or chemistry beyond what is covered in the course even though she thinks the intermolecular force topic in chemistry is interesting to her when asked to nominate one. In contrast to Jack and Ellie, Lucy's confidence in this regard can be described as more malleable and may represent sufficient self-efficacy to study even in light of her reported challenges in approaching the content.

Ava

Ava was an interviewee who didn't study regularly and used cramming to study chemistry. She replied to all 28 text message inquiries with 19 of them describing not studying, and the 9 text messages that described studying happened leading up to the test dates. However, when Ava articulated her study approaches in the interview, she employed deep level study approaches even though her study frequency was on the low end.

Study approaches

Ava expressed that she read the textbook in depth, especially in the week of the test. She liked to make use of the end of chapter summaries, and occasionally she would do example problems in the textbook:

“I felt like it explained everything and then even in the end of the chapter it gave short block summaries of different concepts or whatever, so after you read them in depth, even if you didn't understand it in depth, you could flip to the back of the chapter and have like little sections like that, that helped a lot and especially I happened to be in a situation where I was cramming, I could always flip to those sections, so I really relied on the textbook.” (Ava)

Ava also described an active approach for lectures: “I print out the lectures ahead of time and I write my notes on the actual slides and then I can like actually point out what's important”. She described the interaction between herself and others as two types. First, in working with her peer leader, she shows self-efficacy in putting forth her own understanding: “I would ask my TA during the peer meeting, or if I didn't understand something I would go to him and be like… ok, well, if I do it this way, am I right or wrong, and that would help me.” Second, when working with her friends, she could help others and others could also help her. “So it's like I know A, you know B, and now we can put it together. In the situation now I did really well on the first test, so it's just like ok I can help somebody else”. In terms of approaches to learning, Ava shows signs of confirming understanding, analogous to Level 3 in the learning approaches framework, even with her infrequent reports of studying.

Metacognition

Ava reflected on her study approaches and showed indications of metacognitive skills. She thought flashcards and group work were helpful for her. She has also planned to change her approach in General Chemistry II: “I didn't try to put together as the full picture until I got to General Chemistry II, and I realized that would probably help more trying to put the little pieces together to a big picture.” For the text messages, she said that they were good reminders and also a cause for change: “I would look at all my past text messages and see that I replied ‘no’ seven or eight times and I'm just like maybe I should open my textbook and at least look at something.”

Affective factors

Ava described not being confident before General Chemistry I because she felt her prior knowledge in chemistry was not strong, but felt more confident after General Chemistry I as she has learned more knowledge in chemistry. “I feel pretty confident now [in General Chemistry II], before coming into college, especially right before my General Chemistry I, I was really hesitant on it because my chemistry background wasn't that great.” Ava expressed interest in chemistry, she likes to read articles related to chemistry concepts, search chemistry concepts online and find related articles she was interested in until she understands the concepts.

Discussion: cross-case analysis

Analysis across the six interviews combined with the text message responses of the six interviewees showed that both the frequency and quality of studying matter for at-risk students' academic performance in college General Chemistry. The quality and frequency of studying for the six interviewees are represented in Fig. 3. In fact, for these six students, different mixtures of these two features of study habits direct different academic performance. First, the two students who meet both criteria (high frequency and high quality) earned the highest grades of the six cases (see Table 5a). Both Mary and Bella studied more regularly over the General Chemistry course, and they also employed deep level study approaches when studying chemistry. It is likely that these study habits helped them earn good grades in the course despite their at-risk status. Mary, in particular, entered the course with the lowest SAT math of the six cases (and second lowest among the sample of 122 students) and finished the course with a grade of A−. Second, when neither high frequency nor high quality of studying is present, at-risk students would likely perform poorly. This description would match the cases of Jack and Ellie in the study. Both of them knowingly did not study regularly and described surface level study approaches; these study habits led to the lowest possible grade in this cohort.
image file: c6rp00101g-f3.tif
Fig. 3 Distribution of the quality and frequency of studying of six interviewees.

Third, when only one of the criteria is met, the quality of studying may matter more than the frequency of studying. Ava, who studied less frequently but adopted deep level study approaches performed better on her final course grade than Lucy, who studied more frequently but used surface level study approaches. Similarly, Ava over-performed her predicted final test score while Lucy under-performed her predicted score. According to the data analysis of the four groups in our study, we proposed a hypothetical model that both the frequency and quality of the study habits can be closely related to at-risk students' academic performance, and the quality of study habits might be more important than the frequency of the study habits. However, this model has to be tested by a bigger, more diverse sample in order to propose a generalizable claim. Determining the relative importance of quality versus frequency remains an open question that will be important to better understand how to assist at-risk students.

The interviews were consistent in that examples of good metacognitive skills and positive affective factors coincided with higher quality studying. For example, students' self-monitoring of their study approaches was only present for students who indicated deeper level studying approaches: Mary recognized that tutoring others helped her own learning and Ava realized that she needed to make connections among chemistry topics and not just memorize them as separate facts. Mary and Ava also kept track of their text message responses and utilized them to help promote their studying. In addition, high achieving students Mary and Bella showed more self-efficacy in learning chemistry and more interest in chemistry than low achievement students Jack and Ellie. Interestingly, both Lucy and Ava mentioned that they were not confident in General Chemistry I, but after General Chemistry I, the level of confidence increased because of studying. It seems likely that self-efficacy and frequency of studying are interrelated, where increases in one can beget increases in the other. In sum, students' metacognitive skills and affective factors such as self-efficacy and interest in chemistry can help us understand why and how at-risk students can succeed in chemistry. These traits are closely related to students' study habits (frequency and quality), which impacts their academic performance.

Limitations

The number and representativeness of the students in this sample serve as a limitation to the results presented. The multiple regression analysis was conducted on 122 students, of which only 28 were considered at-risk based on SAT math. The sample size is sufficient to provide statistical significance and the sample was seen as comparable to a broader population in terms of the measures of interest (see comparison of Tables 1 and 2). However, it is possible that this relationship is particular to the research setting and replication is necessary to further an understanding of how generalizable the claims made are. The interviews conducted provided an exploration into the variety of frequency and quality in study habits, but cannot provide a generalizable statement regarding the likelihood of each combination. Follow-up investigations into the relationship between frequency and quality of study habits to academic performance with large sample sizes from a quantitative perspective would be necessary to further an understanding of these characteristics.

Additionally, the interviews conducted only examined factors related to student self-efficacy and metacognition in seeking to understand characteristics that can explain study habits. Other student factors such as time available to study, perceptions of knowledge generation or familial/social expectations for education may certainly prove to be relevant in understanding study habits. Finally, the interview cohort did not include students who failed or withdrew from first-semester General Chemistry and these students may provide unique, additional characteristics of study habits for at-risk students.

Conclusions and implications

This study classified at-risk students and non-at-risk students according to their SAT math scores. As expected from the research literature, the at-risk students performed worse on each of the in-class measures of chemistry learning. The data also showed that at-risk students put in as much effort in terms of attending class and completing homework, and reported studying outside of class at a higher rate. This may indicate that the at-risk students perceive a lack of preparation and subsequently study more. Results of multiple regression showed that studying frequency plays a more important role in student academic performance in college General Chemistry for the students who have lower SAT scores as compared to students who have higher SAT scores.

Furthermore, findings through text message responses and interviews suggested that both the frequency and quality of studying are important to academic performance for at-risk students. The results of this study lead to several implications for instructors who are teaching college chemistry courses. First, instructors should encourage at-risk students to believe they can succeed by studying (Cook et al., 2013). More importantly, instructional supports should be developed to promote at-risk students studying more and to develop deep level study approaches. In order to increase the frequency of studying, instructors can suggest students keep records of when and how they study in certain time ranges, and use those records to keep track of their study. The text message methodology presented here is one possible approach for doing so.

For the quality of studying, it is essential for instructors to help students develop deeper level study approaches. Instructors can provide specific guidance for at-risk students to use study materials, for example, by making annotated notes while reading the textbook or working the practice problems in the textbook. Likewise, for lecture notes, students can be encouraged to read the notes before the lectures and take their own notes during the lecture to support understanding the content instead of capturing all that is said. After the lectures, it is better to actively summarize or rewrite notes using a student's own words instead of only reviewing the notes taken. In terms of practicing problems, the Learning Approaches for Chemistry framework's emphasis on students generating their own understanding and using others primarily to confirm their understanding is prescient. Thus, efforts to promote students attempting to practice problems independently before comparing with an answer key or asking for help would be recommended. In terms of group work, the successful at-risk students presented here demonstrated independent learning by helping others in groups or using groups to confirm their understanding. The importance of explaining concepts when participating in a group matches learning theories and past research on how group work is effective (Webb 1989, 1992; Slavin, 1996). An instructional implication that follows would be the practice of assigning and rotating roles within the group, where one role has an explicit function of providing explanations when the group is called upon.

For researchers who are interested in designing interventions aimed at helping at-risk students, improving the frequency and quality of study habits are appropriate targets. Past research reviewed herein has described promising intervention techniques that may improve study habits. Future research can be aided by matching these interventions with measures to assess students' study habits with the methodology used here as one potential path for doing so. Another potentially fruitful area for research is to investigate the impact of pedagogy and classroom environment on students' study habits.

Appendix

Interview protocol

1. Background questions

• What is your major? Why did you take General Chemistry I?

• What, if any, chemistry classes have you taken before General Chemistry I in college? In high school?

• Why did you take these classes?

• Are you satisfied with your performance in these classes?

• How would you describe your performance in previous chemistry classes?

• How would you characterize your study approaches?

• Please describe any changes in how you study when transitioning from high school to college.

• How important is studying with peers in high school versus studying with peers in college?

• How confident are you in chemistry?

• How satisfied are you with your resulting grade in General Chemistry I?

• If you could return to when you were in General Chemistry I, would you do anything different in your studying for General Chemistry I? If yes, what would you do?

• How is studying for chemistry different than studying for other classes?

2. Study habit text message clarification

• To what extent did participating in the text message project influence your study approach?

• If you did not respond to a text message, what was the reason?

• Describe how you used [X] in your studying. X = the approaches the students indicated in their responses. e.g. textbook, homework, peer activity…

• What study approaches did you think were helpful for General Chemistry I?

• What study approaches did you think were not helpful for General Chemistry I?

3. External study habit questions

• To what extent did the course instructor influence your study approach

• Was there a particular way the teacher presented the material that you liked a lot? Was there a particular way you did not like?

• How many peers in chemistry do you interact with? How important are these interactions? Describe the nature of these interactions, what types of discussions do you have with your peers in chemistry?

• What prevented you from studying for General Chemistry I?

• What factors were outside of the chemistry content?

• To what extent do you memorize content in General Chemistry I?

• Would you characterize the content in General Chemistry I as having one or a small set of themes or as a list of separate facts?

• How much of your studying for General Chemistry I was in practicing math examples versus conceptual understanding?

• What did you do when you were not sure about a concept in your studying?

• Do you think you had too many tests or not enough tests in General Chemistry I?

• Do you read about science/chemistry beyond what is covered in the course?

• What was the most interesting thing you learned in General Chemistry I? Why was it interesting?

• How did your study approach change between General Chemistry I and General Chemistry II?

• How confident are you in learning General Chemistry II now?

• How are you getting ready for your upcoming General Chemistry II test?

• Where do you like to study?

• Describe your ideal study environment.

Acknowledgements

Partial support for this work was provided by the National Science Foundation's Improving Undergraduate STEM Education (IUSE) program under DUE-1432085. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

References

  1. Benford R. and Gess-Newsome J., (2006), Factors affecting student academic success in gateway courses at Northern Arizona University (online), http://www2.nau.edu/˜facdev-p/TR/Factors.pdf [30 November 2015].
  2. Bentley A. B., Gellene G. I., (2005), A six-year study on the effects of a remedial course in the Chemistry curriculum, J. Chem. Educ., 82, 125–130.
  3. Biggs J. B., (1987), Student approaches to learning and studying, Melbourne, Australia: Australian Council for Educational Research.
  4. Biggs J. B., Kember D. and Leung D. Y. P., (2001), The Revised Two Factor Study Process Questionnaire: R-SPQ-2F, Br. J. Educ. Psychol., 71, 133–149.
  5. Carmichael J. W. J., Bauer J. S., Sevenair J. P., Hunter J. T. and Gambrell R. L., (1986), Predictors of first-year chemistry grades for Black Americans, J. Chem. Educ., 63, 333–336.
  6. Chambers K. A., (2005), Improving performance in first year chemistry, Doctoral Dissertation, Texas Tech University.
  7. Chan J. Y. K. and Bauer C. F., (2014), Identifying at-risk students in General Chemistry via cluster analysis of affective characteristics, J. Chem. Educ., 91(9), 1417–1425.
  8. Chan J. Y. K. and Bauer C. F., (2016), Learning and studying strategies used by General Chemistry students with different affective characteristics, Chem. Educ. Res. Pract., DOI: 10.1039/C5RP00205B.
  9. Cohen J., (1988), Statistical power analysis for the behavioral sciences, 2nd edn, New Jersey: Lawrence Erlbaum Associates.
  10. Cohen Goodman M., (1998), The effect of journal writing on the reading comprehension, study habits and attitude of underprepared college students. Dissertation Abstracts International, 59(3), 722A.
  11. College Board, (2016), SAT, Inside the Test, Math Test, http://https://collegereadiness.collegeboard.org/sat/inside-the-test/math accessed: May 3, 2016.
  12. Cook E., Kennedy E. and McGuire S. Y., (2013), Effect of teaching metacognitive learning strategies on performance in General Chemistry courses, J. Chem. Educ., 90(8), 961–967.
  13. Cracolice S. M. and Busby B. D., (2015), Preparation for college General Chemistry: more than just a matter of content knowledge acquisition, J. Chem. Educ., 92(11), 1790–1797.
  14. Crede M. and Kuncel N. R., (2008), Study habits, skills, and attitudes: the third pillar supporting collegiate academic performance, Perspect. Psychol. Sci., 3, 425–453.
  15. Gabriel K. F., (2008), Teaching underprepared students: strategies for promoting success and retention in higher education, Sterling, VA: Stylus.
  16. Gafney L. and Varma-Nelson P., (2008), Peer-Led Team Learning: Evaluation, Dissemination, and Institutionalization of a College Level Initiative, in Cohen K. C. (ed.) Innovations in Science Education and Technology, New York: Springer Science.
  17. Gosser D. K., Cracolice M. S., Kampmeier J. A., Roth V., Strozak V. S. and Varma-Nelson P., (2001), Peer-led team learning: a guidebook, Upper Saddle River, New Jersey: Prentice-Hall.
  18. Griff E. R. and Matter S. F., (2008), Early identification of at-risk students using a personal response system, Br. J. Educ. Technol., 39, 1124–1130.
  19. Gultice A., Witham A. and Kallmeyer R., (2015), Are your students ready for Anatomy and Physiology? Developing tools to identify students at-risk for failure, Adv. Physiol. Educ., 39, 108–115.
  20. Hailikari T. K. and Nevgi A., (2010), How to diagnose at-risk students in chemistry: the case of prior knowledge assessment, Int. J. Sci. Educ., 32, 2079–2095.
  21. Hall D. M., Curtin-Soydan A. J. and Canelas D. A., (2014), The science advancement through group engagement program: leveling the playing field and increasing Retention in Science, J. Chem. Educ., 91, 37–47.
  22. Harris D. E., Hanuum L. and Gupta S., (2004), Contributing factors to student success in anatomy & physiology: lower outside workload & better preparation, Am. Biol. Teach., 66, 166–175.
  23. Hecker M. J., (1995), Assessing the effectiveness of a developmental educational program using GPA attainment, student retention, and student involvement as measurement variables, Dissertation Abstracts International, 55(9), 2739A.
  24. Hektner J. M., Schmidt J. A. and Csikszentmihalyi M., (2007), Experience sampling method: measuring the quality of everyday life, Thousand Oaks, California: Sage Publications, Inc.
  25. Heredia K., Xu X. Y. and Lewis J. E., (2012), The application and evaluation of a two-concept diagnostic instrument with students entering college general chemistry, Chem. Educ. Res. Pract., 13(1), 30–38.
  26. House J. D., (1995), Noncognitive predictors of achievement in introductory college chemistry, Res. High. Educ., 36, 473–490.
  27. Lewis S. E., (2011), Retention and reform: an evaluation of peer-led team learning, J. Chem. Educ., 88, 703–707.
  28. Lewis S. E. and Lewis J. E., (2007), Predicting at-risk students in general chemistry: comparing formal thought to a general achievement measure, Chem. Educ. Res. Pract., 8(1), 32–51.
  29. Lewis S. E., Shaw J. L., Heitz J. O. and Webster G. H., (2009), Attitude counts: self-concept and success in General Chemistry, J. Chem. Educ., 86(6), 744–749.
  30. Marton F. and Saljo R., (1976), On qualitative differences in learning. I: outcome and process, Br. J. Educ. Psychol., 46, 4–11.
  31. Marton F. and Saljo R., (1984), Approaches to learning, in Marton F., Hounsell D. and Entwistle N. (ed.) The experience of learning, Edinburgh: Scottish Academic Press, pp. 36–55.
  32. Mason D. and Verdel E., (2001), Gateway to success for at-risk students in a large-group introductory Chemistry Class, J. Chem. Educ., 78, 252–255.
  33. Meckstroth W. K., (1974), A chemistry course for underprepared students, J. Chem. Educ., 51, 329.
  34. Pickering M., (1975), Helping the high risk freshman chemist, J. Chem. Educ., 52, 512–514.
  35. Potgieter M., Ackermann M. and Fletcher L., (2010), Inaccuracy of self-evaluation as additional variable for prediction of students at risk of failing first-year chemistry, Chem. Educ. Res. Pract., 11, 17–24.
  36. Russell A. A., (1994), A rationally designed general chemistry diagnostic test, J. Chem. Educ., 71, 314–317.
  37. Shields S. P., Hogrebe M. C., Spees W. M., Handlin L. B., Noelken G. P., Riley J. M. and Frey R. F., (2012), A transition program for underprepared students in General Chemistry: diagnosis, implementation and evaluation, J. Chem. Educ., 89, 995–1000.
  38. Sinapuelas M. L. S. and Stacy A. M., (2015), The relationship between student success in introductory university chemistry and approaches to learning outside of the classroom, 52(6), 790–815 DOI:10.1002/tea.21215.
  39. Slavin R. E., (1996), Research for the future: research on cooperative learning and achievement: what we know, what we need to know, Contemp. Educ. Psychol., 27, 43–69.
  40. Spencer H. E., (1996), Mathematical SAT test scores and college chemistry grades, J. Chem. Educ., 73, 1150–1153.
  41. Tai R. H., Sadler P. M. and Loehr J. F., (2005), Factors influencing success in introductory college chemistry, J. Res. Sci. Teach., 42, 987–1012.
  42. Tro N. J., (2014), Chemistry: a molecular approach, Upper Saddle River, NJ: Pearson Education, Inc.
  43. Valentine J. C., Hirschy A. S., Bremer C. D., Novillo W., Castellano M. and Banister A., (2011), Keeping at-risk students in school: a systematic review of college retention programs, Educ. Eval. Policy Anal., 33, 214–234.
  44. Wagner E. P., Sasser H. and DiBiase W. J., (2002), Predicting students at risk in General chemistry using pre-semester assessments and demographic information, J. Chem. Educ., 79(6), 749–755.
  45. Walmsley F., (1977), A Course for the underprepared Chemistry student, J. Chem. Educ., 54, 314–315.
  46. Webb N. M., (1989), Peer interaction and learning in small groups, Int. J. Educ. Res., 13, 21–39.
  47. Webb N. M., (1992), Testing a theoretical model of student interaction and learning in small groups, in Hertz-Lazarowitz R. and Miller N. (ed.) Interaction in cooperative groups: the theoretical anatomy of group learning, New York: Cambridge University Press, pp. 102–119.
  48. Xu X., Villafane S. M. and Lewis J. E., (2013), College students' attitudes toward chemistry, conceptual knowledge and achievement: structural equation model analysis, Chem. Educ. Res. Pract., 14(2), 188–200.
  49. Ye L., Oueini R., Dickerson A. P. and Lewis S. E., (2015a), Learning beyond the classroom: Using text messages to measure General Chemistry students' study habits, Chem. Educ. Res. Pract., 16, 869–878.
  50. Ye L., Oueini R. and Lewis S. E., (2015b), Developing and implementing an assessment technique to measure linked concepts, J. Chem. Educ., 92(11), 1807–1812.
This journal is © The Royal Society of Chemistry 2016