Gendered responses to online homework use in general chemistry

Abstract

Online homework assignments have been shown to enhance student performance. Our research on gendered responses to these assignments adds new and useful information. We investigated differences between male and female students' responses to online homework in large-enrollment general chemistry courses. Replacing in class quizzes with online homework assignments significantly improved general chemistry success rates for both sexes—however, male students' average success rate improvement was double that of female students'. Therefore, online homework use decreased the female–male achievement gap, previously in favor of female students, to a difference that was no longer significant. Males self-reported less effective study habits than females; thus, we hypothesize that online homework might encourage more effective study habits for male students. However, in spite of greater performance gains for male students, females self-reported more positive views toward online homework use than males. Therefore, females might benefit from online homework's positive impact on their confidence in understanding the material. In addition to improving student performance, online homework also appeared to enhance student retention in the course. Overall, online homework provides a time and cost effective means to enhance pedagogy in large classes for both male and female students.

---

Introduction and framework for exploring gendered responses to online homework

Cultivating instructor knowledge about gender issues is particularly important to recruit, retain and ensure the success of students in science, technology, engineering and math (STEM) majors. Although females outnumber males in terms of bachelor's degree recipients and have achieved parity, or even exceed male majors, in areas such as biology, they are still significantly underrepresented in the physical sciences. Moreover, females are less likely to obtain doctorates, post-doctoral positions, and employment in science professions (NAS, NAE and IOM, 2007a; Hill et al., 2010). Yet, among some ethnic and racial groups, males underperform; specifically, more African-American females than males earned physical science bachelor's degrees (as reviewed in Hill et al., 2010). Both male and female students leave the STEM disciplines, at least in the U.S. and other Western nations, at an alarming rate (NAS, NAE and IOM, 2007b). To better engage students with the material, instructor use of a variety of educational strategies is recommended (as reviewed in Osborne, 2003). This is particularly effective at encouraging women in science (Hill et al., 2010). However, introducing variety and moving beyond lecture and multiple choice exams can be difficult for instructors of large enrollment classes. Our research examines the use of online homework, an educational strategy to enhance student performance, in large enrollment chemistry classes. We focus on gendered differences in student performance and attitudes.

Researchers interested in looking for gendered impacts on student performance should proceed carefully. Sax (2008) cautions that, “…studies tend to compare college men and women in terms of their characteristics and abilities, such as assertiveness or mathematical competence, but pay little attention to the ways in which various experiences may contribute to those gender differences…” (p. 61). Thus, instead of focusing on static, fixed sex differences, Sax's model examines the interaction of gender with other factors within an environmental context. We apply this framework toward an analysis of the gendered impacts of online homework on student performance in introductory, large enrollment chemistry classes. Our aim is to determine whether there are gendered differences in the demonstrated positive impacts of online homework. Are these assignments enhancing the success of male and female students to different degrees? Are there different reasons for impacts on male student performance versus female student performance that can be revealed by an examination of student attitudes toward the assignments? While we acknowledge that gender interacts with other social factors such as race, ethnicity and cultural context among others, a full consideration of these factors is beyond the scope of this study. Thus, we caution that our conclusions are based on a mostly Caucasian population of students at a large public, land grant post-secondary institution in the largely rural Appalachian region of the U.S.

Although female students have made substantial progress in many STEM fields, barriers remain to full equity. For example, in the U.S., females now take as many secondary school math and science credits as males and their grades are higher, but fewer females enter college intending to major in science and related fields (NAS, NAE and IOM, 2007a; Hill et al., 2010). Further, Canadian males who took high school coursework in math and physical sciences were more likely to persist in these majors at college than females with similar high school preparation (Adamuti-Trache and Andres, 2008). The lack of persistence by mathematically talented females may result from females' tendency to underestimate their performance in math when compared to similarly performing male students (as reviewed in Hill et al., 2010). Although middle and secondary students of both sexes overestimate the amount of mathematical ability needed to do science (Kitts, 2009), female middle and secondary students have been found to be less confident about their abilities in science than male students (Beghetto, 2007). This can impact course selection: females tend to select less advanced chemistry courses, in spite of high performance in the prerequisites (Cousins, 2007).

Even though females tend to receive higher grades in science and math courses, males outperform females on standardized tests; and the gap is particularly significant when these tests do not directly relate to material covered in a course (as reviewed in Hill et al., 2010). Data indicate that prior preparation and gendered differences in the knowledge that students bring to the classroom can partially explain these results. Prior knowledge has been demonstrated to be a major determinant of student success in chemistry (reviewed in Seery, 2009). Bell (2001) investigated exam performance differences, which favor males in physics and females in the life sciences, and found an effect of sex only on those questions testing memorized facts indicating that gendered socialization impacts knowledge familiar to each sex. Kost et al. (2007) concluded that male students in introductory physics classes were better prepared when they entered the classroom, even though the introductory class had assumed no prior preparation. Thus, they argue that teachers must recognize that males and females may differ in the background knowledge they bring to the class. There may also be gendered differences in test taking strategies: Although overall performance at solving chemistry problems was equivalent between the sexes, males tended to start faster, whereas females thought about the problem first (Bennett, 2008). Thus, timed tests might penalize females more than males.

In college, students of both sexes identify poor teaching as a reason for leaving STEM majors (Seymour and Hewitt, 1997). Additionally, students who did not switch also identified poor teaching as a problem, though they often benefited from faculty support, which encouraged their persistence (Seymour and Hewitt, 1997). Females are more likely to seek help from instructors (Sax, 2008) and are more skilled at developing peer networks than male students (Seymour and Hewitt, 1997). These trends could explain the finding that in an organic chemistry course, non-cognitive variables such as anxiety and confidence had greater impacts on course grades for male than for female students (Turner and Lindsay, 2003). Overall, males are more likely to view success as an innate ability, whereas females see it as something that is developed with work (Smist and Owen, 1994; as reviewed in Hill et al., 2010). Research indicates that recognizing the positive impacts of hard work can lead to persistence (as reviewed in Hill et al., 2010).

Good pedagogy can address these issues. Specifically, “The Seven Principles for Good Practice in Undergraduate Education” provide a framework within which to envision productive strategies to encourage student success by addressing some of the gendered factors that impede student learning (Chickering and Gamson, 1987). The seven principles are: (1) encourage student–faculty contact, (2) encourage cooperation among students, (3) encourage active learning, (4) give prompt feedback, (5) emphasize time-on-task, (6) communicate high expectations, and (7) respect diverse talents and ways of learning.

Research has demonstrated the positive impacts of many of these principles. For example, it is well-known that encouraging active learning (principle 3) enhances learning (Bonwell and Eison, 1991; Farrell et al., 1999; Oliver-Hoyo et al., 2004; Varma-Nelson and Coppola, 2004; Knight and Wood, 2005; Brooks and Crippen, 2006; Handelsman et al., 2007; Poock et al., 2007). Providing students with immediate feedback (principle 4) improves learning and testing performance, equally, for students of both sexes (Epstein et al., 2001; Epstein and Brosvic, 2002). In addition, Epstein and Brosvic found that students preferred immediate feedback testing methods over delayed feedback methods. Furthermore, research has also shown enhanced learning with increased time-on-task (principle 5) (Cooper and Valentine, 2001; Keith et al., 2004; Varma-Nelson and Coppola, 2004). In fact, research in the field of educational psychology has shown a positive correlation, which increases as students proceed from elementary, through middle, and into secondary school, between time spent on out-of-class homework (increased time-on-task) and achievement (Cooper and Valentine, 2001; Keith et al., 2004; Cooper et al., 2006). Finally, interacting with faculty (principle 1) encourages scientific orientation in both males and females (Sax, 2008).

Instructors of large-enrollment classes (n > 100) may struggle with how to increase student–faculty contact (principle 1), encourage active learning (principle 3), give prompt feedback (principle 4), and emphasize time-on-task (principle 5) without overwhelming themselves with course administrative duties (e.g., grading of paper-based homework or quizzes). Web-based, practice and assessment systems (online homework systems) are attractive because they provide instructors with formative assessment tools that encourage students to engage with content outside of lectures (principle 3), provide students with immediate feedback on question correctness and, thus, understanding of content (principle 4), and increase the amount of time students engage with the content (principle 5). Although these systems cannot provide a substitute for specific interactions with faculty (principle 1), they may indicate that instructors are providing a supportive mechanism to help students tackle difficult concepts.

To combat decreased success rates in large enrollment (n > 100) general chemistry courses at our post-secondary institution, instructor use of the seven principles was supported by use of online homework beginning in fall 2006. Since then, all general chemistry courses at our institution have used online homework as formative assessment in place of quizzes. Commercially developed general chemistry online homework systems are available from most textbook publishing companies and the attributes of each system have been discussed in the literature (Brooks and Crippen, 2006; Evans, 2009; Harris, 2009; Hendrickson, 2009; Miller, 2009; Rowley, 2009; Shepherd, 2009; Zhao, 2009).

The effectiveness of online homework systems in promoting learning relative to ungraded homework is documented in the literature (Penn et al., 2000; Cheng et al., 2004). One of us (Richards-Babb et al., 2011) has extensively studied and compared historical success rates in general chemistry and has found that replacing quizzes by online homework significantly (p < 0.0005) improved students' success in general chemistry by between 3.8% and 12.1%. However, whether or not online homework offers learning improvements relative to graded homework is still an open question (Bonham et al., 2001; Charlesworth and Vician, 2003; Cole and Todd, 2003; Arasasingham et al., 2005; Fynewever, 2008). In addition, several researchers have found that students as a group are generally positive toward online homework use (Charlesworth and Vician, 2003; Freasier et al., 2003; Arasasingham et al., 2005; Williams et al., 2008; Richards-Babb et al., 2011). However, whether male and female students have different impacts from and/or attitudes toward online homework use has not been extensively addressed; although, Charlesworth and Vician (2003) observed more appreciation of online resources by females than males. To add to the knowledge base in this area, we focused on gendered differentials and addressed the following specific research questions:

1. Are there any differences between male and female students in the impacts of online homework on student success in general chemistry?

2. Is online homework equivalent to other methods (e.g., weekly quizzes) of formative assessment?

3. Are there any differences between male and female students in their attitudes toward online homework use?

Intervention, aims, and sample

Our research was designed to investigate these questions. Our intervention involved a change in the general chemistry course formative assessment tool from weekly quizzes to a web-based, practice and assessment system (i.e., online homework). The online homework average (fall 2006 and beyond) directly replaced the quiz average (pre-fall 2006) as 10% of the course grade. Otherwise, the course remained unchanged with respect to lecture content and delivery, and grade component distributions. The WileyPLUS system (Zhao, 2009) was used to administer online homework from fall 2006 to fall 2008, whereas, due to a textbook change, the Mastering Chemistry system (Shepherd, 2009) was used in spring 2009 and beyond. In the U.S., student access codes for online homework systems are routinely packaged along with the corresponding general chemistry texts. Instructors construct their own online homework assignments by (i) choosing from the large pool of questions within the online homework system and/or (ii) writing their own questions. Available within the system are end-of-chapter questions in a variety of formats (e.g., multiple choice, text entry, algorithmic, true/false, matching) from the corresponding text. In addition, most online homework systems provide student accessible supplementary resources (e.g., electronic text or ebook, demonstration videos, simulations, worked examples, or tutorials). Our instructors created weekly online homework assignments that contained an average of 15–30 questions on topics relevant to lectures and exams. For comparison purposes, a sampling of quiz, online homework, and exam questions used by Richards-Babb is given in Table 1. So as not to penalize students for the learning process, students were given three attempts to correctly answer each question within the online homework assignments.

Table 1 Comparison of CHEM 116 quiz and online homework questions and corresponding exam questions

General Chemistry Topic	Sample Quiz Question (pre-fall 2006) (non-multiple choice: show all work)	Sample Online Homework Question^a (fall 2006 and beyond)	Sample Exam Question (multiple choice: five different choices)
a In the U.S., most publishers offer online homework systems for free along with a corresponding general chemistry text.
Molarity of Ions in Solution	10.0 g CaCl2 is dissolved in enough water to prepare 500 mL of solution. What is the molarity of chloride ion in this solution?	In a solution of FeCl2, the chloride concentration is 0.010 M. How many grams of FeCl2 are in 550 mL of this solution? (numerically algorithmic or multiple choice)	A solution is made by dissolving 20.0 g NaCl and 10.0 g CaCl2 in enough water to prepare 250 mL of solution. What is the molarity of chloride ion in this solution?
Net Ionic Equations	Predict products and write a balanced net ionic equation for the reaction, KOH + HNO2, that occurs in aqueous solution.	Select the balanced molecular, ionic, and net ionic equations for the reaction: NaHS + HCl → (up to eight different choices given)	Predict products and write the net ionic equation for the reaction, Ba(C2H3O2)2 + HNO3, occurring in aqueous solution. The spectator ions are:
Titration of Weak Acid by Strong Base	20.0 mL of 0.100 M HNO2 is titrated with 12.0 mL of 0.120 M KOH. What is the pH of the resulting solution?	In the titration of 20 mL of 0.20 M HCOOH by 0.15 M LiOH, what is the pH of the solution after the addition of 18 mL of base? (numerically algorithmic or multiple choice)	10.0 mL of 0.30 M HC2H3O2 is titrated with 10.0 mL of 0.20 M KOH. What is the pH of the resulting solution?

---

We focused our research efforts on the second-half or second-semester of the two part general chemistry course (CHEM 116). Undergraduate students majoring in STEM disciplines populate this course at our post-secondary institution. This study is timely because of (i) the steady increase in use of online homework as a formative assessment tool at institutions within the U.S. and beyond and (ii) the ready availability of commercially developed online homework systems from textbook publishing companies. Investigating whether male and female students differ in their response to these instructional strategies adds new information to research on the use of online homework assignments.

To investigate whether online homework's demonstrated ability to improve general chemistry success rates differs between males and females (research question 1), we gathered sex-differentiated student data for all CHEM 116 courses taught at our institution from fall 2001 to fall 2009. Gathered data was further partitioned according to the intervention—no online homework treatment (pre-fall 2006) vs. with online homework treatment (fall 2006 and beyond). Although different instructors taught the CHEM 116 course over the time-period of this study, the course itself was administered by Richards-Babb and course syllabi and grading schemes were coordinated—across sections and across years.

To evaluate the equivalence of the two formative assessment tools—online homework versus quizzes (research question 2)—specific grade component information for a subset of CHEM 116 courses was collected. The sample for this study included only students enrolled in Richards-Babb's CHEM 116 courses over the study's time period (fall 2001 to fall 2009). Prior to online homework treatment (pre-fall 2006), when quizzes were used for formative assessment, Richards-Babb taught seven sections of CHEM 116 with total male and female enrollments of 407 (48.2%) and 438 (51.8%), respectively. With online homework formative assessment and treatment (fall 2006 and beyond), Richards-Babb taught eight sections of CHEM 116 with total male and female enrollments of 430 (47.3%) and 480 (52.7%), respectively.

To investigate whether students' attitudes toward online homework use differ between males and females (research question 3), the attitudinal survey instrument “CHEM 116 Online Homework Evaluation” was administered to a subset of students—all students remaining in the CHEM 116 course at the end of the fall 2006 and fall 2007 semesters. Although the survey data were self-reported, given the nature of its administration and the high rate of return (332 of 461 or 72.0%), we believe that a representative sample of student responses was obtained.

Sex-differentiated data from the demographic questions embedded in the attitudinal survey is shown in Table 2. Of the 332 students who completed the survey, 37.0% were males, 54.8% were females, and 8.1% (or n = 27) were excluded from the analysis due to lack of sex identification (24 of 27) or use of pen (3 of 27) instead of pencil. (Pen marks are not detected by scantron analysis of surveys.) The female to male survey completion ratio (182 : 123 or 1.48 : 1) was larger than the female to male student ratio in the combined fall 2006 and 2007 classes (308 : 263 or 1.17 : 1). Thus, it should be noted that attitudinal survey data was tabulated based upon a subset of students—those students who actually attended class and completed the survey on the day it was administered.

Table 2 Demographic data and comparative statistics for students completing the “CHEM 116 Online Homework Evaluation” attitudinal survey instrument^a

Student Self-Reported Demographic Data from Attitudinal Survey	General Chemistry: Males n = 123 of 332 (37.0%)	General Chemistry: Females n = 182 of 332 (54.8%)	Comparison of Male to Female Demographic Data: z-statistic and two-sided p-value
a n = 332 students completed the survey. However, 27 (8.1%) survey responses were excluded due to (i) lack of gender identification (24 of 27) or (ii) use of pen (3 of 27) instead of pencil.
Class Status: 1st Year	9.8%	2.7%	z = 2.62
	(12 of 123)	(5 of 182)	0.005 < p < 0.01
			significant at 99% level
Class Status: 2nd Year	52.0%	61.5%	z = 1.65
	(64 of 123)	(112 of 182)	0.05 < p < 0.10
Class Status: 3rd year & above	38.2%	35.7%	z = 0.44
	(47 of 123)	(65 of 182)	0.50 < p
First Attempt at CHEM 116	73.6%	81.7%	z = 1.68
	(89 of 121)	(147 of 180)	0.05 < p < 0.10
Second Attempt or above at CHEM 116	26.4%	18.3%	z = 1.68
	(32 of 121)	(33 of 180)	0.05 < p < 0.10

---

The equivalence of the two groups surveyed (males and females), was compared using non-sex specific demographics. Only the percentage of first year students (freshmen) was significantly different between females and males; although the number of first year students enrolled was low, 2.7% of females and 9.8% of males. Most of the students were second year (sophomores) or third year (juniors) and above. The unusually low number of first year students in the subset of students surveyed was a consequence of the “off-sequence” nature of the fall semester CHEM 116 course. Fall semester and spring semester CHEM 116 courses are referred to as “off-sequence” and “on-sequence”, respectively. The off-sequence CHEM 116 course was populated by students who (i) were less academically prepared upon entering college, (ii) took remedial or preparatory coursework in their first year, and/or, as a result, (iii) were second or third year students one or more semesters behind their entering cohort. However, first years students primarily populate the on-sequence CHEM 116 course. In addition, female and male survey respondents did not differ significantly in the number of attempts at passing the course. Due to the low number of minority students enrolled (approximately 7% of our institutional student body), we did not attempt to disaggregate the data for race or ethnicity. This research was reviewed by our Institutional Review Board (IRB) and was deemed to properly follow federal and institutional regulations for protection of human participants.

Research question 1: impacts of online homework on student success

Method

Gathered data consisted of (i) second-semester general chemistry (CHEM 116) success rates and corresponding (ii) college-entrance readiness indicators (CERI) for college coursework. This data was differentiated by sex and further partitioned by intervention—no online homework treatment (pre-fall 2006) versus with online homework treatment (fall 2006 and beyond). More details on the sample used can be found in the “Interventions, aims, and sample” section of this manuscript. Our institutional Information for Decision Enabling and Analysis System (IDEAS) provided this data. Success rates were defined as the number of students of each sex succeeding in the course (i.e., earning final letter grades of A, B, or C) divided by the total number of students of that sex enrolled in the course. Collected CERI indicators were secondary school grade point average (gpa), American College Testing (ACT) composite scores and math subscores and Scholastic Aptitude Test (SAT) scores and math subscores. SAT and ACT are national college entrance tests that are taken by U.S. students seeking college admittance. Many U.S. college admission offices use ACT or SAT scores as one criterion for admission. Past research has revealed positive correlations between CERI (e.g., secondary school grade point averages and SAT and ACT scores/math subscores) and chemistry performance (Carmichael et al., 1986; Nordstrom, 1990; Bunce and Hutchison, 1993; Spencer, 1996; Lewis and Lewis, 2007). CERI comparisons between the two sample populations—no online homework treatment versus with online homework treatment—established sample equivalence. Standard comparative statistical techniques for evaluating inferences about proportions and inferences about means were used to calculate p and z or t-values (Moore, 2007).

Findings and discussion

As shown in Table 3, the average success rate of female students in general chemistry significantly improved by 5.0% with online homework treatment. This success rate enhancement was even more significant for men—male students exhibited more profound success rate improvements (10.1%) in general chemistry coursework with online homework treatment than female students.

Table 3 Sex-differentiated general chemistry success rates and comparative statistics with and without online homework treatment

Gender	Average Success Rate With Online Homework Treatment^a	Average Success Rate No Online Homework Treatment^b	Difference (Treatment-No Treatment	z & p-values (one-sided)
a Includes data from seven semesters of CHEM 116, fall 2006 through fall 2009. b Includes data from ten semesters of CHEM 116, fall 2001 through spring 2006.
Females	63.5%	58.5%	5.0%	z = 2.90
	(n = 1448)	(n = 1853)		0.001 < p < 0.0025
				Significant at 99.5% Level
Males	61.8%	51.6%	10.1%	z = 5.77
	(n = 1420)	(n = 1832)		p < 0.0005
				Significant at 99.9% Level
Difference (Females-Males)	1.7%	6.9%
z & p-values (one-sided)	z = 0.94	z = 4.19
	0.15 < p < 0.20	p < 0.0005
	Not Significant	Significant at 99.9% Level

---

Even more informative is the comparison of female students' success rates to male students' success rates with and without online homework treatment as shown in Table 3. Without online homework (pre-fall 2006), female students were 6.9% more likely to earn final letter grades of A, B, or C than their male counterparts. This difference was highly significant and is indicative of an achievement gap, the males being significantly less successful than the females. With online homework treatment (fall 2006 and beyond), average success rates overall were still 1.7% higher for females than for males; but (i) this difference had decreased by 5.2% from its pre-online homework value of 6.9% and (ii) the success rate difference between females and males was no longer significant. In fact, male students outperformed female students by a small (but insignificant) margin during the spring semesters! This data suggests that online homework use served to close the achievement gap between female and male students in our course.

Thus far, all indications are that use of online homework improves general chemistry success rates for both female and male students. Alternatively, could improved success rates be due to better pre-collegiate academic preparation instead of online homework use? Were the two sample populations—no online homework treatment versus with online homework treatment—equivalent in terms of readiness for college coursework? To address this question, sex-differentiated CERI were obtained, and averages calculated and compared—pre-fall 2006 (no online homework treatment) to fall 2006 and beyond (with online homework treatment) and females to males.

As shown in Table 4, the only CERI that was significantly different was the ACT math subscore for males. It was significantly higher for males enrolled fall 2006 and beyond (24.8) than for those enrolled before fall 2006 (24.3). However, the ACT math subscore for females enrolled fall 2006 and beyond (23.7) was also higher than for those enrolled before fall 2006 (23.4)—but, not significantly so. Thus, although the difference in the ACT math subscore for males is significant, it (i) is small, (ii) is paralleled by a similar difference in the ACT math subscore for females, and therefore (iii) we argue that it cannot alone account for the closing of the achievement gap between females and males in general chemistry coursework. As a result, we believe that our two sample populations are roughly equivalent and that our success rate data is not confounded by inherent CERI differences between the two populations.

Table 4 Incoming ACT combined and math subscores (CERI) for students enrolled in the CHEM 116 course prior to and after the online homework intervention

(i) Females	Average and Standard Deviation Fall 2006 and Beyond (Online Homework Treatment)^a	Average and Standard Deviation Pre-Fall 2006 (No Online Homework Treatment)^b	Comparison: t-statistic and two-sided p-value
a Includes data from seven semesters of CHEM 116, fall 2006 through fall 2009. b Includes data from ten semesters of CHEM 116, fall 2001 through spring 2006. c ACT stands for American College Testing.
ACT ^c combined/ACT math	24.8 ± 3.5/23.7 ± 3.9	24.7 ± 3.4/23.4 ± 3.9	t = 0.84, p = 0.40;
			t = 1.33, p = 0.18
	(n = 834)	(n = 1087)	Not Significant

---

(ii) Males	Average and Standard Deviation Fall 2006 and Beyond (Online Homework Treatment)^a	Average and Standard Deviation Pre-Fall 2006 (No Online Homework Treatment)^b	Comparison: t-statistic and two-sided p-value
ACT ^c combined/ACT math	25.1 ± 3.6/24.8 ± 4.3	24.5 ± 3.5/24.3 ± 4.1	*t = 3.45, p = 6 × 10^−4;
			*t = 2.64, p = 9 × 10^−3
	(n = 765)	(n = 1003)	Significant

---

Conclusions

Replacing quizzes by online homework assignments significantly improved general chemistry success rates for both female and male students—albeit, not to the same extent. Success rate improvement for male students was double that of female students and online homework use decreased the female–male achievement gap to a non-significant level. Incoming college-entrance readiness indicators (e.g., secondary school gpa and SAT scores) were relatively constant for students of both sexes over the time period in question. Therefore, these indicators were discounted as causing the observed success rate enhancements upon implementation of online homework. Thus, our findings mirror those described in the literature where female students receive higher grades than male students in coursework (NAS, NAE and IOM, 2007a; Hill et al., 2010). Online homework provides a mechanism that enhances the performance of male students and reduces the female–male achievement gap while, also, providing positive impacts on the performance of female students.

Research question 2: equivalence of online homework and quizzes as formative assessment tool

Method

Specific grade component information (e.g., exam averages, quiz/online homework averages) was collected for a subset of courses taught by Richards-Babb (see “Interventions, aims, and sample” section for sample details). Gathered data was partitioned by sex and by formative assessment tool—online homework versus quizzes. For the data subsets, averages and standard deviations were calculated along with correlations—(i) online homework to grade components and (ii) quizzes to grade components. Standard comparative statistical techniques for evaluating inferences about means were used to calculate t-values.

Findings and discussion

Female students earned significantly higher scores than male students on both quizzes (67.6% vs. 64.0%, p < 0.001, t = 3.19, one-sided) and online homework assignments (80.6% vs. 73.3%, p < 0.0005, t = 6.29, one-sided) (see Table 5). For both sexes, both quizzes and online homework assignments were correlated with other grade components (e.g., exam scores), though (i) the relationships were stronger between quiz scores and grade components and (ii) both sexes received higher scores on the online homework than on the quizzes (see Table 5). The allowance for multiple attempts (three maximum) probably contributed to higher scores on the online homework than on the quizzes (Richards-Babb et al., 2011). The two formative assessment tools (online homework vs. quizzes), although equivalent in terms of content, coverage, and question mechanics, had different effects on students' final numerical grades. Could inherent differences in the character of the quiz and online homework activities be responsible for these effects? Quizzes required advance preparation, afforded one chance to answer questions correctly, and were “closed book” (i.e., use of outside resources was prohibited). In contrast, online homework did not require advance preparation, afforded several chances to answer questions correctly, and was “open book” (i.e., use of outside resources was allowed). The totality of these differences could account for the better correlation between quizzes and grade components relative to online homework and grade components.

Table 5 CHEM 116 quiz averages^a (pre-fall 2006 and no online homework treatment) and online homework averages^b (fall 2006 and beyond) correlated to grade components for males relative to females

General Chemistry (No Online Homework Treatment)^a	Correlation: Quiz Average vs. Exam Average	Correlation: Quiz Average vs. Final Exam	Correlation: Quiz Average vs. Final Numerical Grade	Correlation: Quiz Average vs. Numerical Grade Less Quiz Contribution	Quiz Average	Quiz Standard Deviation
a Only includes data from seven CHEM 116 sections taught by Richards-Babb during the no online homework treatment time period from fall 2001 through spring 2006. b Only includes data from eight CHEM 116 sections taught by Richards-Babb during the online homework treatment time period from fall 2006 to fall 2009.
Females	0.78	0.70	0.84	0.80	67.6%	15.9%
					(n = 438)
Males	0.76	0.69	0.84	0.79	64.0%	16.9%
					(n = 407)

---

General Chemistry (With Online Homework Treatment)^b	Correlation: Online Homework Average vs. Exam Average	Correlation: Online Homework Average vs. Final Exam	Correlation: Online Homework Average vs. Final Numerical Grade	Correlation: Online Homework Average vs. Numerical Grade Less Online Homework Contribution	Online Homework Average	Online Homework Standard Deviation
Females	0.57	0.53	0.68	0.60	80.6%	14.8%
					(n = 480)
Males	0.54	0.50	0.68	0.58	73.3%	19.7%
					(n = 430)

---

Can it be argued that improved success rates are simply due to numerically improving students' performance on 10% of their grade, online homework versus quiz? Remember, in fall 2006, the online homework average directly replaced the quiz average as 10% of the general chemistry final numerical grade. Consider the female students. On quizzes (pre-fall 2006), the average female scored 67.6%; whereas, on online homework (fall 2006 and beyond) the average female scored 80.6%, a significant increase of 13.0% (one-sided: p < 0.0005, t = 12.8). This amounts to an increase in the final numerical grade or “grade boost” of 1.3% when switching from quizzes to online homework. A similar situation arises for male students. Quiz averages were 64.0%; online homework averages were 73.3%, a significant increase of 9.3% (one-sided: p < 0.0005, t = 7.30) for a final numerical grade increase of 0.93%. This line of reasoning would suggest that female students should receive a larger final numerical grade boost from online homework than male students and thus, online homework use should serve to widen the achievement gap between females and males. On the contrary, we observed that online homework use served to close the achievement gap between female and male students in general chemistry.

In addition, female and male average scores for other grade components—first through fourth exams (given after 4th, 7th, 10th, and 13th weeks of instruction, respectively), laboratory average, and final exam—for all courses taught by Richards-Babb over the time-period of this study, were preliminarily explored by comparing average scores pre-fall 2006 (no online homework treatment) and fall 2006 and beyond (with online homework treatment). For females, once online homework replaced quizzes, significant increases in first exam scores (by + 2.9%, t = 2.50), second exam scores (by + 2.0%, t = 1.96), quiz/online homework averages (by + 13.0%, t = 12.8), and final numerical grades (by + 1.5%, t = 1.98) but a significant decrease in fourth exam scores (by −3.1%, t = 2.46) were observed. For males, significant increases in first exam scores (by + 2.5%, t = 1.98) and quiz/online homework averages (by + 9.3%, t = 7.30) but a significant decrease in fourth exam scores (by −3.8%, t = 2.82) and no significant change in final numerical grades (t = 0.0015) were observed.

Yet again, and based on the changes in final numerical grades (1.5% for females and none for males), this should serve to widen the achievement gap between females and males, not reduce it. What is going on? We believe that online homework use improves students' active learning and time-on-task and, therefore, students' performance (performance effect) which, in turn, improves student retention (retention effect). Notice that both male and female students performed better on the first exam (by >+ 2%) once online homework replaced quizzes. Perhaps the average or C letter grade students—more likely to earn lower scores on the first exam and, as a result, more likely to withdraw from the class earlier in the semester—are being retained due to the supportive environment offered by the online homework and higher scores earned on the first exam. If a large number of these students are retained in the class, this would lead to higher success rates but not, necessarily, higher numerical averages on grade components.

Conclusions

Students of both sexes received higher average scores on the online homework than on previously used quizzes resulting in grade boosts of 1.3% and 0.93% for females and males, respectively. However, this would serve to widen the female-male achievement gap; not lessen it. In addition, preliminary analyses of general chemistry grade components (e.g., exam averages and final exam score) point to a potential 1.5% final numerical grade boost for female students; but, none for male students with online homework use. We believe that this is evidence for the fact that (i) online homework does more than just simply improve the numbers by improving performance on one component of the overall grade and (ii) online homework not only improves students' active learning and time-on-task and, therefore, students' performance but also improves student retention. The potential retention effect is important because of the need to recruit and retain more students to STEM fields (Seymour and Hewitt, 1997; NAS, NAE and IOM, 2007b). Interventions such as the one discussed herein could play a key role by targeting a group of students who have the potential to succeed but who are apt to become discouraged and drop out without support. Tobias (1990) argued that retaining this so-called “second tier” was vital to strengthening the STEM workforce.

We hypothesize that the online homework activity forces students to put in more hours studying, concurrent with its completion, than they may have otherwise put in to study and prepare in advance for quizzes. This amounts to more time-on-task and consequent greater knowledge gains. Since it is more the males that lack the self-regulation needed for advance quiz study (Xu and Corno, 2006), the benefits to them (of regulated study through online homework) are greater and this is more important than the increased contribution the online homework makes to their final numerical grade.

Research question 3: students' attitudes toward online homework use

Method

The “CHEM 116 Online Homework Evaluation” survey consisted of 4 demographic questions (Table 2), 36 attitudinal Likert-type statements (see Table 6) and 4 free-response questions (Richards-Babb et al., 2011). Likert-type statements covered topics of online homework completion, understanding, attitudes, study habits, perceived effect on grades, as well as other questions of interest to the investigators. This survey was administered to a subset of students from Richards-Babb's fall 2006 and fall 2007 classes and student survey responses were combined prior to separation by sex. Details on the students sampled by this survey can be found in the “Interventions, aims, and sample” section (see Table 2). Student participation in the attitudinal survey was voluntary and anonymous. Responses to the attitudinal survey were separately aggregated by sex for each Likert-type question and survey data was quantitatively treated by assigning a numerical value to each lettered response, for example, a (strongly agree) = 5, b (agree) = 4, c (neither agree or disagree) = 3, d (disagree) = 2, and e (strongly disagree) = 1, and calculating averages and standard deviations. To compare attitudinal differences, t-test analyses were performed on the spread of student responses for each question (males vs. females). Corresponding levels of significance (p-values) for a directional (one-sided) test were calculated and are shown in Table 6 for statements in which p < 0.05 (90% one-sided confidence interval) indicating significant response differences between genders. Although t-test methods are more appropriate for numerical data, Romano et al. (2006) found that conclusions from t-tests are “somewhat robust” in “evaluating group differences for discrete ordinal data” (p. 20). We recognize the issues arising from using t-test analyses for treating ordinal survey data and caution that our survey analyses are useful only as qualitative indicators of attitudinal trends.

Table 6 Online homework attitudinal responses and comparative statistics for males relative to females

Positive Statements^a	Males: Average Response^b	Females: Average Response^b	Significance Levels: t-statistic and one sided p-values
a Data only given for statements in which a significant difference (p < 0.05) between male and female responses was found. b Likert-scale used: a (strongly agree) = 5, b (agree) = 4, c (neither agree or disagree) = 3, d (disagree) = 2, and e (strongly disagree) = 1. c All nighter refers to postponing exam study until the night before the exam and then staying up all through the night to study.
1: I completed all of the online homework assignments.	4.2	4.7	t = 3.37
	(n = 123)	(n = 182)	p = 0.0005
32: I recommend that the online homework assignments be used for future general chemistry classes.	4.3	4.5	t = 1.68
	(n = 122)	(n = 182)	p = 0.04^7

---

Negative Statements^a	Males: Average Response^b	Females: Average Response^b	Significance Levels: t-statistic and one sided p-values
4: Besides online homework and laboratory homework, I did none of the other homework recommended on the homework sheet.	3.5	3.1	t = 2.05
	(n = 122)	(n = 181)	p = 0.02
10: I never tried to figure out my mistakes on questions I answered wrong within the online homework.	2.6	2.3	t = 2.26
	(n = 124)	(n = 182)	p = 0.01
12: In the future, I would be less apt to take a course that included online homework.	2.3	1.9	t = 2.48
	(n = 123)	(n = 181)	p = 0.007
14: The online homework assignments were a waste of time.	2.3	1.7	t = 4.03
	(n = 123)	(n = 180)	p = 5 × 10^−5
16: The online homework assignments did not further my understanding of chemistry concepts.	2.2	1.9	t = 2.05
	(n = 124)	(n = 181)	p = 0.02
23: To study for the chemistry exams, I typically pulled all nighters^c.	2.7	2.3	t = 2.44
	(n = 123)	(n = 182)	p = 0.008
26: I only studied chemistry on the days the online homework was due and the night before the exam.	3.2	2.7	t = 3.30
	(n = 123)	(n = 182)	p = 0.0006

---

Findings and discussion

Although attitudes toward online homework use were generally positive, females self-reported more positive views of the online homework than males. For example, females were in significantly better agreement with statements of completing all of the online homework assignments (statement 1 in Table 6) and recommending that the online homework be used for future general chemistry classes (statement 32) than males. In contrast, males self-reported less positive (or more negative) views of the online homework than females. Males were more likely to agree with statements of: (i) being less apt to take a course that included online homework in the future (statement 12), (ii) considering the online homework assignments a waste of time (statement 14), and (iii) believing the online homework assignments did not further their understanding of chemistry concepts (statement 16). Furthermore, males self-reported less effective study habits of, for example, never trying to figure out mistakes on questions answered wrong within the online homework (statement 10) and staying up all night to study (statement 23).

Conclusions

Although attitudes toward online homework use were generally positive for all students, females on average self-reported more positive views of the online homework than males—a finding that is in line with prior research (Charlesworth and Vician, 2003). Female students were in significantly better agreement than males with statements indicating completion of all of the online homework assignments and recommending the online homework for future classes. Whereas males agreed more with negative statements of being less apt to take a course with online homework, viewing the online homework as a waste of time, and believing that online homework did not further their understanding of chemistry concepts. Males also self-reported less effective study habits of not trying to figure out mistakes and only studying the night before online homework due dates or exams. These patterns could provide some explanation for why female students tend to receive higher course grades than males in STEM courses (Seymour and Hewitt, 1997; NAS, NAE and IOM, 2007b). If female students have better study habits, then one would expect their course grades to exceed those of males with poorer habits. Sax (2008) also found that, in general, females spend more time working or studying than males, and that these patterns are reflected in females' higher performance in coursework. It is possible that the closure of the achievement gap between male and female students that we identified results from male students being held accountable for studying regularly. Unfortunately, due to survey anonymity we could not directly link a particular student's survey responses to his/her success, that is, letter grade, in the class. In addition, qualitative responses to open-ended questions, on use of online homework as a learning tool and how online homework use affected study habits, that would help to triangulate these differences could only be sex-differentiated and coded in a few cases. Future research, which employs these methods, will be used to examine our conclusions about the gendered impacts of online homework.

Summary conclusions

It would seem that—although men were not as appreciative of the mandatory nature of online homework—it was the male students who benefited the most from its use. Could this have something to do with what each gender credits with their academic success (attributions for success and failure)? Females, in general, attribute their academic success to amount of effort, whilst males attribute academic success to inherent ability (Ryckman and Peckham, 1987; Smist and Owen, 1994; as reviewed in Hill et al., 2010). For female students, the effort involved in completing online homework along with any associated learning gains may reinforce the belief in effort as fundamental to academic success—thus, females would be expected to view online homework in a more positive light. Conversely, the average male attributes his academic success to inherent ability. Therefore, it is not surprising that males have more negative views of the online homework than females and tend to consider the online homework as a waste of time. Thus, we conclude that both sexes benefit from online homework, but we see a conditional effect of gender in the mechanism. We hypothesize that the nature of the online homework activity forces male students to adopt better study strategies—study more, study more frequently, and limit cramming—that, in turn, lead to greater knowledge gains. For females, it reinforces their belief that work improves achievement and provides a direct reward for their efforts. Further research is required to evaluate the accuracy of these proposed mechanisms.

This result is significant and more research in this area is needed, because one of the recommendations from the American Association of University Women's (AAUW's) report on increasing the numbers of females in STEM fields is to encourage students to see achievement and success as a result of hard work, rather than innate ability (Hill et al., 2010). Online homework's demonstrated positive effect on female performance could be used to bolster this attitude among female students and, perhaps, address some of the confidence issues that have been proposed as an explanation for female students' lack of persistence. Seymour and Hewitt (1997) note that since females tend to underestimate their ability, they need to learn how to accurately assess their performance to avoid declines in confidence. Unfortunately, in many cases, Sax (as reviewed in 2008) found that interactions with faculty actually led to declines in female students' rankings of their mathematical ability. Online homework might offer a means to provide female students with prompt feedback that bolsters confidence because the process is more anonymous than confronting an instructor with mistakes. Additionally, the chance for multiple attempts encourages the belief that finding solutions is a result of working at problems rather than immediate discernment of the correct answer. Perhaps positive impacts on confidence explain some of the reasons why female students liked the assignments. Future research should follow up by exploring this potential effect.

Our results on student performance and online homework could also be used to encourage males to see the relationship between work and performance. Other work shows that gendered student attitudes toward homework and homework management strategies can be changed at the middle and secondary school levels where most research has concentrated. Mau and Lynn (2000) found that achieving higher math and science test scores is correlated with doing more homework for both male and female secondary school students, although the correlation was stronger among females. Wagner et al. (2008) found that although females and males do the same kinds of activities during homework, females devote more time to the tasks. Among secondary school students, females were more likely to complete homework and to arrange their surroundings and manage emotions to be able to complete the work effectively (Xu, 2006). Xu (2005) found that middle and secondary school females were more likely to do homework for intrinsic reasons (e.g., strengthening study skills, being responsible, disciplined and independent, as well as enhancing learning gains); students motivated by intrinsic reasons performed better. However, these gendered effects were malleable; if males got assistance with their work, they were more likely to complete it for intrinsic reasons (Xu, 2005). Middle school students' attitudes toward homework were more positive if they were intrinsically motivated and used good management strategies; these factors were more prevalent among females but were encouraged among all students by receiving aid from parents (Xu, 2007). Our research provides new and useful information because it indicates that online homework can encourage these beneficial patterns in college students.

Overall, we found that online homework succeeds in strengthening instructors' application of Chickering and Gamson's (1987) seven principles, especially principles (3) encourage active learning, (4) provide prompt feedback, and (5) emphasize time-on-task. Additionally, online homework seems to provide a means to address principle (1) encourage student–faculty contact by providing support for student problem solving and principle (7) respect diverse talents and ways of learning by the differential mechanisms that apparently promote the success of male and female students. For this reason, we recommend that institutions adopt online homework, particularly in large classes where it can be difficult to implement the seven principles. However, instructors should be aware of potential differences between male and female students in their understanding of the utility of these assignments. Discussion of the relationship between studying and success rates might reinforce female students' study skills and enhance their confidence, while encouraging male students to value the assignments and to see their performance as improvable with studying.

Future work should examine the impact of instructor messages about the value of assignments on student performance and attitudes. In addition, follow-up work will study the effects of online homework on student performance and retention in other chemistry courses, as well as, sex-differentiated details (from open-ended questions) on use of online homework as a learning tool and how online homework affects study habits.

References

1. Adamuti-Trache M. and Andres L., (2008), Embarking on and persisting in scientific fields of study: Cultural capital, gender, and curriculum along the science pipeline, Int. J. Sci. Educ., 30, 1557–1584.
2. Arasasingham R. D., Taagepera M., Potter F., Martorell I. and Lonjers, S., (2005), Assessing the effect of web-based learning tools on student understanding of stoichiometry using knowledge space theory, J. Chem. Educ., 82, 1251–1262.
3. Beghetto R. A., (2007), Factors associated with middle and secondary students' perceived science competence, J. Res. Sci. Teach., 44, 800–814.
4. Bell J., (2001), Investigating gender differences in the science performance of 16-year-old pupils in the UK, Int. J. Sci. Educ., 23, 469–486.
5. Bennett S. W., (2008), Problem solving: Can anybody do it?, Chem. Educ. Res. Pract., 9, 60–64.
6. Bonham S., Beichner R. and Deardorff D., (2001), Online homework: Does it make a difference?, Phys. Teach., 39, 293–296.
7. Bonwell C. C. and Eison J. A., (1991), Active learning: creating excitement in the classroom, Association for the Study of Higher Education (ASHE), Washington, DC, ERIC Clearinghouse on Higher Education, 1–121.
8. Brooks D. W. and Crippen K. J., (2006), Web-based practice and assessment systems in science, in Handbook of College Science Teaching, ed. Mintzes J. J. and Leonard W. H., Arlington, VA, NSTA Press, 251–259.
9. Bunce D. M. and Hutchison K. D., (1993), The use of GALT (Group Assessment of Logical Thinking) as a predictor of academic success in college chemistry, J. Chem. Educ., 70, 183–187.
10. Carmichael, Jr. J. W., Bauer J., Sevenair J. P., Hunter J. T. and Gambrell, R. L., (1986), Predictors of first-year chemistry grades of black Americans, J. Chem. Educ., 63, 333–336.
11. Charlesworth P. and Vician C., (2003), Leveraging technology for chemical sciences education: An early assessment of WebCT usage in first-year chemistry courses, J. Chem. Educ., 80, 1333–1337.
12. Cheng K. K., Thacker B., Cardenas R. L. and Crouch C., (2004), Using an online homework system enhances students' learning of physics concepts in an introductory physics course, Am. J. Phys., 72, 1447–1453.
13. Chickering A. W. and Gamson Z. F., (1987), Seven principles for good practice in undergraduate education, American Association for Higher Education (AAHE) Bulletin, 39, 3–7.
14. Cole R. S. and Todd J. B., (2003), Effects of web-based multimedia homework with immediate rich feedback on student learning in general chemistry, J. Chem. Educ., 80, 1338–1343.
15. Cooper H. and Valentine J. C., (2001), Using research to answer practical questions about Homework, Educ. Psychol., 36, 143–153.
16. Cooper H. M., Robinson J. C. and Patall E. A., (2006), Does homework improve academic achievement? A synthesis of research, 1987–2003, Rev. Educ. Res., 76, 1–62.
17. Cousins A., (2007), Gender inclusivity in secondary chemistry: A study of male and female participation in secondary school chemistry, Int. J. Sci. Educ., 29, 711–730.
18. Epstein M. L. and Brosvic G. M., (2002), Students prefer the immediate feedback assessment technique, Psychol. Rep., 90, 1136–1138.
19. Epstein M. L., Epstein B. B. and Brosvic G. M., (2001), Immediate feedback during academic testing, Psychol. Rep., 88, 889–894.
20. Evans J., (2009), OWL (Online Web-Based Learning), J. Chem. Educ., 86, 695–696.
21. Farrell J. J., Moog R. S. and Spencer J. N., (1999), A guided inquiry general chemistry course, J. Chem. Educ., 76, 570–574.
22. Freasier B., Collins G. and Newitt P., (2003), A web-based interactive homework quiz and tutorial package to motivate undergraduate chemistry students and improve learning, J. Chem. Educ., 80, 1344–1347.
23. Fynewever H., (2008), A comparison of the effectiveness of web-based and paper-based homework for general chemistry, Chem. Educ., 13, 264.
24. Handelsman J., Miller S. and Pfund C., (2007), Scientific Teaching, New York, NY, W. H. Freeeman and Company, 23–45.
25. Harris H., (2009), Electronic homework management systems: Reviews of popular systems, J. Chem. Educ., 86, 691.
26. Hendrickson S., (2009), WebAssign, J. Chem. Educ., 86, 698–699.
27. Hill C., Corbett C. and St. Rose A., (2010), Why So Few? Women in Science, Technology, Engineering and Mathematics, Washington, D.C., AAUW.
28. Keith T. Z., Diamond-Hallam C. and Fine J. G., (2004), Longitudinal effects of in-school and out-of-school homework on high school grades, School Psychol. Quart., 19, 187–211.
29. Kitts K., (2009), The paradox of middle and high school students' attitudes towards science versus their attitudes about science as a career, J. Geosci. Educ., 57, 159–164.
30. Knight J. K. and Wood W. B., (2005), Teaching more by lecturing less, Cell Biol. Educ., 4, 298–310.
31. Kost L. E., Pollack S. J. and Finckelstein N. D., (2007), Investigating the source of the gender gap in introductory physics, 2007 Physics Education Research Conference, ed. Hsu L., Henderson C. and McCullough L., 136–139.
32. Lewis S. E. and Lewis J. E., (2007), Predicting at-risk students in general chemistry: Comparing formal thought to a general achievement measure, Chem. Educ. Res. Pract., 8, 32–51.
33. Mau W. and Lynn R., (2000), Gender differences in homework and test scores in Mathematics, Reading and Science at tenth and twelfth grade, Psychol., Evol., & Gender, 2, 119–125.
34. Miller R. S., (2009), SmartWork, J. Chem. Educ., 86, 697.
35. Moore, D. S. (2007). The Basic Practice of Statistics (4th ed.), New York, NY, W. H. Freeman and Co.
36. National Academy of Sciences (NAS), National Academy of Engineering (NAE) and Institute of Medicine (IOM), (2007a), Beyond bias and barriers: Fulfilling the potential of women in academic science and engineering, Washington, D.C., The National Academies Press.
37. NAS, NAE and IOM, (2007b), Rising above the gathering storm: Energizing and employing America for a brighter economic future, Washington, D.C., The National Academies Press.
38. Nordstrom B. H., (1990), Predicting performance in freshman chemistry, American Chemical Society National Meeting, Boston, MAhttp://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/24/21/3a.pdf (accessed March 29, 2010).
39. Oliver-Hoyo M. T., Allen D., Hunt W. F., Hutson J. and Pitts A., (2004), Effects of an active learning environment: Teaching innovations at a Research I Institution, J. Chem. Educ., 81, 441.
40. Osborne J., (2003), Attitudes towards science: A review of the literature and its implications, Int. J. Sci. Educ., 25, 1049–1079.
41. Penn J., Nedeff V. M. and Gozdzik G., (2000), Organic chemistry and the internet: a web-based approach to homework and testing using the WE-LEARN system, J. Chem. Educ., 77, 227–231.
42. Poock J. R., Burke K. A., Greenbowe T. J. and Hand B. M., (2007), Using the science writing heuristic in the general chemistry laboratory to improve students' academic performance, J. Chem. Educ., 84, 1371.
43. Richards-Babb M., Drelick J., Henry Z. and Robertson-Honecker J., (2011), Online homework, help or hindrance: what students think and how they perform, J. Coll. Sci. Teach., 40, 70–82.
44. Romano J., Kromrey J. D., Coraggio J. and Skowronek, J., (2006), Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen's d for evaluating group differences on the NSSE and other surveys?, Annual Meeting of the Florida Association of Institutional Research, Cocoa Beach, FL, February 1–3, 1–33.
45. Rowley S., (2009), ARIS (Assessment, Review, and Instruction System), J. Chem. Educ., 86, 691.
46. Ryckman D. B. and Peckham P., (1987), Gender differences in attributions for success and failure situations across subject areas, J. Educ. Res., 81, 120–125.
47. Sax L., (2008), The gender gap in college: Maximizing the developmental potential of women and men, San Francisco, John Wiley & Sons, Inc.
48. Seery M. K., (2009), The role of prior knowledge and student aptitude in undergraduate performance in chemistry: a correlation-prediction study, Chem. Educ. Res. Pract., 10, 227–232.
49. Seymour A. and Hewitt N., (1997), Talking about leaving: Why undergraduates leave the sciences, Boulder, CO, Westview Press.
50. Shepherd T. D., (2009), Mastering Chemistry, J. Chem. Educ., 86, 694.
51. Smist J. M. and Owen S. V., (1994), Explaining science self-efficacy, Annual Meeting of the American Educational Research Association, New Orleans, LA, April 5–8, 2–13.
52. Spencer H. E., (1996), Mathematical SAT test scores and college chemistry grades, J. Chem. Educ., 73, 1150–1153.
53. Tobias S., (1990), They're not dumb, they're different: Stalking the second tier, Arizona, USA, Research Corporation.
54. Turner R. C. and Lindsay H. A., (2003), Gender differences in cognitive and noncognitive factors related to achievement in organic chemistry, J. Chem. Educ., 80, 563–568.
55. Varma-Nelson P. and Coppola B. P., (2004), Team learning, in Chemists' Guide to Effective Teaching, ed. Pienta N. J., Cooper M. M. and Greenbowe T. J., Upper Saddle River, NJ, Pearson, 155–169.
56. Wagner P., Schober B. and Spiel C., (2008), Time investment and time management: An analysis of time students spend working at home for school, Educ. Res. Eval., 14, 139–153.
57. Williams N. A., Bland W. and Christie G., (2008), Improving student achievement and satisfaction by adopting a blended learning approach to inorganic chemistry, Chem. Educ. Res. Pract., 9, 43–50.
58. Xu J., (2005), Purposes for doing homework reported by middle and high school students, J. Educ. Res., 99, 46–55.
59. Xu J., (2006), Gender and homework management reported by high school students, Educ. Psychol., 26, 73–91.
60. Xu J. and Corno, L., (2006), Gender, family help, and homework management reported by rural middle school students, J. Res. Rural Educ., 21, 1–13.
61. Xu J., (2007), Middle-school homework management: More than just gender and family involvement, Educ. Psychol., 27, 173–189.
62. Zhao N., (2009), WileyPLUS with CATALYST, J. Chem. Educ., 86, 692–693.

---