Structured learning environments are required to promote equitable participation

Abstract

It is critical that we understand and address features of learning environments that encumber students historically underrepresented in STEM fields. Here we consider social elements of group work that can either support or impede learning. We tracked gender-bias in student–teaching assistant (TA) interactions in 184 small groups across 27 introductory chemistry laboratories in fall 2017. We demonstrate that in some environments male students interacted with TAs disproportionately more than female students. To promote verbal participation of women in introductory chemistry courses, we advocate for improved TA training programs that teach a host of equitable teaching strategies to enhance the climate of the classrooms and consequently, improve learning. Fostering a structured, inclusive classroom environment is the first step towards achieving equity more broadly across STEM.

---

Introduction

Although women are increasingly enrolling in Science, Technology, Engineering, and Mathematics (STEM) majors at the university level, women still represent a minority of senior positions, are more likely to leave the discipline, are promoted more slowly, and—according to current trends—may not reach equal representation within the next century (Sciences et al., 2007; Shaw and Stanton, 2012; Holman et al., 2018). The specific features that influence the uneven advancement of men and women in STEM can vary based on many factors, complicating attempts to encourage female persistence at scale.

While the attrition of women is most pronounced at the postdoctoral level (Martinez et al., 2007), multiple studies have demonstrated the presence of a gender achievement gap in undergraduate STEM lecture courses [e.g., biology (Eddy et al., 2014; Ballen et al., 2017a, 2017b), physics (Lorenzo et al., 2006; Koester et al., 2016), chemistry (Rovner, 2014; Koester et al., 2016), and mathematics (McGraw et al., 2006)]. However, Matz et al. (2017) observed no such gap in laboratory courses across five large institutions. This lack of gender-biased performance in laboratory courses may be due to low-stakes assessments (Ballen et al., 2017a, 2017b), increased student engagement (Holmes, 1992; Guzzetti and Williams, 1996), or improved attitudes toward the class (Crombie et al., 2003). To our knowledge, demographic gaps in student outcomes other than grades have never been measured in laboratory environments, where teaching assistants (TAs) typically guide students through hands-on activities that reinforce lecture content.

We expect TAs to oversee students as they complete laboratory assignments; that they know how to employ equitable teaching strategies that maximize access to learning for all; and that they encourage and manage the participation of all students, not just the ones who are already engaged, even though TAs have little experience or formal training in teaching (Tanner, 2013). It is critical that we monitor the impact of classroom dynamics on learning and performance, and provide TAs with the training so they can design structured learning environments in which participation is expected from all students. Negative experiences early in a student's college degree have the potential to shape major and career choices, and self- and peer- recognition as competent scientists (Cech et al., 2011).

The first science course faced by many undergraduates is chemistry, a discipline known nationally as a gateway subject to other STEM fields (Koester et al., 2016). Undergraduate chemistry classrooms have recently achieved numerical gender equity, as approximately 50% of American chemistry majors are female (National Science Foundation, 2017). This may lead to the assumption that women no longer face gender-specific obstacles in the chemistry classroom. However, a well-documented gender achievement gap remains in undergraduate chemistry lecture courses (Rovner, 2014; Koester et al., 2016). This gap is not present in chemistry lab courses, suggesting that a more practical, ‘hands-on’ class may not disadvantage female students (Matz et al., 2017). Previous studies have focused on grades as the central measure of gender equity, but recent work has identified in-class verbal participation as an important metric of student success (Eddy et al., 2014).

In fall 2017, hundreds of University of Minnesota students enrolled in introductory chemistry laboratories. To quantify one component of classroom equity, we tracked gender-bias in TA interactions with students in the laboratory at three different scales: (1) from total interactions summed across all laboratory sections; (2) from 27 individual introductory science laboratory sections; and (3) from 184 small groups with varying gender ratios within those laboratory sections. We hypothesized that the laboratory environment would reduce barriers to participation, and that the gender gaps in participation observed in large lecture halls (e.g., Eddy et al., 2014) would not be present in these environments.

Experimental

Lab structure

We conducted observations in the introductory chemistry labs at the University of Minnesota. Chemistry I is a lecture and lab course focusing on basic chemical principles designed for STEM and non-STEM majors. Chemistry II is an identically structured course that delves into further detail and includes more complex laboratory procedures. Students voluntarily enroll in a lecture section taught by a professor and one of many associated lab sections taught by a graduate or undergraduate student TA. In both courses, lectures typically have in excess of 300 students. Lab sections are much smaller, with a maximum of 28 students.

Within each lab section students are further divided into groups, which remain constant throughout the semester. Groups are assigned at the TA's discretion. Some TAs place students based on favorite color or TV show as determined by a brief questionnaire, while others place students randomly. All lab experiments and some assignments are done in this group, and groups are never changed or mixed unless a student drops out of the course. Labs meet for three hours once per week. During class TAs move freely around the lab while students work in their groups. Students may approach the TA with questions, or the TA might approach a group and ask questions about their lab procedure.

Observations

A team of undergraduate observers collected data across 27 chemistry sections over fall 2017. Observers introduced themselves to TAs at the beginning of class without directly revealing the data they would collect. Observers also reassured TAs that their performance would not be evaluated so as to avoid influencing their teaching style. During class, the observer moved throughout the lab to record student–TA interactions without directly interacting with any students. Here an ‘interaction’ is defined as a verbal exchange between a student and the TA that is not (1) mandatory or (2) universal within the student group. For example, TAs commonly spoke with students to return graded lab reports or take attendance. TAs also asked groups if they needed any help, and received a general negative response from all group members at once. Neither of these instances is considered an interaction. Common examples of interactions include a student approaching the TA to ask a question, the TA posing a question to the group and receiving an answer from one student, a student asking the TA for help finding/using a piece of laboratory equipment, and a student making a joking comment to the TA about the lab procedure. For each interaction, observers recorded the apparent gender of the student and the apparent gender composition of the student's group. Gender was coded as a binary variable (0 = female, 1 = male) and group composition was recorded as the proportion of female students. In no instance was a student's apparent gender too ambiguous to be assigned by an observer. In cases where one student was interacting with the TA and a second student joined the conversation, two interactions were recorded. Prior to beginning observations, we conducted approximately 1 hour training sessions for observers to characterize classroom participation as broad types of interactions that occur over a class period, which were eventually further collapsed into a single type of interaction (“Circulating Instructor Question or Comment”) outlined by previous studies (Ballen et al., 2017a, 2017b; Ballen et al., 2018). At the conclusion of the training, we found inter-rater reliability was well within acceptable range among observers’ ability to identify ‘Circulating Instructor Questions or Comments’ (Cohen's kappa > 0.90). Because the vast majority (2950 of 3019, or >97.5%) of interactions were classified as “Circulating Instructor Question or Comment” due to the nature of the lab courses, all analyses consider only this type of interaction. Observers recorded all interactions that occurred during the first hour of lab, and only lab sections that were observed on three occasions were included in analyses. After we completed observations, we obtained additional institutional data, with which we calculated sex composition of each lab section. We also obtained the sex of each TA according to institutional records.

Statistical analyses

We employed a series of analyses to evaluate whether women participated to the extent one would expect based on the proportion present at different scales of resolution, following approaches taken by Eddy et al. (2014), Ballen et al. (2017a, 2017b), and Ballen et al. (2018). After checking all assumptions required for a t-test, we summed the total amount of student–TA interactions across all lab sections, and performed a two-tailed t test to compare the number of women who participated to those present according to total enrollment information. Second, we summed the total amount of student–TA interactions across all small groups in each laboratory section, and examined how the amount of women who participated compared to expectations based on enrollment in the laboratory section using an exact binomial test for goodness of fit. Third, we tested whether the proportion of student–TA interactions that were female in a small group was similar to the expected proportion based on the percentage of women in the group (0%, 25%, 33%, 50%, 66%, 75%, 100% female) using exact binomial tests. For instance, in a group with two men and two women, we tested whether the number of interactions between the TA and female group members significantly deviated from 50% of the total interactions. We excluded data from single-gender groups from this analysis, as only one type of student–TA interaction is possible in each group. To be included in the analysis, observers recorded at least 15 interactions between a student and the laboratory section TA.

Results

When we compared the total proportion of student–TA interactions marked as female across all sections to the total proportion of female students enrolled in the sections as shown in institutional data, results indicated that the total proportion of female participation did not significantly differ from the proportion of total female enrollment (P = 0.2692). The second scale we examined was individual lab sections; we found that women participated at disproportionally low levels in five of the 27 sections, indicating that female under-participation is present in some classes despite the result of the summary statistic (Fig. 1). The 5 significant classes were taught by 5 different TAs. In no laboratory section did female students participate more than would be expected based on their numeric proportions, suggesting that, while female under-participation is not detectable in all classes, the factors responsible for under-participation appear to affect women more than men.

Fig. 1 Laboratory section-level analysis of gendered participation. Bars show the proportion of women participating over three observed laboratory periods considering the observed percentages of women enrolled in individual lab sections (black bars). Women participated less than (yellow bars) or equal to (grey bars) what would be expected given their proportions in the lab sections (*p < 0.05, **p < 0.01, ***p < 0.001). At this scale, we observed no instances in which women participated more than expected based on enrollment.

At the finest scale of resolution, we examined small group-level data to determine how varying gender compositions in lab groups might impact verbal participation (Fig. 2; Table 1, Appendix). Collectively, our results support the previously argued notion that women are more likely to participate when they are in groups with other women (Joecks et al., 2013; Sullivan et al., 2018). Though we noted TA gender for all sections, we did not obtain an adequate sample size to assess whether TA gender had a significant impact on female verbal participation.

Fig. 2 Small group composition-level analysis of gendered participation. To determine whether gender composition within student groups affected verbal participation, we examined female participation as a function of small group gender ratios. Within Chemistry I and Chemistry II, lab sections are grouped by TA gender. Rows depict the percentage of women in each of the groups that we observed. Groups with fewer than 15 recorded interactions were considered to have insufficient data, and are indicated by blue boxes; pink boxes indicate when women participated approximately equally to the proportion of female group members; green boxes indicate when women participated more than would be expected based on the proportion of female group members; yellow boxes show when women participated less than expected given the group's proportion of female group members (*p < 0.05, **p < 0.01, ***p < 0.001). We chose 15 interactions as our lower boundary after the lab observers came to consensus that it provides adequate data to quantify differences in participation.

Table 1 Exact binomial tests of verbal participation and gender compositions in lab groups

Lab section	Female interactions	Total interactions	p-value	%Female in group
1				0
2				0
3	0	1	N/A	0
4				0
5				0
6	0	36	N/A	0
7	0	62	N/A	0
8	0	61	N/A	0
9				0
10				0
11				0
12				0
13	0	25	N/A	0
14	0	9	N/A	0
15				0
16				0
17				0
18				0
19				0
20	0	4	N/A	0
21				0
22				0
23				0
24				0
25	0	36	N/A	0
26				0
27				0
1				25
2	4	19	1	25
3				25
4				25
5				25
6				25
7				25
8	4	17	1	25
9				25
10	8	63	0.02781	25
11				25
12				25
13	3	10	N/A	25
14				25
15				25
16	0	8	N/A	25
17				25
18				25
19				25
20	10	24	0.09447	25
21				25
22				25
23	6	16	0.2531	25
24				25
25				25
26				25
27				25
1				33
2	1	2	N/A	33
3	1	1	N/A	33
4	2	2	N/A	33
5				33
6	2	15	0.1671	33
7				33
8				33
9				33
10	1	17	0.01771	33
11				33
12				33
13				33
14				33
15				33
16				33
17	2	4	N/A	33
18	0	5	N/A	33
19	1	3	N/A	33
20	1	1	N/A	33
21				33
22	3	26	0.02042	33
23	0	11	N/A	33
24	0	1	N/A	33
25	5	16	1	33
26				33
27	2	4	N/A	33
1	26	58	0.5118	50
2	25	46	0.6587	50
3	4	15	0.1185	50
4	29	72	0.1249	50
5	32	92	0.004609	50
6	13	47	0.003088	50
7	21	48	0.4709	50
8				50
9	13	29	0.7111	50
10	61	195	1.83 × 10^−7	50
11	16	41	0.211	50
12	9	39	0.001065	50
13	33	63	0.8013	50
14	33	71	0.6353	50
15	19	34	0.6076	50
16	30	57	0.7914	50
17	49	81	0.07479	50
18	39	81	0.8243	50
19	17	39	0.5224	50
20	10	20	1	50
21	37	68	0.5446	50
22	8	28	0.0357	50
23	3	16	0.02127	50
24	1	12	N/A	50
25	11	21	1	50
26	5	7	N/A	50
27	12	20	0.5034	50
1	8	12	N/A	66
2	6	7	N/A	66
3	15	22	1	66
4	0	1	N/A	66
5	5	11	N/A	66
6				66
7				66
8	16	18	0.04563	66
9	20	40	0.04377	66
10	12	23	0.1872	66
11	21	24	0.02945	66
12	6	8	N/A	66
13	4	12	N/A	66
14				66
15	4	12	N/A	66
16	2	3	N/A	66
17	11	18	0.6286	66
18	15	22	1	66
19	18	23	0.2735	66
20				66
21				66
22	15	23	1	66
23	7	10	N/A	66
24	10	19	0.2315	66
25	2	4	N/A	66
26	12	15	0.4135	66
27	4	5	N/A	66
1				75
2	1	1	N/A	75
3	1	1	N/A	75
4	4	10	N/A	75
5	12	13	N/A	75
6				75
7				75
8	26	34	1	75
9	33	50	0.1436	75
10				75
11	36	56	0.08775	75
12	40	76	2.64 × 10^−5	75
13	2	4	N/A	75
14	6	10	N/A	75
15	25	35	0.696	75
16				75
17				75
18				75
19	13	16	0.7748	75
20	28	38	0.8519	75
21	43	56	0.8777	75
22	24	26	0.04131	75
23	0	3	N/A	75
24	33	38	0.1314	75
25	5	11	N/A	75
26	36	49	0.8689	75
27	27	34	0.6931	75
1	14	14	N/A	100
2	3	3	N/A	100
3	16	16	N/A	100
4	3	3	N/A	100
5				100
6	37	37	N/A	100
7	27	27	N/A	100
8				100
9	16	16	N/A	100
10				100
11				100
12	11	11	N/A	100
13				100
14	23	23	N/A	100
15	1	1	N/A	100
16				100
17	28	28	N/A	100
18	1	1	N/A	100
19	11	11	N/A	100
20	22	22	N/A	100
21	10	10	N/A	100
22	5	5	N/A	100
23	18	18	N/A	100
24	8	8	N/A	100
25				100
26	17	17	N/A	100
27	6	6	N/A	100

---

Discussion

In recent decades, empirical education research has led to a dramatic paradigm shift in ideas about how we should prepare future faculty—in this case TAs—for teaching science (O'Neal et al., 2007; Meadows et al., 2015). Early calls for reform pointed to the fact that the majority of 2- and 4-year colleges and universities offered no formal training for graduate teaching assistants (Rushin et al., 1997). Research into programs that do offer TA training shows they effectively promote the use of evidence-based teaching practices and equitable techniques in graduate-led laboratories, improve TAs’ attitudes toward and confidence in teaching, and improve undergraduate performance (Tanner and Allen, 2006; Gardner and Jones, 2011; Marbach-Ad et al., 2012).

Box 1. Equitable teaching strategy: multiple hands, multiple voices

TA training programs should teach strategies that foster an inclusive classroom environment, such as encouraging equal visibility of students regardless of their identity and background. One way to accomplish this is through a technique called ‘multiple hands, multiple voices’ in which a TA or instructor communicates a “multiple hands” expectation, waits until they see three to five hands in the air, and then calls on the last person who put their hand up and works backwards. One of the most common reasons people do not ask questions is because it takes a long time to think of a question, or they cannot work up the nerve (Carter et al., 2017). Therefore, the last person who raises a hand may be the one who most needs encouragement to speak.

Within teaching training programs, which course or training elements generate positive results for students—in terms of performance, engagement, participation, or inclusion? In the larger lecture hall, we know that instructors who take a podium-planted, hands-off approach to teaching, and who attempt to push through as much material as possible in a single class period, maintain or intensify incoming demographic gaps in performance (Haak et al., 2011; Ballen et al., 2017a, 2017b). We also know that students are more likely to remember and enjoy material if they engage in brief structured activities, and work collaboratively and cooperatively (Springer et al., 1999; Prince, 2004). We suspect the same is true of the laboratory-based classroom. And, while empirically sound resources for promoting inclusive teaching exist (summarized in Tanner, 2013), future research will profit from the development of an accessible ‘toolkit’ specific to TAs in a laboratory setting (Box 1 and Box 2).

Box 2. Equitable teaching strategy: a diversity of role models

While we are not advocating eliminating male TAs, prior work (Cotner et al., 2011) has demonstrated the impact of role models on student confidence; as such, our colleagues could implement low-cost techniques to communicate and encourage broad engagement in science. For example, actively seeking and sharing stories of discovery that feature women and minorities can make science feel more inclusive and lower barriers to participation (Chamany et al., 2008). Similarly, incorporating personal narratives of counter-stereotypical scientists into course materials may alter how students perceive who does science and enhance science self-identity (Schinske et al., 2016).

Undergraduate chemistry students stand to benefit from TA-led inclusive teaching practices, and future research will profit from additional quantitative research on the impacts of changing practices, along with qualitative research that focuses on individual students’ perceptions of climate and reasons for participating—or not. Chemistry majors face laboratory classrooms with comparable numbers of male and female students and without quantifiable gender achievement gaps (Matz et al., 2017; National Science Foundation, 2017), but attrition of female scientists continues further along the academic pipeline (Martinez et al., 2007). This may be due in part to the gender participation gap we have observed in introductory chemistry labs at the University of Minnesota. Formal TA training programs focused on structuring classes to increase equitable teaching strategies may reduce the participation gap in chemistry lab classes, supporting students’ science identities and reducing the attrition of female scientists (Schinske et al., 2016).

Conclusions

In our population, we show that in some chemistry lab sections male students interacted with TAs disproportionately more than women, but we did not observe any cases in which women interacted more. We also found that women were more likely to participate when more women were in their small group (Fig. 2). To broaden verbal participation of students in introductory chemistry courses, we advocate for improved TA training programs that teach a host of equitable teaching strategies to enhance the climate of the classrooms and consequently, improve learning. Educating instructors on how to craft structured learning environments that promote participation from all students is the first step towards achieving equity more broadly across STEM.

Conflicts of interest

There are no conflicts of interest to declare.

Appendix

References

1. Ballen C. J., Salehi S. and Cotner S., (2017a), Exams disadvantage women in introductory biology, PLoS One, 12, 1–14.
2. Ballen C. J., Wieman C. E., Salehi S., Searle J. B. and Zamudio K. R., (2017b), Enhancing diversity in undergraduate science: self-efficacy drives performance gains with active learning, CBE Life Sci. Educ., 16, ar56.
3. Carter A., Croft A., Lukas D. and Sandstrom G., (2017), Women's visibility in academic seminars: women ask fewer questions than men, arXiv preprint arXiv:1711.10985.
4. Cech E., Rubineau B., Silbey S. and Seron C., (2011), Professional role confidence and gendered persistence in engineering, Am. Soc. Rev., 76, 641–666.
5. Chamany K., Allen D. and Tanner K., (2008), Making biology learning relevant to students: integrating people, history, and context into college biology teaching, CBE Life Sci. Educ., 7, 267–278.
6. Cotner S., Ballen C., Brookes D. and Moore R., (2011), Instructor gender and student confidence in the sciences: a need for more role models? J. Coll. Sci. Teach., 40, 96–101.
7. Crombie G., Pyke S. W., Silverthorn N., Jones A. and Piccinin S., (2003), Students' perceptions of their classroom participation and instructor as a function of gender and context, J. High. Educ., 74, 51–76.
8. Eddy S. L., Brownell S. E. and Wenderoth M. P., (2014), Gender gaps in achievement and participation in multiple introductory biology classrooms, CBE Life Sci. Educ., 13, 478–492.
9. Gardner G. E. and Jones M. G., (2011), Pedagogical preparation of the science graduate teaching assistant: challenges and implications, Sci. Educ., 20, 31.
10. Guzzetti B. J. and Williams W. O., (1996), Gender, text, and discussion: examining intellectual safety in the science classroom, J. Res. Sci. Teach., 33, 5–20.
11. Haak D. C., HilleRisLambers J., Pitre E. and Freeman S., (2011), Increased structure and active learning reduce the achievement gap in introductory biology, Science, 332, 1213–1216.
12. Holman L., Stuart-Fox D. and Hauser C. E., (2018), The gender gap in science: how long until women are equally represented? PLoS Biol., 16, e2004956.
13. Holmes J., (1992), Women's talk in public contexts, Discourse Soc., 3, 131–150.
14. Joecks J., Pull K. and Vetter K., (2013), Gender diversity in the boardroom and firm performance: what exactly constitutes a “critical mass?”, J. Bus. Ethics, 118, 61–72.
15. Koester B. P., Grom G. and McKay T. A., (2016), Patterns of gendered performance difference in introductory STEM courses, arXiv preprint arXiv:1608.07565.
16. Lorenzo M., Crouch C. H. and Mazur E., (2006), Reducing the gender gap in the physics classroom, Am. J. Phys., 74, 118–122.
17. Marbach-Ad G., Schaefer K. L., Kumi B. C., Friedman L. A., Thompson K. V. and Doyle M. P., (2012), Development and evaluation of a prep course for chemistry graduate teaching assistants at a research university, J. Chem. Educ., 89, 865–872.
18. Martinez E. D., Botos J., Dohoney K. M., Geiman T. M., Kolla S. S., Olivera A., Qiu Y., Rayasam G. V., Stavreva D. A. and Cohen-Fix O., (2007), Falling off the academic bandwagon: women are more likely to quit at the postdoc to principal investigator transition, EMBO Rep., 8, 977–981.
19. Matz R. L., Koester B. P., Fiorini S., Grom G., Shepard L., Stangor C. G., Weiner B. and McKay T. A., (2017), Patterns of gendered performance differences in large introductory courses at five research universities, AERA Open, 3, 1–12.
20. McGraw R., Lubienski S. T. and Strutchens M. E., (2006), A closer look at gender in NAEP mathematics achievement and affect data: intersections with achievement, race/ethnicity, and socioeconomic status, J. Res. Math. Educ., 37, 129–150.
21. Meadows K. N., Olsen K. C., Dimitrov N. and Dawson D. L., (2015), Evaluating the differential impact of teaching assistant training programs on international graduate student teaching, Can. J. High. Educ., 45, 34.
22. National Science Foundation D. o. S. R. S., Women, minorities, and persons with disabilities in science and engineering, Arlington, VA: National Science Foundation, 2017.
23. O'Neal C., Wright M., Cook C., Perorazio T. and Purkiss J., (2007), The impact of teaching assistants on student retention in the sciences: lessons for TA training, J. Coll. Sci. Teach., 36, 24.
24. Prince M., (2004), Does active learning work? A review of the research, J. Eng. Educ., 93, 223–231.
25. Rovner S. L., (2014), Women faculty positions edge up, Chem. Eng. News, 92, 41–44.
26. Rushin J. W., Saix J. D., Lumsden A., Streubel D. P., Summers G. and Bernson C., (1997), Graduate teaching assistant training: a basis for improvement of college biology teaching & faculty development? Am. Biol. Teach., 59, 86–90.
27. Schinske J. N., Perkins H., Snyder A. and Wyer M., (2016), Scientist spotlight homework assignments shift students' stereotypes of scientists and enhance science identity in a diverse introductory science class, CBE Life Sci. Educ., 15, ar47.
28. Sciences N. A. o., Engineering N. A. o. and Medicine I. o., Beyond bias and barriers: fulfilling the potential of women in academic science and engineering, Washington, DC, 2007.
29. Shaw A. K. and Stanton D. E., (2012), Leaks in the pipeline: separatin demographic inertia from ongoing gender differences in academia, Proc. R. Soc. B, 279, 3736–3741.
30. Springer L., Stanne M. E. and Donovan S. S., (1999), Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a meta-analysis, Rev. Educ. Res., 69, 21–51.
31. Sullivan L. L., Ballen C. J. and Cotner S., (2018), Small group gender ratios impact biology class performance and peer evaluations, PLoS One, 13,  DOI:10.1371/journal.pone.0195129.
32. Tanner K. D., (2013), Structure matters: twenty-one teaching strategies to promote student engagement and cultivate classroom equity, CBE Life Sci. Educ., 12, 322–331.
33. Tanner K. and Allen D., (2006), Approaches to biology teaching and learning: on integrating pedagogical training into the graduate experiences of future science faculty, CBE Life Sci. Educ., 5, 1–6.

---