Planning and evaluating chemistry outreach: a case study of one collegiate group's approach

Abstract

Chemistry outreach, a type of informal science education commonly practiced by college students, has primarily been studied by looking at individuals' approaches and perspectives. However, it is much more common for college students to plan and conduct chemistry outreach events as part of a group/club/chapter, not independently. In this case study, we looked at the planning and evaluation processes for a single student group conducting a chemistry outreach event. Using pre- and post-outreach event focus groups, we sought to understand how the group was structured and how they collectively planned and implemented their outreach event. Using the Framework for Effective Chemistry Communication and Communities of Practice, we identified the steps the group took to plan, implement, and evaluate their event, as well as the group's structure and member dynamics. Findings showed a misalignment between goals used to plan the outreach event, and those evaluated when reflecting on the event's success. Additionally, the Communities of Practice framework was able to highlight the membership structure underlying the group's planning process, as well as areas for growth and improvement. Overall, findings provide further support that collegiate chemistry organizations/student groups need more resources and training to successfully conduct chemistry outreach.

---

Background

Informal science education

Informal science education (ISE) involves learning experiences outside of a formal classroom setting; these informal learning environments are quite diverse and can include museums, everyday experiences, afterschool programs, and even social media. In fact, informal learning experiences account for the majority of any individual's learning—only approximately 19% of a student's K-12 education occurs in formal classroom settings, with the percentage significantly decreasing as you move toward postsecondary education and employment in the workforce (National Research Council, 2009).

Over the past 15 years, ISE has grown through increased scholarly investigations and grant funding, particularly through the Advancing Informal STEM Teaching (AISL) program from the National Science Foundation (National Science Foundation, n.d.). Some scholars have tried to bridge formal and informal learning to maximize the impact on students, including through service-learning experiences. Service learning typically involves formal classroom instruction with the end goal of students applying their formal knowledge to implement informal learning experiences for the public. Bowe et al. (2023) examined the effects of service-learning experiences on students’ sense of competence, self-efficacy, relatedness, and autonomy in introductory chemistry courses. Morgan Theall and Bond (2013) used service-learning experiences where students planned informal science events as part of their formal chemistry laboratory classes. While these chemistry-specific examples provide insights into informal learning experiences, they still rely on formal learning experiences where the informal application of material is a graded component of the course. In chemistry, however, informal science education is more commonly practiced as voluntary, community-service experiences, rather than compulsory as part of a course; such voluntary experiences are typically labeled “chemistry outreach” experiences.

Chemistry outreach

While chemistry outreach is a common practice across the chemical enterprise, both for students and professionals, it has not been studied by many researchers. A report by the National Academies of Sciences, Engineering, and Medicine (2016) provided insights into informal chemistry education experiences and practices, with a push for more scholarly investigations. However, this report focused solely on the experiences and insights of professionals conducting chemistry outreach, not collegiate students who report reaching over 1 million people every year through their voluntary chemistry outreach events (Connelly, 2015; Pratt, 2017).

In addressing this gap, a collection of publications from Pratt and Yezierski (2018–2019) focused exclusively on the chemistry outreach practices of collegiate students and showed the need for further study in this area. The first article looked at the purposes of conducting chemistry outreach and the most prevalent demonstrations/activities associated with chemistry outreach events (Pratt and Yezierski, 2018a). Findings showed that undergraduate students and faculty/staff have different ideas regarding the purpose of chemistry outreach. However, audience learning was the most prevalently discussed purpose for both populations. In a follow-up study, the authors investigated students’ content knowledge related to common outreach demonstrations (Pratt and Yezierski, 2018b). Findings suggested that students held misunderstandings and non-canonical ideas, despite being primarily third- and fourth-year chemistry students approaching graduation, suggesting that audience learning at these events may be shallow and/or inaccurate. This was further supported by findings related to the teaching and learning ideas espoused by these collegiate students for their outreach events (Pratt and Yezierski, 2019a); the collegiate students discussed shallow and/or unsupportive teaching ideas, including that it is acceptable to teach inaccurate content when the audience is young. Such misalignment between their chemistry outreach goals, their content knowledge/understanding, and their beliefs about teaching and learning are concerning but not surprising given that these students report minimal training experiences and little-to-no interactions with faculty advisors (Pratt and Yezierski, 2019b); in fact, students commonly discussed “winging it” and/or using trial and error when conducting outreach events.

The aforementioned studies provided the first scholarly insights into chemistry outreach practices for collegiate students. However, these studies focused on individual student ideas and experiences, and did not consider the contextual nature of their outreach events; it is more common for students to plan and implement chemistry outreach events as part of a group/club/organization, as opposed to individually. The study described herein addresses this gap and focuses on understanding one student group and how they approached planning and implementing a chemistry outreach event.

Communities of Practice

Communities of Practice (CoP) was employed in this study to understand the structure and group dynamics of the sampled student organization, as they planned their outreach event. CoPs are groups of people with shared interests, goals, concerns, and/or passions with a focus on gaining more knowledge within their domain through interactions as a community. Wenger described a CoP as “groups of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” (Wenger et al., 2002, p. 4). In a more recent publication, CoP was defined as “a learning partnership among people who find it useful to learn from and with each other about a particular domain. They use each other's experience of practice as a learning resource, and they join forces in making sense of and addressing challenges they face individually or collectively” (Wenger et al., 2011, p. 9). Both definitions allude to three main components that define a CoP: the domain, the community, and the practice. The domain is the area of shared interest that the group focuses on. The community is the interactions of the members as they gain knowledge in their domain. The practice is the use of the accumulated knowledge of the domain by CoP members. Fig. 1 summarizes the components and elements of a CoP and is described in detail below.

Fig. 1 Overview of Communities of Practice.

At the most basic level, learning in a CoP involves competence and experience. The competence of a CoP is built and established over time, while experience refers to the experiences of members in the world. Both competence and experience are measures of success for a CoP, depending on their goals (i.e., higher competence = higher success). Learning in a CoP takes place at the intersection of competence and experience. Inside a CoP, two major processes occur: participation (acting and interacting of CoP members) and reification (producing artifacts for the CoP). Membership has varying degrees of participation—core, active, and peripheral members—with sequentially decreasing levels of participation (i.e., core members include coordinators or designated leaders, while peripheral members are generally those who are new/less knowledgeable and do not participate in many events). That said, member participation shifts as the community's focus or practice evolves (e.g., some core members become less involved as active members, while some peripheral members become more involved as active members). A great deal of growth within a CoP relies on legitimate peripheral participation and interactions between those with less knowledge and those with more knowledge, resulting in learning and growth (Lave and Wenger, 1991).

Wenger also discusses elements that describe how a CoP functions; these elements are actions that allow for the growth of the CoP, and include leadership, membership, connectivity, events, artifacts, and learning projects. There are multiple forms of leadership, and these roles change from person to person over time. Regardless, CoPs depend on leadership to help develop the community. Membership is also needed to ensure the growth of a CoP. Processes that allow new members to cycle into the community help to maintain what Wenger describes as “critical mass” in a CoP (Wenger, 2000). This means that there are not so many members that the focus of the CoP is lost, but there are enough members to continue towards their goal. Connectivity refers to encouraging interactions between members of a CoP in multiple ways to share ideas. Public events are meant to bring the CoP together, ensure connectivity, and apply their new knowledge. As a CoP establishes experience, they produce artifacts in the form of documents, tools, stories, etc. These artifacts change and/or are maintained based on the community's needs. Lastly, learning projects are used to deepen the extent of a CoPs knowledge and further push their practice forward.

It is also important to discuss the boundaries of a CoP. Boundaries are where “competence and experience tend to diverge [resulting in] a boundary interaction” (Wenger, 2000). Wenger describes three types of boundary ideas: brokering, boundary objects, and boundary interactions. Brokering is where individual CoP members purposefully reach out to another CoP to share/gain knowledge (i.e., formalized knowledge sharing between CoPs). Boundary objects are resources that facilitate such communication and knowledge sharing, unique to individual CoPs. These objects can take the form of documents, terminology, stories, rules, and unspoken norms. Regardless, these objects help CoPs with connectivity and bringing new members into the community by providing resources and tools to facilitate learning and growth; these objects can also be used in multiple ways, not only within CoP development but also as part of brokering with other CoPs. Boundary interactions are more informal knowledge-sharing activities, typically involving individual members interacting with others outside of their CoP (e.g., attending a scientific conference). Such interactions allow new knowledge to be gained and brought back to the CoP. Overall, all parts and elements of a CoP must coalesce for a CoP to be effective and learn as a community, as summarized in Fig. 1. Examples of successful CoPs in the chemistry community that focus on formal learning environments include the Interactive Online Network of Inorganic Chemists (IONiC) (Raker et al., 2020), Organic Chemistry Educational Resources (OrganicERs) (Leontyev et al., 2020), and the Analytical Sciences Digital Library (ASDL) (Larive and Kuwana, 2007).

Collegiate organizations as Communities of Practice

Previous research has used CoP to analyze undergraduate groups conducting chemistry outreach. In two studies by Santos-Díaz and Towns, CoP was used as a lens to interpret how collegiate organizations approach chemistry outreach. Findings suggested alignment between CoP components and the activities of a collegiate group (Santos-Díaz and Towns, 2020). However, these authors narrowed their analyses to the leadership element of a CoP and how that manifested in the studied collegiate organization. Their findings showed that students’ prior experiences impacted the boundary processes. Additionally, leaders, who often lead the planning process of chemistry outreach events, had traditional and laissez-faire leadership styles (Santos-Díaz and Towns, 2021). While other aspects of a CoP were not investigated by Santos-Díaz and Towns, they do provide anecdotal examples of how a CoP may manifest in a collegiate chemistry organization (Santos-Díaz and Towns, 2020). Together, these studies support that the CoP framework is applicable, and useful, for interpreting the leadership activities of a collegiate organization conducting chemistry outreach. However, more work is needed to further understand how the other elements and components of a CoP coincide with collegiate organization activities and whether or not it is a useful framework for future work in this area; since most chemistry outreach is conducted by groups/clubs, understanding how the group functions and makes decisions is important. Santos-Díaz and Towns focused explicitly on the leadership element, while our study provides a more holistic look at how an undergraduate organization aligns with the CoP framework; the CoP framework informed our data collection questions and was used as an analytic lens in the analysis of group dynamics.

The Framework for Effective Chemistry Communication

While the CoP framework allows for understanding about group dynamics and interactions, the Framework for Effective Chemistry Communication in Informal Environments (referred to as Framework herein) provides a lens to better understand the specific planning and evaluation processes of a student group conducting chemistry outreach. Part of the National Academies report on Effective Chemistry Communication in Informal Environments, this Framework consists of five steps to conduct and implement informal chemistry events (see Fig. 2): (1) set goals and outcomes for event, (2) identify resources and materials needed for event, (3) design and prepare for the event, (4) conduct the event, and (5) assess, reflect, and follow up on the success of the event to inform future ones (National Academies of Sciences, Engineering, and Medicine, 2016). While this Framework is straightforward and akin to Backwards Design ideas in formal education settings (Wiggins and McTighe, 2005), it provides a lens for considering how to plan, implement, and evaluate an outreach event. In this study, we used the Framework to inform our data collection questions as well as an analytic lens to interpret the specific planning and implementation process of the studied collegiate group.

Fig. 2 Overview of the framework for effective chemistry communication.

This study seeks to add to the existing literature by first considering how a collegiate organization functions, through the lens of CoP. Secondly, as previous research has examined individual goals for outreach, we seek to build on these findings by looking at the steps/process for how the studied students plan an outreach event as a collective, and the goals that guide their processes. Understanding both how the group functions and their planning and evaluation processes are necessary before interventions and resources can be developed to improve outreach practices, as previously suggested (Pratt and Yezierski, 2018a, 2018b; 2019a, 2019b).

Research questions

This study focuses on understanding a single collegiate student group conducting chemistry outreach by focusing on their group dynamics and how those dynamics result in planning and evaluation processes. As such, we seek to address the following research questions:

1. How does an individual chapter of a collegiate chemistry organization function as a Community of Practice?

2. How does an individual chapter of a collegiate chemistry organization collectively plan, implement, and evaluate a single chemistry outreach event, through the lens of the Framework for Effective Chemistry Communication for informal environments?

These highly interrelated research questions complement each other as the first question focuses on group structure and dynamics, while the second question focuses on how said group structure and dynamics manifest in specific planning and evaluation processes.

Methods

A case study methodology was adopted for this study as it helps better understand a phenomenon in a certain context, without manipulating any behaviors of the participants (Baxter and Jack, 2008). This study is an exploratory, single case study as the goal was to explore the planning and evaluation processes of a single chapter of a collegiate chemistry organization without a pre-conceived outcome (Yin, 2014). In case studies, the unit of analysis is defined as the case of the study; in the study described herein, the case is defined as the planning and evaluation experiences of a collegiate chemistry organization around a single chemistry outreach event. The case is bounded by the context of the single chemistry outreach event, organized and conducted by the student group. Our focus is on the student group's collective experiences, rather than on individual group members’ experiences, as the group collectively planned and implemented the outreach event. Researchers JMP and MC collectively designed the study and the interview guides used in the focus groups. The focus groups were conducted by JMP. Notably, JMP and MC have background in education research and chemistry outreach. The analysis explored the group dynamics and experiences of the student group planning, evaluating, and reflecting on their chemistry outreach event. Analyses were primarily conducted by researcher RCC. RCC has minimal experience with chemistry outreach. As such, salient patterns are more trustworthy as they result from discussions and agreements amongst those with varied expertise in outreach. Results are discussed in a context-sensitive manner looking first at the group's structure and dynamics, followed by their experiences and goals within the context of their chemistry outreach event. All researchers came to unanimous agreements about results through discussions, see additional details below.

Participants and context

Participants for this study were drawn from members of a single collegiate chemistry organization at a large university in the Northeast of the United States. At the university, 70% of the student population identifies as white, and 57% identifies as women. Study participants consisted of thirteen undergraduate students associated with the university's chemistry student organization. Due to the focus of this organization, all study participants were pure or applied chemistry majors. This specific chapter is relatively new, having been installed near the start of the COVID-19 pandemic. Unfortunately, the pandemic heavily impacted this chapter resulting in a loss of institutional and chapter knowledge, particularly around planning and implementing outreach events in their local context. As a result, the specific chemistry outreach event that bounds this case study was the first large-scale, in-person outreach event conducted by this chapter since before the pandemic. Previous outreach experiences for chapter members were predominantly virtual and/or for individual participants or a single class (i.e., smaller scale).

Study participants were recruited via convenience sampling; members of the chapter were invited to voluntarily participate in the study and were afforded opportunities to not take part during each data collection timepoint (i.e., each focus group discussion). Overall, data collection involved two focus group discussions with consented participants: one pre-outreach event and one post-outreach event. In the pre-outreach event focus group, four men and seven women participated (N = 11), with ten participants identifying as white. In the post-outreach event focus group, five men and six women participated (N = 11), with all participants identifying as white. The overall demographic composition for both the pre- and post-focus group participants mirrors that of the university, being predominately white women. Out of the thirteen participants across both data collection timepoints, three held leadership positions within the organization, including the current chapter president, the current chapter vice president, and the incoming chapter president/president-elect.

The chemistry outreach event conducted by the organization that bounds this case consisted of a hands-on component and a larger demonstration show component. Approximately 70 third-grade students from a local elementary school attended the event, hosted on the university's campus. The hands-on activities were hosted in teaching laboratory spaces in the chemistry building and involved splitting the third graders into smaller groups to conduct hands-on activities facilitated by individual chapter members. These hands-on activities included using milk, food coloring, and dish soap to discuss attraction & repulsion ideas (Colors on the Moooove!, 2014) as well as making slime using glue to discuss polymers and structure–property relationships (Time for Slime!, 2014).

The large demonstration show was held in an auditorium space in another university building and involved members of the student organization conducting demonstrations in front of the third graders as a collective audience; demonstrations included Coke and Mentos, Elephant's Toothpaste, and How to Make a Dry Ice Balloon (Mentos and Diet Coke!, 2015; Finio, 2019; Helmenstine, 2019).

Data collection

Thirteen students participated in at least one of two semi-structured, focus group discussions, both conducted by the same researcher (JMP) during the Spring 2023 semester. Focus groups were chosen as the goal was to elicit the collective ideas of the collegiate group rather than semi-structured, one-on-one interviews which would emphasize individuals’ goals and approaches. The first focus group was conducted 3 days before the outreach event and focused on the chapter's goals and planning process; it lasted approximately 28 minutes. The second focus group, conducted 4 days after the outreach event, focused on reflection and evaluating event success; it lasted approximately 24 minutes. Focus group discussions emphasized collective understanding and ideas by allowing participants to engage in back-and-forth dialogues that built off each other's ideas. The use of probing questions, clarifying questions, and silence were harnessed to ensure participants had ample opportunities to process information and share their insights without feeling pressured. Additionally, given that the studied group was a collegiate organization, the group itself already had established relationships amongst members that further added to the sense of rapport and camaraderie necessary for insightful focus group discussions. It is prudent to note that while multiple strategies were employed to support participants in adding to the discussion, not all participants verbally contributed to the focus group discussions; some did not speak at all and only provided nonverbal assents to others’ ideas (captured via field notes from the focus group facilitator and added as annotations to the transcripts). As such, while all participants engaged in the discussions to capture collective thoughts and ideas, quotations and verbalized responses were from more active participants. Pre- and post-focus group discussion guides are provided in Appendices 1 and 2, respectively. Prior to any data collection, approval was obtained from the University of Rhode Island's Institutional Review Board (IRB2223-188). Both focus group discussions were audio recorded and transcribed verbatim for analysis.

Data analysis

Two researchers (RCC and JMP) independently coded the pre- and post-outreach event focus group discussions using inductive, open coding (Patton, 2015). Open coding was used to capture the planning and reflection processes as well as the students’ explicit goals for the outreach event, including goals for their third-grade audience and for themselves as facilitators. Researchers then collaboratively compared codes and descriptions until they reached 100% agreement on an initial codebook (McAlister et al., 2017). Afterward, multiple rounds of coding and discussion occurred until the codebook was stable, all data were analyzed, and all disagreements were resolved.

After open coding, the pre-focus group data were deductively coded using previous literature on chemistry outreach (Fereday and Muir-Cochrane, 2006). Specifically, outreach event goals and purposes discussed by study participants were compared with previously reported goals from a national chemistry outreach survey (Pratt and Yezierski, 2018a). Goals identified through open coding that overlapped with previous survey findings were redefined as deductive codes; novel goals not previously reported remained as inductive codes. Given that the group planning process for chemistry outreach has not been previously studied, all codes related to planning and evaluating the event were novel and thus inductive codes. After inductive and deductive coding, thematic analysis was used to identify overarching patterns and ideas to address the research questions (Fereday and Muir-Cochrane, 2006). As part of this thematic analysis process, our theoretical frameworks were used as lenses to interpret patterns and make sense of the student group's processes. Specifically, Communities of Practice was used to understand the group's structure and function as a community working towards a common goal, and the Framework for Effective Chemistry Communication was used to understand the overall process the group used to plan, implement, and evaluate their event. Individual elements and components of CoP were compared to the codes from the analysis to determine how the CoP framework defined the studied group's dynamics. The Framework was used as an example of an effective event planning and evaluation process as it is based on research by the National Academies of Science, Engineering, and Medicine. To determine alignment with this framework, goals were coded and connected to the studied group's process. These were then compared to the framework for full analysis.

To support the trustworthiness of findings and interpretations, multiple strategies suggested for case studies were used (Lincoln and Guba, 1985; Baxter and Jack, 2008). In support of the credibility and dependability of interpretations, a third education researcher (MC) not involved with the inductive, deductive, and thematic analyses independently reviewed transcripts and summarized their overall interpretations. All researchers then came together as a team to discuss and compare interpretations to make revisions until consensus was reached. The combination of insider/outsider perspectives helped ensure interpretations were meaningful and not the result of individuals’ biases. Additionally, the inductive, deductive, and thematic analyses involved multiple researchers coding and analyzing data independently, followed by debriefing meetings to make revisions and come to consensus. MC also added to the trustworthiness of these findings by providing additional peer debriefing and scrutiny to better address biases and blind spots (Janesick, 2015). Finally, an audit trail of all coding schema and revisions was kept and reflected on between each step of the analysis process, helping support consistency in the analyses and the overall credibility of findings (Merriam and Tisdell, 2016).

Results and discussion

Research question 1: group structure and dynamic through the lens of a Community of Practice

Previous literature has used the Communities of Practice framework to conceptualize how collegiate student organizations conduct chemistry outreach (Santos-Díaz and Towns, 2020, 2021). However, this work mostly focused on individuals’ experiences and only suggested that these collegiate student organizations conducting chemistry outreach are Communities of Practice, without analyzing group structure and practices in depth. As such, we seek to add to the literature a more thorough discussion of collegiate organizations and how they function, through the lens of Communities of Practice (Fig. 1). This was done by looking for components noted earlier that comprise CoPs. As previously described, a CoP is defined by three parts: domain, community, and practice. The domain refers to the shared interest that the CoP focuses on, in this case chemistry. The community are the members, and the interactions between members, to exchange knowledge. The practice is the ways the CoP applies their knowledge, in this case through chemistry outreach. This alignment of domain, community, and practice with a collegiate organization conducting chemistry outreach is supported by previous work using the CoP framework to interpret leadership in these collegiate groups (Santos-Díaz and Towns, 2020, 2021).

Participation. First, we looked at the membership structure, given the centrality of membership to a CoP. There were clear leaders of discussion in the focus groups. Table 1 shows the distribution of contributions by focus group participant. We define a contribution as adding a new idea or point to the group discussion (i.e., not including nonverbal assents or clarifications of previous points). For example, Man 4 has 0 formal contributions during the pre-focus group; however, he nonverbally assented multiple times to other member's contributions throughout the discussion. Because these assents did not add new ideas to the discussion, they were not counted as contributions. All focus group participants listed with 0 contributions provided nonverbal assents for other points throughout the discussions. As such, we used participant's degree of participation in the focus groups as a proxy measure of their role in the organization. Looking at Table 1, some members were clearly more involved/active than others based on their contributions to the focus group discussions.

Table 1 Member contributions during focus group discussions^a

Person	Number of contributions in pre-focus group	Number of contributions in post-focus group
a Contributions do not include reiteration or clarification of a previous point, nor nonverbal assents to previous points.
Man 1 (Vice President)	1	3
Man 2	5	7
Man 3	1	0
Man 4	0	6
Man 5	n/a (not present at pre-focus group)	0
Woman 1 (Chapter President)	22	17
Woman 2 (President Elect)	15	11
Woman 3	6	2
Woman 4	2	0
Woman 5	0	0
Woman 6	0	0
Woman 7	0	n/a (not present at post-focus group)

---

Wenger (2000) discussed that the degree of participation is dependent on one's competence and experience within the context of the CoP. Competence is “established over time by the community” while experience is “within the context of a given CoP and beyond.” For the chemistry outreach event that bounds this case, all college students had similar competence as this outreach event was the first of its kind planned by this chapter. Experience is based on everyone's past experiences outside of the chapter. Using these parameters alone, all members have the same level of participation; however, when looking at the contributions to the focus group, we see varying roles within the chapter when planning the outreach event.

CoP defines three different members within a community: core, active, and peripheral. In our analysis of the chapter's membership structure, we added a level inside core members for coordinators/leaders, which Wenger mentions as examples of potential core members (Wenger, 2000). This was done as a coordinator (i.e., Woman 1/Chapter President) had significant impacts on the planning process for the outreach event, described more in-depth below as part of research question 2; she initiated the outreach event and planning process, as well as managed the larger logistics for the outreach event. Core members were defined as those who spoke and contributed during the focus group discussions, as core membership requires a higher level of engagement with community activities than active members. As such, participants who participated in the focus group discussions but only contributed through nonverbal assents or verbal agreements/clarifications were labeled as active members. Peripheral members were defined as those who were part of this chapter/community but did not participate in the focus group discussions (i.e., were not part of this study); less active/involved members were discussed multiple times during the focus group discussions, providing evidence that this chapter does have peripheral members.

CoP also includes the idea of outsiders: those who are not part of the community of practice but are interacting with them. In this case, the outsiders were defined as the third-grade students who were the attendees of the outreach event. That said, they were not observed in this study as our focus was on the planning and evaluation process of the collegiate organization itself.

Elements. The elements of CoP are actions of the community that characterize it, including leadership, membership, connectivity, events, artifacts, and learning projects (Wenger, 2000). Of these elements, only leadership, events, and membership ideas were evidenced in the focus groups.

Leadership describes the role of certain members (referred to as coordinators) to push the CoP towards its goals or purposes. These leadership roles change over time as the goals of the CoP change or adapt, and as membership changes. In the focus groups, the idea of a leader was mentioned multiple times as the drive for planning the outreach event, evidenced by Man 2:

“I feel like once [President] kind of took the charge and told us it was like going down, we all kind of like collectively just started… just throwing ideas at the wall and seeing what stuck.”

The coordinator (i.e., the Chapter President) “took charge” which led to the rest of the event planning. It was also part of the chapter's observed planning process where one coordinator took charge of the organization and delegated tasks for the outreach event (described more in-depth below). Events refer to any gathering of those within the CoP to better understand their domain and to work towards their goals, likely including recurring meetings to bring members together. During the focus groups, Woman 2 mentioned using weekly chapter meetings as a time to plan for the outreach event, “Yea we probably spent a whole chapter [meeting] finding like small experiments that also like scaled-up experiments for the demo [show].” Membership refers to the processes for adding members to their community. This was also discussed during the pre-focus group. Woman 1 suggested, “We can take photos of us doing [the outreach event] and stuff. I think that it would even be good to like… in terms of like recruitment and getting [new] people.” She positioned the outreach event as a potential recruitment activity that would potentially help add new members to their community.

More connections were made to these processes by the elements the studied CoP did not evidence. Elements that were not mentioned, or were missing from their current community activities, were connectivity, artifacts, and learning projects. Connectivity refers to creating relationships between those who need help and those who can help within a CoP, generally based on experience and knowledge. In this case, we considered this as the relationships between the faculty advisor and the collegiate chapter members, as well as the more experienced members supporting less experienced members. Using the definition of a CoP, the faculty advisor may not be considered part of the CoP in this case based on discussions with the chapter that revealed minimal interactions between the chapter advisor and the students. Woman 1 mentioned a single meeting with the faculty advisor, but no other help or support from them. This is not shocking given earlier work on chemistry outreach practices that indicate students work relatively independently, despite university requirements for faculty advisors (Pratt and Yezierski, 2019b). In terms of connectivity based on experience, all the members had similar amounts of experience regarding chemistry outreach due to the impact of the COVID-19 pandemic and the loss of institutional and community knowledge. Therefore, there is a lack of evidence that this chapter/community has the connectivity element. However, students mentioned wanting more connections and support from faculty in their department, especially when evaluating their event. Woman 1 admitted:

“I think the overall perception in this department of [chemistry] outreach isn’t that strong so I was hesitant to even mention the [outreach event] to other faculty members, but I think in the future just more faculty support, like financial and otherwise would be helpful.”

Clearly, there are some gaps in the communication and connectivity between faculty and the members of this CoP/chapter. This could be a result of the students not reaching out to those with more experience, as suggested by the quote above. Additionally, the desire for more mentorship and support from faculty and chapter advisors echoes previous findings regarding collegiate student chemistry outreach practices (Pratt and Yezierski, 2019b).

Artifacts are symbols, documents, tools, processes, etc. that are maintained by the CoP for future use. Many members of this specific CoP described wanting such artifacts, during both the planning and evaluation discussions. One example is from Man 4 as he described what the group wanted for future outreach events:

“…a kind of set checklist…to set things up would be…a lot more streamlined in the future…Step like 1, make sure you have all this in order, here's who to talk to, this is the first step. Second step…make sure you have your materials. Just kind of a very…rigorous process for [outreach] instead of kind of make-it-up as we go along.”

Based on this chapter's relative newness as an organization and the effects of COVID-19, the chapter had little-to-no precedence for conducting in-person, large chemistry outreach events. This led to a “make-it-up as we go” approach from the student facilitators, much like the “winging it” approach previously reported (Pratt and Yezierski, 2019b). The final element of a CoP is learning projects, which are actions that serve to look at the growth of the CoP and the progress the CoP has made toward its goals. In this case of chemistry outreach, an example of a learning project would be training for outreach, with those with more experience teaching those with less experience; such training would support community learning and growth, as well as the lacking connectivity element discussed previously. Learning projects and artifacts also go hand-in-hand, and the students discussed wanting support, resources, tools, and/or training for chemistry outreach that would fill the gaps in their collective knowledge. Like the lack of connectivity, these ideas echo previous research where collegiate chemistry outreach practitioners want procedures with age-appropriate explanations or want information about how to successfully conduct training for outreach (Pratt and Yezierski, 2019b). These results are not new, nor unexpected, but reiterate the need for training to support chemistry outreach.

Boundaries. Boundaries of a CoP are any interactions or overlap of the CoP with other entities. This can include other CoPs as well as individuals/outsiders. Wenger describes three different types of boundary processes: brokering, boundary objects, and interactions. The CoP in this study has boundary interactions which serve “(1) to provide direct exposure to a practice and be fully immersed in it; (2) to serve people who need some service, are curious or intend to become members” (Santos-Díaz and Towns, 2020). The outreach event provided direct exposure to chemistry for the third-grade attendees, thus the outreach event itself is an example of a boundary interaction.

That said, while planning, Woman 1 mentioned wanting to increase brokering interactions between their chapter/CoP and other universities/CoPs with outreach programs to gain more knowledge (boundary objects). Boundary objects are essentially artifacts shared between CoPs; brokering is interacting with another CoP or outsiders to exchange knowledge and create connections between CoPs.

Based on these findings, we summarized this case/chapter/specific CoP in Fig. 3. In this figure, components of a mature CoP that were not evidenced in the studied CoP are greyed and crossed out. However, the majority of these greyed-out/crossed off components are areas this specific student organization were aware they are missing, as they discussed wanting many of those components. For example, Woman 1 discussed wanting boundary objects/artifacts: “I wish the department as a whole did more things like this so that way, we had like…even just like an idea bank to draw from.” In many instances, the participants discussed wanting various artifacts to better help their CoP grow, better broker new connections, and improve their chemistry outreach overall. This lends to the studied group's lack of experience and competence. Since they are a relatively new organization that was heavily impacted by the pandemic, there are parts of the CoP missing. These missing areas allude to the lack of maturity in terms of a CoP for this collegiate group. However, findings provide empirical evidence to support claims by Santos-Díaz and Towns that the CoP framework can successfully be used to describe and evaluate a collegiate chemistry organization conducting chemistry outreach.

Fig. 3 Summary of the studied case as a community of practice. Components missing from the studied CoP are grayed and crossed out.

Research question 2: outreach event planning and evaluation processes

Previous research investigating outreach planning and evaluation have primarily studied individuals, despite most outreach being conducting by groups. The Framework (Fig. 2) was created based off professional scientists that practice outreach, necessitating an analysis of how it aligns with collegiate organizations who report reaching over 1 million people every year through their voluntary chemistry outreach events (Connelly, 2015; Pratt, 2017). Results to research question 1 support that the studied group is an emerging/developing CoP. Below, we address how this CoP planned and evaluated their event as a collective.

Planning process. Since the Framework emphasizes setting goals as the first step of the process, our analysis looked for both the planning process of the group as well as their various goals for their outreach event. The focus group discussion was non-linear, so the researchers identified the students’ timeline/planning progression through inductive coding; four group planning steps were identified, as noted in Fig. 4.

Fig. 4 Planning process of the group.

The first step of their process was one coordinator taking charge and initiating event planning. During the pre-focus group, the President of the chapter (Woman 1, Chapter President) stated, “I know I personally am like very passionate about science outreach… I kind of like took charge on this project.” Throughout the focus group, the idea of this leader coordinating and guiding the planning process was echoed by other members of the group. The idea to host a chemistry outreach event came from this leader/chapter President, who also managed all logistical planning between the university and the elementary school. Other members of the chapter followed the President's lead.

The second step of their planning process was core members of the group generating ideas for the outreach event. For example, Man 2 said, “I feel like once [President] kind of took the charge and told us it was like going down, we all kind of like collectively just started… just throwing ideas at the wall and seeing what stuck.” Again, the President played a pertinent role in the process as her actions directed other members. Man 2 also noted, “Once she (President) gave us like a general format of how she wanted [the outreach event] to go down, it was kind of [up to] us to like figure out those smaller aspects.”

This alluded to the third step: coordination between coordinator, core, and active members for event logistics. The chapter President also described how chapter members self-organized for the outreach event in terms of those leading the hands-on activities and those presenting the demonstration show. These three planning steps stress the heavy reliance on chapter leadership during the planning process for the outreach event, mirroring previous work on leadership experiences in these collegiate organizations (Santos-Díaz and Towns, 2021). However, in this third step, broader chapter membership became involved, more than the leader and core members who controlled the first two steps of the process. The addition of active members in this step of the planning process provides further support for the CoP interpretation of collegiate groups conducting chemistry outreach.

The last step in the group's planning process was having a final plan for the outreach event. Here, the focus was on the goals the group used to inform event plans since the Framework emphasizes setting goals. Woman 3 summarized their event and tied each component to specific goals: “I think that the smaller [hands-on] experiments are kind of to like actually teach [kids] about different like science concepts…in a simple way. And like the big [demonstration] show is to be like ‘ooh, like super cool’ like large scale like really awesome science experiments.” These goals of audience learning and considering that science is enjoyable align with previous findings by Pratt and Yezierski who found that some goals for chemistry outreach events are audience learning and audience awareness that science is fun (Pratt and Yezierski, 2018a). Overall, this chapter discussed five goals for their outreach event that informed their planning process: (1) audience learning, (2) audience awareness that science is fun, (3) increasing accessibility to science, (4) generating interest, and (5) awareness of what science is; all of these are deductive goals that echo previous findings on chemistry outreach goals and purposes (Pratt and Yezierski, 2018a).

After interpreting the planning process for the chapter, findings were compared to the Framework (National Academies of Sciences, Engineering, and Medicine, 2016). The Framework discusses a logical, stepwise approach to planning, implementing, and evaluating an informal chemistry event, not dissimilar to backwards design approaches in formal education settings (Wiggins and McTighe, 2005). Specifically, the planning components of the Framework focus on setting goals and outcomes, then identifying resources and materials, and finally designing and preparing for the event (see Fig. 2). Throughout the analysis of the studied chapter's process, we found that the group did not follow a stepwise approach. Rather, they incorporated all three of these steps/components concurrently, shifting back-and-forth between them. The group was constantly going between designing activities, making/refining goals, and identifying the necessary resources. Consider this interaction between Woman 2 and Woman 1:

“I think blowing up a balloon with vinegar and baking soda would be really cool, but it would have to be large scale.” (Woman 2, President-elect). “We have balloons upstairs actually.” (Woman 1, President)

In the middle of the pre-focus group, Woman 2 began considering another potential demonstration, which she relates back to the goal of audience awareness that science is fun: “would be really cool” (Pratt and Yezierski, 2018a). Woman 1 then continued the conversation by identifying the potential materials they would need for the new demonstration. While the group shifted between the first three elements of the Framework throughout their planning process, it was more common to consider two elements simultaneously before shifting to the third. Woman 2 described the group's second step in the planning process of setting goals/outcomes and designing/preparing for the event together, “I know like when I like- when we were like googling stuff, I was looking mainly at what kind of concepts would be easy to teach in like an engaging way. But also, like conceptually for [kids]…able for [kids] to understand.”

It is beneficial that this chapter considered goals/outcomes as part of their planning process, which is contrary to previous findings that focused on individuals’ outreach experiences where goals/outcomes were rarely considered (Pratt and Yezierski, 2018b, 2019b). Additionally, the simultaneous shifting across the planning elements can help the group plan more freely (i.e., brainstorming) as the goals and resources can be revised throughout their planning process. However, the shifting between components as opposed to a stepwise process can also lead to confusion amongst the various members involved and a misalignment between goals and actions.

Additionally, it is worth noting that this group did not explicitly consider chemical safety in their planning process. While guiding authorities like the American Chemical Society and the Alpha Chi Sigma Fraternity have chemical safety guidelines and demonstration show planning guides that emphasize identifying, planning for, and mitigating safety hazards, the studied group only implicitly considered safety by thinking about the physical locations of the events and not specifically for attendees. For example, the group removed demonstrations that involved fire from any consideration as the demonstration show was conducted in an auditorium space in a non-chemistry building. Despite this, the group did ensure that safety goggles were provided to participants during the hands-on activities and that they wore appropriate PPE during all events, suggesting that safety was a consideration but did not strongly impact their overall practices. This mirrors previous findings that suggested that collegiate students conducting outreach have minimal understanding of their demonstrations and safety is not on the forefront of their minds (Pratt and Yezierski, 2018b).

Evaluation process. While analyzing the evaluation process, we found that the chapter followed four main steps (Fig. 5) based on their discussion. Researchers organized their approach into a logical sequence based on how the ideas were discussed in the post-focus group.

Fig. 5 Evaluation process of the group.

The first step was reflections and self-assessment of goals for the outreach event. During the post-focus group that focused on evaluation, Man 2 reiterated some of their previously stated goals and evaluated their success in achieving those goals, which aligns with the Framework: “I think one of the goals was just to introduce kids to science…Show them like how fun it can be. And I'd say we achieved that.” One of these goals, awareness that science is fun, was discussed in the pre-focus group before the event and was used as part of the group's planning process. Man 2 reflected on their event and believed they had successfully shown their third-grade audience that science is fun. However, as shown in Man 2's quote, a new goal not previously discussed by the chapter was considered in the evaluation, exposure to science: “introduce kids to science.” While this goal has been seen in previous literature, it was not mentioned by the group in their planning process/pre-focus group discussion (Pratt and Yezierski, 2018a). This shows a mismatch between goals for planning purposes and those evaluated. Woman 2 continued to evaluate additional goals not previously discussed by the group, “I think a main goal was for just for [the event] to run smoothly and for everything to work out well, and it did.” These goals may have been implicit to individuals, but never explicitly shared within the group/stated in the focus group discussions. This new goal of logistic success, mentioned by Woman 2, was not mentioned during the group's planning process, nor has it been previously reported as a goal for chemistry outreach. This may not be surprising as such a goal could be considered an overarching goal for all events, in line with the spirit of the Framework for Effective Chemistry Communication in Informal Environments that focuses on evaluating event success. However, the consideration of new/different goals when evaluating events is concerning.

The second step the chapter followed was the evaluation of the preparation process (i.e., practicing). During this step, the students reflected on their process for preparing for the outreach event, which predominantly focused on practicing the activities/demonstrations before the event. Woman 2 mentioned, “I think if we had practiced like…a few more times it would have been good.” Here, she reflected on the fact that they did practice as part of their preparation, a novel finding given previous work on chemistry outreach that suggests students rarely practice prior to events (Pratt and Yezierski, 2019b). However, Woman 2 continues and suggests future events would benefit from additional practice—to support the goal of logistic success. During this discussion during the post-focus group, the students also mentioned a moment during their event where the elephant's toothpaste demonstration did not go as well as they had hoped. Woman 1 described that situation: “Yea I think the elephant toothpaste thing was… I don’t think we had high hopes after doing our practice runs…I think we knew that it would not…was not going to be spectacular. I feel… I still think it went well.” As discussed, despite practicing their activities/demonstrations prior to the event, there were still inevitable mistakes. Woman 1 (Chapter President) still counted this activity as a success, however, because it had the desired effect on the audience, thus the goal of “audience enjoyment” and “awareness that science is fun” was met.

The third step of their reflection process was collecting data from outreach event. All the data considered during their reflection and evaluation were informal and included four sources: (1) questions from kids during the outreach event, (2) comments from kids during the outreach event, (3) reactions of kids during the outreach events, and (4) messages from teachers and parents after the outreach event. Man 4 best discussed the questions he received from kids at the event, “I had a kid ask me about the uhm vacuum attachments…And it was it was cool…like one of the kids was actually interested in what goes on and how stuff works!” From Man 4's perspective, this evidences the achievement of the goal generate interest, one of the evaluated goals that also informed their planning process. It also supports the goal of exposure to science, another goal not mentioned in the planning process yet was part of the evaluation process. In another example, Woman 1 discusses data from other sources:

“I had several kids actually tell me they had fun. I had a ton of kids tell me like, ‘oh my gosh…this is so cool! This is awesome!’…you heard ALL of the kids in sync be like ‘WOW!’ and it was just…it was really cute…And also the teachers have already like messaged me and said that they've gotten messages from a ton of parents saying that their kids went home and said they had so much fun.”

Several third graders explicitly told Woman 1 they had fun, which supports the achievement of the goal audience enjoyment. While this goal was also found in previous literature, it did not contribute to the group's planning process (Pratt and Yezierski, 2018a). Woman 1's quote also discusses kids’ reactions which support the generate interest goal. While these discussions were explicit and show clear considerations from the chapter in how they reflected and evaluated themselves, other goals were considered during this discussion that were implicitly considered, including increase accessibility to science and awareness of what science is. These goals were mentioned in the pre-focus group during their planning for the outreach event, but the chapter did not explicitly reflect on the achievement of these goals during the post-focus group (Fig. 6). Lastly, Woman 1's quote above also discusses data through communications with the third-grade teachers and parents after the event. These messages provided data that reinforced that the kids had fun (audience enjoyment). Based on the data collected and considered by the chapter, audience enjoyment had the most support for being successfully met. While this is positive in terms of outcomes for event attendees, it is concerning from an event planning perspective as this goal was not used to inform the group's planning process (i.e., not discussed during the pre-focus group), which is contrary to the recommendation of the Framework. It is worth noting that this study solely focused on discussions with the college student event facilitators and did not involve observations of the outreach event itself. As such, all discussions of data and event details came directly from the student facilitators, relayed to the research team via the focus group discussions.

Fig. 6 Relationships between planning process, evaluation processes, and event goals.

The last step of the group's evaluation process was the reevaluation of goals for future outreach events. As part of this, the group considered new goals to inform future outreach events. One such example was initially discussed by Man 2, “We should try to do some more [outreach events]…it's kind of a way to not only give back to our community but also like chip at that stigma of us being college students about being rowdy and stuff.” In this statement, Man 2 describes two new goals for future events: community service and break the stigma of college students. Both of these goals are novel and have not been previously discussed in the literature on chemistry outreach goals/purposes. Overall, the chapter discussed four new goals to consider for future outreach events: community service, break the stigma of college students, audience enjoyment, and gaining connections outside of the group. All four of these goals were only discussed in the evaluation process of the group; only audience enjoyment has been previously discussed in the literature and was reflected upon to evaluate the outreach event that bounds this case study. The other goals are novel goals for chemistry outreach but were not used by the group to evaluate their outreach event, only to inform future events.

Overall, the chapter did reflect and evaluate the success of their event, in line with the last step of the Framework (assess, reflect, follow up). Combined with the aforementioned discussion of how the group planned their event, we have a better understanding of how this specific CoP planned and implemented their outreach event. Summarized in Fig. 7, the Framework is adapted to represent the overall process of the studied group—including the concurrent consideration of the first three steps of the process, as evidenced by this group.

Fig. 7 Overview of group's planning and evaluation process.

Even though the group set goals for their outreach event and evaluated them, the group's evaluation process showed inconsistencies between goals that informed the planning process and goals assessed in the evaluation process. Fig. 6 summarizes the overall relationships between each step of the planning and evaluation processes, and the chapter's various goals for the outreach event. During the evaluation, the student group primarily reflected on new goals rather than on their previously set goals from planning the event. A significant observation is the fact that the goal audience learning informed every step of the group's planning process, yet there was no mention of it during the evaluation process. While the chapter deemed the event successful, the mismatch between goals used to plan the event and those evaluated is concerning. Such a concern points towards the need for more chemistry outreach resources (e.g., guides, training) to improve collegiate students’ outreach practices, echoing findings from previous work that studied individual outreach facilitators (Pratt and Yezierski, 2019b). This may also suggest that students feel the need to sound successful when discussing their events, rather than critically evaluating their experiences. Additionally, the idea of creating reasonable/measurable goals may not have been clear prior to the event, relating to the group's minimal competence and experience in chemistry outreach as they are a relatively new chapter. It serves to mention that the act of having the focus group discussions may have impacted their process in discussing their goals, but based on the discussion, goal setting was already done prior to the pre-focus group discussion. The difference may be seen in terms of depth of discussion and would have less effects on the steps or goals stated during the focus groups. Overall, findings support a need for guidance and/or training so that this organization becomes more aware of the processes necessary for critically planning and evaluating outreach events.

Limitations

This research was a case study looking at a specific student organization planning a single outreach event at a single university; therefore, our results are limited. However, the Framework and CoP lenses used to interpret the actions of this group can provide insights that support transferability to interpret other contexts/groups. That said, the nature of a case study is to observe a specific situation and provide potential insights based on the results found. Thus, our findings may be limited as a case, but provide insights that can support future, larger investigations into chemistry outreach practices. Additionally, we did not observe or interact with peripheral members; rather, the discussions by focus group attendees that included information about other chapter members provided support that peripheral members exist for the studied group. It is also important to point out that the pre-focus group/data collection may have impacted the study as asking the students about their goals beforehand may have primed them to think differently when conducting the event or reflecting on it during the post-focus group. Lastly, we did not observe the outreach event itself to further minimize researcher impact on the study. As such, all data consisted of students’ self-reflections and evaluations during the focus groups.

Conclusions and implications

This study provides deeper insights into the practices of a collegiate chemistry organization conducting chemistry outreach. The CoP lens provided insights into the organization's structure and function, while the Framework provided deeper insights into the group's specific planning process. Previous studies from Santos-Díaz and Towns (2020, 2021) suggested that a collegiate organization aligns well with the CoP framework, but they focused only on the leadership component. In this study, we provide evidence that the entirety of the CoP framework aligns with a collegiate organization and provides insights into their group dynamics. The studied chapter is relatively new and heavily impacted by the COVID-19 pandemic; as such, we see evidence that while they follow the structure and interactions of a CoP, they are an emerging CoP with areas for growth (e.g., brokering, artifacts) that the members are aware of. In terms of specific event planning processes, we see a concurrent approach that simultaneously considered goals, resources, and activities; this is contrary to the proposed Framework and may help interpret why this chapter evidenced misalignment between event goals, event planning, and event evaluation/reflection. The Framework is a guide written by the National Academies of Sciences, Engineering, and Medicine—among others—based in research for effectively communicating chemistry knowledge in outreach settings. Therefore, more alignment with this framework could provide a better foundation for this group when conducting future outreach events.

This study strengthens the argument that a student organization can be considered a CoP, providing further support for using the CoP framework in future investigations of chemistry outreach practices. As this study focused on an individual case/chapter, future work should look at different student groups/chapters and how they plan outreach events, including their boundary interactions (e.g., faculty advisor involvement or connections with other student organizations). Furthermore, expanding to a larger sample of multiple student organizations/chapters is necessary to identify more transferability of these findings; as this is a case study, the results are highly context dependent. In addition to future work considering multiple chapters/contexts, more work is needed in the informal chemistry education/chemistry outreach realm as most studies focus on structured informal science (e.g., museum, planetarium), rather than voluntary, community-service-like events that are common for collegiate students.

Another interesting finding in our results was the idea that many of the pre-focus group goals were found in previous outreach studies. That said, during the evaluation process, this group generated novel goals. This was found for only the post-focus group, not the pre-focus group. This may allude to students having different mindsets during their planning and evaluation processes, which was not fully explored in this study and is a potential area for future work.

In addition to the future research avenues described above, our findings provide further support that specific training experiences for collegiate chemistry outreach are needed. In both focus group discussions, students repeatedly requested more resources and training to better support their planning and evaluation processes. By creating opportunities for such training, there may be an increase in overall outreach event success and impact on attendees. Previous research about CoPs indicates that targeting gaps in connectivity, rather than promoting connectivity as a whole, is more successful (Cross et al., 2006). Thus, training experiences may seek to first target connectivity gaps within a CoP/group before broadening to other areas/elements.

Author contributions

J. M. P. and M. C. conceptualized and designed the project. J. M. P. conducted both focus group discussions. J. M. P. and R. C. C. primarily analyzed the data. M. C. provided peer scrutiny and additional insights on the analyses. J. M. P., R. C. C., and M. C. all wrote and revised the manuscript.

Conflicts of interest

There are no conflicts to declare.

Appendices

Appendix 1 Semi-structured, pre-event focus group protocol

This focus group discussion centers around students’ perspectives, beliefs, and ideas related to planning their outreach event. The goal is to understand why they are planning the event, what their intended outcomes are (both for their audience and for themselves), and to understand how they go about designing, planning, and implementing their activities.

Pre-event focus group questions:

• What is the goal of this event? Are there multiple goals?

○ What outcomes are you hoping for?

○ Are there different outcomes for the participants and the practitioners?

○ Are your goals content focused? Emotion (affect) focused? Skills (psychomotor) focused?

• Describe the process the group uses to plan the event.

○ Is there a committee or individual that plans it?

○ Are multiple voices/perspectives included in the planning?

○ How are roles/duties assigned to plan and implement the event?

• Where did you learn how to plan an outreach event?

○ What resources or individuals have helped develop these skills?

• Are there resources or supports you wished were available to help you in this process?

• Is there anything specific you want us to know about how you approach planning your events or the outcomes you intend?

Appendix 2 Semi-structured, post-event focus group protocol

This focus group discussion centers around students’ reflection on their outreach event. The goal is to understand how they perceive the success of the event and what data (formative, summative, numeric, reflective, and/or observational) were gathered to understand the success of the event. Focus on gaining insights into their approach, reflection on their experiences, and data to inform future events.

Post-event focus group questions:

• Remind us: What was the goal (or goals) for this event?

○ Were there different outcomes for the participants and the practitioners?

• How did the event go?

• Do you think the event was successful?

○ How do you know?

○ What about it was successful? What specific parts/activities/actions were successful?

○ What “data” did you gather to help you assess the success?

• Examples: formative/summative, numeric/survey, qualitative/reflective, observational

• What was “good” that you want to continue doing in future events?

• What would you like to improve for future events?

• Are there resources or supports you wished were available to help you in this process?

• Is there anything specific you want us to know about how your event went, your experiences, the participants/their experiences, etc.?

Acknowledgements

We extend our gratitude to the students who participated in this project; without their voluntary participation, this study would not have been possible. We also thank members of the Pratt Research Group for their feedback, peer scrutiny, and insights over the course of this project which helped improve the work.

References

1. Baxter P. and Jack S., (2008), Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers, Qualitative Rep., 13(4), 544–559.
2. Bowe K. A., Green A. R., Gorske Y. J. K., Lesher E. and Bauer C. F., (2023), Speaking to Learn and Learning to Speak: Service-Learning Involving Communication in the Chemistry Curriculum. J. Chem. Educ., 100(8), 2836–2846 DOI:10.1021/acs.jchemed.2c00985.
3. Colors on the Moooove!, (2014), American Chemical Society. https://www.acs.org/education/whatischemistry/adventures-in-chemistry/experiments/colors-move.html (Accessed 2023).
4. Connelly T. M., (2015), Why is Chapter Community Outreach So Important? inChemistry, 24(1), 3.
5. Cross R., Laseter T., Parker A. and Velasquez G., (2006), Using Social Network Analysis to Improve Communities of Practice. California Manage. Rev., 49(1), 32–60 DOI:10.2307/41166370.
6. Fereday J. and Muir-Cochrane E., (2006), Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development, Int. J. Qualitative Methods, 5(1), 80–92 DOI:10.1177/160940690600500107.
7. Finio B., (2019), Make Elephant Toothpaste. Scientific American. https://www.scientificamerican.com/article/make-elephant-toothpaste/(Accessed 2023).
8. Helmenstine A. M., (2019), How To Make a Dry Ice Balloon. Thought Co. https://www.thoughtco.com/make-a-dry-ice-balloon-606411 (Accessed 2023).
9. Janesick V. J., (2015), Peer Debriefing, in The Blackwell Encyclopedia of Sociology, G. Ritzer (ed.) John Wiley & Sons, Ltd DOI:10.1002/9781405165518.wbeosp014.pub2.
10. Larive C. K. and Kuwana T., (2007), The Analytical Sciences Digital Library: A Useful Resource for Active Learning, in Active Learning: Models from the Analytical Sciences, P. A. Mabrouk (ed.) ACS Symposium Series 970, American Chemical Society, Washington, DC, pp. 188–198.
11. Lave J. and Wenger E., (1991), Situated learning: legitimate peripheral participation, Cambridge University Press DOI:10.1017/CBO9780511815355.
12. Leontyev A., Houseknecht J. B., Maloney V., Muzyka J. L., Rossi R., Welder C. O., Winfield L., (2020), OrganicERs: Building a Community of Practice for Organic Chemistry Instructors through Workshops and Web-Based Resources, J. Chem. Educ., 97(1), 106–111.  DOI:10.1021/acs.jchemed.9b00104.
13. Lincoln Y. S. and Guba E. G., (1985), Naturalistic inquiry, Sage Publications.
14. McAlister A., Lee D., Ehlert K., Kajfez R., Faber C. and Kennedy M., (2017), Qualitative Coding: An Approach to Assess Inter-Rater Reliability, in 2017 ASEE Annual Conference & Exposition Proceedings, ASEE Conferences, p. 28777 DOI:10.18260/1-2--28777.
15. Mentos and Diet Coke!, (2015), American Chemical Society. https://www.acs.org/education/whatischemistry/adventures-in-chemistry/experiments/mentos-diet-coke.html (Accessed 2023).
16. Merriam S. B. and Tisdell E. J., (2016), Qualitative research: a guide to design and implementation, 4th edn, Jossey-Bass.
17. Morgan Theall R. A. and Bond M. R., (2013), Incorporating Professional Service as a Component of General Chemistry Laboratory by Demonstrating Chemistry to Elementary Students, J. Chem. Educ., 90(3), 332–337 DOI:10.1021/ed300356b.
18. National Academies of Sciences, Engineering, and Medicine, (2016), Effective Chemistry Communication in Informal Environments, National Academies Press DOI:10.17226/21790.
19. National Research Council, (2009), Learning Science in Informal Environments: People, Places, and Pursuits, Washington, DC: The National Academies Press DOI:10.17226/12190.
20. National Science Foundation, Advancing Informal STEM Learning (AISL). https://new.nsf.gov/funding/opportunities/advancing-informal-stem-learning-aisl (accessed August 2023).
21. Patton M. Q., (2015), Qualitative research & evaluation methods: integrating theory and practice, 4th edn, SAGE Publications, Inc.
22. Pratt J. M., (2017), Alpha Chi Sigma and Chemistry Outreach: Promoting the Second Object, HEXAGON, 108(1), 8–9.
23. Pratt J. M. and Yezierski E. J., (2018a), Characterizing the Landscape: Collegiate Organizations’ Chemistry Outreach Practices, J. Chem. Educ., 95(1), 7–16 DOI:10.1021/acs.jchemed.7b00627.
24. Pratt J. M. and Yezierski E. J., (2018b), College Students Teaching Chemistry through Outreach: Conceptual Understanding of the Elephant Toothpaste Reaction and Making Liquid Nitrogen Ice Cream, J. Chem. Educ., 95(12), 2091–2102 DOI:10.1021/acs.jchemed.8b00688.
25. Pratt J. M. and Yezierski E. J., (2019a), “You Lose Some Accuracy When You’re Dumbing it Down”: Teaching and Learning Ideas of College Students Teaching Chemistry through Outreach, J. Chem. Educ., 96(2), 203–212 DOI:10.1021/acs.jchemed.8b00828.
26. Pratt J. M. and Yezierski E. J., (2019b), Goodwill without Guidance: College Student Outreach Practitioner Training, J. Chem. Educ., 96(3), 414–422 DOI:10.1021/acs.jchemed.8b00882.
27. Raker J. R., Pratt J. M. and Watson L., (2020), Building Community: A Reflection on the Interactive Online Network of Inorganic Chemists, in Advances in Teaching Inorganic Chemistry: Classroom Innovations and Faculty Development, R. M. Jones (ed.) Washington, DC: ACS Symposium Series 1370, American Chemical Society, vol. 1, pp. 131–139.
28. Santos-Díaz S. and Towns M. H., (2020), Chemistry outreach as a community of practice: investigating the relationship between student-facilitators’ experiences and boundary processes in a student-run organization, Chem. Educ. Res. Pract., 21(4), 1095–1109 10.1039/D0RP00106F.
29. Santos-Díaz S. and Towns M. H., (2021), An all-female graduate student organization participating in chemistry outreach: a case study characterizing leadership in the community of practice, Chem. Educ. Res. Pract., 22(2), 532–553 10.1039/D0RP00222D.
30. Time for Slime!, (2014), American Chemical Society. https://www.acs.org/education/whatischemistry/adventures-in-chemistry/experiments/slime.html (Accessed 2023).
31. Wenger E., (2000), Communities of Practice and Social Learning Systems, Organization, 7(2), 225–246 DOI:10.1177/135050840072002.
32. Wenger E., McDermott R. A. and Snyder W., (2002), Cultivating communities of practice: a guide to managing knowledge, Harvard Business School Press.
33. Wenger E., Traynor B. and de Laat M., (2011), Promoting and assessing value creation in communities and networks: A conceptual framework, The Netherlands: Ruud de Moor Centrum, Open University of the Netherlands.
34. Wiggins G. and McTighe J., (2005), Understanding by Design, 2nd edn, Association for Supervision and Curriculum Development.
35. Yin R. K., (2014), Case Study Research Design and Methods, 5th edn, Thousand Oaks, CA: Sage Publications.

---