Justin M.
Pratt
*a,
Rebecca
Chan - Chao
a and
Merryn
Cole
b
aDepartment of Chemistry, University of Rhode Island, Kingston, RI 02881, USA. E-mail: justin.pratt@uri.edu
bDepartment of Teaching & Learning, University of Nevada, Las Vegas, Las Vegas, NV 89154, USA
First published on 2nd February 2024
Chemistry outreach, a type of informal science education commonly practiced by college students, has primarily been studied by looking at individuals' approaches and perspectives. However, it is much more common for college students to plan and conduct chemistry outreach events as part of a group/club/chapter, not independently. In this case study, we looked at the planning and evaluation processes for a single student group conducting a chemistry outreach event. Using pre- and post-outreach event focus groups, we sought to understand how the group was structured and how they collectively planned and implemented their outreach event. Using the Framework for Effective Chemistry Communication and Communities of Practice, we identified the steps the group took to plan, implement, and evaluate their event, as well as the group's structure and member dynamics. Findings showed a misalignment between goals used to plan the outreach event, and those evaluated when reflecting on the event's success. Additionally, the Communities of Practice framework was able to highlight the membership structure underlying the group's planning process, as well as areas for growth and improvement. Overall, findings provide further support that collegiate chemistry organizations/student groups need more resources and training to successfully conduct chemistry outreach.
Over the past 15 years, ISE has grown through increased scholarly investigations and grant funding, particularly through the Advancing Informal STEM Teaching (AISL) program from the National Science Foundation (National Science Foundation, n.d.). Some scholars have tried to bridge formal and informal learning to maximize the impact on students, including through service-learning experiences. Service learning typically involves formal classroom instruction with the end goal of students applying their formal knowledge to implement informal learning experiences for the public. Bowe et al. (2023) examined the effects of service-learning experiences on students’ sense of competence, self-efficacy, relatedness, and autonomy in introductory chemistry courses. Morgan Theall and Bond (2013) used service-learning experiences where students planned informal science events as part of their formal chemistry laboratory classes. While these chemistry-specific examples provide insights into informal learning experiences, they still rely on formal learning experiences where the informal application of material is a graded component of the course. In chemistry, however, informal science education is more commonly practiced as voluntary, community-service experiences, rather than compulsory as part of a course; such voluntary experiences are typically labeled “chemistry outreach” experiences.
In addressing this gap, a collection of publications from Pratt and Yezierski (2018–2019) focused exclusively on the chemistry outreach practices of collegiate students and showed the need for further study in this area. The first article looked at the purposes of conducting chemistry outreach and the most prevalent demonstrations/activities associated with chemistry outreach events (Pratt and Yezierski, 2018a). Findings showed that undergraduate students and faculty/staff have different ideas regarding the purpose of chemistry outreach. However, audience learning was the most prevalently discussed purpose for both populations. In a follow-up study, the authors investigated students’ content knowledge related to common outreach demonstrations (Pratt and Yezierski, 2018b). Findings suggested that students held misunderstandings and non-canonical ideas, despite being primarily third- and fourth-year chemistry students approaching graduation, suggesting that audience learning at these events may be shallow and/or inaccurate. This was further supported by findings related to the teaching and learning ideas espoused by these collegiate students for their outreach events (Pratt and Yezierski, 2019a); the collegiate students discussed shallow and/or unsupportive teaching ideas, including that it is acceptable to teach inaccurate content when the audience is young. Such misalignment between their chemistry outreach goals, their content knowledge/understanding, and their beliefs about teaching and learning are concerning but not surprising given that these students report minimal training experiences and little-to-no interactions with faculty advisors (Pratt and Yezierski, 2019b); in fact, students commonly discussed “winging it” and/or using trial and error when conducting outreach events.
The aforementioned studies provided the first scholarly insights into chemistry outreach practices for collegiate students. However, these studies focused on individual student ideas and experiences, and did not consider the contextual nature of their outreach events; it is more common for students to plan and implement chemistry outreach events as part of a group/club/organization, as opposed to individually. The study described herein addresses this gap and focuses on understanding one student group and how they approached planning and implementing a chemistry outreach event.
At the most basic level, learning in a CoP involves competence and experience. The competence of a CoP is built and established over time, while experience refers to the experiences of members in the world. Both competence and experience are measures of success for a CoP, depending on their goals (i.e., higher competence = higher success). Learning in a CoP takes place at the intersection of competence and experience. Inside a CoP, two major processes occur: participation (acting and interacting of CoP members) and reification (producing artifacts for the CoP). Membership has varying degrees of participation—core, active, and peripheral members—with sequentially decreasing levels of participation (i.e., core members include coordinators or designated leaders, while peripheral members are generally those who are new/less knowledgeable and do not participate in many events). That said, member participation shifts as the community's focus or practice evolves (e.g., some core members become less involved as active members, while some peripheral members become more involved as active members). A great deal of growth within a CoP relies on legitimate peripheral participation and interactions between those with less knowledge and those with more knowledge, resulting in learning and growth (Lave and Wenger, 1991).
Wenger also discusses elements that describe how a CoP functions; these elements are actions that allow for the growth of the CoP, and include leadership, membership, connectivity, events, artifacts, and learning projects. There are multiple forms of leadership, and these roles change from person to person over time. Regardless, CoPs depend on leadership to help develop the community. Membership is also needed to ensure the growth of a CoP. Processes that allow new members to cycle into the community help to maintain what Wenger describes as “critical mass” in a CoP (Wenger, 2000). This means that there are not so many members that the focus of the CoP is lost, but there are enough members to continue towards their goal. Connectivity refers to encouraging interactions between members of a CoP in multiple ways to share ideas. Public events are meant to bring the CoP together, ensure connectivity, and apply their new knowledge. As a CoP establishes experience, they produce artifacts in the form of documents, tools, stories, etc. These artifacts change and/or are maintained based on the community's needs. Lastly, learning projects are used to deepen the extent of a CoPs knowledge and further push their practice forward.
It is also important to discuss the boundaries of a CoP. Boundaries are where “competence and experience tend to diverge [resulting in] a boundary interaction” (Wenger, 2000). Wenger describes three types of boundary ideas: brokering, boundary objects, and boundary interactions. Brokering is where individual CoP members purposefully reach out to another CoP to share/gain knowledge (i.e., formalized knowledge sharing between CoPs). Boundary objects are resources that facilitate such communication and knowledge sharing, unique to individual CoPs. These objects can take the form of documents, terminology, stories, rules, and unspoken norms. Regardless, these objects help CoPs with connectivity and bringing new members into the community by providing resources and tools to facilitate learning and growth; these objects can also be used in multiple ways, not only within CoP development but also as part of brokering with other CoPs. Boundary interactions are more informal knowledge-sharing activities, typically involving individual members interacting with others outside of their CoP (e.g., attending a scientific conference). Such interactions allow new knowledge to be gained and brought back to the CoP. Overall, all parts and elements of a CoP must coalesce for a CoP to be effective and learn as a community, as summarized in Fig. 1. Examples of successful CoPs in the chemistry community that focus on formal learning environments include the Interactive Online Network of Inorganic Chemists (IONiC) (Raker et al., 2020), Organic Chemistry Educational Resources (OrganicERs) (Leontyev et al., 2020), and the Analytical Sciences Digital Library (ASDL) (Larive and Kuwana, 2007).
This study seeks to add to the existing literature by first considering how a collegiate organization functions, through the lens of CoP. Secondly, as previous research has examined individual goals for outreach, we seek to build on these findings by looking at the steps/process for how the studied students plan an outreach event as a collective, and the goals that guide their processes. Understanding both how the group functions and their planning and evaluation processes are necessary before interventions and resources can be developed to improve outreach practices, as previously suggested (Pratt and Yezierski, 2018a, 2018b; 2019a, 2019b).
1. How does an individual chapter of a collegiate chemistry organization function as a Community of Practice?
2. How does an individual chapter of a collegiate chemistry organization collectively plan, implement, and evaluate a single chemistry outreach event, through the lens of the Framework for Effective Chemistry Communication for informal environments?
These highly interrelated research questions complement each other as the first question focuses on group structure and dynamics, while the second question focuses on how said group structure and dynamics manifest in specific planning and evaluation processes.
Study participants were recruited via convenience sampling; members of the chapter were invited to voluntarily participate in the study and were afforded opportunities to not take part during each data collection timepoint (i.e., each focus group discussion). Overall, data collection involved two focus group discussions with consented participants: one pre-outreach event and one post-outreach event. In the pre-outreach event focus group, four men and seven women participated (N = 11), with ten participants identifying as white. In the post-outreach event focus group, five men and six women participated (N = 11), with all participants identifying as white. The overall demographic composition for both the pre- and post-focus group participants mirrors that of the university, being predominately white women. Out of the thirteen participants across both data collection timepoints, three held leadership positions within the organization, including the current chapter president, the current chapter vice president, and the incoming chapter president/president-elect.
The chemistry outreach event conducted by the organization that bounds this case consisted of a hands-on component and a larger demonstration show component. Approximately 70 third-grade students from a local elementary school attended the event, hosted on the university's campus. The hands-on activities were hosted in teaching laboratory spaces in the chemistry building and involved splitting the third graders into smaller groups to conduct hands-on activities facilitated by individual chapter members. These hands-on activities included using milk, food coloring, and dish soap to discuss attraction & repulsion ideas (Colors on the Moooove!, 2014) as well as making slime using glue to discuss polymers and structure–property relationships (Time for Slime!, 2014).
The large demonstration show was held in an auditorium space in another university building and involved members of the student organization conducting demonstrations in front of the third graders as a collective audience; demonstrations included Coke and Mentos, Elephant's Toothpaste, and How to Make a Dry Ice Balloon (Mentos and Diet Coke!, 2015; Finio, 2019; Helmenstine, 2019).
After open coding, the pre-focus group data were deductively coded using previous literature on chemistry outreach (Fereday and Muir-Cochrane, 2006). Specifically, outreach event goals and purposes discussed by study participants were compared with previously reported goals from a national chemistry outreach survey (Pratt and Yezierski, 2018a). Goals identified through open coding that overlapped with previous survey findings were redefined as deductive codes; novel goals not previously reported remained as inductive codes. Given that the group planning process for chemistry outreach has not been previously studied, all codes related to planning and evaluating the event were novel and thus inductive codes. After inductive and deductive coding, thematic analysis was used to identify overarching patterns and ideas to address the research questions (Fereday and Muir-Cochrane, 2006). As part of this thematic analysis process, our theoretical frameworks were used as lenses to interpret patterns and make sense of the student group's processes. Specifically, Communities of Practice was used to understand the group's structure and function as a community working towards a common goal, and the Framework for Effective Chemistry Communication was used to understand the overall process the group used to plan, implement, and evaluate their event. Individual elements and components of CoP were compared to the codes from the analysis to determine how the CoP framework defined the studied group's dynamics. The Framework was used as an example of an effective event planning and evaluation process as it is based on research by the National Academies of Science, Engineering, and Medicine. To determine alignment with this framework, goals were coded and connected to the studied group's process. These were then compared to the framework for full analysis.
To support the trustworthiness of findings and interpretations, multiple strategies suggested for case studies were used (Lincoln and Guba, 1985; Baxter and Jack, 2008). In support of the credibility and dependability of interpretations, a third education researcher (MC) not involved with the inductive, deductive, and thematic analyses independently reviewed transcripts and summarized their overall interpretations. All researchers then came together as a team to discuss and compare interpretations to make revisions until consensus was reached. The combination of insider/outsider perspectives helped ensure interpretations were meaningful and not the result of individuals’ biases. Additionally, the inductive, deductive, and thematic analyses involved multiple researchers coding and analyzing data independently, followed by debriefing meetings to make revisions and come to consensus. MC also added to the trustworthiness of these findings by providing additional peer debriefing and scrutiny to better address biases and blind spots (Janesick, 2015). Finally, an audit trail of all coding schema and revisions was kept and reflected on between each step of the analysis process, helping support consistency in the analyses and the overall credibility of findings (Merriam and Tisdell, 2016).
Person | Number of contributions in pre-focus group | Number of contributions in post-focus group |
---|---|---|
a Contributions do not include reiteration or clarification of a previous point, nor nonverbal assents to previous points. | ||
Man 1 (Vice President) | 1 | 3 |
Man 2 | 5 | 7 |
Man 3 | 1 | 0 |
Man 4 | 0 | 6 |
Man 5 | n/a (not present at pre-focus group) | 0 |
Woman 1 (Chapter President) | 22 | 17 |
Woman 2 (President Elect) | 15 | 11 |
Woman 3 | 6 | 2 |
Woman 4 | 2 | 0 |
Woman 5 | 0 | 0 |
Woman 6 | 0 | 0 |
Woman 7 | 0 | n/a (not present at post-focus group) |
Wenger (2000) discussed that the degree of participation is dependent on one's competence and experience within the context of the CoP. Competence is “established over time by the community” while experience is “within the context of a given CoP and beyond.” For the chemistry outreach event that bounds this case, all college students had similar competence as this outreach event was the first of its kind planned by this chapter. Experience is based on everyone's past experiences outside of the chapter. Using these parameters alone, all members have the same level of participation; however, when looking at the contributions to the focus group, we see varying roles within the chapter when planning the outreach event.
CoP defines three different members within a community: core, active, and peripheral. In our analysis of the chapter's membership structure, we added a level inside core members for coordinators/leaders, which Wenger mentions as examples of potential core members (Wenger, 2000). This was done as a coordinator (i.e., Woman 1/Chapter President) had significant impacts on the planning process for the outreach event, described more in-depth below as part of research question 2; she initiated the outreach event and planning process, as well as managed the larger logistics for the outreach event. Core members were defined as those who spoke and contributed during the focus group discussions, as core membership requires a higher level of engagement with community activities than active members. As such, participants who participated in the focus group discussions but only contributed through nonverbal assents or verbal agreements/clarifications were labeled as active members. Peripheral members were defined as those who were part of this chapter/community but did not participate in the focus group discussions (i.e., were not part of this study); less active/involved members were discussed multiple times during the focus group discussions, providing evidence that this chapter does have peripheral members.
CoP also includes the idea of outsiders: those who are not part of the community of practice but are interacting with them. In this case, the outsiders were defined as the third-grade students who were the attendees of the outreach event. That said, they were not observed in this study as our focus was on the planning and evaluation process of the collegiate organization itself.
Leadership describes the role of certain members (referred to as coordinators) to push the CoP towards its goals or purposes. These leadership roles change over time as the goals of the CoP change or adapt, and as membership changes. In the focus groups, the idea of a leader was mentioned multiple times as the drive for planning the outreach event, evidenced by Man 2:
“I feel like once [President] kind of took the charge and told us it was like going down, we all kind of like collectively just started… just throwing ideas at the wall and seeing what stuck.”
The coordinator (i.e., the Chapter President) “took charge” which led to the rest of the event planning. It was also part of the chapter's observed planning process where one coordinator took charge of the organization and delegated tasks for the outreach event (described more in-depth below). Events refer to any gathering of those within the CoP to better understand their domain and to work towards their goals, likely including recurring meetings to bring members together. During the focus groups, Woman 2 mentioned using weekly chapter meetings as a time to plan for the outreach event, “Yea we probably spent a whole chapter [meeting] finding like small experiments that also like scaled-up experiments for the demo [show].” Membership refers to the processes for adding members to their community. This was also discussed during the pre-focus group. Woman 1 suggested, “We can take photos of us doing [the outreach event] and stuff. I think that it would even be good to like… in terms of like recruitment and getting [new] people.” She positioned the outreach event as a potential recruitment activity that would potentially help add new members to their community.
More connections were made to these processes by the elements the studied CoP did not evidence. Elements that were not mentioned, or were missing from their current community activities, were connectivity, artifacts, and learning projects. Connectivity refers to creating relationships between those who need help and those who can help within a CoP, generally based on experience and knowledge. In this case, we considered this as the relationships between the faculty advisor and the collegiate chapter members, as well as the more experienced members supporting less experienced members. Using the definition of a CoP, the faculty advisor may not be considered part of the CoP in this case based on discussions with the chapter that revealed minimal interactions between the chapter advisor and the students. Woman 1 mentioned a single meeting with the faculty advisor, but no other help or support from them. This is not shocking given earlier work on chemistry outreach practices that indicate students work relatively independently, despite university requirements for faculty advisors (Pratt and Yezierski, 2019b). In terms of connectivity based on experience, all the members had similar amounts of experience regarding chemistry outreach due to the impact of the COVID-19 pandemic and the loss of institutional and community knowledge. Therefore, there is a lack of evidence that this chapter/community has the connectivity element. However, students mentioned wanting more connections and support from faculty in their department, especially when evaluating their event. Woman 1 admitted:
“I think the overall perception in this department of [chemistry] outreach isn’t that strong so I was hesitant to even mention the [outreach event] to other faculty members, but I think in the future just more faculty support, like financial and otherwise would be helpful.”
Clearly, there are some gaps in the communication and connectivity between faculty and the members of this CoP/chapter. This could be a result of the students not reaching out to those with more experience, as suggested by the quote above. Additionally, the desire for more mentorship and support from faculty and chapter advisors echoes previous findings regarding collegiate student chemistry outreach practices (Pratt and Yezierski, 2019b).
Artifacts are symbols, documents, tools, processes, etc. that are maintained by the CoP for future use. Many members of this specific CoP described wanting such artifacts, during both the planning and evaluation discussions. One example is from Man 4 as he described what the group wanted for future outreach events:
“…a kind of set checklist…to set things up would be…a lot more streamlined in the future…Step like 1, make sure you have all this in order, here's who to talk to, this is the first step. Second step…make sure you have your materials. Just kind of a very…rigorous process for [outreach] instead of kind of make-it-up as we go along.”
Based on this chapter's relative newness as an organization and the effects of COVID-19, the chapter had little-to-no precedence for conducting in-person, large chemistry outreach events. This led to a “make-it-up as we go” approach from the student facilitators, much like the “winging it” approach previously reported (Pratt and Yezierski, 2019b). The final element of a CoP is learning projects, which are actions that serve to look at the growth of the CoP and the progress the CoP has made toward its goals. In this case of chemistry outreach, an example of a learning project would be training for outreach, with those with more experience teaching those with less experience; such training would support community learning and growth, as well as the lacking connectivity element discussed previously. Learning projects and artifacts also go hand-in-hand, and the students discussed wanting support, resources, tools, and/or training for chemistry outreach that would fill the gaps in their collective knowledge. Like the lack of connectivity, these ideas echo previous research where collegiate chemistry outreach practitioners want procedures with age-appropriate explanations or want information about how to successfully conduct training for outreach (Pratt and Yezierski, 2019b). These results are not new, nor unexpected, but reiterate the need for training to support chemistry outreach.
That said, while planning, Woman 1 mentioned wanting to increase brokering interactions between their chapter/CoP and other universities/CoPs with outreach programs to gain more knowledge (boundary objects). Boundary objects are essentially artifacts shared between CoPs; brokering is interacting with another CoP or outsiders to exchange knowledge and create connections between CoPs.
Based on these findings, we summarized this case/chapter/specific CoP in Fig. 3. In this figure, components of a mature CoP that were not evidenced in the studied CoP are greyed and crossed out. However, the majority of these greyed-out/crossed off components are areas this specific student organization were aware they are missing, as they discussed wanting many of those components. For example, Woman 1 discussed wanting boundary objects/artifacts: “I wish the department as a whole did more things like this so that way, we had like…even just like an idea bank to draw from.” In many instances, the participants discussed wanting various artifacts to better help their CoP grow, better broker new connections, and improve their chemistry outreach overall. This lends to the studied group's lack of experience and competence. Since they are a relatively new organization that was heavily impacted by the pandemic, there are parts of the CoP missing. These missing areas allude to the lack of maturity in terms of a CoP for this collegiate group. However, findings provide empirical evidence to support claims by Santos-Díaz and Towns that the CoP framework can successfully be used to describe and evaluate a collegiate chemistry organization conducting chemistry outreach.
The first step of their process was one coordinator taking charge and initiating event planning. During the pre-focus group, the President of the chapter (Woman 1, Chapter President) stated, “I know I personally am like very passionate about science outreach… I kind of like took charge on this project.” Throughout the focus group, the idea of this leader coordinating and guiding the planning process was echoed by other members of the group. The idea to host a chemistry outreach event came from this leader/chapter President, who also managed all logistical planning between the university and the elementary school. Other members of the chapter followed the President's lead.
The second step of their planning process was core members of the group generating ideas for the outreach event. For example, Man 2 said, “I feel like once [President] kind of took the charge and told us it was like going down, we all kind of like collectively just started… just throwing ideas at the wall and seeing what stuck.” Again, the President played a pertinent role in the process as her actions directed other members. Man 2 also noted, “Once she (President) gave us like a general format of how she wanted [the outreach event] to go down, it was kind of [up to] us to like figure out those smaller aspects.”
This alluded to the third step: coordination between coordinator, core, and active members for event logistics. The chapter President also described how chapter members self-organized for the outreach event in terms of those leading the hands-on activities and those presenting the demonstration show. These three planning steps stress the heavy reliance on chapter leadership during the planning process for the outreach event, mirroring previous work on leadership experiences in these collegiate organizations (Santos-Díaz and Towns, 2021). However, in this third step, broader chapter membership became involved, more than the leader and core members who controlled the first two steps of the process. The addition of active members in this step of the planning process provides further support for the CoP interpretation of collegiate groups conducting chemistry outreach.
The last step in the group's planning process was having a final plan for the outreach event. Here, the focus was on the goals the group used to inform event plans since the Framework emphasizes setting goals. Woman 3 summarized their event and tied each component to specific goals: “I think that the smaller [hands-on] experiments are kind of to like actually teach [kids] about different like science concepts…in a simple way. And like the big [demonstration] show is to be like ‘ooh, like super cool’ like large scale like really awesome science experiments.” These goals of audience learning and considering that science is enjoyable align with previous findings by Pratt and Yezierski who found that some goals for chemistry outreach events are audience learning and audience awareness that science is fun (Pratt and Yezierski, 2018a). Overall, this chapter discussed five goals for their outreach event that informed their planning process: (1) audience learning, (2) audience awareness that science is fun, (3) increasing accessibility to science, (4) generating interest, and (5) awareness of what science is; all of these are deductive goals that echo previous findings on chemistry outreach goals and purposes (Pratt and Yezierski, 2018a).
After interpreting the planning process for the chapter, findings were compared to the Framework (National Academies of Sciences, Engineering, and Medicine, 2016). The Framework discusses a logical, stepwise approach to planning, implementing, and evaluating an informal chemistry event, not dissimilar to backwards design approaches in formal education settings (Wiggins and McTighe, 2005). Specifically, the planning components of the Framework focus on setting goals and outcomes, then identifying resources and materials, and finally designing and preparing for the event (see Fig. 2). Throughout the analysis of the studied chapter's process, we found that the group did not follow a stepwise approach. Rather, they incorporated all three of these steps/components concurrently, shifting back-and-forth between them. The group was constantly going between designing activities, making/refining goals, and identifying the necessary resources. Consider this interaction between Woman 2 and Woman 1:
“I think blowing up a balloon with vinegar and baking soda would be really cool, but it would have to be large scale.” (Woman 2, President-elect). “We have balloons upstairs actually.” (Woman 1, President)
In the middle of the pre-focus group, Woman 2 began considering another potential demonstration, which she relates back to the goal of audience awareness that science is fun: “would be really cool” (Pratt and Yezierski, 2018a). Woman 1 then continued the conversation by identifying the potential materials they would need for the new demonstration. While the group shifted between the first three elements of the Framework throughout their planning process, it was more common to consider two elements simultaneously before shifting to the third. Woman 2 described the group's second step in the planning process of setting goals/outcomes and designing/preparing for the event together, “I know like when I like- when we were like googling stuff, I was looking mainly at what kind of concepts would be easy to teach in like an engaging way. But also, like conceptually for [kids]…able for [kids] to understand.”
It is beneficial that this chapter considered goals/outcomes as part of their planning process, which is contrary to previous findings that focused on individuals’ outreach experiences where goals/outcomes were rarely considered (Pratt and Yezierski, 2018b, 2019b). Additionally, the simultaneous shifting across the planning elements can help the group plan more freely (i.e., brainstorming) as the goals and resources can be revised throughout their planning process. However, the shifting between components as opposed to a stepwise process can also lead to confusion amongst the various members involved and a misalignment between goals and actions.
Additionally, it is worth noting that this group did not explicitly consider chemical safety in their planning process. While guiding authorities like the American Chemical Society and the Alpha Chi Sigma Fraternity have chemical safety guidelines and demonstration show planning guides that emphasize identifying, planning for, and mitigating safety hazards, the studied group only implicitly considered safety by thinking about the physical locations of the events and not specifically for attendees. For example, the group removed demonstrations that involved fire from any consideration as the demonstration show was conducted in an auditorium space in a non-chemistry building. Despite this, the group did ensure that safety goggles were provided to participants during the hands-on activities and that they wore appropriate PPE during all events, suggesting that safety was a consideration but did not strongly impact their overall practices. This mirrors previous findings that suggested that collegiate students conducting outreach have minimal understanding of their demonstrations and safety is not on the forefront of their minds (Pratt and Yezierski, 2018b).
The first step was reflections and self-assessment of goals for the outreach event. During the post-focus group that focused on evaluation, Man 2 reiterated some of their previously stated goals and evaluated their success in achieving those goals, which aligns with the Framework: “I think one of the goals was just to introduce kids to science…Show them like how fun it can be. And I'd say we achieved that.” One of these goals, awareness that science is fun, was discussed in the pre-focus group before the event and was used as part of the group's planning process. Man 2 reflected on their event and believed they had successfully shown their third-grade audience that science is fun. However, as shown in Man 2's quote, a new goal not previously discussed by the chapter was considered in the evaluation, exposure to science: “introduce kids to science.” While this goal has been seen in previous literature, it was not mentioned by the group in their planning process/pre-focus group discussion (Pratt and Yezierski, 2018a). This shows a mismatch between goals for planning purposes and those evaluated. Woman 2 continued to evaluate additional goals not previously discussed by the group, “I think a main goal was for just for [the event] to run smoothly and for everything to work out well, and it did.” These goals may have been implicit to individuals, but never explicitly shared within the group/stated in the focus group discussions. This new goal of logistic success, mentioned by Woman 2, was not mentioned during the group's planning process, nor has it been previously reported as a goal for chemistry outreach. This may not be surprising as such a goal could be considered an overarching goal for all events, in line with the spirit of the Framework for Effective Chemistry Communication in Informal Environments that focuses on evaluating event success. However, the consideration of new/different goals when evaluating events is concerning.
The second step the chapter followed was the evaluation of the preparation process (i.e., practicing). During this step, the students reflected on their process for preparing for the outreach event, which predominantly focused on practicing the activities/demonstrations before the event. Woman 2 mentioned, “I think if we had practiced like…a few more times it would have been good.” Here, she reflected on the fact that they did practice as part of their preparation, a novel finding given previous work on chemistry outreach that suggests students rarely practice prior to events (Pratt and Yezierski, 2019b). However, Woman 2 continues and suggests future events would benefit from additional practice—to support the goal of logistic success. During this discussion during the post-focus group, the students also mentioned a moment during their event where the elephant's toothpaste demonstration did not go as well as they had hoped. Woman 1 described that situation: “Yea I think the elephant toothpaste thing was… I don’t think we had high hopes after doing our practice runs…I think we knew that it would not…was not going to be spectacular. I feel… I still think it went well.” As discussed, despite practicing their activities/demonstrations prior to the event, there were still inevitable mistakes. Woman 1 (Chapter President) still counted this activity as a success, however, because it had the desired effect on the audience, thus the goal of “audience enjoyment” and “awareness that science is fun” was met.
The third step of their reflection process was collecting data from outreach event. All the data considered during their reflection and evaluation were informal and included four sources: (1) questions from kids during the outreach event, (2) comments from kids during the outreach event, (3) reactions of kids during the outreach events, and (4) messages from teachers and parents after the outreach event. Man 4 best discussed the questions he received from kids at the event, “I had a kid ask me about the uhm vacuum attachments…And it was it was cool…like one of the kids was actually interested in what goes on and how stuff works!” From Man 4's perspective, this evidences the achievement of the goal generate interest, one of the evaluated goals that also informed their planning process. It also supports the goal of exposure to science, another goal not mentioned in the planning process yet was part of the evaluation process. In another example, Woman 1 discusses data from other sources:
“I had several kids actually tell me they had fun. I had a ton of kids tell me like, ‘oh my gosh…this is so cool! This is awesome!’…you heard ALL of the kids in sync be like ‘WOW!’ and it was just…it was really cute…And also the teachers have already like messaged me and said that they've gotten messages from a ton of parents saying that their kids went home and said they had so much fun.”
Several third graders explicitly told Woman 1 they had fun, which supports the achievement of the goal audience enjoyment. While this goal was also found in previous literature, it did not contribute to the group's planning process (Pratt and Yezierski, 2018a). Woman 1's quote also discusses kids’ reactions which support the generate interest goal. While these discussions were explicit and show clear considerations from the chapter in how they reflected and evaluated themselves, other goals were considered during this discussion that were implicitly considered, including increase accessibility to science and awareness of what science is. These goals were mentioned in the pre-focus group during their planning for the outreach event, but the chapter did not explicitly reflect on the achievement of these goals during the post-focus group (Fig. 6). Lastly, Woman 1's quote above also discusses data through communications with the third-grade teachers and parents after the event. These messages provided data that reinforced that the kids had fun (audience enjoyment). Based on the data collected and considered by the chapter, audience enjoyment had the most support for being successfully met. While this is positive in terms of outcomes for event attendees, it is concerning from an event planning perspective as this goal was not used to inform the group's planning process (i.e., not discussed during the pre-focus group), which is contrary to the recommendation of the Framework. It is worth noting that this study solely focused on discussions with the college student event facilitators and did not involve observations of the outreach event itself. As such, all discussions of data and event details came directly from the student facilitators, relayed to the research team via the focus group discussions.
The last step of the group's evaluation process was the reevaluation of goals for future outreach events. As part of this, the group considered new goals to inform future outreach events. One such example was initially discussed by Man 2, “We should try to do some more [outreach events]…it's kind of a way to not only give back to our community but also like chip at that stigma of us being college students about being rowdy and stuff.” In this statement, Man 2 describes two new goals for future events: community service and break the stigma of college students. Both of these goals are novel and have not been previously discussed in the literature on chemistry outreach goals/purposes. Overall, the chapter discussed four new goals to consider for future outreach events: community service, break the stigma of college students, audience enjoyment, and gaining connections outside of the group. All four of these goals were only discussed in the evaluation process of the group; only audience enjoyment has been previously discussed in the literature and was reflected upon to evaluate the outreach event that bounds this case study. The other goals are novel goals for chemistry outreach but were not used by the group to evaluate their outreach event, only to inform future events.
Overall, the chapter did reflect and evaluate the success of their event, in line with the last step of the Framework (assess, reflect, follow up). Combined with the aforementioned discussion of how the group planned their event, we have a better understanding of how this specific CoP planned and implemented their outreach event. Summarized in Fig. 7, the Framework is adapted to represent the overall process of the studied group—including the concurrent consideration of the first three steps of the process, as evidenced by this group.
Even though the group set goals for their outreach event and evaluated them, the group's evaluation process showed inconsistencies between goals that informed the planning process and goals assessed in the evaluation process. Fig. 6 summarizes the overall relationships between each step of the planning and evaluation processes, and the chapter's various goals for the outreach event. During the evaluation, the student group primarily reflected on new goals rather than on their previously set goals from planning the event. A significant observation is the fact that the goal audience learning informed every step of the group's planning process, yet there was no mention of it during the evaluation process. While the chapter deemed the event successful, the mismatch between goals used to plan the event and those evaluated is concerning. Such a concern points towards the need for more chemistry outreach resources (e.g., guides, training) to improve collegiate students’ outreach practices, echoing findings from previous work that studied individual outreach facilitators (Pratt and Yezierski, 2019b). This may also suggest that students feel the need to sound successful when discussing their events, rather than critically evaluating their experiences. Additionally, the idea of creating reasonable/measurable goals may not have been clear prior to the event, relating to the group's minimal competence and experience in chemistry outreach as they are a relatively new chapter. It serves to mention that the act of having the focus group discussions may have impacted their process in discussing their goals, but based on the discussion, goal setting was already done prior to the pre-focus group discussion. The difference may be seen in terms of depth of discussion and would have less effects on the steps or goals stated during the focus groups. Overall, findings support a need for guidance and/or training so that this organization becomes more aware of the processes necessary for critically planning and evaluating outreach events.
This study strengthens the argument that a student organization can be considered a CoP, providing further support for using the CoP framework in future investigations of chemistry outreach practices. As this study focused on an individual case/chapter, future work should look at different student groups/chapters and how they plan outreach events, including their boundary interactions (e.g., faculty advisor involvement or connections with other student organizations). Furthermore, expanding to a larger sample of multiple student organizations/chapters is necessary to identify more transferability of these findings; as this is a case study, the results are highly context dependent. In addition to future work considering multiple chapters/contexts, more work is needed in the informal chemistry education/chemistry outreach realm as most studies focus on structured informal science (e.g., museum, planetarium), rather than voluntary, community-service-like events that are common for collegiate students.
Another interesting finding in our results was the idea that many of the pre-focus group goals were found in previous outreach studies. That said, during the evaluation process, this group generated novel goals. This was found for only the post-focus group, not the pre-focus group. This may allude to students having different mindsets during their planning and evaluation processes, which was not fully explored in this study and is a potential area for future work.
In addition to the future research avenues described above, our findings provide further support that specific training experiences for collegiate chemistry outreach are needed. In both focus group discussions, students repeatedly requested more resources and training to better support their planning and evaluation processes. By creating opportunities for such training, there may be an increase in overall outreach event success and impact on attendees. Previous research about CoPs indicates that targeting gaps in connectivity, rather than promoting connectivity as a whole, is more successful (Cross et al., 2006). Thus, training experiences may seek to first target connectivity gaps within a CoP/group before broadening to other areas/elements.
Pre-event focus group questions:
• What is the goal of this event? Are there multiple goals?
○ What outcomes are you hoping for?
○ Are there different outcomes for the participants and the practitioners?
○ Are your goals content focused? Emotion (affect) focused? Skills (psychomotor) focused?
• Describe the process the group uses to plan the event.
○ Is there a committee or individual that plans it?
○ Are multiple voices/perspectives included in the planning?
○ How are roles/duties assigned to plan and implement the event?
• Where did you learn how to plan an outreach event?
○ What resources or individuals have helped develop these skills?
• Are there resources or supports you wished were available to help you in this process?
• Is there anything specific you want us to know about how you approach planning your events or the outcomes you intend?
Post-event focus group questions:
• Remind us: What was the goal (or goals) for this event?
○ Were there different outcomes for the participants and the practitioners?
• How did the event go?
• Do you think the event was successful?
○ How do you know?
○ What about it was successful? What specific parts/activities/actions were successful?
○ What “data” did you gather to help you assess the success?
• Examples: formative/summative, numeric/survey, qualitative/reflective, observational
• What was “good” that you want to continue doing in future events?
• What would you like to improve for future events?
• Are there resources or supports you wished were available to help you in this process?
• Is there anything specific you want us to know about how your event went, your experiences, the participants/their experiences, etc.?
This journal is © The Royal Society of Chemistry 2024 |