Evaluating the effectiveness of POGIL-PCL workshops

Abstract

The POGIL-PCL (Process-Oriented Guided Inquiry Learning in the Physical Chemistry Laboratory) project has developed a series of workshops to introduce faculty to POGIL-PCL and to facilitate the development of new experiments. More than 60 faculty members from various institutions have attended these workshops. Workshop participants were surveyed in order to evaluate the effectiveness of the workshop and better understand why faculty choose to adopt POGIL-PCL. Of the participants who completed the survey, 77% had used POGIL-PCL experiments. The results of the survey show that personal influence is a major factor in all stages of the adoption process.

---

Introduction

Workshops are often used as a means of disseminating new teaching methodologies to interested faculty. Some studies on the effectiveness of workshops have focused mainly on the rate of adoption of a particular teaching method (Burke et al., 2004; Lewis and Lewis, 2006; Baker et al., 2009; Baker et al., 2014), while others have focused on the development of a community of instructors dedicated to reform (Paul and Volk, 2002; Wood and Gentile, 2003; Cox, 2004; Wood and Handelsman, 2004; Connolly and Millar, 2006; Lynd-Balta et al., 2006; Deignan, 2009; Sirum et al., 2009; Sirum and Madigan, 2010). Prior studies have addressed the usefulness of workshops designed to promote implementation of various active learning techniques, such as Just-in-Time Teaching, Problem-Based Learning, Peer-Led Guided Inquiry, and POGIL (Burke et al., 2004; Sirum and Madigan, 2010; Murray et al., 2011; Baker et al., 2014; Stains et al., 2015). Studies by Baker et al. (2014) and Stains et al. (2015) have investigated the impact of workshops designed to introduce faculty to multiple active learning techniques at three-day workshops. These studies have found that intensive workshops can have a significant impact on instructors' use of active learning techniques and that instructors often had a more student-centered approach to teaching after attending workshops (Burke et al., 2004; Gibbs and Coffey, 2004; Henderson, 2008; Murray et al., 2011; Baker et al., 2014; Stains et al., 2015). However, the extent of change is often variable, and some researchers have raised concerns about whether these changes are sustained long-term (Emerson and Mosteller, 2000; Lewis and Lewis, 2006; Murray et al., 2011; Stains et al., 2015). An understanding of the process of reform and the factors that lead to change is crucial in order to promote the implementation of innovative teaching methods. We begin with a discussion of the POGIL-PCL project and then explore the factors that influence instructors' decisions to adopt POGIL-PCL.

POGIL-PCL Workshops

The POGIL-PCL project has developed guided-inquiry experiments for the physical chemistry laboratory at POGIL-PCL workshops. The project objectives are to design, write, revise, and test guided-inquiry experiments, to encourage faculty to adopt POGIL-PCL, and to foster the community of physical chemistry faculty. POGIL-PCL workshops were designed to accomplish these goals. In addition, much of the work of the project continues after participants attend the workshops, including additional writing, reviewing, and testing. The workshops are described in detail in this section, and the structure of POGIL-PCL experiments is described more fully in the next section.

A sample agenda for the two-day workshops is shown in Appendix 1. The workshops open with participants doing a POGIL-PCL experiment. Immersion in a POGIL-PCL experiment allows faculty to get a sense of how the experiments work from students' perspectives and to observe how the experiments are facilitated. After this experience, participants then discuss the key elements of a POGIL-PCL experiment, the differences between traditional and POGIL experiments, and the facilitation of a POGIL-PCL experiment. Next, workshop participants review previously written experiments using a screening rubric (Appendix 2). The rubric emphasizes the key elements of POGIL-PCL experiments. By using the rubric, faculty better understand the general experiment structure and simultaneously learn about additional experiments.

A key part of the workshops is the opportunity to write a POGIL-PCL experiment. The writing portion of the workshops has been instrumental in expanding the number of POGIL-PCL experiments available, which is one of the goals of the project. Workshop participants work in groups of two to four people on the experiment they have chosen. Participants have revised existing POGIL-PCL experiments, converted standard physical chemistry experiments into POGIL experiments, or used their own ideas for new POGIL-PCL experiments. Up to eight hours over the course of the workshop are dedicated to writing and editing experiments. Instructors continue to work on the experiments after they leave the workshop in a variety of ways, including:

• writing and/or revising their own experiments and instructor's handbook

• testing the experimental procedure for newly developed experiments at their institution (alpha-testing)

• writing reviews of new experiments written by others

• implementing a fully-developed experiment (exactly as written by others) at their institution, and writing a review (beta-testing)

One criticism of some education research and dissemination methods is that instructors are simply given the materials and told what to do; the instructors have little or no involvement in developing materials. The lack of interaction between researchers and instructors often leads to problems with implementation (Henderson and Dancy, 2008). The writing portion of POGIL-PCL workshops addresses this concern by allowing instructors to focus on developing materials suitable for use in their own laboratories and by fostering collaboration among participants and researchers (facilitators). Development-style workshops have been developed for the POGIL Biochem project (Murray et al., 2011). The collaboration occurs not only between participants and facilitators but also between new participants and experienced participants (those who have previously attended workshops). The planned time for social interaction and networking among participants and facilitators further reinforces the collaboration and network among the new and experienced participants.

At the end of the workshop, participants provide feedback for the workshop facilitators. Participants are asked to fill out a Workshop Assessment form listing strengths of the workshops, the improvements the workshop needs, and any insights they gained from the workshop. The feedback provided by workshop participants is used to improve future workshops.

The workshops described in the previous paragraphs and illustrated in Appendix 1 are two-day workshops. The POGIL-PCL Project also holds half-day workshops at the Biennial Conference on Chemical Education (BCCE). Because the workshops at the BCCE are only a few hours long, the agenda emphasizes experiencing and then discussing a POGIL-PCL experiment. The workshops at BCCE provide instructors with an introduction to POGIL-PCL, but, because the BCCE workshops are so short, participants do not have as much time for the socializing and networking with other physical chemistry faculty that occurs at the two-day workshops. The BCCE workshops also do not include time for writing and development of experiments. Approximately half of the faculty who have attended the BCCE workshops have later attended the two-day workshops.

The workshops were advertised via email, announcements at pogil.org, conference presentations at regional and national American Chemical Society meetings, and at the 2012 Biennial Conference on Chemical Education.

The POGIL-PCL experiment structure

POGIL-PCL is a student-centered approach that utilizes guided inquiry and collaborative learning (Pavelich and Abraham, 1979; Allen et al., 1986; Deckert et al., 1998; Farrell et al., 1999; Spencer, 1999; Rudd et al., 2001; Lord and Orkwiszewski, 2006; Poock et al., 2007; Rudd et al., 2007; Schroeder and Greenbowe, 2008). The model of the POGIL-PCL learning cycle shown in Fig. 1 was developed through the discussion and testing of experiments at the workshops (Hunnicutt et al., 2015). Students work through multiple “data-think cycles” (see Fig. 1) in which they answer guiding questions, formulate a hypothesis, run an experiment, and analyze the resulting data. Specific experimental parameters such as concentrations or masses of reagents, volumes of reagents, and the number of trials necessary to obtain enough data, are determined by the students in small groups or as a whole class. The experiments end with in-depth questions that require students to apply the concepts they have learned. POGIL-PCL experiments rely on data-pooling by the students, so one experiment is often carried out simultaneously by the whole class. A few instructors who attended workshops have reported using POGIL-PCL experiments in a round-robin setting, in which different teams of two to four students carry out different experiments in the same class period.

Fig. 1 Diagram of the POGIL-PCL cycle. The experiment begins with the title as a question, includes multiple “data-think” cycles, and concludes with a set of post-experiment questions focusing on application. (Reproduced by permission from Hunnicutt et al., 2015.)

Process of change

Some of the faculty who attend POGIL-PCL workshops have previously attended workshops and currently use POGIL in their laboratory or lecture courses, while others are new to POGIL. The faculty who attend workshops include:

• those who have heard about POGIL and are interested in learning more,

• those who have begun to use POGIL at their institutions, and

• those who primarily use POGIL at their institutions.

The process of change from traditional laboratory teaching methods to using POGIL goes through several stages. The diffusion of innovations theory developed by Rogers describes the process of implementing change in five stages: knowledge, persuasion, decision, implementation, and confirmation (Rogers, 2003; Sahin, 2006). The first stage, knowledge, involves becoming aware of the innovation and learning how it is used. In the persuasion stage, an instructor begins to form an opinion based on the perceived advantages, disadvantages, and complexities of the innovation, the compatibility with their current environment, and the opinions of their peers. After learning of the innovation and forming an opinion about it, the instructor makes a decision to adopt or reject the innovation. An instructor may initially decide to adopt then later reject the innovation or discontinue use, or initially reject the innovation and later adopt it. Following the decision to adopt, the instructor enters the implementation stage. During the implementation stage, an instructor may seek support from more experienced users of the innovation. The final stage in this process of change is the confirmation stage, in which instructors seek support for their decision to adopt (Rogers, 2003; Sahin, 2006).

Instructors who attend POGIL-PCL workshops may be at any of the five stages in the diffusion of innovations. Instructors who are in the knowledge stage may be dissatisfied with the experiments they currently use and come to the workshops to learn about another way to teach a lab course, while those who are in the persuasion stage have heard about POGIL and want to learn more about it in order to decide whether they want to try POGIL experiments in their laboratory course. Some are in the implementation stage and are looking for support to deal with the challenges they have encountered or for more experiments for their course. More experienced participants may be in the confirmation stage and come to the workshop for the support from like-minded instructors.

After the workshop, instructors continue to move through these stages. Those who choose to implement may try previously developed experiments, or they participate in beta-testing. Beta-testing is the process of testing and reviewing experiments that are under development; once an experiment has been successfully tested, it may be considered ready for distribution to the community. After beta-testing the experiments with their students, instructors provide feedback to the authors of the experiment. This interaction plays an important part in developing experiments for the POGIL-PCL project and in supporting instructors in both the implementation and confirmation stages. Developers provide the experiments to instructors to implement with different groups of students, and instructors provide the developers with feedback on what did and did not work with the experiment.

As instructors move through the change process, many factors affect their willingness to adopt a new teaching method. These factors include interest in new methods, support from colleagues and administrators, time involved in implementing the new method, research showing the effectiveness of the new method, availability of resources, and the ease or difficulty of implementation (Jacobsen, 1998; Bennett and Bennett, 2003; Gess-Newsome et al., 2003; Rogers, 2003; Henderson, 2005; Sahin, 2006; Pundak and Rozner, 2008; Henderson et al., 2011; Olitsky, 2015). Dissatisfaction with current teaching methods is often considered a prerequisite for change (Gess-Newsome et al., 2003; Henderson, 2005; Pundak and Rozner, 2008; Olitsky, 2015). Personal influence is also considered to be a major factor in the adoption process, particularly for those who adopt later (Rogers and Beal, 1957). Those who adopt early, when an innovation is still new, may not have as many sources of personal influences because few people have adopted the innovation. Those who adopt later on are likely to seek information and support from early adopters who have more experience with the innovation (Rogers and Beal, 1957).

Research questions

POGIL-PCL workshops are designed to support instructors at various stages of implementation and to involve faculty in developing materials for POGIL-PCL experiments. In this study, we seek to understand what factors influence instructors at different stages of adopting POGIL-PCL. Specifically, the research questions are:

• What factors initially interest instructors in changing their teaching method?

• What factors affect whether instructors adopt new teaching methods?

• To what extent are the workshops effective? Do participants adopt POGIL-PCL experiments after the workshop?

Methods

Workshop assessment

At the end of each workshop, participants were asked for their feedback on the strengths of the workshop, the improvements needed for future workshops, and any insights they gained from the workshop. The SII (strength, improvements, and insights) forms were collected at five workshops from January 2012 through January 2014. The responses were analyzed for common themes such as networking, hands-on experience with POGIL, and insights into teaching.

Survey

The POGIL-PCL Survey was administered via REDCap (Harris et al., 2009). Participants who attended two-day POGIL-PCL workshops or half-day POGIL-PCL workshops at BCCE between January 2012 and January 2014 were contacted via email and asked to complete the survey. No identifying information was collected in the survey. Approval for the survey was obtained from the Institutional Review Board at Virginia Commonwealth University on July 29, 2014. The survey was open to workshop participants on September 5, 2014 and closed on October 13, 2014, giving participants approximately five weeks to respond. The response rate was 69% (62 email invitations were sent out; 43 completed responses were obtained). The survey included questions focusing on participants' experiences and attitudes both before and after the workshop.

Data analysis

For most of the survey questions, respondents were asked to check all answers that applied. Percentages for each answer choice were calculated from the total number of responses for each question. For several of the questions, there was an “other” response respondents could select if they did not agree with any of the available responses or if they had another response to add. After the survey was closed, the “other” responses were categorized with the provided responses or new categories were created based on common themes in the written answers. For example, for the question “How did you find out about the POGIL-PCL workshop,” several of the respondents who selected “other” stated that they heard about it at a POGIL national meeting or ACS meeting; these responses were placed in the category, “I attended a seminar or meeting in which POGIL-PCL was discussed.” Similarly, several of the respondents who selected “other” stated that they were invited by one of the organizers. These responses were categorized with the response “It was recommended by a colleague,” and the category was expanded to “It was recommended by a colleague, or I was invited by one of the organizers.” For the question, “What made you decide not to use POGIL-PCL,” a common theme among the “other” responses was that the instructor had not taught a lab course since the workshop; a new category was created to include those responses.

Results and discussion

The questions in the POGIL-PCL Survey were divided into two categories: those that address the factors that initially interest instructors in changing their teaching methods and those that address whether instructors adopt new teaching methods. Responses were analyzed for common themes in order to understand what motivates faculty in the earlier stages (knowledge and persuasion) and later stages (implementation and confirmation) of the change process. The SII responses were also analyzed for common themes related to the survey questions.

SII responses

Several common themes were seen in the SII responses, such as the value of networking, hands-on experience with POGIL-PCL labs, time to develop experiments, and insights into teaching physical chemistry laboratories. Since participants completed the SII forms before leaving the workshops, the SII forms yield insight into participants' initial impressions of the POGIL-PCL workshops. Most of the improvements suggested were related to scheduling or organization. Over time, there is a noticeable decrease in the percentage of workshop participants who suggested improvements. For two out of the first three workshops (January 2012 and February 2013), all of the respondents suggested improvements; in June 2012, 50% of the respondents suggested improvements. The percentage decreased to 37% for the July 2013 workshop and 40% for the January 2014 workshop, which suggests that the workshop effectiveness has improved over time. For every workshop, all participants listed multiple strengths, including collaboration or networking with other physical chemistry faculty and the opportunity to experience a POGIL-PCL experiment.

Survey: initial factors that engage instructors

One of the goals of the survey was to determine what factors initially interest instructors in changing their teaching methods. This goal addresses the knowledge and persuasion stages in the diffusion of innovations theory. While some instructors who attend the workshops may be in the later stages (implementation and confirmation), the goal was to understand instructors' initial thought processes when they first learned about POGIL-PCL. Three questions on the POGIL-PCL Survey addressed the factors that initially interest instructors to possibly change their teaching methods:

(1) How did you find out about the POGIL-PCL workshop?

(2) Which of the following describes your reason(s) for coming to a POGIL-PCL workshop?

(3) Which of the following would encourage (or has encouraged) you to use POGIL in your lab(s)?

The possible responses and percentages for each response for question 1 (How did you find out about the POGIL-PCL workshop?) are shown in Fig. 2. The two most common responses were “I attended a seminar or meeting in which POGIL-PCL was discussed” (45%) and “It was recommended by a colleague, or I was invited by one of the organizers” (40%). In contrast, a much smaller percentage of respondents selected the response “I read about it on the POGIL website” (6%). The data here indicate that actively speaking about the workshop with a person, such as a colleague or a seminar speaker, has a greater influence on an instructor's decision to come to a workshop and learn more about the POGIL-PCL than passive, impersonal communication such as reading about the workshop on the website. Rogers and Beal, as well as Jacobsen, also found that personal influence was a major factor in an individual's decision to adopt change (Rogers and Beal, 1957; Jacobsen, 1998). Rogers and Beal argue that impersonal communication (like mail) is easy to ignore and that individuals who are unsure about a new idea are more likely to be reached by person-to-person communication (Rogers and Beal, 1957).

Fig. 2 Percent of total responses for the question, “How did you find out about the POGIL-PCL workshop?” Respondents were asked to select all answers that applied.

Another question addressed instructors' reasons for coming to a workshop. The top three responses in Fig. 3 were given in the survey; the bottom three responses were common themes among respondents who selected “other.” The belief that students do not get much from traditional labs or experiments, the desire for students to be active learners, and feeling that students are more concerned with getting the correct answer than the process of learning accounted for over 90% of the responses. A desire to get students more engaged in lab and involved in the process of learning appears to play an important role in instructors' decision to come to a workshop, although the workshop itself may have influenced respondents' answers to this question. The survey responses are consistent with the discussion at the workshops and with the ultimate design of the POGIL-PCL experiments, as illustrated in Fig. 1, that arose out of these workshop discussions. Comments from instructors also supported this conclusion. One instructor commented, “I wanted access to labs that are more appropriate for upper level labs-that get students to think about what they are doing, why they are doing it, and what the results mean.” Another instructor commented, “I want my students to learn to think. Thinking rarely seems to occur in traditional labs.”

Fig. 3 Percent of total responses for the question, “Which of the following describes your reason(s) for coming to a POGIL-PCL workshop?” Respondents were asked to select all answers that applied.

The third question addresses the factors that encourage instructors to implement POGIL-PCL in their laboratory. Instructors' responses are shown in Fig. 4. Research and statistics showing the effectiveness and long-term benefits of POGIL had a large impact on instructors' decisions to implement POGIL; however, hands-on experience with POGIL labs or experiments received the highest response rate (22%). Hands-on experience was also a common theme among the strengths participants cited on the SII forms. The hands-on experience provided at the workshops allow instructors to see how the experiments work from the student's perspective and to observe how those who currently use POGIL in their lab courses facilitate the experiments. This experience is a key component of the third, or decision, stage of Roger's theory of the diffusion of change (Rogers, 2003). Another study also emphasized the importance of hands-on experience at workshops (Burke et al., 2004). Again, direct personal influence was also a factor in encouraging instructors to implement POGIL. Personal influence in this case occurred during and after the workshop in the form of support from the POGIL community, learning about POGIL from colleagues, and interacting with other physical chemistry faculty.

Fig. 4 Percent of total responses for the question, “Which of the following would encourage (or has encouraged) you to use POGIL in your lab(s)?” Respondents were asked to select all answers that applied. (The last answer, “personal experience using POGIL in lab,” was a common theme among those who selected “other.”)

Survey: factors that affect whether instructors adopt new teaching methods

To understand the factors that affected whether instructors implemented POGIL, survey respondents were asked, “After the POGIL-PCL workshop, how have you used POGIL in your lab(s)?” The goal here was to determine if instructors moved into the implementation stage after attending the workshop. If a respondent indicated they had used POGIL at all, they were asked what types of follow-up would be helpful to them. If they indicated they had not used POGIL since the workshop, respondents were asked what made them decide not to use POGIL.

Table 1 shows the percentages, sorted by frequency, of instructors who had used POGIL in their lab. Of the instructors who responded, 77% had used POGIL to some extent in their lab. (“To some extent” refers to the sum of the categories “mostly used POGIL labs,” “used POGIL labs regularly,” and “tried a few POGIL labs.”) As shown in Fig. 5, email/Skype with POGIL facilitators and follow-up workshops were the most commonly selected forms of support. Personal influence was a key factor in motivating instructors to change their teaching methods; the responses here indicate personal influence is also an important factor when instructors are making the decision to implement POGIL. Burke et al. (2004) also found that many workshop participants value the interaction provided at workshops, as it allows them to network with colleagues in their field and discuss challenges associated with implementation with more experienced practitioners.

Table 1 Percentage of respondents who use POGIL at each frequency

Mostly used POGIL labs regularly	19%
Used POGIL labs regularly	23%
Tried a few POGIL labs	35%
Do not use POGIL labs	23%

---

Fig. 5 Percent of total responses for the forms of support (email/Skype, follow-up workshops, or visiting with a facilitator or experienced POGIL practitioner) that would be helpful for instructors as they begin to implement POGIL. Respondents were asked to select all answers that applied.

Of the respondents who said they had not used POGIL since the workshop, the most commonly cited reason was that the instructor had not taught a lab course since the workshop (57%), followed by the course being taught round-robin style (29%). Using POGIL-PCL in a round-robin format could be an area for future study, given that a round-robin setting does provide some logistical challenges because POGIL-PCL experiments rely heavily on collaboration both within and between groups of students.

Survey respondents were also asked about their past experience with POGIL. Instructors were asked to recall whether their experiences had been mostly positive, somewhat positive due to either issues with the activities or resistance from students and/or other faculty, or if they had a lot of difficulty with the activities or resistance from students and/or other faculty. Over fifty percent of the survey respondents reported that their experience had been mostly positive. Given that instructors were more encouraged to implement POGIL in their laboratory when they had hands-on experience with the experiments, it seems that prior experience with POGIL would have an effect on an instructor's decision to use POGIL-PCL experiments. It was hypothesized that instructors whose prior experience was more positive would be more likely to use POGIL-PCL experiments. Fig. 6B shows that, when instructors reported having a mostly positive experience in the past, 83% had used POGIL-PCL experiments to some extent, with 32% mostly using POGIL-PCL experiments. (Again, “to some extent” refers to the sum of the categories “mostly used POGIL labs,” “used POGIL labs regularly,” and “tried a few POGIL labs.”) However, for instructors whose experience was not mostly positive (i.e. somewhat positive or a lot of difficulty), 67% had used POGIL-PCL experiments to some extent, and only 5% were mostly using POGIL-PCL experiments. As expected, those who had a more positive experience with POGIL before coming to the workshop were more likely to try POGIL-PCL after the workshop.

Fig. 6 (A) Percentage of instructors who reported having a mostly positive experience, somewhat positive experience, or a lot of difficulty with POGIL in the past. (B) Left: Percentage of instructors who reported using POGIL-PCL when their prior experience was mostly positive, right: percentage of instructors who reported using POGIL-PCL when they reported having a somewhat positive experience or a lot of difficulty.

It should be noted, however, that, even when instructor's experience was not mostly positive, some still chose to try POGIL-PCL after attending the workshop. Despite reporting difficulty with the activities and/or resistance from students and/or faculty in the past, 67% of instructors (Fig. 6B) had chosen to use POGIL-PCL experiments, with sizable percentages having tried a few experiments or regularly using the experiments in their labs. This result speaks to the effectiveness of the workshops. Although some instructors recalled less than positive experiences with POGIL in the past, they chose to attend a workshop and decided to try some of the experiments.

Open questions

There are several remaining questions regarding the workshops and the adoption of POGIL-PCL. Several instructors reported that they had not used POGIL-PCL experiments because they have not taught a laboratory course since the workshop. This study was conducted 7 months after the last workshop. Lewis and Lewis (2006) conducted a study on workshop effectiveness more than a year after the last workshop in a series of workshops and still found that a larger percentage of participants from the last workshop reported planned use of the activities rather than actual use (compared to participants from earlier workshops). This brings up the question of how long to wait after workshops are conducted to administer an implementer's survey. Conducting a survey too early means that some participants will not have had a chance to implement the new teaching method; however, waiting too long may make it more difficult to capture instructors' initial impressions and influences. In our case, more than 75% of survey respondents had used the experiments in their laboratories. Another follow-up survey would need to be done to determine whether that percentage would be significantly different if the survey were conducted at times long after the workshops attended.

There is also the question of whether instructors will continue to use POGIL years after they have attended the workshops. Five or ten years after their initial decision to implement POGIL, will instructors still be using POGIL-PCL? One study by Stains et al. (2015) found that, while instructors who attended workshops changed their teaching methods immediately after attending a workshop, these changes were not sustained a year later. In this study, the responses from instructors, particularly those regarding their reason for coming to a POGIL-PCL workshop, indicate that workshop participants believe in the goals of the POGIL-PCL project. Future studies will be necessary to determine how many instructors continue to use POGIL-PCL.

One question that is perhaps more difficult to answer is how instructors who have attended the workshops implement POGIL-PCL experiments. Do they implement the experiments as written, or do they modify them? How do different instructors facilitate the experiments? These questions can be answered to a certain extent by asking instructors about their implementation and has been addressed in a follow-up survey. When viewed collectively rather than individually, self-assessments can provide some indication of workshop success (D'eon et al., 2008). Faculty could also be interviewed individually or in focus groups. However, it is difficult to fully answer these questions without observing individual instructors because faculty are self-reporting their activities. Alternatively, faculty could be observed teaching their courses using an observation rubric developed for laboratory courses (Piburn et al., 2000; Smith et al., 2013; Lund et al., 2015).

Conclusion

The POGIL-PCL survey was designed to uncover factors that initially interested instructors in changing their teaching methods and factors that affected whether instructors adopted new teaching methods. The data collected from the survey show that personal influence (contact with colleagues, seminar speakers, facilitators, and other members of the POGIL community) is an important factor in the early stages of adoption when instructors are becoming aware of POGIL-PCL and deciding to learn more about it (knowledge and persuasion) and in the later stages of adoption when instructors begin to implement POGIL-PCL. While instructors who attend the workshops are in different stages of the process of change, 77% of instructors who responded to the survey moved into the implementation stage to some extent. Instructors' initial experience with POGIL also heavily influenced their decision to use POGIL-PCL; of those who had a mostly positive experience 32% were mostly using POGIL-PCL, compared with only 5% of those whose experience was not mostly positive.

Several open questions have been discussed, including how long to wait after the workshop to conduct a survey and how different instructors implement POGIL in their laboratory courses. A follow-up survey has been conducted to address these questions. Future papers will discuss results of this follow-up survey and focus on the development of a POGIL-PCL network over time.

Appendix 1 (sample workshop agenda)

Appendix 2 (screening rubric)

Laboratory Experiment Initial Screening Rubric, POGIL-PCL (Cole and Bauer, 2012)

This rubric guides initial review of a laboratory experiment to determine how well the activity supports process-oriented and guided-inquiry learning in the physical chemistry laboratory. The review pertains only to the written description of the experiment, protocol, and pre/post lab questions included and not to how an instructor might facilitate its use. A “yes” response should indicate that evidence for that characteristic can be found in the activity itself. The evidence must be explicit, i.e. reviewers should not assume that an instructor using the activity will provide anything that seems to be missing. If evidence is found, then a “yes” response is appropriate, irrespective of the perceived quality of that characteristic. If no explicit evidence can be found, then a “no” response is appropriate.

References

1. Allen J., Barker L. and Ramsden J., (1986), Guided inquiry laboratory, J. Chem. Educ., 63, 533.
2. Baker T. R., Palmer A. M. and Kerski J. J., (2009), A national survey to examine teacher professional development and implementation of desktop GIS, J. Geogr., 108, 174–185.
3. Baker L. A., Chakraverty D., Columbus L., Feig A. L., Jenks W. S., Pilarz M., Stains M., Waterman R. and Wesemann J. L., (2014), Cottrell scholars collaborative new faculty workshop: professional development for new chemistry faculty and initial assessment of its efficacy, J. Chem. Educ., 91, 1874–1881.
4. Bennett J. and Bennett L., (2003), A review of factors that influence the diffusion of innovation when structuring a faculty training program, The Internet and Higher Education, 6, 53–63.
5. Burke K., Greenbowe T. J. and Gelder J. I., (2004), The multi-initiative dissemination project workshops: who attends them and how effective are they? J. Chem. Educ., 81, 897.
6. Cole R. and Bauer C., (2012), University Of Iowa and University Of New Hampshire, Personal communication.
7. Connolly M. R. and Millar S. B., (2006), Using workshops to improve instruction in STEM courses, Metropolitan Universities, 17, 53–65.
8. Cox M. D., (2004), Introduction to faculty learning communities, New Directions for Teaching and Learning, 2004, 5–23.
9. D'eon M., Sadownik L., Harrison A. and Nation J., (2008), Using self-assessments to detect workshop success – do they work? Am. J. Eval., 29, 92–98.
10. Deckert A. A., Nestor L. P. and Dilullo D., (1998), An example of a guided-inquiry, collaborative physical chemistry laboratory course, J. Chem. Educ., 75, 860–863.
11. Deignan T., (2009), Enquiry-based learning: perspectives on practice, Teach. High. Educ., 14, 13–28.
12. Emerson J. D. and Mosteller F., (2000), Development programs for college faculty: preparing for the twenty-first century, in Branch R. M. and Fitzgerald M. A. (ed.) Educational Media and Technology Yearbook, 2000, Englewood, CO: Libraries Unlimited.
13. Farrell J. J., Moog R. S. and Spencer J. N., (1999), A guided-inquiry general chemistry course, J. Chem. Educ., 76, 570.
14. Gess-Newsome J., Southerland S. A., Johnston A. and Woodbury S., (2003), Educational reform, personal practical theories, and dissatisfaction: the anatomy of change in college science teaching, Am. Educ. Res. J., 40, 731–767.
15. Gibbs G. and Coffey M., (2004), The impact of training of university teachers on their teaching skills, their approach to teaching and the approach to learning of their students, Active Learning in Higher Education, 5, 87–100.
16. Harris P. A., Taylor R., Thielke R., Payne J., Gonzalez N. and Conde J. G., (2009), Research electronic data capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support, J. Biomed. Inf., 42, 377–381.
17. Henderson C., (2005), The challenges of instructional change under the best of circumstances: a case study of one college physics instructor, Am. J. Phys., 73, 778–786.
18. Henderson C., (2008), Promoting instructional change in new faculty: an evaluation of the physics and astronomy new faculty workshop, Am. J. Phys., 76, 179–187.
19. Henderson C. and Dancy M. H., (2008), Physics faculty and educational researchers: divergent expectations as barriers to the diffusion of innovations, Am. J. Phys., 76, 79–91.
20. Henderson C., Beach A. and Finkelstein N., (2011), Facilitating change in undergraduate STEM instructional practices: an analytic review of the literature, J. Res. Sci. Teach., 48, 952–984.
21. Hunnicutt S. S., Grushow A. and Whitnell R., (2015), Guided-inquiry experiments for physical chemistry: the POGIL-PCL model, J. Chem. Educ., 92, 262–268.
22. Jacobsen D. M., (1998), Adoption patterns of faculty who integrate computer technology for teaching and learning in higher education, Social Sciences and Humanities Research Council of Canada.
23. Lewis S. E. and Lewis J. E., (2006), Effectiveness of a workshop to encourage action: evaluation from a post-workshop survey, J. Chem. Educ., 83, 299.
24. Lord T. and Orkwiszewski T., (2006), Moving from didactic to inquiry-based instruction in a science laboratory, Am. Biol. Teach., 68, 342–345.
25. Lund T. J., Pilarz M., Velasco J. B., Chakraverty D., Rosploch K., Undersander M. and Stains M., (2015), The best of both worlds: building on the COPUS and RTOP observation protocols to easily and reliably measure various levels of reformed instructional practice, CBE-Life Sci. Educ., 14, 1–12.
26. Lynd-Balta E., Erklenz-Watts M., Freeman C. and Westbay T. D., (2006), Professional development using an interdisciplinary learning circle, J. Coll. Sci. Teach., 35, 18–24.
27. Murray T. A., Higgins P., Minderhout V. and Loertscher J., (2011), Sustaining the development and implementation of student-centered teaching nationally: the importance of a community of practice, Biochem. Mol. Biol. Educ., 39, 405–411.
28. Olitsky S., (2015), Facilitating changes in college teaching practices: instructional reform, identity conflict and professional community in a K-20 partnership, Res. Sci. Educ., 45, 625–646.
29. Paul G. and Volk T. L., (2002), Ten years of teacher workshops in an environmental problem-solving model: teacher implementation and perceptions, J. Environ. Educ., 33, 10–20.
30. Pavelich M. J. and Abraham M. R., (1979), An inquiry format laboratory program for general chemistry, J. Chem. Educ., 56, 100.
31. Piburn M., Sawada D., Turley J., Falconer K., Benford R., Bloom I. and Judson E., (2000), Reformed teaching observation protocol (RTOP) reference manual. Tempe, Arizona: Arizona Collaborative for Excellence in the Preparation of Teachers.
32. Poock J. R., Burke K. A., Greenbowe T. J. and Hand B. M., (2007), Using the science writing heuristic in the general chemistry laboratory to improve students' academic performance, J. Chem. Educ., 84, 1371–1379.
33. Pundak D. and Rozner S., (2008), Empowering engineering college staff to adopt active learning methods, J. Sci. Educ. Tech., 17, 152–163.
34. Rogers E. M., (2003), Diffusion of Innovations, New York: Free Press.
35. Rogers E. M. and Beal G. M., (1957), The importance of personal influence in the adoption of technological change, Soc. Forces,36, 329.
36. Rudd J. A., Greenbowe T. J., Hand B. M. and Legg M. J., (2001), Using the science writing heuristic to move toward an inquiry-based laboratory curriculum: an example from physical equilibrium, J. Chem. Educ., 78, 1680–1686.
37. Rudd J. A., Greenbowe T. J. and Hand B. M., (2007), Using the science writing heuristic to improve students' understanding of general equilibrium, J. Chem. Educ., 84, 2007–2011.
38. Sahin I., (2006), Detailed review of Rogers' diffusion of innovations theory and educational technology-related studies based on Rogers' theory, Online Submission, 5.
39. Schroeder J. D. and Greenbowe T. J., (2008), Implementing POGIL in the lecture and the science writing heuristic in the laboratory-student perceptions and performance in undergraduate organic chemistry, Chem. Educ. Res. Pract., 9, 149–156.
40. Sirum K. L. and Madigan D., (2010), Assessing how science faculty learning communities promote scientific teaching, Biochem. Mol. Biol. Educ., 38, 197–206.
41. Sirum K. L., Madigan D. and Klionsky D. J., (2009), Enabling a culture of change: a life science faculty learning community promotes scientific teaching, J. Coll. Sci. Teach, 38, 38–44.
42. Smith M. K., Jones F. H., Gilbert S. L. and Wieman C. E., (2013), The classroom observation protocol for undergraduate STEM (COPUS): a new instrument to characterize university STEM classroom practices, CBE-Life Sci. Educ., 12, 618–627.
43. Spencer J. N., (1999), New directions in teaching chemistry: a philosophical and pedagogical basis, J. Chem. Educ., 76, 566–569.
44. Stains M., Pilarz M. and Chakraverty D., (2015), Short and long-term impacts of the cottrell scholars collaborative new faculty workshop, J. Chem. Educ., 92, 1466–1476.
45. Wood W. and Gentile J., (2003), Meeting report: the first national academies summer institute for undergraduate education in biology, Cell Biol. Educ., 2, 207–209.
46. Wood W. B. and Handelsman J., (2004), Meeting report: the 2004 national academies summer institute on undergraduate education in biology, Cell Biol. Educ., 3, 215–217.

---