Edinburgh Research Explorer Flipped learning in higher education chemistry: emerging trends and potential directions

Flipped learning has grown in popularity in recent years as a mechanism of incorporating an active learning enviroment in classrooms and lecture halls. There has been an increasing number of reports for flipped learning in chemistry at higher education institutions. The purpose of this review is to survey these reports with a view to examining the rationale for adopting the flipped learning approach, how educators have implemented the flipped learning approach into their own practice and how these implementations have been evaluated. The reports are analsyed for emerging themes on the benefits and challenges of integrating this approach in chemistry education at university level, with a view to understanding how we can continue to develop the approaches taken for implementation of flipped learning methods in higher education chemistry. Analsyis of the articles surveyed indicate that the approach is highly popular with students, with educators adopting it as a means of developing an active learning environment, to increase engagement, and to allow time for developing a deeper understanding of the discipline. Despite the approach being open-ended in terms of how it can be implemented, there is some uniformity in how it has been adopted. These approaches are discussed, along with lessons learned from evaluations, with some suggestions for future iterations so that the implementation relies on evidence-based methods.


Introduction
Flipped learning has emerged in recent years as a popular alternative to traditional teaching methods. Originally conceived as a means of allowing all learners to engage with lecture material (Lage et al, 2000), it has been formalised into a pedagogical approach for presenting material to students in advance of class and enabling active learning environments to take place during formal class time. In response to some misinterpretations and misconceptions of what flipped learning is, the Flipped Learning Network issued the following definition (Flipped Learning Network 2014): "Flipped Learning is a pedagogical approach in which direct instruction moves from the group learning space to the individual learning space, and the resulting group space is transformed into a dynamic, interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter." Key to this definition and the approach of flipped learning is that provision of material in advance of class is just one component of flipped learning. Historically, there are several examples of provision of materials in advance of chemistry lectures (e.g. Kristine 1985;Collard Girardot and Deutsch 2002) as a means of getting students to engage with subject material prior to lectures. However flipped learning aims to harness this pre-lecture preparation to subsequently change the format of the lecture time, from a mainly passive activity to one primarily focussed on student activity.
Flipped learning is perhaps unusual as it has emerged directly from classroom practice, promoted as a technique that worked well, rather than something drawn from educational theory. Two chemistry teachers tried the approach in their classroom and after observing some positive effects with their students, wrote a book which has become highly influential (Bergmann and Sams, 2012).
In their scoping review of flipped lectures, O'Flaherty and Phillips surveyed 28 articles across a range of disciplines with a view to exploring the technologies and implementation approaches used, the acceptance of staff and students, the educational outcomes, and the presence of a conceptual framework for developing a flipped learning approach (O'Flaherty and Phillips, 2015). Of relevance here is the last point: the review found that while there was good intention, there were limitations in the capacity of staff to design, implement, and evaluate flipped classrooms in their own practice. One purpose of this review is to generate a roadmap for those interested in pursuing this approach based on experiences emerging recently from those working in higher education chemistry. Why should an educator consider flipped learning? A dilemma with an approach that has risen up from the "chalk-face" is that it can be difficult to rationalise its implementation, other than a desire to change from an unsatisfactory status quo, or an intuition that it may make sense. However, as the method has become more popular, consideration is now been given to ground the flipped learning approach within an educational framework such as cognitive load theory and selfdetermination (motivation) theory (Abeysekera and Dawson, 2015).
Cognitive load theory is based on the notion that the process of learning imposes a load on the working memory, a limited, finite processing space (Mayer, 2005). Material that is new to novice learners will impose an intrinsic load, based on its implicit delivery, an extraneous load, dependent on how difficult it is to extract information from the learning materials, and germane load, the process of integrating the new information with what is already stored in the long term memory. Given that working memory is a finite space, if the intrinsic and extraneous load are substantial, little capacity is available for processing new information, and thus the extent of learning will be low (Johnstone, 1994). Reducing cognitive load by introducing material in advance of lectures already has some basis in chemistry, with reports on completing some advance activity sheets (Sirhan et al, 1999), web assignments (Sirhan and Reid, 2001), and preparatory pre-lecture activities (Seery and Donnelly, 2011;. It is proposed that students' ability to work through material in advance of lectures at a pace that suits individual learners may reduce cognitive load and help learning in a flipped class environment (Abeysekera and Dawson 2015).
As well as considering a rationale for change based on educational theory, there is a continuing frustration among many educators with the over-reliance on one pedagogic approach (the didactic lecture, and variants of it) in chemistry education (Byers and Eilks, 2009). Part of the reason for the predominance of this model is that alternatives must be viable and coherent, and seen to be 'rigorous' by chemistry faculty (Talanquer and Pollard, 2010). Nevertheless, there is an acknowledgement that chemistry, especially at an introductory level is currently taught in a manner that is encyclopaedic, aiming to cover too much in an abstract and disconnected way (Bodner 1992;Goedhart 2015). Developments, when they do occur, can often be seen as piecemeal (Bennett and Overton, 2010) and hence lack overall cohesion and impact. From students' perspective, there is an implicit assumption that the one-direction transfer of information in large lecture halls emphasises a "sink or swim" attitude (Black and Deci, 2000). A potential benefit of the flipped learning approach is that it is not a single-point intervention, such as providing revision quizzes or online lectures for review, but rather a holistic pedagogic scaffold upon which to build a curriculum delivery strategy (Seery and Mc Donnell, 2013).
Implementation and reports in the peer-reviewed literature of flipping chemistry in higher education have lagged behind those in other disciplines such as health sciences, engineering, and mathematics (Abeysekera and Dawson, 2015). In addition, there are a number of trials reported at well as school level, both generally (Goodwin and Miller, 2013) and for chemistry (Schultz et al, 2014;Bergmann and Sams, 2012). For chemistry at university level there are reports of "flipped laboratories" (Teo et al, 2014;Fung, 2015).
However, in the last two years, there has been a number of reports from higher education chemistry classrooms. The purpose of this review is to survey these, with a view to answering the following questions: 1. What is the rationale for lecture flipping in HE chemistry? What basis have authors provided for adopting this method? 2. What approaches have been used with the implementation of lecture flipping? What happens before, during and after class time? 3. How is the implementation monitored and evaluated?
In particular: a. What is the student feedback from the implementation of this approach? b. What evidence is there that the approach leads to an improvement in knowledge, attributes, and/or skills? 4. What can we learn from the studies published so far in continuing to implement and evaluate this technique?

Method
In order to source a comprehensive set of useful articles for this study, a series of criteria were imposed on the results obtained from database searches. Shortlisted articles were collected based on the following conditions: (1) articles must be published in a publication that employs peer-review; (2) articles must implement flipped learning approach in a higher education chemistry along the lines of the definition provided above, namely there should be a pre-lecture component enabling an in-class active learning component; (3) articles should include some evaluation of the approach, either in terms of student opinion, engagement, and/or performance.
To source articles, the Web of Science and ERIC databases were used. Search terms "flip*" or "inverted" were used in conjunction with "chemistry" and the results subsequently filtered by category to identify education related papers, and manually by abstract to identify those pertinent to chemistry education. Citing and cited articles were explored to identify any that fit the criteria. Once a list of articles had been compiled, a further search on Google Scholar was used to identify additional sources, with the term "lecture or class" being added to the search criteria. Having applied these filters, 12 articles (Table 1) were compiled and found to fit the criteria (Butzler, 2015;Christiansen, 2014;Fautch, 2015;Flynn, 2015;Rein and Brookes, 2015;Rossi, 2015; Findings 1. Rationale for flipped learning The selected articles were surveyed to examine the rationale for changing to the flipped learning approach. In several cases, no rationale was provided, or there was simply an (often implicit) dissatisfaction with the current mode of teaching. The desire to improve the quality of (Christiansen, 2014;Yeung and O'Malley, 2014) and engagement with face-to-face time were the motivation some, while in organic chemistry in particular, the approach was considered a way to provide time to cover both the course content and getting sufficient time for working through problems (Rossi, 2015;Fautch, 2015). In other cases, there was a sense of "trying out" a new method, based on some findings that indicated positive benefits to learning Yestrebsky 2015). A more detailed theoretical framework is provided by Flynn who bases her approach in constructivism, arguing that the time allowed in class is providing students with an additional opportunity to construct their own knowledge in the social setting of a classroom (Flynn, 2015). In addition, the possibility of reducing in-class cognitive load is proposed, based on previous work done in chemistry mentioned above (Seery, 2015).

Approaches to flipped learning
Lecture flipping is considered to be a philosophy rather than a particular approach to teaching (Bergmann and Sams, 2012). The articles sourced that dealt with lecture flipping in higher education were surveyed to gain a general sense of the approaches made to teaching at different stages: what happened prior to the lecture; was there any requirement (assessment) for work before or during the lecture; what happened during the lecture; was there any follow up after the lecture. These are surveyed below and summarised in Figure 1. The articles surveyed show a great deal of similarity with the approaches taken.

Prior to Lecture
While advocates of lecture flipping propose that advance material can be provided in a variety of formats (textbook, web-pages, video, etc), the predominant method in the articles surveyed was to use PowerPoint recordings with voice narration, known as screencasts. These were hosted on video sharing sites such as YouTube or institutional virtual learning environments. Some authors noted the value of providing a schedule or calendar to students so that the structure remained clear and consistent throughout (Flynn, 2015;Seery, 2015). Those who hosted externally to their institution have pointed to the usefulness of comments from external users in identifying areas and area where clarity was needed. (Christiansen, 2014). In one case, it was feared that a complete conversion to a flipped approach would overwhelm students, and so a partial flip (one lecture out of three per week) was implemented (Trogden, 2015).
Lengths of screencasts varied, although several authors noted that the lengths were significantly shorter than the equivalent time that would have been used in a lecture (Table 1). This is explained by the fact that typical lectures would require time to settle the class, deal with student queries, and allow for student activities, none of which are a concern for screencasts covering content. Some lecturers opted for a sequence of very short screencasts. Smith (2013) created 14 hours of lecture material for a semester of general chemistry, with individual screencasts ranging from 1 to 12 minutes in length, and five and a half minutes being the average. Flynn (2015) reported that for a first semester organic chemistry course, 28 videos totalling 6.9 hours were prepared. Aiming for between 5 and 10 minutes per video, she found her average video length was 9.11 minutes. Fautch provides a detailed list of lecture topics and states that the average time was typically 20 minutes (Fautch, 2015). Time to prepare screencasts was also noted by several authors. Flynn proposes a ratio of 1:10 produced material: preparation time. Christiansen estimated that lecture flipping preparation took nearly three times as much time as a traditional lecture format would require. As well as videos, accompanying notes were typically provided in advance. Flynn provided material that was too difficult or time consuming to copy out, such as spectra, with space for students to annotate (Flynn, 2015). Seery provided an outline structure of notes and diagrams with spaces for students to work out problems in advance of class (Seery, 2015). Please do not adjust margins Please do not adjust margins case, introductory video recordings of the lecturer were incorporated into Prezi presentation software, with accompanying static lecture notes, audio clips, videos of worked solutions, etc, all available in a mind map format. As Prezi works on a zoom-in-zoom-out navigation, the authors used this approach to allow students test themselves by displaying a question, and having the answer available when the zoom-in was activated. Examples from this implementation are available on the internet (Fitzgerald and Li, 2013).

Incentivising Pre-Lecture Work
As the flipped lecture model requires students to engage with material in advance of class time, most authors considered how this could be incentivised. This was usually achieved by having a quiz to complete after watching the screencast before class (Fautch, 2015;Flynn, 2015;Seery, 2015) or in class time (Christiansen, 2014;Smith, 2013). Typically these quizzes were worth a proportion of the module grade. When stated, it was usually worth 5% (Flynn, 2015) or 10% (Christiansen, 2014;Seery, 2015).
An alternative to providing a quiz prior to or during the lecture was to assign a mark to students for problem solving or other activities during the lecture. To encourage attendance to be above average for traditionally delivered lectures, problem solving was awarded with a small assessed component (amount not stated) for a final year student group (Yeung and O'Malley, 2014). In-lecture problem solving work was awarded a mark (combined total being 20% of module grade), including a peer-assessed mark (see below) by Christiansen (2014). Rein and Brookes used case studies, and in one iteration of their module, awarded 10% to students for presenting a case study in class. The consideration in all of these examples was that the student needed to come to the class time prepared, and hence there was a grade incentive to watch the material in advance of the lecture.

Activities during the lecture time
The rationale for flipped lecturing centres around the fact that it allows for more active learning to take place during the valuable face-to-face time the lecturer has with students. With a few exceptions, problem solving was the dominant activity during class time. Fautch required students in organic chemistry to work through problem sets in groups, with students being asked to report answers on the whiteboard periodically (Fautch, 2015). Smith used the problem-solving section of his general chemistry lecture to present students with some worked examples and followed up with problems that students were required to work through, considering whether they could use similar assumptions to the worked examples, and encouraging them to "think like a chemist"applying chemical reasoning to their approach in working  (2015) Organic I (~400) 9.11 9.11 28 6.9 Organic II (~400) 9.04 9.04 24 3.6 Spectroscopy (~140) 11.31 11.31 17 3.2 Spectroscopy in French (~17) Rein and Brookes (2015) Organic Chemistry (225, (Smith, 2013). Flynn devised some clicker questions from those which had been answered poorly from pre-class work, thus bridging the pre-class activities with the in-class activities.
This bridging of pre-class and in-class was also considered by both Fautch and Seery. Students were asked in pre-class quizzes what areas were causing difficulty. In the case of Fautch, students were told that their pre-class quiz was ungraded unless this question about what was causing difficulty was answered (Fautch, 2015;Seery 2015). These topics ("muddy points") often opened the in-class time with students through the provision of a "mini-lecture" at the beginning of class time, followed by the problem solving activities. Most authors used the "Just in Time Teaching" concept, either informed by pre-class difficulties or difficulties raised in class to re-cover some concepts or ideas that were causing difficulty (Seery, 2015).
While most authors facilitated or required group work during the class time, in some instances it was formalised and included as part of the assessment of the module. As mentioned, Christiansen (2014) included a peer-assessment which was worth 20% for eight assessments over the module. This involved grading one of the problems sets from each group, with each group member being given the same grade, weighted by a peer-assessment. This was calculated from an average of peer-grades from three classmates, with the weighting ranging from 0.5 (F) to 1 (A). In an implementation with non-chemistry majors, students were given a group casestudy, which require each group to prepare a 10-minute presentation on the topics they have learned during the course. Examples included "Fix-a-Flat", based on cationic polymerisation and "Curcumin in Turmeric", based on ketoenol tautomerism, pK a , and UV/vis spectroscopy. Presentations were accorded 10% of the final module grade (Rein and Brookes, 2015).
None of the articles considered the use of Peer-Instruction (PI). Although it is a popular method used in conjunction with flipped lectures in other disciplines (Mazur, 1997), it has yet to be formally reported in chemistry, although the general approach has been described (Lancaster and Read, 2013;Lancaster, 2013;Sleazak, 2014).

Activities after class
Including formal after-class activities was uncommon in the articles surveyed. Only Flynn describes the issuing of post-class assignments, which she describes as more challenging that the pre-class tests (Flynn, 2015). These typically required students to think more deeply about questions that they had covered, considering alternative approaches, etc. They were rewarded with 10% of grade. Smith required students to complete online homework drawn from textbook chapter material once the relevant content had been covered in class. Students could choose when they completed it, with a typical deadline of one to one and a half weeks after class (Smith, 2013).

Evaluation of lecture flipping
The question most educators want answered regarding any new approach to teaching is: does it work? Definitions of what that means varies widely; improvements in satisfaction, improvements in examination grades, additional learning outcomes, developing student autonomy, and more. In general, two perspectives have been considered in reports on lecture flipping in HE chemistry. Firstly, student satisfaction surveys abound. These provide a sense of student acceptance (or not) of the approach, but also offer clues on any changes to student approaches to learning. Secondly, learning gains (if any) are explored, aiming to demonstrate whether the approach leads to an improvement in examination scores. These are discussed below.

Student opinions on lecture flipping
All of the articles surveyed considered student feedback in their evaluation of the approach. There was an overwhelming agreement that students liked the approach. Response scores and student comments repeatedly stated that they preferred the approach to whatever method they were used to elsewhere. Smith surveyed general chemistry students and reported that 81% found the flipped approach "more useful and/or enlightening", with 13% neutral. Students in different years of the same institution gave similar responses: at University of Manchester, 74% of 2 nd year students and 85% of 4 th year students reported that they believed "flipped teaching is better than the traditional lecture-based method" (Yeung and O'Malley, 2014), while at University of Ottawa, already high course evaluations by 1 st Year and 3 rd Year students further improved (Flynn 2015). Some open response surveys were used to elicit opinions from students on what they liked and disliked about the approach. Some common themes emerged and are presented in Table 2 (positive) and Table 3 (negative).
Students reported that the liked to be able to access the material in their own time. While the motivation for this may have been encouraged by some form of pre-lecture or inlecture assessment, there is a general finding that students did access the material as required in advance of class. A typical quote is shown in Table 2. Christiansen reports high quiz grades, indicating that students watched the video prior to completion (Christiansen, 2014). Seery tracked screencast usage directly, and found >90% of students consistently watched the screencasts prior to class (Seery, 2015). Fautch reported high agreement when students were asked if listening to lectures in advance and doing problems in class was effective, and whether it was more effective than listening to lectures in class and doing problems at home (Fautch, 2015). Smith surveyed General Chemistry students on the length of videos (see Table 1 for details on his screencast Please do not adjust margins Please do not adjust margins times) and found that students generally felt the length appropriate, with a reluctance for the videos to be any longer (Smith, 2013).
The organisation of material with advance activities, questions to complete, and in some cases questions after class, provided opportunities for students to engage with the lecture material several times. This appeared to structure their approach to study. Indeed Smith demonstrated that his students watched videos on average of three times per video, suggesting that students used them for more than just preparation for class (Smith, 2013). While students generally liked the approach, there are some indications that it takes them some time to adjust to the new format. Seery tracked screencast usage and time and day of access and suggests that as weeks progressed, a more stable pattern emerged of when students were interacting with the videos, building it into a regular study pattern to prepare for each week's class (Seery, 2015). Fautch captured some student sentiment during the implementation of lecture flipping which demonstrated that while students were amenable to the approach, it took time to adjust (Fautch, 2015).
Awareness of issues facing students, such as time to adjust, is important for educators thinking of implementing the approach. Where negative comments about the approach were captured, they tended to relate to the difficulty of organising time outside the lecture to watch videos, or a preference for the lecture as a means of receiving information rather than watching screencasts (Table 3). However, it does appear that these views were in the minority. Yeung and O'Malley report that they were expecting resistance from final year honours chemistry students about the approach, given that they were used to more traditional methods, but none were forthcoming (Yeung and O'Malley, 2014). Flynn, also expecting some pushback, reports that only "very positive" sentiments were received to the open-response questions (Flynn, 2015).

Evaluation of learning through lecture flipping
Most studies presented some overview of whether learning had improved as a result of implementing the lecture flipped approach. While this was often directly measured by comparing examination scores between years or between groups, other data trends or observations were used to examine any improvement in learning. The majority of studies examined considered exam scores in course work or American Chemical Society (ACS) exams to examine the impact of lecture flipping. The results were divided evenly -half of the studies showed no improvement in exam scores.
Fautch compared student scores between different groups of students taking organic chemistry I and observed a changing grade distribution with those students who took the course by the flipped method. She noted that the proportion of students with grade 3.5 (A-/B+) dropped, while those gaining both a 4.0 (A) and a 3.0 (B) increased. The number of withdrawals from

Feedback Theme Illustrative Quote
Early surveys demonstrated initial difficulties in adjusting ''I think it may turn out well in the end, but so far it has been tough getting used to.'' (Fautch, 2015) Difficult to organise own time outside lecture Please do not adjust margins Please do not adjust margins the course also dropped (Fautch, 2015). A proposed explanation is that flipping encourages students who may have opted to withdraw to stay for the full course. Hence there is some increase in the lower grades, as these students tend to find the course difficult. The increase in the higher grades (A) is attributed to the students who typically do well continuing to do well (or better) in the flipped lecture approach. Flynn also noted a reduction in withdrawal rates across two large enrolment modules (Organic I, Organic II), reporting that they were lower (2%, 3%) when compared to the average of previous years (6%, 7%). Module failure rates were also lower (6%, 7%) when compared to previous year (20%, 17%) (Flynn, 2015). In addition, students in this group had a small but significant improvement in exam scores compared to previous students taught in an active learning classroom.
A shifting of grade distribution was also found in the implementation of the flipped lecture model with a large first year general chemistry group. Two classes, one with 320 students and one with 415 students were taught in parallel, the latter being taught by the flipped approach. Examining the grade distribution, an increase in A and B grades were observed (of 3.5% and 3.9% respectively) with a matching decrease in C grades (7%). D and F grades remained unchanged (Yestrebsky, 2015). Similarly, a grade-letter increase performance was observed in a partial-flipped classroom with students in the middle grades (C to F), with the observation that students tended to stay with the course in this implementation, whereas they would have withdrawn in previous traditional formats (Trogden, 2015). Students in this study were found to have earned a grade-letter increase on what they would have in a traditional course, with A and B students performing similar in both cases. Furthermore in a study considering academic achievement, students in the upper third and lower third of their previous school class were 4.3% and 2.6% more successful than those in a similar lecture based class, whereas students in the middle third were 3.6% more successful in the lecture based class (Butzler, 2015). This latter finding is not fully explored in that article but warrants further analysis as it suggests that flipped learning is not universally improving student grades.
ACS exams were used to measure test scores in a large enrolment general chemistry classes which were taught by traditional (N=340) and flipped approach (N=339). A moderate sized significant difference was recorded in the first year of implementation, but in the second, no significant difference was found between student scores in each group (Baepler et al, 2014).

Discussion
In the last number of decades, there have been repeated calls for innovation and reform of the university chemistry curriculum, especially that at introductory levels. The curriculum has been criticised for leading "to knowledge without understanding" and producing "a system of knowledge that students cannot apply to the world in which they live" (Bodner, 1992). Reflecting on four decades of education research, Johnstone concluded that "many of the problems we identified in the 1970s are still there… This should be telling us something about the direction we are taking and the need for change" (Johnstone 2010). He echoed a call by Hawkes (2005) for professional societies to become more proactive in driving curricular reform. More recently, in a critique of the approach taken in introductory chemistry courses, Talanquer and Pollard wrote that: the first-year chemistry curriculum at most universities is still mostly fact based and encyclopedic, built upon a collection of isolated topics, oriented too much towards the perceived needs of chemistry majors, focused too much on abstract concepts and algorithmic problem solving, and detached from the practices, ways of thinking, and applications of both chemistry research and chemistry education research in the 21st century (Talanquer and Pollard, 2010).
It is likely to be in this context of dissatisfaction with the status quo that has persuaded many education practitioners to adopt a flipped learning approach. There is an (often implicit) sense in the introductory paragraphs of the various articles that something other than what is currently practised is worth trying. Much of this is underpinned by the dominance and generally accepted framework of constructivism, which to educators in classrooms translates as applying active learning approaches (Goedhart, 2015). Lecture flipping could also be viewed through the lens of cognitive load theory. Providing information in advance of lectures may offer students a chance to process it, and thus utilise it in the active environment the flipped classroom enables. While constructivism (Flynn, 2015) and cognitive load (Seery, 2015) are hinted at in some implementations, there is a need for the community to further develop our theoretical basis for integrating flipped learning into our practice. Many educators of course simply aim to have a more active classroom, and use the flipped learning approach to enable that. Indeed it has been suggested that the improvements in a flipped learning classroom may just be the result of implementing an active learning classroom; noted in a study that compared a flipped classroom with a non-flipped active classroom, both based on an active-learning, constructivist approach. No difference was found in attitudes or grades between the two courses (Jensen et al, 2015). However, among the articles surveyed here, there were improvements noted in flipped classroom when it was compared with a non-flipped, but active learning, classroom (Flynn, 2015).
Much of the commentary online and in articles alludes to the fact that there is no single way to implement flipped learning, but the studies in chemistry have shown a remarkable consistency. In all cases, students watched a pre-lecture screencast or video, which was in some instances supplemented by handouts or additional notes or reading. Video lengths varied but most instances generated an average Please do not adjust margins Please do not adjust margins between 10 -20 minutes; probably with an implicit sense that this is the time range after which students' attention in lectures has fallen off (Fitzgerald and Li, 2014). Only two authors elaborate on what they required students to do while watching these pre-lecture screencasts; namely annotate and work through provided notes. In some cases it isn't clear whether students make their own notes or are provided with full or partial notes as part of the pre-lecture package. Flynn also notes caution in making the organisation of the material consistent to students, so that availability and deadlines are clear for the duration of the module (Flynn, 2015). This is likely to be especially important for initial implementations of flipped learning; several comments from students in different studies remark that while they were amenable to the approach, there was an adjustment period.
While presentation of materials in advance of lectures was uniform in approach, there was divergence on whether the work completed by students in their advance preparation of lectures merited some proportion of the module grade. Recently Cooper made the argument that students consider tests and quizzes that are graded most important, and thus if we attribute value to some component of our curricular reform, it must have a grade attributed to it (Cooper, 2015). As well as incentivising students to complete the pre-lecture work by attributing it assessment value, it also offers students feedback on their own understanding of the material prior to lecture. This was formalised in the approaches used in some cases: Fautch (2015) and Seery (2015) both asked students what topics they found difficult and these were addressed in lectures, while Flynn (2015) used questions where the performance was poor as a basis for in-class discussion questions. Allowing students to develop an awareness of the difficulties they are facing and how they can address these is a means of facilitating the development of their metacognitive strategies, whereby students can monitor their own development and understanding (Goedhart, 2015).
One approach to this is to use worked examples, which have an established basis in cognitive load theory for allowing students develop their understanding on topics of difficulty; especially novice learners in a discipline (Kalyuga et al, 2001).
Using worked examples has been documented for chemistry (Crippen and Brooks, 2007), and the approach is particularly suitable for an online environment (Crippen and Earl, 2007;Biesinger and Crippen, 2010). Worked examples were provided by  as part of the suite of resources to help students work through material in advance of their flipped classroom session. These worked consisted videos showing the workings to achieve answers. In the context of cognitive load theory, worked examples are more strictly defined as an approach whereby students complete an ever-increasing proportion of a problem based on their developing knowledge on how to solve that problem type (Behmke and Atwood, 2013).
Enabling student independence was a theme touched on by many authors. Linking to textbook examples and questions to try, as well as using textbook graphics in the screencast aimed to emphasise the role of the textbook for students in their study as a useful resource to further explore a topic (Seery, 2015). There was some indication that students were taking ownership of their own learning in a flipped class approach (Fautch, 2015) When discussing what happens during class time, some authors described their approach to bridge the pre-lecture work with the lecture, by giving mini-lectures on topics of difficulty identified by a pre-lecture quiz or by what students had reported (e.g. Fautch, 2015). None of the shortlisted articles utilised Peer Instruction formally. Flynn elaborates fully on what happened during her class time, with students using clickers giving responses before and after explanations, which allowed the pace of the class and follow-up questions to be determined (Flynn, 2015). Other approaches involved peer learning, where students' group work on problem solving was formally assessed, and included a peer-grade (Christiansen, 2014). Another approach was to use some in-class time to allow students to give presentations (Rein and Brookes, 2015).
In other studies however, there is a vagueness about what happens during class time, and a more robust framework needs to be developed so that there is a basis for what happens in class time and how it builds on pre-lecture work.
Recently, further description on the possible use of peer instruction for higher education chemistry has been outlined by Schell and Mazur (2015). It is proposed that it works well with flipped lectures as it enables students to prepare some prior knowledge prior to class. Peer instruction in class typically involves a mini-lecture on a particular concept, followed by a conceptual question. An example of a conceptual question is provided by Schell and Mazur: "Spontaneous reactions occur:

C) Both (A) and (B)"
This question is designed so that it will elicit discussion with students. Having been presented with the question, students respond using personal response systems ("clickers") or similar devices. If the average correct response is below 30%, the concept is revisited with a mini-lecture. If it is above 70%, there is a brief explanation on the correct answer before proceeding. If the correct response rate falls between 30-70%, the students are allowed some time to discuss with their peers, before being asked again to submit their answers. The idea is that the discussions allow students to develop their understanding of the topic (Mazur, 1997).
Given that the flipped learning approach increase the formal out-of-lecture workload for students, it is perhaps surprising that students across all studies overwhelmingly supported and enjoyed this approach. While there was some caution regarding an adjustment time needed to get used to it, it appears only a small proportion of feedback and sentiment was negative. These comments usually alluded to the fact that the role of a lecture was to receive information, or that a student didn't have time beyond the lecture hour to cover the material required.
Why is the approach so popular with students? One possibility is that it offers a scaffold and organisational structure for students to engage with materials. Making the schedule consistent and the learning goals clear and up-front mean that students are aware of expectations and responsibilities (Flynn, 2015), and perhaps derive a sense of satisfaction from completing work regularly. Evidence from other studies suggest that this may be the case. Students in a statistics class felt that they were learning more than students who had been in a traditional class (Touchton, 2015). There is much to learn from motivation theory that could be applied here. (Abeysekera and Dawson, 2015) In many studies, there was an understandable tendency to see if the flipped learning approach was "better" than the traditional approach by comparing grade averages in different groups. While this kind of data can be useful for promotion and advocacy of a particular technique, many results here show that caution is needed in conclusions that are made from such comparisons.
These studies indicate that highperforming students will continue to do well in the flipped approach. Several authors commented that students who would previously withdraw, or score poorly, tended to complete the module. While these marks were lower (and hence reduce the average score), the approach benefited these students as they successfully completed the module. Thus as with any average, the underlying detail provides a much richer analysis. Given these preliminary observations, cluster analysis is likely to be a more suitable approach. A useful template is that recently completed measuring students planning and monitoring behaviours to identify at-risk students (Chan and Bauer, 2014). There is much work to be done in this area.

Conclusions
What can we learn from the work published so far on flipped learning in chemistry higher education? There are several positive outcomes emerging: students tend to like, enjoy, and engage with the format; there are similar performance outcomes, if not better, to that found with traditional approaches; and there is some evidence that students who may not have traditionally stuck with a course do so with the flipped format. The approach has led to some variety in how in-class work is managed, and afforded some opportunities for bridging work before, during, and after class to provide a framework for student engagement with the module.
Along with these positive themes emerging from studies about flipped learning in chemistry, there are some aspects that warrant consideration from education researchers and practitioners. While the approach is considered a philosophy rather than one particular method, there is a need to establish a more robust framework for how this teaching approach is implemented. The predominant learning theory in chemistry education is constructivism, which aims to base students approach to learning by integrating new ideas and information so that it makes sense with what they already know (Bodner, 1986;Taber, 2011). Teaching under the umbrella of constructivism would therefore mean that teachers don't just tell students what they need to know, but provide structured activities so that students can develop their knowledge within the parameters of their own prior understanding. There is a sense that educators discussed above reporting their implementation of flipped teaching are aiming to use the inclass time to create a structured environment where they can interact with new information with guidance from their lecturer. Nevertheless, the over-reliance on the pre-lecture screencast in one form or another in all twelve reports means that the concept of information transfer underpins the implementation of flipped learning.
Is this a conflict? Flipped learning allows a re-balancing of time between "time spent telling students what [the lecturer] thinks" and "[time] spent asking them what they think" (Herron, quoted in Bodner, 1986). Thus it could be argued that flipped learning aligns with a constructivist approach as it is an approach that facilitates active learning situations where students can work to create new knowledge (to them). Indeed it has been argued that a blend of autonomous learning through computer assisted learning, socially mediated learning through group work, and direct instruction is advantageous (Schraw et al, 2005).
Nevertheless, there are likely to be opportunities to extend the findings from chemistry education research further, so that the pre-lecture experience is not simply a passive information retrieval. Screencasts themselves could be more interactive, or be user-dependent, so that students could explore their understanding (e.g. see Yang et al, 2004). Students with identified misconceptions based on their response could be diverted through some activities that would assist their comprehension. Another approach would be to incorporate worked example, as defined by cognitive load theory (Crippen and Brooks, 2009). These would provide a useful gradient between pre-lecture screencast and in-lecture work, and incorporate the need for incentivising pre-lecture work by awarding some grade value. The purpose from an educational perspective however would shift from identifying (and rewarding/penalising) what a student does or does not know prior to a lecture towards one where the purpose of assessment is assisting in learning. Furthermore, the peercomponent of flipped learning could be expanded so that it began prior to the class time and if necessary continued after Please do not adjust margins Please do not adjust margins it. Discussion for a have been described for chemistry to enable peer interactions Smith et al, 2014).
As well as innovation in the practice of implementing flipped learning approaches, more thought is needed in evaluating their educational impact. While there will undoubtedly be a number of reports in the future on the impact of flipped learning, it is wise to caution what these will say. It's already clear that comparing average performances between control and experimental groups misses nuances that are already emerging from the studies shown, and examining what happens to students individually, through qualitative work or cluster analysis, will likely offer more valuable information.
The flipped learning approach is likely to be a significant teaching and learning method over the next decade as more educators seek to improve the value and quality of their inclass time by creating a space for active learning. Progress on this will enhance the likelihood that the approach, which is already in favour with students, will be viewed as a rigorous one that can finally challenge the hegemony of the didactic lecture in higher education chemistry.