A novel practical pedagogy for terminal assessment

Abstract

The present paper reports upon the design and implementation of a novel practical work pedagogy that is shown to increase students’ retention of practical procedures. The chemistry exams, for 15 to 16-year olds in England, will be assessed entirely through terminal examination questions from 2018. Longer term retention of learning is critical if these students are to minimise any discrepancy in attainment that may arise from following curricula with a coursework component. The novel design, underpinned by Cognitive Load Theory and Social Constructivism, involves pre-laboratory preparation and employs oracy to promote thinking during practical work. Three equivalent chemistry groups within the same school undertook neutralisation and crystallisation practical tasks and their practical work exam question attainment data is analysed. The novel pedagogy is trialled as a neutralisation task with one group and affective outcomes are determined through Likert scale activity feedback questionnaire. Attainment data shows a statistically significant higher mark in practical exam question attainment for the intervention compared to the control group when tested ten weeks after the teaching episode.

---

Introduction

Practical work assessment in school chemistry

That “Chemistry is necessarily an experimental science; its conclusions are drawn from data, and its principles supported by evidence derived from facts” (Faraday, 1827, p. B) remains uncontested, but the role, delivery and assessment of practical work are subject to debate. In the UK the requirement for secondary school students to engage in practical work as an integral part of science education, was codified in 1904 (Board of Education, 1904). Practical work is both time consuming and expensive, but its inclusion in assessment criteria justifies its costs, and ensures its retention.

In 1988 the General Certificate of Secondary Education (GCSE) was introduced bringing with it teacher assessed coursework. Coursework aimed to provide a mechanism by which practical skills, in contrast to practical work itself, of a large number of students could be assessed. The National Curriculum for Science allocated a minimum of 20 percent of the total marks to experimental and observational work in the laboratory or its equivalent (Jenkins, 1998). In 2011 coursework was replaced by controlled assessment; coursework could be redrafted resulting in attainment that did not reflect scientific skill, whereas aspects of the controlled assessment were undertaken in exam conditions. Practical skills assessment comprised of two written papers undertaken in exam conditions, enabling large numbers of students to be assessed by their teachers. Controlled assessment accounted for 25 percent of the final GCSE science mark but may have involved as few as two practical episodes. The most recent round of curriculum reform in neighbouring Ireland, Northern Ireland, Scotland and Wales, have all resulted in either in a coursework or controlled assessment component in their 14–16 chemistry exams (Perry, 2013) however, from September 2016 in England, practical skills will be assessed solely based on examination. The exams will contain questions that specifically draw on the experience students have gained from doing practical work (Office of Qualifications and Examinations Regulation (Ofqual), 2015), the exam boards are required to detail the apparatus that students should use and the techniques they should develop, through a minimum of eight practical activities upon which the examination questions are based. Schools are only required to confirm, rather than provide evidence to exam boards, that their students have undertaken the tasks and kept a record of their work. At least fifteen percent of the total marks available in GCSE Chemistry will be dedicated to these exam questions; “this proportion is large enough to have a significant effect on a student's grade, but not so large as to distort assessment or hinder coverage of other requirements” (Ofqual, 2015, p. 2). A further twenty percent of the total marks available in GCSE Chemistry have been allocated to assessing mathematical skills.

Practical work in GCSE lessons

Secondary School practical lessons often form a part of a teaching sequence, building upon students’ prior knowledge and skills. This sequence is drawn to the students’ attention through learning objectives and teacher instruction, “setting the scene” (Wellington, 2000, p. 150). To quote Millar (2009, p. 5) “It would be unreasonable to expect durable long-term learning of a scientific idea or concept to result from a single, relatively brief, practical activity. Learning, where it occurs, is likely to result from a sequence of lessons which involve activities of various kinds, including some carefully planned practical activities at appropriate points.”

The choice of the practical activity to be undertaken and where it occurs in a teaching sequence may be dictated by a departmentally produced scheme of work (Abrahams, 2005); exam boards in England such as the Assessment and Qualification Alliance (AQA) provide an exam specification and suggested schemes of work which include practical activities (http://www.aqa.org.uk/resources/science/gcse/chemistry/teach/schemes-of-work). The Department for Education has specified eight required practical activities to be conducted as part of the curriculum and assessment of GCSE chemistry (AQA, 2017, p. 2) for which exam boards have developed their own suggested activities that ensure the apparatus and technique requirements are met. Again, providing an example from AQA, their suggested activities include the following reminders “written papers will include questions requiring knowledge gained from carrying out the specified practicals” (AQA, 2018 front cover) and, “not having done some of the practicals, despite the school's best efforts, will not stop a student from entering for the GCSE. However, it may affect their grade, because there may be questions in the exams that they won’t be able to answer” (AQA, 2016b, p. 4).

Abrahams and Millar (2008) reported that teachers viewed their responsibility in terms of the delivery of an activity judged to be appropriate by others, a ‘foolproof’ experiment from which the right answer will emerge as long as the instructions are followed (Kirschner, 1992, p. 278). The procedure is introduced to the students by the teacher using at least one of the following devices; oral instructions that may be supported by written instructions or diagrams on a board or screen; the teacher may demonstrate aspects of the equipment or the procedures or even the entire task before the students begin and there may be a labsheet that dictates the method which may also include a diagram of the apparatus (Millar, 2009). Tiberghien et al., (2001) reported that most teachers use labsheets to direct the students’ activities during the practical activity, the teacher may adopt a “supervisor” role (Wellington, 2000, p. 150), a “good” teacher will then move around the laboratory intervening as required, providing equal attention to groups asking for assistance and those that don’t (Hamidu, 2014, p. 84). Classroom studies indicate that teachers and students primarily focus on following the “recipe” (Clackson and Wright, 1992, p. 41) and handling an apparatus (Abrahams and Millar, 2008; Hodson, 1993; Lunetta et al., 2007; Tiberghien et al., 2001). Abrahams (2005) observed teachers limiting the time allocated to presentation and explanation of practical work or materials or spending little or no time considering the concepts behind the activity, giving precedence instead to the time available for students to manipulate objects and materials.

What should be improved?

Hodson (1990) described laboratory work as unproductive and confusing as it is often used without clear thought-out purpose. The emphasis on closed “recipe” following practical tasks as previously described enable hands-on activity without affording the opportunity for engaging minds. In England the GCSE exam specification explains three reasons for carrying out practical work; firstly, to support and consolidate scientific concepts, secondly to develop investigative skills such as controlling variables and finally to develop practical skills that include taking measurements using specialist equipment (AQA, 2016a, p. 91). It has been argued that the purpose should be more than skill development but also include, in addition to opportunities to develop conceptual understanding, affective aims such as interest and enjoyment (Johnstone and Al-Shuailib, 2001). Abrahams’ work (2009) in the affective value of practical work in secondary school science reported that much of the perceived motivation was in fact situational interest that passes as the lesson ends and thus, unlike motivation, it is not necessarily as effective in helping to develop student learning.

Providing students with a set of procedures to follow in a manner akin to a “cookbook” (Tobin et al., 1994, p. 51) is common practice among teachers, (Ibrahim et al., 2014), the persistence in use of recipe style activities has been attributed by some teachers as being due the need to provide structure so students can complete the practical task with in the limited time, but Abrahams and Millar (2008) report, in the same study, issues related both to the reading and understanding of such written instructions. One problem with the traditional pedagogy described here is that working memory space becomes overload during laboratory episodes resulting in a lack of space available for cognitive processing (Johnstone and Wham, 1982) the consequence of which is that students adopt the default position of using their written instructions as a “mind-in-neutral” recipe (Johnstone, 2006, p. 58) rather than thinking about what they are doing and why.

Psychologists use the theoretical constructs of long-term memory and working memory to describe the functioning of cognitive architecture (Aben, Stapert and Blokland, 2012). Long-memory contains the cognitive structures that make up an individual's knowledge base, schema (Sweller, 1988). The human mind uses schema to organise, retrieve and encode chunks of information, once formed schema tend to be stable over time (Gobet, Lane, and Lloyd-Kelly, 2015). There has been no measure of a limit to the number of schema the long-term memory can hold conversely, the capacity to hold information in the working memory is very limited (Sweller, van Merrienboer and Paas, 1998, Cowan, 2010).

Johnstone (2006) explains that the working memory has two functions, firstly as a temporary store for incoming information and secondly, to process this information. Processing results in an action being taken, a response, and/or the information being transferred into the long-term memory. Processing requires that incoming information interacts with schema retrieved from the long-term memory, if the new information can be fitted to an existing schema it will be transferred into the long-term memory. Fitting new information into an existing schema is an “active, constructive process” (Sweller et al., 1998, p. 255) which produces the complex schema. Complex schema built up over time not only hold chunks of related knowledge but can be treated as a single entity by the working memory thus freeing up space for additional information to be processed. The limited capacity of the working memory inhibits complex reasoning involving many new pieces of information thus, it is the existence of prior knowledge held in complex schema that distinguish between a novice and a master (Sweller et al., 1998).

The load placed on working memory, that is the number of new or unfamiliar entities, must be minimised to enable transfer of information into the long-term memory. Instructions and learning opportunities must consider cognitive load when being designed as the information contained in instructional material must first be processed by working memory (de Jong, 2010) before a response can be elicited. Working memory has two processing systems that initially process visual and verbal information independently. Thus, information presented in more than one modality is divided across two of these systems, effectively reducing the cognitive load applied (Kirschner, 2002).

Johnstone (2009), fearful that students could be successful in their laboratory class with very little understanding of what they were doing, suggested a need for both pre-laboratory preparation and assessment for laboratory outcomes as a means of rewarding their efforts. A recent review of work on pre-laboratory preparation (Agustian and Seery, 2017) puts forth a convincing case for the inclusion of pre-laboratory activities that seeks in part to address the issue of cognitive load and offers a framework for the incorporation of pre-laboratory preparation in higher education. The success of pre-laboratory video to prepare students for laboratory work is demonstrated by Schmidt-McCormack et al. (2017).

Cooperative learning (also referred to a collaborative learning) has been reported to increase student achievement (Warfa, 2016). Raviv, Cohen and Aflalo, (2017) conducted a small-scale study of collaborative learning in a school laboratory and reported an improvement in skill acquisition and conceptual understanding in addition to positive affective outcomes among the 12 year-old female participants. Transforming group work into a collaborative task requires materials to be modified and activities to be structured so that every member of the group is responsible for contributing to the group work and the groups success (Leikin and Zaslavsky, 1999) Collaborative learning has its foundations in Social Constructivism (Vygotsky, 1934), a learning theory that emphasises the role of language and culture in cognitive development. Knowledge is thought not to be transmitted from one individual to another but co-constructed through social interaction: First, a student learns to adopt the language and symbols used by a more capable other to describe their processes of logical reasoning or conceptualisation, then the student internalises the ideas and perceptions expressed by the more capable other and, lastly these external actions are transformed to become internal to the student. In this way a more capable student provides “scaffolding” (Wood et al., 1976, p. 90) until all members of the group are able to complete the task themselves. The more capable other may arise through the structure of the activity, Smith, Hinckley and Volk (1991) for example, divided up the tasks in their activity so that each student in a group was assigned a particular part of a lesson or unit and then, in turn, each student acted as the more capable other, helping the other members of the group learn that section of the material.

Pre-laboratory preparation and assessment for laboratory outcomes. The work of Seery et al. (2017) detailed a method that employed exemplar videos, peer- assessment and digital badges to prepare, assess and reward laboratory skill development (referred to here as the Seery Protocol). The Seery Protocol employs a peer observation checklist to guide students as they film each other performing the techniques. Students may opt to reshoot the video as a result of the feedback. Thus, the Seery Protocol incorporates both a mastery learning model; assessment- feedback- repeat (Bloom 1968); Johnstone's model (2009), pre-laboratory preparation and assessment for laboratory outcomes. Furthermore, incorporating digital badges to reward skill development has been shown to motivate learners (Abramovich et al., 2013). A modified version of the Seery Protocol has been previously reported (Hennah and Seery 2017) in which, secondary school students aged 16 to 18 years were required to both watch exemplar videos and answer pre-laboratory questions. The questions had been designed to highlight pertinent techniques or to encourage students to think about why a particular action was required.

Aim of the study

The primary aim of this study was to design and implement a novel pedagogy for the preparation and delivery of a practical episode involving students aged 14–15 years old and to determine its success through student attainment in exam style questions.

Research questions

1. Does the novel pedagogy affect the procedure exam question attainment of students aged 14 to 15 years compared to the attainment of their traditionally taught peers?

2. Does the novel pedagogy affect student retention of practical procedures?

Designing an alternative pedagogy

The rationale for change is to transform practical work pedagogy from “hand-on” into “hands-on and minds-on” tasks. The literature presented in this paper reports the positive impact of pre-laboratory preparation on undergraduate teaching laboratory work, these instructional principles will be applied within a Secondary School setting. Success in GCSE Chemistry is determined in totality upon terminal assessment, thus the need to develop practical episodes that enable procedure retention over time is paramount to success.

To address the concerns about lack of clear purpose (Lewthwaite, 2014), the practical activity analysis inventory (PAAI) was employed (Millar, 2009). The PAAI aims to ensure clarity of both the task objectives, and to identify what the students were intended to gain from them. Here the primary objective for students, carrying their first titration, is to learn how use the equipment correctly. The second objective is to learn how the values obtained become the titres from which the concentration of the analyte may be determined. Thus, the emphasis is placed upon gathering data and how to use that data, linking “hands-on” procedure directly to the “minds-on” concept of concentration.

Student Task 1 pre-laboratory preparation. The exemplar titration video (Kucharski and Seery, 2016) was selected for use as it provided a clear step-wise introduction to titration and best-practice techniques, although designed for use with undergraduates its implementation of the Seery Protocol with post-16 teaching groups has also been reported by the author as increasing student confidence (Hennah and Seery, 2017). The Seery Protocol has been modified so that in addition to watching the exemplar titration video students will also be required to complete a pre-laboratory question sheet. The questions include images snipped from the exemplar video to direct the students to think about both the procedure and the underlying concepts. Using visual prompts, taken directly from the video, was used as a bridge between the abstract concept in the question and the procedure as a support for students trying to link concepts and practical tasks. Student Task 1 is available in Appendix 1 (ESI†). The students must submit this work before carrying out the practical task to encourage task completion and to provide the opportunity for both written and verbal feedback to clarify understanding.

Prompt slide. The need to simplify laboratory worksheets has long been argued (McDowell and Wadding, 1985; Sweller, 1994) and here a practical prompt slide replaces the written method provided by a labsheet (Fig. 1) for greater simplification. The slide acts as a “talking point”,‡ a strategy for stimulating students’ speaking, listening, thinking (Dawes et al., 2010); a scaffold to support students’ social construction of the procedure as a device to support learning (Vygotsky, 1978; Bransford et al., 1999; Mercer, 2013). Replacing written instructions, which have previously been identified as problematic (Abrahams and Millar, 2008), with the prompt slide and verbal discussion provides more than one presentation modality and may help reduce the cognitive demand of the instructions (Kirschner, 2002) freeing working memory to process information and enhance learning (de Jong, 2010).

Fig. 1 Prompt slide acts as a talking point to support reconstruction of the method through group talk.

Student Task 2 collecting data. A second written task is to be completed during the laboratory session, which has been designed to encourage students to recall their pre-laboratory task and to provide scaffolding for recording data and completing the novel titration calculation. In addition, students will receive a peer-assessment observation sheet consistent with the Seery Protocol and lastly a Likert-scale student feedback form to collect affective data (Hennah and Seery, 2017). These materials are available in Appendix 3 (ESI†).

Methodology

Ethical approval was first obtained from the school management to carry out this research. Students were informed about the purpose of the research and given information about what data was being gathered, and that they had the right to withdraw.

The research questions were addressed through quantitative methods based on student attainment scores from tests given under exam conditions. To ensure the test marking was equivalent between groups with different teachers, a marking matrix was used and samples of all the marking were moderated and agreed. Marking was checked prior to moderation and Cohen's kappa (Cohen, 1960) was run to determine the agreement between the judgements k = 0.769 (95% CI, 0.564 to 0.975), p < 0.0005. Although this agreement is good (Landis and Koch, 1977) it is the moderated marks that have been used.

Research question 1 attainment data collection

At the end of each school term students’ progress is assessed under exam conditions using a short examination paper (approximately 40 minutes) comprised of past exam questions or exam style questions based on the content covered during the term. Each end of term assessment includes a practical work question and where applicable a linked calculation question. Student attainment only in the practical work questions will be collected. Assessment deadlines are set within the school calendar. The teaching content is specified within this calendar thus any modifications to pedagogy cannot result in a requirement for more teaching time. The experimental methods for neutralisation and crystallisation are provided in Appendix 2 (ESI†).

There are three GCSE chemistry exam classes included in this study, control groups C1 and C2 and the test group T, each group has a different teacher but the teaching time, materials and laboratory resources available for the neutralisation (Oxford University Press, 2016) and crystallisation (AQA, 2016c) required practical activities considered in this work, are the same for each group. The presentation of crystallisation practical utilises only traditional pedagogy for all three groups. The control groups C1 and C2 utilise traditional pedagogy whereas, test group T follow the novel neutralisation pedagogy.

Group T attainment data from neutralisation practical exam question will be compared to attainment data from crystallisation practical exam question responses; that is the same group, same teacher but different teaching approaches. The attainment data from the three classes neutralisation practical exam question and crystallisation practical exam question will be compared to make visible variation due to task demand between the crystallisation and neutralisation required practical activities and for random errors arising from teacher and student variations. Titration calculation question attainment data has also been recorded for the three groups.

The baseline comparison. Students are given forecast grades for GCSEs based upon their Key Stage 2 (KS2) results from primary school (age 10 to 11 years). Using each student's KS2 English Reading and Mathematics test attainment scores as a starting point, national transition matrices are used to generate a target grade for the end of Key Stage 4 (KS4) when the GCSE exam is taken (UK Government, 2015). Schools calculate a minimum achievement grade (MAG) for each student for each subject. This is the baseline from which stakeholders are able to judge if a student or a class has been successful.

Three chemistry classes of 31 male students aged between 14 and 15 years old were used in this study. Two of the classes were taught entirely with traditional pedagogy, control 1 (C1) and control 2 (C2), whereas the third, Test (T) were taught the neutralisation procedure using the novel pedagogy. Analysis of student attainment data occurred only if the student had taken part in both practical events and that their exam papers were available for moderation and analysis.

A one-way Anova was conducted to compare the Minimum Attainment Targets (MAG) of the three groups C1 (n = 23), C2 (n = 25) T (n = 26). Normality checks and Levene's test of homogeneity (1960) were carried out and the assumptions were met. There was no statistically significant variance in the mean MAG between the groups [F = 2.314, p = 0.106] Levene's test supports the null hypothesis that group variances are equal.

Research question 2 attainment data collection. The practical exam question data discussed previous is collected at the end of each school term, approximately two weeks following completion of the practical task. The practical exam question attainment data for group T crystallisation (traditional pedagogy) and group T neutralisation were compared using standard z-scores (Lavrakas, 2008). T group students were also given an unannounced test using the same neutralisation and crystallisation procedure questions ten weeks later. Standard z-scores were calculated a for the data collected to compare attainment in procedure exam questions following traditional pedagogy (crystallisation) and the novel pedagogy (neutralisation) and to compare the attainment in these questions following an interval of ten weeks.

Results

Research question 1: does the novel pedagogy affect procedure exam question attainment of students aged 14 to 15 compared to the attainment of their traditionally taught peers?

Group T were provided with the same time and equipment to complete their practical work as the control groups. The same required practical was carried out by all three groups however, group T the students were not provided with a practical task sheet but were required to collaboratively construct their method with a talking point prompt to scaffold their talk. The greatest difference in pedagogy occurred in the level of pre-laboratory preparation as the students were required to watch an exemplar video and complete questions and receive feedback on their answers prior to undertaking the practical work. Attainment scores for each of the groups C1, C2 and T have been plotted as box plots (Fig. 2–4). Attainment scores for a titration calculation question has been included in addition to the crystallisation and neutralisation procedure questions scores as titration calculations are of fundamental importance to determining the success of a titration teaching episode. The two control groups C1 and C2 were taught using the traditional pedagogy only.

Fig. 2 Box plot of group attainment in crystallisation method question. All groups taught using traditional pedagogy.

Fig. 3 Box plot of group attainment in neutralisation method question. Control groups C1 and C2 taught using traditional pedagogy whereas the novel pedagogy was used with group T.

Fig. 4 Box plot of group attainment in titration calculation question. Control groups C1 and C2 taught using traditional pedagogy whereas the novel pedagogy was used with group T.

ANCOVA analysis for covariance

A one-way ANCOVA was conducted to determine a statistically significant difference between groups C1, C2 and T attainment scores from the crystallisation method (traditional pedagogy), neutralisation method (T Group novel pedagogy) and the titration calculation questions. Levene's test and normality checks were carried out and the assumptions met for each of the three data sets.

There was a statistically significant difference F(2,70) = 4.13 p = 0.046 between the attainment scores, controlling for class average MAG, for the crystallisation question. The Group mean score (adjusted) were calculated to be C1 = 3.838, C2 = 3.972, T = 3.517, random errors arising from teacher difference may have contributed to this result as the traditional pedagogy had been implemented for the crystallisation practical work with all three groups.

A statistically significant difference between groups C1, C2 and T attainment in the neutralisation method question F(2,70) = 3.39 p = 0.039 and for titration calculation question attainment score, F(2,70) = 6.51 p = 0.003 controlling for class average MAG in both analyses. The significant differences in neutralisation and titration attainment result from control group difference, random errors arising from teacher difference may have contributed to this variance.

Research question 2: does the novel pedagogy affect student retention of practical procedures?

Data from the end of term exams, approximately two weeks following the teaching episodes, were collected and the mean scores calculated (Fig. 5), although the Group T mean score is higher following the novel (neutralisation) pedagogy than for the traditional (crystallisation) pedagogy, analysis of z-scores (F_1,26 = 1.08, p = 0.42) revealed no statistically significant difference. Group T students were given an unannounced test using the same neutralisation and crystallisation procedure questions ten weeks later, attainment scores for both neutralisation and crystallisation had fallen, but the means score for the traditional pedagogy are lower (Fig. 5). Analysis of z-scores for this data suggests that there is a statistically significant difference in attainment following the novel pedagogy after ten weeks compared to attainment following the traditional pedagogy (F_1,26 = 2.28, p = 0.02).

Fig. 5 Procedural exam style question attainment following traditional (crystallisation) and novel (neutralisation) pedagogies. Students were tested two weeks after the practical activity (short term) and then approximately ten weeks later (medium term). The box plots indicate that group T have retained the neutralisation method information better than the crystallisation information.

Furthermore analysis of the difference between attainment after 2 weeks compared to 10 weeks revealed a statistically significant difference for the traditional pedagogy, Z crystallisation scores – Z 10 week crystallisation score t(25) = 4.99 p = 0.00 but not for the novel pedagogy Z neutralisation scores – Z 10 week neutralisation score t(25) = 1.83 p = 0.08.

The traditional pedagogy, in this small-scale study, has resulted in students performing better in procedure exam questions after ten weeks than they performed in procedure exam questions following traditional pedagogy after the same time interval. There is also a statistically significant difference in their procedure exam style question attainment after ten weeks following the novel pedagogy than there was following the traditional pedagogy. This data indicates that the novel pedagogy is having a positive impact on the students’ performance in procedural exam questions after ten weeks.

Implementation of the novel pedagogy

As discussed previously, 15 percent of the GCSE Chemistry exam marks are allocated to questions requiring knowledge gained from carrying out the eight specified practical activities. Time constraints imposed by a broad curriculum and limited teaching time dictate that practical work must be effective in developing both the procedural knowledge and knowledge of the underlying concepts. Research literature has repeatedly found that practical work does not offer a significant advantage in developing students’ scientific conceptual understanding as measured using pen and paper tests (Abrahams, 2017, p. 404). The novel pedagogy included pre-laboratory tasks explicitly planned (Abrahams and Reiss, 2012) to encourage students to think about what they observe during the pre-laboratory video. In addition to exam question attainment students’ responses to pre and post-laboratory task questions were collected (Task 1 and Tasks 2) and affective data from a student questionnaire (activity feedback response).

Activity feedback responses. Attainment data from group T (n = 31) were collected, a reliability analysis was carried out on the perceived task values scale comprising 7 items. Cronbach's alpha showed the questionnaire to reach acceptable reliability, α = 0.97.

The novel pedagogy required students to watch the pre-laboratory video, complete and submit the Task 1 questions (Appendix 1, ESI†), 30 of the 31 group T students watched the video before the practical episode. This is attributed to both novelty and the necessity of completing and submitting the Task 1 pre-lab questions. Feedback responses expressed as Likert scale averages can be seen in Fig. 6. The students’ average response is above 3 (neither agree nor disagree) for all the items except to the question asking whether or not they had narrated their procedure. Narrating the procedure was included in the novel pedagogy to encourage students to justify their actions and in doing so connect their “hands-on” activity with their “minds-on” concepts. How students can be better encouraged to do this is a subject to addressed in future iterations when oracy techniques are more familiar to them. The average student score as 2 (agree) when asked if the novel pedagogy helped them develop their understanding of the technique and its purpose and when asked if they felt it helped to develop their lab skill.

Fig. 6 Group T student questionnaire responses following the novel neutralisation practical episode. The responses have been stated as a class average Likert scale score.

The final open-ended question was predominately left without comment. The three comments left referred specifically the video; the video did not make it clear why titration was carried out in this way; it was hard to understand; the video was confusing because the maths wasn’t clear and used different symbols. The exemplar video was selected for its clarity of best practice techniques but had been designed to be used by chemistry undergraduates. It will be interesting to observe in future iterations whether an age and experience appropriate video would result in more positive affective results. Anecdotally, the same individual who reported not having watched the pre-laboratory video also reported three strongly disagree responses.

Task 1 pre-laboratory question

All students handed in the task sheet, the extent to which they responded correctly, incorrectly or provided no response has been summarised in Fig. 7. The questions were teacher- marked and whole class verbal feedback verbal feedback was given to the class before the practical episode. The school expectation and practice require students to capture verbal feedback by annotating their work in green pen.

Fig. 7 Group T student responses to pre-laboratory questions. The responses show students are less successful in answering questions that require an understanding of the underpinning concepts compared to questions that require knowledge about procedures.

The highest percentages of correct answers are to those questions that require recall from the exemplar video whereas the lowest scores were for questions 8 and 9. Question 8 asks why a pink colour appeared at the top of the flask as the titrant is added but disappeared when the conical flask is swirled, although students realised the solutions were being mixed the majority inferred that the indicator was being mixed in. Question 9 asks why washing the burette tip and conical flask can produce a colour change, most answers inferred that the water was reacting. These misconceptions indicate that the students struggle to link the procedure with the underlying chemical concepts. The results in Fig. 7 suggest that the pre-laboratory materials were preparing the students for the physical processes of the practical episode but did not support linking procedure and conceptual understanding. Question 10 asked students to calculate the titre, testing their understanding of the terminology. Only 60% responded correctly, most incorrect answers gave the end volume indicating that insufficient time had be allocated to rehearsal of titration specific language and as such the terminology acts to increase the cognitive load (Johnstone and Al-Shuailib, 2001). Exam questions require concentration to be calculated most often using the unit moles per cubic decimetre. As question 11 indicates, that the confusion between the units used on glassware and those used in calculations persists.

Prompt slide

The practical work focusses on manipulative skills rather than data generation to reduce the “noise” (Millar, 2004, p. 20) in the experiment. The students rehearse the procedure via exemplar video, then introduced to the school equipment and how it is set up before carrying out the procedure themselves. Working in groups, students selected and set up the equipment using talk and the prompt slide replicate the method. The students were able to conduct the neutralisation safely and effectively without being issued with a full written method following a detailed demonstration linked to the projected prompt image and the peer assessment checklist. Only one group, comprised of 2 students commented that they did not have a method sheet.

Collaborative learning episodes can be made more effective through training learners to employ language as a tool for collective reasoning, “Exploratory Talk”. This mode of talk occurs when everyone: listens actively, asks questions, shares relevant information, contributions builds on what has gone before, there is a sense of shared purpose, and the group seeks agreement for joint decisions (Mercer and Littleton, 2009). Although the prompt slide used in this study provided a “talking point” the students remained untrained in “Exploratory Talk”. Future iterations will place more emphasis on oracy and how laboratory talk can be structured more effectively (Kind et al., 2011).

Task 2 collecting data

Task 2 activity sheet is available in Appendix 3 (ESI†). The Task 2 data shows an increase in the number of questions that are left without a response. Additional work was found in class books rather than the Task 2 sheets indicating that results were being recorded and calculations completed. Appendix 4 (ESI†) summaries the students’ responses to the Task 2 questions. The Peer Observation Checklist had actions ticked off but no responses. Future iterations must consider limiting the activities beyond data collection that are required during a practical episode whilst encouraging links between the practical procedures and the underlying concepts.

Digital badges

Anecdotal reports suggest that students were not motivated by the reward of a digital badge. The reasons for this are likely to be two-fold. Firstly, school students have very little, if any, experience of digital badges beyond those issued in games. Fourteen and fifteen-year olds are not yet thinking about application processes and curriculum vitae development, so the potential of digital badges has yet to appeal to them. Secondly, it has been reported that obtaining badges for taking part fails to provide the motivation that occurs when a badge must be authentically earned (Abramovich et al., 2013). In this study the completion and submission of pre-laboratory Task 1, Data collection Task 2 and the Likert scale questionnaire were the parameters for receiving a badge, separating practical skill from assessment unlike the direct assessment achieved by submitting video evidence of practical skills. An image of the digital badge and criteria as issued to the students via email from the school VLE, has been reproduced in Appendix 5 (ESI†).

Limitations

The activities selected in this study had to fit within the confines of curriculum timetabling, and an exemplar video had to be available for the novel pedagogy. Neutralisation and crystallisation are both practical episodes specified by the exam board, they occur in sequential terms within the school calendar but are designed to develop different laboratory skills. In addition, the neutralisation procedure is a higher tier activity, meaning the content is more demanding and aims to distinguish between those students capable of achieving grades 5 up to 9. The sequential order of the episodes, dependent on school assessment timetable, resulted in Group T being retested on neutralisation (novel pedagogy) and crystallisation (traditional pedagogy) practical exam questions after different time intervals, the ten weeks were measured from the completion of the crystallisation task. This implies that improved retention of neutralisation (novel pedagogy), the first practical episode, was greater than reported here.

Each analysis conducted met the assumptions required to use inferential statistics. That control cohort C1 showed a statistically significant difference (Fig. 2 and 3) in attainment between the two procedure questions and that control cohort C2 showed a statistically significant difference in calculation question attainment (Fig. 3) as compared to cohorts C1 and T are suggestive of additional factors, affective, epistemic, or instructor interaction, influencing outcome. As a results comparison of attainment made within a cohort are perhaps more secure than those made between cohorts. To ensure marking bias was reduced, a rubric was employed in the first instance followed by moderation and agreement of marking samples. The marking sample revealed one group's work was not in agreement with the other two, although Cohen's Kappa suggested good agreement (Landis and Koch, 1977), where discrepancy was noted items were remarked and the agreed mark was submitted.

The use of student minimum target grades, based on primary school attainment, as a baseline is not secure however, as a population, variance between individuals with different primary school experiences should be accommodated for, ANOVA analysis of the groups’ Minimum Attainment Targets (MAG) and Levene's test of homogeneity indicate that group variances are equal and ANCOVA analysis using MAG as a covariant was employed. At the time of writing GCSE grade boundaries remain unavailable hence, no inference between attainment scores and GCSE grades have been made. In addition to which extrapolations between six-mark questions scores and GCSE grades would not be secure.

Learning experiences should be designed to reduce working memory load to promote schema acquisition (Sweller, 1994) Task 2 and the peer-observation sheet were text dense and required completion during the practical work future iterations will need to aim address this. In algebra using worked examples has been shown to promote learning more effectively, measured by student performance in subsequent tests, than problem solving (Carroll, 1994), similarly the use of questions here may not be an effective tool for promoting learning.

Conclusions

This work set out to design and test a novel pedagogy for practical work in GCSE chemistry because of concerns raised by the introduction of terminal assessment of which fifteen percent of the marks will be dedicated to exam questions that specifically draw on the experience students have gained from doing practical work. Although no direct measure of cognitive load was made, the instructional design included modifications reported to limit cognitive load (de Jong, 2010; Kirschner, 2002).

The novel pedagogy used pre-laboratory preparation outside of lesson time to extend the learning opportunity and build familiarity with the procedure prior to implementation. Students reported affective benefits of this novel approach and this was supported by a statistically significant increase in retention. If students feel that they understand something or feel that a method of instruction is successful, it can indicate that learning has occurred, such confidence is not consistent with situational motivation. The affective benefits reported by the students suggests that in this instance the novel pedagogy is beneficial.

There was no statistically significant difference between the test cohort's initial attainment grades following each mode of instruction. When the students were retested, without warning, the novel pedagogy resulted in higher attainment grades than those obtained following the traditional pedagogy. That this novel pedagogy has resulted in a statistically significant increase in the retention of knowledge, suggests that pre- laboratory episodes for GCSE candidate may be beneficial. The author recognises that further work is required to draw conclusions beyond this study, further testing and a larger sample size are both required to do so.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

The exemplar videos and badge images have been made freely available by Michael Seery (Seery, 2017). The author would like to thank the reviewers for their contributions to the improvement of the manuscript.

Notes and references

1. Aben B., Stapert S. and Blokland, A., (2012), About the distinction between working memory and short-term memory, Front. Psychol., 3, 301,  DOI:10.3389/fpsyg.2012.00301, retrieved March 2018.
2. Abrahams, I., (2005), Between Rhetoric and Reality: The Use and Effectiveness of Practical Work in Secondary School Science, PhD thesis, http://etheses.whiterose.ac.uk/14069/, retrieved May 2018.
3. Abrahams, I., (2009), Does practical work really motivate? A study of the affective value of practical work in secondary school science, Int. J. Mod. Sci. Technol., 31(17), 2335–2353.
4. Abrahams I., (2017), Minds-On Practical Work for Effective Science Learning, in Taber K. S. and Akpan B., (ed.), Science Education. New Directions in Mathematics and Science Education, Sense Publishers, Rotterdam.
5. Abrahams I. and Millar R., (2008), Does practical work really work? A study of the effectiveness of practical work as a teaching and learning method in school science, Int. J. Mod. Sci. Technol., 30(14), 1945–1969.
6. Abrahams I. and Reiss M. J., (2012), Practical work: Its effectiveness in primary and secondary schools in England J. Res. Sci. Teach., 49(8), 1035–1055.
7. Abramovich S., Schunn C. and Higashi R., (2013), Are badges useful in education? It depends upon the type of badge and expertise of learner, Educ. Technol. Res. Dev., 61(2), 217–232.
8. Agustian, H. Y. and Seery M. K., (2017), Reasserting the role of pre-laboratory activities in chemistry education: a proposed framework for their design. Chem. Educ. Res. Pract., 18, 518.
9. Assessment and Qualification Alliance (AQA), (2016a), Chemistry (8462) Specification Version 1.0, http://filestore.aqa.org.uk/resources/chemistry/specifications/AQA-8462-SP-2016.PDF, retrieved July 2018.
10. Assessment and Qualification Alliance (AQA), (2016b), GCSE Chemistry: Required practical handbook V 3.8, http://filestore.aqa.org.uk/resources/chemistry/AQA-8462-PRACTICALS-HB.DOCX, retrieved July 2018.
11. Assessment and Qualification Alliance (AQA), (2016c), GCSE Chemistry: Required practical handbook V 3.8, Making salts p. 11, http://filestore.aqa.org.uk/resources/chemistry/AQA-8462-PRACTICALS-HB.DOCX, retrieved July 2018.
12. Assessment and Qualification Alliance (AQA), (2017) How practical skills are assessed, http://filestore.aqa.org.uk/resources/science/AQA-GCSE-SCIENCE-PIE.PDF, retrieved July 2018.
13. Bloom B. S., (1968), Learning for Mastery, UCLA – CSEIP – Evaluation Comment retrieved from http://programs.honolulu.hawaii.edu/intranet/sites/programs.honolulu.hawaii.edu.intranet/files/upstf-student-success-bloom-1968.pdf, retrieved June 2018.
14. Board of Education, (1904), Regulations for secondary schools, HMSO, London.
15. Bransford J. D, Brown A. L. and Cocking R. R., (1999), How people learn: Brain, mind, experience, and school, National Academy Press, Washington, DC.
16. Carroll W., (1994), Using worked examples as an instructional support in the algebra classroom, J. Educ. Psychol., 86(3), 360–367.
17. Clackson S. G. and Wright D. K., (1992), An appraisal of practical work in science education, Sch. Sci. Rev., 74(266), 39–42.
18. Cohen J., (1960), A coefficient of agreement for nominal scales, Educ Psychol Meas., 20, 37–46.
19. Cowan N., (2010), The Magical Mystery Four: How Is Working Memory Capacity Limited, and Why? Curr. Dir. Psychol. Sci., 19(1), 51–57.
20. Dawes L., (2004), Research report: Talk and learning in classroom science, Int. J. Mod. Sci. Technol., 26(6), 677–695.
21. Dawes L., Dore B., Loxley P. and Nicholls L., (2010), A talk focus for promoting enjoyment and developing understanding in science, English Teaching: Practice and Critique, 9(2), 99–110, http://education.waikato.ac.nz/research/files/etpc/files/2010v9n2nar1.pdf, retrieved July 2018.
22. de Jong T., (2010), Instructional Sci., 38, 105,  DOI:10.1007/s11251-009-9110-0, retrieved March 2018.
23. Faraday, M., (1827), Chemical Manipulation: Being Instructions to Students in Chemistry, on the methods of performing experiments of demonstration or of research, with accuracy and success, Google Books, W. Phillips, London, accessed 18 Nov. 2017.
24. Gobet F., Lane P.C. R., and Lloyd-Kelly M., (2015), Chunks, Schemata, and Retrieval Structures: Past and Current Computational Models. Front. Psychol., 6, 1785.
25. Hamidu M. Y., Ibrahim A. I. and Mohammed A., (2014), The Use of Laboratory Method in Teaching Secondary School Students: a key to Improving the Quality of Education, Int. J. Sci. Eng. Res., 5, 9, 81–86.
26. Hennah N. and Seery M., (2017), Using Digital Badges for Developing High School Chemistry Laboratory Skills, J. Chem. Educ., 94(7), 844–848.
27. Hodson D., (1990), A critical look at practical work in school science, Sch. Sci. Rev., 71, 33–40.
28. Hodson D., (1993), Re-thinking Old Ways: Towards A More Critical Approach to Practical Work in School Science, J. Stud. Sci. Educ.22(1), 85–142.
29. Ibrahim N. H., Surif J., Hui K. P. and Yaakub S., (2014), “Typical” teaching method applied in chemistry experiment, Procedia – Social and Behavioural Sciences, 116, 4946–4954.
30. Jenkins E., (1998), The Schooling of Laboratory Science. Practical work in science, in Wellington J. (ed.), Practical work in school science. – Which way now?, Routledge, London.
31. Johnstone A. H. and Al-Shuailib A., (2001), Learning in the laboratory; Some thoughts from the literature. Univ. Chem. Educ., 5(2), http://www.rsc.org/images/Vol_5_No2_tcm18-7041.pdf, accessed 18 Nov. 2017.
32. Johnstone A. H., (2006), Chemical education research in Glasgow in perspective, Chem. Educ. Res. Pract., 2006, 7, 49–63.
33. Johnstone A. H., (2009), You Can't Get There from Here 1, J. Chem. Educ., 87(1), 22–29.
34. Johnstone A. H. and Wham A. J. B., (1982), Demands of practical work, Educ. Chem., 19, 71–73.
35. Kind P. M., Kind V., Hofstein A. and Wilson J., (2011), Peer Argumentation in the School Science Laboratory—Exploring effects of task features, Int. J. Mod. Sci. Technol., 33(18), 2527–2558.
36. Kirschner, P. A., (2002), Cognitive load theory: Implications of cognitive load theory on the design of learning, Learn. Instr., 12, 1–10.
37. Kucharski M. M. and Seery M. K., (2016), Titration exemplar video, https://www.youtube.com/watch?v=rK7Egs-Sjus, retrieved March 2018.
38. Landis J. R. and Koch G. G., (1977), The measurement of observer agreement for categorical data, Biometrics, 33, 159–74.
39. Lavrakas P. J., (2008), Z-Score. Encyclopaedia of Survey Research Methods, http://methods.sagepub.com/reference/encyclopedia-of-survey-research-methods/n639.xml, accessed July 2018.
40. Leikin R. and Zaslavsky O., (1999), Cooperative learning in mathematics, Math. Teach., 92(3), 240–246.
41. Lewthwaite B., (2014), Thinking about practical work in chemistry: Teachers' considerations of selected practices for the macroscopic experience, Chem. Educ. Res. Pract., 15(1), 35–46.
42. Levene H., (1960), In Contributions to probability and statistics, in I. Olkin et al., (ed.), Essays in honour of Harold Hotelling, Stanford University Press, pp. 278–292.
43. Lunetta V. N., Hofstein A. and Clough M., (2007), Learning and teaching in the school science laboratory: an analysis of research, theory, and practice, in N. Lederman and S. Abel (ed.), Handbook of research on science education, pp. 393–441.
44. Kirschner, (1992).
45. McDowell E. and Waddling R., (1985), Improving the design of laboratory worksheets, J. Chem. Educ., 62(11), 1037.
46. Mercer, N. and Littleton, K., (2009), Dialogue and the Development of Children's Thinking, Routledge, London.
47. Mercer N., (2013), The Social Brain, Language, and Goal-Directed Collective Thinking: A social Conception of Cognition and Its Implications for Understanding How We Think, Teach, and Learn, Educ. Psychol., 48(3), 148–168.
48. Millar R., (2004), The role of practical work in teaching and learning science, Washington, DC, National Academy of Sciences, http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_073330.pdf, accessed 18 Nov. 2017.
49. Millar R., (2009), Analysing practical activities to assess and improve effectiveness: The Practical Activity Analysis Inventory, https://www.rsc.org/cpd/teachers/content/filerepository/frg/pdf/ResearchbyMillar.pdf, accessed 18 Nov. 2017.
50. Office of Qualifications and Examinations Regulation (Ofqual), (2015), Assessment of Practical Work in New Science GCSEs – Summary, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/408513/2015-03-03-assessment-of-practical-work-in-new-science-gcses-summary.pdf, accessed 18 Nov. 2017.
51. Oxford University Press, (2016), Carrying Out a Titration, http://oxo_AQA16_C407_pr03_xxaann.pdf, retrieved July 2018.
52. Perry C., (2013), GCSE and A level reform, http://www.niassembly.gov.uk/globalassets/documents/raise/publications/2013/education/11413.pdf, accessed 18 Nov. 2017.
53. Raviv A., Cohen S. and Aflalo E., (2017), How Should Students Learn in the School Science Laboratory? The Benefits of Cooperative Learning, Res. Sci. Educ., 1–15.
54. Seery, M., (2017), Badging Lab Skills, https://badginglabskills.wordpress.com/author/michaeldit/, accessed 18 Nov. 2017.
55. Seery M., Agustian H., Doidge E., Kucharski M., O’Connor H. and Price A., (2017), Developing laboratory skills by incorporating peer-review and digital badges, Chem. Educ. Res. Pract., 18(3), 403–419.
56. Schmidt-McCormack, J. A., Muniz, M. N., Keuter, E. C., Shaw S. K. and Cole, R. S., (2017), Design and implementation of instructional videos for upper-division undergraduate laboratory courses: Chem. Educ. Res. Pract., 18, 749.
57. Smith M. E., Hinckley C. C. and Volk G. L., (1991), Cooperative learning in the undergraduate laboratory, J. Chem. Educ.68(5), 413.
58. Sweller J., (1988), Cognitive Load During Problem Solving: Effects on Learning, Cognitive Science, 12(2), 257–285.
59. Sweller J., (1994), Cognitive load theory, learning difficulty, and instructional design, Learn. Instr., 4(4), 295–312.
60. Sweller J., van Merrienboer J. J. G. and Paas F. G. W. C., (1998), Cognitive Architecture and Instructional Design. Educ. Psychol. Rev., 10, 251–296.
61. Tiberghien A., Veillard L., Le Marechal J.-F., Buty C. and Millar R., (2001) An Analysis of Labwork Tasks Used in Science Teaching at Upper Secondary School and University Levels in Several European Countries, Sci. Educ., 85(5), 483–508.
62. Tobin K., Tippins D. and Gallard A., (1994), Research on instructional strategies for teaching science, in Gabel D. L. (ed.), Handbook of research on science teaching and learning, Macmillan, New York.
63. UK Government, (2015), Analysis and use of Key Stage 2 Data in GCSE Predictions, https://www.gov.uk/government/publications/analysis-and-use-of-key-stage-2-data-in-gcse-predictions, accessed 18 Nov. 2017.
64. Vygotsky L., (1934), Thought and Language, translated by Alex Kozulin (1986), The MiT Press Cambridge, Massachusetts Institute of Technology, http://s-f-walker.org.uk/pubsebooks/pdfs/Vygotsky_Thought_and_Language.pdf.
65. Vygotsky L., (1978), Mind in Society: Development of Higher Psychological Processes, Harvard University Press, Cambridge.
66. Warfa A. M., (2016), Using Cooperative Learning To Teach Chemistry: A Meta-analytic Review, J. Chem. Educ., 93(2), 248–255.
67. Wellington J., (2000), Teaching and Learning Secondary Science: contemporary issues and practical approaches, Routledge, London.
68. Wood D., Bruner J. and Ross, G., (1976), The role of tutoring in problem solving, J. Child Psychol. Psychiatry, 17, 89–100.

---

Footnotes

† Electronic supplementary information (ESI) available. See DOI: 10.1039/c8rp00186c

‡ Oracy pedagogy employs tools such as “talking points” (Lyn Dawes, 2004) to develop and assess students’ ability to use spoken language (http://www.educ.cam.ac.uk/research/projects/oracytoolkit/).

---